How to Detect Bots in Email Marketing Campaigns: GTCbio Case Study

Implementing the best email marketing tools doesn’t necessarily guarantee success. There are many technical challenges to overcome. Among them, email security bots, firewalls and spam filters.

Email security bots, what are they?

Spam bots, firewalls, and filters are email scanning programs and scripts running on top of email servers. They’re specifically designed to scan inbound emails and email attachments for potential threats, including malware, phishing, clickjacking, scam, spoofed emails or spammy content.
 

Why are they bad for your email marketing campaigns?

E-mail scanners are, in fact, ‘good bots’. Their main purpose is to protect users from unsafe content. Unless an email contains malicious links, the recipients will be able to open and read it as usual.

For digital marketing professionals, however, email click bots pose a number of challenges, such as:

• Inflated open and clickthrough rates in email performance reports,

• Inaccurate activity tracking for event-based email marketing,

• Disrupted sales and lead nurturing automation processes due to false triggers.

Moreover, bots may become an even bigger issue if you let your subscribers opt in/out from your emails simply by clicking a link in the email, without asking them to confirm their decision on the website. Whenever bots click the ‘Unsubscribe’ link, you risk losing your carefully nurtured subscribers. Whenever bots click ‘Confirm subscription’, you risk starting new relationships off the wrong foot.

In sum, if your email service provider registers bot clicks and you leave them untreated, this might significantly distort your email performance reports, hamper your decision-making processes, and even harm the relationships with your customers and leads.
 

Email firewalls: is this problem unique to the B2B world?

The problem with email security bots is not unique to B2B businesses. B2C projects can face this kind of problem, too. The reason is that, free mailbox providers also use security bots and firewalls to check emails before they reach the recipients. For example, our experience shows that emails sent to Rambler Mail may have unnaturally high CTRs because of firewalls checking links for potential threats.

The answer is, keep an eye on your emails’ performance indicators. If you notice abnormal increase in OR and CTR, this might be because of security bots checking your email. That's why it's worth taking your best effort to identify and exclude bot user agents from your reports.
 

What can you do to detect and reduce security bot clicks in your emails?

Here’s how you can identify whether clicks were made by bots or human beings.

For starters, you could place a transparent 1x1 pixel image with a link in the body of your email, directing to a noindex, nofollow honeypot page. Since the image is so tiny and the page cannot be accessed otherwise, humans won’t be able to interact with it. With this, the number of clicks on the link would equal the number of clicks performed by link scanning software. The downside of this method is that, people who view web versions or your emails will be likely to see and click the link as well.

Another option would be, to meticulously check through all your subscribers’ domains for email opens and clicks. For example, the default report in our system lets you see the campaign’s open and click rate statistics per domain, ranked by the number of email addresses at the domain. See the screenshot below, demonstrating the extreme case of inflated email performance metrics.

Note that by default, our platform shows a limited number of domains, those where you send the most emails to. All the other domains are aggregated under the Other entry. If that’s where you witness the numbers so big that they distort your overall performance stats, you might want to sweep your email list for ‘bad apples’. Create a dynamic group to include only the domain you want to check. Email campaign performance reports in our platform show data retroactively, which means you’ll be able to immediately see the data for your already existing and completed campaigns.

Don’t forget to monitor your reports on a regular basis, as new bots are constantly appearing and evolving across webmail providers and mail clients.

The drawback of this approach is that it’s a tedious, time-consuming task, especially if there are hundreds of different domains in your email list.

Finally, one of the most effective solutions would be, to contact your ESP’s support team and ask them if they can check through your campaigns to determine exactly which mail client software, browser, OS originated the clicks in your emails. This approach is also not 100% accurate — unfortunately, none is. Still, you’ll be able to get a reliable list of email addresses with aggressive link scanning software to filter out from your reports.
 

How to deal with security bot clicks

If you detect bots messing up your email performance data, a reasonable workaround would be to shift your focus of attention and ground your decisions on other marketing performance indicators. However, this won't help you answer the key email marketing question, “How do my email campaigns perform?”

As an option, you could try reducing email security bot clicks by asking your subscribers to give you their personal email addresses. This might not help you solve the problem completely, as free webmail providers have their own, albeit less aggressive, firewalls and spam filters.

Still, the better alternative would be to isolate all the sources of email security bot traffic and treat them differently. For example, in our platform you can group such domains together in a separate dynamic group. Thus, you can obtain accurate information on your email campaigns.

Keep reading to learn how our team helped one of our clients detect and neutralize security bots in their performance reports.
 

Client Case Study: our platform Helps GTCbio Detect Email Security Bots

GTCbio, a B2B conference production company specializing in healthcare, biotechnology and life sciences, organized a number of conferences aimed to bring together doctors, postgraduates and scientists. They used our platform to send emails promoting their events to the target audiences in the US, East Asia, and Europe.

Challenge

The company started experiencing abnormally high email performance metrics for their email marketing metrics. There was a sudden steep increase in the open and click-through rates with no online registrations.

Upon analyzing user interactions with the emails, our platform experts noticed the same suspicious activity patterns across emails delivered to email addresses at certain domains. In these emails, users seemed to have clicked all the links and CTA buttons, within only a few minutes after a campaign was sent. At the same time, some of the links in an email were clicked twice or even more times. Such behavior is typical of bots, not human beings.

Roots of this problem

Most subscribers to the company’s email list were scientists and medical representatives. They used the information from the company’s emails to achieve their work-related objectives. Hence, they opted in with their work email addresses. Because of this, almost all addresses in the company’s email list were at custom, corporate domains owned by different universities, pharmaceutical companies, hospitals and government organizations. Corporate email domains often have more aggressive firewalls and anti-spam filters than free regular webmail providers.

Solution

In order to detect non-human and invalid activity in GTCbio’s email campaigns, our team decided to take the following steps:

• Check if there are contacts who opened all the emails delivered to them.

• Check if there were contacts who click all the links in the emails.

• See if there were suspiciously short intervals (a few seconds) between email clicks.

• Monitor the time logs recorded by the system. If the emails were read within seconds after sending, this would indicate bot activity.

Consequently, the following characteristics were examined across all GTCbio’s campaigns:

• Domain names,

• The number of clicks for each domain,

• The number of email addresses at this domain,

• The average number of clicks per email address,

• The total number of delivered emails,

• The exact time when the clicks were recorded,

• The number of clicks within each email campaign.

A deeper analysis of all the collected data helped reveal the full picture. The image below shows a part of the resulting report with conditional formatting: red for domains with suspicious behavior, yellow for domains that require extra attention, and green for domains whose characteristics were within normal range. For example, the number of clicks from emails delivered to addresses at a corporate domain wisvis.com clearly indicated that bots were in action.

After discussing the results with the technical support team, we decided to programmatically filter bot-like user agents out of GTCbio’s email performance reports in our platform.

For this purpose, our tech professionals provided a list of user agents the clicks were recorded from, using the user-agent header field. Through this list, it became apparent that many different domains were using the same email protection tool to verify links in the inbound emails. Such domains were isolated and examined separately. In this way, GTCbio could see real, undistorted information about their email marketing success.

In addition, our team suggested reaching out to the company’s customers via additional communication channels. In particular, the GTCbio’s sales department called these subscribers over the phone and asked whether they wanted to attend an event.

Results

With the help of our team, GTCbio’s marketers managed to:

• Obtain more accurate email performance reports,

• Make more efficient marketing decisions, based on the number of online registrations instead of click-through rates,

• Improve customer communication with direct phone calls.

This client case study is based upon a comprehensive email marketing audit our professionals conducted for GTCbio.

Special Request Inline