Google Adwords is the king of Pay-per-click (PPC) advertising, but it's not the only player in town. If you are in the business of Search Engine Marketing chances are that you are also using Bing Ads: the Adwords alternative that Microsoft built for the Bing search engine.
Adwords is Google's main source of revenue, and we all know that Google is making the big bucks. Despite the efforts of the Bing Ads team, the engineering resources behind Adwords are just something that Bing Ads can't match. The technological gap between the two platforms is especially evident when you try to automate things. Like ingesting click performance reports from the Bing platform into your ETL pipeline.
Our ETL pipeline
At FindHotel we are using the AWS cloud to build a completely automated ETL pipeline, and most of our ETL processes involve at least these steps:
- Ingest raw data from an external source into our S3 bucket
- Run various transformation on the raw data using AWS computing resources
- Dump the transformed data back to S3
For Google Adwords, ingesting click reports into S3 is relatively easy, using the excellent Adwords API, or using the simpler, but almost as powerful Adwords scripts. Bing Ads also has an API, but it's definitely far behind what's being offered by Adwords, especially in terms of documentation. Let's put it this way: learning how to use the Bing API is not what we considered to be fun. We don't do non-fun things unless we have no other choice.
Python magic to save the day
Although we are not very fond of the Bing Ads API, the Bing Ads GUI has a nice feature that allows you to schedule daily click performance reports. You can also choose that the reports will be sent to you by email, either as an attachment or as a download link in the email body. So we don't really need a complicated application running somewhere and interacting with the Bing Ads API. We just need a way to process all these emails automatically.
As it turns out, this is really easy to do. Using Python's smtpd library we had a mail server running in an EC2 instance in a matter of minutes. From there it takes only a few additional lines of code to write a custom email processor that moves the reports to the right place in our S3 bucket using the boto3 library. The actual processing that is specific to Bing reporting, and that you may like to change to suit your own purposes can be as simple as this:
def process_attachment(self, payload, filename): boto3.client('s3').put_object( ACL='private', Bucket='yourbucket', Body=payload, Key=filename)
I know what you are thinking: this is a just a hack, a short term solution. Maybe, but this solution has some undeniable advantages beyond being much easier to implement:
- We leverage Bing's resources: Bing's reporting service does all the heavy duty work, at zero cost to us.
- We can easily reuse what we learned. Triggering actions by email is something that you can use for lots of cool things. We will give some examples in a future blog post.
- You can easily incoporate SSL encryption for enchanced security. Simply use secure-smtpd instead of Python's standard smtpd library.
Try it yourself
You will find all the code in our Github account, including everything you need for a test deployment in the AWS cloud.