Data: scraping below the surface

This blog was posted by Zahida Din on November 17, 2015 as part of the CultureHive Digital Marketing Academy. You can find out more about the project here.

My experiment looks at the return on investment for post-show emails. The campaign for The Three Little Pigs threw up some interesting results…

  • Customers who attended the Friday performances were sent an email highlighting four shows for children of the same age. They received this at 5pm on the same day as their visit.
  • Customers who attended on Sunday where sent the email on Monday, ie the next day.

The results a week later…

  • Those mailed the same day (Friday attenders) – 67% open rate – £3.29 generated per email sent.
  • Those mailed the following day (Sunday attenders) – 57% open rate – £2.36 generated per email sent.

Sounds impressive, doesn’t it? Generating £3.29 income for a fraction of a penny, in expenditure, can’t be bad.

Whilst the results look really promising on the surface, a little digging around in the data shows that post-show emails are not quite as lucrative as they first appear.

The Three Little Pigs had performances on Saturday too; the audience didn’t receive a post-show email. This group acted as a control. Running their data through the same analysis showed that, across the same week period, £2.09 was generated per customer… by doing nothing!

So what did we learn?

  • Firstly, it’s easy to attribute sales to marketing activity, but without a control group we can’t be certain.
  • Secondly, no one unsubscribed from the mailing list, as a result we can assume customers didn’t find the post-show emails too annoying, in fact the emails had fairly good open rates (57 – 67%). There’s potential here for communicating key messages to customers already engaged with your organisation.

It would have been easy to assume the emails had generated close to £1k in sales, but once we took sales from the control group into consideration the margin dropped to just £200.

In an age where statistics are spun to suit the needs of those publishing them, the experiment is a helpful reminder that sometimes it’s worth scraping below the surface to work out what the data is really telling us.

Leave a Comment

Your email address will not be published. Required fields are marked *