1 Comment

1. Johnny Chatterton

   Posted April 5, 2010 at 3:22 PM

   Analytics and Testing

   What do did we want from the session?

   Ed from Torchbox — interested in seeing how analytics is being used to track subtler things.
   Rachel from Rechord — Analytics and testing is so important and we want to absorb everything we can about how it’s being used successfully and not
   Brian F from Greenpeace — testing and analytics is our god
   Rob from Rechord — Interested in all systems that can save organisations money
   Tom from Torchbox — same as Ed
   Jay from Tearfund — analytics is something I want to get my head around to make my supporters more engaged

   Analytics is overwhelming, so what do you focus on?

   First of all, understand what the terms mean. Do Google’s Conversion University http://www.google.com/support/conversionuniversity/?hl=en which is a series of walkthrough presentations on what analytics is and how to get the info you want out of it.

   Germany have problems with the legal aspects of Google Analytics. They can’t use it anymore.
   That gave them a hard time. Now they’re at a point where they’re about to get rid of it totally and use Piwik. http://piwik.org/

   This is the open source alternative to Google Analytics. It looks promising although GA keeps expanding and expanding. They’re at beta 0.5.5 right now.
   Omniture = incredibly expensive
   Webtrends = expensive and didn’t do anything that GA couldn’t do.

   Complexity in Google Analytics; that’s when you get to installing Goals.
   Greenpeace put each of their thankyou pages in its own subdirectory to track goals and conversions properly. You can also annotate the timeline in GA — this is where we sent the survey and so on.
   Numbers don’t totally match in funnels in GA Goals. We’ve recognised a problem with a particular page using funnels and we can go in and fix it.

   Our numbers were 3x higher in GA than in Piwik. We can’t understand why this is. If numbers are wrong, use server logs to calibrate it and see what the data actually is.

   A lot of people have recommended getting in an analytics expert to analyse the figures as a start-up. But there’s analytics ‘experts’ and analytics experts. If anybody knows a good person, please recommend one.

   How do you combine GA with a newsletter system?
   Mailchimp offers that out of the box. They use a tracking code that they add and GA knows about that. It’s against Google’s terms of service to track users individually though and it’s also against German law.

   It comes back to not just WHAT you’re tracking but WHY. Do you have the resources to improve whatever it is you’re measuring?

   First: define your success.
   Second: how are you going to reach that point?

   Johnny: We used ClickTale to identify why there was a high bounce rate on some petition pages. We realised that people were confusing the email newsletter signup form with the petition. http://www.clicktale.com/

   Crazyegg is also good, because you can segment by entry method, so you can draw some conclusion about the purpose of the visit. http://www.crazyegg.com/

   You can use split testing to increase conversions.

   Are there any specific examples of goals that people had and how they worked to test and increase the success?

   We worked with the Humane Society on a petition signing campaign. We bought media on loads of different channels. We wanted to use Analytics during the campaign to figure out what money should be spent on during the campaign with Google Adwords versus Facebook. After the campaign we went back and looked at Cost Per Acquisition for each of the channels. Adwords via Google Grants was most successful and Facebook after that.

   We ran test on our donate page using Google Optimiser. We tested form field changes, copy changes and layout. We got a 33% increase in conversion from the control, which resulted in an 8% increase in BACS donations. We’re looking at doing more multivariate testing and funnels as well. It was tax time in Oz and we thought we could get enough traffic during May and June then going live with the best form after that. But there were glitches which meant we couldn’t get it live in time, so it took 6 months to get it running. We got 600-800 converstions each month during tax time, I can’t remember the exact numbers but we needed around 1,000 to get statistically significant numbers. We did a lot of work on multivariate testing with an agency. They tested gift strings according to previous donation history. They got an uplift but it wasn’t really significant. They found seasonal variation in colours too.

   Last night Google released funnels for AdWords.

   Always remember that any results you get are due to your audience.
   Greenpeace found that a picture of a little girl beside a donate button was the one that worked best against a picture of an orang-utan or a whale. We think that’s because of the psychological phenomenon of people being more likely to donate something if there’s a pair of eyes looking at them.

   How far do people go with tracking users and putting that data into a CRM? My organisation wants to build up a profile of individual users. Does anybody spend lots of time data mining for this sort of thing? It’s totally legal in the UK as long as you have it in your privacy policy. But is it recommended for NGOs? You can get a scary amount of data about individuals. It’s quite an uncomfortable area. The trade-off with Amazon is that you get a good service and it’s useful. But they are collecting information on individually identifiable book purchases, eeek.

   Privacy stuff can hurt our ability to know when a corporate target visits the site. We lost that information when we moved to Google analytics because we could know which corporations were visiting the site.
   But you can kind of do this by excluding specific corporation IDs from a duplicate analytics account and then you can see the difference in figures and know that’s how many corporations visited your site.

   What about email testing?
   We’re often wrong, and it’s really refreshing to learn things that way, but also very disappointing… I’m an expert in this field and I’m supposed to know this?! But that happens to everybody. It’s human behaviour… unpredictable.

   One chunk of our list tends to get worse results than all the others. We realised that it’s the time when it’s sent out. Time of day is really important. Never send anything on Fridays. We now send stuff out at 8am so that it gets there by 10am to avoid this.
   We found Friday was the best day because we were sending out fun things on Friday lunchtime though.
   Wednesday is the best day for open rates, but Friday may be the best day for CTRs.
   There is some software that will send out emails at the correct time of day for a person based on their history of responding at certain times of day. This is really important because you can miss a big segment of your list if you’ve got Jews, Christians or Muslims which are a big proportion of your 50,000 emails.
   Campaign Monitor will do a 5% split A/B test and send out the winning email to the rest of the list automatically.
   There is a tool that will combine clicktale-like functionality on the website with email campaign monitoring. So you can see the exact journey.

   Return-path and Goodmail will track whether emails end up in spam. This can make a big difference between 80% and 98% delivery rates.
   Our vendor takes their SLA very seriously and will ban us if we get too many people clicking on things saying ‘this is spam’.

   Does anybody test when the best time is to post links on Facebook or Twitter?
   No, but we’re going to now…
   There are a lot of people on Facebook during the Christmas and other holiday seasons.

   Does anybody measure which activists retweet the best for you?
   We do… we will DM our biggest supporters. We have a list of bloggers that will post stuff for us, and we send them things. It’s all about having those contacts and not abusing them. You’ve got to spend a lot of time building up friendships with people.