Wednesday, June 07, 2006

Results from our Coupon Experiment

During the month of May 2006, we conducted an experiment with the coupon code feature of our online store.  On May 1^st, I posted a coupon code here on my blog.  The coupon was good for a 20% discount during the month of May.  Now I am ready to report the results of the experiment.

The Data

• During the month of May, the coupon code was used for 9.5% of the transactions processed through the SourceGear store.
  
  
• Most of the coupon transactions were significantly smaller than the average size of the transactions for the month as a whole.
  
  
• 31% of the coupon transactions came from outside the United States.  For the month as a whole, 34% of the transactions came from outside the United States.
  
  
• All of the coupon transactions used a credit card as the method of payment.  For the month as a whole, 23% of the transactions were paid with a purchase order, but none of them used the coupon.
  
  
• I selected 25 of the coupon users and asked them how the coupon affected their purchasing decision:
• 4 people did not respond to the question.
• 5 people said that the coupon swung their decision toward Vault.  They were undecided about their Vault purchase, and the coupon affected their decision.  Note also:  The average size of those 5 transactions was dramatically less than the average transaction size for the month as a whole.
• 3 people said they were already planning to buy our product, but the coupon prompted them to do it sooner than they had planned.
• 1 person said the coupon caused them to buy more licenses than they had planned.
• 12 people said the coupon did not affect their decision in any way.  They were already planning to make a purchase at full price, but they found the coupon, so they used it.  (Of those 12 people, 3 of them volunteered that they simply Googled for the coupon code when they saw the place for it on the order form.  This is consistent with indications from the Apache log files on my blog server.)

The Observations and Conclusions

Positives and Negatives

Looking at the customers who responded to my question, I see several positives:

• 5 people said that the coupon swung their decision toward Vault.  These are customers we would not have gained.
  
  
• 1 person said the coupon caused them buy more licenses than they had planned. 
  
  
• 3 people said they were already planning to buy our product, but the coupon caused them to do it sooner than they had planned.  A pessimist might say that this is a bad thing, since we would have gotten full price from these people later.  I think this is a good thing because even if they planned to buy later, they might have ended up changing their mind.

In contrast, 12 people said the coupon did not affect their decision.  They were planning to make a purchase at full price, but they were able to pay less because they found the coupon.  An optimist could find positive things to say about those 12 transactions.  After all, we don't want to be in the habit of complaining when we people trade us cash for our software.

But the fact is that these 12 transactions can also be seen in a negative light.  Those customers paid 20% less than they were willing to pay.  The aggregate amount saved by those customers is effectively a cost of having the coupon.

A rather simplistic judgment of the coupon experiment would be to add up an estimate of the incremental revenue produced by the coupon and subtract the total amount saved by the 12 folks who would have been willing to pay full price.  (I'm sure there are several reasons why this analysis is simplistic, but the first one that comes to mind is that it ignores the future positive effects of having 5 customers we would not otherwise have.)

Anyway, I'm not disclosing actual revenue numbers here, but I will say this:  It appears that the net revenue from this coupon experiment was positive, but it was not a very large number.  (Keep in mind that I only asked for feedback from 25 of the coupon customers, and only 21 of them replied.)

Measuring Reach

One interesting way to use coupon codes is to track the effectiveness and reach of an advertisement.  For example, we might put a coupon code in a print ad and then watch how many times the coupon gets used.  This would give us some sort of indication of the effectiveness of that ad.

In this case, the coupon appeared in a blog entry, not in a print ad.  Can we look at the data from this experiment and draw any conclusions about the reach and effectiveness of my blog?  I'm not sure, but it seems like I would be extrapolating from very little data.  I'll predict that if we place a coupon in a print ad, it will show up in less than 9.5% of our transactions.  Nonetheless, the 9.5% number seemed kind of low to me.  Think of it this way:  Over 90% of our customers in May could have saved money on their purchase if they were reading my blog.  :-)

For now, I will decline to draw any solid conclusions on this issue.  The 9.5% number may end up being useful for comparisons later.

Segmenting the Market

Traditional coupons are a way of charging lower prices to customers who are more price-sensitive.  Joel Spolsky explains this concept rather well in his article on pricing.  I summarize it like this: 

Grocery store coupons are a form of market segmentation.  People who are willing to take the time to clip coupons from the Sunday newspaper inserts are rewarded for their efforts with discounted prices.  The value of the coupons (the amount of the discount) is specifically tuned to ensure that the time spent clipping them is worth approximately minimum wage.  Most affluent customers eventually realize that clipping coupons is a very low-paying job, so they stop doing it.  Therefore, the groceries get sold to two different groups of people at two different prices, and the "consumer surplus" is captured.

Obviously, some of our customers are more price-sensitive than others.  It makes sense that our coupon would be appealing to the more price-sensitive folks in our customer base.  Some of the data from our experiment seems consistent with this:

• Most of the coupon transactions were significantly smaller than the average size of the transactions for the month as a whole.
  
  
• Looking at the 5 people who said the coupon swung their decision, all of them spent dramatically less than our average customer spends.
  
  
• Nobody who used a coupon paid with a purchase order.  Purchase orders are more commonly used in larger companies with purchasing departments, which we can assume are likely to be somewhat less price-sensitive.

But one piece of data suggests that if our coupon was an attempt at market segmentation, it failed miserably:  12 out of 25 people questioned used the coupon to get a discount even though they would have been willing to pay full price.  Why did this happen?

I observe that the coupon on my blog was simply far too easy to find.  Just Google for "SourceGear coupon code" and it shows up as the second result.  If we want to decrease the percentage of people who used the coupon simply because they saw it, we need to add more hassle.  The grocery store example suggests that a relatively small amount of tedium will be an effective obstacle.  Clipping coupons from the Sunday newspaper isn't really that difficult, but lots of people just don't bother.

Final Thoughts

The transparency in this note is, well, atypical.  I am writing this because I think people might find it interesting, but I am taking somewhat of a risk.  I hope that nothing I have written here causes offense to any of our customers.  :-)

If you are reading this and you happen to be one of our more "price-sensitive" customers, please know that we appreciate your business.  I feel humbled by every customer who chooses our products, from the small shop with one license to the large enterprise with hundreds of seats.  

Also, if you are one of the coupon customers who responded to my question, thank you again for taking the time to provide useful information.

I think this coupon experiment was interesting.  It's over for now, and there are currently no valid coupon codes in the SourceGear store.  But I think we'll probably try this concept again sometime in the future.  As I've hinted here, I want to see what kind of different results we might get by placing a coupon code someplace other than here on my blog.