Implementing a price testing strategy is like conducting an orchestra. Just like an orchestra needs every individual performer and section playing together in harmony, there are many different parts of your business that have to work in lockstep to run effective price tests. Otherwise, you just end up with noise.
Price testing can’t just be done in an ad hoc way. You need a data-driven framework to give your tests direction and to generate results that are actionable for your teams. Before we get into how you can build that framework, let’s talk about why price testing is vital for success.
Why Should You Test Your Prices?
There are so many different pricing strategies, and finding the best one for your products, categories, and market requires you to try more than one pricing model to hone in on the right one.
Finding the ideal pricing method is an ever-moving target. Prices aren’t just determined by internal metrics like desired profit margin, market share, revenue goals, or something else. External factors also have an impact. If the cost of a major input for your product skyrockets, your pricing has to change. Macroeconomic conditions that affect consumer spending habits will force you to adapt. Existing and new competitors change their prices constantly, which means you have to consider how their behavior affects your goals.
The rise of online marketplaces and their ability to update pricing based on real-time supply and demand data has forced retailers to continually monitor and update their prices to remain competitive. Continuous testing is less about finding the perfect time to do a test and more about creating a framework that lets you approach the process in a systematic way.
How to Build a Framework for Price Testing
Price testing can’t be done without a model for how the tests should be conducted. The right approach for your business will differ based on your capabilities and your goals. Here’s a framework you can use to evaluate your current capabilities and understand where you can improve.
Set Your Goals
Before you start testing your prices, it helps to understand what you actually want to accomplish. For example, some tests are goal-based:
- Growing the market for a new category
- Growing market share of an existing market
- Growing top-line revenue
- Growing profit margin
When you understand your goals, you can generate hypotheses that you can test in pursuit of those goals. You can ask questions like, “If we raise prices on this product by 5 percent, will our profit margin increase by X?” Or, “If we lower prices by 1 percent across our kitchen appliance category, will market share increase by X?
You can also test pricing based on specific actions. Does bundling increase sales? If we leave prices the same but change messaging, does that impact our results? Does a cost-plus strategy have a better impact than a value-based strategy?
Choose Your Groups and Categories
With a goal and hypothesis in mind, select a category where you can test your pricing. Ideally, this would be a category where you are selling a significant volume. Once you have a category, choose the products you want to test pricing on.
For example, if you are selling appliances, you could use blenders as an experimental group, where you’ll alter the prices, and food processors as a control group, with prices staying constant. These are similar enough that you can gather meaningful data on how consumers react to price changes for kitchen appliances.
It’s important that the groups are similar in their functionalities and consumer expectations. Comparing appliances to winter coats wouldn’t give you valuable information.
Collect Data and Start Testing
By this point, you have set the stage with the goals you want to accomplish and the tests you want to run. Now, it’s time to start collecting data to see what you learn from the changes you make.
Change your prices incrementally, so you can see precisely where the market stops supporting your price. Leverage competitive pricing data to see how your price changes impact their behavior, as well as find benchmarks for your pricing strategy.
Do this for a long enough timeframe as it takes to get enough volume to analyze. This can be anywhere from one to three months depending on the volume of sales you’re generating.
Analyze Your Results
When it comes to analyzing results, use a simple variance analysis. This can give you a strong idea of whether your revenue and profit are increasing or decreasing. Measure the difference in daily averages for metrics like revenue and profit before and after your price test. Compare the differences between your test and control group to see how much repricing impacted your bottom line.
If you’re looking to up your game even further, integrate statistical tests to measure your changes in revenue and profit. These will help you indicate with greater confidence if repricing was the main driver of your lift in revenue or profits.
Common Mistakes in Price Testing
Price testing is complex, and even the best make mistakes. Let’s look at a few common pitfalls and understand how they can be avoided.
Failure to Isolate Different Inputs
One of the most fundamental aspects of running an experiment is isolating the variables you want to test. For example, say you’re testing a lower price for a product at the same time your marketing team is aggressively pushing ads for the same product. Sales go up.
It seems like the test yielded great results, but how do you determine if that increase in sales came from your price change or your marketing efforts? Unfortunately, you don’t. You’re back to square one.
Failure to Get a Statistically Significant Sample
The smaller the sample size, the more variation you’ll see relative to average behavior. That’s why price testing requires a large enough sample of customers who saw one price and customers who saw another. Small sales volume can give you misleading results.
There are other things to consider as well. For example, if you only test prices with customers who are on your email list, that’s a biased sample because those people are already devoted customers. It’s not representative of your total market. Seasonality is another confounding variable that can throw off your testing.
Failure to Consider Competitor Pricing
Your prices aren’t the only ones consumers will be exposed to, especially online. Given how easy it is for consumers to compare different prices across marketplaces and individual retailers, you have to understand how your price tests will be impacted by competitor pricing.
If a competitor comes in to undercut your prices noticeably, that could steal short-term sales volume from you, impacting your test. Likewise, if a key competitor has similar items to the ones you’re testing priced higher than yours, the increase in sales volume for you could be more attributable to that, rather than anything you’re doing with your prices. It’s hard to control for this, but a strong price testing framework will take this into account.
Failure to Move Beyond A/B Testing
A/B tests have their place, but it should not be your primary price testing strategy. It might work for web developers testing different landing pages, but it’s not appropriate for testing different price points at scale. Here’s why:
- Even though most consumers are aware of dynamic pricing, you can still make them resent you if they find out they got charged more than someone else because you’re testing prices on them.
- You can anchor your price at a low price point for the consumers who see a low price during a test. If that price goes back up, you’ll potentially upset them and make them go search elsewhere for a similar product.
- A/B testing is great for showing how one price point performs relative to another, but it’s not ideal for understanding what your products are actually worth. The idea of price testing is to find out what customers think of your value. This is ultimately what you want to uncover.
Price testing is all about continually honing your method as you learn and adjust. Every experiment will lead to new hypotheses and things to test, which will allow you to accumulate learnings and better serve your customers and achieve your goals.
Editor’s Note: Contributing writers are Min-Jee Hwang and Brian Smyth. This post was originally published in February 2016 and has since been updated and refreshed for readability and accuracy.
