Ecommerce testing guide 2022
What is ecommerce website testing?
Ecommerce website testing is the process of testing e-commerce websites, servers, and E-commerce software to ensure they are operating correctly. It ensures there are no software or hardware errors preventing customers from paying, receiving products, and leaving positive reviews.
In this post, we’ll give you the ultimate guide on ecommerce QA testing, including why you should be testing today, as well as answering some of the most frequently asked questions about ecommerce website testing. We’ll also get into the nitty-gritty of how to QA test your Shopify or ecommerce website, walking you through the process step-by-step, as well as sharing hidden tips and tricks. Lastly, we’ll touch on a powerful new trend used by some of the largest Merchants in the world, automated QA testing – and how automation can save you a lot of time, money, and headache.
Why you should be performing ecommerce website testing
Gain peace of mind
Your online store is your business, and we so often lay awake at night wondering if things are working or if we made a mistake with a recent change we made to our website. But with a basic QA testing plan and strategy, you can gain peace of mind knowing that your Shopify online store or ecommerce website is functioning correctly and that your customers are able to both navigate and make a purchase from your website.
Theme changes and third-party apps can cause issues
As you run the day-to-day of your online store, you may be constantly making updates to your store theme or installing third-party apps from the Shopify App Store to extend the functionality of your website. With every change you make or app you install, it leaves your store vulnerable to breaking changes that may impact your store's ability to function – for example, a theme change may result in broken add-to-cart buttons, or a third-party app may add a new script that slows down your checkout, causing conversion rates to plummet.
Without an effective QA strategy, you may not catch these sorts of issues right away – leaving you to only find out later by looking at your sales data for the day.
Ecommerce downtime can be expensive
The cost of downtime obviously depends on how many sales your store is doing per day, but overall downtime will cost your business a lot of money.
At a baseline downtime or online store issues will impact your conversion rate leading to a loss of sales and revenue, but there are other external costs that you should consider, such as money spent on Facebook or the cost of sending emails or SMS to customers that lead to a broken store.
There is also the intangible cost of loss of trust with customers – as it’s likely a customer who visits a broken store may never return.
To help you assess the risk, we’ve put together a handy downtime cost calculator where you can assess how much downtime or store issues may cost you in the next year.
Build customer empathy
By testing your online store you are putting yourself in your customer's shoes, and viewing your ecommerce website from your customer's perspective. This can lead to a deeper sense of empathy for your customer, often revealing new ways to improve your online store experience, which can lead to better conversion rates.
Frequently asked questions about ecommerce website testing
What types of QA testing are there?
There are many different types of QA testing, but in this post we’re going to cover the most used types of testing in the ecommerce industry:
End-to-End Testing: this is where testing occurs on a live online store, where the tester mimics real user behaviour in order to simulate a real-word scenario. The purpose of this is to verify that the specific work flows tested work as expected
Performance Testing: performance testing is a stress test of your online store to ensure it is performing well under different loads – these tests often check for page speeds, and mobile useability. These sorts of tests are good for assessing how SEO ready your website is, as well as if there’s any load time issues that should be address to improve conversion rates.
Automated Testing: this type of testing takes end-to-end tests and automates them so they run on a set interval without the need for human intervention. More on this later on in this post.
How much time is needed to effectively QA an ecommerce website?
Generally, Shopify QA testing shouldn’t take too much time – it often very much depends on the complexity of your online store as well as the complexity of your products/business. For example, a headless online store would be much more complex than a store that is running on a basic Shopify theme, or a store with products that have many different variations may be more difficult to test than a single product store.
How often should I test my online store?
This really depends on the Merchant, the complexity of your online store, and how often you are making changes to your store.
If you rarely make changes to your online store you may not need to test very often as there’s not a lot that could go wrong. However, if you are constantly making changes to your theme, adding apps, or are running a business generating a high amount of revenue, then the risk is much higher and you may want to test your store more frequently.
In the case that your store is simple and changes happen less frequently you could get away with testing every couple of weeks. For a higher-risk store, you will want to be testing at least every day to ensure your store is fully operational.
The general rule of thumb is you should base how often you QA test on how much you stand to lose – if you stand to lose $10s of thousands of dollars for 1 hour of downtime, then you may want to test more often. If this is too cumbersome for you then you may want to consider using a service like Uptime to set up automated Shopify QA testingfor you.
What are the most common things to test on an online store?
Here is a list of the most comment test flows that Merchants often test on their ecommerce websites:
Ability to proceed from the home page to a product or collections page
Ability to add a product to your cart from a product page
Ability to add cart upsells from within the cart page or cart drawer
Ability to proceed to checkout either directly from the product page (ie. buy it now button or express checkout buttons) or from the cart page/drawer
Ability to add a discount code on checkout
Ability to complete checkout and place an order
A good place to start when thinking about test flows is the customer journeys from your Facebook ads, search ads, or email/SMS sends – these are often critical paths that should be tested thoroughly. You don’t want to be throwing money down a hole!
Does testing my online store impact my analytics?
Running manual useability testing (as in manually opening your online store in a browser and clicking around) will, unfortunately, impact your analytics.
With Uptime, we’ve developed a way to automate your online store and test without impacting your analytics – and that includes both your Shopify analytics admin reports as well as your Google Analytics.
How to test an ecommerce website
Performance testing
Sometimes online store issues can be caused simply by performance issues – meaning a customer's ability to effectively load your store quickly. Often times adding third-party apps or making inefficient changes to your theme can cause increased online store load times which can have a large impact on both your conversion rate and your Google SEO ranking.
To assess the performance of your online store you can use the Google Lighthouse testing tool. Here’s how to run a test:
Open a new browser window in incognito and load your online store
Go to this website and enter your online store URL
Select the Run Audit button and view the results
The results may be a bit cryptic if you are not a developer, but the score should give you an idea of whether or not this is an area of QA and improvements you should invest more in.
How to place a test order on Shopify
Placing a test order on your Shopify store is a great way to gain peace of mind knowing that customers are able to purchase products.
The most basic way to perform a test order is to simply visit your store in your browser, and make a real purchase. This is also the best end-to-end test as you’ll be performing the test in the same way a real live customer would.
Turning on Shopify test mode
Test mode means you will not be able to accept real credit card payments for orders. You should make sure to not enable this if you have active customers visiting your store. This approach of how to test Shopify checkout is normally best used on a Shopify test store.
In order to create a Shopify test transaction and test Shopify checkout, you will need put your Shopify store in test mode. You can do this by performing the following:
Load your Shopify Admin, and go to the settings page
Select the Payments option, then select Manage next to Shopify Payments (if you’re not using Shopify Payments then refer to this blog post)
Scroll to the very bottom of the page, and toggle on the Test mode then hit Save
Congrats, your store is now ready to test! You can now proceed to place a test checkout with a Shopify test credit card. When you arrive at checkout make sure to enter the following Shopify test card information:
Name: any two words will do
Expiry: you can use any date in the future
Security Code: you can use any three digits you like
Card Number: you can choose from the following test credit card types
Visa: 4242424242424242
Mastercard: 5555555555554444
American Express: 378282246310005
Discover: 6011111111111117
Diners Club: 30569309025904
JCB: 3530111333300000
How to delete test orders in Shopify
Test orders, even one made with Shopify test cards, will show up in your Orders tab, so you will probably want to refund and delete the order so it does not complicate your analytics. Here are the simple steps to remove test orders in your Shopify Admin:
In your Shopify Admin, navigate to the Orders tab
Click into the order you would like to refund and delete
First, select the Refund button in the top right-hand corner of the order window, and fulfill the refund
Now return back to the order, click the More actions button in the top right-hand corner, and select Archive
Finally, scroll to the bottom of the order page and select the Delete Order button to fully delete the order from your store
Automated Shopify QA testing
Automated testing is a set-it-and-forget solution for QA testing – think of it as hiring someone to periodically check and QA your ecommerce website. It can save Merchants a lot of time and energy, as well as give peace of mind knowing your store is running correctly.
How does automated online store testing work?
Automated testing works by simulating real user clicks, mouse movements, and typing on your online store.
These are run in the form of test flows, which are pre-recorded bits of code that tells a computer what it should do when it lands on your ecommerce store. So if you want to test your add to cart buttons, you can simply record a flow that loads a product page and clicks the add to cart button. Then, once uploaded to the Uptime app, this test will run every 24hrs or whenever you make an update to your theme to ensure everything is working correctly.
Why use automated ecommerce store testing?
As a Merchant, automated Shopify QA testing can save you a lot of time and a lot of money. As we mentioned earlier, depending on the complexity of your store and the amount of impact downtime would have on your business, QA testing can take a lot of time and create a lot of stress.
With automation your QA testing is completely hands off, you simply record the flows and Uptime will check them periodically in the background, and alert you the moment something is wrong so you can take action.
As an added bonus Uptime also monitors the Shopify platform status, so you’ll get notified if any Shopify services outside of your online store are having issues.
Summing up ecommerce online store testing
There you have it! The ultimate guide for all things ecomerce website testing. We hope this guide made it clear as to why QA testing your Shopify online store is important, as well as answering all the open questions you had.
You are now equipt with the tools to create test orders on your online store in order to run effective end-to-end testing – giving you peace of mind knowing that your store is working as expected.
We also learned about a new up-and-coming trend in the ecommerce QA space called automated testing, which allows you to be totally hands-off with your testing strategy, allowing a Shopify app to run a test suite of flows on your online store. This will help save you both time and money by detecting issues early so you can take immediate action.
If you need additional support or have more questions about automated online store testing you can reach us at any time at [email protected] or book time with us for a free store QA analysis here.
