Online marketing has a range of tests.
The one that gets a lot of play and media attention is split tests, and with good reason – it’s a reasonably versatile, low-cost test, and you can apply it to existing pages and functions. However, if you’re launching something new or overhauling existing web site functionality, split tests will not be useful until the end of the process.
Also, if what you launched is fundamentally broken, split tests will not do you much good. Using split tests on a broken system is like adding lipstick on a pig to make it pretty – there’s only so much it can accomplish.
What marketers need to have is a broad view of the tests required for online marketing. That includes not just tests for refining what exists, but also tests done before and during development.
Before Development
Before you develop something, changes are dirt-cheap.
It’ll take a few dollars to change a low-fidelity wireframe on PowerPoint. It’s not that much more expensive to change a high-fidelity prototype on HTML or interactive PDF, without full functionality.
By contrast, it can take several hundred or several thousand dollars to change something that’s live. You have to reanalyze the backend, redevelop the functionality, and manage the project.
What we’re saying is, you should change the system a lot to correct for what’s wrong in the early phases of the project, when it’s cheap to make changes. You shouldn’t be making significant changes after development, when those changes are costly. To make sure you get the changes you need early on, you need to run tests.
Usability Test on Competing System
If you have a competitor that has something close to what you’re launching, you can actually interview users to perform tasks on THEIR environment. Then you can ask if the site’s section meets expectations, and if users have any difficulties navigating, ensuring they talk aloud in the process.
What this does is identify elements that work and potential pitfalls before you actually work on your low-fidelity test.
Low-Fidelity Prototype Test
A low-fidelity prototype test can take various shapes and forms. It can be a wireframe on a PowerPoint. It can be a set of screens on specialized software, like Balsamiq. It can even be a model on paper, but that’ll take more advanced interviewing skills.
Whichever version of it you choose to build, what you’ll need to do with it is the same.
You’ll need to …
- show it to a set of users,
- ask them to follow the flow, and
- see if they can accomplish the task.
If they can’t because your elements are in the wrong place, or the next set of screens are not what they expected, you can adjust before you get any further.
High-Fidelity Prototype Test
A high-fidelity prototype is just a more developed version of your wireframe.
It can be in PDF or HTML, but even without the full functionality of what you’re building, it’s supposed to get users closer to the experience of the real thing.
Your task with the high-fidelity prototype is the same – you’ll just interview users with a more developed system.
For smaller projects, this part can be skipped.
During Development
Most of your corrections should have been made before the development phase started. That said, there’s one test that needs to run before you finalize your changes and launch.
User Acceptance Test
The User Acceptance Test or UAT just ensures that whatever was agreed upon in the planning phase is actually what launches.
Essentially, you’ll have a checklist of functions and steps to check the functions, and a person or a team going through each of those in a numbered list. If everything works as expected, the build gets to launch. If one or more functions fail, those get fixed before the actual feature is shipped.
If you built what you set out to build, it’s time to launch and verify what you built works for users.
After Development
After you launch, there’s still the matter of verifying that what you developed actually works.
Usability Test on Your System
Remember what you did on your competitor’s environment at the beginning of the process? Now it’s time to do the same thing on your environment.
Ask users to perform tasks on your launched functionality or site area, and make sure they talk aloud about difficulties.
If you did well during the low-fidelity prototype testing phase, you shouldn’t find anything serious, but it still helps to verify.
Split/Multivariate Test
Of course, after you’ve ensured everything works well, there will still be refinements.
That’s where split tests come in. You can use champion and challenger pages at this phase, serve a percentage of your users the challenger page, and proceed to make changes from there.
The important thing to note is that split tests are about refinements, and the ones that you need to run before this are largely about basic usability.
Test Early, Test Often
You need to know which test to pull at the right time, and you can’t rely on just split tests to make sure everything works. If you test early, you’ll avoid costly mistakes down the line, so you’ll actually end up saving the company money by running tests.
If you run the right kind of tests before, during, and after development, you’ll be that much closer to making every part of your web site work.
