When you get a test plan, don’t just jump straight into developing. If you think that this will save your time, think again. If your test development process is not adequately thought; it is more likely that you are not developing most efficiently and struggling to meet your deadlines. After thousands hours of test development analysis and speaking with our experienced developers, QA Engineers, we came up with a robust process of AB test development. It will help you to save your development time, increase efficiency and quality of the work, and will help you deliver the testing variation code on time.
Here are the steps:
- Analysis of the website
- Development approach
- Initialise markup
- Digging deeper
- Implement functionality
- Pre-QA
1. Analysis of the website:
Website Analysis is the most crucial phase of the process that can save a lot of time. So read the test plan carefully, go to the website for which you are going to build the test. Check whether this site has jQuery or not, which javascript library/plugin they are using, is this an SPA, which tool they are using (Optimizely, VWO, Adobe Target, AB Tasty etc), does the site have any dataLayer , are they using any CSS framework (like Bootstrap, Foundation, Bluma etc), check the checkout funnel of the website if necessary. What to check, is dependent on the test requirement, and you can decide about it after reading the test plan.
For example, if there is jQuery on the site, and, we know that jQuery is the best library for that. So we don’t need to waste the time by writing long plain JavaScript to change the text of all headings. (you can do it with one line of jQuery code, right?)
If the website is a SPA, be careful about the CSS selectors. You can use history.pushState to detect the change of the state(URLs, DOM, event etc) and based on that you might need to do the modification on your variation. You can also use MutationObserver to detect the DOM changes; cool javascript feature, I know!
The test is in Optimizely!! Why do you need to create some custom polling function? Optimizely has the utility library – utils, take the full facility of it. Want some sample code? here you go. Do you love to work with CLI? Optimizely has a REST API, instead of creating and maintaining projects using the Optimizely web dashboard you can create an experiment programmatically. Those tools have their own API, utility library don’t forget to take the benefit of it.
The website has Bootstrap!! Why are you wasting time to create a button colourfull? – just add a class.
2. Development approach:
This phase is the foundation where you are planning how you are going to write your code. It involves requirements gathering -like assets needed, any APIs or library that you are going to use, how you are going to do the markup, how to implement the functionality, how you are going to use the functionality of the control etc. Invest some time on it will keep you safe from uncertain situations that will force you to change the development approach in the middle of the development.
3. Initialize markup:
In this phase, you are going to create the DOM layout and design all the visual content; where your variation takes shape. So, you are going to do all the HTML and CSS stuff here. The control DOM structure & the functionality you are changing in the variation must be kept in mind whilst you are doing the markup. Check all primary function of the layout that needs to present in the variation; such as correct text, colours, images, accurate and responsive design etc.
If you are following a code structure to develop the variations, that is great. If not, spend some time to come up with a structure to have a common template that you can reuse in the future.
4. Digging deeper:
You just coded the markup. Are you sure it will work for all devices, it will fulfil your all requirements that are mentioned in the test plan? Did you miss anything? What if you have a different design for mobile that is not achievable with your markup? You might need to have some unique selectors for a variation only for goals/metrics setup. You just created a list of a banner that might need to be part of a slider/carousel on mobile devices. Is this markup compatible with the new functionality described in the test plan? So dig deeper, consider all the scenario that needs to be in your code and improve your markup if needed.
5. Implement functionality:
If you need to do something on the scroll, need to change slider/carousel images automatically, need to open a popup when a user clicks on a link; this is the correct time to implement all JavaScript functionality that you need to make your variation interactive.
6. Pre-QA:
Wait! You are not done yet. I’m sure you don’t want to back and forth by doing the bug fixing. You have done the hard part, you have given a lot of effort. Did you check at least one browser per device? What if your code does not work well after putting them in the tools? Is there a flickering issue or a loading issue? What if the code is not working correctly on Edge? Bugs are annoying, it makes you lose your head sometimes. So before sending the variation to the client/QA team do a quick QA on your own. If you find any issues, deal with it – update the code accordingly.
We have a separate article regarding the bug fixing, check it out. Bugs can be solved efficiently with a proper process.
Considerations:
When you developing an AB test you consider the following things:
- Functionality of control
- CSS: Use CSS from the control as much you can
- Javascript: Use Javascript from the control as much you can
- Prefer CSS solution: If you can do something with CSS don’t do it with javascript
- Don’t create anything if it is not needed
- Loading approach: Don’t forget to consider the dependency of loading the variation
