Developing software requires testing to ensure the quality of your product. That testing takes time and money, resources that you no doubt want to put directly toward development. But shipping a product that no one can use, or worse, that customers find frustrating because of errors and glitches, will leave you wishing you’d spent that extra cash to get the job done right. So assuming that testing is essential, how can you get the most out of it?
Levels of Testing
First, let’s start with different levels of testing: there are essentially four. Each are important to be able to catch issues both big and small.
- Unit: Unit tests occur at the smallest level. A developer tests the execution of the functions he is responsible for.
- Integration: Integration tests are best accomplished by a dedicated tester (rather than the developer). That person could be a part of the development team or a separate quality team. Here, you are testing a full set of features and “white box” testing to look at the cooperation between components.
- System: System tests are best accomplished by testers that are completely outside the development team, even a separate testing company. This “black box” testing is used to see how the software works from the user’s standpoint without looking at the way the pieces work together “under the hood.”
- Acceptance: Acceptance tests are performed by pilot users for a real look at the user’s standpoint. This round of testing helps developers learn the acceptance criteria of users so that they can adjust elements if needed.
Testing at each level is important to make sure that each piece is working properly, they are working properly together, and the user is able to use the finished product easily. Starting testing early on keeps costs low: discovering issues as early as possible saves an incredible amount of time and effort later on down the line.
Automate Your Tests
Whenever possible, automate your tests! Paying to develop automated tests may seem salty up front, but in the end it will save you countless hours of labor that manual testers would charge to do. Obviously you can’t automate the user experience, but every other level of testing should be automated as much as possible.
We’re not the only ones who feel this way:
“A manual test is an expense. A test which is consolidated in a replayable script is an investment.” —Softfluent
“Repetitive tasks like manual testing numb the mind and invite mistakes. Testers slowly become less focused on their task, and more likely to let defects slip through. But a computer isn’t affected by this phenomenon. So automation ensures that each test is rigorously executed every time.” —Atlassian
Start with something as simple as a bug tracker, and then work up to bigger systems as you are able. Software testing is an essential part of building a good product.
Listen to Customer Feedback
With your automated testing in place, you are well on your way to developing software that will be as high-quality as possible. Not only that, you’ll be developing it as efficiently as possible. But then what? Once users begin to use your product (be it at the beta or wide-release stage) you will no doubt start to hear back with their opinions and experiences. How do you decide what to listen to and what to ignore?
Some would say there are two schools of thought on this: the Steve Jobs “We tell customers what they want” school and the Lean Startup “get this out to the customer, see what they say and then change the product” school. Really the two aren’t as far off as you might think. Steve Jobs didn’t ignore customers, he just understood that customers don’t know what they want until you show it to them. But he did create products that solved problems for customers, gave them the products and then observed what they did and said so that the next version or next product could be even better.
That is not actually far off from the Lean Startup principles, which suggest that a company must ask “Should this product be built?” instead of “Can this product be built?” Instead of spending a long time developing a product and “perfecting” it by their own standards, a company should spend enough time to develop a product and then get it out to potential customers to see what they think. From there, the product can evolve into something better that attracts more and more customers. Why spend so much time and money developing something under the assumption that customers will want it? Find out if they want it as early as possible.
User feedback is critical to successfully developing software. That said, you certainly cannot (and should not) take every piece of user feedback into account. Sometimes (often) different users will have opposite opinions. Pay attention to how many users bring up an issue or ask for a new feature. If you’re hearing a lot of the same feedback, it’s time to start really paying attention.
Even then, you shouldn’t automatically move forward with everything users want. Ultimately you must listen to what your customers are telling you and decide which feedback best synthesizes with your vision of the product and what you want to accomplish. If you’ve set up testing and ways to hear from customers early on in the development process (and then again near the end, and again after shipping,) you’ll be getting the most out of your testing.
Photo courtesy of wikimedia