The key to defining and creating good tests is to have a good understanding of all of the pieces that make up a good product. There are many things that can go into this understanding, and I will attempt to cover a few of the key ones here. One thing to keep in mind as you are reading over this post is that on some level, what you need to get from all of them is an agreed upon set of acceptance criteria that will form the basis of your test plans. You can’t just do ad-hoc/exploratory testing, and hope that you will cover everything that you need to cover (trust me, you won’t)…you need actual identifiable deliverables that you can put a check mark next to, in addition to all of the ad-hoc testing you will be doing. The creation of these acceptance criteria should not be the responsibility of any one party or team, rather it should be a group effort, where everyone provides input.
The business as a whole is a very important stakeholder in the success of the product, and as such, you should pay careful attention to the things they find important, and ensure that you have adequately covered their needs from a test case perspective. No matter how good your test cases are, unless they take into account the business’ expectations, they are not complete.
So what are some things to look for? First and foremost, you should have a very good understanding of what priorities the business has placed on specific pieces of functionality. This will help you not only prepare the proper test cases, but will help you prioritize and schedule them for testing. It is also very helpful to understand the business’ objectives and goals so when a conflict comes up as to priorities, you have a better sense of how to resolve it without having to call 10 meetings.
The Product Team
The biggest piece in designing your test cases should always be the product itself. You will need to know the product inside and out, and you should be the person/team that knows the product better than anyone else. This requires you to be involved along every step of it’s creation, from concept to release. You need to know the ins and outs, all the dark corners, and all the gotchas of the system in order to create test cases that cover the right scenarios. If you are only testing to the requirements you were given from the project manager or developers or even the business, there is no guarantee that they have accounted for all the possible scenarios you might encounter (and they usually haven’t). You need to make sure that the product that gets released is of the highest possible quality, and that means finding the holes.
I like to look at my job as one in which I get to break software, instead of as one where I just look for bugs. I am actively trying to break the app every time I run a test, and that is the mindset you should have. If you know what causes the system to break, you can create a test case to prevent it from happening in the future. This same concept should be applied to any bug that comes in from another group as well. You should be taking that information and creating a test case to ensure it is resolved and does not get out anymore.
One of the key components in a good test case that often gets overlooked is the expectations of the end-user. Far too many testers have no idea how the end-users actually use the product, and as such, they often look at it in a more clinical manner, and not as a real user might. Something you often hear, but is incredibly true, is that no matter how a system is built or designed, the end-users will always find a way to use it in a manner that was never intended, and fully expect it to work like that. This means you need to understand how the end-user expects to use the system, and plan accordingly. It pays to have someone from the QA team spend some time with an end-user for a day or so and see how they use the product, or the existing product yours will replace. Have user summits to solicit feedback from the end-users and to ask questions of them. Get inside of their heads and understand how they think and work. This product is for them, and it should be tested with them in mind. This may cause some tough conversations with the development team, and possibly even the project manager or business owner, but that just mean that those conversations needed to happen anyways. It doesn’t pay to waste time building an app that will never get used due to it not being what the end-user needs.
Remember, an end-user can take on various forms; it could be a customer that walks into Best Buy and buys a copy of your software, it could be Joe from shipping who is using the new inventory system you are developing, it could even be Bob from another development team that will be using your finished product to make or create his own product. There are many forms the end-user can take…the key is identifying them, and then getting to know how the product is going to be used.