Smoke testing is a term that originated in plumbing and refers to physically checking a series of closed pipes for leaks. It doesn’t refer to extremely detailed testing of the pipes, but simply an initial preliminary test to ensure that nothing in the system will fail catastrophically. The same ‘smoke-tests’ are used in manufacturing and repair many things from electronics to wood winds. Our team has been using this process to make software development more efficient by eliminating waste in our QA process.
As some background, when our software goes from development to QA, we follow a rigorous testing process. Currently, our teams specs come in the form of wire-frame mockups (where applicable), a requirements document and detailed specifications written in the form of tests. As with most teams, defects or changes found during QA are entered into a defect log where our QA team needs to explain the defect with sufficient detail to reproduce the bug and includes:
- Time and Date
- Environment (alpha, beta, …)
- User logged in
- Series of steps required to recreate bug
- Expected results
- Observed results
- Which test/specification is failing?
After this information is entered, those defects are reviewed by a developer who often needs to contact the QA/Analyst for more information. Next, we develop code and tests to address the defect. Our code then goes through another code review process. Any problems caught in code review are fixed then code is merged or committed into our branch. Next we prepare a build and deploy to a test environment where the defect can be re-tested by our QA team. This process works… but is time consuming. The process itself, outside of the fix, adds fixed overhead to each defect. Each defect no matter how small take a not insignificant amount of time from our team to address, track, develop, review, deploy and re-test.
As a solution to this time-eater we have been using ‘smoke testing’ between the Development and QA process. What this means is that after the developer is ‘done’ but before code review and testing, we invite a QA/Analyst to our desk for a demo. We walk through the specifications, mockups and tests together ensuring to demo each feature. This process usually takes under 1 hour and without fail, finds issues small and large that are written down by the developer to complete and re-demo prior to passing to QA. Because these issues are found before code review and deploy, they never enter the testing process discussed above and the work of defect logging, discussion, re-development, re-code review, re-build and re-testing never need to occur.
We have found this process to be in-valuable to our team in terms of efficiency and eliminating waste. Give it a go on your project and I bet you’ll be pleasantly surprised.