Jargon: we test to find if a bug exists. We debug to find the location of a known bug.
Testing Techniques
Black Box (functional) testing
we only consider inputs and outputs - do not use
any knowledge of the code.
we create some input data, and its expected outcome.
also need to cover error input.
we use sample data - not the whole set!
White Box (code-based) testing
The programmer looks at the code, and tries to ensure coverage
choose data so that every branch of an IF is covered.
loops are executed once, several times...
ensure all error code can be executed.
Exhaustive testing
...is not possible. There are too many combinations of inputs.
Dijkstra: 'Testing can only reveal the presence of bugs, not their absence'
In other words, we can never know when we have found the last bug.
We can use 'equivalence partitioning' to represent a 'run' of data by one item.
For example, the ages 18 to 59 might be represented by e.g. 30 in a particular program.
Static Verification (no code is run)
walkthroughs, inspections
...are reviews in which a group of people 'walk through' documents. An inspection is more formal,
but a walkthrough may only involve the team of programmers.
it can be used - e.g in spec, design, coding...
can find errors early, hence good value.
needs setting up to avoid pressure on individuals - we want people to be open
about admitting errors.
The Stages of Testing
Unit testing -
avoids 'big bang'
test a 'module' e.g a method or class
by programmer - white box
may need to write bits of extra code, e.g:
a driver - code to call up the method in question
a stub - a dummy version of a method that is called by the method in question.
System testing
We fit together all the unit-tested stuff, in a systematic way
in-house
black box
formal test plan
Acceptance Testing
involves customer
black box
uses real data
when done at customer's site - 'Alpha' testing
You will also hear of 'beta testing', where a product is released early to selected customers.
Which approach is best?
Studies show that different approaches find different errors - hence might use a combination
of e.g white box and walkthrough for unit tests.
Summary
testing is crucial, and costly - recall our other lecture on the stages/costs in life-cycle.