How do they know when testing should be ended? When the faults are few and far between,
they might say 'We could go on forever, trying to find the last bug, but nothing is perfect...'
Section 12.2
Ideas
Dijkstra: 'Testing can only show the presence of bugs, not their
absence'
or - how do we know if a bug we find is the last one? (we don't)
Jargon - testing - is the software correct?
debugging - trying to find the source of a known error.
Section 12.3
Main stages
-
unit testing - e.g 10 to a few 100 lines, a class..., a procedure a sub ...
-
integration testing - the whole program - can be many 1000s of lines
-
alpha testing - in-house
-
beta testing - external
Section 12.4
Some Terms
-
validation - are we building the right product - not covered here.
-
verification - means testing.
-
static testing - without running the program (e.g code inspections)
-
dynamic testing - running the program.
-
black box testing - testing without looking at the source code
-
glass box (white box) testing - basing tests on the source code
Section 12.5
Glass box (or white box) Testing
Done by programmer (hence problem of unwillingness to find flaws in
ones own work). Look at the code and choose data so that e.g.
-
every branch of an if is executed
-
every error message can be made to appear
Problem: not possible to try every PATH through program. (Analogy - journey)
This form of testing can be used for unit testing. For testing the whole system, it
is not feasible (too many lines of code !)
Section 12.6
Black box testing, and choosing data
- based on inputs and outputs - we do not use the code. We are testing a whole system, and the code can be massive. We look at the program specification to see what it is supposed to do, then choose our data.
We can focus on boundary values (which can show 'off-by-one' errors) and use
equivalence partitioning.
Example: 'An age is input, in range 16 to 64...' We identify 3
regions:
-
less than 16
-
16 to 64
-
65 and above
Rather than input every age from 16 to 64, (16, 17, 18, 19...)we assume that e.g. 30 will
be equivalent. We try to choose one item to represent a goup of
items.
We should also test the boudaries carefully (e.g test 15, 16, 17)
Note: there are still many combinations. We CANNOT
exhaustively test a program!
This form of testing is often used for system testing. Note that we choose the test data by reading the
program specification, not the code.
Section 12.7
Inspections and Walkthroughs
Static - Involves a team reading the code, and discussing. E.g:
-
have we covered every error case...
-
are the names clear...
-
does it meet our coding standards...
It does work, but watch/avoid personal clashes (e.g comment on 'our'
code, not 'your' code.
Integration testing
done after unit testing, to avoid a 'big bang' . Can integrate modules (classes, procedures...) one-
by-one, to help pin down error location.
Section 12.8
Automated testing - as used in Extreme Programming - XP
It can be helpful if we can run all the tests automatically. To make this easier, we
can put our input data in a file, rather than type it in.
We can also call up procedures and check the answer, as in:
...we have written a function - big - to find the biggest of 3 numbers. here is how
we could test it:
private sub testbig()
if big(1,2,3)<> 3 then
MessageBox.Show("test1 failed - answer should be 3")
end if
if big(3,2,1) <> 3 then
MessageBox.Show("test 2 failed - answer should be 3")
end if
if big(1,3,2) <> 3 then
MessageBox.Show("test 3 failed - answer should be 3")
end if
etc... test for 3 equal numbers, -ve etc...
...
end sub
all we need to do is call testbig to do the tests. if big gets re-written, the test
is there, ready to run.
Note that in the above code, we did not use 'else' between the tests, becuase we want it to run through all
of our tests, not finish at the first one it finds to be true.
The design chapter has some brief info on XP
Section 12.9
Which Method is best?
Different methods locate different errors. Unit testing and system testing is essential, but
walkthroughs might show errors which other ways do not show.
Stuff on alpha, beta testing (and other testing material(optional!)) on wikipedia
| 