Software automated testing 123
Esteban Manchado Velázquez
Background
- Old codebases
- “I didn't write this @&#~$ !”
Background
- Old codebases
- “I didn't write this @&#~$ !”
- Many regressions
- Time wasted finding regressions
- QA testing instead of building
Quote
The main thing that distinguishes legacy code from non-legacy code is tests, or
rather a lack of tests. [...]
To me, the difference is so critical, it overwhelms any other
distinction. With tests, you can make things better. Without them,
you just don't know whether things are getting better or worse.
— Working Effectively with Legacy Code
Why automated testing?
- Ensures "no regressions"
- Confidence
- Refactoring safe net
- Keeps your programs cleaner
- Finding bugs before committing saves time
- It makes you feel warm and fuzzy inside!
Types of tests
- Unit
- Functional
- Other categorizations, not important now
Unit tests
- AKA whitebox or developer
- Test "modules", or "units"
- Care about APIs and design
- Call functions/methods and check result
Functional tests
- AKA blackbox, user
- Test "programs", or "systems"
- Don't care about design/implementation
- Use services, simulate users
- Used for acceptance tests
Free advice
- Don't try only typical cases
- Make a test for your bugs
- First make the test, see it fail...
- ...then fix the code
- Always run the tests before committing
- Don't focus on the current implementation
- Test in a controlled environment
But...
- ...tests are hard to write!
But...
- ...tests are hard to write!
- Chances are your code stinks
- Refactor
- Functional tests first?
Now, behold!
There be dragons^Wcamels!
How do I write that?
- Standard Perl scripts named t/*.t
- use Test::More
How do I write that?
- Standard Perl scripts named t/*.t
- use Test::More
- You must have a plan
- Asserts are functions like ok, is, ...
- Fancier functions in Test::Deep and others
Example
use Test::More tests => 3; # Specify plan
use Some::Module; # Load module to be tested
is(power_sum(1, 2), 3, "Obvious case");
is(power_sum(20, 7), 27, "Another obvious, but different");
is(power_sum(2, -2), 0, "A bit more interesting");
is(power_sum(-2, -1), -3, "Other weird but possible cases");
ok(is_even(2), "Obvious case");
ok(!is_even(3), "Obvious case, but different");
ok(is_even(0), "Strange case");
ok(is_even(-2), "Another strange case");
ok(is_even(-3), "Another strange case, making sure");
like(get_email, /\w+@\w+\.\w{2,3}/,
"Lame e-mail check with regexps");
Sample output (pass)
estebanm@estebanm-desktop:~/WebApp/WebAppFunctionalTests$ make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" blahblah...
t/001-login...........ok
t/002-edit_profile....ok
t/003-newsfeed_new....ok
All tests successful.
Files=3, Tests=32, 36 wallclock secs ( 0.61 cusr + 0.08 csys = 0.69 CPU)
Sample output (failure)
estebanm@estebanm-desktop$ make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" blahblah...
t/001-login...........ok 4/8
# Failed test 'get_title, '(?-xism:My blog title)''
# at t/001-login.t line 11.
# 'My blog title - by Esteban Manchado Velázquez'
# doesn't match '(?-xism:My blog title)'
t/001-login...........ok 7/8# Looks like you failed 1 test of 8.
t/001-login...........dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 5
Failed 1/8 tests, 87.50% okay
t/002-edit_profile....ok
t/003-newsfeed_new....ok
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/001-login.t 1 256 8 1 12.50% 5
Failed 1/3 test scripts, 66.67% okay. 1/32 subtests failed, 96.88% okay.
make: *** [test_dynamic] Error 255
xUnit style tests for Perl
- xUnit = Class and conventions for unit testing
- One Perl implementation is Test::Class
- Object Oriented, Agile and buzzword-compliant
Test::Class example
package Test::Lock;
use base 'Test::Class';
use Test::More;
use Lock;
sub basic_usage_works : Test(12) {
my $l1 = Lock->new({ key => 'test1' });
ok($l1->acquire(), 'L1 lock acquired');
ok($l1->acquired(), 'L1 lock is correctly acquired');
lock_exists_ok('test1.lock'); # custom assert
# [...]
}
sub lock_exists_ok {
my ($lock_key) = @_;
return ok((-s $lock_key),
"Lock file for key [$lock_key] should exist");
}
Other Test::Class goodies
# Executed once before every test method
sub setup : Test(setup) {
unlink 'test.lock';
unlink 'test1.lock';
}
# Executed once after every test method
sub teardown : Test(teardown) {
# Remove temporary files or whatever
};
Wrapping up
- Tests: only way to improve code
- But don't stop manual testing!!!11
- Just don't rely on it
TESTS, MÖTHAFÖCKA, DO YOU WRITE THEM‽
