Iced Tea in Crete 2012:Development and Testing Tools

JUnit

 * as tool for exploring new languages
 * custom JUnit runners (e.g. SpringJUnitTestRunner, for AJ)
 * common base class for default annotations

TestNG

 * different groups, annotate tests, (fast tests)
 * more memory as it loads all classes upfront

Asserts

 * scala, groovy better error messages from expressions
 * extract custom assert methods
 * fest-assert http://code.google.com/p/fest/

Hamcrest

 * assertThat(value, hasItems)
 * integrated in JUnit (oolder version)
 * some default matchers come with JUnit

Mocking

 * http://martinfowler.com/articles/mocksArentStubs.html
 * created mocks/stubs
 * brittle tests with mocks, you test behavior not resulting state
 * mock as little behavior as possible


 * Mockito: (http://code.google.com/p/mockito/)
 * adds more default Hamcrest Matchers, easy to use, fluent setup of behavior,
 * verify # of interaction, nice defaults
 * annotations for injecting mocks


 * EasyMock (http://www.easymock.org/EasyMock3_1_Documentation.html)
 * very simple, mock and replay
 * JMock(2):


 * PowerMock: mock final methods, static calls and java.lang classes
 * self made stubs

MultiThreading?

 * separate threading behavior from other behavior
 * use java debugger architecture to suspend/continue tests
 * in JUnit @Concurrent Annotations
 * better concurrency support in TestNG
 * write your own testing infrastructure with CountDownLatch (or Phaser in Java7)
 * watch out for shared state, also if you want to run your tests in parallel
 * hard to do in current frameworks

SpringJunit4TestRunner.class

 * automatically rolls back by default so doesn't keep state
 * @Rollback(false)
 * @Before executed in TX, @BeforeTransaction is executed outside of TX
 * @Transacitonal on Test-case makes each method transactional

Cleanup DB?

 * VM(ware) with snapshot and no persisent disk for reset
 * HyperV
 * saved state in db

Database

 * DBUnit ?
 * too big an effort to maintain (all the xml files) - own framework
 * In Memory DB?
 * different contexts
 * problems with schemas ? different SQL dialects? (better with ORM)
 * HSQL, ImpermanentGraphDatabase


 * LiquiBase?
 * HttpUnit?


 * Infrastructure that automatically starts up embedded or external servers
 * fast, slow, external tests (with external services - JMS, Kerberos, LDAP, ...)

BDD

 * write acceptance criteria in business domain language
 * readable/writable by business users
 * the tool reads those statements and extracts relevant information which are fed to the tests
 * Given When Then / As a ...., I want ...., so that ...

Cucumber

 * http://cukes.info/
 * in lots of languages
 * step definitions use regular expressions to extract the data for input and verification

JBehave(2)

 * difficult to express specs

ScalaTest

 * http://www.scalatest.org/user_guide/tests_as_specifications

Fit / Fitnesse

 * Fit (http://fit.c2.com/) by Ward Cunningham - uses tabular specs in Word/HTML documents to define Acceptance tests
 * FitLibrary
 * Fitnesse (Uncle Bob) Wiki based implentation (http://fitnesse.org/)
 * plug in spreadsheets (excel)

Performance/Scaling

 * JMeter for http performance testing (http://jmeter.apache.org/usermanual/get-started.html)
 * curl, ab load testing
 * Profilers (VisualVM, YourKit, JProfiler)


 * make sure the client and the system under test run on different systems
 * record: gc-logs
 * record stats: min, max, avg, mean, stddev look at the outliers
 * vmstat, iostat
 * Kirks Performance Test courses
 * you have to have performance requirements so that you know when you meet your performance needs