Automated Acceptance Testing Video
Elisabeth Hendrickson provides a lightning talk regarding how important automated acceptance testing is.
For what it's worth, I agree with her. Automated acceptance tests are very important. So, why don't more teams do this? It has been my experience that many teams evolve quickly to perfect a developer centric view of testing - focusing on the testing of interfaces, classes, methods, etc. Teams work hard to integrate these tests into automated builds and really rally around making "tests" first class citizens during the development process. This is absolutely necessary; however, the evolution of an automated testing mindset should not stop there. In too many teams, any form of automated testing begins and ends with developers. When the development team releases the software to the acceptance phase they hand it over to the BA's who run through scores of manual tests (or simply perform exploratory testing) to find defects. Of course, this is a lot of work - and because of this I have often seen organizations provide for this by creating a really long QA "phase" of a project - where all acceptance testing is done at the same time. All of a sudden, this seems very "waterfally" in nature - which kinda blows the entire point.
The problem with automated acceptance tests is that they are relatively hard to create and, more significantly, hard to maintain. I have seen many projects attempt to fully automated all functional tests - using large teams of QA specialists to create and maintain the tests. This was EXTREMELY hard work and in many cases did NOT provide the expected results. In fact, on many projects test case maintenance added an enormous amount of effort and time onto the project - and resulted in an extremely NON-AGILE attitude about requirement change (small changes in requirements may be simple to code however result in a lot of work for those maintaining the automated acceptance tests - therefore the "cost of change" becomes quite significant and thus change is discouraged).
When I look to the future, I want to see a much improved emphasis on a fully integrated experience for creating and maintaining automated acceptance tests.