OOPSLA Day 1: Architecture meets testing

As an architect, do you...

• involve yourself in the tasks of 
• Devising a testing strategy?
• Helping test managers build a test architecture?
• Promoting test design methods?
• consider testability as a quality attribute to be addressed just as you do with any other?  
• really practice test driven design (i. e. starting from the definition of the architecture)?
• care about the effectiveness and efficiency of the team who is gonna test your stuff?

As far as I see, it´s unlikely that you answer "yes" to most of the questions above. But you should.
Why? Because your architecture is not something that abstract as the maya in hinduism. It's the very foundation on which working software will be built. So your responsibility is not just delivering an architecture (not to say a Power Point), but a strategy for building working software based on that architecture.

Peter Zimmerer's tutorial gave me great insight. Lately I've discussed some ideas about testing strategies and testability with my team's new test manager, and raised questions such as:

• How do I design my systems and programs in order to be more testable?
• How should the testing teams apply their effort in the most valuable way? 

Besides this we started using sonar as a code quality management all-in-one-place tool. But Peter's talk just revealed that I had still a lot of work to do. Some of the lessons are:

• Devise a risk based testing strategy.
• Metrics about bugs, coverage and testing progress should be your drivers for test exit criteria.
• Coverage has very different meanings: risks, quality attributes, features, requirements, use cases, user profiles, operations, data, platforms, etc.
• Practice architecture testing. Don´t wait for functionality to be built.
• If you practice TDD, there´s no reason for not doing it from the very beginning: architecture.
• Integration testing != integration. 
• Integration testing usually takes most of testing effort.
• Design for testability. Testability = f ( Observability,  Controllability, Modularity, ... )
• Internal code quality affects directly external system quality.

This one last point may be better represented with a very recent story.

As I've said, I dropped a sonar instance a few weeks ago, and the very first project which got analyzed had respectable quality metrics (it's OSGi based and highly modularized), except for the web module. Then integration of the app began, and the oldest code in the web module (no need to say that the project had been asleep for a while due to organization priority changes) started giving problems: ugly code, specifically in 2 classes.

Then testing began, and although I had to stop making commits to the project, I saw that these 2 classes continued to give trouble to the rest of the team. I jumped to the sonar board and guess what I found: those 2 classes where the  ones with the most violations and the worst metrics. They urgently need a refactoring.

So, this is not talking. Code quality metrics matter for very different purposes.

If you are not that bored enough by now, you may check out this tool: Testability Explorer. It calculates a metric about the suitability of your code for unit testing. How it does what it does? According to the project site, it computes

• Non-Mockable Total Recursive Cyclomatic Complexity (What?!!! Fortunaley there's an explanation for it).
• Global mutable state.
• Law of Demeter violations.

I have found it very useful. Although I've had not time to delve into the relation of the above metrics to compute the final result or integrate the analyzer as part of sonar (a must!)

Next post will be about the workshop on assessment of contemporary modularization techniques =)