Category Archives: TDD

100% Code Coverage via automated tests in Symfony applications

On 2017-04-26 the Symfony User Group Cologne gave me the opportunity to speak about Code Coverage.

If you create software that is planned for continuous enhancements and maintenance over several years then you’ll need a sustainable strategy for code quality.
Code coverage by automated tests is one important metric, and its value should equal 100% at all times.
My talk shows why this is so and how it can be achieved in PHP / Symfony based applications.

See the full presentation at Speaker Deck

Natural Enemies of “Moving Fast” in Software Development Projects, Part 1

“Writing tests is the natural enemy of moving fast.”

Some of our dev teams create software using Adobe Experience Manager (AEM, formerly known as CQ).
When you do AEM or Apache Sling then the annual adaptTo tech meetup in Berlin is the place to go.

This year, i.e. September 2016, I heard an interesting phrase.
“Writing tests is the natural enemy of moving fast.”

I was surprised.
Sure, if you’ve written only production code in a project and only later on start writing the tests then that will take a lot of time, it will likely be painful and it will slow down progress.

But the 2nd time I heard the phrase was in a talk about TDD, and that’s when I was really surprised.
I’ve been doing XP and TDD since 2005 and – as far as my experience goes – writing tests (and preferably early) is key if you want to move fast in any project of considerable size (… and AEM projects usually are of considerable size).

But I might be wrong, so let’s do an example coding task.

The irritat0r coding task


Let’s create a functionality that displays personalized texts to web users
in order to irritate the hell out of them
(which is my understanding of what advertising is basically all about).

The text shall look like this:
Hey ${salutation}, did you know that ${message}?“.

    • If the user is known, e.g. a logged in principal, then salute her by the first name.
    • If the user is anonymous then salute her by just “you”.
    • As for the message it shall be possible to send a String message.


  • Hey Andreas, did you know that the longest recorded flight of a chicken was 13 seconds?
  • Hey you, did you know that it is 8 times more likely to get killed by a pig than by a shark?

In version 1 we will implement the following message types:

      • text message, i.e. a fixed text, like
      • message that displays how many kilometers earth has moved around the sun since the user’s birthday, which of course only applies to logged-in users who provided their birthday.

To keep things proper irritat1ng the application shall choose the message randomly from a pool of all appropriate messages.

In version 1 the functionality shall be available as

  • a HTTP response of type text/plain.
  • a JSON representation
  • an AEM component

As for non-functional requirements:

  • Production code shall be 100% covered by tests
  • The code shall be part of a larger system
  • The code shall allow for being changed and extended for new functionalities


Solution Outline

First, let’s do a short design session:

  • We’ll create a Java / maven OSGi bundle called “irritat0r“.
  • We’ll create an AEM servlet called “Irritat0rAemServlet” which allows access to the HttpSession and will send the irritat0r text via the HttpServletResponse
  • We’ll get the user’s id from the HttpSession if HttpSession contains such an id
  • If it does we’ll use the (existing) UserService and get the User object by that id.
  • We’ll create a function that creates the salutation (for known user or anonymous)
  • We’ll create a function that gets 1 appropriate message from the pool of messages.
  • We’ll create a function that assembles the text out of the salutation and the message.
  • (intentionally left blank… will follow in part 2)

So, where should we start?

We could start with the OSGi part

To do so we would need some OSGi container which needs to be set up and started.
Luckily, OSGi requires us to start it only once – but we’d need some way to deploy the bundle into the container.

We could deploy manually

  • Compile the code
  • Package it as a jar including the meta data
  • Open Felix (or Equinox etc)
  • Choose Bundle
  • Upload
  • Restart Bundle

That’s what needs to be done each time we’ve changed the code and we want to see the effect.

I guess we agree that this is a waste of time so let’s state that
“Manual work is a natural enemy of moving fast.”


We could use Maven

(or Gradle etc)

So the manual part is transformed into an automated process. Good.

But we’d still need to run Maven each time we’ve changed the code and we want to see the effect.

You may have become accustomed to this procedure but, believe me, somewhere out there PHP, Ruby, JavaScript Developers and even HTML coders are laughing at a random Java hacker right now.
How long do your Maven scripts take? 30 seconds? 2 minutes? Now multiply that times the number of changes you do to your code.

I believe that
“Deploying to verify code changes is a natural enemy of moving fast.”


We could fake it

There’s a reason why ZERO TURNAROUND has been successful with JRebel, and the reason is Maven. And ant. And, you name it.

Using JRebel would indeed remove the need to run Maven for verifying code changes – at least for code without OSGi annotations. We’ve been using JRebel in AEM projects. It works for “normal” code but it cannot (reliably) redeploy OSGi services and Servlets, so it has been Maven time for us again.

Also, JRebel is not cheap – it will cost you ~ $475 (as of 2016-10-10).
Per year.
Per seat.


But there’s another reason

and it’s less obvious: It’s what happens to the way you think.

You probably know that writing tests for an OSGi service is not the easiest task you can think of.
You might need to install OSGi Mocks (which are in fact stubs, not mocks), Google for code examples and fiddle around with them.
You’ll probably not make the 100% coverage but you’ll have an excuse because, hey, “that’s hard to test” or “that part needn’t be tested”.

Eventually, you’ll get used to a code coverage below 100% before the project even started.

And you’ll be so happy you made it run after all the hard work that you might feel inclined to add more code – i.e. domain logic –  to the OSGi service.
As if it was to make it pay off the pain.

Chances are that this additional code will not be fully covered with tests either, partly because it will be inside of a class that is hard to test, partly because you have already put up with less than 100% coverage.
Test coverage will be the first “victim” of that approach, and the Broken Window Theory will take its toll on you.

Second, putting domain logic into the OSGi service would break Clean Code practices like the Single Responsibility Principle.
Also, as OSGi is the container and hence by definition located on the outside layer of the application, it would violate the Clean Design principle that “All dependencies must only point inwards“.
See also “Ports & Adapters“.

Those violations will eventually lead to messed up dependencies, and, boy, if you really want to slow down development then screwing up your dependencies is a pretty efficient way to do so.

That’s why
“Losing control of dependencies is a natural enemy of moving fast.”

And thus
“Binding domain logic to external devices, ports etc is a natural enemy of moving fast.”

Third, if you need to deploy to verify your code you’ll probably verify your code less often.
If you know that running Maven takes 2 minutes you might not want to run it after each code change.
Instead you’ll keep piling up code changes and deploy them all at once, every 15 minutes or so, and go get a cup coffee while Maven is running.
Sounds familiar?
(Maybe Maven is secretly sponsored by the coffee machine industry, but that’s just a theory.)

Chances are that parts of the code you created within those 15 minutes turn out to be flawed. Sometimes it will be “just a bug”.
But sometimes you will realize that you’ve chosen the wrong approach and now have to delete, change and redo a major part of the code. And as you piled up code changes this might lead to a significantly large piece of rework. That’s of course a waste of time and it will slow down progress.

And that’s why
“Verifying code changes in chunks of more than just a few lines is a natural enemy of moving fast.”.


…and there’s nothing to learn here

Lastly, why would we want to start by testing OSGi?

It’s safe to assume that OSGi works, that’s why we use it in the first place.
There’s nothing to learn about the domain here.

We could start with the Servlet/JSP part

Alternatively we could start by coding the Servlet or maybe some JSP.

However we’d be facing the same problems:

  • We’ll need to deploy each time to see the effect.
  • Testing web output will require additional effort like setting up Selenium, HtmlUnit etc
  • Getting 100% coverage will be hard to do and we might put up with less than 100%.
  • Again, you might want to make the pain pay off and add more code, i.e. domain logic, to the Servlet or – even worse – the JSP.

There may be something to learn here, though, i.e. the look and feel of the front end.


…but you might be F5’ing a lot

Making sure the front end looks the way it should is an important task, and I admit it’s still pretty hard to do so without manual work. (BTW: has anyone evaluated Galen yet? I’m curious…)

However, if that requires running mvn install each time you want to preview you UI then that’s a waste of time. (HTML coders like immediate feedback, e.g. by using Firebug, etc.)

Also, once you have set up this process you might feel inclined to use the UI to also test functionality, not just look and feel.
As a result you’ll be pressing F5 a lot (-> waste of time), comparing the browser’s content to some expectation about the outcome.
In order to do so you’ll be using – your eyes.

First, it’s a manual process and we already know that’s a natural enemy.

Second, it’s exhausting. You might feel as if you were moving fast.
You’re not.
In fact you’re just keeping yourself busy and the way you experience time will change.

Third, your eyes will only see what they are looking for.
If your code change has broken stuff that your eyes are not looking for then you’ll miss that bug.
Let alone if stuff has got broken in other areas of the site, i.e. pages you did not F5. You’ll miss those bugs as well.

And delivering a bug, even more so an avoidable bug, thus making testers report it back and then fixing it, is a HUGE waste of time (cf. Cost to Fix), so
“Bugs are natural enemies of moving fast.”

And that’s why
“Testing domain functionality via the UI is a natural enemy of moving fast.”

First pay-off

We haven’t written a single line of test or production code yet.
However, the Test Driven Development approach has already paid off I think:
TDD kept us from starting with the wrong tasks.
It prevented us from using “Natural Enemies of Moving Fast”.

For some real coding stay tuned for Part 2 which I’ll publish soon and which will include a lot of code examples, I promise.


Spoiler alert

Where should you start?
Start at the core, i.e. the domain logic.

You’ll easily be at 100% coverage right from the beginning and you’ll stay at 100% from that point of time on.

What do you think?
Please leave a comment and let me know.

If you like this post then please share it.

Happy hacking!