A few weeks ago, I had a passionate debate with my old friend Nicolas Frankel about the usefulness of the code coverage metric. We started on Twitter and then Nicolas wrote a blog entry stating “Your code coverage metric is not meaningful” and so useless. Not only am I thinking exactly the opposite, but I would even say that not tracking the code coverage on new code is almost insane nowadays.

Read the rest of this page »

Starting with Java Ecosystem version 2.2 (compatible with SonarQube version 4.2+), we no longer drive the execution of unit tests during Maven analysis. Dropping this feature seemed like such a natural step to us that we were a little surprised when people asked us why we’d taken it.

Read the rest of this page »

A new year provides a good opportunity to look back at what was achieved the previous year. This is what I am going to do in this post for the Sonar platform.

Let’s start with a short version of this retrospective. Last year was made of:

• 6 releases of Sonar platform
• 200 releases of ecosystem products
• 65,000 downloads of Sonar
• 12,000+ messages on mailing lists

So I suppose, we can call this a pretty active year for the community. Now, the longer version:

Read the rest of this page »

Code coverage by unit tests is one of the practices that we are pushing the most at SonarSource. We have currently more than 10,000 unit tests running daily to cover the platform and keep adding every day. Of course, Sonar is not the place to start from to do Test Driven Development, Sonar is not an IDE, Sonar will not create unit tests for you and Sonar will not replace precious Agile developers. But once the first bunch of code has been commited, you should certainly use it to manage unit tests and code coverage. The objective of this post is to show how to use Sonar to meet this requirement.

Read the rest of this page »

It has now been almost 2 years since JaCoCo was integrated as a plugin in Sonar and more than 18 months since Evgeny published his post to describe how to Measure Code Coverage by Integration Tests with Sonar. A lot happened since then on this JaCoCo integration into Sonar: creation of an ANT task for JaCoCo, support for branch coverage, integration of JaCoCo in Sonar…

This is why I thought it would be a good time to post an updated version of Evgeny’s post.

Read the rest of this page »

One of the main objective for Sonar in 2011 is to go a step further in the support of Continuous Inspection. Indeed, prior to version 2.5, Sonar could already take a snapshot of the overall quality of an application and view the evolution of quality measures across snapshots with the TimeMachine service. But this was not sufficient to provide at quick answers to some very valuable questions such as :

• what changed in my application over last 30 days ?
• did quality improve during version 2.7 software increment ?
• which violations have been created since 1st of January and by who ?
• how much is new code covered by unit tests ?
• which projects have increased their technical debt during last 3 months ?
• …

Read the rest of this page »

You probably know already that JaCoCo is the most performant code coverage engine. But you might not know that you can now combine it with Sonar to assess the code coverage by integration tests. This was the most voted Sonar issue (SONAR-613) and the latest version of the Sonar JaCoCo Plugin solves it. I am now going to explain how.

Read the rest of this page »

By default, Sonar embarks two tools to calculate code coverage by unit tests on java projects : Cobertura and Clover. But last week, we also released plugins for two other coverage tools : Emma and JaCoCo. Both plugins are available in the Sonar Plugin Library. So I thought it would be a good time to compare all of them and share the results with the community.

Read the rest of this page »

A debate occurred within the team about a year ago to decide whether Sonar should enable to report on error details of failing unit tests. One year later, it sounds bizarre that this debate could take place, given the comprehensive solution provided by Sonar to review and improve unit tests and code coverage.

Here is a one minute video to summarize this one year of effort :

Of course, when you start developing a new application, the Sonar web server is not the place to start from to do Test Driven Development; Sonar is not an IDE, Sonar will not create unit tests for you and Sonar won’t replace your precious Agile developers. But once the first bunch of code has been commited, you should certainly use it to monitor and review the code health and more particularly the unit tests and code coverage.

At a glance, you get it all on the Sonar project dashboard : total number of tests, tests in error, tests in failure, skipped tests, code coverage and total duration of unit tests. On any of those measures, you not only get an indication of the short term trend through the usual green and red arrows but also alerts if any threshold has been reached.

Then, to get the bigger picture, you can access the TimeMachine to see evolution of the latest versions/snapshots. In the following example, number of unit tests and code coverage are increasing whereas unit tests duration decreases, which is pretty good :

Obviously, when you have seen all the numbers, you will want to understand how it breaks down. So on every measure, you can click to drill down.

In this example, you see that a unit test fails in FindbugsRulesRepositoryTest.java but what is wrong ?

And what’s true for unit tests in error is also true for code coverage: drill down and then display code coverage detail:





In fact, Sonar offers multiple entry points (aka hunting tools) to improve code coverage. That really depends of what you are looking for.

1. To make sure coverage is homogeneous across packages, simply use the treemap

2. To understand why a module has low coverage, drill down on the percentage

3. For quick wins, check the classes cloud: any big red class is a target

4. To reduce risk on the project, the best entry point is the “Most complex & less tested files” in the Hotspot where files that have the highest uncovered cyclomatic complexity are shown


To get all that, you just have to :

1. 1. Download Sonar
2. 2. Unzip Sonar
3. 3. Launch the Sonar web server (sonar.xx start)
4. 4. Launch the Sonar Maven plugin (mvn sonar:sonar)