Eating The Dog Food... In Public

At SonarSource, we've always eaten our own dog food, but that hasn't always been visible outside the company. I talked about how dogfooding works at SonarSource a couple years ago. Today, the process is much the same, but the visibility is quite different.

When I wrote about this in 2015, we used a private SonarQube server named "Dory" for dogfooding. Every project in the company was analyzed there, and it was Dory's standards we were held to. Today, that's still the case, but the server's no longer private, and it's no longer named "Dory".

Today, we use next.sonarqube.com (nee Dory) for dogfooding, and it's open to the public. That means you can follow along as, for instance, we run new rule implementations against our own code bases before releasing them to you. We also have a set of example projects we run new rules against before they even make it to Next, but seeing a potentially questionable issue raised against someone else's code hits a different emotional note than seeing it raised against your own.

Of course, that's the point of dogfooding: that we feel your pain. As an example, take the problem of new issues raised in the leak period on old code. Since we deploy new code analyzer snapshots on Next as often as daily, it means we're always introducing new rules or improved implementations that find issues they didn't find before. And that means that we're always raising new issues on old code. Since we enforce the requirement to have a passing quality gate to release, this causes us the same problem you face when you do a "simple" code analyzer upgrade and suddenly see new issues on old code. Because we do feel that pain, SonarQube 6.3 includes changes to the algorithm that sets issue creation date so that issues from new rules that are raised on old code won't be raised in the leak period.

Obviously, we're not just testing rules on Next; we're also testing changes to SonarQube itself. About once a day, a new version of SonarQube itself is deployed there. In fact, it happens so often, we added a notification block to our wallboard to keep up with it:

By running the latest milestone on our internal instance, each UI change is put through its paces pretty thoroughly. That's because we all use Next, and no one in this crowd is meek or bashful.

Always running the latest milestone also means that if you decide to look over our shoulders at Next, you'll get a sneak peek at where the next version is headed. Just don't be surprised if details change from day to day. Because around here, change is the only constant.