xUnit Roslyn Analyzers

Putting your code under the scope

David Pine

5 minute read

Intro

At my day job, I wear many hats. I’m currently wearing the “upgrade our team’s tooling” hat. I counted 99 *.xproj and project.json based projects spanning four repositories that need to be updated to the revitalized *.csproj project format. With this the team can upgrade to the latest version of Visual Studio (finally upgrading to 2017 from 2015) and start taking advantage of the following:

  • Supported tooling, no longer on preview tooling bits
  • Allows us to explore C# 7 (and peruse C# 7.1 ) with production code
  • Leverage the latest SDK and .NET Core CLI
  • Other obvious reasons
  • And so on…

Upgrading tooling is not always glamorous, but there are times that I find myself excited. I thoroughly enjoy learning about the new tooling innovations and efforts around simplifying the development work-flow, especially when it involves C# and Visual Studio.

An “OMG, WTF” Moment

I mandate that our projects “treat warnings as errors”. After migrating the first solution over, 15 of the 30 projects had compilation errors. It’s worth mentioning that we maintain a 1-to-1 ratio between class libraries and unit-test projects. Much to my surprise the 15 projects that had compilation errors were only unit-test projects. The reason that they stopped compiling was due to seemingly countless xUnit warnings, generated by various Roslyn analyzers. I was shocked by the number of warnings that were discovered.

The Realization

As one might imagine – there are a lot of unit tests in 15 projects. This was an opportunity for Roslyn and xUnit’s analyzers to shine, and shine they did! At first I was frustrated…thinking something like “FML, of course there is an issue” but upon further investigation I discovered that these were all perfectly reasonable warnings. In fact, several of them made me question how some tests ever passed at all. I was impressed by the power of this tooling addition. While Roslyn has been around for a while, I have not personally observed a large uptake in its abilities. Needless to say, I was thrilled to see the xUnit team leveraging it so much – this is inspiring!

Visual Studio 2017 will display the analyzers that are available within a project under Dependencies > Analyzers.

Analyzers

As you can see they have put a lot of thought and effort into this. There are tons of various rules, here is a comprehensive listing of them all.

xUnit

I’m dating my xUnit efforts here a bit, but I’ve been a fan of xUnit for a long time now! Back in the days of “DNX”, or as Scott Addie might ask DMX? . In a time when I was still on Wordpress , before I was enlightened my Steve Hicks who says:

If you aren’t putting out as many posts as BuzzFeed, you can probably just build your blog statically Steve Hicks

Here is a post from February 2016 where I flexed some of the xUnit capabilities. Again, I’ve been a fan of xUnit for a long time now. It is incredibly

Roslyn

If you’re unfamiliar with Roslyn, it is the .NET Compiler Platform.

Roslyn provides open-source C# and Visual Basic compilers with rich code analysis APIs. It enables building code analysis tools with the same APIs that are used by Visual Studio.

Visual Studio relies on Roslyn for lots of things. All of the aforementioned xUnit rules are defined as Roslyn analyzers. These analyzers are executed within Visual Studio, providing a great developer experience. They help you write better code.

Visual Studio – Warnings

This section will serve as a brief listing of some of the warnings that I encountered. When writing a unit-test that uses Theory paired with InlineData, if there are duplicates – this is flagged.

When you mistakenly define a unit-test method parameter that is not used, it’s flagged.

When your unit-test is attempting to check whether a collection “does not contain” something, and does so with an Assert.False(collection.Any(...)); – it is flagged.

This one might be my favorite – if you incorrectly Assert.NotNull on a value-type (that can never be null) – it is flagged.

When you call Assert.Equal and the expected argument is null – it is flagged.

I also ran into two variations of the xUnit:2013 warning which states the following:

Do not use equality check to check for collection size

In one scenario I had some unit test code similar to Assert.Equal(0, values.Count);, the analyzer instead suggested Assert.Empty(values);. This was my intent anyways, so of course I was happy to accept the recommended change.

Likewise I had an Assert.Equal(1, values.Count); and the analyzer suggested Assert.Single(values);. I was ensuring that the collection had only a single value, so once again xUnit was correct. Again, my appreciation for this tooling enhancement cannot be overstated.

Thank you

Further reading

If you’re curious about xUnit or Roslyn, please checkout the resources below:

comments powered by Disqus