Code coverage for Maven projects in NetBeans

The problem

When activated for a Java project the code coverage plugin hooks into NetBeans' ant based build process to instrument generated class files using the EMMA code coverage library before performing any action that would execute code.

Although NetBeans supports maven projects as a native project type Maven projects, naturally, use maven for performing the build. This means that the code coverage plugin no longer has any method of instrumenting the classes and therefore is not enabled.

The loophole

Poking around the file structure of a NetBeans Java project with code coverage activated you find the following files created;

• nbproject/coverage.properties
• coverage/emmascript.xml
• coverage/coverage.emma
• coverage/template.emma

The first of those files is just a simple java properties file with references to the location of the two .emma files. The second is a small ant project definition with one task that performs instrumentation and the remaining two files are the results of instrumentation and running the instrumented code.

The presence of these files in an easy to access form got me wondering if just generating valid EMMA files would be enough to get the coverage feedback I desired. A couple of emails to the author of the code coverage plugin and a quick peek at the source code, confirmed that when opening a Java file, as long as the project contains the coverage.properties file, then it will attempt to render the coverage highlights in the editor.

The solution

With this information at hand it became obvious that I just need to have maven run the EMMA instrumentation from my maven build before running any tests. There is an EMMA plugin for maven that can perform this instrumentation however the documentation doesn't reveal any option for in-place instrumentation of classes, without this I would need to mess around with the classpath when running tests and that's a lot more effort than just running the EMMA instrumentation with the maven exec plugin. First we need to add EMMA as a project dependency:

<dependency>
  <groupId>emma</groupId>
  <artifactId>emma</artifactId>
  <version>2.0.5312</version>
</dependency>

A quick side note to Java 1.6 users on 64bit architectures. If you receive errors during instrumentation you should use a patched version of EMMA, see the bug report for details.

Then configure the exec plugin:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <phase>process-classes</phase>
      <goals>
        <goal>java</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <mainClass>emma</mainClass>
    <commandlineArgs>instr -verbose -m overwrite -ip ${project.build.directory}/classes -outfile ${project.build.directory}/coverage.em</commandlineArgs>
  </configuration>
</plugin>

The above config will, during the process-classes phase of the maven lifecycle, instrument the project's class files and write some metadata to the the target/coverage.em file. When executing the test goal in maven the instrumented classes will be tested and they will write out runtime coverage information to a coverage.ec file in the root of the project. Being Maven friendly developers we don't like build artifacts lying around all over the place so to have this file also created under the target directory we need to add a system property to the test execution using a plugin configuration similar to the following:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <systemProperties>
      <property>
        <name>emma.coverage.out.file</name>
        <value>${project.build.directory}/coverage.ec</value>
      </property>
    </systemProperties>
  </configuration>
</plugin>

With those two changes in place we now have EMMA instrumenting the code and producing coverage report files however NetBeans doesn't know where these files are. One solution to this is to hand write the coverage.properties file but;

1. I don't like to manually write files for each project
2. The code coverage plugin has a habit of overwriting this file in order to turn itself off (not investigated why)

So my solution was to also generate this using maven:

<plugin>
  <groupId>org.codehaus.groovy.maven</groupId>
  <artifactId>gmaven-plugin</artifactId>
  <executions>
    <execution>
      <phase>initialize</phase>
      <goals>
        <goal>execute</goal>
      </goals>
      <configuration>
        <source>
        def nbdir = new File(project.basedir, "nbproject")
        def coverageProps = new File(nbdir, "coverage.properties")

        ant.mkdir(dir:nbdir)

        ant.echo(file:coverageProps, message:"""
        #coverage viewer module properties for project
        # Generated by Maven Configuration from sharebear.co.uk
        coverage.activity=ON
        project.type=java
        coverage.templateFile=target/coverage.em
        coverage.coverageFile=target/coverage.ec
        """)
        </source>
      </configuration>
    </execution>
  </executions>
</plugin>

In hindsight I could have implemented this last bit using the ant runner plugin but I'm trying to use Groovy whenever I get the chance, in order to familiarise myself with the language.

Putting it all together I have wrapped the above three configurations into a maven profile so that I only produce instrumented classes when I need them and don't affect a regular build making a minimal configuration look like the following:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>uk.co.sharebear</groupId>
  <artifactId>coverage-hacking</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>coverage-hacking</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>emma</groupId>
      <artifactId>emma</artifactId>
      <version>2.0.5312</version>
    </dependency>
  </dependencies>
  <profiles>
    <profile>
      <id>coverage</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.codehaus.groovy.maven</groupId>
            <artifactId>gmaven-plugin</artifactId>
            <executions>
              <execution>
                <phase>initialize</phase>
                <goals>
                  <goal>execute</goal>
                </goals>
                <configuration>
                  <source>
                  def nbdir = new File(project.basedir, "nbproject")
                  def coverageProps = new File(nbdir, "coverage.properties")

                  ant.mkdir(dir:nbdir)

                  ant.echo(file:coverageProps, message:"""
                  #coverage viewer module properties for project
                  # Generated by Maven Configuration from sharebear.co.uk
                  coverage.activity=ON
                  project.type=java
                  coverage.templateFile=target/coverage.em
                  coverage.coverageFile=target/coverage.ec
                  """)
                  </source>
                </configuration>
              </execution>
            </executions>
          </plugin>
          <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <executions>
              <execution>
                <phase>process-classes</phase>
                <goals>
                  <goal>java</goal>
                </goals>
              </execution>
            </executions>
            <configuration>
              <mainClass>emma</mainClass>
              <commandlineArgs>instr -verbose -m overwrite -ip ${project.build.directory}/classes -outfile ${project.build.directory}/coverage.em</commandlineArgs>
            </configuration>
          </plugin>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <systemProperties>
                <property>
                  <name>emma.coverage.out.file</name>
                  <value>${project.build.directory}/coverage.ec</value>
                </property>
              </systemProperties>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
</project>

Using this configuration if you activate the coverage profile in the project's context menu in NetBeans, then run a clean test, any java files opened after that will have coverage information rendered for them. A very simple sample project with the above pom file can be found on my mercurial server here.

Final thoughts

This solution doesn't enable the coverage reporting menu, for that you would need to modify the code coverage plugin sources to enable the coverage menu for maven projects and I don't fancy maintaining a fork of the plugin.

As part of the investigation of how to approach this I did take a look around the code for the ongoing work to implement code coverage for scripting languages such as Ruby and Python to see if there was an alternate way I could render usage reporting. Unfortunately from what I could see this code is more closely tied to the project type meaning that getting code coverage functionality for a maven project that includes scripting languages such as Groovy and JRuby sources would be a lot more difficult.