Monday, September 6, 2010

Where are all my stubs?



Update: you must also include a pluginRepositories section. See below for XML snippet.



Update: See here for a sample project and the source code of the compiler integration.




The standard way of compiling joint Groovy-Java code outside of Eclipse has always been through the use of stubs:

  1. Generate Java stub files for the Groovy files
  2. Compile the Java files using the stubs to compile against
  3. Compile the Groovy files

Although this works reasonably well in many situations, there are some complications and problems with this approach, most of which have already been described recently in detail on the groovy-dev mailing list here and here, so I won't go into them in this post.

About a year ago, we introduced Groovy-Eclipse 2.0, which compiles Groovy code by plugging into the JDT compiler and does not need to generate stub files. And as Andy Clement describes, it is possible to run the compiler in batch mode on the command line.

And now, with a little bit of glue code required, I have released a snapshot of the compiler with both ant and maven integration. Although, this is still early work, I do hope that this approach will solve many of the problems that Groovy programmers are having with stub generation. I'll describe below how they both work.

Ant integration for Groovy-Eclipse


Ant integration for the batch compiler is fairly simple.

  1. Download the groovy-eclipse-batch-0.5.0.jar from its temporary location.
  2. Add this jar to your ~/.ant/lib directory.
  3. Once you have that, you need to set the build.compiler property to org.codehaus.groovy.eclipse.ant.GroovyCompilerAdapter.

This will cause ant's javac task to delegate the Groovy-Eclipse compiler for the actual compilation. This means that it is possible to pass any combination of Groovy and Java files to the compiler and most parameters applicable for javac are still available when using the compiler adapter.

A very simple script that uses the Groovy compiler adapter looks like this:

<target name="compile">
  <property name="build.compiler"
           value="org.codehaus.groovy.eclipse.ant.GroovyCompilerAdapter">
  <javac srcdir="src" destdir="bin"/>
</target>

This script sets compiler adapter and compiles all source files in src, placing the resulting class files in bin. Both *.java files and *.groovy files are included in the compilation.

Maven integration for Groovy-Eclipse


Groovy-Eclipse can now also be used from maven. To do so, add the following to your pom.xml.

The artifacts are currently in the SpringSource snapshot maven repo. You must add it as a regular repository:

<repositories>
  <repository>
  <id>springsource</id>
  <url>http://maven.springframework.org/snapshot</url>
  <releases><enabled>true</enabled></releases>
  <snapshots><enabled>true</enabled></snapshots>
  </repository>
</repositories>

as well as a plugin repository:

<pluginRepositories>
  <pluginRepository>
  <id>springsource</id>
  <url>http://maven.springframework.org/snapshot</url>
  </pluginRepository>
</pluginRepositories>

And in your plugin section, you must change the compiler used by the maven-compiler-plugin. Like the javac ant task, the maven-compiler-plugin does not actually compile, but rather delegates the compilation to a different artifact:

<build>
...
<plugins>
  <plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.3.1</version>
    <configuration>
      <compilerId>groovy-eclipse-compiler</compilerId>
      <verbose>true</verbose>
    </configuration>
    <dependencies>
      <dependency>
        <groupId>org.codehaus.groovy</groupId>
        <artifactId>groovy-eclipse-compiler</artifactId>
        <version>0.0.1-SNAPSHOT</version>
      </dependency>
    </dependencies>
  </plugin>
  ...
</plugins>
</build>

This will allow Groovy files to be compiled. The maven-compiler-plugin prefers all source files to be in src/main/java and src/test/java, but if you prefer you can use the standard Groovy convention and keep your files in src/main/groovy and src/test/groovy. You can do so by adding the following plugin to your build section of the pom:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <version>1.5</version>
  <executions>
    <execution>
      <id>add-source</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          src/main/groovy
          src/test/groovy
        </sources> 
      </configuration>
    </execution>
  </executions>
</plugin>

This approach is still in an alpha state and has not been widely tested. It was hard to find reasonably large Groovy-Java projects that use maven for me to try this on. The largest project I have compiled in this way is the GPars project (GPars uses gradle for its build, but I adapted its build.gradle to a pom.xml and successfully ran maven on it). This project includes 168 Java and Groovy files in main as well as 338 Groovy files in test. In a not particularly scientific manner, I did a few runs of building the main and test classes using both Groovy-Eclipse and GMaven and the results are that Groovy-Eclipse is reasonably faster than GMaven for this project:

  • Time to compile main and test classes using GMaven: 36s
  • Time to compile main and test classes using Groovy-Eclipse: 28s

In addition to being largely untested in the wild, there are a few caveats when using Groovy-Eclipse:

  • Since stubs are not generated, GroovyDoc and any other artifacts that rely on stubs cannot be generated.
  • This only supports Groovy 1.7.
  • Third (ant only), your project must have at least one Java file in it (this can be an empty stub), or else ant will finish without compiling anything. There is a patch for this (Bug 48829), but I am waiting for it to be contributed back to ant.
  • Fourth (maven only), your maven project must have at least one groovy file or else compilation will not occur. (Though, if your project doesn't have any Groovy files, then why are you using a Groovy compiler?)

There is still some work to be done, but it is ready enough for people to start trying it out. Feedback is greatly appreciated. You can reply to this blog post, send a message to the mailing list, or raise an issue on jira.

19 comments:

  1. Looks promising, but unusable at the moment:

    [ERROR] Found 1 error and 0 warnings.
    [INFO] 1error

    No other info about line number etc.

    ReplyDelete
  2. Can you please provide a bit more information? Are you using ant or maven? Make sure you are running with the verbose flag turned on.

    Also, you you forward me your pom.xml/build.xml. I can take a look to see if there's anything wrong with it. My email is: andrew at eisenberg dot as.

    ReplyDelete
  3. I'm using maven, but I can't reproduce this issue at the moment.

    Verbose was true - copied from you example.

    BTW, could you please fix maven pom samples:
    groupid should be groupId, and artifactid - artifactId.

    Thanks

    ReplyDelete
  4. Fixed. Let me know if you are able to reproduce the problem.

    ReplyDelete
  5. Haven't gotten this to work yet. The maven compiler plugin isn't picking up the groovy files. Is the source published somewhere, along with an example?

    ReplyDelete
  6. I haven't published the source yet, nor an example. I'll get something out for you to look at.

    ReplyDelete
  7. Ron, I created a new post that contains a link to the compiler integration source code and a sample project:
    http://contraptionsforprogramming.blogspot.com/2010/10/more-on-groovy-eclipse-and-maven.html

    I did see there was one thing missing in the pom.xml snippet from above. You also need to include a pluginRepository section. Perhaps this is the problem that you were having.

    I'd recommend looking at the sample project to see if that can solve your problem. Please let me know if this works for you.

    ReplyDelete
  8. Thanks, working great now. The problem was the maven compiler plugin configuration item was named compilerid instead of compilerId. The example above should be compilerId.

    ReplyDelete
  9. Ah...strange that it worked for me the other way...I made the change.

    ReplyDelete
  10. Ye very promising...far better than creating stubs for groovy

    ReplyDelete
  11. I am using maven/groovy, I am getting compilation error compiling test sources, compiler cant see junit classes. In the pom I have to set the junit artifact scope to 'compile' for the compilation to be successful. When I set the scope to 'test' the compilation fails. I tried the pom file here-> https://svn.codehaus.org/groovy/eclipse/trunk/extras/groovy-eclipse-maven-tests/pom.xml still did not work.

    ReplyDelete
  12. HI Saad,

    What is the failure message?

    ReplyDelete
  13. I get a compilation error

    \swacorp\cebstk\eclipsecp\PathStrategy.groovy - #11/12]
    [writing com\swacorp\cebstk\eclipsecp\GenerateClassPath.class - #13]
    [completed C:\Apps\eclipse_helios_3.6_pide_workspace\cebstk\src\main\groovy\com
    \swacorp\cebstk\eclipsecp\GenerateClassPath.groovy - #12/12]
    [12 units compiled]
    [13 .class files generated]
    ----------
    1. ERROR in C:\Apps\eclipse_helios_3.6_pide_workspace\cebstk\src\test\groovy\com
    \swacorp\cebstk\GenerateServiceRegistryXmlUnitTest.groovy (at line 3)
    import org.junit.Test
    ^
    Groovy:unable to resolve class org.junit.Test
    ----------
    2. ERROR in C:\Apps\eclipse_helios_3.6_pide_workspace\cebstk\src\test\groovy\com
    \swacorp\cebstk\GenerateServiceRegistryXmlUnitTest.groovy (at line 6)
    @Test
    ^
    Groovy:class org.junit.Test is not an annotation in @org.junit.Test
    ----------
    2 problems (2 errors)[INFO] ----------------------------------------------------
    ---------
    [ERROR] COMPILATION ERROR :
    [INFO] -------------------------------------------------------------
    [ERROR] Found 2 errors and 0 warnings.
    [INFO] 1error
    [INFO] -------------------------------------------------------------
    [INFO] ------------------------------------------------------------------------
    [ERROR] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Compilation failure
    Found 2 errors and 0 warnings.

    ReplyDelete
  14. SaadKhawaja,

    This is because the build-helper-maven-plugin maven snippet is wrong. It should be:

    <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.5</version>
    <executions>
    <execution>
    <id>add-source</id>
    <phase>generate-sources</phase>
    <goals>
    <goal>add-source</goal>
    </goals>
    <configuration>
    <sources>
    <source>src/main/groovy</source>
    </sources>
    </configuration>
    </execution>
    <execution>
    <id>add-test-source</id>
    <phase>generate-test-sources</phase>
    <goals>
    <goal>add-test-source</goal>
    </goals>
    <configuration>
    <sources>
    <source>src/test/groovy</source>
    </sources>
    </configuration>
    </execution>
    </executions>
    </plugin>

    ReplyDelete
  15. Thanks Leo. I updated the wiki here:
    http://docs.codehaus.org/display/GROOVY/Groovy-Eclipse+compiler+plugin+for+Maven

    This page will have the most up to date information on the maven compiler.

    ReplyDelete
  16. Did the maven binary for 0.5.1-SNAPSHOT recently change? My builds no longer compile and I had to revert to 0.0.1-SNAPSHOT...

    ReplyDelete
  17. Hi,

    I recently updated to so that the groovy-eclipse-batch jar is based off of Groovy 1.7.10. However, there was a problem with the initial change. It should be fixed now.

    This is the problem that I fixed and if this is what you were seeing then it should be fixed now:

    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.1:compile (default-compile) on project enumtest: Execution default-compile of goal org.apache.maven.plugins:maven-compiler-plugin:2.3.1:compile failed: A required class was missing while executing org.apache.maven.plugins:maven-compiler-plugin:2.3.1:compile: org/apache/xbean/classloader/NonLockingJarFileClassLoader

    ReplyDelete
  18. Shouldn't you be incrementing the version number as to not break existing builds? Is there a plan for a release build :)

    ReplyDelete
  19. I haven't gotten around to making a release build, but since there has been so much interest recently, I'll try to get one out soon.

    ReplyDelete