Friday, March 26, 2010

AJDT-PDE builds Redux

AspectJ-aware PDE builds in AJDT have always been a bit of a kludge. So, I'm happy to say that as of Eclipse 3.6 (Helios), things will be changing for the better and AJDT will rely on a different mechanism to perform PDE builds.

PDE, or the Plugin Development Environment, is the tool set that helps developers build plugins for the Eclipse platform itself. In Eclipse 3.5 and earlier, AJDT shipped with its own version of the PDE tools (AJDT-PDE) that were largely a copy of Eclipse's PDE, but with a few key changes so that a different compiler could be plugged in. It is an understatement to say that this was inefficient. The AJDT-PDE had to be re-created for each point release of Eclipse so that it could be re-based off of the new version of PDE. This process was fallible and inevitably, some features of PDE were missing or broken in AJDT-PDE.

I have been working with the PDE team to create a more sustainable solution to AspectJ-aware PDE builds. From AJDT for Eclipse 3.6 and onwards, the AJDT-PDE will be removed. The AspectJ-aware export wizards will be deprecated and the org.eclipse.ajdt.pde.build plugin will be redundant. Instead, users can use the standard PDE build, with the following changes to the build.properties file of your plugin project:

# required
compilerAdapter=org.eclipse.ajdt.core.ant.AJDT_AjcCompilerAdapter
sourceFileExtensions=*.java, *.aj

# optional
compilerArg=-aspectpath other.jar


The first line specifies that the AspectJ compiler should be used instead of the Java compiler. The second line specifies what file types should be considered source files. And optionally, you can specify a number of command line arguments using the compilerArg property. Only AspectJ-specific arguments like inpath and aspectpath need to be specified here. For all standard Java options, like source and target you can use the standard PDE-Java options javacSource and javacTarget.

From here, you can use the standard PDE export wizards or PDE headless build scripts to build your project. That's it. There is now nothing special needed for AspectJ-aware PDE builds.

Furthermore, it is possible to create your own compiler adapter so that you can use your own custom compiler for your own JVM language (and avoid using javac). You need to do the following:


  1. Implement org.apache.tools.ant.taskdefs.compilers.CompilerAdapter and delegate to the compiler of your choice

  2. Ensure that this new compiler adapter is on the ant classpath. I recommend using a org.eclipse.ant.core.extraClasspathEntries extension point so that your plugin automatically contributes the classpath to Eclipse's ant runtime. But, you could also specify it in your global ant classpath properties page.

  3. In your build.properties file, fill in the proper values for the compilerAdapter, the sourceFileExtensions, and the compilerArg properties.


That's really all there is to it. For all the gory details on how this works see Bug 303960.

Friday, March 19, 2010

Groovy-Eclipse 2.0.1 Released and see you at EclipseCon!

We are proud to release to announce the release of Groovy-Eclipse 2.0.1. This release is primarily a service refresh of 2.0, but there are a couple of interesting new features including debug stack frame filtering and Eclipse 3.5.2 support. You can read the New and Noteworthy.

The update site for the release is here:
  http://dist.springsource.org/release/GRECLIPSE/e3.5/
This is also a good time to promote my talk at EclipseCon. I will be speaking about Groovy-Eclipse on Wednesday, March 24 at 2pm.

Getting Groovy with Eclipse: Next Generation Tool Support for Dynamic Languages. Come hear how Groovy-Eclipse has extended the JDT compiler to provide deep integration between Groovy and Java. Also hear about how Groovy-Eclipse provides editing support for Groovy's dynamicism through type inferencing.

See you at EclipseCon!

Wednesday, March 17, 2010

AJDT for Helios now available

Just a quick note to let you know that AJDT targeting Helios (Eclipse 3.6) is now available from the following update site:
  http://download.eclipse.org/tools/ajdt/36/dev/update
AspectJ-aware PDE export facilities are currently broken. I am not planning on fixing it. Instead, I have been working with the PDE team on an alternative solution. See Ability to override javac task during pde builds (Bugzilla). I'll write more about this later, but the solution is much more elegant than what is currently available in AJDT for 3.5 and earlier.

Also, as a bonus, this version includes intertype declaration (ITD) -aware Java search. Finally, ITDs are completely first-class citizens in the workspace. I'll write more about this later, too.

Friday, March 5, 2010

Intertype Declaration support in Java projects

A user recently had a problem with Roo and AspectJ. This particular user had a Roo project being referenced by a regular Java project. Roo is a rapid application development tool for Java. It makes heavy use of AspectJ's Intertype Declarations (ITDs). So, all Roo projects re AspectJ projects. And here is where the problem is. When an AspectJ project makes use of ITDs, Java projects that reference it cannot reference the ITDs without errors. Well actually, the Java projects compile and run correctly, but there are spurious error messages that appear in the editor, and ITDs do not appear in content assist.

A simplified version of this problem is below:

fail

Here, there are two projects, an AspectJ project and a Java project. The AspectJ project has an aspect that declares an ITD on a Java class. This Java class and its ITD is then referenced in the Java project. Even though the Java project compiles and runs correctly, an error appears in the editor.

Thankfully, there is a simple fix to this. Open up the .project file for the Java project and add the following line to the nnatures element:
  <nature>org.eclipse.ajdt.ui.ajnature</nature>


(Note: this line should be the first nature if you want the AJ decorator to appear on the project icon.)

What this does is tell AJDT that this project should be treated like an AspectJ project, but it does not change the compiler, so the Java compiler is still used. The result is that ITDs are now correctly recognized in the editor:

success

In the future, I'll probably add a quick fix or some other automated way of doing this in future releases of AJDT, but for now this manual process is a reasonable workaround.

Wednesday, February 24, 2010

Approaches to Extending JDT

As most of you who have already read my blog know, I write a lot about extending Eclipse's Java Development Tools (JDT) for use with Java-like languages such as AspectJ and Groovy. The problem, as I've mentioned before, is that while the JDT is very good at providing tool support for Java in Eclipse, it provides no support for Java-like languages. To date, I've taken two approaches to addressing this problem. Now, there is a third approach, Object Teams, that has recently made a home at Eclipse.

JDT Weaving



With AJDT, the AspectJ development tools, I developed JDT Weaving. This approach uses Equinox Weaving to weave into the JDT plugins. Yes, we are eating our own dog food here and using AspectJ to implement AJDT. But, this approach has worked well for AJDT since AJDT only requires a small number of well-defined hooks into JDT to get its tight integration.

Feature patches



For Groovy-Eclipse, our tool support for the Groovy language, the JDT Weaving approach was not feasible since we needed a larger number of hooks into JDT. And the nature of these hooks made it difficult to use AspectJ to advise them. So, instead we are using a feature patch. This means that Groovy-Eclipse ships its own (slightly modified and heavily tested) version of the JDT core plugin. This also means that Groovy-Eclipse needs to be re-released for each new version of JDT Core (however, this turns out not to be much of a problem since our release cycle is much shorter than JDT's).

Object Teams



Now, Object Teams is a third approach, which I find very exciting (actually, this approach has been around for several years, but it has only recently moved to Eclipse.org). Object Teams is itself an Aspect-Oriented extension to Java, with a component, O/T Equinox, that provides integration with Eclipse's Equinox OSGi framework. O/T Equinox has been further adapted to provide easy extensibility to JDT. The O/T blog has a slick example of this approach. I'm looking forward to exploring this approach more.

So what?



Although it's great that we have been able to get around JDT's shortcomings and there are multiple ways of doing this, but this proliferation of approaches is a little scary. There is no guarantee that these approaches will play nicely with each other. For a while, Eclipse installs with Groovy-Eclipse would break the Scala plugin (the Scala plugin uses a variant of JDT weaving). Although this problem has now been fixed, we need to be especially careful that a similar problem does not happen in the future. Similarly, the Object Teams tool support and Groovy-Eclipse are incompatible because they both ship with feature patches for JDT Core.

Of course, the best solution would be to push our changes into JDT Core itself, we have been unsuccessful as of yet. Until then, we need open communication between all JDT hackers to make sure that we are working towards the same goal.

If you're interested in learning more or have some ideas of your own, come have a chat with me at EclispeCon.

Thursday, December 24, 2009

Extending Groovy-Eclipse for use with Domain-Specific Languages

One of the great things about Groovy is how easy it is to use the language to create domain specific languages (DSLs). And one of the great things about Eclipse is extensibility through its plugin architecture. Within Groovy-Eclipse, we are using the second to leverage the first. We have created a set of Eclipse extension points that allow Groovy programmers to create tool support for their own custom Groovy DSL. In this post, I will walk you through the extension points we created and show you how we use them to implement Grails support in STS.

We provide three extension points, one for adding new smarts to the inferencing engine: one for adding new content assist proposals, and one for adding new highlighting rules to the editor.

Extending Syntax Highlighting



The simplest way to extend Groovy-Eclipse is through the org.codehaus.groovy.eclipse.ui.syntaxHighlightingExtension extension point.

Here is what STS does to use the extension point:

<extension
point="org.codehaus.groovy.eclipse.ui.syntaxHighlightingExtension">
<highlightingExtender
extender="com.springsource.sts.grails.editor.groovy.GrailsSyntaxHighlighting"
natureID="com.springsource.sts.grails.core.nature">
</highlightingExtender>
</extension>


As you can see, an extender class (com.springsource.sts.grails.editor.groovy.GrailsSyntaxHighlighting) is associated with a project nature (com.springsource.sts.grails.core.nature). And now, whenever a Groovy Editor is opened for a Grails project, all of the syntax highlighting rules from the extender class is added to the Groovy Editor. For Grails, the extension is simple:

public class GrailsSyntaxHighlighting implements IHighlightingExtender {

public List getAdditionalGJDKKeywords() {
return Arrays.asList(
// domain fields
"constraints", "belongsTo", "hasMany", "nullable", "belongsTo", "mapping",
"hasMany", "embedded", "transients", "id", "tablePerHierarchy", "version",
// domain methods
"list", "save", "delete", "get",
// controller fields
"log", "actionName", "actionUri", "controllerName", "controllerUri",
"flash", "log", "params", "request", "response", "session",
"servletContext",
// controller methods
"render", "redirect"
);
}

public List getAdditionalRules() {
return null;
}
public List getAdditionalGroovyKeywords() {
return null;
}

}


New GJDK keywords are added and nothing else. Also, note that GrailsSyntaxHighlighting implements IHighlightingExtender. And here is what the additional syntax highlighting can give you:

syntax_highlighting

Notice that the special Grails domain class fields such as belongsTo and mapping are highlighted, and below in the controller class, keywords like params and render are highlighted.

Extending the inferencing engine



The Groovy-Eclipse inferencing engine is used to infer the types of expressions within a Groovy file. Because of its dynamic nature, determining the types of all Groovy expressions in a file is undecidable. The good news is that most programs are well-behaved and follow a simple set of rules through which we can infer the types of most expressions. Meta-programming in Groovy can add new members to Groovy objects and classes. This is a feature used by most Groovy DSLs.

Groovy-Eclipse allows DSL programmers to specify the meta-programming through the org.eclipse.jdt.groovy.core.typeLookup extension point. Here is what the extension point looks like in the Grails tool support in STS:

<extension
point="org.eclipse.jdt.groovy.core.typeLookup">
<lookup lookup="com.springsource.sts.grails.editor.groovy.types.GrailsTypeLookup">
<appliesTo projectNature="com.springsource.sts.grails.core.nature"/>
</lookup>
</extension>


Here the class com.springsource.sts.grails.editor.groovy.types.GrailsTypeLookup is defined to be a type lookup for projects that have the com.springsource.sts.grails.core.nature (i.e., this lookup is only activated for Grails projects).

Let's take a look at the GrailsTypeLookup class:

public class GrailsTypeLookup extends AbstractSimplifiedTypeLookup implements ITypeLookup {

private IGrailsElement element;
private GrailsProject gp;

public void initialize(GroovyCompilationUnit unit,
VariableScope topLevelScope) {
gp = GrailsCore.get().getGrailsProjectFor(unit);
if (gp != null) {
element = gp.getGrailsElement(unit);
element.initializeTypeLookup(topLevelScope);
}
}

@Override
protected TypeAndDeclaration lookupTypeAndDeclaration(
ClassNode declaringType, String name, VariableScope scope) {
IGrailsElement declaringElt = gp.getGrailsElement(declaringType);
return declaringElt.lookupTypeAndDeclaration(declaringType, name, scope);
}
}


According to the extension point specification, GrailsTypeLookup must extend ITypeLookup and we choose to let it extend AbstractSimplifiedTypeLookup in order to reduce the amount of coding required.

The initialize method is called when type inferencing is starting for a Groovy file. Here, it is possible to stuff things into the top level scope (such as global variables). For Grails, we determine what kind of Grails element we are performing inference on (e.g., a domain class, controller class, taglib, etc) and modify the top level scope appropriately.

More magic happens in the lookupTypeAndDeclaration method. Again, the type lookup delegates to the specific Grails element to determine what the type is.

Now, let's take a look at what this can do for us. Notice that hovering over Grails keywords will bring up a JavaDoc of the inferred type of that keyword:

hovers_in_controller

The Grails response field in controller classes is of type HttpServletResponse. Similarly, this allows us to get HttpServletResponse aware content assist proposals:

content_assist1

Extending Content Assist



The final step is to hook extensible content assist into the DSL. This can be tricky. For example, in Grails, there are certain fields that if defined have special meaning. There is the constraints field where constraints for domain classes are defined, and the mapping field where object-relational mappings are defined. The closure attached to each of these fields have special keywords that they expect.

This is possible to control through the org.codehaus.groovy.eclipse.codeassist.completion.completionProposalProvider extension point. Here is how it is used in STS:
<extension
point="org.codehaus.groovy.eclipse.codeassist.completion.completionProposalProvider">
<proposalProvider
proposalProvider=
"com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider">
<appliesTo projectNature="com.springsource.sts.grails.core.nature"/>
</proposalProvider>
</extension>


This extension point wires a com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider to the Grails project nature. And so (as with the other extension points), theis extra content logic will only occur when inside a Grails project.

com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider implements org.codehaus.groovy.eclipse.codeassist.processors.IProposalProvider. This interface has three methods to implement:


  • getNewFieldProposals: Return a list of fields that can be defined at the content assist location. For example, here is where all special fields available in Grails domain classes are proposed. This method is only called when content assist is invoked when inside a class body (i.e., only where it is appropriate to define new fields).

  • getNewMethodProposals: Return a list of new methods that can be defined at the invocation location. As with the new fields method, this method is only called when content assist is invoked in a location that is possible to define new methods.

  • getStatementAndExpressionProposals: This method returns all possible special content assist proposals when in the context of an expression or statement. For example, here is where special controller class fields like params, request, and response are inserted.



Let's take a look at what this can do. In this screenshot, you can see that when performing content assist on a reference to a Grails domain class, you can access Grails specific methods like count:

extra_content_assist

And when inside the constraints block (and only when within that block) content assist is augmented with possible constraints to add:

content_assist_constraints

Conclusion



We have worked hard to make sure that Groovy-Eclipse is extensible. It is already being used by some DSLs such as EasyB and by BonitaSoft.

These extension points and APIs are still a work in progress if you have any questions, or require some changes to anything, please raise a bug or send a message to the mailing list.

Monday, December 14, 2009

Getting GMaven to play nicely with Groovy-Eclipse

There has been a lot of talk on the Groovy mailing list lately about how to get GMaven working with Groovy-Eclipse and coincidentally I saw my friend Mike last night who works at boats.com and he told me that his office has solved this problem. Here is what he told me:



Okay, the first things you will need in your POM file are the GMaven mojo and the GMaven runtime. The dependencies for the mojo seem to be a big buggered out of the box, so you will have to tweak the GMaven runtime dependencies. Here is how you do it:


<dependencies>
<dependency>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-mojo</artifactId>
<version>1.0</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all-minimal</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.groovy.maven.runtime</groupId>
<artifactId>gmaven-runtime-1.5</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.codehaus.groovy.maven.runtime</groupId>
<artifactId>gmaven-runtime-1.6</artifactId>
<version>1.0</version>
</dependency>
</dependencies>


Next is your build configuration. Eclipse will need to be able to find your Groovy source files, so you will need to add your Groovy source as an explicit resource directory.

The mvn:eclipse plugin will also need to add the Groovy nature to your project.

You will also need to add directives for the GMaven plugin. This is where you will have to do a manual tweak: when using mvn to build your project, gmaven generates some stubs for your groovy code before the java code compiles, so that any java->groovy dependencies will be fulfilled. The Groovy code is then compiled and will overwrite the .class files from the previous step. This breaks things in eclipse, since if you attempt to execute code in your workbench, eclipse sees the stubs and somehow their .class files are what end up in your binary output directory.

Therefore, if you want to execute a gmaven project from Eclipse, you simply need to comment out the stub generation directive and delete any generated stub classes. This is what it looks like:


<build>
<resources>
<resource>
<directory>src/main/groovy</directory>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<additionalProjectnatures>
<projectnature>
org.eclipse.jdt.groovy.core.groovyNature
</projectnature>
</additionalProjectnatures>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<goals>
<!--
<goal>generateStubs</goal>
-->
<goal>compile</goal>
<!--
<goal>generateTestStubs</goal>
-->
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

The result is a groovy/java project that can be managed with maven but can be transparently developed (and more importantly, debugged!) in Eclipse.

There is one minor issue that has come up however: GMaven 1.6 stub generation is broken for enums: it generates public constructors, which do not compile. I submitted an issue to codehaus (GMAVEN-51), and apparently the workaround is to use the gmaven 1.7 runtime. I haven't tried it yet, since things are working okay for us thus far.

Mike


So, it does seem like there is a bit of a problem with stubs being recognized by Groovy-Eclipse when they should not be. The solution would be to generate the stubs in a directory that is not seen by Eclipse, rather than in the default output directory. But not knowing much about GMaven (or Maven for that matter) works, I don't know how feasible this is. Can anyone think of a better solution?