Thursday, December 24, 2009

Extending Groovy-Eclipse for use with Domain-Specific Languages

One of the great things about Groovy is how easy it is to use the language to create domain specific languages (DSLs). And one of the great things about Eclipse is extensibility through its plugin architecture. Within Groovy-Eclipse, we are using the second to leverage the first. We have created a set of Eclipse extension points that allow Groovy programmers to create tool support for their own custom Groovy DSL. In this post, I will walk you through the extension points we created and show you how we use them to implement Grails support in STS.

We provide three extension points, one for adding new smarts to the inferencing engine: one for adding new content assist proposals, and one for adding new highlighting rules to the editor.

Extending Syntax Highlighting



The simplest way to extend Groovy-Eclipse is through the org.codehaus.groovy.eclipse.ui.syntaxHighlightingExtension extension point.

Here is what STS does to use the extension point:

<extension
point="org.codehaus.groovy.eclipse.ui.syntaxHighlightingExtension">
<highlightingExtender
extender="com.springsource.sts.grails.editor.groovy.GrailsSyntaxHighlighting"
natureID="com.springsource.sts.grails.core.nature">
</highlightingExtender>
</extension>


As you can see, an extender class (com.springsource.sts.grails.editor.groovy.GrailsSyntaxHighlighting) is associated with a project nature (com.springsource.sts.grails.core.nature). And now, whenever a Groovy Editor is opened for a Grails project, all of the syntax highlighting rules from the extender class is added to the Groovy Editor. For Grails, the extension is simple:

public class GrailsSyntaxHighlighting implements IHighlightingExtender {

public List getAdditionalGJDKKeywords() {
return Arrays.asList(
// domain fields
"constraints", "belongsTo", "hasMany", "nullable", "belongsTo", "mapping",
"hasMany", "embedded", "transients", "id", "tablePerHierarchy", "version",
// domain methods
"list", "save", "delete", "get",
// controller fields
"log", "actionName", "actionUri", "controllerName", "controllerUri",
"flash", "log", "params", "request", "response", "session",
"servletContext",
// controller methods
"render", "redirect"
);
}

public List getAdditionalRules() {
return null;
}
public List getAdditionalGroovyKeywords() {
return null;
}

}


New GJDK keywords are added and nothing else. Also, note that GrailsSyntaxHighlighting implements IHighlightingExtender. And here is what the additional syntax highlighting can give you:

syntax_highlighting

Notice that the special Grails domain class fields such as belongsTo and mapping are highlighted, and below in the controller class, keywords like params and render are highlighted.

Extending the inferencing engine



The Groovy-Eclipse inferencing engine is used to infer the types of expressions within a Groovy file. Because of its dynamic nature, determining the types of all Groovy expressions in a file is undecidable. The good news is that most programs are well-behaved and follow a simple set of rules through which we can infer the types of most expressions. Meta-programming in Groovy can add new members to Groovy objects and classes. This is a feature used by most Groovy DSLs.

Groovy-Eclipse allows DSL programmers to specify the meta-programming through the org.eclipse.jdt.groovy.core.typeLookup extension point. Here is what the extension point looks like in the Grails tool support in STS:

<extension
point="org.eclipse.jdt.groovy.core.typeLookup">
<lookup lookup="com.springsource.sts.grails.editor.groovy.types.GrailsTypeLookup">
<appliesTo projectNature="com.springsource.sts.grails.core.nature"/>
</lookup>
</extension>


Here the class com.springsource.sts.grails.editor.groovy.types.GrailsTypeLookup is defined to be a type lookup for projects that have the com.springsource.sts.grails.core.nature (i.e., this lookup is only activated for Grails projects).

Let's take a look at the GrailsTypeLookup class:

public class GrailsTypeLookup extends AbstractSimplifiedTypeLookup implements ITypeLookup {

private IGrailsElement element;
private GrailsProject gp;

public void initialize(GroovyCompilationUnit unit,
VariableScope topLevelScope) {
gp = GrailsCore.get().getGrailsProjectFor(unit);
if (gp != null) {
element = gp.getGrailsElement(unit);
element.initializeTypeLookup(topLevelScope);
}
}

@Override
protected TypeAndDeclaration lookupTypeAndDeclaration(
ClassNode declaringType, String name, VariableScope scope) {
IGrailsElement declaringElt = gp.getGrailsElement(declaringType);
return declaringElt.lookupTypeAndDeclaration(declaringType, name, scope);
}
}


According to the extension point specification, GrailsTypeLookup must extend ITypeLookup and we choose to let it extend AbstractSimplifiedTypeLookup in order to reduce the amount of coding required.

The initialize method is called when type inferencing is starting for a Groovy file. Here, it is possible to stuff things into the top level scope (such as global variables). For Grails, we determine what kind of Grails element we are performing inference on (e.g., a domain class, controller class, taglib, etc) and modify the top level scope appropriately.

More magic happens in the lookupTypeAndDeclaration method. Again, the type lookup delegates to the specific Grails element to determine what the type is.

Now, let's take a look at what this can do for us. Notice that hovering over Grails keywords will bring up a JavaDoc of the inferred type of that keyword:

hovers_in_controller

The Grails response field in controller classes is of type HttpServletResponse. Similarly, this allows us to get HttpServletResponse aware content assist proposals:

content_assist1

Extending Content Assist



The final step is to hook extensible content assist into the DSL. This can be tricky. For example, in Grails, there are certain fields that if defined have special meaning. There is the constraints field where constraints for domain classes are defined, and the mapping field where object-relational mappings are defined. The closure attached to each of these fields have special keywords that they expect.

This is possible to control through the org.codehaus.groovy.eclipse.codeassist.completion.completionProposalProvider extension point. Here is how it is used in STS:
<extension
point="org.codehaus.groovy.eclipse.codeassist.completion.completionProposalProvider">
<proposalProvider
proposalProvider=
"com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider">
<appliesTo projectNature="com.springsource.sts.grails.core.nature"/>
</proposalProvider>
</extension>


This extension point wires a com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider to the Grails project nature. And so (as with the other extension points), theis extra content logic will only occur when inside a Grails project.

com.springsource.sts.grails.editor.groovy.contentassist.GrailsProposalProvider implements org.codehaus.groovy.eclipse.codeassist.processors.IProposalProvider. This interface has three methods to implement:


  • getNewFieldProposals: Return a list of fields that can be defined at the content assist location. For example, here is where all special fields available in Grails domain classes are proposed. This method is only called when content assist is invoked when inside a class body (i.e., only where it is appropriate to define new fields).

  • getNewMethodProposals: Return a list of new methods that can be defined at the invocation location. As with the new fields method, this method is only called when content assist is invoked in a location that is possible to define new methods.

  • getStatementAndExpressionProposals: This method returns all possible special content assist proposals when in the context of an expression or statement. For example, here is where special controller class fields like params, request, and response are inserted.



Let's take a look at what this can do. In this screenshot, you can see that when performing content assist on a reference to a Grails domain class, you can access Grails specific methods like count:

extra_content_assist

And when inside the constraints block (and only when within that block) content assist is augmented with possible constraints to add:

content_assist_constraints

Conclusion



We have worked hard to make sure that Groovy-Eclipse is extensible. It is already being used by some DSLs such as EasyB and by BonitaSoft.

These extension points and APIs are still a work in progress if you have any questions, or require some changes to anything, please raise a bug or send a message to the mailing list.

Monday, December 14, 2009

Getting GMaven to play nicely with Groovy-Eclipse

There has been a lot of talk on the Groovy mailing list lately about how to get GMaven working with Groovy-Eclipse and coincidentally I saw my friend Mike last night who works at boats.com and he told me that his office has solved this problem. Here is what he told me:



Okay, the first things you will need in your POM file are the GMaven mojo and the GMaven runtime. The dependencies for the mojo seem to be a big buggered out of the box, so you will have to tweak the GMaven runtime dependencies. Here is how you do it:


<dependencies>
<dependency>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-mojo</artifactId>
<version>1.0</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all-minimal</artifactId>
</exclusion>
<exclusion>
<groupId>org.codehaus.groovy.maven.runtime</groupId>
<artifactId>gmaven-runtime-1.5</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.codehaus.groovy.maven.runtime</groupId>
<artifactId>gmaven-runtime-1.6</artifactId>
<version>1.0</version>
</dependency>
</dependencies>


Next is your build configuration. Eclipse will need to be able to find your Groovy source files, so you will need to add your Groovy source as an explicit resource directory.

The mvn:eclipse plugin will also need to add the Groovy nature to your project.

You will also need to add directives for the GMaven plugin. This is where you will have to do a manual tweak: when using mvn to build your project, gmaven generates some stubs for your groovy code before the java code compiles, so that any java->groovy dependencies will be fulfilled. The Groovy code is then compiled and will overwrite the .class files from the previous step. This breaks things in eclipse, since if you attempt to execute code in your workbench, eclipse sees the stubs and somehow their .class files are what end up in your binary output directory.

Therefore, if you want to execute a gmaven project from Eclipse, you simply need to comment out the stub generation directive and delete any generated stub classes. This is what it looks like:


<build>
<resources>
<resource>
<directory>src/main/groovy</directory>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<additionalProjectnatures>
<projectnature>
org.eclipse.jdt.groovy.core.groovyNature
</projectnature>
</additionalProjectnatures>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<goals>
<!--
<goal>generateStubs</goal>
-->
<goal>compile</goal>
<!--
<goal>generateTestStubs</goal>
-->
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

The result is a groovy/java project that can be managed with maven but can be transparently developed (and more importantly, debugged!) in Eclipse.

There is one minor issue that has come up however: GMaven 1.6 stub generation is broken for enums: it generates public constructors, which do not compile. I submitted an issue to codehaus (GMAVEN-51), and apparently the workaround is to use the gmaven 1.7 runtime. I haven't tried it yet, since things are working okay for us thus far.

Mike


So, it does seem like there is a bit of a problem with stubs being recognized by Groovy-Eclipse when they should not be. The solution would be to generate the stubs in a directory that is not seen by Eclipse, rather than in the default output directory. But not knowing much about GMaven (or Maven for that matter) works, I don't know how feasible this is. Can anyone think of a better solution?