Is it Groovy or is it Java? Do we really care?

Well, I care, but the editor doesn't. Following up on the work that Andy Clement has done to modify the JDT compiler so that it can process Groovy as well as Java, I've spent the last few weeks working on editor support for Groovy in Eclipse. Yes, there is already an Eclipse plugin for Groovy, but this particular plugin is tied to a split compiler (JDT for Java files and Groovy compiler for Groovy files). And I wanted to see if we could use the joint compiler to drive Groovy tool support.

So far, with minimal changes it seems that it is possible. I've been able to implement much of the standard Java editing support for Groovy files (e.g., open declaration, search, syntax highlighting, content assist, etc) as long as the editor is passed the right kind of structure.

Let's recap what Andy has done so far...

After getting a little intro to groovy joint compilation (and groovy compiler structure) from Jochen Theodorou, I wondered how hard it would be to go about things slightly differently. Let the Eclipse JDT compiler take control of building a mixed code base but whenever it needed to deal with groovy - call the groovy compiler. If JDT could take control I could see all sorts of things 'just working' in Eclipse, problems upon which we are still expending large amounts of efforts to try and solve for AspectJ. If I could just plug groovy type reference resolution and classfile generation into JDT correctly then references between java and groovy artifacts would work and JDT would ensure incremental compilation worked, even across restarts of eclipse (this latter problem still hasn't been solved for AspectJ).

So originally, this worked great for joint compilation, but as soon as you try to open this in an editor, things just don't behave correctly...



The outline view is out of order, problems and errors don't show up where you expect them to, and source code navigation is broken.

After a bit of mucking around, I found that there is a single root to this problem: source locations. The JDT tooling and the editor is very specific about the source locations that it requires. JDT requires the start and end locations of identifiers, method and type bodies, and all declarations. However, the Groovy compiler (which builds its AST from antlr), only provides line and column information for AST nodes. Although it is possible to translate from line/column to file offset, offsets for identifiers are lost.

And, with a little bit of hacking, I was able to recreate source offsets for field, method, and class names and declarations. It turns out that this is enough to spring to like much of your favorite editor functionality:



Same program, same error as before, but now the outline view is correct, the error marker is in the proper location,* "Open Declaration" and searches navigate to the proper location. And even source hovers work:



So, let's take a deeper look at how this works.

The JDT uses three representations of abstract syntax of a Java file. There is the compiler AST, which is used to generate byte code and perform analysis. There is DOM (document object model) AST, which is used for refactoring and source code manipulation. And, there is the model (the IJavaElement hierarchy), which is more abstract, and is used for navigation and in places like the Package Explorer and Outline views.

There is also support to translate from the compiler AST to either of the other two ASTs. Earlier, I described (or rather Andy described) the way JDT can be used to drive the joint compilation of Groovy source code. The result is a JDT compiler AST with (crucially) the right source code offsets. This, when translated to the right kind of AST at the right time, can be fed to the different parts of JDT that require it. And, thereby, things just work.

A few caveats, of course. Right now, we are only generating a minimal compiler AST (class, method and field declarations only---method bodies are ignored for now). This, however, does seem to be enough for this basic functionality. Also, there are some major differences between Groovy and Java syntax (closures, regular expressions, and lists operations). We are currently sidestepping this problem by ignoring method bodies, statements and expressions.

At some point, however, we are going to have to tackle these loose ends. We will need to be more complete about our ASTs and fill in method bodies (this part is *just* lots of coding). And also, we will need to figure out how to translate Groovy-specific syntax into JDT AST, probably by sub-classing the existing API (this will require some thorough design work *and* lots of coding). So, there is still a lot of work to be done.

What does this mean? Well for one thing it means that the implementation of the Groovy editor is just this:

public class GroovyEditor extends CompilationUnitEditor {
  private GroovyImageDecorator decorator = new GroovyImageDecorator();
  
  @Override
  public Image getTitleImage() {
    return decorator.decorateImage(null, getEditorInput().getAdapter(IFile.class));
  }
}

Yes, it is just a minimal subclass of the standard CompilationUnitEditor that merely ensures that the icon in the title bar indicates Groovy, not Java.

So, who cares if it's Groovy or Java? Certainly not the editor.

* There are currently Groovy compiler limitations with error reporting in that error locations only include the character or two of the error.