Light-weight IDE extensibility for custom DSLs in Groovy

This post gives a bit of a teaser for my EclipseCon talk next week.

Most of you reading this probably know that Groovy-Eclipse is the Eclipse tool support for Groovy and one of the major features it provides is type inferencing in the editor. For example, in the following screenshot:

Groovy-Eclipse knows that the type of myString is java.lang.String. That is why substring is not underlined, but uh_oh is. All references that cannot be statically resolved are underlined regardless whether or not the program will fail at runtime (Groovy is a dynamic language, and we can't be certain about what happens at runtime).

This works quite well in common situations. However, things get complicated because Groovy provides strong support for creating domain specific languages (DSLs). Typically, when a DSL is used, the editor will be confused. Consider, for example, this simple DSL to calculate distances and convert between units. When used in Groovy-Eclipse, the result looks like this:

Uh-oh, underlines. The references do not make sense to the editor, even though this executes without problem.

Thankfully, Groovy-Eclipse's inferencing engine was designed from the beginning to be extensible. There are three ways to extend the inferencing engine:

1. Roll your own Eclipse plugin
2. Inferencing suggestions
3. DSL descriptors

And I'll talk about each of them in the next sections.

Roll your own Eclipse plugin

This is a powerful technique and gives you fine-grained control over exactly how your DSL integrates with Groovy-Eclipse. I have already written a full description of all the extension points available and how to use them to create a plugin that extends Groovy-Eclipse. This was the state of Groovy-Eclipse circa 2010.

Although there have been many successes with this approach, we found that overall this was quite a heavy-weight way for others to leverage the benefits of type inferencing:

• Most Groovy DSL developers are not Eclipse experts and so do not have the experience or desire to learn Eclipse APIs, create plugins and features, produce an update site, and test it against various versions of Eclipse
• Many Groovy programmers (the ones who consume the DSLs) are coming from a scripting world, use vi or textmate, and are used to a light-weight editor. Installing plugins was a big complaint for them. They want things to just work

Clearly, another approach was needed.

Inferencing suggestions (very light-weight extensibility)

Inferencing suggestions give the end user control to augment the inferencing engine with specific suggestions. Using a quick-assist (CTRL+1), the user is given an option to add a suggestion to the currently selected identifier in the editor:

This brings up the inferencing suggestions dialog, where the user can fill in as little or as many details as desired:

And now, back in the editor, the suggestion is now used for inferencing:

This feature has some big advantages over the plugin model:

• it gives end-users more control over their own environment
• it is dead simple to use and no other plugins are required
• no need for any Eclipse API knowledge
• little knowledge of Groovy required

But in some situations, this feature moves too far in the direction of simplicity over expressiveness.

• can only target a single property/method at a time, which makes large DSLs painful to work with
• not easy to share across users and projects

Inferencing suggestions was not sufficient and we needed yet another kind of extensibility.

DSL descriptors (light-weight extensibility)

A DSL descriptor (DSLD) is a Groovy script that describes your Groovy DSL. This script is compiled and executed by Groovy-Eclipse so that the inferencing engine can understand the DSL. They can be shipped with libraries and as long as Groovy-Eclipse can find these files on the classpath, they can contribute to type inferencing. I have already written a good introduction to DSLD.

We created the DSLD language so that library developers could have fine-grained control over how their library and DSL gets interpreted in Groovy-Eclipse and also so that end users can transparently use this functionality without needing to install a new plugin or do any extra work.

Taking our distances DSL from before, we can write a simple DSLD that encapsulates the DSL in a format consumable by Groovy-Eclipse:

contribute( currentType( subType( Number ) ) ) {
  [ “m”, “yd”, “cm”, “mi”, “km” ].each {
    property name:it, type:"Distance”, 
        doc: """Converts a {@link Number} to a {$it}
                @author Joe
                @since 1.0"""
  }
}

When this file is saved to something like distances.dsld and placed in the project or on the classpath, the distances expression is appropriately recognized in the editor:

(Which, you should notice looks the same as the inferencing suggestions hover above.)

What kind of extensibility is best?

That's a loaded question and of course, it depends. Here's a simple table to highlight the costs and benefits of each approach:

	Extension points	DSLD	Inferencing suggestions
Ease	heavy-weight	light-weight	very light-weight
Flexibility	lots	some	little
Eclipse knowledge required?	lots	none	none
Groovy knowledge required?	lots	lots	little
How to use/install	update site	included with library	preferences/quick-assist

I'd still recommend that you come see the talk at EclipseCon (if you are lucky enough to be attending the conference). I'll go into more detail on implementation and I'll show some live demos on how this all works. See you at EclipseCon!