Root CUIP Metalevel

StringTemplate: a great template engine for code generation

Paper Templates, licensed CC Attribution by Edinburgh City of Print

When building a new tool for modelling and code generation like Essential, one has to rethink again what template engine use to drive all the machinery. In code generation contexts, template engines are a good field for innovation and your choice will be with you probably for the full lifetime of your tool.

In this post, I will try to introduce and explain why StringTemplate is a superb engine for doing code generation and why you should consider it if dealing with a code generation scenario.

As commented before in other post, I felt in love with StringTemplate in 2005 when I discovered ANTLR and StringTemplate during the development of a MDD workbench for a customer. Prior to that date, I had tested many template engines and also developed on my own a pair of them. For one reason or another, I was not comfortable with the language, the coupling of the template with the transformation control, or the maintainability of the templates in the long term. As the time goes by, this last point arises as crucial to maintain and evolve your architecture.

Therefore, one could say that I am being quite loyal to StringTemplate for the last 5 years, and that too much for technology, moreover, for a product software. And such loyalty is definitely not my merit for not been promiscuous in this issue, but a merit of StringTemplate: I’m still looking for new ways for improvement and didn’t find one better to my needs. (Note: this is probably the best compliment I never paid to a piece of good software.)

Some products shine thanks to the many features they provides, StringTemplate sparkles for avoiding adding features in the wrong place. I will elaborate a bit more on that.

MVC

Model View Controller is an old and good “classic” architectural pattern providing a solution to organize the pieces of a user interface. Originally proposed by Trygve Reenskaug, in Smalltalk in 1979. It has been lately again on fashion, especially in the building of web sites, multi-channel user interfaces and promoting UI testability thanks to the application of the principle of Separation of Concerns that the pattern provides.

There are many variants or dialects to MVC out there, but the simple one is enough to describe our scenario: code generation.

Code generation can be seen as a classic MVC domain with the three roles easily identifiable:

  • A model: the variable data in consideration, the metadata to be generated in this case.
  • A view: the template with the form of the target language and providing holes or slots to be later being fill in (the output).
  • And the transformation controller: the piece in charge of selecting with model parts should be selected, combine and finally inserted in which template holes/slots to render the final and concrete view.

The main problem with traditional template engines is that there are lots of template engines that mix the last two roles (for example all coming from web development in the flavour of JSP or  ASP variants). The view and the transformation controller are intermingled and this situation, in the long time, is not good. I found this strict separation of responsibilities crucial for the maintainability of the templates.

And StringTemplate does exactly that. It syntax and features, although could be strange for the first time, was carefully selected to not allow you to do control choices in the view. It allows to define templates, declare named holes (or slots) and an inline for-each and if statements.

Let’s review a basic sample of StringTemplate syntax:

group mail;

genMail(msg)::=<<
Mail: Subject $msg.Subject$
From: $msg.From:genAddress()$
To: $msg.To:genAddress(); separator=”, “$
>>

genAddress()::=<<$it.Name$ ($it.Email$)>>

The sample contains two templates: one for a mail, and a second one for each address. Templates can have parameters explicit parameters like name, but also there is an implicit keyword (it) to reference the current iterating context.

Named holes or slots are defined are marked with $expression$ where expression is a constrained expression resolution of the model properties received.

In the sample, genAddress() template will be invoked to resolve the From and To properties. Assuming that From is a monovaluated property and that To is a list, the template iterates throw the collection received and evaluates the template for each item. Note the separator modifier used to describe how items should be rendered.

That’s all in the template, the view. It’s forbidden to compute any value inside the template. The idea is that only read operations are going to be done into the model to get the data needed to fill the template.

How to consume the template?

From the transformation side (the controller) consuming a template is something such straightforward as the following one:

//1. Select the template
StringTemplate tlp = group.InstanceOf(“genMail”);
//2. Set parameters
tlp.setAttribute(“msg”, message);
//3. Get the result
string result = tlp.ToString();

It is frequent to find people complaining StringTemplate about not having enough freedom for example to call functions from the inside. But this is not a missing feature, this done on purpose to assure no side effects of such computations. The motto is; if you need to compute something, do it the transformation stage.

This only has covered a very basic intro to StringTemplate. If you are interested you can browse for more contents in the main documentation of StringTemplate.

Of course, Essential uses StringTemplate!

ST & ANTRL on Code Generation 2011

By the way, the father of StringTemplate and the superb meta-compiler tool called ANTLR is Terrence Parr. He teaches compiler construction in the University of San Francisco.

I was planning to prepare this post for some months, but in the meanwhile, Mark Dalgarno (the main organizer of Code Generation Conferences) has surprises us announcing the next keynote for CG2011.

I am quite happy to see that, as announced by Mark, that in the next edition of Code Generation 2011 Terrence is going to be present as one of the invited keynotes!

So, if you are interested in all of these topics, join us and meet Terrence on May 25-27, 2011 in Cambridge, UK on Code Generation 2011.

The call for speakers for CG2011 is open till January 15th 2011.

To know more:

  1. Enforcing Strict Model-View Separation in Template Engines, Terrence Parr
  2. StringTemplate
  3. ANTLR

7 comments.

  1. [...] This post was mentioned on Twitter by Kees Dijk, Pedro J. Molina. Pedro J. Molina said: Just posted: StringTemplate: a great template engine for #codegeneration http://bit.ly/hp7FV2 #stringtemplate #metalevel [...]

  2. Using ABSE and Lua, your example would be something like:

    On a “message” Atom:

    set(“to_line”,forall(“recipient”,”csv”,”genAddress”))
    code([[
    Mail: Subject $Subject
    From: $From
    To: $to_line
    ]])

    On a “recipient” Atom:

    function genAddress()
    return var(“name”) .. ” (” .. var(“address”) ..”)”
    end

    Which is not much different from StringTemplate. Main difference between ABSE and StringTemplate, in terms of code generation, is that ABSE templates are more like mini-programs and were conceived to support interactive modeling.

    To consume the template:

    Nothing needed. Consuming templates is internally done by the interactive modeler. So, in fact, when you are building a model in AtomWeaver you are automatically creating that “consume code”.

  3. OK, I’ll bite :-). Here’s the same thing in MERL, the MetaEdit+ Reporting Language:

    ‘Mail: Subject ‘ :Subject ‘
    From: ‘ :From ‘
    To: ‘ dowhile :To { id ‘, ‘ }

    In addition you’d have an Identifier Generator for the Address objects:

    :Name ‘ (‘ :Email ‘)’

    Really only a couple of differences:
    1) MERL quotes its fixed text, rather than its commands/expressions. In simple examples, there are longer blocks of fixed text, so the template style seems better, but once you are generating anything real the majority is the generator code rather than the fixed text, so switching the quoting around makes sense.

    2) MERL doesn’t need a variable pointing to “this” or the current iterated element – rather than saying “this.Name” you just say “:Name”.

    Here’s a quick intro to MERL:
    http://www.metacase.com/support/45/manuals/mwb/Mw-5_3_1.html

  4. Thank Rui and Steven for sharing your experience with others template engines like ABSE/Lua and MERL.

    Doing things in a maintainable way (in this case, keeping a clean view and apart of the control code) is doable with many templates engines. Ones enforce a strict separation, others like the ones based in ASP or JSP requires some discipline in the template author to not to put control code inside the template.

    Steven: so I see, constant text in MERL is treated as a string literal.

    Another small and bright feature nicely resolved in StreamTemplate is auto-ident and the separator modifier like in the sample: separator=", " or separator="\n"
    I like it a lot. :)

    I see the ',' in the Steven sample.
    Rui: There are something similar in LUA for creating a separator between email addresses?

    There was a classic TV advertisement in the Spanish TV that says: “Search, compare, and if you found something better, buy it!”

    Bite, bite, please! Don’t be shy. :)

  5. “Rui: There are something similar in LUA for creating a separator between email addresses?”

    First, please note that Lua is a general purpose scripting language (http://www.lua.org) and not a specific DSL for ABSE.

    So, since ABSE does not use a DSL, it uses a library. In my opinion, using a library instead of a DSL eliminates one more “accidental complexity” barrier for adoption.

    Answering your question, the forall() function executes a given function on all child Atoms of a specified type and outputs the results on a selected format. In my example, the selected format is “csv” which naturally solves the “last comma” problem.

  6. Excellent introduction to StringTemplate. I’ve used this template library for a few projects at work and absolutely love it. I’ll be using this post as my go-to explanatory material of why it’s so good.

  7. Thanks you very much Nicholas.
    Feel free to link it. :)

Post a comment.