Plaza de España, Sevilla (By CCSA Tom Raftery)

La comunidad Alt.NET Hispano ha publicado la VAN (desconferencia) sobre Generación de código con Essential que tuvimos el pasado dia 11 de junio. Si estás interesado en MDD, generación de código y como aplicarlo con Essential, este video grabado para la comunidad en español es un buen punto de partida.

Mi agradecimiento a Alt.NET Hispano por el interés en la materia y la invitación a divulgarlo.

Descarga directa del video (562 Mb).

In this long post, I want to share my personal view about these days, and for sure, take note it could be partial and subjective. So, be kind to review also the comments as seen by others like Johan den Haan, Markus Völter, Angelo Hulshout, Marco Bambrilla, or Mariot Chauvin to cite a few and more expected to come. Find the majority of the pointers at the http://modeldrivensoftware.net

In this edition, the conference has been deeply covered via twitter using #cg2011 and #lwc11

As expected, I will be only be able to comment about the sessions I personally have attended. Running three tracks in parallel always force us to choose one and miss two other great sessions.

Day 0. The Language Workbenches Competition

Ten tools were presented in a marathonian session exactly allocating 40 minutes per tool. The Angelo’s egg-timer was implacable: Whenever it rang reaching the agreed time, the speakers suddenly stopped talking unable to end a simple phrase.

The challenge was a competition without a winner or loser, mainly because there was no prize to win (may be next year a generous sponsor could change that). The main objective is to compare how different tools are able to complete a common problem facing modeling, model transformations and code generation to multiple platforms.

Johan den Haan has prepared a detailed report of the tools presented. Take a look on it.

From my side, I have created a feature matrix for comparing the different approaches stressing what I see as different in each tool. It is not complete: some information is missing and some other not yet contrasted so take it as a beta version to be improved.

With respect to my beloved tool Essential, I have to say that I received a very nice feedback from the audience. See a sample of the twitted comments:

kthoms: @pmolinam Has a high voltage notebook – projector crashed on plugging in. Now let’s see how he could improvise the situation #lwc11

stevekmcc:  @pmolinam Showing #Essential: major theme is separation of concerns, so opposite to Rascal #lwc11

stevekmcc: #Essential looks very clean compared to other text or text-projection workbenches #lwc11

delphinocons: RT @stevekmcc: #Essential looks very clean compared to other text or text-projection workbenches #lwc11 <== I agree. Relaxing to the eyes.

delphinocons: #lwc11 The Essential metamodel is smaller than EMFs, and a little larger than MetaEdit+’s

meinte37: @pmolinam is the first to finish before the egg timer at #lwc11. That’s because he cuts down to the Essential-s

JohanDenHaan: @pmolinam nice presentation and demo, good mix of slides and demo. Great timing!

Day 1. Code Generation 2011 starts

I started the day giving an initial talk about Introduction to Model Driven Software Development.

Later on, Obeo guys (Mariot Chauvin & Stéphane Begaudeau) presented how to use Obeo Designer to create sensible graphical DSL tools on the top of the Eclipse framework. Obeo uses model interpretation over the EMF models to create in runtime GMF based editors.

Panel: Build or Buy – who should develop and own your DSLs and generators?

We played a nice game with the following roles:

• Juha-Pekka Tolvanen (MetaCase), the moderator
• Jos Warmer (Independent) , the homegrow tool maker
• Pedro J. Molina (Capgemini) , the consultant
• Johan den Haan (Mendix), the tool vendor

More soon than later, Andrew Watson and Markus Völter join with the rest of the audience in the discussion. Of course, there is not an easy answer to that question and depends a lot of the particular context and requiring thinking about, but not only on:

• the experience of the in-house developers with MDD
• the level of criticality of the process for the business to be automated (core or non -core to business)
• the existence of tools able to provide the requested features
• the need to split the business process from the current technology

Type Systems for DSLs.

Markus Völter presented a nice library he has been creating to help with the type checking in expressions of DSLs. After a gentle introduction to type checking, he introduced three different approaches to the problem:

• recursive type-checkers,
• union based (a la MPS) and
• table driven (a declarative way of the first one).

Day 2. Getting speed

Terrence Parr Keynote was one of the more awaited talks in the conference. Many of us use their tools on a daily basis like ANTLR and StringTemplate. So having him here is quite special. In his keynote titled “Why program by hand in five days what you can spend five years of your life automating?” he let us some pearls like the following ones highly twitteable:

• “XML is not for humans beings, just for machines.” (Donald Knuth, the father of TeX support this, and I am not going the one to contradict this)
• “Programmers are lazy.” Yes, we are. We don’t like repetition of tasks, trivial or not, because we get bored, so we always try to automate things. And this is our main driver and source of fun.
• “Automate the things more prone to human errors”. Automate the tedious work!
• Running a test does not increase the quality of the code.

MetaEdit+ Hands On

MetaCase MetaEdit+ was of my pending tools to try from previous editions of CG. I had the tool, I had the chances but not the inline tutor to allow me to get advantage of the tool. Risto, Steven and Juha-Pekka did a great work in the “hands on” session been able to not to left away any of the practitioners. They have did this lots of time and they are good teaching it.

Definitely, the learning curve of MetaEdit is not big, but for sure, you need the initial proper training to get the grasp of the tool to start creating your own graphical DSLs. Been a tool implemented in Smalltalk and using a repository, MetaEdit provided an edition experience quite different to the ones more used to edit traditional files.

Generating Graphical DSLs

Marko Boger, one of the fathers of the Poseidon tool, presented his work about applying DSLs to describe graphical DSLs. The idea was well received, and later on in the local pub called The Castle Inn an informal Birds of a Feather session was settled to create the Spray project to join forces and create a common core DSLs to help in generating graphical DSLs.

Modeling the UI

To end the day I gave a talk about how we take in account to model the User Interface of a business application. The topic is important because the IU is not always pondered as it should be. The UI is the only aspect the user will see from any system we build. Approaches to modeling and code generation based in a pattern based approach were presented with the help of briefs live demos using Essential and Io tools.

Day 3. Closing a great edition

Ed Merks Keynote

Ed is the leader of the Eclipse Modeling Framework based in Montreal and now working also for Itemis. In his keynote he remarked some of the prejudices people use to have against code generation and (UML) modeling, leaving also some great quotes:

• “Resistance is futile: Best practices will be assimilated by code generators.”
• “XMI sucks because XML sucks. XMI sucks for transitivity.”

Ed talks reinforces my idea that, talking about software, Europeans are usually more instructed in the university in formal methods and theories where Americans uses to be quite more practical and take a more empirical approach to software engineering or computer science.

It is interesting to see that mostly all the participants in the LWC with the exception of Intentional are Europeans. Well, Charles Simony (the owner) and Mats Helander (the presenter) both from Intentional are also Europeans. So, there is no interest in Language Workbenches in other continents apart from Europe?

Goldfish Bowl: “Code Generation as a normal programming practice”

Jos Warmer lead this session were all the interested participants shared thoughts about how to integrate code generation with traditional styles of development.

How to create a good template: Go to your best developers and tell them: “write this DAO code file as it were the last time in your life you are going to write it”. Then abstract it and you got it. If the further in time found a bug, they can always came back and fix the template also.

Final panel: “Models, DSLs, Transformations: The Next 5 years”

In the closing plenary session, future directions and incoming challenges were addresses by the panelist and the audience.

• Andrew Watson focused on resolving business problems instead of technological problems.
• Jos Warmer encourages us to build better tools and better languages.
• Johan den Haan pinpointed some incoming trends: Mobile, cloud, social, multi-core and poliglot languages.
• Wim Bast commented about the need for parallelization and the best way to achieve it is to move to a more declarative DSLs allowing us to exploit the intrinsic parallelization in the execution of such algorithms. Wim pointed out also about the difficulty of making prediction about the future.

So, I put my two cents remembering the genial quote of Alan Kay: “The best way to predict the future is to invent it”.

My position is that we have the tools, we have the knowledge… so, we should put all our efforts in improving the tools. Good tools will make MDD mainstream soon or later. So, I want to see it happening: let’s improve our tools now better sooner than later.

Conclusions

The LWC has been a great idea made real. Having the opportunity to compare solutions and styles of tools has been quite productive for all of us. All the participants were happy with the idea of repeating next year and the next edition is looking for the next challenge.

Twitter this years has expanded the real-time conference far away that the walls of the College. Colleagues like Rui Curado, Xavier Seignard, Jordi Cabot, Angel J. Lopez or the AltNet Hispano group to cite a few I am aware were tracking the conference and taking part via twitter. This enriches the experience and makes it more participative.

Some tools, highly integrated with the Eclipse environment were suffering during the conference live demo failures. This is not an exception, but a warning signal that some task that should be easy are getting complex and complex when the number of dependencies to run a simple task is out of control. The friction level of these environments is definitely something to keep as low as possible. Model interpretation versus traditional code generation can be helpful also to reduce startup times and opening another instance of the same environment.

The new born project Spray (as reported by Karsten) can be quite helpful as a neutral DSL definition for describing graphical DSL in an technological agnostic way. The risk however is if it is too close to a certain technology stack like GMF or Graphiti it will only be used in such context. To my understanding, a Spray definition should be implementable also in Microsoft DSL Tools  to put a extreme counterpart.

If a contest for the novel tool of the year will have ever existed in CG, IMHO this year had been awarded to The Whole Plaftorm by Riccardo Solmi and Enrico Persiani. They have a nice and cool looking projectional tool that was deeply unknown to all of us. So, welcomed to the club!

XML is definitely not for humans (Terrence Parr) and XMI sucks for XML transitivity (Ed Merks).

Finally Mark Dalgarno and his team were superb organizing all the logistics to make all of this possible. Thanks to everyone and see you there next year.

The Code Generation 2011 conference and the Language Workbenches Competition 2011 Workshop are quite close in the calendar.

I want to celebrate it with the MDD community sharing my work on Essential (a tool designed for acquiring speed with Model Driven Development). On 11th may 2011, version 0.4.44 has been released as the first public beta.

Essential is a meta-modeling and code generation tool providing specific DSLs to define and consume:

• Metamodels
• Models
• Templates (using StringTemplate) &
• Transformations (Model2Text and Model2Model)

Try it also with the sample projects created for the LWC 2011 challenge.

Finally, I will also be there presenting Essential as a solution to the challenge. It’s a nice excuse to go there, just in case!

Now I can hear in the background the sound of knives being sharpened, nevertheless with an Olympic spirit. }:)

If you want to see the State of the Art of the next generation Software Engineering tools in action, don’t miss the opportunity and join us. See you there!

When building a new tool for modelling and code generation like Essential, one has to rethink again what template engine use to drive all the machinery. In code generation contexts, template engines are a good field for innovation and your choice will be with you probably for the full lifetime of your tool.

In this post, I will try to introduce and explain why StringTemplate is a superb engine for doing code generation and why you should consider it if dealing with a code generation scenario.

As commented before in other post, I felt in love with StringTemplate in 2005 when I discovered ANTLR and StringTemplate during the development of a MDD workbench for a customer. Prior to that date, I had tested many template engines and also developed on my own a pair of them. For one reason or another, I was not comfortable with the language, the coupling of the template with the transformation control, or the maintainability of the templates in the long term. As the time goes by, this last point arises as crucial to maintain and evolve your architecture.

Therefore, one could say that I am being quite loyal to StringTemplate for the last 5 years, and that too much for technology, moreover, for a product software. And such loyalty is definitely not my merit for not been promiscuous in this issue, but a merit of StringTemplate: I’m still looking for new ways for improvement and didn’t find one better to my needs. (Note: this is probably the best compliment I never paid to a piece of good software.)

Some products shine thanks to the many features they provides, StringTemplate sparkles for avoiding adding features in the wrong place. I will elaborate a bit more on that.

MVC

Model View Controller is an old and good “classic” architectural pattern providing a solution to organize the pieces of a user interface. Originally proposed by Trygve Reenskaug, in Smalltalk in 1979. It has been lately again on fashion, especially in the building of web sites, multi-channel user interfaces and promoting UI testability thanks to the application of the principle of Separation of Concerns that the pattern provides.

There are many variants or dialects to MVC out there, but the simple one is enough to describe our scenario: code generation.

Code generation can be seen as a classic MVC domain with the three roles easily identifiable:

• A model: the variable data in consideration, the metadata to be generated in this case.
• A view: the template with the form of the target language and providing holes or slots to be later being fill in (the output).
• And the transformation controller: the piece in charge of selecting with model parts should be selected, combine and finally inserted in which template holes/slots to render the final and concrete view.

The main problem with traditional template engines is that there are lots of template engines that mix the last two roles (for example all coming from web development in the flavour of JSP or  ASP variants). The view and the transformation controller are intermingled and this situation, in the long time, is not good. I found this strict separation of responsibilities crucial for the maintainability of the templates.

And StringTemplate does exactly that. It syntax and features, although could be strange for the first time, was carefully selected to not allow you to do control choices in the view. It allows to define templates, declare named holes (or slots) and an inline for-each and if statements.

Let’s review a basic sample of StringTemplate syntax:

group mail;

genMail(msg)::=<<
Mail: Subject $msg.Subject$
From: $msg.From:genAddress()$
To: $msg.To:genAddress(); separator=”, “$
>>

genAddress()::=<<$it.Name$ ($it.Email$)>>

The sample contains two templates: one for a mail, and a second one for each address. Templates can have parameters explicit parameters like name, but also there is an implicit keyword (it) to reference the current iterating context.

Named holes or slots are defined are marked with $expression$ where expression is a constrained expression resolution of the model properties received.

In the sample, genAddress() template will be invoked to resolve the From and To properties. Assuming that From is a monovaluated property and that To is a list, the template iterates throw the collection received and evaluates the template for each item. Note the separator modifier used to describe how items should be rendered.

That’s all in the template, the view. It’s forbidden to compute any value inside the template. The idea is that only read operations are going to be done into the model to get the data needed to fill the template.

How to consume the template?

From the transformation side (the controller) consuming a template is something such straightforward as the following one:

//1. Select the template
StringTemplate tlp = group.InstanceOf(“genMail”);
//2. Set parameters
tlp.setAttribute(“msg”, message);
//3. Get the result
string result = tlp.ToString();

It is frequent to find people complaining StringTemplate about not having enough freedom for example to call functions from the inside. But this is not a missing feature, this done on purpose to assure no side effects of such computations. The motto is; if you need to compute something, do it the transformation stage.

This only has covered a very basic intro to StringTemplate. If you are interested you can browse for more contents in the main documentation of StringTemplate.

Of course, Essential uses StringTemplate!

ST & ANTRL on Code Generation 2011

By the way, the father of StringTemplate and the superb meta-compiler tool called ANTLR is Terrence Parr. He teaches compiler construction in the University of San Francisco.

I was planning to prepare this post for some months, but in the meanwhile, Mark Dalgarno (the main organizer of Code Generation Conferences) has surprises us announcing the next keynote for CG2011.

I am quite happy to see that, as announced by Mark, that in the next edition of Code Generation 2011 Terrence is going to be present as one of the invited keynotes!

So, if you are interested in all of these topics, join us and meet Terrence on May 25-27, 2011 in Cambridge, UK on Code Generation 2011.

The call for speakers for CG2011 is open till January 15^th 2011.

To know more:

1. Enforcing Strict Model-View Separation in Template Engines, Terrence Parr
2. StringTemplate
3. ANTLR

The Hello World sample is a nice starting point to show the syntax and capabilities of every new language. This test is also useful for code generators and Domain Specific Languages (DSLs) also as a proof of concept.

Following this honorable tradition, I have created a video showing the capabilities of Essential: the tool I am working on for doing agile Model Driven Development.

In this 10 minutes video you will get a general idea of the DSL the language provides to create:

• metamodels
• models
• templates
• and control transformations

In order to see the details, jump to Vimeo, activate the High Definition mode (HD) and set full screen (sorry embebed version is not good enough).

Essential IDE – Hello World sample from Pedro J. Molina on Vimeo.

More info about it and 12 usage scenarios in the last Code Generation 2010 presentation about Tailored Code Generators.

Share your impressions!

Currently, the main efforts on MDD, MDE, MDSD (model-driven-whatever you prefer…) are focused in the development of this kind of tools perceived as a hot research area for Software Engineering.

In this scenarion, Cambridge, at Code Generation 2010 was the perfect place for sparkling the idea of promoting a contest to show and compare the advances of different language workbenches.

The Language Workbench Competition born with the objective to serve as a point of comparison between different tools in this exciting and fast moving area.

The competition is now open to the public. So anyone interested can enroll and implement the proposed challenge just published.

On the other hand, if you want to know more about Language Workbenches, modeling and code generation add this page to you bookmarks and come back in few months to see some proposals.

The promoters of the idea are: Markus Völter, Eelco Visser, Steven Kelly, Angelo Hulshout, Jos Warmer, Bernhard Merkle, Karsten Thoms and myself.

So this a call to arms but with sportsmanship!

Angelo and Markus has already started the calling.

When creating a DSL (Domain Specific Language) one of the most important choices is to decide about what items in your domain are going to be considered variable, changeable and which ones are going to be considered fixed, carved in stone.

The former need to be specified in your DSL, in your design or may be coded. The latter are considered immutable and will remain static for all your derived applications for ages.

Considering that everything is static it is obviously useless. On the contrary, considering every aspect variable drives to another no-end getting nothing tangible again as a result. Therefore, in the middle we will have to search for the virtue.

The main issue here is to study a domain and ponder between variable parts and fixed parts. It is not a trivial thing to do from the very beginning. Experience in DSL construction and specially, experience in the domain helps to train your smell, but there are not clear rules for it, nevertheless.

It is not only about knowing your requirements. It is about trying to predict how your requirements will change across time and what types of requirements have more likelihood and tendency to change.

Adding variability

A variable part could be, for example, the background color of your application. If so, you need to add syntax and semantics to your DSL to capture such property. Let’s say you can express somewhere in your specification:  { background-color = peach; }

We can select the peach color for app1, and may be ivory for app2.

However, nothing is for free and this freedom comes with the followings possible drawbacks:

• You need to increase the size of your language (DSL), editors, model checkers, compilers and code generation or interpreters.
• Users have to provide a value for such property unless you have also provided a sensible default value in case of missing information.
• Homogeneity across applications vanishes with respect to background-color. Now it’s a user choice (the one in control of the modeling tool).
• Specs are more complex.

Adding commonality

On the other hand, if you consider the background of your application should be always the same because you are following, for example, a user interface style guide then, the background color is a fixed issue. Its value is provided by design by a style guide, by an architect, or design choice and the user modeling has no control over it.

In this scenario, the DSL is smaller. No need to specify the background color, it is implicit, it is no included in the model/specification.

With this kind of choice, we are betting for standardization. A shared library, a runtime framework or an interpreter will take care of supplying the right color in the right moment.

• Users can not change the background color, specs are smaller.
• Standardization is improved across applications.
• User has no control on the feature.

But, what is the right choice?

It depends. There is no right choice with the information given till the moment. To answer the question we need to consider if the background color is a fundamental feature in our domain and it is needed to be different from application to application or may be, on the contrary, the color should be used in an homogeneous way following a predefined style guide.

Again, the domain imposes the rules to follow. Studding the domain and its variability is crucial to create a consistent DSLs focused in gathering the key features of the domain in a model: the important and variable ones. The important and fixed ones must be also identified but they shouldn’t be included into the model, but into the framework or the runtime.

Standards, policy assurance, compliance

Everything related to standard procedures, compliance and in-house stile guidelines are first-class candidates for standardization. If done in that way, your developers will not have to remember all that weird standard and compliance rules when developing a specific artifact.

A code generator will provide the right value for them. It will do it silently, without errors neither oversights. All the boring code dedicated to plumbing applications like: naming guidelines, service publication, serialization, persistence, adapters, proxies, skeletons, stubs, DAO code are driven by strict standards and best practices and are natural candidates for strong automation by code generators.

Moreover, if the regulation or the standard changes, the change will have impact in the following assets:

• a single change to a framework will be enough
• or a change to a code generator and then forcing a regeneration process and redeploy.

In both cases, it is cheaper that manually reviewing a set of in-production applications.

For example, think about replacing your data-layer access code from a DAO pattern and SQL to an ORM based approach like Hibernate.

Business Know-How

The core of the business Know-How is the important and the variable parts we are interested in to be collected in a specification. Such features need to be modeled, and if possible, abstracted from the technology that will implement it.

If we do it in this way, the model can survive the current technology.

Why we could be interested in do it in such a way?

Just because technology evolves like fashion. Today everyone likes red T-shirts, tomorrow blue jeans will be sublime! Basic, Cobol, C, Java, C#, Ruby… what is the next language to use in 5 years time?

Use your best bet, whatever platform better fulfills your requirements, but I it could be nice to see the business process surviving the technology.   We don’t know in which direction, but technology will evolve, and will change for sure.

Maintaining a language or a DSL

When a DSL or a language needs a review you will be probably considering adding new features to the language.

Each new feature will increase the variability and increase the complexity of the language. Before deciding to add a new modeling feature make a cost/benefits analysis and double check that the valued added by the improvement is greater than the cost of implementing it.

I like to follow the golden rule proposed by Gordon S. Novak about automatic programming:

“Automatic Programming is defined as the synthesis of a program from a specification. If automatic programming is to be useful, the specification must be smaller and easier to write than the program would be if written in a conventional programming language.”

Conclusion

Whenever is possible:

• Business Know-How should be captured by models, specs, DSLs.
• Technical Know-How should be captured by code generators, model interpreters, best practices and patterns.

So, at the end of the day I like the following pair of quotes to sum up about what to include in a model:

• The Spanish writer Baltasar Gracián in the XVII century said “Lo bueno si breve, dos veces bueno.” (a literal translation from Spanish could be: “Good things if brief, twice good.”)
• On the other side, Albert Einstein (XX century) counterpoints “Things should be as simple as possible, but not simpler.”