The Metalevel

A title like this could sound utopian and probably it is in the extreme, but at the same time provides two advantages:

1. a suggestive title brings more blogs readers , and
2. these kind of Quests keep me active developing and testing my ideas about MDD.

Sorry for 1, the joke. But Let me explain 2 a little further if you are interested:

Some months ago, I started to develop a new tool for doing Model Driven Development. I always start new developments with contained expectatives. I consider most of them throw away propotypes just to test a new way of doing the same thing, but luckily, with less effort and better productivity. But when you found the way, you follow it again whenever you need it, isn’t it?

So, this time is one of those moments: I am quite happy with the results, and I will continue building on the top of it to make it grow.

Today (almost?) everyone agrees on code generation can make your life notably easier as long as you have:

1. a stable domain,
2. a clear knowledge of your domain (domain expert) and,
3. tools: modelling editors, model checkers, and code generator adapted to your domain and your chosen target architecture and language.

The trickiest one to get is, not surprisingly, the third one. And this issue: the quality and the applicability of the MDD tools is the main stopper when considering applying MDD for a software development project.

So, my motivation during years is not only to create code generators one more time, but do it in a way that it will be cheaper to obtain results in next projects. Therefore, reducing the requirements for the entry level will help the MDD adoption to gain speed.

Ok, wait a minute, are you reinventing the wheel again? Building another meta-code-generator?

Yes I know, there are very good and mature tools for doing MDD: such as EMF/GMF, XText, ATL, TextUml, Metacase, MS DSL Tools, etc.

However, guided by my intuition and the experiences in the domains I have worked in, I have a strong opinion on of how some things should be done and for the moment I didn’t found the perfect tool to satisfy my needs.

But, instead of complain, I decided to take to the action and add my two cents implementing my view about MDD tool support: taking the good ideas of the standards available and reinventing the parts that don’t fit.

The tool

After this introduction, the name of the tool is not going to surprise you. It is named Essential.

The goals of the project are following ones:

• to declaratively describe metamodels, models, templates, and transformations using textual DSLs
• to provide a comfortable editor for each of these four pillars,
• to provide model checkers to assure the integrity of the four, and
• to build code generators and transformation interpreters to achieve the output we are looking for.

In the next posts I will depict the DSLs used in Essential and some architectural choices. This will help us to discuss the essence of the problems found in MDD.

I want to thanks Javier Hernandez for his constant help, good discussions and counterpoints about choosing design alternatives for Essential. And also to Nicolas Cornaglia and Ángel Marín for their active beta testing as early adopters and providing good test fields.

Hello everybody!

I want to share with you a set of videos to show what I understand when talking about Forward Engineering applied to MDD.

First of all, a legal disclaimer: my apologies for the quality of the videos and for my rusty English: I am starting to play with video editing tools and recording software so I expect to improve my recording and editing skills on the way. Anyway, I found (I hope) they have enough quality to explain the main ideas. So seeing it is totally up to you! You have been warned! 

Also note: see the videos in High Quality mode (HQ) in Youtube. Otherwise, details of the samples probably will not be visible.

In a previous post I introduced a sample of the code that can be generated from a very basic conceptual model. I have created three videos to show you the main steps involved.

1. Modeling (Video 1/3). The first video uses a minimalistic class model created inside Visual Studio 2005 with Microsoft DSL Tools. The sample creates a basic blog structure in less than 5 minutes. Note that in the specification there are no technological choices (neither the types are bind to a concrete language representation).
2. Code Generation (Video 2/3). A quick step: Selecting a code generation (selecting a target architecture), fixing the design choices offered by the code generation and pressing the red button: Generate! A full .NET Solution is generated in less than 5 seconds ready to compile.
3. A quick code review of the generated code (Video 3/3). Finally, I am sure you have curiosity to take a look to the output code, don’t you? This third video shows a walkthrough to show:
   • DB Scripts (table creation, foreign keys, drop scripts)
   • Database creation
   • Logic layer: POCOs (Plain Old CLR objects), NHibernate mappings and a Business Service Layer with fully functional CRUD operations.

So this is it. It is a proof of concept of how fast and direct MDD tools can be starting from a minimalistic model.

When talking about using or buying modeling & code generation products my advice is:

• Don’t use models just for documentation. They will be outdated soon or later. On the contrary, a living (generating) model is always in sync with its target application.
• Don’t resign yourself to just using code generation of skeletons. As you just have seen the current technology allows you to generate much more.
• Don’t be content if anyone try to sell you a model too close or tied in any way to a given target language. Today we have just generated C#, but tomorrow may be we prefer Ruby? Python?
• Don’t resign yourself to use a tool married with a specific database. You know, technology changes faster that we usually expect.

When talking about Model Driven Development (MDD), code generation and related technologies, the key terms, its meanings and usage could not be clear enough to the involved audience due to our different backgrounds and the use of overloaded terms in several specific jargons.

A good way to avoid such confusion is to make explicit a short clear definition of the terms used.

Following this idea, today I want to share the Conceptual Map I traditionally use to explain the areas involved in code generation. This map helps me to explain how more complex interactions and natural dependences and properties arise in this domain. See embedded video.

We will start considering MDD using two orthogonal dimensions:

• Types/Classes vs Instances &
• Two levels of abstraction: Programs/code (technology dependent, concrete)  vs Models (technology independent, abstract).

Key concepts

1. Model: A description of the problem to be solved.
2. Metamodel: A type description for providing the modeling language.
3. Template: A pattern representation of the code you want to create.
4. Transformation: Application of an algorithm to transform one or more models in one o more outputs.
5. Output: Your target asset: source code, doc or models again.

Additional concepts

• Mappings = Template to Metamodel item bindings
• Architecture Know How = Mappings + Templates (embedding best practices and patterns)
• Business Know How  = Metamodel + Model
• Model Checker = Validation of Model with respect to Metamodel
• Generator = Transformation(Templates, Mappings, Model) –> Output

A pair of properties to mention.

Stability:

1. Metamodel: Quite stable once developed.
2. Model: Stable till the business change (requirements change).
3. Template: Stable till not changing or evolving the technology (2-3 years cycle?).
4. Transformation: Impacted by technology or metamodel changes.
5. Output (Source code): Regenerated as needed. Discardable.

Dependences:

1. Metamodel: Not dependent of third parties.
2. Model: Altered if metamodel changes.
3. Template: Not dependent of third parties (just the technology).
4. Transformation: Impacted by template changes, mapping changes and  metamodel changes.
5. Output (Source code): Impacted by everything. Regeneration is a good property as seen before. But now is a necessity.

Changes in the formers will cascade to the latter.

Strong Separation of Concerns & skills

Apart of the classical advantages provided by MDD approaches such as quality, productivity, time to market, & technology independence; one of the good properties no so times recalled is the strong separation of skills that this method of software engineering provides:

• Business analyst can concentrate efforts in using modeling editors to describe the business and provide feedback to enhance metamodels (when needed)
• On the other side, Software Architects and technology gurus can concentrate in tuning the technical architecture to achieve the most performing implementations they can provide.

One you achieve this stage, you can start thinking of reusing and evolving your business models and architectures at speeds that most of traditional SW developers never seen before.

There is one famous quote from Grady Booch saying:

“The entire history of software engineering is one of rising levels of abstraction (abstraction is the primary way we as humans deal with complexity).”  Grady Booch

As many others, I think that MDD is the next step to adopt in such direction. When it will happen globally is just a question of achieving the critical mass:

• having good enough tools,
• and having practitioners enough and success cases to prove the value in the “not so new” way of working.

I’m optimistic about it. We are in the way and getting speed…

The call for speakers of Code Generation 2010 is now open.

If you are a practitioner, researcher or a tool maker in the MDD arena this is your conference.
You have till January 15th 2010 to send your submission.

See you there!

After coming back to Valencia from Montevideo, I’ve found the time to organize my ideas and explain as promised what I saw there.

As commented before, my expectatives were exceeded. The Genexus Event organized by Artech has a great quality level: more than 3.600 participants, having more than 120 sessions in tree days. I’m impressed! These kinds of things are not improvised, and the organization did a wonderful job for the event. Congrats!

In this post I will comment about the things I saw and liked (specially sharing the links to the videos and abundant material) and about my, now, better understanding of the tool Genexus.

Note: There is a some of material with on-line translation to English, the rest is only in original version (Spanish).

More… »

Selecting a good Template Engine is an important issue when dealing with code generation.

Since 1998 I’ve been testing almost lots of them and forming an opinion about it at the same time the technology evolved. Been at CG2009 and attending the session of Kathleen Dollard on Template Specialization brought to my mind all the times I wasn’t satisfied with my current template engine and changed to try for a new one.

Doing commercial code generators, I’ve been tested lost of techniques: direct string concatenation, XSLT, direct code generation (or what Kathleen calls brute force code gen), ASP, JSP based approaches, developing custom template engines (two of them), taking a look to Code Smith, T4 and others, using NVelocity and finally arriving till StringTemplate (for the moment).
More… »

This was my contribution for the Code Generation 2009 conference.

The central idea of my speech was devoted to achieve scalability in MDD tools & methods for industrial application of the MDD technology.

I was so glad to see that Markus Völter and Steven Kelly also just 30 minutes before my talk, were addressing the same key issues in their inspiring keynote: scalability, partitioning, modularization of DSL. This fact reinforce me in the idea that we are really growing as a community (CG) with common problems and aligned in the solutions improving the tools & techniques more and more every year.

Scaling MDD for production: enabling SoC at model time

There was also time for an small but complete live demo of a small tool socDriven running and demonstrating the concepts explained. The tool was implementing a compiler and merger for a small textual DSL. The DSL was parsed using the OSLO M_Grammar to create the AST. A merged engine in a second phase glued the partial specifications producing as output:

• the same specs merged in a unique file,
• the same spec ready to be open with a graphical editor with DSL Tools,
• a full code generation solution on VS2008 containing:
  • scripts for SQL Server
  • a logic layers with POCOs & full NHibernate mappings
  • a default CRUD service definition,
  • a default Web Service definition,
  • and a Windows Form UI ready to query and execute CRUD operations.

After the generation of the solution, it can be opened with VS, create the DB launching the scripts provided and pressing just F5 for building and running it all! == (a.k.a the utopia of 0 custom code) };-)

Please, feel free to add your comments!