Root CUIP Metalevel

Posts tagged “DSL”.

MDD talk at UCLM/ABC

To give a talk about my favourite topic, it’s always a pleasure. To give it at home, it’s an invaluable pleasure.

I’ve been invited to talk about MDD in my native university (University of Castilla-La ManchaUCLM at Albacete, Spain) where I started my Computer Science studies. So, I have a lot of good feelings remembering my student days every time I come back there.

On Monday, February 22th, I will be there sharing my points of view and experiences about applying MDD.

Update:

When & where:  February 22th, 18:00 H. Salon de actos de la Escuela Superior de Ingeniería Informática, Albacete (location).

Modelling with Essential

After presenting the metamodel DSL, today is the turn for the modelling DSL.

As commented previously, Martin Thiede has a nice prototype called Concrete of a Web editor for model and metamodels. Such work and video also helps to understand the modelling and metamodelling duality.

Coming back to Essential, our tool is also providing a DSL to instantiate a metamodel. But a sample will probably be more illustrative:

Suppose the following simple metamodel:

namespace Core.Class
{
    class ClassModel
    {
        composition List<Class> Classes opposite Model;
    }
    class  Class
    {
        string Name;
        composition List<Attribute> Attributes;
    }
    class Attribute
    {
         string name;
         string Type;
    }
}

You could consider it as a very basic OO model. Intentionally I haven’t added too many details to keep it simple.

If you continue building it; adding operations, arguments, association support and more… at the end, you can have a metamodel comprising a full class diagram of UML, for example. The complexity of the metamodel you want to use is up to you. Having the control of the concepts inside the metamodel you can always add or remove features.

Modelling tools focused in UML, for example, provides this metamodel closed. You can’t change it, it’s a hard-wired standard and you are not allowed to change the core directly (only by extensions points like stereotypes and profiles).

Modelling Language

Now, let’s see how we can create a model using the previous metamodel:

using Core.Class;
namespace MyModel
{
    ClassModel  myClassModel
    {
        Classes = [Customer, Invoice];
    }
    Class Customer
    {
        Name = "Customer";
        Attributes = [Name, Surname];
    }
    Attribute Name
    {
        Name= "Name";
        Type= "string";
    }
    Attribute Surname
    {
        Name= "Name";
        Type= "string";
    }
    Class Invoice
    {
        Name = "Invoice";
        Attributes = [
            Attribute Amount
            {
                Name= "Amount";
                Type= "decimal";
            },
            Attribute InvoiceDate
            {
                Name= "InvoiceDate";
                Type= "date";
            }
        ];
     }
}

In the sample, a basic declaration of two classes (Customer and Invoice) has been instantiated using the concept Class defined in the metamodel. Each metamodel concept is instantiated using it as you would be using a class in Java, C# or C++. Properties are assigned in the natural way one would expect in such languages.

As expected, the using operator imports the metamodel to be employed.

In the lines 26-37 you can see how properties can be also defined inline for list contexts.

Under the curtains, the tool is providing code completion, syntax colorization, error management and real-time parsing as you could expect from a modern IDE.

Essential uses DSL text for serialization but also XML can be derived. This XML document is conformant with an XSD derived from the corresponding metamodel.

Intended usage

Consider now, a basic DB metamodel:

namespace Meta.Database
{
    class Schema
    {
		string Name;
		composition List<Table> Tables;
    }
    class Table
    {
		string Name;
		composition List<Column> Columns;
    }
    class Column
    {
		string Name;
		string SqlType;
    }
}

With this base in mind, you can define a business model containing (Customers, Invoices, etc.) based on a class metamodel and then, look for ways to transform your class model in, for example, a DB model representation. This is called, a M2M transformation (Model to Model). Essential will provide specialized languages to do that.

After that, your derived DB model can again be generated to final code, for example SQL DDL scripts in a given certain dialect. This is generally called a M2T transformation (Model to Text) and again Essential provides tools to describe the transformations.

Basically, to complete all this stuff, we need to introduce two more specialized languages: one for applying transformations, and another one for templates to be converted to final text.

I will focus on templates side on the next post.

Concrete: a Web modeling editor

Martin Thiede has released a video-demo about his last project: Concrete.

It is an original web based meta-model and model editor.

Take a look on it, it makes worthy!

Essential.Meta: A concise DSL for metamodeling

L'umbracle, Valencia, from the Collection: Life on Mars? By iFovea via flickr.com

In my previous post, I introduced Essential, a custom-developed tool for doing Model Driven Development.

As promised, in this post I will describe the primitives of the first language of the tool: Essential.Meta. This meta-language is helpful to describe the structural aspects of a metamodel.

The input requirements for the language are:

  1. Usable for meta-structure description
  2. Human readable/maintainable
  3. Concise as possible
  4. Easy to understand
  5. Object oriented
  6. Technology agnostic
  7. Textual
  8. Extensible
  9. Scalable
  10. Reusable definitions

Ok, that was the initial wish list. Now, let’s review how we can cope with it and show a short example of how it looks like:

namespace Meta.TownModel
{
    enum Sex  {  Male, Female }

    class Town
    {
        string Name;
        List<Person> Habitants;
        composition List<House> Houses;
    }

    class Person
    {
        string Name;
        string? MiddleName;
        string Surname;
        Sex Sex;
        int? Age;
        List<Person> Children;
        House Home;
    }

    class House
    {
        string Address;
        composition List<Room>0..* Rooms opposite 1..1 House;
    }

    class Room
    {
        string Name;
        decimal? Dimensions;   //In square meters
    }
}

The sample describes a meta-model for describing a town, with its buildings (houses) and inhabitants (persons). Nothing complex, nothing weird if assuming I am using a C/C++/Java/C# syntax and this doesn’t suppose a problem for you. I selected a C-like syntax instead a Pascal or VB flavour due to its less verbosity and a probably biased to what we used get to.

The namespace defines a scope for naming. All of our definitions are enclosed in a namespace. Therefore names of concepts should not collide inside a namespace. Same names are allowed in different namespaces.

As you can see, the main concepts in this language are classes and enumerations.

Enumerations define closed sets of named values helpful to describe domains. Classes on the other hand, have the classical meaning and properties or attributes can be defined for each class.

As usual in object languages each property has a type that could be primitive, an enum, a class type or a composed type.

The primitive data types supported are: string, bool, int, long, decimal, char, date, time & datetime. That should be more than enough, for the moment.

A note to readers: data types here are platform neutral, they are not java string for example, neither a C# string. It was horrible for me to see in a lot of diagramming UML tools whose names I will not cite, early mapping types to implementation languages like java:string and similar things… I sure you understand me. Such feature could be nice if you are doing reverse engineering or documenting code (a bit late BTW). But definitely horrible to see when doing conceptual modelling and haven’t chosen yet the final implementation platform.

Another relevant feature of the language is the cardinality operators. Each property has a cardinality attached to it. Following a Convention over Configuration approach, simple properties like string Name; has an implicit cardinality 1..1. You can also write to make it explicit in the language: string 1..1 Name;

List of things, on the contrary has an implicit cardinality of 0..* (zero to many). However you can further constraint it to 1..* (compulsory multi-valued) or arbitrarily to any pair of minimal and maximal values List<Holes> 18..24 GolfCampHoles;

The next feature to review is the composition keyword. A composition indicates that the property can’t exist alone, without its container. If the container is destroyed, the composed children will disappear also. Containment should be unique for each object; it cannot be contained in two direct containers at the same time. However, transitive containment it’s allowed.

There are some times, when we will be interested not only in one side of the relation between two classes, but also in the others’ end. That’s the case of the Children property. How about reaching the parents? To express it, the language introduces the opposite keyword. In our small sample it can be fully expressed (cardinalities included) as:

List<Person>0..* Children opposite 0:2 Parents;

Make sense, isn’t it? A person can have 0 or many children and let’s say zero, one or two well known parents.

I am still considering if the keyword List<T> could be also removed from the language. One can argue that cardinalities can be more than enough to express the same. However, I am still considering including other composed ADTs (abstract data types) in the future like Set<T> or Stack<T> to cite some of them. So, doors are still open to reconsider it.

Another feature we wanted to add is extensibility: If you are using this metamodel but needs to extend it for your own needs, the language is prepared to allow it. Just add another definition like this in another file and you got it:

namespace Meta.TownModel
{
    class Town
    {
        Person Mayor;
    }
    class Person
    {
        List<Person>0..* Friends opposite 0..* Friends;
    }
    class City : Town   //<--Inheritance sample
    {
        //extra city properties…
    }
}

Now, your town must have a mayor and persons can have friends!

The language allows partial definitions in the same or different files. If the namespace and class name matches, the properties of the Town class and Person class are extended by merging partial definitions.

From the cardinality point of view, Friends is a curious symmetric relation. It is a many-to-many relation but both ends  (the roles) are normally called Friends and not: MyFriends and OthersGuysConsideringMeAFriend but how knows! }:)

As briefly seen, this language is useful to impose rules over data. It constrains the objects/concepts we can describe and the allowed properties.

Intended usage

OK. Some of you are probably thinking: and what in hell it this useful for?

The mail goal is to use the language to describe the domain in which we are planning to apply MDD. With this description, we will describe the concepts, properties and relations of the problem domain we are interested in.

If you are really, really pragmatic, and the analysis and design of the domain for the joy of it doesn’t satisfy you, well, consider then that we can derive (manually or better 100% generated if you prefer) some other interesting and more earthly artefacts:

  1. UML class models (structure), XMI, etc.
  2. XML Schema (XSD),
  3. SQL Schema (relational tables to persist the data),
  4. Classes in Java (POJOs), C# (POCOs), or any other language implementing a pure Domain Model
  5. XML de/serialization code to read/save XML documents been conformant with (2)
  6. Data Access code (DAO) to connect (3) with (4).
  7. Maps to your favorite ORM tool to connect again (3) & (4)
  8. Etcetera, see Domain Driven Design and other approaches.

That’s all for today! This was the first DSL implemented in Essential targeting metamodeling. For more details, a full reference of the Essential.Meta language is described here.

On the next post, we will talk about a second DSL in Essential: the model (object level) language used to instantiate the concepts we just have created.

Thanks for reading! And please share your thoughts about it!

Searching for the essential building bricks of MDD

Red Brick Wall. Photo by Cibergabi via Flickr.com

A title like this could sound utopian and probably it is in the extreme, but at the same time provides two advantages:

  1. a suggestive title brings more blogs readers :) , and
  2. these kind of Quests keep me active developing and testing my ideas about MDD.

Sorry for 1, the joke. But Let me explain 2 a little further if you are interested:

Some months ago, I started to develop a new tool for doing Model Driven Development. I always start new developments with contained expectatives. I consider most of them throw away propotypes just to test a new way of doing the same thing, but luckily, with less effort and better productivity. But when you found the way, you follow it again whenever you need it, isn’t it?

So, this time is one of those moments: I am quite happy with the results, and I will continue building on the top of it to make it grow.

Today (almost?) everyone agrees on code generation can make your life notably easier as long as you have:

  1. a stable domain,
  2. a clear knowledge of your domain (domain expert) and,
  3. tools: modelling editors, model checkers, and code generator adapted to your domain and your chosen target architecture and language.

The trickiest one to get is, not surprisingly, the third one. And this issue: the quality and the applicability of the MDD tools is the main stopper when considering applying MDD for a software development project.

So, my motivation during years is not only to create code generators one more time, but do it in a way that it will be cheaper to obtain results in next projects. Therefore, reducing the requirements for the entry level will help the MDD adoption to gain speed.

Ok, wait a minute, are you reinventing the wheel again? Building another meta-code-generator?

Yes I know, there are very good and mature tools for doing MDD: such as EMF/GMF, XText, ATL, TextUml, Metacase, MS DSL Tools, etc.

However, guided by my intuition and the experiences in the domains I have worked in, I have a strong opinion on of how some things should be done and for the moment I didn’t found the perfect tool to satisfy my needs.

But, instead of complain, I decided to take to the action and add my two cents implementing my view about MDD tool support: taking the good ideas of the standards available and reinventing the parts that don’t fit.

The tool

After this introduction, the name of the tool is not going to surprise you. It is named Essential.

The goals of the project are following ones:

  • to declaratively describe metamodels, models, templates, and transformations using textual DSLs
  • to provide a comfortable editor for each of these four pillars,
  • to provide model checkers to assure the integrity of the four, and
  • to build code generators and transformation interpreters to achieve the output we are looking for.

In the next posts I will depict the DSLs used in Essential and some architectural choices. This will help us to discuss the essence of the problems found in MDD.

I want to thanks Javier Hernandez for his constant help, good discussions and counterpoints about choosing design alternatives for Essential. And also to Nicolas Cornaglia and Ángel Marín for their active beta testing as early adopters and providing good test fields.

Forward engineering with MDD: A proof of concept

Hello everybody!

I want to share with you a set of videos to show what I understand when talking about Forward Engineering applied to MDD.

First of all, a legal disclaimer: my apologies for the quality of the videos and for my rusty English: I am starting to play with video editing tools and recording software so I expect to improve my recording and editing skills on the way. Anyway, I found (I hope) they have enough quality to explain the main ideas. So seeing it is totally up to you! You have been warned!  :)

Also note: see the videos in High Quality mode (HQ) in Youtube. Otherwise, details of the samples probably will not be visible.

In a previous post I introduced a sample of the code that can be generated from a very basic conceptual model. I have created three videos to show you the main steps involved.

  1. Modeling (Video 1/3). The first video uses a minimalistic class model created inside Visual Studio 2005 with Microsoft DSL Tools. The sample creates a basic blog structure in less than 5 minutes. Note that in the specification there are no technological choices (neither the types are bind to a concrete language representation).
  2. Code Generation (Video 2/3). A quick step: Selecting a code generation (selecting a target architecture), fixing the design choices offered by the code generation and pressing the red button: Generate! A full .NET Solution is generated in less than 5 seconds ready to compile.
  3. A quick code review of the generated code (Video 3/3). Finally, I am sure you have curiosity to take a look to the output code, don’t you? This third video shows a walkthrough to show:
    • DB Scripts (table creation, foreign keys, drop scripts)
    • Database creation
    • Logic layer: POCOs (Plain Old CLR objects), NHibernate mappings and a Business Service Layer with fully functional CRUD operations.

So this is it. It is a proof of concept of how fast and direct MDD tools can be starting from a minimalistic model.

When talking about using or buying modeling & code generation products my advice is:

  • Don’t use models just for documentation. They will be outdated soon or later. On the contrary, a living (generating) model is always in sync with its target application.
  • Don’t resign yourself to just using code generation of skeletons. As you just have seen the current technology allows you to generate much more.
  • Don’t be content if anyone try to sell you a model too close or tied in any way to a given target language. Today we have just generated C#, but tomorrow may be we prefer Ruby? Python?
  • Don’t resign yourself to use a tool married with a specific database. You know, technology changes faster that we usually expect.

Report on Genexus Meeting 2009

After coming back to Valencia from Montevideo, I’ve found the time to organize my ideas and explain as promised what I saw there.

As commented before, my expectatives were exceeded. The Genexus Event organized by Artech has a great quality level: more than 3.600 participants, having more than 120 sessions in tree days. I’m impressed! These kinds of things are not improvised, and the organization did a wonderful job for the event. Congrats!

In this post I will comment about the things I saw and liked (specially sharing the links to the videos and abundant material) and about my, now, better understanding of the tool Genexus.

Note: There is a some of material with on-line translation to English, the rest is only in original version (Spanish).

More… »

Ready for Genexus Meeting 2009

Everything is ready for the XIX Genexus International Meeting. (14-16 of September, Montevideo)

Final programinvited speakers have been published.

As anticipated before, on Monday at 14:30 H (GMT-3), I will be there talking about The State of the Art of the Model Driven Development (MDD).

Using the 20% of UML

Via Jordi Cabot I found a quite interesting comment made by one of the “Three Amigos” about UML usage in his blog-post titled Taking the temperature of UML:

“Still, UML has become complex and clumsy. For 80% of all software only 20% of UML is needed. However, it is not easy to find the subset of UML which we would call the “Essential” UML. We must make UML smarter to use.”

Ivar Jacobson

The full post is a good review about the origins, the growth, and the current status of UML made by one of its three fathers. Ivar also argues about the current size of UML (really fat nowadays) and the necessity to put it on diet to achieve a kind of “Essential UML”. I like the idea of essential UML. It makes worthy to read it.

On the other hand, I want to stress the quote of Ivar Jacobson:

“For 80% of all software only 20% of UML is needed.”

I agree. My own quick usage stats of UML models when creating business software could be the following ones (please share your experience):

More… »

Choosing a Template Engine for code generation

Selecting a good Template Engine is an important issue when dealing with code generation.

Since 1998 I’ve been testing almost lots of them and forming an opinion about it at the same time the technology evolved. Been at CG2009 and attending the session of Kathleen Dollard on Template Specialization brought to my mind all the times I wasn’t satisfied with my current template engine and changed to try for a new one.

Doing commercial code generators, I’ve been tested lost of techniques: direct string concatenation, XSLT, direct code generation (or what Kathleen calls brute force code gen), ASP, JSP based approaches, developing custom template engines (two of them), taking a look to Code Smith, T4 and others, using NVelocity and finally arriving till StringTemplate (for the moment).
More… »