After presenting the metamodel DSL, today is the turn for the modelling DSL.

As commented previously, Martin Thiede has a nice prototype called Concrete of a Web editor for model and metamodels. Such work and video also helps to understand the modelling and metamodelling duality.

Coming back to Essential, our tool is also providing a DSL to instantiate a metamodel. But a sample will probably be more illustrative:

Suppose the following simple metamodel:

namespace Core.Class
{
    class ClassModel
    {
        composition List<Class> Classes opposite Model;
    }
    class  Class
    {
        string Name;
        composition List<Attribute> Attributes;
    }
    class Attribute
    {
         string name;
         string Type;
    }
}

You could consider it as a very basic OO model. Intentionally I haven’t added too many details to keep it simple.

If you continue building it; adding operations, arguments, association support and more… at the end, you can have a metamodel comprising a full class diagram of UML, for example. The complexity of the metamodel you want to use is up to you. Having the control of the concepts inside the metamodel you can always add or remove features.

Modelling tools focused in UML, for example, provides this metamodel closed. You can’t change it, it’s a hard-wired standard and you are not allowed to change the core directly (only by extensions points like stereotypes and profiles).

Modelling Language

Now, let’s see how we can create a model using the previous metamodel:

using Core.Class;
namespace MyModel
{
    ClassModel  myClassModel
    {
        Classes = [Customer, Invoice];
    }
    Class Customer
    {
        Name = "Customer";
        Attributes = [Name, Surname];
    }
    Attribute Name
    {
        Name= "Name";
        Type= "string";
    }
    Attribute Surname
    {
        Name= "Name";
        Type= "string";
    }
    Class Invoice
    {
        Name = "Invoice";
        Attributes = [
            Attribute Amount
            {
                Name= "Amount";
                Type= "decimal";
            },
            Attribute InvoiceDate
            {
                Name= "InvoiceDate";
                Type= "date";
            }
        ];
     }
}

In the sample, a basic declaration of two classes (Customer and Invoice) has been instantiated using the concept Class defined in the metamodel. Each metamodel concept is instantiated using it as you would be using a class in Java, C# or C++. Properties are assigned in the natural way one would expect in such languages.

As expected, the using operator imports the metamodel to be employed.

In the lines 26-37 you can see how properties can be also defined inline for list contexts.

Under the curtains, the tool is providing code completion, syntax colorization, error management and real-time parsing as you could expect from a modern IDE.

Essential uses DSL text for serialization but also XML can be derived. This XML document is conformant with an XSD derived from the corresponding metamodel.

Intended usage

Consider now, a basic DB metamodel:

namespace Meta.Database
{
    class Schema
    {
		string Name;
		composition List<Table> Tables;
    }
    class Table
    {
		string Name;
		composition List<Column> Columns;
    }
    class Column
    {
		string Name;
		string SqlType;
    }
}

With this base in mind, you can define a business model containing (Customers, Invoices, etc.) based on a class metamodel and then, look for ways to transform your class model in, for example, a DB model representation. This is called, a M2M transformation (Model to Model). Essential will provide specialized languages to do that.

After that, your derived DB model can again be generated to final code, for example SQL DDL scripts in a given certain dialect. This is generally called a M2T transformation (Model to Text) and again Essential provides tools to describe the transformations.

Basically, to complete all this stuff, we need to introduce two more specialized languages: one for applying transformations, and another one for templates to be converted to final text.

I will focus on templates side on the next post.

It is an original web based meta-model and model editor.

Take a look on it, it makes worthy!

The first one is related to Enterprise Architecture. The second one is a tool for applying MDD. Both projects are not related in any way. But it’s curious how we collide choosing the same name.

I don’t want to confuse people and fortunately the name is not the key part of our work. So, probably, a rename for our side (as the newcomers) would be gentle an a good choice. I’m open to suggestions for a new name.

In my previous post, I introduced Essential, a custom-developed tool for doing Model Driven Development.

As promised, in this post I will describe the primitives of the first language of the tool: Essential.Meta. This meta-language is helpful to describe the structural aspects of a metamodel.

The input requirements for the language are:

1. Usable for meta-structure description
2. Human readable/maintainable
3. Concise as possible
4. Easy to understand
5. Object oriented
6. Technology agnostic
7. Textual
8. Extensible
9. Scalable
10. Reusable definitions

Ok, that was the initial wish list. Now, let’s review how we can cope with it and show a short example of how it looks like:

namespace Meta.TownModel
{
    enum Sex  {  Male, Female }

    class Town
    {
        string Name;
        List<Person> Habitants;
        composition List<House> Houses;
    }

    class Person
    {
        string Name;
        string? MiddleName;
        string Surname;
        Sex Sex;
        int? Age;
        List<Person> Children;
        House Home;
    }

    class House
    {
        string Address;
        composition List<Room>0..* Rooms opposite 1..1 House;
    }

    class Room
    {
        string Name;
        decimal? Dimensions;   //In square meters
    }
}

The sample describes a meta-model for describing a town, with its buildings (houses) and inhabitants (persons). Nothing complex, nothing weird if assuming I am using a C/C++/Java/C# syntax and this doesn’t suppose a problem for you. I selected a C-like syntax instead a Pascal or VB flavour due to its less verbosity and a probably biased to what we used get to.

The namespace defines a scope for naming. All of our definitions are enclosed in a namespace. Therefore names of concepts should not collide inside a namespace. Same names are allowed in different namespaces.

As you can see, the main concepts in this language are classes and enumerations.

Enumerations define closed sets of named values helpful to describe domains. Classes on the other hand, have the classical meaning and properties or attributes can be defined for each class.

As usual in object languages each property has a type that could be primitive, an enum, a class type or a composed type.

The primitive data types supported are: string, bool, int, long, decimal, char, date, time & datetime. That should be more than enough, for the moment.

A note to readers: data types here are platform neutral, they are not java string for example, neither a C# string. It was horrible for me to see in a lot of diagramming UML tools whose names I will not cite, early mapping types to implementation languages like java:string and similar things… I sure you understand me. Such feature could be nice if you are doing reverse engineering or documenting code (a bit late BTW). But definitely horrible to see when doing conceptual modelling and haven’t chosen yet the final implementation platform.

Another relevant feature of the language is the cardinality operators. Each property has a cardinality attached to it. Following a Convention over Configuration approach, simple properties like string Name; has an implicit cardinality 1..1. You can also write to make it explicit in the language: string 1..1 Name;

List of things, on the contrary has an implicit cardinality of 0..* (zero to many). However you can further constraint it to 1..* (compulsory multi-valued) or arbitrarily to any pair of minimal and maximal values List<Holes> 18..24 GolfCampHoles;

The next feature to review is the composition keyword. A composition indicates that the property can’t exist alone, without its container. If the container is destroyed, the composed children will disappear also. Containment should be unique for each object; it cannot be contained in two direct containers at the same time. However, transitive containment it’s allowed.

There are some times, when we will be interested not only in one side of the relation between two classes, but also in the others’ end. That’s the case of the Children property. How about reaching the parents? To express it, the language introduces the opposite keyword. In our small sample it can be fully expressed (cardinalities included) as:

List<Person>0..* Children opposite 0:2 Parents;

Make sense, isn’t it? A person can have 0 or many children and let’s say zero, one or two well known parents.

I am still considering if the keyword List<T> could be also removed from the language. One can argue that cardinalities can be more than enough to express the same. However, I am still considering including other composed ADTs (abstract data types) in the future like Set<T> or Stack<T> to cite some of them. So, doors are still open to reconsider it.

Another feature we wanted to add is extensibility: If you are using this metamodel but needs to extend it for your own needs, the language is prepared to allow it. Just add another definition like this in another file and you got it:

namespace Meta.TownModel
{
    class Town
    {
        Person Mayor;
    }
    class Person
    {
        List<Person>0..* Friends opposite 0..* Friends;
    }
    class City : Town   //<--Inheritance sample
    {
        //extra city properties…
    }
}

Now, your town must have a mayor and persons can have friends!

The language allows partial definitions in the same or different files. If the namespace and class name matches, the properties of the Town class and Person class are extended by merging partial definitions.

From the cardinality point of view, Friends is a curious symmetric relation. It is a many-to-many relation but both ends  (the roles) are normally called Friends and not: MyFriends and OthersGuysConsideringMeAFriend but how knows! }:)

As briefly seen, this language is useful to impose rules over data. It constrains the objects/concepts we can describe and the allowed properties.

Intended usage

OK. Some of you are probably thinking: and what in hell it this useful for?

The mail goal is to use the language to describe the domain in which we are planning to apply MDD. With this description, we will describe the concepts, properties and relations of the problem domain we are interested in.

If you are really, really pragmatic, and the analysis and design of the domain for the joy of it doesn’t satisfy you, well, consider then that we can derive (manually or better 100% generated if you prefer) some other interesting and more earthly artefacts:

1. UML class models (structure), XMI, etc.
2. XML Schema (XSD),
3. SQL Schema (relational tables to persist the data),
4. Classes in Java (POJOs), C# (POCOs), or any other language implementing a pure Domain Model
5. XML de/serialization code to read/save XML documents been conformant with (2)
6. Data Access code (DAO) to connect (3) with (4).
7. Maps to your favorite ORM tool to connect again (3) & (4)
8. Etcetera, see Domain Driven Design and other approaches.

That’s all for today! This was the first DSL implemented in Essential targeting metamodeling. For more details, a full reference of the Essential.Meta language is described here.

On the next post, we will talk about a second DSL in Essential: the model (object level) language used to instantiate the concepts we just have created.

Thanks for reading! And please share your thoughts about it!

A title like this could sound utopian and probably it is in the extreme, but at the same time provides two advantages:

1. a suggestive title brings more blogs readers , and
2. these kind of Quests keep me active developing and testing my ideas about MDD.

Sorry for 1, the joke. But Let me explain 2 a little further if you are interested:

Some months ago, I started to develop a new tool for doing Model Driven Development. I always start new developments with contained expectatives. I consider most of them throw away propotypes just to test a new way of doing the same thing, but luckily, with less effort and better productivity. But when you found the way, you follow it again whenever you need it, isn’t it?

So, this time is one of those moments: I am quite happy with the results, and I will continue building on the top of it to make it grow.

Today (almost?) everyone agrees on code generation can make your life notably easier as long as you have:

1. a stable domain,
2. a clear knowledge of your domain (domain expert) and,
3. tools: modelling editors, model checkers, and code generator adapted to your domain and your chosen target architecture and language.

The trickiest one to get is, not surprisingly, the third one. And this issue: the quality and the applicability of the MDD tools is the main stopper when considering applying MDD for a software development project.

So, my motivation during years is not only to create code generators one more time, but do it in a way that it will be cheaper to obtain results in next projects. Therefore, reducing the requirements for the entry level will help the MDD adoption to gain speed.

Ok, wait a minute, are you reinventing the wheel again? Building another meta-code-generator?

Yes I know, there are very good and mature tools for doing MDD: such as EMF/GMF, XText, ATL, TextUml, Metacase, MS DSL Tools, etc.

However, guided by my intuition and the experiences in the domains I have worked in, I have a strong opinion on of how some things should be done and for the moment I didn’t found the perfect tool to satisfy my needs.

But, instead of complain, I decided to take to the action and add my two cents implementing my view about MDD tool support: taking the good ideas of the standards available and reinventing the parts that don’t fit.

The tool

After this introduction, the name of the tool is not going to surprise you. It is named Essential.

The goals of the project are following ones:

• to declaratively describe metamodels, models, templates, and transformations using textual DSLs
• to provide a comfortable editor for each of these four pillars,
• to provide model checkers to assure the integrity of the four, and
• to build code generators and transformation interpreters to achieve the output we are looking for.

In the next posts I will depict the DSLs used in Essential and some architectural choices. This will help us to discuss the essence of the problems found in MDD.

I want to thanks Javier Hernandez for his constant help, good discussions and counterpoints about choosing design alternatives for Essential. And also to Nicolas Cornaglia and Ángel Marín for their active beta testing as early adopters and providing good test fields.

I want to share with you a set of videos to show what I understand when talking about Forward Engineering applied to MDD.

First of all, a legal disclaimer: my apologies for the quality of the videos and for my rusty English: I am starting to play with video editing tools and recording software so I expect to improve my recording and editing skills on the way. Anyway, I found (I hope) they have enough quality to explain the main ideas. So seeing it is totally up to you! You have been warned! 

Also note: see the videos in High Quality mode (HQ) in Youtube. Otherwise, details of the samples probably will not be visible.

In a previous post I introduced a sample of the code that can be generated from a very basic conceptual model. I have created three videos to show you the main steps involved.

1. Modeling (Video 1/3). The first video uses a minimalistic class model created inside Visual Studio 2005 with Microsoft DSL Tools. The sample creates a basic blog structure in less than 5 minutes. Note that in the specification there are no technological choices (neither the types are bind to a concrete language representation).
2. Code Generation (Video 2/3). A quick step: Selecting a code generation (selecting a target architecture), fixing the design choices offered by the code generation and pressing the red button: Generate! A full .NET Solution is generated in less than 5 seconds ready to compile.
3. A quick code review of the generated code (Video 3/3). Finally, I am sure you have curiosity to take a look to the output code, don’t you? This third video shows a walkthrough to show:
   • DB Scripts (table creation, foreign keys, drop scripts)
   • Database creation
   • Logic layer: POCOs (Plain Old CLR objects), NHibernate mappings and a Business Service Layer with fully functional CRUD operations.

So this is it. It is a proof of concept of how fast and direct MDD tools can be starting from a minimalistic model.

When talking about using or buying modeling & code generation products my advice is:

• Don’t use models just for documentation. They will be outdated soon or later. On the contrary, a living (generating) model is always in sync with its target application.
• Don’t resign yourself to just using code generation of skeletons. As you just have seen the current technology allows you to generate much more.
• Don’t be content if anyone try to sell you a model too close or tied in any way to a given target language. Today we have just generated C#, but tomorrow may be we prefer Ruby? Python?
• Don’t resign yourself to use a tool married with a specific database. You know, technology changes faster that we usually expect.

A good way to avoid such confusion is to make explicit a short clear definition of the terms used.

Following this idea, today I want to share the Conceptual Map I traditionally use to explain the areas involved in code generation. This map helps me to explain how more complex interactions and natural dependences and properties arise in this domain. See embedded video.

We will start considering MDD using two orthogonal dimensions:

• Types/Classes vs Instances &
• Two levels of abstraction: Programs/code (technology dependent, concrete)  vs Models (technology independent, abstract).

Key concepts

1. Model: A description of the problem to be solved.
2. Metamodel: A type description for providing the modeling language.
3. Template: A pattern representation of the code you want to create.
4. Transformation: Application of an algorithm to transform one or more models in one o more outputs.
5. Output: Your target asset: source code, doc or models again.

Additional concepts

• Mappings = Template to Metamodel item bindings
• Architecture Know How = Mappings + Templates (embedding best practices and patterns)
• Business Know How  = Metamodel + Model
• Model Checker = Validation of Model with respect to Metamodel
• Generator = Transformation(Templates, Mappings, Model) –> Output

A pair of properties to mention.

Stability:

1. Metamodel: Quite stable once developed.
2. Model: Stable till the business change (requirements change).
3. Template: Stable till not changing or evolving the technology (2-3 years cycle?).
4. Transformation: Impacted by technology or metamodel changes.
5. Output (Source code): Regenerated as needed. Discardable.

Dependences:

1. Metamodel: Not dependent of third parties.
2. Model: Altered if metamodel changes.
3. Template: Not dependent of third parties (just the technology).
4. Transformation: Impacted by template changes, mapping changes and  metamodel changes.
5. Output (Source code): Impacted by everything. Regeneration is a good property as seen before. But now is a necessity.

Changes in the formers will cascade to the latter.

Strong Separation of Concerns & skills

Apart of the classical advantages provided by MDD approaches such as quality, productivity, time to market, & technology independence; one of the good properties no so times recalled is the strong separation of skills that this method of software engineering provides:

• Business analyst can concentrate efforts in using modeling editors to describe the business and provide feedback to enhance metamodels (when needed)
• On the other side, Software Architects and technology gurus can concentrate in tuning the technical architecture to achieve the most performing implementations they can provide.

One you achieve this stage, you can start thinking of reusing and evolving your business models and architectures at speeds that most of traditional SW developers never seen before.

There is one famous quote from Grady Booch saying:

“The entire history of software engineering is one of rising levels of abstraction (abstraction is the primary way we as humans deal with complexity).”  Grady Booch

As many others, I think that MDD is the next step to adopt in such direction. When it will happen globally is just a question of achieving the critical mass:

• having good enough tools,
• and having practitioners enough and success cases to prove the value in the “not so new” way of working.

I’m optimistic about it. We are in the way and getting speed…

In the economic crisis scenario, some voices argues about the death of SOA (more, more and more) just even before few organizations has already started to adopt it.

However, the SOA approach has valuable principles for Enterprise Software organization for the long term maintenance, integration and minimizing the TCO.

Johan den Haan has a nice post about it: SOA is dead; long live Model-Driven SOA.

As a supporter of MDD, I agree with Cesar and Johan, MD SOA could be a way to deliver the good principles behing SOA.

SOA is always a tough topic to explain for first time visitors. To help to introduce it I will link a superb presentation about it: Meet mike… (credits for Eduard Hildebrandt).

MDD with Oslo

On the other hand, as Cesar pointed, Microsoft Oslo project has/had (after the rebranding) potential to be a good platform for domain modelling with textual DSL (MGrammar) and visual DSL (Quadrant).

As commented in the previous posts, it’s a pity that for the moment modeling efforts at Microsoft has been focused too much into one particular tree “the SQL Server scope” and loose the full forest vision.

Via Jorge Ubeda I received the bad news. Accordingly to Douglas Purdy’s post, Microsoft OSLO is going to be rebranded to SQL Server Modeling CTP.

The people in the MDD community using .NET technologies has put a strong believe in Microsoft promise of delivering a big step forward in MDD with the OSLO project. At least, that what Microsoft sell us in the first CTP.

Now, in its new reincarnation: the M language, Quadrant and the Repository will still provide value for the database colleagues, for sure. But for the MDD community, this time, Microsoft has lost totally the point. That is quite different respect to the initial selling proposition we all bought at www.modelsremixed.com for example!

Model Driven Development is about increasing the level of abstraction, it is about technology and architecture independence. Tying the models with a database provides no help in achieving such objectives.

And what about MGrammar? It is a great tool for doing textual DSLs in the .NET environment. Any plans to support and improve it as a product or will end also tied to a database?

As Jorge points JJ. Dubray seems to be in the right path when anticipating the results.

It is a pity, a great opportunity to empower MDD with the right tools has been lost. Anyway, others tools will come and do the job instead.

The call for speakers of Code Generation 2010 is now open.

If you are a practitioner, researcher or a tool maker in the MDD arena this is your conference.
You have till January 15th 2010 to send your submission.

See you there!