Polyglot Data Model

Starting with version 5.2.0 of Hackolade Studio, users can now define structures once in a technology-agnostic polyglot data model, complete with denormalization and complex data types, then represent these structures in a variety of physical data models respecting the specific aspects of each target technology.

 

You may want to first read about the background for our polyglot data model approach, and as well as the definition of the polyglot data modeling concept.

 

The polyglot and target models are separate models.  But they are related to ensure consistency.  In other words, the behavior is similar to references to external definitions, but at the model level.  The polyglot data model is the master, and targets models are dependent on the polyglot model.  If changes are required to an object, they must be done in the polyglot model, then the target models must be updated to reflect the changes.

 

Exceptions are possible: some objects in a polyglot model can be flagged as polyglot-only so they do not appear in the target models.  Conversely, you can deactivate entities and attributes in the target model, or you can add objects in the target models that are not reflected in the polyglot model.  What is important, mostly, is that, if attributes or properties of an object change in the polyglot model, you may update dependent target models to reflect those changes.

 

Push and Pull from the target model

Many customers having accumulated physical data models in Hackolade prior to the introduction of the new functionality will want to convert their target model into a polyglot model.  This is done by opening the physical target model, selecting the menu Tools > Polyglot > Convert to Polyglot Model...

 

You must choose whether to make your target dependent of the Polyglot model being created.  This is the recommended approach as the Polyglot model should now become the master of changes, and the target model should be dependent of the Polyglot model.

 

 

You may then choose the location where you want to store the polyglot model.  You're now ready to maintain and update the polyglot data model.

 

You may also create a polyglot data model from scratch and build it, as you would with physical models.  Except that a polyglot data model cannot be forward-engineer to a database instance.

 

Once you're ready to create a physical data model for a given target, the process is that a new model must be created for the target of your choice.  then, from the target model, you derive your physical model from the polyglot data model by selecting the menu Tools > Polyglot > Derive from Polyglot Model:

 

 

You may then choose the polyglot data model file:

 

You are then presented with a dialog where you can narrow down the selection of entities to reference in the target model.  By default, all objects are selected.  Using multi-select (ctrl-click or shift+click) , you may change the selection of containers, entities, and definitions to derive in the target model.

 

 

Attributes of entities cannot be individually deselected in the above screen. But 2 options are available, depending on the use case:

1) in the polyglot model, each object, including attributes can be marked as "Polyglot only", meaning that they will not be included in any target models.

2) in a target model you can deselect the IsActivated property, which will cause the attribute to be filtered out (or commented if the target syntax allows it) in the forward-engineering script.

 

 

Data type mapping

On the polyglot data model side, a long list of physical data types has been compiled across the different targets support by different Hackolade plugins.  It was on purpose that this approach was decided instead of providing only a list of logical data types.  That is so the most relevant level of granularity could be chosen at the polyglot level to be automatically mapped to the appropriate physical data type without minimal user intervention.

 

A data type matrix mapping table is used to provide bi-directional transformation: when converting a target model to polyglot, and when deriving a target model from polyglot.

 

Naming conventions

Polyglot models being technology-agnostic, it is normal that only business names are created and maintained here.  

Then when a physical model is derived for any target, it is often the case that the rules to generate physical names from business names are different.  For example, in MongoDB, JSON, Avro, OpenAPI, developers tend to prefer camelCase, whereas some targets like Cassandra cannot have any upper case characters and therefore snake_case is a typical conversion.

To generate technical names in physical target models when deriving from a polyglot model, you must first enable Business-to-Technical coupling for the chosen target in Tools > Options > Naming Conventions.  Target-specific transformations rules are applied during the process of deriving from the polyglot model.

 

Maintain and update the target model

Once a target model has been created and derived from a polyglot model, it contains all the derived objects.  You may make some changes to the target model:

- make changes to the properties of entities and attributes without changing them in the polyglot model.  This is used if you need minor deviations;

- add new entities and views that are specific to the physical target model, for example based on access patterns;

- remove entities from the target model.  This action does not delete the entity in the polyglot model;

- add, delete or modify attributes, again without effect on the Polyglot model.

 

You may also join entities from multiple polyglot models by repeating the operation to derive from polyglot.

 

You may supplement the target model with metadata, index and partitioning information specific to the target technology, then forward-engineer artifacts according to your use case.

 

If the polyglot model changes during the time you're editing the target model, you may refresh the derived objects by clicking the update button which can be found in the model-level Properties Pane:

If you have multiple links, possibly to the same polyglot model so you can maintain different profiles, it may be useful to indicate a meaningful name for the link.

 

An impact analysis dialog is displayed to let you decide which objects to include as part of this refresh operation.  Maybe these objects were not selected originally, or they could have been added to the polyglot model since the previous refresh or derive operation, or they could have been deleted in the target model.  

 

 

When opening a target model with a reference to one or mode polyglot models, you are presented with the option to update the objects.

 

You may also permanently break the connection to the polyglot model.  This operation should be done with care of course, as it cannot be re-established afterwards without restarting all the steps from scratch.  The operation replaces all references to the objects of the polyglot model by the objects themselves.  But further modifications in the polyglot model will no longer have any effect in the target model.