Leverage a Polyglot data model
In the previous tutorial, we reviewed how to create a model for REST APIs using Swagger or OpenAPI specification. By the end of this tutorial, you will master how to perform data modeling for polyglot storage and transmission, using our Polyglot Data Modeling capabilities.
A polyglot data model is a common physical model, where denormalization and complex data types are encouraged. While some people like to compare our polyglot model to a logical model because both are technology-agnostic, you should read this article where we emphasize the differences. We strongly advise against creating normalized logical models when using the polyglot functionality. You should instead use this functionality to create denormalized models with complex data types and, if needed, rely on the built-in functionality that will automatically normalize nested objects in RDBMS targets.
With a polyglot data model, you can define data structures once, then easily instantiate them in data models for any of the technologies Hackolade supports:
The polyglot and target models are separate models. But they are related to ensure consistency. In other words, the behavior is similar to references to external definitions, but at the model level. The polyglot data model is the master, while target models are derived, and remain dependent on the polyglot model. If changes are required to an object, they must be done in the polyglot model, then the target models inherit these changes.
Exceptions are possible, as described further down.
Create a polyglot data model
A polyglot data model is just another Hackolade data model. You can create entities with attributes and relationships like you would with a specific Hackolade plugin. It has a more generic terminology and a superset of all data types found in the different technology targets.
To create a new polyglot data model, simply choose Polyglot as your target, and follow the instructions in this tutorial.
Alternatively, if you have an existing target data model, you can convert it to a polyglot model so the polyglot model becomes the "master" from which other target models can be derived.
Derive a target data model
Important: from a polyglot model, you cannot directly forward-engineer DDLs, scripts, schemas, etc for technology targets. You must first create a target data model for the technology of your choice, then pull (or derive) from the polyglot model. Then you're able to forward-engineer the artifacts from the target model.
Once you're ready to create a physical data model for a given technology, the process starts with the creation of a new model for the target of your choice. Then, from the target model, you derive your physical model from the polyglot data model by selecting the menu Tools > Polyglot > Derive from Polyglot Model:
You may then choose the polyglot data model file:
You are then presented with a dialog where you can narrow down the selection of entities to reference in the target model. By default, all objects are selected. Using multi-select (ctrl-click or shift+click) , you may change the selection of containers, entities, and definitions to derive in the target model.
Attributes of entities cannot be individually deselected in the above screen. But 2 options are available, depending on the use case:
1) in the polyglot model, each object, including attributes can be marked as "Polyglot only", meaning that they will not be included in any target models.
2) in a target model you can deselect the IsActivated property, which will cause the attribute to remain in the model, but be filtered out (or commented if the target syntax allows it) in the forward-engineering script.
If your target is a relational database, the process to derive from polyglot will automatically convert complex structures into normalized tables with foreign key relationships.
Adjust the target model to the specific needs of the target
You may supplement the target model with metadata, index and partitioning information specific to the target technology, then forward-engineer artifacts according to your use case.
Once a target model has been created and derived from a polyglot model, it contains all the derived objects. Several adjustments to the target model are possible:
- changes the properties of entities and attributes without changing them in the polyglot model. This is used if you need minor deviations;
- add new entities and views that are specific to the physical target model, for example based on access patterns;
- remove entities from the target model. This action does not delete the entity in the polyglot model.
You may also join entities from multiple polyglot models by repeating the operation to derive from polyglot.
Update target model with changes in polyglot model
When you make changes to a polyglot model and save it, these changes can be reflected in the target models derived from that polyglot model. If the target model is already open, you may refresh the derived objects by clicking the update button which can be found in the model-level Properties Pane:
If new objects have appeared in the polyglot model (or there are objects in the polyglot model that were not selected in an earlier step), a dialog appears to allow their selection so references could be added to the target model.
When opening a target model with a reference to one or more polyglot models, you are presented with the option to update the objects.
You may also permanently break the connection to the polyglot model. This operation should be done with care of course, as it cannot be re-established afterwards without restarting all the steps from scratch. The operation replaces all references to the objects of the polyglot model by the objects themselves. But further modifications in the polyglot model will no longer have any effect in the target model.
Note: when converting a target model to polyglot, links are not currently created for the original target model to become dependent on the polyglot. A new model would have to be derived from the polyglot. Example: you have a MongoDB model, and you want to convert to polyglot. First you do the conversion to polyglot, then you create a new MongoDB model and derive it from polyglot.
Tip: if you need to convert a data model from a given technology to another one (example: from MongoDB to Avro, or Parquet, etc.), it is 2-step process: first you convert your original model to polyglot, then you create a new model in the other technology plugin and you derive from the polyglot model. You can optionally break the link with the polyglot model.
In this tutorial, we reviewed how to perform data modeling for polyglot storage and transmission, using our Polyglot Data Modeling capabilities. This concludes for now our tutorial series. Next, we will develop a series of How-To Guides, goal-oriented recipes taking you through the steps to solve real-world situations.