Add reusable definitions

The purpose of reusable definitions is to increase productivity/convenience and consistency to achieve higher data quality and governance.  It is a handy feature of Hackolade Studio to facilitate the work of data modelers: it provides the ability to create and maintain object definitions that can be re-used in multiple places. 

 

Watch this short video to learn how to use Hackolade Studio and leverage reusable definitions.  

 

A simple example of a re-usable definition is an address, but many other domain concepts are candidates for this feature.  A definition object may be used in many places inside a single data model, and in different models too.  Definitions are created once, then referenced in many different places.  In case the structure or properties of the object evolve, you only need to make the change in one place, and all the places where the object is referenced will evolve accordingly.

 

Hackolade Studio helps breaking down large enterprise models into smaller reusable models from which pieces can be referenced elsewhere:

 

As these reusable objects are maintained in one place, it is easier to ensure consistency, quality, and governance.  Changes and evolutions can be done in a single place, and propagated throughout all the models referencing the definition.  This propagation is either done when opening a referencing model or by running a Command-Line Interface update command.

 

 

Definitions can be maintained at 3 distinct levels: at the entity-level (called internal definitions), at the model-level, and external.  Internal definitions may have limited use because they can only be reused (or referenced) within the same entity (a collection or table). But internal definitions ensure JSON Schema compatibility.  Model definitions, on the other hand, can be referenced in any entity of the same Hackolade data model.  Finally, external definitions are external files (Hackolade data models or JSON Schema) that can be referenced, in whole or in part, by any other Hackolade Studio data models.

 

In the ERD view and hierarchical schema view, the references are are noted:

- (i) for internal

- (m) for model

- (e) for external

 

Note: in some targets such as RDBMS, "model definitions" are also known as "user-defined types" or UDTs.  In Swagger, they are called "definitions" while in OpenAPI, they're called "components".  Their behaviors throughout the application are identical.

 

There are 2 simple ways to create a definition.  Either it is created ahead of time, then referenced in different places.  Or it is created from an existing instance of an object.  Let's illustrate the second method.  

 

Convert an object into a model definition

Let's create a simple JSON document structure with a sub-object for the address.  

 

Let’s convert this object into a model definition by doing a right-click on the “address” object.  In the contextual menu, choose Reference > Convert to Definition > Model.   This action is not available through the menus or the toolbar.  It is only possible via the contextual menus, in the Object Browser, the ERD or the hierarchical schema tree view of each entity tab.

 

Two things happened:

1) in the ERD and also in the schema tree view, the attribute is replaced by a reference to the definition:

 

2) and in the Model Definitions tab, there is now a definition.:

 

This is where definitions can be maintained.  Any change made here is automatically reflected in all places referencing this definition.  It is also possible to maintain many additional definitions here. 

 

 

Reference an existing definition

This action is not available through the menus or the toolbar.  It is only possible via the contextual menus, in the Object Browser, the ERD or the hierarchical schema tree view of each entity tab.

 

Right-click on an attribute of the entity where you want to add the reference.  In the contextual menu, choose Append Attribute > Reference > Model:.  

 

 

 

In the dialog, search and select the object or attribute of your choice:

then click the Apply button.

 

 

Maintain definitions

Go to the Model Definitions tab change the zip code to a string field, and add a country attribute.

 

We can see that the changes in the model definition are immediately reflected wherever referenced:

 

Note: you may assign to references a different name than in its master definition. You may also enter a reference description to supplement the description in the master definition.

 

JSON Schema preview

To see the JSON Schema representation of the person structure, let's open the entity in a separate tab, and got to the JSON/YAML Preview tab..  We can see that the address object does not contain the structure -- it is only a pointer to the model definition, using JSON Schema's $ref syntax.

 

 

There are 2 other interesting settings for definitions: Resolved and Internal:

 

If you choose Resolved, the $ref pointer is replaced by the actual structure of the definition:

 

Whereas, if you choose Internal, the model definition is converted into an internal definition for the entity, according to the JSON Schema standard:

 

 

Deviate from a definition

There is sometimes a need for the reference to have different attributes and properties than its "master" definition.  But a reference to a definition is just a pointer to its definition.  As a result, it inherits ALL the content of the master definition, including all its attributes and their properties, with no possible deviation.  This is NOT a restriction of the software.  It is simple logic and is also dictated by the way references to definitions are implemented in general, and in JSON Schema in particular.  

 

There are 2 possible workarounds:
1) create another definition with a different schema and/or properties to match the new needs;
2) or disconnect from the original definition through the functionality "Replace by attributes".  You can start with a reference to the definition, then break the reference and create your deviation.  Of course, there’s a catch: if the schema or properties of the definition change, these changes will only be reflected where the definition has been referenced, i.e. not in places where you have broken the reference.  You cannot eat your cake and eat it too… You cannot break from the master definition, then still expect that, if the definition changes, this attribute will benefit from the changes.

 

Right-click on the reference either in the ERD or in the hierarchical schema view, then choose Reference > Replace by Attributes. 

 

 

 

Display Where-Used

If your definitions are used in multiple places, you may find useful to see where.  You can click on a definition in the Object Browser and also in the Model Definitions or Internal definitions tabs, then select Where-Used:

 

 

A dialog box will let you see the different references to the selected definition.  You can select one and click the Go to button, and the Object Browser will take you to that object.

 

External definitions

External definitions are external files (any Hackolade data model or JSON Schema, plus SwaggerHub openAPI specs in Swagger/OpenAPI target models) that can be referenced, in whole or in part, by any other Hackolade Studio data models.

 

In the contextual menu, choose Append Attribute > Reference > External.   This action is not available through the menus or the toolbar.  It is only possible via the contextual menus, in the Object Browser, the ERD or the hierarchical schema tree view of each entity tab.

 

 

 

Important: when referencing external files, it is important to realize that data model files can sometimes move around and be shared between different users.  For example, a library of definitions may be stored in a shared folder, in which case you may want to use an absolute path to the file.  Whereas if data models and their definitions are in the same repository (even if in different folders of that repo), it would be preferable to use a relative path to the definition file.

 

References can also be made to files accessible at a URL.

 

 

 

 

You are then presented with a dialog to choose one or more objects to reference:

 

Warning: you should avoid doing recursive external definitions.  In other words, don't create from a given data model an external reference to an object in the same model file.  Also, you must avoid circular references.

 

When you open a data model with references to external files, you are prompted to choose whether to update the definitions or not:

 

 

While in a model with external references, you may at any time update the path to the external file (ellipsis button), or refresh the references in case the external file changed (refresh button.)

 

 

 

Important Note: Definitions are a Hackolade concept that is not necessarily carried over to the target technology, unless specifically supported (such as UDTs for Cassandra, Hive, RDBMS...)  If definitions are not supported, they are resolved during the forward-engineering process.