Documentation

gitHub

Re-usable objects definitions

Note: You may wish to view the how-to video on this subject.

 

To facilitate the work of data modelers, Hackolade has introduced a handy feature: the ability to create once, object definitions that can be re-used in multiple places.  A library of definitions standardizes content and insures consistency.  This dictionary also simplifies the work of data modelers so maintenance can be performed in one place and be automatically propagated to all places where the definition is referenced.

 

A good simple example of a re-usable definition is an address.  This type of object may be used in many places inside a model, and in different models too.  Even within one single entity, you may have, for example a billing address and a shipping address.

 

Definitions can be maintained at 3 distinct levels: at the entity-level (called internal definitions), at the model-level, and external.  In the ERD view and hierarchical schema view, the references are are noted:

- (i) for internal

- (m) for model

- (e) for external

 

Note: Definitions are not carried over to many of the NoSQL databases.  They are only used in physical modeling in Hackolade.  To make things easy for the data modeler, Object Browser, ERD, entity hierarchical schema views, and documentation all show the referenced attributes as if they had been defined directly in the collection(s).

 

It is easy to create a re-usable definition from scratch, then reference it in a collection.  Or by selecting an attribute in an existing collection, and converting it to a re-usable definition, so it can be referenced elsewhere.

 

Internal definitions

Internal definitions have limited use because they can only be reused (or referenced) within the same entity (a collection or table). But internal definitions insure JSON Schema compatibility.  

 

Model definitions

Model definitions can be referenced in any entity of the same Hackolade model.  

 

If a definition is created at the model level, it can be referenced in any entity of the same model, whereas it will be limited to its own entity if it is created as an internal definition.  

 

With some RDMBS as well as with Cassandra and Hive for example, Hackolade's model definitions correspond to the concept of User-Defined Types or UDTs.

 

External definitions

External definitions are Hackolade models (or JSON Schema files) that can be referenced in whole or in part, by other models. 

 

Any entity (collection, table, ...) - or attribute herein - within a Hackolade can be an external definition for, and be referenced by, another Hackolade model.  In other words, it does not have to be a definition within that model.

 

External models can be used to create a company-wide library of re-usable objects.  You may ask yourself whether it is best to have one single model hold all of your company's re-usable definitions, or to have one model per object, or various degrees in-between.  There are pros and cons to each approach.   We would advise against putting all objects in a single model.  One model per domain, or even one model per sub-domain would make more sense from these perspectives:

- maintenance responsibilities (maybe a department does not want its definitions be altered by another department)

- ease of navigation (directory names and model names should be descriptive so modelers don't have to scan tons of objects to find the ones that they're interested in)

- performance (not a huge factor, but smaller models are easier to navigate than bigger ones.)

Depending on size, you may even use sub-directories to help manage the library of definition models.  But do remember that, if you move referenced models around, you will break the path in the referencing models.

 

In order for changes to a referenced external model to be activated in a referencing model, the latter needs to be opened in the application.   Alternatively, you may manually refresh a reference in an open referencing model

Definitions reference refresh

 

If you have defined a separate model with the structure of an address, you may reference it in any other Hackolade model:

Definitions - contextual menu referencing ext

 

next, you're prompted to choose the reference model, either from your file system or a URL:

 

Defintions - Choose external dialog

 

You then can choose an individual attribute, or an entire structure:

Definitions external choose

 

 

 

Note: it is not intended to combine definition levels.  In other words, if you have a Hackolade model with attribute structures intended to be re-used in other models, you should define these as simple attributes, not as internal or model definitions.

 

External references are produced using the $ref implementation described here and RFC 3986.

 

Initially create a definition

A definition can be created from scratch, just like any collection, either by selecting the Internal Definitions lower tab at the collection level:

 

Definitions - Lower collection tabs

 

or by selecting the Model Definitions lower tab at the model ERD level:

Definitions - Lower model tabs

 

 

In our address example, the definition may look like this:

Definitions - Address example

 

Once the definition is created, it can be referenced in a collection, via the contextual menu:

 

Definitions - Contextual menu

As a result, a reference is created in the collection.  The application displays all the elements of the definition, as if they were in the collection, but only a reference to the definition is actually stored in the collection:

 

Definitions - billing and shipping references

The reference is marked with the middle line in the hierarchical schema box, showing the name of the referenced definition, followed by either (m) for model definitions, or (i) for internal definitions.

 

At any point, it is possible, if desired, to replace a reference by the object definition structure: 

Definitions - replace by attributes

 

Convert an existing attribute into a definition

If you want to make an attribute available for use elsewhere in the same collection, or in another collection, you can convert it with just a few clicks:

Definitions - convert reference to def

This will create the internal or model definition, and replace the attribute (and its children if any) by a reference to the definition.

 

Where-Used

For internal and model definitions, it may be useful to find all occurrences of definition references within a model.  This where-used function (aka impact analysis, aka lineage) is available from the contextual menu (or the Ctrl+F11 shortcut key):

Definition Where-Used contextual menu

 

 

Choosing this option displays a dialog displaying all instances of references to the selected definition, and the path to those instances.

 

Definition where-used dialog

You may select an instance and go to the hierarchical schema view of that reference.   Lineage is enabled from the Object Browser and definition tab (model and internal.) 

 

Forward-engineering "referenced" vs "resolved" definitions

When displaying the JSON Schema preview or forward-engineering JSON Schema, the user may choose between the standard "Referenced definitions"

 JSON Schema preview Referenced definitions

 

 

JSON Schema FE Referenced definitions

 

 

or "Resolved definitions"

JSON Schema preview Resolved definitions

 

JSON Schema forward-engineering Resolved definitions

 

or "Internal".  This option converts to an internal definition, the references to model definition and external definitions:

JSON Schema preview Internalized definitions

 

JSON Schema forward-engineering Internalized definitions