GitHub repo setup

To enable service-to-service communication, GitHub has what it calls GitHub apps. These apps can be used to access files on repositories using the GitHub API without the need for a service account. 

 

The repository administrator of an organization can create a GitHub app and install it within the organization on GitHub. The app only requires read-only access permission to the content.  When installing it, the administrator can select the repository or repositories to which this app is allowed access.

 

The app also supports webhooks to provide notifications to the Model Hub instance when certain events occur on GitHub.  In our case, these events will occur when new models, or new versions of existing models, are pushed to the main branch of each of the linked repositories.  

 

We use a GitHub app manifest to simplify the user experience and pre-fill the app creation form with the appropriate parameters to create the endpoint and secret.  Once the GitHub app is created, we store in the OCI Vault: the client ID, the private key, and the webhook secret for the model sync to work.  Since this is sensitive information, it must be treated with care, meaning no logging and no way for the user to retrieve it once it is saved.

 

 

Initiate the GitHub App creation from the Model Hub

Go to the the Git Providers tab of the admin panel, then click on the button + setup Git Provider

 

 

Enter the name of your GitHub organization and click the button Register GitHub app.  

 

Allow and configure the creation of the GitHub App in your organization

You may need to authenticate with GitHub.

 

 

 

The process may take up to one minute or so.  When our automatic process to generate the app is completed, the application returns this screen:

 

 

 

You must now click on the Intall button so that the creation can take effect on GitHub:

 

 

You must select the repository or repositories for which you want to authorize the app to provide models to the Model Hub:

 

 

You may edit the repository list at a later time if you want to change which ones should be replicated to the Model Hub:

 

 

Create a GitHub webhook

A GitHub webhook is a mechanism that sends HTTP requests with event data to a specified external URL whenever specific events, such as code pushes or pull requests, occur in a repository or organization.

 

This is how the Hackolade Model Hub gets notified of the creation of a new model or change to an existing model in any folder of the default branch for your repository, or repositories.

 

To create a GitHub, go to the Settings menu of your repository, then selec Webhooks in the left menu, then click on the button Add webhook.

 

 

Fill the form

• Payload URL: /gateway/sync endpoint of your HUB domain (eg: https://<yourcompany-prod>.hackolade.com/gateway/sync)
• Content type: application/json 
• Secret: generate a secret using a password generator.  Keep this password in a safe location. 
• SSL verification: keep the default value
• Which events would you like to trigger this webhook?:  Just the push event
• Active: yes

 

Then click on Add webhook.

 

 

 

 

The replication of models into the Model Hub database should start automatically.  Depending on the size of your model repository (or repositories), the initial process may take some time.

 

 

Troubleshooting

Go to the Repositories tab of the admin panel to view the sync status of your repository of repositories.

 

If you think that your Model Hub database is not in sync with a repository, it is possible to manually start one of 3 separate actions

 

If you have reasons to believe that the database is not receiving models from your GitHub repository, you can view on GitHub whether the webhook is delivering as expected.

 

There is an easy way to access the GitHub screen.  Go to the Repositories tab of the admin panel, then click on the Edit button of the GitHub box:

 

then click the Advanced button.  

 

You may also manually access the same screen by going to GitHub > your organization > Developer settings > GitHub app > select the app created above > go to Advanced.

 

 

 

Configure an additional repo for replication to the Model Hub

It is possible that you might need to add one or more repositories to the existing list of repos already being replicated to the Model Hub.  This action must take place on GitHub.

 

Go to GitHub > your organization > GitHub apps > for the app created above > click the Configure button.  Or from the Model Hub admin panel > Repositories, click on the Edit button of the GitHub repository provider, then click on the Advanced button.  In both cases, you reach a page where you can click the button Select repositories to configure your choice.