7->8 Gitlab Adapter

The Gitlab adapter will allow the transfer of various data types from Gitlab project to the SBE platform and vis-versa.

GitLab is a web-based DevOps platform that provides a complete set of tools for the software development lifecycle, including source code management (SCM), continuous integration/continuous deployment (CI/CD), issue tracking, and more. It allows teams to collaborate on code, track changes using Git version control, and automate workflows for building, testing, and deploying applications.

At its core, GitLab functions as a Git repository manager with features similar to GitHub or Bitbucket, but with the added benefit of integrated CI/CD pipelines, project management tools, and robust access controls — all within a single interface.

Interaction between the Gitlab Adapter and GitLab happens through the GitLab REST API. The adapter uses standard HTTP protocols (GET, POST, PUT, DELETE) to create, edit and remove Epics, Issues, Pipelines, Jobs and files.

SBE Adapter/Plugin Documentation Template

• SBE Adapter/Plugin Documentation Template

  • 1. Getting Started: Using the Adapter

    • 1.1 Operations Overview

    • 1.2 Accessing the Adapter

    • 1.3 Attaching & Connecting

    • 1.4 Publishing Items

    • 1.5 Refreshing Items (Including Subscribed Items)

    • 1.6 Verify

    • 1.7 Advanced Operations

    • 1.8 Troubleshooting for End Users

  • 2. Document Overview

    • 2.1 Document Overview

    • 2.2 Document Orientation

  • 3. Adapter Use Cases

    • 3.1 Adapter Overview

    • 3.2 Typical Use Cases

  • 4. Supported Versions

    • 4.1 Supported Adapter Products

    • 4.2 External Tool Versions Supported

    • 4.3 Differences Across Tool Versions

    • 4.4 Supported Plug-Ins and Add-Ons

  • 5. Digital Tool Best Practices

    • 5.1 Tool Configuration Considerations

    • 5.2 Usage Tips & Gotchas

    • 5.3 Tool Limitations and Workarounds

  • 6. Installation

    • 6.1 Installation Instructions

    • 6.2 Configuration

    • 6.3 Data Source Type Definition

  • 7. Channels and Mappings

    • 7.1 Channel Definition

    • 7.2 Approaches to Mapping

  • 8. Security and Access

    • 8.1 Authentication Methods

    • 8.2 Authorization and Roles

    • 8.3 Secure Communication

    • 8.4 Identity Integration

  • 9. Troubleshooting

    • 9.1 Known Issues

    • 9.2 Logs and Diagnostics

    • 9.3 Common Misconfigurations

    • 9.4 Escalation and Support

  • 10. Release Notes

    • 10.1 Version History

      • 8.3

      • 7.12

      • 7.11

      • 7.10

      • 7.9

      • 7.7

      • 7.6

      • 7.5

      • 7.4

      • 7.3

      • 7.1

      • 7.0

    • 10.2 Breaking Changes

  • 11. Technical Reference

    • 11.1 Adapter API Endpoints

    • 11.2 Identity

    • 11.3 Configuration File Format Reference

    • 11.4 Schema Support

    • 11.5 Glossary of Terms

    • 11.6 Compliance and Certification

Adapter & Extension Package Documentation go together

Plugins are Separate

1. Getting Started: Using the Adapter

1.1 Operations Overview

The Gitlab Adapter is a server-based SBE adapter. This allows users to interact with data from Gitlab within the Digital Thread via Standard HTTP protocols.

Attach

• Attaching is the process of connecting your Gitlab source project to the digital thread. This process is performed behind the scenes by SBE when initiating one of the operations described below (publish, refresh, etc). Necessary details of your gitlab project, specifically the address/url of your Gitlab instance (as schema, host, port and URI path) and project name/id, are provided as server and channel properties, respectively, as described in Section 1.3.

Publish

• The Publish operation moves data from a connected system into the SBE platform.

• The Gitlab Adapter publishes Epics, Issues (including Tasks), Pipelines and Jobs as SBE models on the digital thread (DT) with the various object properties mapped to corresponding model properties.

• Files, both JSON and non-JSON can be published, but must be individually specified in the server properties, as described in Section 1.3.

• Non-JSON files are mapped to SBE models with the file contents stored as a single property value, as mapped.

• JSON files (denoted by the .json extension) are disassembled, with each level of the JSON represented by a separate SBE model, each containing the properties on that level of the JSON, and “Contains” links to those models representing the sub-layers of the JSON structure, each containing their own properties.

• A Publish of Gitlab data can be initiated from the SBE web-UI by following the steps outlined in Section 1.4.

Refresh

• The Refresh operation moves data from the SBE platform to a connected system.

• The Gitlab adapter refreshes both authoritative (those originating in the connected Gitlab project) and Subscription Epics and Issues (including Tasks). Upon refresh, new Epics and Issues will be created in the attached Gitlab project for any models subscribed to the channel.

• Authoritative files (JSON and non-JSON), those originating from the attached project and specified in the server properties, can be updated via refresh from the DT. However, the Gitlab Adapter cannot currently create new files.

• JSON files are reconstructed from the associated SBE models during refresh. Any property edits with this set of models will be reflected in the updated file.

Rebase

• Like Refresh, the Rebase operation, also known as a “soft-refresh”, moves data from the digital thread to the connected system. However, in this operation, Gitlab records will be updated with the latest version of the corresponding SBE model unless the item has been edited in Gitlab since the most recent publish to the digital thread.

• This is the equivalent of merging the data with all merge conflicts being resolved in favor of the data within the connected Gitlab project.

Verify

• The Verify operation examines the data in the digital thread and the connected Gitlab project to “verify” that the data is in sync. Results, viewable in the “Detailed” (Level 3) Tracking Logs, show an item-by-item comparison of each corresponding Gitlab item/SBE model pair, indicating any discrepancies between the two in properties, relations or metadata.

Reset

• The Reset operation checks the digital thread for stale (deleted) subscriptions and removes the corresponding, previously refreshed, items from the connected Gitlab project.

1.2 Accessing the Adapter

As a thin client service, the connection between Gitlab and the digital thread is managed by SBE and all operations are performed via the SBE web interface. Once the connection is established, as described in Section 1.3, all data operations described above can be initiated and monitored within the SBE platform.

1.3 Attaching & Connecting

Establishing a connection between your Gitlab project and the digital thread can be completed using the Data Source Manager on the SBE platform, which is linked from the SBE Admin page. Once in the Data Source Manager, follow the steps below to create a Data Source Type, Data Source and Channel.

1. Data Source Type

   1. The Data Source Type (DST) is a generic designator that tells SBE the identity of the external system that a Data Source, Channel or Connection are targeting. The Gitlab adapter handles all operations on Channels tied to Gitlab, as indicated by an associated DST with name “GITLAB”.

   2. The GITLAB DST can be associated with multiple Data Sources, thus allowing connection to multiple Gitlab hosts and projects on different channels. To accommodate this, the DST specifies two sets of properties, Data Source Properties and Channel Properties.

   3. Create a Data Source Type, GITLAB, as shown below. Client Type should be THIN. The DST should include the Data Source and Channel properties shown below.

2. Data Source

   1. The Data Source (DS) specifies the Gitlab Server to which the digital thread will connect. This server location (url) is specified by the “host”, “port”, “uriScheme” and “uriPath” Data Source Properties, set up on the DST above. The full set of Data Source Properties include:

      1. host: Host name of the Gitlab instance 

      2. port: Port at which Gitlab Instance runs (eg: 443) 

      3. uriScheme: http or https

      4. uriPath: any path prefix

      5. proxyHost: if applicable

      6. proxyPort: if applicable

      7. authType: Token or Basic (default value is Token)

      8. dateTimeFormat: Format of datetime in your gitlab project (default value is ‘yyyy-MM-dd’T’HH:mm:ss.SSSXXX')

      9. dateFormat: Format of date in your gitlab project (default value is 'yyyy-MM-dd')

   2. Click the Data Source tab. Select the newly created ELASTICSEARCH DST in the “Data Source Type” dropdown. Name the Data Source and add a default ontology and description if desired. Enter at least the required Data Source properties (uriScheme, host & port).

3. Channel

   1. The Channel details the two endpoints of data operations between your Gitlab instance and the digital thread.

   2. Click the “Attach to Digital Thread” tab:

      1. Select your newly created Data Source (above) from the “Data Source” dropdown. Click “Next”.

      2. Under “Configure channel properties”, select the ontological context for your dataset if it wasn’t set previously and set the Channel Properties to point to a specific project on your gitlab instance that you. Channel properties include:

         1. projectId: Internal ID of the project

         2. projectName: Name of the project

         3. branch: Name of the branch, used when publishing files

         4. filePath: path to file(s) being published; separated by ‘|' (e.g. “sample.txt|jsonSample.json”)

      3. Select an existing or create a new Partition and Branch in the digital thread to connect to your Gitlab project. This Branch is the location on the digital thread where all data will be published to and/or refreshed from. Click “Create Channel” to create the new channel.

1.4 Publishing Items

1. Create a Data Source Type, Data Source and Channel as described in Section 1.3 above.

   1. Ensure that the URL details of the Gitlab instance are included in the Data Source Properties the specific project and any files to be published are included in the Channel Properties, and your Gitlab mappings are loaded.

2. In the Data Source Manager, click the “View Channels” tab. Select your Gitlab Data Source from the “Select Data Source” dropdown and the Channel from which you want to publish from the list of channels below. See Section 1.3, Item 3, for a detailed description of the “indices” Channel property.

3. Ensure that an ontological context is set in the “Selected Ontology” field and click “Publish”.

4. Click “Tracking Center” to monitor the progress of the Publish operation.

1.5 Refreshing Items (Including Subscribed Items)

1. Assuming that a Data Source Type, Data Source and Channel have been set up, initiating a Refresh operation, as with Publish, can be performed from the “View Channels” tab of the Data Source Manager.

2. Click “Hard Refresh” to refresh data from the digital thread to the connected Gitlab project.

   1. ASOT models, those originating from the Gitlab project via a previous Publish operation on the channel, will be updated, including any changes to properties or relations, in the same indices from which they originated. Existing issues/epics/tasks and files will be overwritten with updated versions from the digital thread.

   2. Subscribed models, those originating from projects or systems outside of the channel will be written to the project. Subscription refresh is currently only available or Epics and Issues/Tasks.

1.6 Verify

1. Assuming that Data Source Type, Data Source and Channel have been set up, and an initial Publish and/or Refresh have been completed, a Verify operation can be utilized to ensure that records in Elasticsearch are in sync with their counterparts in the digitial thread.

2. In the View Channels tab of Data Source Manager, select your Data Source and Channel and click “Verify”.

3. Click the link to open the Tracking Center. Click the hyperlinked Tracking ID beside the Verify Operation to go to the tracking page of the Verify operation.

4. Upon completion of the final stage of the Verify operation, as indicated by a green circle/checkmark above the “assessed” stage, click the small arrow to the right of the page header and select “Download Logs (Detailed)”.

   

5. View the detailed logs to see an item-by-item comparison of each corresponding Gitlab item/SBE model pair, indicating any discrepancies between the two in properties, relations or metadata.

1.7 Advanced Operations

Soft-Refresh

• The Rebase/Soft-Refresh operation is a variation of the Hard Refresh, as described in Section 1.1. A Soft-Refresh is initiated by the “Soft Refresh” button on the “View Channels” tab of the Data Source Manager.

Reset

• Reset is currently supported by the Gitlab adapter, and can be initiated by a direct HTTP request to that platform. However, the option is not currently supported in the SBE web-UI.

1.8 Troubleshooting for End Users

2. Document Overview

2.1 Document Overview

This document provides essential information for using, configuring, and supporting the SBE Vision adapter for Gitlab. There is a different version of this document for each major release of the SBE Platform.

2.2 Document Orientation

This document is designed to inform users with various roles:

• End Users should begin with Section 1 to understand how to access and operate the adapter, and Section 5 for issues pertaining to the setup, configuration, and use of the digital tool itself.

• Digital Thread Specialists should focus on Section 1, and also consult Sections 3, 4, and 5 for deployment and semantic mapping. Section 11 contains details related to mapping items from this tool into a semantic ontology.

• Administrators should refer to Section 6 and beyond for setup, security, support, and version management.

3. Adapter Use Cases

3.1 Adapter Overview

The purpose of this adapter is to allow the data contained within instances of Gitlab to connect with the SBE Digital Thread platform. Given that Gitlab is a server-based tool, the usage of this adapter is primarily through the SBE UI or http requests to the SBE instance. This adapter was built using the SBE Java Pro-SDK product.

3.2 Typical Use Cases

Lead By: Solutions

• Why this adapter exists

• What problem it solves in the system development lifecycle

• reqs ↔ architecture with diff/merge in between

• Example: “Sync requirements from Tool A to Tool B”

• Step-by-step walkthroughs or diagrams

4. Supported Versions

Led By: Engineering
Source: Existing Docs & CCTV

4.1 Supported Adapter Products

7.x

8.0, 8.1, 8.2, 8.3

4.2 External Tool Versions Supported

All 8.x versions of the Gitlab Adapter utilize the Gitlab REST API. The Gitlab adapter will work with any Gitlab server/instance and publish/refresh the set of objects described in this document that are supported by that specific version/configuration of Gitlab. If, for example, your Gitlab configuration does not support Epics, the adapter will simply process issues, pipelines, jobs and files in all data operations.

4.3 Differences Across Tool Versions

Versions 8.0 was the first to support SBE 8.0 Melrose. Version 8.0 is equivalent to SBE 7.0 Woburn version 7.4, supporting only publish of Gitlab data (Epics, Issues, Pipelines, Jobs and JSON files) into the digital thread.

Version 8.1 added support for hard/soft Refresh of authoritative data, Subscriptions (including hard/soft Refresh and Publish), Verify and Publish/Refresh of non-JSON files.

Reset support was added in v8.3.

4.4 Supported Plug-Ins and Add-Ons

N/A

5. Digital Tool Best Practices

Source: Existing Info & Knowledge

5.1 Tool Configuration Considerations

• The Gitlab adapter requires minimal configuration. All configuration is done via the Data Source and Channel properties.

• The target Gitlab instance must be set in the ‘uriScheme’, ‘host’, ‘port’ and ‘uriPath’ (optional, as necessary) Data Source Properties. The ‘authType’ property defaults to Token, and must be set to 'Basic” if specified by the Gitlab instance.

• The specific project and branch on your Gitlab instance to be attached to the digital thread is specified by the ‘projectId’, ‘projectName’ and ‘branch’ Channel properties.

• Any files to be included on your channel must be specified in the filePath Channel property, as a list separated by '|'.

5.2 Usage Tips & Gotchas

• N/A

5.3 Tool Limitations and Workarounds

• N/A

6. Installation

Led By: Engineering
Source: Atlassian Page, DevOps Team & Knowledge

6.1 Installation Instructions

• The Gitlab Adapter is a server-based tool that runs entirely on an existing SBE deployment.

6.2 Configuration

6.3 Data Source Type Definition

1. Data Source Type

   1. The Data Source Type (DST) is a generic designator that tells SBE the identity of the external system that a Data Source, Channel or Connection are targeting. The Gitlab adapter handles all operations on Channels tied to Gitlab, as indicated by an associated DST with name “GITLAB”.

   2. The GITLAB DST can be associated with multiple Data Sources, thus allowing connection to multiple Gitlab servers, projects and branches on different channels. To accommodate this, the DST specifies two sets of properties, Data Source Properties and Channel Properties.

   3. The Gitlab Adapter requires a Data Source Type named “GITLAB”, as shown below. Client Type should be THIN and the DST should be created with the Data Source and Channel properties shown below:

2. Data Source

   1. The Data Source (DS) specifies the Gitlab Server to which the digital thread will connect. This server location (url) is specified by the ‘host’, ‘port’, ‘uriScheme’ and ‘uriPath’ Data Source Properties, set up on the DST above. The full set of Data Source Properties include:

      1. host: Host name of the Gitlab instance 

      2. port: Port at which Gitlab Instance runs (eg: 443) 

      3. uriScheme: http or https

      4. uriPath: any path prefix

      5. proxyHost: if applicable

      6. proxyPort: if applicable

      7. authType: Token or Basic (default value is Token)

      8. dateTimeFormat: Format of datetime in your gitlab project (default value is ‘yyyy-MM-dd’T’HH:mm:ss.SSSXXX')

      9. dateFormat: Format of date in your gitlab project (default value is 'yyyy-MM-dd')

7. Channels and Mappings

7.1 Channel Definition

1. The Channel details the two endpoints of data operations between your Gitlab instance and the digital thread.

2. Click the “Attach to Digital Thread” tab:

   1. Select your newly created Data Source (above) from the “Data Source” dropdown. Click “Next”.

   2. Under “Configure channel properties”, select the ontological context for your dataset if it wasn’t set previously and set the Channel Properties to point to a specific project on your gitlab instance that you. Channel properties include:

      1. projectId: Internal ID of the project

      2. projectName: Name of the project

      3. branch: Name of the branch, used when publishing files

      4. filePath: path to file(s) being published; separated by ‘|' (e.g. “sample.txt|jsonSample.json”)

   3. Select an existing or create a new Partition and Branch in the digital thread to connect to your Gitlab project. This Branch is the location on the digital thread where all data will be published to and/or refreshed from. Click “Create Channel” to create the new channel.

7.2 Approaches to Mapping

• Example mappings of gitlab items to SBE’s MBE ontology can be provided upon request.

• All properties of interest must be mapped in order to be published into or refreshed out of the digital thread. Identifying “_externalName” values for all properties match those output by the Gitlab REST API.

• Similar to properties, relations must be mapped (with the exception of JSON files broken down into multiple models) in order to be included in the digital thread. For example, the Pipeline object must contain a Relation mapping to ‘Jobs’ in order to preserve this link through DT data operations. Similarly, the ‘Relates’ and ‘Blocks’ relations between native Gitlab Issues/Tasks must be mapped to be included in the DT.

8. Security and Access

Lead By: Engineering

8.1 Authentication Methods

• See RFC 006 - Authentication & Authorization for the standard on how to authenticate to use SBE and adapter services.

• A user must be logged into the SBE platform as a user with appropriate permissions (Read, Write, and/or Manage) for the operation being performed.

• Gitlab credentials (token or username/password) will be requested upon initiation of any data operation in the platform.

8.2 Authorization and Roles

• Role-based access models

• User-to-role mapping

8.3 Secure Communication

• TLS configuration

• Firewall or VPN guidelines

8.4 Identity Integration

• LDAP, SSO, or identity federation

9. Troubleshooting

Lead By: Solutions
Source: Existing Docs & Knowledge

9.1 Known Issues

• Compatibility limitations

• Common operational issues

• Maximum item-size limitations

• Performance Issues

9.2 Logs and Diagnostics

• Where logs are located

• Enabling debug output

9.3 Common Misconfigurations

• Setup pitfalls

• Misuse scenarios

9.4 Escalation and Support

• How and when to contact SBE support

• Required logs or artifacts

10. Release Notes

Lead By: Engineering
Source: Existing Docs, GitLab

10.1 Version History

8.3

• Adds support for Reset operation

• Fixes Bug in Verify of links

• Fixes bug preventing refresh of labels on Issues

• Upgrade to Java-sdk 8.11

8.2

• Upgrade to Java-sdk 8.9

8.1

• Adds support for Hard/Soft Refresh

• Adds support for Subscriptions

• Adds support for Verify operation

• Adds support for non-JSON (as well as JSON) file Publish/Refresh

• Upgrade to Java-sdk 8.8

8.0

• Support SBE 8.0 Melrose

• Includes all features up to gitlab-adapter v7.4

10.2 Breaking Changes

• N/A

11. Technical Reference

Lead By: Engineering
Source: ARS & RFCs

11.1 Adapter API Endpoints

• The Gitlab Adapter operates entirely via the SBE platform. There are no external API endpoints to the adapter. The adapter itself communicates with the Gitlab server via the Gitlab REST API. Users can also access and manage their Gitlab projects via the Gitlab UI or via independent http requests to the Gitlab REST API.

11.2 Identity

• All items within a Gitlab project, and the project itself, can be uniquely identified by a combination of the item type (e.g. Issue or Pipeline) and a Gitlab generated and assigned ID. These two variables are utilized as the unique external identifier for all ASOT and subscription items within a connected Gitlab project.

11.3 Configuration File Format Reference

• N/A

11.4 Schema Support

The Gitlab adapter supports data operations (limitations described herein) on Epic/Issue/Task, Pipeline, Job and File objects. The schemas for these metaclasses are defined by Gitlab and the Gitlab REST API. Included properties and links are listed below for each metaclass.

Metaclass: Issue

Issues, Epics and Tasks are subsets of the Issue metaclass.

 Links:

Metaclass: Pipeline

Attributes:

Links:

Metaclass: Job

Attributes:

Metaclass: JsonItem

This class represents a JsonFile.

Name: JsonItem

{
   "sample": {
      name: value
   }
}

Note: The properties on JsonItem are published based on the json name value pairs

There are 2 relations on JsonItem, that need to be mapped. 

11.5 Glossary of Terms

11.6 Compliance and Certification