NetBox Data Provider

Description

The NetBox Data Provider connects a NetBox instance with Matrix42 – in both directions:

• Import (NetBox → Matrix42): Computers and network devices, including manufacturers, SKUs and roles, are imported from NetBox into Matrix42 and kept in sync. Role mapping controls which NetBox roles are imported as which asset type.
• Export (Matrix42 → NetBox): Regions, sites, locations, SKUs (device types) and assets (devices) are created and updated in NetBox from Matrix42. Which objects and fields are transferred is configured entirely in Matrix42.

1 Installation

1.1 Prerequisites

1.2 Installing the extension

The NetBox Data Provider extension is installed via the Extension Gallery (Administration > Extension Gallery).

The installation provides, among other things:

• the NetBox data provider (generic connector) with the NetBox Configuration dialog,
• the workflows for import and export,
• the engine activations Netbox - Import and Netbox - Export (locked, without schedule),
• the web service for the export and the NetBox script for download.

1.3 Configuring the data provider

1. Open Administration > Integration > Data Providers.
2. Select the NetBox data provider (in the example “NetBox DEMO”) and choose Edit.
3. In the Configurations area, add a new configuration with + or open an existing configuration by double-clicking it.
4. In the NetBox Configuration dialog, enter the connection to NetBox on the General tab and save with Save & Close.

The NetBox Configuration dialog has four tabs:

General tab

2 Import

The import reads devices and device roles via the NetBox REST API (/api/dcim/devices, /api/dcim/device-roles/) and creates or updates computers, network devices, manufacturers, SKUs and roles in Matrix42. It is run by the generic connector on the selected data provider pool.

2.1 Configuring the import

The import settings are on the Import tab of the NetBox Configuration.

General

Import settings (inventory)

2.2 Configuring roles

The Configure Roles area defines which NetBox roles are mapped to which asset types in Matrix42. The list shows Name, Slug and Asset Type for each role.

Importing NetBox roles

Before configuring the roles, run the data provider once with a valid basic configuration and without role mapping. This imports the roles from NetBox into Matrix42.

Assigning NetBox roles

Once the roles have been imported, assign the matching Asset Type to the required roles in the Configure Roles area. The next data provider run imports all devices whose NetBox role has an asset type assigned.

2.3 Automatic execution (engine activation “Netbox - Import”)

The engine activation supplied with the extension is preconfigured as follows:

Setup:

1. Open Administration > Services & Processes > Engine Activations, select Netbox - Import and choose Edit.
2. Activated Engines tab: open the Generic Connector engine and check that the NetBox data provider in use is entered under Connectors. If a data provider other than “NetBox” is used, select it here.
3. Schedules tab: add a schedule with + (name, start date, run time, time zone, frequency, recurrence, end) and confirm it.
4. General tab: clear the option that disables the whole activation.
5. Save with Save & Close.

2.4 Attribute mapping

The following tables show which NetBox attributes are imported into which Matrix42 fields. Fields marked (matching) are used to identify existing objects.

Computers (asset group 1)

API call: GET https://{NetBoxUrl}/api/dcim/devices

Network devices (asset group 8)

API call: GET https://{NetBoxUrl}/api/dcim/devices

Manufacturers

The manufacturers are the distinct values of device_type.manufacturer.name.

Roles

API call: GET https://{NetBoxUrl}/api/dcim/device-roles/

SKUs (device types)

The SKUs are the distinct combinations of device_type.model and device_type.manufacturer.name.

3 Export

3.1 Overview

The export creates and updates regions, sites, locations, SKUs and assets from Matrix42 in NetBox. The Export tab of the NetBox Configuration defines which objects are transferred with which fields. The transfer into NetBox is performed by the NetBox script Matrix42 Sync, which retrieves the configuration and data from the extension’s web service.

Flow of an export started from Matrix42:

3.2 Prerequisites

• NetBox 4.x with custom scripts enabled. The Python package requests must be available in NetBox (included in the official NetBox Docker image).
• Network: NetBox must be able to reach Matrix42 via HTTPS. For starting the export from Matrix42, Matrix42 must in turn be able to reach NetBox at the Netbox Url.
• Matrix42 API token (Administration > Integration > Web Services Tokens) of a user who is allowed to read the exported data.
• Environment variables M42_BASE_URL and M42_API_TOKEN in NetBox (see 3.3).
• Device roles: the slugs delivered for role in the field mapping must exist as device roles in NetBox.
• Custom fields: if further custom fields are filled via cf_<name>, they must be created in NetBox and assigned to the respective object type. The script creates the custom field matrix42_eoid itself.
• NetBox API token (v1) on the General tab with permission to run scripts (only for starting the export from Matrix42).

3.3 Setup

1. Create a Matrix42 API token under Administration > Integration > Web Services Tokens.
2. Enable the export: on the Export tab of the NetBox Configuration, select Enable Export and save. The Configuration ID is needed later in NetBox.
3. Download the sync script: on the Export tab, choose DOWNLOAD SCRIPT. The downloaded file netbox_sync.py always matches the installed version of the web service.
4. Set the environment variables in NetBox and restart NetBox. For Docker installations, the variables must be set in both the NetBox container and the worker container, because scripts run as background jobs:

   environment:
     M42_BASE_URL: 'https://matrix42.example.com'   # base URL without /m42Services
     M42_API_TOKEN: '<Matrix42 API token>'

5. Upload the script to NetBox: in NetBox, go to Customization > Scripts, choose Add and upload the file netbox_sync.py. The scripts Matrix42 Sync and Matrix42 Link Existing Objects then appear.
6. Enter the script ID: determine the numeric ID of the Matrix42 Sync script (visible in the URL /extras/scripts/<ID>/ or via GET /api/extras/scripts/) and enter it on the Export tab under Netbox Script Id.
7. Create the device roles in NetBox that are used in the field mapping of the assets.
8. Configure the entities: on the Export tab, set the export checkbox, filter, status mapping and field mapping for each entity (see 3.5 to 3.7).
9. Link existing NetBox data: if NetBox already contains objects that are to be maintained from Matrix42, link them once with Matrix42 Link Existing Objects (see 3.8.1).
10. Test run: run Matrix42 Sync in NetBox without Commit, check the log and then run it with Commit (see 3.8.2).
11. Automate via the engine activation Netbox - Export (see 3.8.3).

3.4 Export tab – settings

3.5 Entities

The Export tab contains a separate area for each entity. The entities are processed in the following order so that references to objects that have already been transferred can be resolved:

Each area contains the following fields:

3.6 Field mapping

Each row of the field mapping assigns a Matrix42 value to a NetBox field.

Rows without a columns expression or NetBox target are ignored. Each exported entity needs at least one complete row.

Required row: Every entity needs the row [Expression-ObjectID] → cf_matrix42_eoid. It links the Matrix42 object to the NetBox object. Records without this value are not transferred.

Field policy

If no field policy is specified, Always applies.

NetBox fields with special handling

All other NetBox targets are written directly to the NetBox field of the same name (e.g. name, serial, asset_tag, description, comments, model, u_height, part_number, facility). Other reference fields such as location, parent or platform are not supported.

Which fields NetBox requires when creating an object depends on the object. The slug of regions, sites, locations and device types is set automatically to m42-<ID>.

Example: field mapping of the assets

Example: field mapping of the SKUs

3.7 Status mapping

The status mapping assigns a NetBox status to each Matrix42 status. It is available for site, location and assets and is used when the field mapping delivers the status field.

• Statuses without an assigned NetBox status are ignored. Objects with such a status are not transferred and are reported as errors in the log.
• The NetBox status must be valid for the respective object:

3.8 Running the export

3.8.1 One-time: linking existing NetBox objects

If NetBox already contains regions, sites, locations, device types or devices that are to be maintained from Matrix42 in the future, they must be linked before the first sync. Otherwise the sync creates duplicates or fails because names are already taken.

The Matrix42 Link Existing Objects script assigns the records delivered by Matrix42 to NetBox objects that are not yet linked. For each entity it compares a selectable field and, if the values match, writes the custom field matrix42_eoid and the tag m42-managed. It does not create objects and does not change any other fields.

Rules:

• The comparison ignores upper and lower case as well as leading and trailing spaces.
• The match field must be delivered as a NetBox target in the field mapping of the entity. If the value should not be written during the sync, use the field policy Never.
• If a value occurs on several unlinked NetBox objects, none of them is linked.
• Objects that are already linked remain unchanged. Differences in the match field are reported as warnings.

Recommended procedure:

1. Run the script without Commit.
2. Correct the reported differences in Matrix42 or NetBox and repeat step 1 until the log is clean.
3. Run the script with Commit.
4. Then run Matrix42 Sync.

3.8.2 Manually in NetBox

In NetBox, open the Matrix42 Sync script under Customization > Scripts, enter the parameters and run it:

The result contains, per entity, the number of created, updated, unchanged, failed and no longer delivered (disappeared) objects, e.g.:

regions: 0 created, 0 updated, 1 unchanged, 0 failed, 0 disappeared | sites: … | assets: 0 created, 0 updated, 1 unchanged, 0 failed, 0 disappeared

Alternatively, Matrix42 Sync can be scheduled directly in NetBox (options Schedule at and Recurs every when running the script).

3.8.3 Automatically from Matrix42 (engine activation “Netbox - Export”)

The extension supplies the engine activation Netbox - Export. It starts the workflow Netbox - Trigger Export, which starts the script stored under Netbox Script Id in NetBox for this configuration, waits for the NetBox job to finish and stores the status, time, result and log in the Last Export … fields on the Export tab.

Setup is the same as for the import (see 2.3): add a schedule on the Schedules tab, clear the option that disables the whole activation on the General tab and save.

Prerequisites for starting from Matrix42:

• Netbox Url and Token (v1) are set on the General tab; the token is allowed to run scripts.
• Netbox Script Id on the Export tab contains the ID of the Matrix42 Sync script.
• Enable Export is selected.

3.9 Sync behavior

• Linking: Matrix42 objects and NetBox objects are matched exclusively via the custom field matrix42_eoid (label “Matrix42 EOID”). The script creates the custom field on the first run and assigns it to Region, Site, Location, Device Type and Device.
• Tagging: All objects created or changed by the sync get the tag m42-managed. Automatically created manufacturers, tenants and tags are tagged as well.
• New objects: Objects without a matching matrix42_eoid are created. Regions, sites, locations and device types get the slug m42-<ID>.
• Unchanged objects are not saved and do not create a changelog entry.
• Objects no longer delivered (e.g. because of a changed filter or objects deleted in Matrix42) are only reported in the log. They are neither changed nor deleted in NetBox.
• Errors: Connection, HTTP or format errors while retrieving data abort the entire run, so that the sync never works with incomplete data. Errors in individual records are logged; the remaining records are still processed.
• Simulation: Without commit, all changes are rolled back at the end. The result then ends with SIMULATION - nothing was saved.

3.10 Web service reference

Base URL: https://<Matrix42 server>/m42Services/api/mountx/netbox

Authentication

1. POST /m42Services/api/ApiToken/GenerateAccessTokenFromApiToken with the header Authorization: Bearer <Matrix42 API token>. The response contains an access token in the RawToken field.
2. All further calls with the header Authorization: Bearer <RawToken>.

Endpoints

Each record contains the NetBox targets defined in the field mapping as keys. If an entity is not enabled for export, the list is empty.

Example:

GET https://matrix42.example.com/m42Services/api/mountx/netbox/skus?configurationId=<Configuration ID>
Authorization: Bearer <RawToken>
Accept: application/json

{
  "Skus": [
    {
      "model": "Catalyst 9300",
      "manufacturer": "Cisco",
      "cf_matrix42_eoid": "<ID of the SKU>",
      "u_height": 1
    }
  ]
}

Web service error messages

3.11 Troubleshooting

The following messages appear in the log of the NetBox job or in the Last Export Log field.