API v1 - Environment

Overview

This is a guide to using the Matillion ETL Environment API. The Environment API can be used to explore and manage environments using PATH/environment endpoints, and to explore and manage environment variables using PATH/variable endpoints, as described in this article.

An environment describes a single Matillion ETL connection to a database on which Matillion ETL jobs can be run. Environments can be set up at user level via Project → Add Environment within the instance.

Environment variables are name:value pairs that are stored inside the Matillion ETL client and fully configurable by its users. Environment variables can be used throughout a project, in many components in any of the project's jobs. Environment variables must be declared before being used. You declare a variable by selecting Project → Manage Environment Variables.

The environment endpoint /environment is a part of /project, and the variable endpoint /variable is a part of /environment, as shown in the endpoint descriptions below. These endpoints provide unique paths to specific environments and environment variables within a project. Used with the HTTP methods GET, POST, and DELETE, this allows you to retrieve and manipulate any environment and environment variable details from your Matillion API instance.

Prerequisites

• Users responsible for using Matillion ETL API services require access to the Matillion ETL instance.
• Users will need to know how to make REST API calls, either employing a REST API GUI client such as Postman or employing a command-line interface such as cURL.
• Matillion ETL API endpoints require authorization to make any REST API call, so ensure a username and password for the Matillion ETL instance is configured before making any API call.

Environment API endpoints

API Base URLs

The Environment API is accessed through a base URL with one of the following forms.

Environment endpoints:

http(s)://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/environment/name/<environmentName>/

Variable endpoints:

http(s)://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/environment/name/<environmentName>/variable/

The API is a standard REST-based API that uses HTTP or HTTPS GET, POST, and DELETE requests. The specific endpoints within this API are shown in the following tables, along with the HTTP request method required to use them. Full descriptions of each endpoint and the expected server response from using it can be found in Endpoints and server response, below.

Environment endpoints

Variable endpoints

URL Parameters

The parameters surrounded by < and > in the endpoint URLs are placeholders that you should replace with appropriate values in your API calls.

Endpoints and server response

This section gives full descriptions of each endpoint, with examples of the expected server response from using that endpoint.

PATH /environment

GET /name

This is a GET method REST API request that will provide the name of the environment within the project in the instance.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/name

Server Response:

API Environment

GET /id

This is a GET method REST API request that will provide the id of the environment within the project in the instance.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/id

Server Response:

796

GET /test

This GET request is used to test the selected environment within the project.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/test

Server Response:

{
"success": true,
"msg": "Testing environment [API Environment] was successful.",
"id": 0
}

GET /export

This GET request exports details of the selected environment configuration, in JSON format.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/export

Server Response:

{
"objects": [
{
"name": "API Environment",
"credentialsName": "Instance Credentials",
"gcCredentialsName": "GCP",
"azureCredentialsName": "aws_azure",
"schema": "public",
"encrypted": false,
"url": "mtln-rs-006.ck4qmenbus4m.eu-west-1.redshift.amazonaws.com",
"port": "5432",
"database": "admin",
"driver": null,
"user": "admin",
"passwordName": "API PROJECT-test",
"passphraseName": null,
"role": "",
"variables": {},
"defaultStorageLocation": "cf-templates-bvdzucx2zo3e-us-east-1",
"concurrentConnections": 1,
"passwordType": "PASSWORD",
"connectionOptions": {}
}
],
"version": "master",
"environment": "redshift"
}

POST /update

This POST request updates details of the selected environment with new details passed as post data in the request.

A typical use of this would be to clone an existing environment by using GET /export to retrieve its details and then passing those details into a new environment using POST /update.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/update WITH POST DATA arg0

POST Body:

The post body containing the new details must be specified in the JSON format shown in this example. Note how this format is the same as that produced by the server response of a GET /export request.

{
"objects": [
{
"name": "test",
"credentialsName": "Instance Credentials",
"gcCredentialsName": "GCP",
"azureCredentialsName": "aws_azure",
"schema": "public",
"encrypted": false,
"url": "mtln-rs-006.ck4qmenbus4m.eu-west-1.redshift.amazonaws.com",
"port": "5432",
"database": "admin",
"driver": null,
"user": "admin",
"passwordName": "API PROJECT-test",
"passphraseName": null,
"role": "",
"variables": {},
"defaultStorageLocation": "cf-templates-bvdzucx2zo3e-us-east-1",
"concurrentConnections": 1,
"passwordType": "PASSWORD",
"connectionOptions": {}
}
],
"version": "master",
"environment": "redshift"
}

Server Response:

{
"name": "Environments",
"statusList": [
{
"success": true,
"name": "test"
}
],
"success": true
}

PATH /variable

GET /value

This uses a GET request to return the default value of a specified environment variable.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/name/<variableName>/value

Server Response:

date

POST /delete

This uses a POST request to delete the default value of an environment variable in the specified environment. See also DELETE /name.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/name/<variableName>/delete

Server Response:

{
"success": true,
"msg": "Successfully removed variable [test_API] from environment [API Environment].",
"id": 0
}

POST /set/value/

This uses a POST request to set a new value for a specified environment variable. The value must be provided as <variableValue> in the URL, as shown below.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/name/<variableName>/set/value/<variableValue>

Server Response:

{
"success": true,
"msg": "Successfully set value [2] for variable [test_API].",
"id": 0
}

POST /set/instance

This uses a POST request to set a new value for a specified environment variable. The value must be provided as <variableValue> in the URL, as shown below.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/name/<variableName>/set/instance?variableValue=<variableValue>

Server Response:

{
"success": true,
"msg": "Successfully set value [4] for variable [test_API].",
"id": 0
}

DELETE /name (in environment)

This uses a DELETE request to delete the default value for a variable in the specified environment. See also POST /delete

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/name/<variableName>

Server Response:

{
"success": true,
"msg": "Successfully removed variable [test_api] from environment [API Environment].",
"id": 0
}

DELETE /name (in project)

This uses a DELETE request to completely delete the specified variable from the project.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/variable/name/<variableName>

Server Response:

{
"success": true,
"msg": "Successfully removed variable [test_api] from project [API Project].",
"id": 0
}

GET /variable/export

This GET request exports all environment variables in the selected environment, as name:value pairs in JSON format.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/export

Server Response:

{
"objects": [
{
"name": "Example_variable",
"value": "88.8"
},
],
"version": "master",
"environment": "redshift"
}

POST /variable/import

This POST request imports environment variables into the specified environment. The variable names and values are passed as post data in the request.

Base URL:

http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/environment/name/<environmentName>/variable/import

POST Body:

The post body containing the variables must be specified in the JSON format shown in this example. Note how this format is the same as that produced by the server response of a GET /variable/export request.

{
"objects": [
{
"name": "Example_Variable",
"value": "22.1"
}
],
"version": "master",
"environment": "redshift"
}

Server Response:

{
"name": "Environment Variables",
"statusList": [
{
"success": true,
"name": "Example_Variable"
}
],
"success": true
}