API v1 - Credentials

Overview

This is a guide to providing details on the Credential API services offered by Matillion ETL. The Credential API gives details on "Amazon Web Services Credentials" , "Google Cloud Platform Credentials", and "Microsoft Azure Platform Credentials" within the Matillion ETL instance.

The 'Credentials' refers to Roles and Permissions set on the hosting platform of your Matillion ETL instance and does not refer to the database your client connects to.

The Credentials in the Matillion instance can be accessed and via Project → Manage Credentials. The Manage Credentials sets User Defined Credentials, which are specific to an environment. Thus, it is possible to use a different set of credentials for each environment. Whereas, Instance credentials (credentials tied to the instance hosting Matillion ETL) do not change with environment and are overruled by User Defined credentials.

Credential API provides the "resource" data ("Resources" refers to the information returned by an API). These resources usually have various endpoints which are combined with multiple HTTP methods GET, POST, and DELETE for each endpoint.

Credential API Endpoints

API Base URL

http(s)://<InstanceAddress>/rest/v1/<credential>

API Endpoints and Function

Credential API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The Credential API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified. The available API endpoints are listed below:

</tbody>

Graphical Representation

To illustrate the Credential API, endpoints and methods to the further, below is the graphical flow of the /credential endpoint showing possible PATH, GET , POST, and DELETE options.

URL Parameters and Description

Below is the list of endpoint parameters and their brief description:

Endpoints and Server Response

This chapter describes the Credential APIs endpoints and examples. These APIs offers REST-based web service, offering ease of use and a flexible choice of programming language. These APIs can be used to access and analyse the notice and its id associated with the Matillion ETL instance.

All the APIs listed in this chapter are available to use with GET/POST/DELETE HTTP methods .

GET/credentials

In the example below, we will be retrieving a resource information, perform a GET request for that resource endpoint.

Whenever you reach the resource endpoint, the API will expose API metadata for that resource, including PATH, GET and POST and DELETE method options available. In the example below, the metadata would show PATH options for the "credential" available within the current instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential

• Server Response

  {
  "endpoints": [
  {
  "httpMethod": "PATH",
  "name": "CredentialsBaseService",
  "children": [...],
  "type": "RestResponse",
  ...
  },
  {
  "httpMethod": "POST",
  ...
  },
  {
  "httpMethod": "DELETE",
  ...
  },
  {
  "httpMethod": "GET",
  ...
  },...
  ]
  }

PATH/aws

This is the PATH with /credential/aws endpoint for Amazon Web Services to get the credentials details available for the aws within the instance. This PATH is further combined with HTTP methods GET, POST, and DELETE

The list of endpoints for the PATH/aws

• GET/aws
• GET/test
• GET/export
• POST/delete
• POST/update
• POST/updateFromExport
• DELETE/credentialName
• PATH/instance

Below is the detailed description of these endpoints with example response from the server.

GET/aws

This example is a GET method REST API request to get the list of aws credential object available in the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws

• Server Response

  [
  "Instance Credentials",
  "Manual Credentials",
  "awstest"
  ]

GET/test

This example is a GET method REST API request to test the current aws credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>/test

• Server Response

  {
  "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
  "result": "PASS",
  "connectionMessage": "Redshift API: success
  S3 API: success
  SQS API: success
  SNS API: success
  CloudWatch API: success
  RDS API: success
  EC2 API: Check credentials.
  KMS: success
  DMS: success",
  "specifier": null,
  "details": {
  "timestamp": 1595846515047,
  "message": null,
  "ackID": -1
  }
  }

GET/export

To export the selected AWS credential within the Matillion instance, provide the <credentialName> and use the /export endpoint. This example using GET method REST API call to export the "name" and "secret" of the selected aws credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>/export

• Server Response

  {
  "objects": [
  {
  "name": "aws",
  "secret": "gsdjwjh23412jk"
  }
  ],
  "version": "master",
  "environment": "redshift"
  }

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected aws credential from the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>/delete

• Server Response

  {
  "success": true,
  "msg": "AWS Credential AWSTest deleted",
  "id": 784
  }

POST/update

The /update endpoint will allow to update the selected aws credential. This will be a POST method API call as we will have to attach the details as "name" and "secretKey" (in JSON form, as exported), in the body as a JSON file to update into the Matillion ETL instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>/update WITH POST DATA arg0

• POST Body(JSON)

  {  "name": "<credentialname>",
  "secretKey": "<secretkey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  secretkey	String	The key for the selected AWS credential.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential aws",
  "id": 782
  }

POST/updateFromExport

The /updateFromExport endpoint will allow to update the selected aws credential using exported credential. Now you have already exported the aws credential (see previous example), we will use the exported information ("name" and "secret") in POST Body and update the selected aws credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>/updateFromExport WITH POST DATA arg0

• POST Body (JSON)

  {  
  "name": "<credentialname>",
  "secret":<secretkey> 
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  secret	String	The exported credentials which have been encoded to be unreadable.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential awstest",
  "id": 782
  }

DELETE/credentialName

This will be a DELETE HTTP method API call that will remove the aws credential object from the listing.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/name/<credentialName>

• Server Response

  {
  "success": true,
  "msg": "AWS Credential awstest deleted",
  "id": 253
  }

PATH/instance

This PATH is a part of the PATH/aws. This will provide the AWS Credentials Instance Service includes PATH, GET, POST and DELETE HTTP methods and datatypes for AWS objects and all associated endpoints . This will be a GET request to retrieve the details.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/aws/instance?credentialName=<credentialName>

• Server Response

    "endpoints": [
  {
  "httpMethod": "PATH",
  "name": "AWSCredentialsInstanceService",
  "children": [
  {
  "httpMethod": "POST",
  "name": "updateUnencrypted",
  "description": "Takes a body containing name, access key and secret key and updates this credential",
  "path": "/update",
  "arguments": [...],
  ],..
  "type": "AWSCredentialsInstanceService"
  }
  ],
  "dataTypes": [...]
  }
  ]
  }

PATH/gcp

This is the PATH with /credential/gcp endpoint for Google Cloud Platform to get the credentials and details available for the gcp within the instance. This PATH is further combined with HTTP methods GET, POST, and DELETE

The list of endpoints for the PATH/gcp

• GET/gcp
• GET/test
• GET/export
• POST/delete
• POST/update
• POST/updateFromExport
• POST/create
• POST/import
• DELETE/credentialName
• PATH/instance

Below is the detailed description of these endpoints with example response from the server.

GET/gcp

This example is a GET method REST API request to get the list of gcp credential object available in the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp

• Server Response

  [
  "GCP",
  "GCP_Test"
  ]

GET/test

This example is a GET method REST API request to test the selected gcp credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/test

• Server Response

  {
  "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
  "result": "PASS",
  "connectionMessage": "BigQuery: success
  GoogleCloudStorage: success
  PubSub: success
  KMS: success",
  "specifier": null,
  "details": {
  "timestamp": 1595850444709,
  "message": null,
  "ackID": -1
  }
  }

GET/export

To export the selected GCP credential within the Matillion instance, provide the <credentialName> and use the /export endpoint. This example using GET method REST API call to export the "name" and "secret" of the selected GCP credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/export

• Server Response

  {
  "objects": [
  {
  "name": "GCPtest",
  "secret": "==fQpsdQBsb..."
  }
  ],
  "version": "master",
  "environment": "redshift"
  }

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected aws credential from the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/delete

• Server Response

  {
  "success": true,
  "msg": "GCP Credential GCPtest deleted",
  "id": 790
  }

POST/update

The /update endpoint will allow to update the selected gcp credential. This will be a POST method API call as we will have to attach the details as "name" and "secretkey" (in JSON form, as exported), in the body as a JSON file to update the selected gcp credential object.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/update WITH POST DATA arg0

• POST Body(JSON)

  {  
  "name": "<credentialname>",
  "secretKey": "<secretKey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  secretkey	String	The key for the selected credential.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential gcp",
  "id": 782
  }

POST/updateFromExport

The /updateFromExport endpoint will allow to update the selected gcp credential object using exported credential. Now you have already exported the gcp credential (see previous example), we will use the exported information ("name" and "secret") in POST Body and update the selected aws credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/updateFromExport WITH POST DATA arg0

• POST BODY(JSON)

  {  
  "name": "<credentialname>",
  "secret": "<secretKey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  secret	String	The exported credentials which have been encoded to be unreadable.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential gcptest",
  "id": 782
  }

POST/create

The /create endpoint will allow to create a new gcp credential object. This will be a POST method API call as we will have to attach the details to create ("Name" and "privateKey") (in JSON form, as exported), in the body as a JSON file to create gcp object.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/create WITH POST DATA arg0

• POST BODY(JSON)

  {  
  "name": "<credentialname>",
  "privateKey": "<privateKey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  privateKey	String	The privateKey which has been encoded to be unreadable.

• Server Response

  {
  "success": true,
  "msg": "Created GCP Credential GCPtest",
  "id": -1
  }

POST/import

Now that you have an exported gcp credential object (see previous example), this time we use the API to import that object into a Matillion ETL instance. Note that, when importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new.This will be a POST method API call as we will have to attach the details to be imported (in JSON form, as exported), in the body as a JSON file to import into the Matillion ETL instance.

http://<InstanceAddress>/rest/v1/credential/gcp/import WITH POST DATA arg0

{
"objects": [
{
"name": "GCP1",
"secret": "ICN0aX5uSG9pcGJ0eXNuImB"
}
],
"version": "master",
"environment": "redshift"
}

{
"name": "Credentials",
"statusList": [
{
"success": true,
"name": "GCP1"
}
],
"success": true
}

DELETE/credentialName

This will be a DELETE HTTP method API call that will remove the gcp credential object from the listing.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>

• Server Response

  {
  "success": true,
  "msg": "GCP Credential GCP1 deleted",
  "id": 66535
  }

PATH/instance

This PATH is a part of the PATH/gcp. This will provide the GCP Credentials Instance Service includes PATH, GET, POST and DELETE HTTP methods and dataTypes for GCP and all associated endpoints . This will be a GET request to retrieve the details.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/instance?credentialName=<credentialName>

• Server Response

  {
  "endpoints": [
  {
  "httpMethod": "PATH",
  "name": "GCPCredentialsInstanceService",
  "children": [
  {
  "httpMethod": "POST",
  "name": "updateUnencrypted",
  "description": "Takes a body containing name and private key and updates this credential",
  "path": "/update",
  "arguments": [...],
  "type": "GCPCredentialsInstanceService"
  }
  ],
  "dataTypes": [...]
  },
  {
  "type": "CredentialExport",
  "fields": [...]
  ]...
  }

PATH/azure

This is the PATH with /credential/azure endpoint for Microsoft Azure Platform to get the credentials and details available for the azure within the instance. This PATH is further combined with HTTP methods GET, POST, and DELETE

The list of endpoints for the PATH/azure

• GET/azure
• GET/test
• GET/export
• POST/delete
• POST/update
• POST/updateFromExport
• POST/create
• POST/import
• DELETE/credentialName
• PATH/name/<credentialName>
• PATH/instance

Below is the detailed description of these endpoints with example response from the server.

GET/azure

This example is a GET method REST API request to get the list of azure credential object available in the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>/test

• Server Response

  [
  "aws_azure",
  "cactus aazure credentials",
  "Azure-test"
  ]

GET/test

This example is a GET method REST API request to test the selected azure credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>/test

• Server Response

  {
  "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
  "result": "PASS",
  "connectionMessage": "Blob Storage: success",
  "specifier": null,
  "details": {
  "timestamp": 1595857388819,
  "message": null,
  "ackID": -1
  }
  }

GET/export

To export the selected Azure credential within the Matillion instance, provide the <credentialName> and use the /export endpoint. This example using GET method REST API call to export the "name", "secret", "version", and "environment" of the selected Azure credential.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>/export

• Server Response

  {
  "objects": [
  {
  "name": "aws_azure",
  "secret": "0KbH5sdWA6IiQgc35pbmlIb2Rwcn55Y2UiIGwgCiI9WiU0Tnhxa2htWFBmZ1hvOT04WG...."
  }
  ],
  "version": "master",
  "environment": "redshift"
  }

POST/delete

This will be a POST method API call. The /delete endpoint will allow to delete the selected azure credential from the instance.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>/delete

• Server Response

  {
  "success": true,
  "msg": "Azure Credential azuretest deleted",
  "id": 796
  }

POST/update

The /update endpoint will allow to update the selected azure credential object. This will be a POST method API call as we will have to attach the details as "name", "tenanantId", "clientId", and "secretkey" (in JSON form, as exported), in the body as a JSON file to update the selected gcp credential object.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/name/<credentialName>/update WITH POST DATA arg0

• POST Body(JSON)

  {
  "name": "<Credential name>",
  "tenantID": "<tenantID>",
  "clientID": "<clientID>",
  "secretKey": "<secretKey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  tenantID	String	The tenantID for the Azure credential to authenticate.
  clientID	String	The clientID for the credential object.
  secretKey	String	The key for the selected credential.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential Azuretest",
  "id": 782
  }

POST/updateFromExport

The /updateFromExport endpoint will allow to update the selected azure credential object using exported credential component. Now you have already exported the azure credential (see previous example), we will use the exported information ("name" and "secret") in POST Body and update the selected azure credential object.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>/updateFromExport WITH POST DATA arg0

• POST BODY(JSON)

  {  
  "name": "<Credentialname>",
  "secret": "SecretKey"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  secret	String	The exported credentials which have been encoded to be unreadable.

• Server Response

  {
  "success": true,
  "msg": "Successfully updated credential AzureTest",
  "id": 782
  }

POST/create

The /create endpoint will allow to create a new azure credential object. This will be a POST method API call as we will have to attach the details to create ("name", "tenantID", "clientID", and "privateKey") (in JSON form, as exported), in the body as a JSON file to create gcp object.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/gcp/create WITH POST DATA arg0

• POST BODY(JSON)

  {
  "name": "<Credential name>",
  "tenantID": "<tenantID>",
  "clientID": "<clientID>",
  "secretKey": "<secretKey>"
  }

  Below is the description of the fields included in the POST body:

  Field name	Data type	Description
  name	String	The name of the credential object.
  tenantID	String	The tenantID for the Azure credential to authenticate.
  clientID	String	The clientID for the credential object.
  secretKey	String	The key for the selected credential.

• Server Response

  {
  "success": true,
  "msg": "Created Azure Credential AzureTest",
  "id": -1
  }

POST/import

Now that you have an exported azure credential object (see previous example), this time we use the API endpoint to import that object into a Matillion ETL instance. Note that, when importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new.This will be a POST method API call as we will have to attach the details to be imported (in JSON form, as exported), in the body as a JSON file to import into the Matillion ETL instance.

http://<InstanceAddress>/rest/v1/credential/azure/import WITH POST DATA arg0

{
"objects": [
{
"name": "AzureTest",
"secret": "gadgjs28738kl"
}
],
"version": "master",
"environment": "redshift"
}

{
"name": "Credentials",
"statusList": [
{
"success": true,
"name": "Azuretest"
}
],
"success": true
}

DELETE/credentialName

This will be a DELETE HTTP method API call that will remove the azure credential object from the listing.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/name/<credentialName>

• Server Response

  {
  "success": true,
  "msg": "Azure Credential AzureTest deleted",
  "id": 66767
  }

PATH/name/credentialName

This PATH is a part of PATH/azure. This endpoint will provide the an Azure Credential metadata including PATHs, GET, POST, and DELETE HTTP methods and dataTypes for a selected azure credential object. This will be a GET request to retrieve the details.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/<credentialName>

• Server Response

  {
  "endpoints": [
  {
  "httpMethod": "PATH",
  "name": "AzureCredentialsInstanceService",
  "children": [
  {
  "httpMethod": "POST",
  "name": "updateUnencrypted",
  "description": "Takes a body containing name, tenant ID, client ID, secret key and updates this credential",
  "path": "/update",
  "arguments": [...],
  "type": "AzureCredentialsInstanceService"
  }
  ],
  "dataTypes": [...]
  }
  ]
  }

PATH/instance

This PATH is a part of the PATH/azure. This will provide the Azure Credentials Instance Service includes PATH, GET, POST and DELETE HTTP methods and dataTypes for azure credential object and all associated endpoints . This will be a GET request to retrieve the details.

• Base URL

  http://<InstanceAddress>/rest/v1/credential/azure/instance?credentialName=<credentialName>

• Server Response

  {
  "endpoints": [
  {
  "httpMethod": "PATH",
  "name": "AzureCredentialsInstanceService",
  "children": [
  {
  "httpMethod": "POST",
  "name": "updateUnencrypted",
  "description": "Takes a body containing name, tenant ID, client ID, secret key and updates this credential",
  "path": "/update",
  "arguments": [...],
  "type": "AzureCredentialsInstanceService"
  }
  ],
  "dataTypes": [...]
  }
  ]
  }