Dataset
Manage the datasources referenced in Talend Cloud applications and the configuration and metadata associated to these datasources (attributes, data types, data quality).
- Contact
- Talend
Endpoints
- https://api.eu.cloud.talend.comURL for the AWS Europe region
- https://api.ap.cloud.talend.comURL for the AWS Asia Pacific region
- https://api.us.cloud.talend.comURL for the AWS United States East region
- https://api.au.cloud.talend.comURL for the AWS Australia region
- https://api.us-west.cloud.talend.comURL for the Azure United States West region
Security scheme
This scheme can be referenced across the API
Public
This Bearer authentication can be set in the Authorization header of your requests.
Authentication tokens and Personal Access Tokens are supported.
Personal Access Tokens can be generated in the Profile Preferences page of the Talend Cloud Portal.
| Name | Description |
|---|---|
| Format | Bearer <token> |
Get the list of datasets
GET /datasetsA dataset is a collection of data retrieved from a connection. Use this method to list all the datasets that have been created for a given user account.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| flat | A dataset is flat when it only contains simple types (not nested) | string Optional | |
| tqlQuery | Filter on dataset’s list by using TQL (Talend Query Language) | string Optional | |
| favorite | Filter on dataset’s favorite status | string Optional | |
| sameSchemaAs | Filter on datasets which have the same schema as the targeted dataset | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c |
| sortOrder | Order for the sort: ascending or descending, should be paired with the sortBy parameter | string Optional | |
| sortBy | Sort on datasets' list with a field name, should be paired with the sortOrder parameter | string Optional | |
| cloudrunner | Filter on dataset’s connection remote engine type | string Optional |
Response
200Status 200
successful operation
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of DatasetResponse |
| |||||||||||
Get a dataset
GET /datasets/{datasetId}A dataset is a collection of data retrieved from a connection. Use this method to get information on a dataset based on its id.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | string Required |
Response
Delete a dataset
DELETE /datasets/{datasetId}A dataset is a collection of data retrieved from a connection. Use this method to delete a dataset based on its id.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | string Required |
Response
Set the certification state of a dataset
PUT /datasets/{datasetId}/certificationUse this method to add a certification level to a dataset that has trusted data. There are three certification statuses:
- no certification
- certification pending
- certified
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | Id of dataset to update the certification | string Required |
Body
application/json
Response
List custom attributes values for a dataset
GET /datasets/{datasetId}/attributesA predefined list of values or free text can be used to add metadata to a dataset based on the configured attribute definition. Use this method to list all the custom attributes values that have been applied on a given dataset based on its id.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| limit | integer Optional | 100 INT32 | |
| category | string Optional | ||
| offset | The number of rows to ignore before the beginning of the result set | integer Optional | INT32 |
Response
Set or update a custom attribute value for a dataset
PUT /datasets/{datasetId}/attributes/{id}Use this method to set or update a custom attribute value applied on a given dataset.
- In the case of a
List of valuesdefinition, you can select another value among the list of authorized ones. - In the case of a
Textdefinition, you can simply edit the value.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | The dataset id | string Required | |
| id | The cad or business id | string Required |
Body
application/json
Response
Delete a custom attribute value for a dataset
DELETE /datasets/{datasetId}/attributes/{id}Use this method to remove a custom attribute value applied on a dataset. When configuring a custom attribute definition, a default value can be set. After removing an existing value, the attribute value of the dataset will be reset to the default one, or to an empty value if no default value was configured beforehand.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | The dataset id | string Required | |
| id | The cad or business id | string Required |
Response
Create a custom attribute definition
POST /datasets/attributesCustom attributes allow you to create a tagging system for your datasets using free text or predefined values for a given category. Use this method to create a new custom attribute definition. The two types of attributes definitions are
List of values and Text.Response
List custom attributes definitions
GET /datasets/attributesCustom attributes allow you to create a tagging system for your datasets using free text or predefined values for a given category. Use this method to list all the custom attributes definitions created on the account.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| limit | Limit the number of elements returned | integer Optional | 100 INT32 |
| category | string Optional | custom | |
| includeMetadata | Add metadata to the response, as the creator full name | boolean Optional | true |
| offset | The number of rows to ignore before the beginning of the result set | integer Optional | INT32 |
Response
Update a custom attribute definition
PATCH /datasets/attributes/{definitionId}Custom attributes allow you to create a tagging system for your datasets using free text or predefined values for a given category. Use this method to update the configuration of a given custom attribute definition based on its id.The PATCH operation allows you to apply specific changes to the attribute configuration without replacing it entirely.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| definitionId | string Required |
Body
application/json
Response
204Status 204
successful operation
Get a custom attribute definition
GET /datasets/attributes/{definitionId}Custom attributes allow you to create a tagging system for your datasets using free text or predefined values for a given category. Use this method to retrieve the information on a given custom attribute definition based on its id.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| definitionId | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| includeMetadata | Add metadata to the response, such as the creator full name | boolean Optional | true |
Response
Delete a custom attribute definition
DELETE /datasets/attributes/{definitionId}Custom attributes allow you to create a tagging system for your datasets using free text or predefined values for a given category. Use this method to delete a given custom attribute definition based on its id.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| definitionId | string Required |
Response
204Status 204
successful operation
List metadata of all fields for a dataset
GET /datasets/{datasetId}/columnsUse cases
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | Id of the dataset | string Required |
Response
200Status 200
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of ColumnDescription |
| |||||||||||
Create or update metadata for multiple fields of a dataset
POST /datasets/{datasetId}/columnsRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | Id of the dataset | string Required |
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of ColumnDescription |
| |||||||||||
Response
204Status 204
The metadata has been successfully updated
Create or update metadata for a specific field of the dataset
PUT /datasets/{datasetId}/columns/{path}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| datasetId | Id of the dataset | string Required | |
| path | Path of the field to update the description | string Required |
Body
application/json
Response
204Status 204
The metadata has been successfully updated
CustomAttributeDefinition
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | UUID | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c | |||||||||
| label | Label of the attribute | string Required | Business domain | |||||||||
| description | Detailed description | string Optional | Attribute to configure the business domain of a dataset | |||||||||
| type | Type of the attribute | string Required | ||||||||||
| defaults | array of string Optional |
| ||||||||||
| mandatory | Indicates if this attribute is mandatory | boolean Required | ||||||||||
| createdAt | Creation timestamp of the attribute | integer Optional | INT32 1621408387341 | |||||||||
| updatedAt | Last update timestamp of the attribute | integer Required | INT64 1621408387341 | |||||||||
| createdBy | Author Required | |||||||||||
| updatedBy | Author Required |
CustomAttributeCreateRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| label | Label of the attribute | string Required | Business domain | |||||||||
| description | Detailed description of the attribute | string Optional | Attribute for our business domains | |||||||||
| mandatory | Indicates if this attribute is mandatory | boolean Required | ||||||||||
| type | Type of the attribute | string Required | ||||||||||
| values | array of string Optional |
| ||||||||||
| defaults | array of string Optional |
|
CustomAttributeCreateResponse
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| definitionId | The id of the created attribute | string Required | 766ba1b6-0500-40fd-a492-c72de568592c |
CustomAttributeUpdateRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| label | Label of the attribute | string Required | Business domain | |||||||||
| description | Description of the attribute | string Optional | Business domains of our organization | |||||||||
| mandatory | Indicates if this attribute is mandatory | boolean Required | ||||||||||
| defaults | array of string Optional |
| ||||||||||
| values | array of string Optional |
|
DatasetResponse
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | UUID | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c | |||||||||
| label | Label of the dataset | string Required | Countries | |||||||||
| certification | Certification status | string Optional | ||||||||||
| entitlements | array of string Optional |
| ||||||||||
| enabled | Indicates if the dataset is enabled or not | boolean Optional | ||||||||||
| version | Technical version of the dataset | integer Required | INT32 0 | |||||||||
| datastoreId | Id of the connection used for the dataset | string Optional | 4233d338-731d-4b43-a5d5-1863f692761c | |||||||||
| datastore | ConnectionInfo Optional | |||||||||||
| properties | Configuration of the dataset | ObjectOptional | ||||||||||
| owner | Id of the user who owns the dataset | string Optional | 62d6bd8e-fb17-48f1-90b2-ebf827f462f8 | |||||||||
| creator | Full name of the user who created the dataset | string Optional | Chad Smith | |||||||||
| rating | RatingInfo Optional | |||||||||||
| draft | Indicates if the dataset is a draft | boolean Optional | ||||||||||
| sharing | SharingState Optional | |||||||||||
| favorite | Indicates if the dataset is a favorite | boolean Optional | ||||||||||
| talendGlobalQuality | TalendQuality Optional | |||||||||||
| trustScore | Trust score of the dataset | number Optional | DOUBLE 57.76 | |||||||||
| trustScoreAxes | array of TrustScoreAxis Optional |
| ||||||||||
| nested | Indicates if the dataset is hierarchical | boolean Optional | ||||||||||
| created | Creation timestamp of the dataset | number Optional | 1619680424393 | |||||||||
| updated | Last update timestamp of the dataset | string Optional | 1619680424393 | |||||||||
| createdBy | User who created the dataset | ObjectOptional | ||||||||||
| ||||||||||||
| updatedBy | User who last updated the dataset | ObjectOptional | ||||||||||
| ||||||||||||
CustomAttributeValueDatasetResponse
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| data | array of AttributeValue Required |
| ||||||||||
| paging | Paging Required |
AttributeValue
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | UUID | string Required | 766ba1b6-0500-40fd-a492-c72de568592c | |||||||||
| label | Label of the attribute | string Required | Security classification | |||||||||
| category | Category of the attribute | string Optional | ||||||||||
| values | array of string Optional |
| ||||||||||
| isDefault | Indicates if the current value for this dataset matches the default value of the attribute | boolean Optional | ||||||||||
| businessId | Business id | string Optional |
AttributeValuePutRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| values | array of string Required |
|
CertificationRequest
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| currentCertification | Certification value to set on the dataset | string Required |
TalendQuality
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| -1 | Number of invalid values in the dataset sample | integer Required | INT32 11315 |
| 0 | Number of empty values in the dataset sample | integer Required | INT32 1776 |
| 1 | Number of valid values in the dataset sample | integer Required | INT32 89436 |
| total | Total number of values in the dataset sample | integer Required | INT32 102527 |
ConnectionInfo
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | ID of the connection used for this dataset | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c |
| label | Label of the connection | string Optional | |
| type | Id of the connection type | string Optional | amRiYy1lZSNKZGJjI2RhdGFzdG9yZSNKZGJjQ29ubmVjdGlvbg |
| typeLabel | Label of the connection type | string Optional | JDBC |
| cloudAgentId | ID of the engine used by this connection | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c |
| icon | Icon identifier of this connection | string Optional | icon1 |
Paging
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| limit | Limit the number of elements returned | integer Required | INT32 100 |
| offset | The number of rows to ignore before the beginning of the result set | integer Required | INT32 1000 |
| total | The total number of elements | integer Required | INT32 |
RatingInfo
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| contributors | Number of users who contributed to this rating | integer Optional | INT32 1 |
| global | Value of the rating | number Optional | DOUBLE 0.75 |
CommandSuccess
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Command id | string Optional | 766ba1b6-0500-40fd-a492-c72de568592c |
| requestId | Request id | string Optional | 166ba1b6-0500-40fd-a492-c72de568592e |
SharingState
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| isOwner | Indicates if the current user is the owner of the dataset | boolean Required | |
| isSharedWithOthers | Indicates if this dataset is shared with other users or groups | boolean Required |
TrustScoreAxis
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Identifier of the axis | string Required | usage |
| label | Label of the axis | string Required | Usage |
| description | string Optional | How often a dataset is used | |
| currentValue | Value for this axis | number Optional | 56.63 |
| maxValue | Max value for this axis | number Optional | 100 |
ErrorMessageBody
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| timestamp | integer Required | INT64 1622618273418 | |
| status | HTTP status | integer Required | INT32 400 |
| error | string Required | Bad Request | |
| exception | string Optional | ||
| message | The request is invalid | string Optional | |
| path | /datasets | string Optional |
ColumnDescription
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| path | The path of the field, as returned in the dataset schema | string Required | address |
| description | The description to set for the related path | string Required | This is the full address of the person |
ColumnDescriptionUpdateRequest
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| description | Description of the field | string Required |
ColumnDescriptionUpdateRequest
This is the full address of the person