Processing
Configure Engines and Clusters, manage and browse executions of your Tasks, Plans and Promotions.
Endpoints
- https://api.eu.cloud.talend.comURL for the AWS Europe region
- https://api.ap.cloud.talend.comURL for the AWS Asia Pacific region
- https://api.us.cloud.talend.comURL for the AWS United States East region
- https://api.au.cloud.talend.comURL for the AWS Australia region
- https://api.us-west.cloud.talend.comURL for the Azure United States West region
Security scheme
This scheme can be referenced across the API
Public
This Bearer authentication can be set in the Authorization header of your requests.
Authentication tokens and Personal Access Tokens are supported.
Personal Access Tokens can be generated in the Profile Preferences page of the Talend Cloud Portal.
| Name | Description |
|---|---|
| Format | Bearer <token> |
Task Executions
Executabletask
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executable | Executable identifier | string Required | 57f64991e4b0b689a64feed0 |
| parameters | Executable parameters | ObjectOptional | |
| logLevel | Execution log level | string Optional | WARN |
| timeout | Execution timeout in minutes. Minimal value is 1. | integer Optional | INT32 |
JobExecutionStatus
Execution info
Object
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | ||||||||||||||||||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| triggerTimestamp | Trigger time of job execution (UTC) | datetime Required | RFC3339 | ||||||||||||||||||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | ||||||||||||||||||||||||
| userType | Type of user who triggered or scheduled the execution | string Required | HUMAN | ||||||||||||||||||||||||
| jobId | Job ID | string Required | 57f64991e4b0b689a64feed0 | ||||||||||||||||||||||||
| jobVersion | The version of the executed job | string Required | 5.2 | ||||||||||||||||||||||||
| environmentVersion | Execution environment version | string Required | 1.3 | ||||||||||||||||||||||||
| executionStatus | Execution status:
| string Required | |||||||||||||||||||||||||
| executionType | Execution type | string Required | SCHEDULED | ||||||||||||||||||||||||
| apiVersion | The version of the API that was used to run the task. Only available if the task was executed through API call to /processing API, If the task was executed by webhook/schedule/plan/UI this field will be absent.Format: YYYY-MMsince 2026-01 | string Optional | 2021-03 | ||||||||||||||||||||||||
| executionDestination | Destination of execution Note: ELASTIC_ENGINE is legacy name of PIPELINE_ENGINE | string Required | REMOTE_ENGINE | ||||||||||||||||||||||||
| containerId | Container ID | string Optional | |||||||||||||||||||||||||
| runProfileId | Run profile ID | string Optional | 157f818f-a901-4425-b592-0f9282687784 | ||||||||||||||||||||||||
| remoteEngineId | Remote engine ID | string Optional | 157f818f-a901-4425-b592-0f9282687784 | ||||||||||||||||||||||||
| remoteEngineClusterId | Remote engine cluster ID | string Optional | |||||||||||||||||||||||||
| numberOfProcessedRows | Number of processed rows | integer Optional | INT64 1234567890 | ||||||||||||||||||||||||
| numberOfRejectedRows | Number of rejected rows | integer Optional | INT64 0 | ||||||||||||||||||||||||
| accountId | Account ID | string Required | 8494b016-b5ef-4b9c-b16d-8b1f824d7616 | ||||||||||||||||||||||||
| workspaceId | Workspace ID | string Required | 57ce63d3e4b0681c36d1a1c4 | ||||||||||||||||||||||||
| planId | Plan execution ID | string Optional | |||||||||||||||||||||||||
| errorType | Error type if error occurs | string Optional | |||||||||||||||||||||||||
| errorMessage | Error message if error occurs | string Optional | |||||||||||||||||||||||||
| timeout | Execution timeout if set (in minutes) | integer Optional | INT32 | ||||||||||||||||||||||||
| status | Task execution status:
| string Required | execution_successful | ||||||||||||||||||||||||
| misfiredExecutionReason | Misfired execution reason | string Optional | TASK_IS_PAUSED |
JobExecutionStatusV21
Execution info
Note: it’s preferable to use JobExecutionStatus as they are equal
Note: it’s preferable to use JobExecutionStatus as they are equal
Object
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | ||||||||||||||||||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| triggerTimestamp | Trigger time of job execution (UTC) | datetime Required | RFC3339 | ||||||||||||||||||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | ||||||||||||||||||||||||
| userType | Type of user who triggered or scheduled the execution | string Required | HUMAN | ||||||||||||||||||||||||
| jobId | Job ID | string Required | 57f64991e4b0b689a64feed0 | ||||||||||||||||||||||||
| jobVersion | The version of the executed job | string Required | 5.2 | ||||||||||||||||||||||||
| environmentVersion | Execution environment version | string Required | 1.3 | ||||||||||||||||||||||||
| executionStatus | Execution status:
| string Required | |||||||||||||||||||||||||
| executionType | Execution type | string Required | SCHEDULED | ||||||||||||||||||||||||
| apiVersion | The version of the API that was used to run the task. Only available if the task was executed through API call to /processing API, If the task was executed by webhook/schedule/plan/UI this field will be absent.Format: YYYY-MMsince 2026-01 | string Optional | 2021-03 | ||||||||||||||||||||||||
| executionDestination | Destination of execution Note: ELASTIC_ENGINE is legacy name of PIPELINE_ENGINE | string Required | REMOTE_ENGINE | ||||||||||||||||||||||||
| containerId | Container ID | string Optional | |||||||||||||||||||||||||
| runProfileId | Run profile ID | string Optional | 157f818f-a901-4425-b592-0f9282687784 | ||||||||||||||||||||||||
| remoteEngineId | Remote engine ID | string Optional | 157f818f-a901-4425-b592-0f9282687784 | ||||||||||||||||||||||||
| remoteEngineClusterId | Remote engine cluster ID | string Optional | |||||||||||||||||||||||||
| numberOfProcessedRows | Number of processed rows | integer Optional | INT64 1234567890 | ||||||||||||||||||||||||
| numberOfRejectedRows | Number of rejected rows | integer Optional | INT64 0 | ||||||||||||||||||||||||
| accountId | Account ID | string Required | 8494b016-b5ef-4b9c-b16d-8b1f824d7616 | ||||||||||||||||||||||||
| workspaceId | Workspace ID | string Required | 57ce63d3e4b0681c36d1a1c4 | ||||||||||||||||||||||||
| planId | Plan execution ID | string Optional | |||||||||||||||||||||||||
| errorType | Error type if error occurs | string Optional | |||||||||||||||||||||||||
| errorMessage | Error message if error occurs | string Optional | |||||||||||||||||||||||||
| timeout | Execution timeout if set (in minutes) | integer Optional | INT32 | ||||||||||||||||||||||||
| status | Task execution status:
| string Required | execution_successful | ||||||||||||||||||||||||
| misfiredExecutionReason | Misfired execution reason | string Optional | TASK_IS_PAUSED |
WorkspaceInfo
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Workspace identifier | string Optional | 57f64991e4b0b689a64feed2 |
| name | Workspace name | string Optional | Personal |
| description | Workspace description | string Optional | workspace detail description |
| owner | Workspace owner | string Optional | admin |
| type | Workspace type | string Optional | custom |
| environment | EnvironmentInfo Optional | ||
| protectedArtifactUpdate | Allows to restrict the scope of task updates when a new artifact version is published.
| boolean Optional |
TaskExecutionsFilters
Object
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| environmentId | Environment filter | string Optional | 6148511d6ccc6d41563574e5 | ||||||||||||||||||||
| workspaceId | Workspace filter | string Optional | 6148511d6ccc6d41563574e5 | ||||||||||||||||||||
| status | Filter on status of execution:
| string Optional | |||||||||||||||||||||
| tags | array of string Optional |
| |||||||||||||||||||||
| lastDays | Number of days in the past. Must be in range [1, 60]. Default value - 60 | integer Optional | INT32 1 | ||||||||||||||||||||
| from | Time period filter start date time (milliseconds).
| integer Optional | INT32 | ||||||||||||||||||||
| to | Time period filter end date time (milliseconds).
| integer Optional | INT32 | ||||||||||||||||||||
| limit | Number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 | ||||||||||||||||||||
| offset | Position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Optional | INT32 |
TaskExecutionsFilters
{
"environmentId": "57f64991e4b0b689a64feed3",
"workspaceId": "57f64991e4b0b689a64feed2",
"tags": [
"critical",
"supply-asia"
],
"limit": 100,
"offset": 10,
"status": "execution_rejected",
"from": 1646159400000,
"to": 1649183400000,
"lastDays": 5
}TaskExecutionStatus
Object
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| parallelExecutionAllowed | Whether parallel run was allowed for the execution | boolean Optional | |||||||||||||||||||||||||
| taskId | Task Id | string Required | 57f64991e4b0b689a64feed0 | ||||||||||||||||||||||||
| planId | Plan Id | string Optional | |||||||||||||||||||||||||
| taskVersion | Task version | string Required | 5.2 | ||||||||||||||||||||||||
| executionType | Execution type | string Required | SCHEDULED | ||||||||||||||||||||||||
| apiVersion | The version of the API that was used to run the task. Only available if the task was executed through API call to /processing API, If the task was executed by webhook/schedule/plan/UI this field will be absent.Format: YYYY-MMsince 2026-01 | string Optional | 2021-03 | ||||||||||||||||||||||||
| status | Task execution status:
| string Required | execution_successful | ||||||||||||||||||||||||
| runtime | Executable runtime | ObjectRequired | |||||||||||||||||||||||||
| |||||||||||||||||||||||||||
| errorMessage | Error message if error occurs | string Optional | |||||||||||||||||||||||||
| executionStatus | Execution status:
| string Required | |||||||||||||||||||||||||
| misfiredExecutionReason | Misfired execution reason | string Optional | TASK_IS_PAUSED | ||||||||||||||||||||||||
| timeout | Execution timeout if set (in minutes) | integer Optional | INT32 | ||||||||||||||||||||||||
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | ||||||||||||||||||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| triggerTimestamp | Trigger time of job execution (UTC) | datetime Required | RFC3339 | ||||||||||||||||||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | ||||||||||||||||||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | ||||||||||||||||||||||||
| userType | Type of user who triggered or scheduled the execution | string Required | HUMAN | ||||||||||||||||||||||||
PageTaskExecutionStatus
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| items | array of TaskExecutionStatus Required |
| ||||||||||
| limit | integer Required | INT32 | ||||||||||
| offset | integer Required | INT32 | ||||||||||
| total | integer Optional | INT32 |
Get Tasks Executions
GET /processing/executables/tasks/executionsGet task executions. URL query strings have length limits. For more complex filters, use the POST method.
You can use the following optional filters, which may be combined as needed:
- Some specific environment as the alternate filter to workspaces
- Some specific workspace as the alternate filter to environments
- Status of execution
- Recent days: Limit to tasks executed within the last n days (for example,
1for yesterday,5for the past five days).- Default: 7 days
- Maximum: 10 days
- Date/time period: A custom period (
from/to) in the past; the maximum range is 10 days. - Offset: An offset for pagination. The maximum offset is 1,000.
Note that the workspace should belong to the environment when both filters (environmentId and workspaceId) are specified in the same call.
When neither the number of days nor a date/time period is provided, the default 7 days time range is applied.
Request
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| limit | Number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 | ||||||||||||||||||||
| offset | Position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Optional | INT32 | ||||||||||||||||||||
| to | Time period filter end date time (milliseconds).
| integer Optional | INT64 | ||||||||||||||||||||
| from | Time period filter start date time (milliseconds).
| integer Optional | INT64 | ||||||||||||||||||||
| lastDays | Number of days in the past. Must be in range [1, 60]. Default value - 60 | integer Optional | INT32 | ||||||||||||||||||||
| workspaceId | Workspace filter | string Optional | 6148511d6ccc6d41563574e5 | ||||||||||||||||||||
| environmentId | Environment filter | string Optional | 6148511d6ccc6d41563574e1 | ||||||||||||||||||||
| status | Filter on status of execution:
| string Optional |
Response
Get available task executions
POST /processing/executables/tasks/executionsGet available task executions. Defining filters in the payload body allows you to combine more filters than using the GET method, which is constrained by URL query string length limitations.
You can use the following optional filters, which may be combined as needed:
- Some specific environment as the alternate filter to workspaces
- Some specific workspace as the alternate filter to environments
- Status of execution
- Recent days: Limit to tasks executed within the last n days (for example,
1for yesterday,5for the past five days).- Default: 7 days
- Maximum: 10 days
- Date/time period: A custom period (
from/to) in the past; the maximum range is 10 days. - Offset: An offset for pagination. The maximum offset is 1,000.
Note that the workspace should belong to the environment when both filters (environmentId and workspaceId) are specified in the same call.
When neither the number of days nor a date/time period is provided, the default 7 days time range is applied.
Request
Body
application/json
application/json
{
"limit": 100,
"offset": 10,
"status": "executing",
"from": 1646159400000,
"to": 1649183400000,
"lastDays": 10,
"environmentId": "57f64991e4b0b689a64feed3",
"workspaceId": "57f64991e4b0b689a64feed2",
"tags": [
"critical",
"finance-eu"
]
}Response
Get Task Executions
GET /processing/executables/tasks/{taskId}/executionsGet task executions
Use cases
- Checking scheduled task runs against your maintenance timetable
- Monitoring a specific task without downloading logs
- Getting a task run log for live monitoring
- Fetching run logs periodically for analysis
- Upgrading a single-triggered task to be multi-triggered
- Troubleshooting a hanging task run
- Using component observability metrics to build reports for task runs
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| taskId | task ID | string Required |
| Name | Description | Type | Attributes and examples | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| limit | the number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 | ||||||||||||||||||||
| offset | the position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Optional | INT32 | ||||||||||||||||||||
| to | Time period filter end date time (milliseconds).
| integer Optional | INT64 | ||||||||||||||||||||
| from | Time period filter start date time (milliseconds).
| integer Optional | INT64 | ||||||||||||||||||||
| lastDays | Number of days in the past. Must be in range [1, 60]. Default value - 60 | integer Optional | INT32 | ||||||||||||||||||||
| status | Execution status:
| string Optional |
Response
Execute Task
POST /processing/executionsExecute Task
Use cases
Response
Terminate Task execution
DELETE /processing/executions/{executionId}Terminate Task execution
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | execution ID | string Required |
Response
204Status 204
Execution stopped succesfully
Get Task execution status
GET /processing/executions/{executionId}Get Task execution status
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | execution ID | string Required |
Response
Plan Executions
PlanExecutable
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executable | Executable identifier | string Required | b91cf8b2-5dd1-4b18-915b-4c447cee5267 |
| executionPlanId | The plan execution identifier in case of re-execution | string Optional | 0798b8d1-0e12-472f-be02-a0f04e792daa |
| stepId | The step execution id to start from. Step execution id can be found in executionId field from response of /processing/executions/plans/{planExecutionId}/steps endpointrequired in case of re-execution | string Optional | 09043c9f-02d0-41f6-b3cb-0ea53ffde377 |
| rerunOnlyFailedTasks | Optionally run only failing tasks | boolean Optional |
PlanExecutionStatus
Plan Execution info
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | |||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Required | RFC3339 | |||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | |||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | |||||||||
| userType | Type of user who triggered or scheduled the execution | string Required | HUMAN | |||||||||
| planId | Plan ID | string Required | b91cf8b2-5dd1-4b18-915b-4c447cee5267 | |||||||||
| executionStatus | Execution (detailed) status | string Required | FINISHED | |||||||||
| status | Status of the execution | string Optional | ||||||||||
| executionType | Execution type | string Optional | SCHEDULED | |||||||||
| apiVersion | The version of the API that was used to run the plan. Only available if the plan was executed through API call to /processing API, If the plan was executed by webhook/schedule/plan/UI this field will be absent.Format: YYYY-MMsince 2026-01 | string Optional | 2021-03 | |||||||||
| plannedExecutableCount | Number of planned executables | integer Optional | INT32 2 | |||||||||
| doneExecutableCount | Number of done executables | integer Optional | INT32 1 | |||||||||
| doneExecutableDetails | array of JobExecutionStatus Optional |
| ||||||||||
| misfiredExecutionReason | Misfired execution reason | string Optional | PLAN_EXECUTIONS_PAUSED |
PagePlanExecutionStatus
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| items | array of PlanExecutionStatus Required |
| ||||||||||
| limit | integer Optional | INT32 | ||||||||||
| offset | integer Optional | INT32 | ||||||||||
| total | integer Optional | INT32 |
StepExecution
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | string Optional | ||
| executionId | string Optional | ||
| startTimestamp | datetime Optional | RFC3339 | |
| finishTimestamp | datetime Optional | RFC3339 | |
| executionStatus | string Optional |
StepExecutionDetails
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string Optional | |||||||||||
| executionId | string Optional | |||||||||||
| startTimestamp | datetime Optional | RFC3339 | ||||||||||
| finishTimestamp | datetime Optional | RFC3339 | ||||||||||
| executionStatus | string Optional | |||||||||||
| plannedExecutableCount | integer Required | INT32 | ||||||||||
| doneExecutableCount | integer Required | INT32 | ||||||||||
| doneTaskExecutableDetails | array of JobExecutionStatus Optional |
| ||||||||||
| donePlanExecutableDetails | array of PlanExecutionStatus Optional |
|
Execute Plan
POST /processing/executions/plansAllows to run a Plan.
This endpoint also allows for re-running some plan execution from some failed step : you should then provide execution id and step from which you wish to restart execution.
Response
422Status 422
Entity exists but cannot be processed, because too old or plan exec didn’t fail
Body
application/json
Get available Plans executions
GET /processing/executables/plans/executionsList all plan executions the user is granted visibility on.
For each plan execution returns execution and plan identifiers, start and finish timestamps, execution status and counters on done and planned executables within plan.
This query is paginated and accepts filters as defined in parameters documented above.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| limit | the number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 |
| offset | the position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Optional | INT32 |
| to | to date time (milliseconds) | integer Optional | INT64 |
| from | from date time (milliseconds) | integer Optional | INT64 |
| lastDays | lastDays | integer Optional | INT32 |
| workspaceId | workspace ID the plans should belong to | string Optional | |
| environmentId | environment ID the plans should belong to | string Optional | |
| status | execution status | string Optional |
Response
200Status 200
OK
Body
application/json
Status 200
application/json
{
"items": [
{
"executionId": "7fb48c7a-5ea5-4cb9-9333-e345f28d9dd5",
"startTimestamp": "2022-10-20T14:20:28.582Z",
"finishTimestamp": "2022-10-20T14:20:39.830Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_SUCCESS",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_successful"
},
{
"executionId": "dc57c82f-7f18-4228-9f82-a81e02bae617",
"startTimestamp": "2022-10-20T14:17:56.390Z",
"finishTimestamp": "2022-10-20T14:18:07.865Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_SUCCESS",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_successful"
},
{
"executionId": "4555c45f-9548-41e9-98b4-a9bfa2ee4e5c",
"startTimestamp": "2022-10-20T13:47:37.611Z",
"finishTimestamp": "2022-10-20T13:47:48.864Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_SUCCESS",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_successful"
},
{
"executionId": "f3cfb077-2ea6-4557-bbb2-fd027f3cae4e",
"startTimestamp": "2022-10-20T13:45:04.027Z",
"finishTimestamp": "2022-10-20T13:45:15.929Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_SUCCESS",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_successful"
},
{
"executionId": "b668cd50-6421-4e57-b15a-79b39ac153a8",
"startTimestamp": "2022-10-20T13:18:01.235Z",
"finishTimestamp": "2022-10-20T13:18:16.093Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_FAILED",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_failed"
},
{
"executionId": "226dc12c-a288-4fe0-96f6-f6d51fa2b9ff",
"startTimestamp": "2022-10-20T12:08:48.823Z",
"finishTimestamp": "2022-10-20T12:18:52.729Z",
"planId": "e7734729-cf3d-4578-939c-30381e95cd10",
"executionStatus": "EXECUTION_SUCCESS",
"plannedExecutableCount": 1,
"doneExecutableCount": 1,
"status": "execution_successful"
}
],
"limit": 100,
"offset": 0,
"total": 6
}Get all steps for a plan execution (order by designed execution, only steps, without error handlers)
GET /processing/executions/plans/{planExecutionId}/stepsGet Steps executions status
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| planExecutionId | Plan execution ID | string Required |
Response
200Status 200
Successful response
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of StepExecution |
| |||||||||||
Get Plan executions
GET /processing/executables/plans/{planId}/executionsGet Plan executions
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| planId | plan ID | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| limit | the number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 |
| offset | the position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Optional | INT32 |
| to | to date time (milliseconds) | integer Optional | INT64 |
| from | from date time (milliseconds) | integer Optional | INT64 |
| lastDays | lastDays | integer Optional | INT32 |
| status | execution status | string Optional |
Response
Get Plan Execution status
GET /processing/executions/plans/{planExecutionId}Get detailed status of one plan execution
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| planExecutionId | Execution id | string Required |
Response
Get the step for plan execution without error handlers
GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}Get Step execution status
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| planExecutionId | Plan execution ID | string Required | |
| stepExecutionId | Step execution ID | string Required |
Response
Get execution status of the handler for the specified step
GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}/error-handlerGet Step handler execution status
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| planExecutionId | Plan execution ID | string Required | |
| stepExecutionId | Step execution ID | string Required |
Response
Promotion Executions
AdvancedPromotionSpec
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| artifactId | Advanced Promotion Artifact Id | string Optional | 57f64991e4b0b689a64feed0 |
| artifactType | Advanced Promotion Artifact Type. In the following list of allowed values, ‘ACTION’ refers to Artifacts, ‘FLOW’ refers to Tasks, ‘PLAN’ refers to Plans and ‘WORKSPACE’ refers to Workspaces | string Optional | WORKSPACE |
Execution
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | string Optional | ||
| executionReport | PromotionExecutionStatusV26 Optional |
PromotionExecutableTaskV26
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executable | Executable identifier | string Required | 57f64991e4b0b689a64feed0 |
| keepTargetResources | Allows to keep target resources untouched. | boolean Optional | |
| keepTargetRunProfiles | Allows to keep target run profiles untouched. | boolean Optional | |
| advanced | AdvancedPromotionSpec Optional | ||
| context | Execution context message | string Optional | 255 Free text describing the context for promotion execution |
PromotionExecutionStatus
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | |||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Required | RFC3339 | |||||||||
| triggerTimestamp | Trigger time of job execution (UTC) | datetime Required | RFC3339 | |||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | |||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | |||||||||
| promotionId | Promotion ID | string Optional | ||||||||||
| pipelineId | Deprecated! Use Promotion ID instead | string Optional | ||||||||||
| keepTargetResources | Keep Target Resources flag | boolean Optional | ||||||||||
| keepTargetRunProfiles | Keep Target Run Profiles flag (not returned for API versions earlier V2.2) | boolean Optional | ||||||||||
| advanced | AdvancedPromotionSpec Optional | |||||||||||
| defective | Defective Promotion flag | boolean Optional | ||||||||||
| status | Execution status | string Optional | PROMOTED | |||||||||
| statusMessage | Execution status message | string Optional | ||||||||||
| workspaces | array of WorkspacePromotionResult Optional |
| ||||||||||
| engines | array of RuntimePromotionResult Optional |
| ||||||||||
| clusters | array of RuntimePromotionResult Optional |
|
PromotionExecutionStatusV26
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| executionId | Job execution ID | string Required | 7b2b122e-d6b8-42de-b0ba-fa2f0d36306e | |||||||||
| startTimestamp | Start time of job execution (UTC) | datetime Required | RFC3339 | |||||||||
| triggerTimestamp | Trigger time of job execution (UTC) | datetime Required | RFC3339 | |||||||||
| finishTimestamp | End time of job execution (UTC) | datetime Optional | RFC3339 | |||||||||
| userId | User who triggered or scheduled the execution | string Required | fupton | |||||||||
| promotionId | Promotion ID | string Optional | ||||||||||
| pipelineId | Deprecated! Use Promotion ID instead | string Optional | ||||||||||
| keepTargetResources | Keep Target Resources flag | boolean Optional | ||||||||||
| keepTargetRunProfiles | Keep Target Run Profiles flag (not returned for API versions earlier V2.2) | boolean Optional | ||||||||||
| advanced | AdvancedPromotionSpec Optional | |||||||||||
| defective | Defective Promotion flag | boolean Optional | ||||||||||
| status | Execution status | string Optional | PROMOTED | |||||||||
| statusMessage | Execution status message | string Optional | ||||||||||
| workspaces | array of WorkspacePromotionResult Optional |
| ||||||||||
| engines | array of RuntimePromotionResult Optional |
| ||||||||||
| clusters | array of RuntimePromotionResult Optional |
| ||||||||||
| context | Execution context message | string Optional | 255 Free text describing the context for promotion execution | |||||||||
| userType | Promotion executor type, either HUMAN or SERVICE | string Optional | SERVICE |
Execute a Promotion
POST /processing/executions/promotionsExecute a Promotion from source Environment to target one, according to all the default configuration set on Tasks.
Detailed rules on promotion executions are available here : https://help.talend.com/r/en-US/Cloud/management-console-user-guide/promotion-rules
In addition to these rules, current execution can be adapted as such :
- Default scope of Promotion is the whole environment. When narrower scope (Workspace, Task, etc.) please provide advanced option parameter accordingly.
- Run Profiles and Resources are defaulty updated on target. You can also choose to keep them untouched using parameters.
Lastly, promotion simulation and analysis is also available: helpful when check before exec is needed. Automatic-undo of a promotion execution is not provided.
Response
202Status 202
Execution started
Run a promotion analysis or fully execute a previously analyzed promotion
POST /processing/executions/promotions/asyncUse the type field to choose the action:
ANALYZEstarts an analysis for the promotion specified in the request body.RESUMEstarts full promotion execution from an analysis withstatusANALYZED. Use theexecutionIdfrom the returned analysis.
Response handling:
200returns a completedPromotionReportwhen the analysis finishes before the async handoff.202returns aPromotionStartedResponsewhen the analysis continues asynchronously. Use the returnedexecutionIdto callGET /processing/executions/promotions/{promotionId}/report/{executionId}to retrieve the report. Poll until itsstatusisANALYZEDbefore sendingRESUME.
Response
200Status 200
The analysis or promotion execution is complete. The response includes the report.
Body
application/json
202Status 202
The analysis or promotion execution continues in the background.
Call GET /processing/executions/promotions/{promotionId}/report/{executionId} with the returned executionId to check progress.
Body
application/json
400Status 400
Bad request: malformed promotion or execution identifier
Body
application/json
404Status 404
Promotion or execution not found, or the execution does not belong to the promotion
Body
application/json
409Status 409
Conflict: the execution is defective, already processed or still being analyzed, or the promotion is already running
Body
application/json
500Status 500
Internal server error
Retrieve the current report for a promotion execution
GET /processing/executions/promotions/{promotionId}/report/{executionId}Use the
executionId returned by POST /processing/executions/promotions/async to check progress after a 202 Accepted response. The report’s status indicates when the analysis or promotion execution is complete.Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| promotionId | Promotion identifier that owns the execution report | string Required | |
| executionId | Execution identifier returned by POST /processing/executions/promotions/async when the promotion continues asynchronously | string Required |
Response
400Status 400
Bad request: malformed promotion or execution identifier
Body
application/json
403Status 403
Forbidden: the user is not assigned to the promotion, or the execution does not belong to it
Body
application/json
404Status 404
Report not found
500Status 500
Internal server error
Get Promotion execution status
GET /processing/executions/promotions/{executionId}Get Promotion execution status
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | Execution ID | string Required |
Response
Promoter
User running the promotion
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| fullName | string Optional | ||
| type | Type of the promoter running a promotion | string Optional | |
| userName | User | string Optional |
RunPromotionRequest
Run a promotion, to either:
- analyze (
type=ANALYZEand provide the attributes ofPromotionParameters) - or execute a promotion after analysis (
type=RESUMEand provide the attributes ofResumeAnalyzedPromotionRequest)
PromotionReport
Report of a promotion execution. The
type attribute is FINISHED.Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| clusters | array of RuntimePromotionResult Optional |
| ||||||||||
| defective | Defective Promotion flag | boolean Optional | ||||||||||
| engines | array of RuntimePromotionResult Optional |
| ||||||||||
| executionId | Job execution ID | string Required | 66b1d5b0b6dd93638c4a4f01 | |||||||||
| finishTimestamp | End time of job execution | datetime Optional | RFC3339 | |||||||||
| message | Execution status message | string Optional | ||||||||||
| promoter | Promoter Optional | |||||||||||
| runParameters | PromotionParameters Optional | |||||||||||
| startTimestamp | Start time of job execution | datetime Optional | RFC3339 | |||||||||
| status | Status of a promotion | string Optional | ||||||||||
| triggerTimestamp | Trigger time of job execution | datetime Required | RFC3339 | |||||||||
| workspaces | array of WorkspacePromotionResult Optional |
| ||||||||||
| type | string Required |
PromotionReport
{"clusters":[],"defective":false,"engines":[],"executionId":"66b1d5b0b6dd93638c4a4f01","finishTimestamp":"2026-08-07T15:00:12.000Z","message":"Promotion completed successfully","promoter":{"fullName":"Jane Doe","type":"HUMAN","userName":"jdoe"},"runParameters":{"context":"Validate promotion before rollout","keepTargetResources":false,"keepTargetRunProfiles":false,"promotionId":"66b1d5b0b6dd93638c4a4f00","scope":{"name":"Daily sync","type":"PLAN","workspaceId":"ab12cd34ef56"},"type":"ANALYZE"},"startTimestamp":"2026-08-07T15:00:00.000Z","status":"PROMOTED","triggerTimestamp":"2026-08-07T14:59:58.000Z","type":"FINISHED","workspaces":[]}PromotionStartedResponse
Response when a promotion has started.
The type attribute is STARTED.
Use the executionId to track the progress of the promotion with the dedicated endpoint:
/processing/executions/promotions/{promotionId}/report/{executionId}.
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | string Optional |
PromotionStartedResponse
{"executionId":"66b1d5b0b6dd93638c4a4f01","type":"STARTED"}PromotionParameters
Parameters for starting a promotion analysis.
Set the
type attribute from RunPromotionRequest to ANALYZE.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| context | Execution context message | string Optional | 255 |
| keepTargetResources | Keep Target Resources flag | boolean Optional | |
| keepTargetRunProfiles | Keep Target Run Profiles flag | boolean Optional | |
| promotionId | Promotion ID | string Required | 1 |
| scope | one of ActionScope , EnvironmentScope , FlowScope , PlanScope , WorkspaceScope Optional | ||
| type | string Required |
PromotionParameters
{"context":"Validate promotion before rollout","keepTargetResources":false,"keepTargetRunProfiles":false,"promotionId":"66b1d5b0b6dd93638c4a4f00","scope":{"name":"Daily sync","type":"PLAN","workspaceId":"ab12cd34ef56"},"type":"ANALYZE"}ResumeAnalyzedPromotionRequest
Parameters for resuming an analyzed promotion and executing it fully.
Set the
type attribute from RunPromotionRequest to RESUME.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | executionId of the analysis to execute. Take it from the analysis report, whose status must be ANALYZED. | string Required | 1 |
| promotionId | Promotion ID. Must be the same as the one used for the analysis. | string Required | 1 |
| type | string Required |
ResumeAnalyzedPromotionRequest
{"executionId":"66b1d5b0b6dd93638c4a4f01","promotionId":"66b1d5b0b6dd93638c4a4f00","type":"RESUME"}RuntimePromotionResult
RuntimePromotionResult
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| analyzeReport | Report Optional | |||||||||||
| id | Artifact ID | string Optional | ||||||||||
| jobType | Job type | string Optional | ||||||||||
| name | Artifact Name | string Optional | ||||||||||
| promotionReport | Report Optional | |||||||||||
| runProfiles | array of ArtifactPromotionResult Optional |
| ||||||||||
| targetId | Target Artifact ID | string Optional | ||||||||||
| targetVersion | Artifact target version | string Optional | ||||||||||
| type | Artifact Type | string Optional | WORKSPACE | |||||||||
| usedBy | array of PromotionResultInfo Optional |
| ||||||||||
| version | Artifact version | string Optional | ||||||||||
| versions | array of ArtifactPromotionResult Optional |
| ||||||||||
| workspaceId | Workspace id | string Optional |
WorkspacePromotionResult
WorkspacePromotionResult
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| actions | array of ArtifactPromotionResult Optional |
| ||||||||||
| analyzeReport | Report Optional | |||||||||||
| clusters | array of RuntimePromotionResult Optional |
| ||||||||||
| connections | array of ArtifactPromotionResult Optional |
| ||||||||||
| engines | array of RuntimePromotionResult Optional |
| ||||||||||
| flows | array of FlowPromotionResult Optional |
| ||||||||||
| id | Artifact ID | string Optional | ||||||||||
| name | Artifact Name | string Optional | ||||||||||
| plans | array of ArtifactPromotionResult Optional |
| ||||||||||
| promotionReport | Report Optional | |||||||||||
| resources | array of ArtifactPromotionResult Optional |
| ||||||||||
| targetId | Target Artifact ID | string Optional | ||||||||||
| targetVersion | Artifact target version | string Optional | ||||||||||
| type | Artifact Type | string Optional | WORKSPACE | |||||||||
| version | Artifact version | string Optional | ||||||||||
| workspaceId | Workspace id | string Optional | ||||||||||
| workspaceType | Workspace type | string Optional |
ActionScope
To promote an artifact of this promotion.
Set the
type attribute from PromotionScope to ACTION.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | ID of the artifact to promote | string Required | 1 |
| name | string Optional | ||
| workspaceId | ID of the workspace that contains the artifact | string Required | 1 |
| type | string Required |
EnvironmentScope
To promote the whole source environment of this promotion.
Set the
type attribute from PromotionScope to ENVIRONMENT.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| type | string Required |
FlowScope
To promote a task of this promotion.
Set the
type attribute from PromotionScope to FLOW.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | ID of the task to promote | string Required | 1 |
| name | string Optional | ||
| workspaceId | ID of the workspace that contains the task | string Required | 1 |
| type | string Required |
PlanScope
To promote a plan of this promotion.
Set the
type attribute from PromotionScope to PLAN.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | ID of the plan to promote | string Required | 1 |
| name | string Optional | ||
| workspaceId | ID of the workspace that contains the plan | string Required | 1 |
| type | string Required |
WorkspaceScope
To promote the whole workspace of this promotion.
Set the
type attribute from PromotionScope to WORKSPACE.Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | ID of the workspace to promote | string Required | 1 |
| name | string Optional | ||
| type | string Required |
Report
Promotion Report
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| messages | array of Message Optional |
| ||||||||||
| status | string Optional |
ArtifactVersionPromotionResult
Artifact versions
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| analyzeReport | Report Optional | |||||||||||
| artifactId | Artifact ID | string Optional | ||||||||||
| id | Artifact ID | string Optional | ||||||||||
| jobType | Job type | string Optional | ||||||||||
| name | Artifact Name | string Optional | ||||||||||
| promotionReport | Report Optional | |||||||||||
| targetId | Target Artifact ID | string Optional | ||||||||||
| targetVersion | Artifact target version | string Optional | ||||||||||
| type | Artifact Type | string Optional | WORKSPACE | |||||||||
| usedBy | array of PromotionResultInfo Optional |
| ||||||||||
| version | Artifact version | string Optional | ||||||||||
| versions | array of ArtifactPromotionResult Optional |
| ||||||||||
| workspaceId | Workspace id | string Optional |
PromotionScope
Scope of the promotion, defined by the “type” field with one of those values:
- “ENVIRONMENT” and attributes from EnvironmentScope to promote a whole Environment
- “WORKSPACE” and attributes from WorkspaceScope to promote a whole Workspace
- “PLAN” and attributes from PlanScope to promote a Plan
- “FLOW” and attributes from FlowScope to promote a Task
- “ACTION” and attributes from ActionScope to promote an Artifact
FlowPromotionResult
FlowPromotionResult
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| analyzeReport | Report Optional | |||||||||||
| id | Artifact ID | string Optional | ||||||||||
| jobType | Job type | string Optional | ||||||||||
| name | Artifact Name | string Optional | ||||||||||
| promotionReport | Report Optional | |||||||||||
| targetId | Target Artifact ID | string Optional | ||||||||||
| targetVersion | Artifact target version | string Optional | ||||||||||
| type | Artifact Type | string Optional | WORKSPACE | |||||||||
| usedBy | array of PromotionResultInfo Optional |
| ||||||||||
| version | Artifact version | integer Optional | INT32 | |||||||||
| versions | array of ArtifactPromotionResult Optional |
| ||||||||||
| workspaceId | Workspace id | string Optional |
ArtifactPromotionResult
Resources Promotion Execution Info
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| analyzeReport | Report Optional | |||||||||||
| id | Artifact ID | string Optional | ||||||||||
| jobType | Job type | string Optional | ||||||||||
| name | Artifact Name | string Optional | ||||||||||
| promotionReport | Report Optional | |||||||||||
| targetId | Target Artifact ID | string Optional | ||||||||||
| targetVersion | Artifact target version | string Optional | ||||||||||
| type | Artifact Type | string Optional | WORKSPACE | |||||||||
| usedBy | array of PromotionResultInfo Optional |
| ||||||||||
| version | Artifact version | string Optional | ||||||||||
| versions | array of ArtifactVersionPromotionResult Optional |
| ||||||||||
| workspaceId | Workspace id | string Optional |
PromotionResultInfo
PromotionResultInfo
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| analyzeReport | Report Optional | ||
| id | Artifact ID | string Optional | |
| name | Artifact Name | string Optional | |
| promotionReport | Report Optional | ||
| targetId | Target Artifact ID | string Optional | |
| targetVersion | Artifact target version | string Optional | |
| type | Artifact Type | string Optional | WORKSPACE |
| version | Artifact version | integer Optional | INT32 |
| workspaceId | Workspace id | string Optional |
Remote Engines
Engine
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | Engine id | string Required | 5cdd2ce2c737973f9d581b98 | |||||||||
| name | Engine name | string Required | Remote Engine | |||||||||
| description | Engine description | string Optional | Some description | |||||||||
| workspace | WorkspaceInfo Optional | |||||||||||
| createDate | Date of creation of the engine | datetime Required | RFC3339 | |||||||||
| updateDate | Date of updating of the engine | datetime Optional | RFC3339 | |||||||||
| runtimeId | Engine runtime id | string Required | 9e8acf72-00d5-451b-9175-97bd44cd6b13 | |||||||||
| availability | Availability status of engine|cluster | string Optional | NOT_AVAILABLE | |||||||||
| managed | Indicates whether target runtime (engine/cluster) is managed or not | boolean Optional | ||||||||||
| status | Engine status | string Required | NOT_PAIRED | |||||||||
| runProfiles | array of string Required |
| ||||||||||
| debug | EngineDebug Optional | |||||||||||
| clusterId | cluster that this engine is part of | string Optional | ||||||||||
| preAuthorizedKey | Preauthorized key (only for not paired engine) | string Optional | 123A822BF26DF49045C90D94CD269F3850BFB308B2606019CC63EDADE55B4391 | |||||||||
| esbCompatibilityVersion | ESB compatibility version | string Optional | 7.3.1 | |||||||||
| version | Remote Engine version | string Optional | 2.14.0 |
EngineDebug
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| host | Debug host of engine | string Required | 192.168.0.1 |
EngineRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | Engine name | string Required | Remote Engine | |||||||||
| environmentId | Id of engine environment | string Required | 5c1111d7a4186a4eafed0587 | |||||||||
| workspaceId | Id of engine workspace | string Optional | 5c2222d7a4186a4eafed0587 | |||||||||
| runProfiles | array of string Optional |
| ||||||||||
| debug | EngineDebug Optional | |||||||||||
| description | Engine|cluster description | string Optional | Some description | |||||||||
| esbCompatibilityVersion | Data Service Runner/Talend Runtime version. Mandatory if TALEND_RUNTIME or MICROSERVICE profile selected | string Optional | 7.2.1 |
RunProfileRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | Run profile name | string Required | Run profile | |||||||||
| description | Description | string Optional | Run profile with default options | |||||||||
| jvmArguments | array of string Required |
| ||||||||||
| type | Run profile type | string Required | JOB_SERVER | |||||||||
| runtimeId | Runtime id | string Required | 5e713618de3c75a450447262 |
RunProfileUpdateRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | Run profile name | string Required | Run profile | |||||||||
| description | Description | string Optional | Run profile with default options | |||||||||
| jvmArguments | array of string Required |
|
RuntimeRunProfile
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string Optional | |||||||||||
| name | string Optional | |||||||||||
| description | string Optional | |||||||||||
| createDate | datetime Optional | RFC3339 | ||||||||||
| updateDate | datetime Optional | RFC3339 | ||||||||||
| type | Run profile type | string Optional | ||||||||||
| jvmArguments | array of string Optional |
| ||||||||||
| runtimeId | string Optional | |||||||||||
| version | Run profile version | integer Optional | INT32 | |||||||||
| processorId | Id of the processor (RE / RE Cluster / Gen2 / Dynamic Engine) | string Optional | ||||||||||
| processorType | Type of processor | string Optional | ||||||||||
| runProfileType | Type of the run profile | string Required | ||||||||||
| scopeId | ID of the environment in which this run profile is defined. A run profile is defined either directly in a Dynamic Engine environment or through Remote Engine or Cluster settings. | string Required |
Create new Remote Engine
POST /processing/runtimes/remote-enginesCreate new Remote Engine
Response
Get all (available) Remote Engines
GET /processing/runtimes/remote-enginesGet all (available) Remote Engines
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| query | search query (FIQL format) | string Optional | workspace.environment.id==5cb47ca4b1b5247f6006529e |
Response
200Status 200
OK
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of Engine |
| |||||||||||
Delete Remote Engine by id
DELETE /processing/runtimes/remote-engines/{id}Delete Remote Engine by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | remote engine id | string Required |
Response
204Status 204
No Content
Get Remote Engine by id
GET /processing/runtimes/remote-engines/{id}Get Remote Engine by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | remote engine id | string Required |
Response
Unpair Remote Engine
DELETE /processing/runtimes/remote-engines/{id}/pairingUnpair Remote Engine.
If data service tasks and Route tasks are available in your license, you can maintain OSGi Data Services as deployed while upgrading your Remote Engine, without need to undeploy and redeploy those services. See Upgrading Remote Engine v2.11.8 onwards for data service Job or Route tasks for more details about this specific upgrade process.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | remote engine id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| keepOsgiDeployments | Boolean to indicate if OSGi Routes or data services should be kept as deployed while unpairing - used for Remote Engine upgrade with data service tasks and Route tasks only | boolean Optional | |
| keepMicroserviceDeployments | Boolean to indicate if microservice Routes or data services should be kept as deployed while unpairing - used for Remote Engine upgrade with data service tasks and Route tasks only | boolean Optional |
Response
200Status 200
OK
Body
text/plain
string
Get all engine run profiles
GET /processing/runtimes/remote-engines/{engineId}/run-profilesGet all engine run profiles
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | remote engine id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| type | string Optional |
Response
200Status 200
OK
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of RuntimeRunProfile |
| |||||||||||
Create engine run profile
POST /processing/runtimes/remote-engines/{engineId}/run-profilesCreate engine run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | remote engine id | string Required |
Body
application/json
Response
409Status 409
Run profile with corresponding name already exists for given runtime
Body
application/json
Delete engine run profile
DELETE /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}Delete engine run profile.
When in use in some task configuration, deletion is aborted unless you provided some new Run Profile id to use. It that latter case, tasks will be using this other Run Profile and the original Run Profile would be deleted.
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | remote engine id | string Required | |
| runProfileId | run profile id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| newRunProfileId | new run profile id to use in tasks | string Optional |
Response
202Status 202
Run Profile deleted.
Body
text/plain
string
Get engine run profile by id
GET /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}Get engine run profile by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | remote engine id | string Required | |
| runProfileId | run profile id | string Required |
Response
Update engine run profile
PUT /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}Update engine run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | remote engine id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
Update engine debug information
PUT /processing/runtimes/remote-engines/{remoteEngineId}/debugUpdate engine debug information
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| remoteEngineId | Remote Engine ID | string Required |
Body
application/json
Debug Remote Engine Parameters
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| host | Host IP | string Optional | |
| isDebugModeOn | Is debug mode on | boolean Optional |
application/json
{
"host":"1.1.1.1",
"isDebugModeOn": "true"
}Response
Remote Engine Clusters
ClusterRequest
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | Cluster name | string Required | Some cluster name | |||||||||
| workspaceId | Workspace identifier | string Required | 5c935c8ba4186a4eafed0592 | |||||||||
| environmentId | Environment identifier | string Required | 5c935c8ba4186a4eafed0592 | |||||||||
| description | Cluster description | string Required | some description | |||||||||
| remoteEngines | array of string Required |
|
EngineCluster
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | Resource id | string Required | 5cdd2ce2c737973f9d581b98 | |||||||||
| name | Resource name | string Required | Remote Engine | |||||||||
| description | Resource description | string Optional | Some description | |||||||||
| workspace | WorkspaceInfo Optional | |||||||||||
| createDate | Date of creation of the resource | datetime Required | RFC3339 | |||||||||
| updateDate | Date of updating of the resource | datetime Optional | RFC3339 | |||||||||
| runtimeId | Resource runtime id | string Required | 9e8acf72-00d5-451b-9175-97bd44cd6b13 | |||||||||
| availability | Availability status of engine|cluster | string Optional | NOT_AVAILABLE | |||||||||
| managed | Indicates whether target runtime (engine/cluster) is managed or not | boolean Optional | ||||||||||
| engines | array of string Optional |
|
RunProfileRescheduleOptions
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| method | Run profile reschedule method | string Required | DEFAULT |
| runProfileId | Run profile to be used from cluster | string Required |
Create Remote Engine Cluster
POST /processing/runtimes/remote-engine-clustersCreate Remote Engine Cluster
Response
Get all (available) Remote Engine Clusters
GET /processing/runtimes/remote-engine-clustersGet all (available) Remote Engine Clusters
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| _s | search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“name==NewCluster” | string Optional |
Response
200Status 200
OK
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of BaseEngine |
| |||||||||||
Get Remote Engine Cluster by id
GET /processing/runtimes/remote-engine-clusters/{clusterId}Get Remote Engine Cluster by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | remote engine cluster id | string Required |
Response
Delete Remote Engine Cluster by id
DELETE /processing/runtimes/remote-engine-clusters/{clusterId}Delete Remote Engine Cluster by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | remote engine cluster id | string Required |
Response
204Status 204
No Content
Add Remote Engine to Remote Engine Cluster
PUT /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}Add Remote Engine to Remote Engine Cluster
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | remote engine cluster id | string Required | |
| engineId | remote engine id | string Required |
Body
application/json
Response
204Status 204
No Content
Remove Remote Engine from Remote Engine Cluster
DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}Remove Remote Engine from Remote Engine Cluster
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | remote engine cluster id | string Required | |
| engineId | remote engine id | string Required |
Response
204Status 204
No Content
Create cluster run profile
POST /processing/runtimes/remote-engine-clusters/{clusterId}/run-profilesCreate cluster run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | cluster id | string Required |
Body
application/json
Response
Get cluster run profiles
GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profilesGet cluster run profiles
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | cluster id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| type | string Optional |
Response
200Status 200
OK
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of RuntimeRunProfile |
| |||||||||||
Update cluster run profile
PUT /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}Update cluster run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | cluster id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
Get cluster run profile
GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}Get cluster run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | cluster id | string Required | |
| runProfileId | run profile id | string Required |
Response
Delete cluster run profile
DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}Delete cluster run profile
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| clusterId | cluster id | string Required | |
| runProfileId | run profile id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| runProfileId | new run profile id | string Optional |
Response
204Status 204
No Content
Remote Engine Gen2
AdvancedRunProfile
Pipeline run profile of type ‘Advanced’
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| definition | Json with definition | string Optional | Example {"id": 1} |
| pushdownPreferenceEnabled | Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). | boolean Optional | |
| warehouseName | Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. | string Optional | |
| createDate | Creation date of the run profile | datetime Optional | RFC3339 |
| updateDate | Update date of the run profile | datetime Optional | RFC3339 |
| id | run profile id | string Optional | 57f64991e4b0b689a64feed0 |
| type | string Optional | advanced |
BigDataRunProfile
Pipeline run profile of type ‘Big Data’
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| distribution | Distribution | string Optional | EMR 5.x (Hadoop 2.7) |
| executionFramework | Execution framework | string Optional | Spark on Yarn |
| microBatchInterval | Micro-batch interval (in ms) | integer Required | INT32 1000 |
| streamingTimeout | Streaming timeout (in ms) | integer Required | INT32 10 |
| yarnQueue | Yarn queue | string Optional | q1 |
| driverCores | Driver cores | integer Optional | INT32 1 |
| driverMemory | Driver cores (in MB) | integer Optional | INT32 512 |
| temporaryStorage | Temporary storage | string Required | /tmp-storage |
| yarnStrategy | YarnStrategy Required | ||
| executorMemoryOverhead | Execution memory overhead (MB) | integer Required | INT32 384 |
| isCheckpointingEnabled | Checkpointing | boolean Optional | |
| checkPointFilePath | Checkpoint file path. Required if checkpointing enabled | string Optional | /file-path |
| parameters | Parameters | ObjectOptional | |
| id | Run profile id | string Optional | 57f64991e4b0b689a64feed0 |
| type | Type | string Optional | |
| createDate | Created on | datetime Optional | RFC3339 |
| updateDate | Updated on | datetime Optional | RFC3339 |
Databrick
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| endpoint | Endpoint | string Required | https://endpoint |
| token | Token (can also be provided in the Remote Engine configuration) | string Optional | 1 |
| stagingLocation | DBFS library staging directory | string Required | dbfs:/staging/ |
| target | DatabricksCluster Required | ||
| cloudProvider | Cloud provider | string Required | AWS |
DatabricksCluster
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| cluster | New or existing cluster | string Required | new |
| id | ID of existing cluster. Required for ‘old’ cluster | string Optional | 132553 |
| sparkVersion | Databricks runtime version. Required for ‘new’ cluster | string Optional | 5.5.x-scala2.11 |
| nodeTypeId | Node type ID. Required for ‘new’ cluster | string Required | m4.large |
| logsDestination | Logs destination. Required for ‘new’ cluster | string Required | dbfs:/cluster-logs |
| workersNumber | Number of workers. Required for ‘new’ cluster | integer Required | INT32 2 |
DatabricksRunProfile
Pipeline run profile of type ‘Databricks’
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| microBatchInterval | Micro-batch interval (in ms) | integer Required | INT32 1000 |
| databricks | Databrick Optional | ||
| parameters | Parameters | ObjectOptional | |
| id | Run profile id | string Optional | 57f64991e4b0b689a64feed0 |
| type | Type | string Optional | |
| createDate | Created on | datetime Optional | RFC3339 |
| updateDate | Updated on | datetime Optional | RFC3339 |
DataIntegrationRunProfile
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | string Required | |||||||||||
| description | string Optional | |||||||||||
| jvmArguments | array of string Optional |
|
DataIntegrationRunProfileResponse
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string Optional | "62444fbce9f05d51f8da4031" | ||||||||||
| name | string Required | "Default run profile" | ||||||||||
| description | string Optional | "run profile description" | ||||||||||
| engineId | string Optional | "6246ea2275ca6f0e05bc4a87" | ||||||||||
| runtimeId | string Optional | "f4422f10-4152-4a3f-adae-3b97ba3ba918" | ||||||||||
| jvmArguments | array of string Optional |
| ||||||||||
| type | string Optional | |||||||||||
| createDate | date Optional | |||||||||||
| updateDate | string Optional |
PipelineEngine
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Resource id | string Required | 5cdd2ce2c737973f9d581b98 |
| name | Resource name | string Required | Remote Engine |
| description | Resource description | string Optional | Some description |
| workspace | WorkspaceInfo Optional | ||
| createDate | Date of creation of the resource | datetime Required | RFC3339 |
| updateDate | Date of updating of the resource | datetime Optional | RFC3339 |
| runtimeId | Resource runtime id | string Required | 9e8acf72-00d5-451b-9175-97bd44cd6b13 |
| availability | Availability status of engine|cluster | string Optional | NOT_AVAILABLE |
| managed | Indicates whether target runtime (engine/cluster) is managed or not | boolean Optional | |
| status | string Optional | ||
| preAuthorizedKey | string Optional | ||
| cloudRunner | boolean Optional | ||
| runStatus | Engine status for Cloud Runner | string Optional |
PipelineEngineRequest
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Engine name | string Required | Remote Engine |
| environmentId | Id of engine environment | string Required | 5c1111d7a4186a4eafed0587 |
| workspaceId | Id of engine workspace | string Optional | 5c2222d7a4186a4eafed0587 |
| description | Engine|cluster description | string Optional | Some description |
PipelineRunProfileResponse
Pipeline run profile response object.
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| id | Run profile id | string Optional | 57f64991e4b0b689a64feed0 |
| type | Type | string Optional | |
| createDate | Created on | datetime Optional | RFC3339 |
| updateDate | Updated on | datetime Optional | RFC3339 |
StandardRunProfile
Pipeline run profile of type ‘Standard’
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| configuration | Configuration Required | ||
| executionFramework | Execution framework | string Required | Spark local |
| pushdownPreferenceEnabled | Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). | boolean Optional | |
| warehouseName | Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. | string Optional |
StandardRunProfileResponse
Pipeline run profile of type ‘Standard’
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| name | Name | string Required | Default run profile |
| description | Description | string Optional | Run profile used by default |
| engineId | Pipeline engine id | string Optional | 5f7eff3245ecf259a1c56daa |
| configuration | Configuration Required | ||
| executionFramework | Execution framework | string Required | Spark local |
| id | Run profile id | string Optional | 57f64991e4b0b689a64feed0 |
| type | Type | string Optional | |
| createDate | Created on | datetime Optional | RFC3339 |
| updateDate | Updated on | datetime Optional | RFC3339 |
| pushdownPreferenceEnabled | Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). | boolean Optional | |
| warehouseName | Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. | string Optional |
YarnStrategy
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| type | Yarn strategy type | string Required | |
| initialExecutors | Initial number of executors (for Dynamic strategy) | integer Optional | INT32 |
| minExecutors | Min. number of executors (for Dynamic strategy) | integer Optional | INT32 |
| maxExecutors | Max. number of executors (for Dynamic strategy) | integer Optional | INT32 |
| coresPerExecutor | Cores per executor | integer Optional | INT32 |
| executorMemory | Executor memory (in MB) | integer Optional | INT32 |
| executorsNumber | Number of executors (for Fixed strategy) | integer Optional | INT32 |
Create new Pipeline Engine
POST /processing/runtimes/pipeline-enginesCreate new Pipeline Engine
Response
Get Pipeline Engines
GET /processing/runtimes/pipeline-enginesGet Pipeline Engines
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| query | search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“status==PAIRED” | string Optional |
Response
200Status 200
OK
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of PipelineEngine |
| |||||||||||
Get Pipeline Engine by id
GET /processing/runtimes/pipeline-engines/{engineId}Get Pipeline Engine by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | pipeline engine id | string Required |
Response
Delete Pipeline Engine by id
DELETE /processing/runtimes/pipeline-engines/{engineId}Delete Pipeline Engine by id
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | pipeline engine id | string Required |
Response
204Status 204
No Content
Unpair Pipeline Engine
DELETE /processing/runtimes/pipeline-engines/{engineId}/pairingUnpair Pipeline Engine
Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | pipeline engine id | string Required |
Response
200Status 200
OK
Body
text/plain
string
Get run profile for engine
GET /processing/runtimes/pipeline-engines/{engineId}/run-profilesRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | run profile id | string Required |
Response
200Status 200
successful operation
Body
application/json
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| array of PipelineRunProfileResponse |
| |||||||||||
Create new advanced run profile
POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advancedRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required |
Body
application/json
Response
201Status 201
Created
Get advanced run profile
GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Response
Update advanced run profile
PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
204Status 204
No Content
Create new Big Data run profile
POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/big-dataRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required |
Body
application/json
Response
201Status 201
Created
Get Big Data run profile
GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/big-data/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Response
Update Big Data run profile
PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/big-data/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
204Status 204
No Content
Create new DataBricks run profile
POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/databricksRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required |
Body
application/json
Response
201Status 201
Created
Get DataBricks run profile
GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/databricks/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Response
Update DataBricks run profile
PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/databricks/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
204Status 204
No Content
Create new standard run profile
POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/standardRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required |
Body
application/json
Response
201Status 201
Created
Update standard run profile
PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/standard/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Body
application/json
Response
204Status 204
No Content
Get standard run profile
GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/standard/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| runProfileId | run profile id | string Required |
Response
Create data-integration run profile
POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/data-integrationRequest
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | string Required |
Body
application/json
Response
201Status 201
Created
Get data-integration run profile by id
GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/data-integration/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | string Required | ||
| runProfileId | string Required |
Response
Update data-integration run profile by id
PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/data-integration/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | string Required | ||
| runProfileId | string Required |
Body
application/json
Response
204Status 204
Status 204
Delete run profile
DELETE /processing/runtimes/pipeline-engines/{engineId}/run-profiles/{type}/{runProfileId}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| engineId | engine id | string Required | |
| type | run profile type | string Required | |
| runProfileId | run profile id | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| newRunProfileId | run profile for replacement id | string Optional |
Response
204Status 204
No Content
Run Profiles
RunProfileType
string
PageBaseRunProfile
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| items | array of BaseRunProfile Required |
| ||||||||||
| limit | integer Optional | INT32 | ||||||||||
| offset | integer Optional | INT32 | ||||||||||
| total | integer Optional | INT32 |
BaseRunProfile
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Id of the run profile | string Required | |
| name | Name of the run profile | string Required | |
| scopeId | ID of the environment in which this run profile is defined. A run profile is defined either directly in a Dynamic Engine environment or through Remote Engine or Cluster settings. | string Required | |
| workspaceId | Id of the workspace this run profile is limited to (optional). Not provided in the case of Dynamic Engine. | string Optional | |
| runProfileType | RunProfileType Required | ||
| processorType | ProcessorType Optional | ||
| processorId | Id of the processor (RE / RE Cluster / Gen2 / Dynamic Engine) | string Required | |
| description | Functional description | string Optional | |
| createDate | datetime Required | RFC3339 | |
| updateDate | datetime Optional | RFC3339 |
DIRunProfile
Run profile that can be used for Data Integration job running on dynamic engines, remote engines or cluster of remote engines
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string Required | |||||||||||
| name | string Required | |||||||||||
| scopeId | ID of the environment | string Required | ||||||||||
| runProfileType | Type of the run profile | string Optional | ||||||||||
| processorId | string Required | |||||||||||
| processorType | Type of processor | string Required | ||||||||||
| description | string Optional | |||||||||||
| createDate | datetime Optional | RFC3339 | ||||||||||
| updateDate | datetime Optional | RFC3339 | ||||||||||
| jvmArguments | array of string Required |
| ||||||||||
| runtimeId | Either the Remote Engine id or the Remote Engine cluster id. | string Optional | ||||||||||
| type | Type of the run profile | string Optional |
Create a run profile
POST /processing/run-profilesRequest
Body
application/json
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | string Required | |||||||||||
| description | string Optional | |||||||||||
| processorType | ProcessorType Required | |||||||||||
| type | Run profile type for data integration Jobs | string Required | ||||||||||
| jvmArguments | array of string Required |
| ||||||||||
| scopeId | ID of the environment. The scopeId must be defined when the processor type is DYNAMIC_ENGINE | string Optional | ||||||||||
| runtimeId | The runtimeId must be defined when the processor type is not DYNAMIC_ENGINE. The value should be either the Remote Engine ID or the Remote Engine cluster id. | string Optional |
Response
Get a run profile by id
GET /processing/run-profiles/{id}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Id of the run profile | string Required |
Response
Update a run profile by id
PUT /processing/run-profiles/{id}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Id of the run profile | string Required |
Body
application/json
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | string Optional | |||||||||||
| scopeId | ID of the environment | string Optional | ||||||||||
| description | string Optional | |||||||||||
| jvmArguments | array of string Optional |
|
Response
Remove a run profile by id
DELETE /processing/run-profiles/{id}Request
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Id of the run profile | string Required |
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| rescheduleRunProfileId | After run profile deletion, tasks will be rescheduled to the run profile specified in this parameter | string Optional | "677fdc3ccc355fed69d32b59" |
Response
204Status 204
Status 204
Search run profiles
POST /processing/run-profiles/searchRequest
Body
application/json
Query filter for run profiles.
Results are sorted alphabetically by name and then by the most recent update.
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| processorTypes | array of ProcessorType Optional |
| ||||||||||
| processorType | ProcessorType Optional | |||||||||||
| processorIds | array of string Optional |
| ||||||||||
| scopeIds | array of string Optional |
| ||||||||||
| name | Contain filter | string Optional | ||||||||||
| runProfileType | RunProfileType Optional | |||||||||||
| limit | Number of items to return. Must be in range [1, 100]. Default value - 100 | integer Optional | INT32 | |||||||||
| offset | Position of the first item. Must be greater than or equal to 0. Default value - 0 | integer Required | INT32 | |||||||||
| sortBy | Specifies the field by which results are sorted. | string Optional | NAME | |||||||||
| sortOrder | Determines the sorting direction of query results.
| string Optional | DESC |
Response
Commons
BaseEngine
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Resource id | string Required | 5cdd2ce2c737973f9d581b98 |
| name | Resource name | string Required | Remote Engine |
| description | Resource description | string Optional | Some description |
| workspace | WorkspaceInfo Optional | ||
| createDate | Date of creation of the resource | datetime Required | RFC3339 |
| updateDate | Date of updating of the resource | datetime Optional | RFC3339 |
| runtimeId | Resource runtime id | string Required | 9e8acf72-00d5-451b-9175-97bd44cd6b13 |
| availability | Availability status of engine|cluster | string Optional | NOT_AVAILABLE |
| managed | Indicates whether target runtime (engine/cluster) is managed or not | boolean Optional |
Configuration
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| parameters | Parameters | ObjectOptional | |
| numberOfThreads | Number of threads | integer Required | INT32 1 |
| microBatchInterval | Micro-batch interval (in ms) | integer Required | INT32 1000 |
EnvironmentInfo
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| id | Environment identifier | string Required | 57f64991e4b0b689a64feed3 |
| name | Environment name | string Optional | Development |
| description | Environment description (not returned when querying a list of objects) | string Optional | environment detail description |
| default | Default environment flag | boolean Optional |
ErrorResponse
Error response object
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| status | TTP status code | integer Required | INT32 401 |
| message | Info about error for user. | string Optional | Forbidden |
| detail | Info about error for developer. | string Optional | Token authorization is required for access |
| requestId | The unique identification of the request involved with this error | string Optional | 16fefb53-035a-4249-af9d-f80a3b47b132 |
Page
Object
| Name | Description | Type | Attributes and examples | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| items | array of Object Optional |
| ||||||||||
| limit | integer Optional | INT32 | ||||||||||
| offset | integer Optional | INT32 | ||||||||||
| total | integer Optional | INT32 |
Executionidentifier
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| executionId | Execution identifier | string Required | 16fefb53-035a-4249-af9d-f80a3b47b132 |
ProcessorType
string
Message
Object
| Name | Description | Type | Attributes and examples |
|---|---|---|---|
| info | string Optional | ||
| level | string Optional |