task_execution.proto API Reference
Request Content-Types: application/json
Response Content-Types: application/json
Schemes: http, https
Version: version not set
Paths
List tasks. TaskView is requested as such: "v1/tasks?view=BASIC"
OPTIONAL. Filter the list to include tasks where the name matches this prefix. If unspecified, no task name filtering is done.
OPTIONAL. Number of tasks to return in one page. Must be less than 2048. Defaults to 256.
OPTIONAL. Page token is used to retrieve the next page of results. If unspecified, returns the first page of results. See ListTasksResponse.next_page_token
OPTIONAL. Affects the fields included in the returned Task messages. See TaskView below.
- MINIMAL: Task message will include ONLY the fields: Task.Id Task.State
- BASIC: Task message will include all fields EXCEPT: Task.ExecutorLog.stdout Task.ExecutorLog.stderr Input.content TaskLog.system_logs
- FULL: Task message includes all fields.
Response Example (200 OK)
{
"tasks": [
{
"id": "string",
"state": "string",
"name": "string",
"description": "string",
"inputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
],
"outputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
],
"resources": {
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
},
"executors": [
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
],
"volumes": [
"string"
],
"tags": "object",
"logs": [
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}
],
"creation_time": "string"
}
],
"next_page_token": "string"
}
Create a new task.
Request Example
{
"id": "string",
"state": "string",
"name": "string",
"description": "string",
"inputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
],
"outputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
],
"resources": {
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
},
"executors": [
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
],
"volumes": [
"string"
],
"tags": "object",
"logs": [
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}
],
"creation_time": "string"
}
Response Example (200 OK)
{
"id": "string"
}
GetServiceInfo provides information about the service, such as storage details, resource availability, and other documentation.
Response Example (200 OK)
{
"name": "string",
"doc": "string",
"storage": [
"string"
]
}
Get a task. TaskView is requested as such: "v1/tasks/{id}?view=FULL"
(no description)
OPTIONAL. Affects the fields included in the returned Task messages. See TaskView below.
- MINIMAL: Task message will include ONLY the fields: Task.Id Task.State
- BASIC: Task message will include all fields EXCEPT: Task.ExecutorLog.stdout Task.ExecutorLog.stderr Input.content TaskLog.system_logs
- FULL: Task message includes all fields.
Response Example (200 OK)
{
"id": "string",
"state": "string",
"name": "string",
"description": "string",
"inputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
],
"outputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
],
"resources": {
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
},
"executors": [
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
],
"volumes": [
"string"
],
"tags": "object",
"logs": [
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}
],
"creation_time": "string"
}
Cancel a task.
(no description)
Response Example (200 OK)
"object"
Schema Definitions
tesCancelTaskResponse: object
CancelTaskResponse describes a response from the CancelTask endpoint.
Example
"object"
tesCreateTaskResponse: object
CreateTaskResponse describes a response from the CreateTask endpoint.
- id: string
-
Task identifier assigned by the server.
Example
{
"id": "string"
}
tesExecutor: object
Executor describes a command to be executed, and its environment.
- image: string
-
Name of the container image, for example: ubuntu quay.io/aptible/ubuntu gcr.io/my-org/my-image etc...
- command: string[]
-
A sequence of program arguments to execute, where the first argument is the program to execute (i.e. argv).
- workdir: string
-
The working directory that the command will be executed in. Defaults to the directory set by the container image.
- stdin: string
-
Path inside the container to a file which will be piped to the executor's stdin. Must be an absolute path.
- stdout: string
-
Path inside the container to a file where the executor's stdout will be written to. Must be an absolute path.
- stderr: string
-
Path inside the container to a file where the executor's stderr will be written to. Must be an absolute path.
- env: object
-
Enviromental variables to set within the container.
Example
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
tesExecutorLog: object
ExecutorLog describes logging information related to an Executor.
- start_time: string
-
Time the executor started, in RFC 3339 format.
- end_time: string
-
Time the executor ended, in RFC 3339 format.
- stdout: string
-
Stdout content.
This is meant for convenience. No guarantees are made about the content. Implementations may chose different approaches: only the head, only the tail, a URL reference only, etc.
In order to capture the full stdout users should set Executor.stdout to a container file path, and use Task.outputs to upload that file to permanent storage.
- stderr: string
-
Stderr content.
This is meant for convenience. No guarantees are made about the content. Implementations may chose different approaches: only the head, only the tail, a URL reference only, etc.
In order to capture the full stderr users should set Executor.stderr to a container file path, and use Task.outputs to upload that file to permanent storage.
- exit_code: integer (int32)
-
Exit code.
Example
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
tesInput: object
Input describes Task input files.
- name: string
- description: string
- url: string
-
REQUIRED, unless "content" is set.
URL in long term storage, for example: s3://my-object-store/file1 gs://my-bucket/file2 file:///path/to/my/file /path/to/my/file etc...
- path: string
-
Path of the file inside the container. Must be an absolute path.
- type: tesFileType
-
Type of the file, FILE or DIRECTORY
- content: string
-
File content literal. Implementations should support a minimum of 128 KiB in this field and may define its own maximum. UTF-8 encoded
If content is not empty, "url" must be ignored.
Example
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
tesListTasksResponse: object
ListTasksResponse describes a response from the ListTasks endpoint.
- tasks: tesTask
-
List of tasks.
- next_page_token: string
-
Token used to return the next page of results. See TaskListRequest.next_page_token
Example
{
"tasks": [
{
"id": "string",
"state": "string",
"name": "string",
"description": "string",
"inputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
],
"outputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
],
"resources": {
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
},
"executors": [
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
],
"volumes": [
"string"
],
"tags": "object",
"logs": [
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}
],
"creation_time": "string"
}
],
"next_page_token": "string"
}
tesOutput: object
Output describes Task output files.
- name: string
- description: string
- url: string
-
URL in long term storage, for example: s3://my-object-store/file1 gs://my-bucket/file2 file:///path/to/my/file /path/to/my/file etc...
- path: string
-
Path of the file inside the container. Must be an absolute path.
- type: tesFileType
-
Type of the file, FILE or DIRECTORY
Example
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
tesOutputFileLog: object
OutputFileLog describes a single output file. This describes file details after the task has completed successfully, for logging purposes.
- url: string
-
URL of the file in storage, e.g. s3://bucket/file.txt
- path: string
-
Path of the file inside the container. Must be an absolute path.
- size_bytes: string (int64)
-
Size of the file in bytes.
Example
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
tesResources: object
Resources describes the resources requested by a task.
- cpu_cores: integer (int64)
-
Requested number of CPUs
- preemptible: boolean (boolean)
-
Is the task allowed to run on preemptible compute instances (e.g. AWS Spot)?
- ram_gb: number (double)
-
Requested RAM required in gigabytes (GB)
- disk_gb: number (double)
-
Requested disk size in gigabytes (GB)
- zones: string[]
-
Request that the task be run in these compute zones.
Example
{
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
}
tesServiceInfo: object
ServiceInfo describes information about the service, such as storage details, resource availability, and other documentation.
- name: string
-
Returns the name of the service, e.g. "ohsu-compbio-funnel".
- doc: string
-
Returns a documentation string, e.g. "Hey, we're OHSU Comp. Bio!".
- storage: string[]
-
Lists some, but not necessarily all, storage locations supported by the service.
Must be in a valid URL format. e.g. file:///path/to/local/funnel-storage s3://ohsu-compbio-funnel/storage etc.
Example
{
"name": "string",
"doc": "string",
"storage": [
"string"
]
}
tesState: string
tesTask: object
Task describes an instance of a task.
- id: string
-
Task identifier assigned by the server.
- state: tesState
- name: string
- description: string
- inputs: tesInput
-
Input files. Inputs will be downloaded and mounted into the executor container.
- outputs: tesOutput
-
Output files. Outputs will be uploaded from the executor container to long-term storage.
- resources: tesResources
-
Request that the task be run with these resources.
- executors: tesExecutor
-
A list of executors to be run, sequentially. Execution stops on the first error.
- volumes: string[]
-
Volumes are directories which may be used to share data between Executors. Volumes are initialized as empty directories by the system when the task starts and are mounted at the same path in each Executor.
For example, given a volume defined at "/vol/A", executor 1 may write a file to "/vol/A/exec1.out.txt", then executor 2 may read from that file.
(Essentially, this translates to a
docker run -vflag where the container path is the same for each executor). - tags: object
-
A key-value map of arbitrary tags.
- logs: tesTaskLog
-
Task logging information. Normally, this will contain only one entry, but in the case where a task fails and is retried, an entry will be appended to this list.
- creation_time: string
-
Date + time the task was created, in RFC 3339 format. This is set by the system, not the client.
Example
{
"id": "string",
"state": "string",
"name": "string",
"description": "string",
"inputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string",
"content": "string"
}
],
"outputs": [
{
"name": "string",
"description": "string",
"url": "string",
"path": "string",
"type": "string"
}
],
"resources": {
"cpu_cores": "integer (int64)",
"preemptible": "boolean (boolean)",
"ram_gb": "number (double)",
"disk_gb": "number (double)",
"zones": [
"string"
]
},
"executors": [
{
"image": "string",
"command": [
"string"
],
"workdir": "string",
"stdin": "string",
"stdout": "string",
"stderr": "string",
"env": "object"
}
],
"volumes": [
"string"
],
"tags": "object",
"logs": [
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}
],
"creation_time": "string"
}
tesTaskLog: object
TaskLog describes logging information related to a Task.
- logs: tesExecutorLog
-
Logs for each executor
- metadata: object
-
Arbitrary logging metadata included by the implementation.
- start_time: string
-
When the task started, in RFC 3339 format.
- end_time: string
-
When the task ended, in RFC 3339 format.
- outputs: tesOutputFileLog
-
Information about all output files. Directory outputs are flattened into separate items.
- system_logs: string[]
-
System logs are any logs the system decides are relevant, which are not tied directly to an Executor process. Content is implementation specific: format, size, etc.
System logs may be collected here to provide convenient access.
For example, the system may include the name of the host where the task is executing, an error message that caused a SYSTEM_ERROR state (e.g. disk is full), etc.
System logs are only included in the FULL task view.
Example
{
"logs": [
{
"start_time": "string",
"end_time": "string",
"stdout": "string",
"stderr": "string",
"exit_code": "integer (int32)"
}
],
"metadata": "object",
"start_time": "string",
"end_time": "string",
"outputs": [
{
"url": "string",
"path": "string",
"size_bytes": "string (int64)"
}
],
"system_logs": [
"string"
]
}