Introduction
Getting Started
Installation
Step-by-step guide to getting you up and running.
Architecture
Take a look under the hood.
API reference
Learn how to interact with Tork using its REST API.
Extend Tork
Extend Tork for your particular use case.
Features
- REST API
- Horizontally scalable
- Task isolation - tasks are executed within a container to provide isolation, idempotency, and in order to enforce resource limits
- Automatic recovery of tasks in the event of a worker crash
- Supports both stand-alone and distributed setup
- Retry failed tasks
- No single point of failure
- Task timeout
- Full-text search
- Runtime agnostic.
- Middleware
- Expression Language
- Conditional Tasks
- Parallel Task
- Each Task
- Subjob Task
- Pre/Post tasks
- Web UI
Quick start
Ensure you have Docker with API Version >= 1.42 (use
docker version | grep API
to check).Download the binary for your system from the releases page.
Hello World
Start Tork in standalone
mode:
./tork run standalone
Create a file called hello.yaml
with the following contents:
# hello.yaml
---
name: hello job
tasks:
- name: say hello
image: ubuntu:mantic #docker image
run: |
echo -n hello world
- name: say goodbye
image: ubuntu:mantic
run: |
echo -n bye world
Submit the job in another terminal window:
JOB_ID=$(curl -s -X POST --data-binary @hello.yaml \
-H "Content-type: text/yaml" http://localhost:8000/jobs | jq -r .id)
Query for the status of the job:
curl -s http://localhost:8000/jobs/$JOB_ID
{
"id": "ed0dba93d262492b8cf26e6c1c4f1c98",
"state": "COMPLETED",
...
}
A slightly more interesting example
The following job:
- Downloads a remote video file using a pre task to a shared
/tmp
volume. - Converts the first 5 seconds of the downloaded video using ffmpeg.
- Uploads the converted video to a destination using a post task.
# convert.yaml
---
name: convert a video
inputs:
source: https://upload.wikimedia.org/wikipedia/commons/1/18/Big_Buck_Bunny_Trailer_1080p.ogv
tasks:
- name: convert the first 5 seconds of a video
image: jrottenberg/ffmpeg:3.4-alpine
run: |
ffmpeg -i /tmp/input.ogv -t 5 /tmp/output.mp4
mounts:
- type: volume
target: /tmp
pre:
- name: download the remote file
image: alpine:3.18.3
env:
SOURCE_URL: '{{ inputs.source }}'
run: |
wget \
$SOURCE_URL \
-O /tmp/input.ogv
post:
- name: upload the converted file
image: alpine:3.18.3
run: |
wget \
--post-file=/tmp/output.mp4 \
https://devnull-as-a-service.com/dev/null
Submit the job in another terminal window:
JOB_ID=$(curl -s -X POST --data-binary @convert.yaml \
-H "Content-type: text/yaml" http://localhost:8000/jobs | jq -r .id)
More examples
Check out the examples folder.