TaskQ v2

Scheduling task queues for web services — execute HTTP requests, shell commands, and more on a schedule.

Features

Multi-type tasks: HTTP requests, shell command execution
Pluggable storage: JSON files, Redis, SQLite, MySQL, PostgreSQL
Flexible scheduling: One-time, repeating, cron-like intervals, start-at
Channel isolation: Group tasks into independent queues
Pause/Resume: Pause individual tasks and resume them later
Failure handling: Automatic retries with configurable delay + failure/success callbacks
Persistence: Tasks survive restarts (all storage backends)
Real-time UI: Vue 3 SPA with SSE live updates, pause/resume, add/edit/delete tasks
Export/Import: Transfer tasks between servers via JSON export/import
YAML config + environment variable overrides + CLI flags
Graceful shutdown: Waits for in-flight tasks to complete
Auth: Token-based API authentication
Health endpoint: Public health check

Quick Start

# Build from source
git clone https://github.com/mindblowup/taskq
cd taskq
go build -o taskq ./cmd/taskq/

# Build the GUI
cd gui && npm install && npm run build && cd ..

# Run (secret auto-generated, GUI served at /gui/)
./taskq --gui-dir ./gui/dist

Installation

From Source (Go 1.25+ required)

git clone https://github.com/mindblowup/taskq
cd taskq
go build -o taskq ./cmd/taskq/

Docker

docker build -t taskq .
docker run -p 8001:8001 taskq

Configuration

TaskQ supports three configuration layers (later overrides earlier):

Defaults → 2. YAML config file → 3. Environment variables → 4. CLI flags

CLI Flags

./taskq [flags]

  -clear                    Clear all uncompleted tasks on startup
  -config string            Path to YAML config file
  -dsn string               Store DSN (for SQL backends)
  -error_log string         Error log file path (default "./taskq_error.log")
  -failure_callback string  URL called on permanent failure
  -gui-dir string           Path to GUI static files directory
  -listen string            HTTP listen address (default ":8001")
  -redis-addr string        Redis server address
  -redis-db int             Redis database number
  -redis-password string    Redis password
  -retry int                Number of retries on failure (default 3)
  -retry_delay int          Seconds between retries (default 30)
  -secret string            Secret token for API auth
  -store string             Store backend (json, redis, sqlite, mysql, postgres)
  -store-path string        JSON store directory path (default ".taskq")
  -success_callback string  URL called on successful execution
  -timeout int              HTTP request/command timeout in seconds (default 20)

YAML Config File

# example_config.yaml
listen: ":8001"
secret: ""                          # auto-generated if empty

store:
  type: "json"                      # json | redis | sqlite | mysql | postgres
  path: ".taskq"                    # JSON store directory
  addr: "localhost:6379"            # Redis address
  password: ""                      # Redis password
  db: 0                             # Redis database number
  dsn: ""                           # SQL DSN
  table_name: "tasks"               # SQL table name

timeout: 20
retry: 3
retry_delay: 30
failure_callback: ""
success_callback: ""
error_log: "./taskq_error.log"
gui_dir: "./gui/dist"

./taskq --config ./example_config.yaml

Environment Variables

Variable	Overrides
`TASKQ_LISTEN`	`listen`
`TASKQ_SECRET`	`secret`
`TASKQ_STORE_TYPE`	`store.type`
`TASKQ_DSN`	`store.dsn`
`TASKQ_REDIS_ADDR`	`store.addr`
`TASKQ_REDIS_PASSWORD`	`store.password`

Storage Backends

JSON File (default)

No external dependencies. Stores tasks as individual JSON files.

./taskq --store json --store-path .taskq

Redis

./taskq --store redis --redis-addr localhost:6379

SQLite

./taskq --store sqlite --dsn "file:tasks.db?cache=shared"

MySQL

./taskq --store mysql --dsn "user:pass@tcp(localhost:3306)/taskq?parseTime=true"

PostgreSQL

./taskq --store postgres --dsn "host=localhost port=5432 user=postgres dbname=taskq sslmode=disable"

Task Types

HTTP Task

Executes an HTTP request to the specified URL.

{
  "name": "send-email",
  "type": "http",
  "url": "https://api.example.com/send",
  "method": "POST",
  "headers": {
    "Authorization": "Bearer token123"
  },
  "data": {
    "to": "user@example.com",
    "subject": "Hello"
  },
  "options": {
    "repeat": 3,
    "every": 3600,
    "startAt": 1740000000
  }
}

Fields:

Field	Type	Required	Description
`name`	string	yes	Task name
`type`	string	no	`"http"` (default) or `"command"`
`url`	string	yes (HTTP)	Full URL for the request
`method`	string	no	HTTP method (default: `"POST"`)
`headers`	object	no	HTTP headers
`data`	object	no	JSON body sent with the request
`options`	object	no	See Options

Command Task

Executes a shell command on the server.

{
  "name": "backup-db",
  "type": "command",
  "command": "/usr/local/bin/backup.sh",
  "args": ["--database", "mydb"],
  "dir": "/opt/backups",
  "env": {
    "AWS_ACCESS_KEY_ID": "xxx"
  },
  "options": {
    "every": 86400,
    "repeat": 0
  }
}

Fields:

Field	Type	Required	Description
`name`	string	yes	Task name
`type`	string	yes	`"command"`
`command`	string	yes	Command to execute
`args`	array	no	Command arguments
`dir`	string	no	Working directory
`env`	object	no	Additional environment variables
`options`	object	no	See Options

Options

Option	Type	Default	Description
`method`	string	`"POST"`	HTTP method (GET, POST, PUT, DELETE, etc.)
`repeat`	int	`1`	Repeat count. `0` = forever. `1` = once
`every`	int	`0`	Seconds between repeats
`startAt`	int	`now`	Unix timestamp to start execution
`timeout`	int	`20`	Timeout in seconds per execution
`retry`	int	`3`	Retry count on failure
`retry_delay`	int	`30`	Seconds between retries
`failure_callback`	string	`""`	URL called (POST) on permanent failure
`success_callback`	string	`""`	URL called (POST) on success

API

All endpoints except /health require authentication via ?secret=<token>.

Health Check (public)

GET /health

Response:

{"status": true, "data": "ok"}

Add Task

POST /add-task?secret=<token>
Content-Type: application/json

{
  "name": "my-task",
  "type": "http",
  "url": "https://example.com/api",
  "data": {"key": "value"},
  "options": {"every": 60, "repeat": 10}
}

Or via channel grouping:

POST /add-task?channel=email&secret=<token>

Response:

{
  "status": true,
  "data": {
    "id": "abc123",
    "name": "my-task",
    "channel": "default",
    "type": "http"
  }
}

List Tasks

GET /list?secret=<token>
GET /list?channel=myChannel&secret=<token>

Update Task

POST /update-task?id=<taskId>&channel=myChannel&secret=<token>
Content-Type: application/json

{
  "name": "updated-name",
  "url": "https://new-url.example.com",
  "options": {"every": 120}
}

Only provided fields are updated; runtime fields (last_exec, next_exec, etc.) are preserved.

Remove Task

DELETE /remove-task?id=<taskId>&secret=<token>
DELETE /remove-task?id=<taskId>&channel=myChannel&secret=<token>

Pause / Resume Task

GET /pause-task?id=<taskId>&channel=myChannel&secret=<token>
GET /resume-task?id=<taskId>&channel=myChannel&secret=<token>

Clear Channel

DELETE /clear?secret=<token>
DELETE /clear?channel=myChannel,otherChannel&secret=<token>

Export Tasks

GET /export?secret=<token>

Returns all tasks grouped by channel:

{
  "status": true,
  "data": {
    "version": 1,
    "tasks": {
      "default": [{ "name": "task1", ... }],
      "email": [{ "name": "task2", ... }]
    }
  }
}

Import Tasks

POST /import?secret=<token>
Content-Type: application/json

{
  "tasks": {
    "default": [{ "name": "task1", "type": "http", "url": "...", "options": {...} }]
  }
}

All imported tasks get new IDs and timestamps on the server.

Real-time Events (SSE)

GET /events?secret=<token>

Server-Sent Events stream. Each message is a JSON event:

{"type": "task_added", "channel": "default", "task_id": "abc123", "task": {...}}

Event types: task_added, task_updated, task_removed, task_paused, task_resumed, task_completed, task_failed, channel_cleared.

Legacy Endpoints (backward compatible)

POST /add-http-task → same as POST /add-task
DELETE /remove-http-task → same as DELETE /remove-task

GUI

TaskQ ships with a Vue 3 single-page application served at /gui/. Features:

Dashboard with channel-organized task cards
Real-time updates via SSE (no polling)
Add/Edit/Delete tasks with type-specific forms
Pause/Resume individual tasks
Export/Import tasks as JSON files
Live countdown for next/previous execution times

Building the GUI

cd gui
npm install
npm run build     # production build to dist/
npm run dev       # dev server with API proxy at /api

Serve with TaskQ:

./taskq --gui-dir ./gui/dist

Channels

Channels let you group related tasks together. Each channel operates independently.

# Add task to "email" channel
POST /add-task?channel=email&secret=<token>

# List tasks in "email" channel
GET /list?channel=email&secret=<token>

# Clear all tasks in "email" channel
DELETE /clear?channel=email&secret=<token>

Default channel is "default" if none specified.

Architecture

┌──────────────────────────────────────────────────────────┐
│                    cmd/taskq/main.go                      │
│   flag.Parse → config.Load → createStore → scheduler     │
│   → registerExecutors → loadTasks → start → signal.Wait  │
└──────────────────────────┬───────────────────────────────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
      ┌────────────┐ ┌──────────┐ ┌──────────┐
      │   Server    │ │Scheduler │ │  Store   │
      │  (routes,  │◄┤(execute, │◄┤(persist) │
      │  auth,SSE) │ │ publish) │ │          │
      └────────────┘ └──────────┘ └──────────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
      ┌────────────┐ ┌──────────┐ ┌──────────┐
      │   HTTP     │ │ Command  │ │ Future   │
      │  Executor  │ │ Executor │ │Executors │
      └────────────┘ └──────────┘ └──────────┘
                           │
              ┌────────────┘
              ▼
      ┌──────────────┐
      │ Event Broker │──► SSE clients
      │  (pub/sub)   │
      └──────────────┘

Packages

Package	Description
`cmd/taskq/`	Entry point
`internal/config/`	Configuration loading (flags, YAML, env)
`internal/store/`	Store interface + JSON/Redis/SQL backends
`internal/task/`	Task model, scheduling logic, options
`internal/executor/`	Executor interface + HTTP/Command executors
`internal/scheduler/`	Task lifecycle management, concurrency, event broker
`internal/server/`	HTTP server, routes, middleware, auth, SSE
`internal/log/`	Structured logging (slog)

Development

Running Tests

# All unit tests
go test -v -count=1 ./internal/...

# Integration tests (SQLite)
go test -v -count=1 -tags=integration ./internal/store/

Building

go build -o taskq ./cmd/taskq/

Docker

docker build -t taskq .
docker run -p 8001:8001 -v $(pwd)/data:/data taskq --store-path /data

Examples

Periodic Database Backup

curl -s -X POST "http://localhost:8001/add-task?secret=test123" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "pg-backup",
    "type": "command",
    "command": "pg_dump",
    "args": ["-U", "postgres", "mydb"],
    "dir": "/backups",
    "env": {"PGPASSWORD": "secret"},
    "options": {
      "every": 86400,
      "repeat": 0
    }
  }'

Webhook with Retry

curl -s -X POST "http://localhost:8001/add-task?secret=test123" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "webhook-order-123",
    "type": "http",
    "url": "https://hooks.example.com/order-created",
    "method": "POST",
    "data": {"order_id": 123, "total": 49.99},
    "options": {
      "timeout": 10,
      "retry": 5,
      "retry_delay": 60,
      "failure_callback": "https://alerts.example.com/task-failed"
    }
  }'

Export / Import Between Servers

# Export from server A
curl -s "http://server-a:8001/export?secret=token" > tasks.json

# Import to server B
curl -s -X POST "http://server-b:8001/import?secret=token" \
  -H "Content-Type: application/json" \
  -d @tasks.json

Migration from v1

v2 introduces breaking changes:

Config: JSON file storage path changed from .taskq to configurable via --store-path
API: New /add-task endpoint replacing /add-http-task (legacy still works)
Single tasks: The API now accepts a single task object instead of always an array
Dependencies: JSON file storage no longer uses jsondb library

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
gui		gui
internal		internal
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
example_config.yaml		example_config.yaml
go.mod		go.mod
go.sum		go.sum

Folders and files

Latest commit

History

Repository files navigation

TaskQ v2

Features

Quick Start

Installation

From Source (Go 1.25+ required)

Docker

Configuration

CLI Flags

YAML Config File

Environment Variables

Storage Backends

JSON File (default)

Redis

SQLite

MySQL

PostgreSQL

Task Types

HTTP Task

Command Task

Options

API

Health Check (public)

Add Task

List Tasks

Update Task

Remove Task

Pause / Resume Task

Clear Channel

Export Tasks

Import Tasks

Real-time Events (SSE)

Legacy Endpoints (backward compatible)

GUI

Building the GUI

Channels

Architecture

Packages

Development

Running Tests

Building

Docker

Examples

Periodic Database Backup

Webhook with Retry

Export / Import Between Servers

Migration from v1

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 1

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages