Skip to content

Commit 89e1931

Browse files
imthatdevimthatdev
committed
feat: enhance storage and jobs management, along other features
1 parent 011286a commit 89e1931

133 files changed

Lines changed: 17037 additions & 2951 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,41 @@ This project follows [Semantic Versioning](https://iconical.dev/versioning).
1111

1212
---
1313

14+
v1.2.0 – Runtime & Queue Intelligence ⚙️
15+
16+
Released: April 12, 2026
17+
18+
### ⚡ Performance & Observability
19+
- Introduced runtime signals tracking for monitoring event loop lag and API latency.
20+
- Implemented server settings caching to reduce repeated lookups and improve response times.
21+
- Added Redis caching for API key lookups and settings retrieval to speed up critical paths.
22+
- Added health check endpoints for API and job queue monitoring.
23+
24+
### 🔄 Jobs & Processing
25+
- Enhanced Jobs handling with retry logic and dead-letter queue support.
26+
- Improved storage cleanup jobs with execution validation and state awareness.
27+
- Added new admin job for automated PWA subscription cleanup.
28+
29+
### 🔖 Bookmarks & RSS Auto-Hoard
30+
- Added Auto-Hoard RSS feed management for Bookmarks with feed create/update/delete, run-now, and schedule controls.
31+
- Added bookmark RSS API routes and worker runner integration so feeds are queued and processed in the background.
32+
- Added a Bookmarks header tools menu with quick actions for Import/Export and Auto-Hoard RSS dialogs.
33+
- Updated RSS management UX so Add RSS feed opens in a dedicated modal separate from the configured-feeds screen.
34+
- Hardened RSS feed settings by normalizing `maxItemsPerFetch` to a clamped integer before persistence.
35+
- Export and Import Bookmarks with ease using JSON and HTML files, making it simple to backup or transfer your bookmarks.
36+
37+
### 📦 Storage & Infrastructure
38+
- Strengthened storage configuration with socket limits and acquisition timeouts.
39+
- Improved storage handling with path traversal protection and safer socket management.
40+
- Enhanced yt-dlp processing with better file naming and optimized threading.
41+
- Docker runs swush app and swush worker as separate services for better scalability and reliability.
42+
43+
### 🧠 Internal & Types
44+
- Introduced admin queue health types for monitoring job system metrics.
45+
- Updated schema definitions for improved API key handling.
46+
47+
---
48+
1449
## v1.1.1 – URL Safety Patch 🔐
1550

1651
**Released: February 25, 2026**

‎README.md‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -102,17 +102,28 @@ Open [http://localhost:3000](http://localhost:3000) in your browser.
102102

103103
> **Note:** The Docker setup has been recently updated and optimized for smaller image size and faster builds.
104104
105+
Compose files now run two services by default:
106+
- `app` (`JOB_RUNNER_ROLE=api`) serves HTTP/API traffic.
107+
- `worker` (`JOB_RUNNER_ROLE=worker`) handles heavy background jobs (transcoding, previews, remote uploads, exports, cleanup).
108+
109+
This keeps API latency stable while long-running jobs are isolated.
110+
105111
### Build and run with remote database (Neon, Supabase, etc.)
106112
```bash
107-
docker compose up -d --build
113+
docker compose -f docker/docker-compose.remote.yml up -d --build
108114
```
109115

110116
### Run with self-hosted PostgreSQL
111117
```bash
112-
docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d --build
118+
docker compose -f docker/docker-compose.yml up -d --build
119+
```
120+
121+
### Scale workers independently
122+
```bash
123+
docker compose -f docker/docker-compose.yml up -d --scale worker=2
113124
```
114125

115-
- The app will be accessible at [http://localhost:3000](http://localhost:3000).
126+
- The app will be accessible at [http://localhost:3419](http://localhost:3419).
116127
- PostgreSQL will be exposed on port `5432` (credentials configured in `.env`).
117128

118129
## ☁️ Cloudflare Cache Rules (HLS Only)

‎docker/docker-compose.remote-windows.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,32 @@
11
services:
2+
redis:
3+
image: redis:7-alpine
4+
container_name: swush-redis
5+
restart: unless-stopped
6+
command: ["redis-server", "--appendonly", "yes"]
7+
healthcheck:
8+
test: ["CMD", "redis-cli", "ping"]
9+
interval: 5s
10+
timeout: 5s
11+
retries: 20
12+
volumes:
13+
- swush-redis:/data:rw
14+
networks:
15+
- swush
16+
217
app:
318
image: iconical/swush:latest
419
container_name: swush-app
520
restart: unless-stopped
621
env_file:
722
- ./.env
23+
environment:
24+
JOB_RUNNER_ROLE: api
25+
RUN_DB_MIGRATIONS: "true"
26+
REDIS_URL: "redis://redis:6379/0"
27+
depends_on:
28+
redis:
29+
condition: service_healthy
830
ports:
931
- "3419:3000"
1032
volumes:
@@ -13,5 +35,26 @@ services:
1335
networks:
1436
- swush
1537

38+
worker:
39+
image: iconical/swush:latest
40+
container_name: swush-worker
41+
restart: unless-stopped
42+
env_file:
43+
- ./.env
44+
environment:
45+
JOB_RUNNER_ROLE: worker
46+
RUN_DB_MIGRATIONS: "false"
47+
REDIS_URL: "redis://redis:6379/0"
48+
depends_on:
49+
redis:
50+
condition: service_healthy
51+
volumes:
52+
- D:/selfhosted/swush/uploads:/data/uploads:rw
53+
networks:
54+
- swush
55+
56+
volumes:
57+
swush-redis:
58+
1659
networks:
1760
swush:

‎docker/docker-compose.remote.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,53 @@
11
services:
2+
redis:
3+
image: redis:7-alpine
4+
container_name: swush-redis
5+
restart: unless-stopped
6+
command: ["redis-server", "--appendonly", "yes"]
7+
healthcheck:
8+
test: ["CMD", "redis-cli", "ping"]
9+
interval: 5s
10+
timeout: 5s
11+
retries: 20
12+
volumes:
13+
- swush-redis:/data:rw
14+
215
app:
316
image: iconical/swush:latest
417
container_name: swush-app
518
restart: unless-stopped
619
env_file:
720
- ./.env
21+
environment:
22+
JOB_RUNNER_ROLE: api
23+
RUN_DB_MIGRATIONS: "true"
24+
REDIS_URL: "redis://redis:6379/0"
25+
depends_on:
26+
redis:
27+
condition: service_healthy
828

929
ports:
1030
- "3419:3000"
1131
volumes:
1232
# Can be removed if you don't want to persist uploads outside the container or using S3 for uploads
1333
- swush-uploads:/data/uploads:rw
1434

35+
worker:
36+
image: iconical/swush:latest
37+
container_name: swush-worker
38+
restart: unless-stopped
39+
env_file:
40+
- ./.env
41+
environment:
42+
JOB_RUNNER_ROLE: worker
43+
RUN_DB_MIGRATIONS: "false"
44+
REDIS_URL: "redis://redis:6379/0"
45+
depends_on:
46+
redis:
47+
condition: service_healthy
48+
volumes:
49+
- swush-uploads:/data/uploads:rw
50+
1551
volumes:
1652
swush-uploads:
53+
swush-redis:

‎docker/docker-compose.synology.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,36 @@ services:
1717
networks:
1818
- swush
1919

20+
redis:
21+
image: redis:7-alpine
22+
container_name: swush-redis
23+
restart: unless-stopped
24+
command: ["redis-server", "--appendonly", "yes"]
25+
healthcheck:
26+
test: ["CMD", "redis-cli", "ping"]
27+
interval: 5s
28+
timeout: 5s
29+
retries: 20
30+
volumes:
31+
- swush-redis:/data:rw
32+
networks:
33+
- swush
34+
2035
app:
2136
image: iconical/swush:latest
2237
container_name: swush-app
2338
restart: unless-stopped
2439
depends_on:
2540
db:
2641
condition: service_healthy
42+
redis:
43+
condition: service_healthy
2744
env_file:
2845
- ./.env
46+
environment:
47+
JOB_RUNNER_ROLE: api
48+
RUN_DB_MIGRATIONS: "true"
49+
REDIS_URL: "redis://redis:6379/0"
2950

3051
ports:
3152
- "3419:3000"
@@ -35,5 +56,28 @@ services:
3556
networks:
3657
- swush
3758

59+
worker:
60+
image: iconical/swush:latest
61+
container_name: swush-worker
62+
restart: unless-stopped
63+
depends_on:
64+
db:
65+
condition: service_healthy
66+
redis:
67+
condition: service_healthy
68+
env_file:
69+
- ./.env
70+
environment:
71+
JOB_RUNNER_ROLE: worker
72+
RUN_DB_MIGRATIONS: "false"
73+
REDIS_URL: "redis://redis:6379/0"
74+
volumes:
75+
- /volume1/docker/swush/uploads:/data/uploads:rw
76+
networks:
77+
- swush
78+
79+
volumes:
80+
swush-redis:
81+
3882
networks:
3983
swush:

‎docker/docker-compose.windows.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,36 @@ services:
1818
networks:
1919
- swush
2020

21+
redis:
22+
image: redis:7-alpine
23+
container_name: swush-redis
24+
restart: unless-stopped
25+
command: ["redis-server", "--appendonly", "yes"]
26+
healthcheck:
27+
test: ["CMD", "redis-cli", "ping"]
28+
interval: 5s
29+
timeout: 5s
30+
retries: 20
31+
volumes:
32+
- swush-redis:/data:rw
33+
networks:
34+
- swush
35+
2136
app:
2237
image: iconical/swush:latest
2338
container_name: swush-app
2439
restart: unless-stopped
2540
depends_on:
2641
db:
2742
condition: service_healthy
43+
redis:
44+
condition: service_healthy
2845
env_file:
2946
- ./.env
47+
environment:
48+
JOB_RUNNER_ROLE: api
49+
RUN_DB_MIGRATIONS: "true"
50+
REDIS_URL: "redis://redis:6379/0"
3051

3152
ports:
3253
- "3419:3000"
@@ -36,5 +57,28 @@ services:
3657
networks:
3758
- swush
3859

60+
worker:
61+
image: iconical/swush:latest
62+
container_name: swush-worker
63+
restart: unless-stopped
64+
depends_on:
65+
db:
66+
condition: service_healthy
67+
redis:
68+
condition: service_healthy
69+
env_file:
70+
- ./.env
71+
environment:
72+
JOB_RUNNER_ROLE: worker
73+
RUN_DB_MIGRATIONS: "false"
74+
REDIS_URL: "redis://redis:6379/0"
75+
volumes:
76+
- "D:/selfhosted/swush/uploads:/data/uploads:rw"
77+
networks:
78+
- swush
79+
80+
volumes:
81+
swush-redis:
82+
3983
networks:
4084
swush:

‎docker/docker-compose.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,36 @@ services:
1717
networks:
1818
- swush
1919

20+
redis:
21+
image: redis:7-alpine
22+
container_name: swush-redis
23+
restart: unless-stopped
24+
command: ["redis-server", "--appendonly", "yes"]
25+
healthcheck:
26+
test: ["CMD", "redis-cli", "ping"]
27+
interval: 5s
28+
timeout: 5s
29+
retries: 20
30+
volumes:
31+
- swush-redis:/data:rw
32+
networks:
33+
- swush
34+
2035
app:
2136
image: iconical/swush:latest
2237
container_name: swush-app
2338
restart: unless-stopped
2439
depends_on:
2540
db:
2641
condition: service_healthy
42+
redis:
43+
condition: service_healthy
2744
env_file:
2845
- ./.env
46+
environment:
47+
JOB_RUNNER_ROLE: api
48+
RUN_DB_MIGRATIONS: "true"
49+
REDIS_URL: "redis://redis:6379/0"
2950

3051
ports:
3152
- "3419:3000"
@@ -35,9 +56,30 @@ services:
3556
networks:
3657
- swush
3758

59+
worker:
60+
image: iconical/swush:latest
61+
container_name: swush-worker
62+
restart: unless-stopped
63+
depends_on:
64+
db:
65+
condition: service_healthy
66+
redis:
67+
condition: service_healthy
68+
env_file:
69+
- ./.env
70+
environment:
71+
JOB_RUNNER_ROLE: worker
72+
RUN_DB_MIGRATIONS: "false"
73+
REDIS_URL: "redis://redis:6379/0"
74+
volumes:
75+
- swush-uploads:/data/uploads:rw
76+
networks:
77+
- swush
78+
3879
volumes:
3980
swush-db:
4081
swush-uploads:
82+
swush-redis:
4183

4284
networks:
4385
swush:

‎docker/entrypoint.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/bin/sh
22
set -e
33

4-
if [ -n "${DATABASE_URL:-}" ]; then
4+
if [ "${RUN_DB_MIGRATIONS:-true}" != "false" ] && [ -n "${DATABASE_URL:-}" ]; then
55
echo "Running database push..."
66
pnpm db:migrate
77
fi

0 commit comments

Comments
 (0)