Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions .claude/settings.local.json

This file was deleted.

6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ tests/lib/
.vscode/
.idea/

# Claude Code local settings (per-developer, not shared)
.claude/settings.local.json

# lazbuild-generated local package-link cache
packagefiles.xml

# OS specific files
## Windows
Thumbs.db
Expand Down
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [0.7.0] - 2026-07-11

### Added

- Error-collection API on both pools (implemented once in `TThreadPoolBase`, inherited by `TSimpleThreadPool` and `TProducerConsumerThreadPool`):
- `Errors: TStringArray` — every captured task error message, oldest first
- `ErrorCount: Integer`
- `ClearErrors` — clears the collection and `LastError`
- `OnError: TThreadPoolErrorEvent` — optional callback fired (on a worker thread) each time a task raises
- `MAX_STORED_ERRORS` constant (1000) caps the collection; oldest messages are dropped beyond it so a high volume of failing tasks cannot exhaust memory
- 8 new unit tests covering the error-collection API across both pools (collection captures all, `OnError` fires per failure, `ClearErrors` resets, cap enforced, `LastError` back-compat)
- New examples:
- `examples/SimpleErrorHandlingBasic` — the easy way: poll `Errors`/`ErrorCount`/`LastError` after `WaitForAll`, no callback or locking
- `examples/SimpleErrorHandling` — advanced error handling with the `OnError` callback and a thread-safe handler
- `examples/ParallelFileHasher` — hashes multiple files concurrently and demonstrates real file I/O failures
- `examples/ParallelUrlFetcher` — fetches multiple URLs concurrently and demonstrates real HTTP/DNS failures

### Changed

- `LastError` is unchanged and remains backward-compatible (still the most recent error)
- Removed the now-redundant per-pool `FErrorLock`; error capture is serialized by the base class, which also fires `OnError` outside its lock to avoid deadlocks
- README/API docs: documented the new `Errors`/`ErrorCount`/`OnError`/`ClearErrors` API and corrected the `LastError`-overwrite caveats; corrected an inaccurate "error messages include thread IDs" claim (messages are raw)
- README "Planned/In Progress": next milestone is a performance & robustness pass (0.8.0)

## [0.6.5] - 2026-06-11

### Added
Expand Down
84 changes: 70 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 🚀 ThreadPool for Free Pascal

[![Version](https://img.shields.io/badge/version-0.6.5-8B5CF6.svg)](CHANGELOG.md)
[![Version](https://img.shields.io/badge/version-0.7.0-8B5CF6.svg)](CHANGELOG.md)
[![License: MIT](https://img.shields.io/badge/License-MIT-1E3A8A.svg)](https://opensource.org/licenses/MIT)
[![Free Pascal](https://img.shields.io/badge/Free%20Pascal-3.2.2+-3B82F6.svg)](https://www.freepascal.org/)
[![Lazarus](https://img.shields.io/badge/Lazarus-4.0+-60A5FA.svg)](https://www.lazarus-ide.org/)
Expand Down Expand Up @@ -124,9 +124,10 @@ A thread pool with fixed-size circular buffer (1024 items) and built-in backpres
- Protected error handling

- **Error Management**
- Thread-specific error capture
- Error messages with thread IDs
- Continuous operation after exceptions
- Worker exceptions are caught automatically; the pool keeps running
- `LastError` for the most recent failure
- `Errors` collection captures **all** failed-task messages (capped, oldest dropped) — *v0.7.0*
- Optional `OnError` callback fired per failed task — *v0.7.0*

> [!NOTE]
> Thread count is determined by `TThread.ProcessorCount` at startup and remains fixed. See [Thread Management](#-thread-management) for details.
Expand Down Expand Up @@ -273,6 +274,38 @@ begin
end.
```

### Capturing all task errors (v0.7.0)

`LastError` only holds the **most recent** failure. To inspect **every** failed
task, use the `Errors` collection (oldest first, capped at `MAX_STORED_ERRORS = 1000`).
This works the same on both pools:

```pascal
var
Msg: string;
begin
Pool.ClearErrors;
for i := 0 to 9 do
Pool.Queue(@RiskyProc, i);
Pool.WaitForAll;

WriteLn(Pool.ErrorCount, ' task(s) failed:');
for Msg in Pool.Errors do
WriteLn(' - ', Msg);

Pool.ClearErrors; // resets the collection and LastError
end;
```

Prefer to react the moment a task fails (instead of polling after `WaitForAll`)?
Assign an `OnError` callback:

```pascal
// IMPORTANT: OnError is called from a worker thread. Keep the handler short and
// thread-safe; synchronize if it touches the UI or shared state.
Pool.OnError := @MyHandler.OnTaskError;
```

### Tips

> [!NOTE]
Expand Down Expand Up @@ -323,7 +356,7 @@ All four `Queue` overloads share the same pattern — pick the one that fits you
| Indexed method | `Queue(@MyObj.MyMethod, i)` | Loop parallelism + object state | Parallel array transform on an object |

> [!NOTE]
> `LastError` is **overwritten** (not appended) each time a task raises an exception. If you queue multiple tasks, only the last error is stored. Check `LastError` immediately after `WaitForAll` and call `ClearLastError` before reusing the pool.
> `LastError` holds only the **most recent** exception. To see **every** failed task, use the `Errors` collection (added in v0.7.0) — see [Capturing all task errors](#capturing-all-task-errors-v070). Call `ClearErrors` before reusing the pool.


## 📚 Examples
Expand Down Expand Up @@ -357,25 +390,44 @@ All four `Queue` overloads share the same pattern — pick the one that fits you
- Queue full handling
- Performance comparison

5. 🛡️ **Error Handling — Basic** (`examples/SimpleErrorHandlingBasic/SimpleErrorHandlingBasic.lpr`)
- **Start here** for error handling — the easy way
- Just queue, `WaitForAll`, then read `Errors`/`ErrorCount`/`LastError`
- No callback, no custom class, no locking required

6. 🛡️ **Error Handling — Advanced** (`examples/SimpleErrorHandling/SimpleErrorHandling.lpr`)
- Adds the `OnError` callback to react *while* tasks run
- Shows the thread-safe handler pattern (needed only because the handler keeps shared state)

7. 📁 **Parallel File Hasher** (`examples/ParallelFileHasher/ParallelFileHasher.lpr`)
- Hashes multiple files concurrently with `ThreadPool.Simple`
- Demonstrates real file I/O failures captured through `Errors`/`OnError`
- Creates and cleans up its own sample files

### Producer-Consumer Examples

5. 🎓 **Simple Demo** (`examples/ProdConSimpleDemo/ProdConSimpleDemo.lpr`)
1. 🎓 **Simple Demo** (`examples/ProdConSimpleDemo/ProdConSimpleDemo.lpr`)
- Basic usage with ProducerConsumerThreadPool
- Demonstrates procedures and methods
- Shows proper object lifetime

6. 🔢 **Square Numbers** (`examples/ProdConSquareNumbers/ProdConSquareNumbers.lpr`)
2. 🔢 **Square Numbers** (`examples/ProdConSquareNumbers/ProdConSquareNumbers.lpr`)
- High volume task processing
- Queue full handling
- Backpressure demonstration
- Performance monitoring

7. 📝 **Message Processor** (`examples/ProdConMessageProcessor/ProdConMessageProcessor.lpr`)
3. 📝 **Message Processor** (`examples/ProdConMessageProcessor/ProdConMessageProcessor.lpr`)
- Queue-based task processing
- Thread-safe message handling
- Graceful shutdown
- Error handling patterns

4. 🌐 **Parallel URL Fetcher** (`examples/ParallelUrlFetcher/ParallelUrlFetcher.lpr`)
- Fetches multiple URLs concurrently with `ThreadPool.ProducerConsumer`
- Demonstrates real HTTP/DNS failures captured through `Errors`/`OnError`
- Uses one HTTP client per task so clients are not shared across threads


## 🛠️ Installation

Expand Down Expand Up @@ -516,20 +568,24 @@ for i := 0 to 99 do
GlobalThreadPool.WaitForAll;
```

### 3. Only the last error is kept
### 3. `LastError` only keeps the most recent error

`LastError` is overwritten on every exception — not appended. If multiple tasks fail, you only see the last one.
`LastError` is overwritten on every exception. If multiple tasks fail and you
only check `LastError`, you see just the last one. Use the `Errors` collection
to capture all of them — see [Capturing all task errors](#capturing-all-task-errors-v070) below.

```pascal
// Queue several tasks that might fail
for i := 0 to 9 do
Pool.Queue(@RiskyProc, i);
Pool.WaitForAll;

// Only the LAST exception is in LastError
// Only the LAST exception is in LastError...
if Pool.LastError <> '' then
WriteLn('At least one task failed: ', Pool.LastError);
Pool.ClearLastError;
WriteLn('Most recent failure: ', Pool.LastError);
// ...but Errors has them all:
WriteLn(Pool.ErrorCount, ' task(s) failed in total');
Pool.ClearErrors;
```

### 4. Freeing the global pool manually
Expand All @@ -547,7 +603,7 @@ GlobalThreadPool.WaitForAll;

## 🚧 Planned/In Progress

- Richer error handling — collect all task errors (not just the last) and an optional `OnError` callback (planned for 0.7.0)
- Performance & robustness pass — event-driven idle workers (remove the Simple pool's poll loop), shutdown review, stress/soak tests (planned for 0.8.0)
- Support for `procedure Queue(AMethod: TProc; AArgs: array of Const);`
- More comprehensive tests
- More examples
Expand Down
32 changes: 30 additions & 2 deletions docs/ThreadPool.ProducerConsumer-API.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,12 +129,36 @@ Pool.WaitForAll;
if Pool.LastError <> '' then
begin
// LastError holds the raw message of the most recent worker exception.
// Earlier failures are overwritten — only the last one is available.
WriteLn('Task error: ', Pool.LastError);
Pool.ClearLastError; // reset before reuse
end;
```

To inspect **every** task failure (not just the last), use the `Errors`
collection added in v0.7.0 — oldest first, capped at `MAX_STORED_ERRORS = 1000`:

```pascal
var
Msg: string;
begin
Pool.ClearErrors;
// ... queue tasks ...
Pool.WaitForAll;

WriteLn(Pool.ErrorCount, ' task(s) failed:');
for Msg in Pool.Errors do
WriteLn(' - ', Msg);
Pool.ClearErrors; // resets the collection and LastError
end;
```

Or assign `OnError` to react the moment a task fails, instead of polling:

```pascal
// Called from a worker thread — keep it short and thread-safe.
Pool.OnError := @Handler.OnTaskError;
```

### Full pattern

```pascal
Expand Down Expand Up @@ -167,10 +191,14 @@ end;
```pascal
property ThreadCount: Integer; // read-only; number of worker threads
property LastError: string; // read-only; last worker exception message
property Errors: TStringArray; // read-only; all captured messages (capped)
property ErrorCount: Integer; // read-only; count of messages in Errors
property OnError: TThreadPoolErrorEvent; // fired (on a worker thread) per failed task
property WorkQueue: TThreadSafeQueue; // access to queue for monitoring/config

procedure WaitForAll;
procedure ClearLastError;
procedure ClearErrors; // clears both Errors and LastError
```

---
Expand Down Expand Up @@ -324,7 +352,7 @@ end;
## Limitations

- Fixed queue capacity — no dynamic resizing
- Only the most recent worker exception is stored in `LastError`
- `LastError` holds only the most recent worker exception (use `Errors` to collect all; capped at `MAX_STORED_ERRORS`)
- No task priority or cancellation support
- No dynamic thread scaling after construction
- Not suitable for real-time or UI-thread work
52 changes: 49 additions & 3 deletions docs/ThreadPool.Simple-API.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,6 @@ begin
if Pool.LastError <> '' then
begin
// LastError holds the raw exception message of the most recent failure.
// If multiple tasks fail, only the last exception is stored.
WriteLn('Error: ', Pool.LastError);
Pool.ClearLastError; // Reset before reusing the pool
end;
Expand All @@ -128,11 +127,57 @@ begin
end;
```

### Collecting all errors (v0.7.0)

`LastError` only holds the most recent failure. To inspect **every** task error,
use the `Errors` collection (oldest first, capped at `MAX_STORED_ERRORS = 1000`):

```pascal
var
Msg: string;
begin
Pool.ClearErrors;
for i := 0 to N - 1 do
Pool.Queue(@RiskyProcedure);
Pool.WaitForAll;

WriteLn(Pool.ErrorCount, ' task(s) failed:');
for Msg in Pool.Errors do
WriteLn(' - ', Msg);

Pool.ClearErrors; // resets the collection and LastError
end;
```

### Reacting to errors as they happen (v0.7.0)

Assign `OnError` to be notified the moment a task fails, instead of polling after
`WaitForAll`:

```pascal
type
TMyHandler = class
procedure OnTaskError(const AMessage: string);
end;

procedure TMyHandler.OnTaskError(const AMessage: string);
begin
// NOTE: called from a worker thread. Keep it short and thread-safe;
// synchronize if you touch the UI or shared state.
Log('task failed: ' + AMessage);
end;

Pool.OnError := @Handler.OnTaskError;
```

### Properties

```pascal
property LastError: string; // Raw message of the most recent worker exception
property ThreadCount: Integer; // Number of worker threads (read-only)
property LastError: string; // Raw message of the most recent worker exception
property Errors: TStringArray; // All captured messages (oldest first, capped)
property ErrorCount: Integer; // Number of messages currently in Errors
property OnError: TThreadPoolErrorEvent; // Fired (on a worker thread) per failed task
property ThreadCount: Integer; // Number of worker threads (read-only)
```

### Methods
Expand All @@ -144,6 +189,7 @@ procedure Queue(AProcedure: TThreadProcedureIndex; AIndex: Integer);
procedure Queue(AMethod: TThreadMethodIndex; AIndex: Integer);
procedure WaitForAll;
procedure ClearLastError;
procedure ClearErrors; // clears both Errors and LastError
```

---
Expand Down
File renamed without changes.
Loading
Loading