Nightly build 2026-06-06

HyperQueue dev

Artifact summary:

• hq-vdev-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-dev-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.26.2

HyperQueue 0.26.2

Fixes

• Updated Cargo.lock. No visible change if you are using precompiled binaries.

Artifact summary:

• hq-v0.26.2-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.26.2-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.26.1

HyperQueue 0.26.1

Fixes

• Fixed a rare server crash when self retracting occurs.

New features

• Added scrolling to CPU table

Artifact summary:

• hq-v0.26.1-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.26.1-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.26.0

HyperQueue 0.26.0

Breaking change

• min-utilization now considers only CPUs, not other resources

New features

• Complete rewrite of the main server scheduler, now based on MILP. This includes major rewrite of worker scheduler. It brings the following new features:
  • Better resource allocation in complex situations
  • Better handling resource variants
  • Correctly solves priorities
  • Better scheduler decisions in situations when single node & multi node tasks are mixed
  • Better handling of situations when a worker cannot run a task because of strict resource requests of resource fragmentation
  • Reduced the number of messages between server/worker when there are many tasks of the same resource requests
  • Non-delayed reaction when a pre-assigned tasks cannot be run because of time request
• Resource weights (configured via --weight). They are used to adjust priority between different tasks or between
  multiple resource request variants within the same task.
• Worker can be started with --min-utilization (so not only through autoalloc).
  Min-utilization is now respected also in main scheduler (no only autoalloc scheduler)
• New "aborted" task state introduced; it is derived from "cancel" state.
  When HQ cancels a task (e.g. because dependency fails), it is now marked as "aborted";
  Task state "canceled" is now reserved for tasks that are canceled by the user.

Artifact summary:

• hq-v0.26.0-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.26.0-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.26.0-rc1

HyperQueue 0.26.0-rc1

Breaking change

• min-utilization now considers only CPUs, not other resources

New features

• Complete rewrite of the main server scheduler, now based on MILP. This includes major rewrite of worker scheduler. It brings the following new features:
  • Better resource allocation in complex situations
  • Better handling resource variants
  • Correctly solves priorities
  • Better scheduler decisions in situations when single node & multi node tasks are mixed
  • Better handling of situations when a worker cannot run a task because of strict resource requests of resource fragmentation
  • Reduced the number of messages between server/worker when there are many tasks of the same resource requests
  • Non-delayed reaction when a pre-assigned tasks cannot be run because of time request
• Resource weights (configured via --weight). They are used to adjust priority between different tasks or between
  multiple resource request variants within the same task.
• Worker can be started with --min-utilization (so not only through autoalloc).
  Min-utilization is now respected also in main scheduler (no only autoalloc scheduler)
• New "aborted" task state introduced; it is derived from "cancel" state.
  When HQ cancels a task (e.g. because dependency fails), it is now marked as "aborted";
  Task state "canceled" is now reserved for tasks that are canceled by the user.

Artifact summary:

• hq-v0.26.0-rc1-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.26.0-rc1-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.25.1

HyperQueue 0.25.1

Fixes

• Fixes multi-node tasks defined in job file.
• Fix selection of MILP solver when building from source

Artifact summary:

• hq-v0.25.1-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.25.1-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.25.0

HyperQueue 0.25.0

Breaking change

• hq job wait now returns non-zero exit code only in case of infrastructure error (e.g. the server is lost).
  If any waited tasks fails or is canceled, it is still returned 0.

New features

• Client notification mechanism. A task can generate a notification,
  which the process that submitted the task can capture.
• hq journal report that exports a statistics into an HTML report
• Resource coupling is now stabilized. Worker resources may now define arbitrary weights between groups.
• Server scheduler more respects priorities
• Server scheduler improved when resource variants are used
• Extended events:
  • New journal events: job-idle, task-notify.
  • worker-new now contains the worker configuration,
  • job-created now contains task information
  • task-started now contains a resource variant id
• We now ensure that after a successful modifying client's operation (submit, cancel, open/close job, queues
  modification),
  the operation is immediately a part of the written journal.
• hq worker info can now be used with a selector, to display information about multiple workers at once.
  • Note that this is a breaking change for the JSON format, as it now outputs the worker infos as an array of objects. Before it was a single object.

Changes

• hq job progress stopped using periodic polling.
  In the current version, the client passively waits for events.
  It saves some resources as the server is not queried every second.

Artifact summary:

• hq-v0.25.0-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.25.0-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.25.0-rc1

HyperQueue 0.25.0-rc1

Breaking change

• hq job wait now returns non-zero exit code only in case of infrastructure error (e.g. the server is lost).
  If any waited tasks fails or is canceled, it is still returned 0.

New features

• Client notification mechanism. A task can generate a notification,
  which the process that submitted the task can capture.
• hq journal report that exports a statistics into an HTML report
• Resource coupling is now stabilized. Worker resources may now define arbitrary weights between groups.
• Server scheduler more respects priorities
• Server scheduler improved when resource variants are used
• Extended events:
  • New journal events: job-idle, task-notify.
  • worker-new now contains the worker configuration,
  • job-created now contains task information
  • task-started now contains a resource variant id
• We now ensure that after a successful modifying client's operation (submit, cancel, open/close job, queues
  modification),
  the operation is immediately a part of the written journal.
• hq worker info can now be used with a selector, to display information about multiple workers at once.
  • Note that this is a breaking change for the JSON format, as it now outputs the worker infos as an array of objects. Before it was a single object.

Changes

• hq job progress stopped using periodic polling.
  In the current version, the client passively waits for events.
  It saves some resources as the server is not queried every second.

Artifact summary:

• hq-v0.25.0-rc1-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.25.0-rc1-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.24.0

HyperQueue 0.24.0

Breaking changes

• The --no-detect-resources flag of the hq worker start command has been removed.
  You can now configure automatically detected resources in a granular way using the new
  --detect-resources flag (see below). --no-detect-resources corresponds to --detect-resources=none.

New features

• New policy tight (and tight!) that is the original implementation of compact.
  It selects minimal number of resources groups and then tries to get maximum resources from
  a biggest group and then maximum resources from the second biggest group, etc.
  The policy compact now behaves as is described in the section "Changes".
• Resource policy compact! is now allowed to take fractional resource request.
• New command hq alloc cat <alloc-id> <stdout/stderr>, which can be used
  to debug the output of allocations submitted by the automatic allocator.
• New command hq server wait that repeatedly tries to connect to a server with a configurable timeout.
  This is useful for deployment scripts that need to wait for server availability.
• New hq alloc add parameter called --wrap-worker-cmd. It can be used to start
  workers on allocated nodes using some wrapping mechanism (e.g. Podman).
• New flag --detect-resources for hq worker start. It can be used to configure which worker resources
  will be automatically detected. You can e.g. say --detect-resources=cpus,gpus/nvidia.
  See documentation for more information.
• The scheduler has better compacting behavior when there are small number of tasks
  and workers appearing/disappering
• Autoallocator keeps log file when probing allocation fails.
• Unstable: Resource "coupling".
  You may specify that some resources are coupled, e.g. cpus and gpus.
  That means that cpus are gpus are organized in numa nodes, and allocation strategy
  will respect that, i.e., it tries to find cpus and gpus from the same numa nodes.
  Note: The current implementation does not detect coupling automatically,
  you have to specify it manually.

Changes

• Allocation policy compact was updated.
  It still tries to find the minimal number of the resource groups,
  but when they are found, resources are evenly taken from the selected groups.
  In rare cases when you need original behavior, use new policy tight.
  It is not a breaking change, because the compact previously did not specified how
  exactly will be resources taken from groups.
• Worker process terminated because of idle timeout now returns zero exit code.

Fixes

• Fixed the issue of possible ignoring idle timeout when time request is used.
• Fixes broken streaming when job file is used.
• Fixed missing fields in export of journal into JSON

Artifact summary:

• hq-v0.24.0-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.24.0-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.

v0.24.0-rc1

HyperQueue 0.24.0-rc1

Breaking changes

• The --no-detect-resources flag of the hq worker start command has been removed.
  You can now configure automatically detected resources in a granular way using the new
  --detect-resources flag (see below). --no-detect-resources corresponds to --detect-resources=none.

New features

• New policy tight (and tight!) that is the original implementation of compact.
  It selects minimal number of resources groups and then tries to get maximum resources from
  a biggest group and then maximum resources from the second biggest group, etc.
  The policy compact now behaves as is described in the section "Changes".
• Resource policy compact! is now allowed to take fractional resource request.
• New command hq alloc cat <alloc-id> <stdout/stderr>, which can be used
  to debug the output of allocations submitted by the automatic allocator.
• New command hq server wait that repeatedly tries to connect to a server with a configurable timeout.
  This is useful for deployment scripts that need to wait for server availability.
• New hq alloc add parameter called --wrap-worker-cmd. It can be used to start
  workers on allocated nodes using some wrapping mechanism (e.g. Podman).
• New flag --detect-resources for hq worker start. It can be used to configure which worker resources
  will be automatically detected. You can e.g. say --detect-resources=cpus,gpus/nvidia.
  See documentation for more information.
• The scheduler has better compacting behavior when there are small number of tasks
  and workers appearing/disappering
• Autoallocator keeps log file when probing allocation fails.
• Unstable: Resource "coupling".
  You may specify that some resources are coupled, e.g. cpus and gpus.
  That means that cpus are gpus are organized in numa nodes, and allocation strategy
  will respect that, i.e., it tries to find cpus and gpus from the same numa nodes.
  Note: The current implementation does not detect coupling automatically,
  you have to specify it manually.

Changes

• Allocation policy compact was updated.
  It still tries to find the minimal number of the resource groups,
  but when they are found, resources are evenly taken from the selected groups.
  In rare cases when you need original behavior, use new policy tight.
  It is not a breaking change, because the compact previously did not specified how
  exactly will be resources taken from groups.
• Worker process terminated because of idle timeout now returns zero exit code.

Fixes

• Fixed the issue of possible ignoring idle timeout when time request is used.
• Fixes broken streaming when job file is used.

Artifact summary:

• hq-v0.24.0-rc1-*: Main HyperQueue build containing the hq binary. Download this archive to
  use HyperQueue from the command line.
• hyperqueue-0.24.0-rc1-*: Wheel containing the hyperqueue package with HyperQueue Python
  bindings.