]> Piment Noir Git Repositories - poolifier.git/commit
feat: task function worker node affinity (#2269)
authorJérôme Benoit <jerome.benoit@piment-noir.org>
Mon, 16 Feb 2026 15:52:42 +0000 (16:52 +0100)
committerGitHub <noreply@github.com>
Mon, 16 Feb 2026 15:52:42 +0000 (16:52 +0100)
commit8d9f9d710c74ad72b7028fad325b8ccbf19a993b
tree1c9eae5f6de68c5fa3ff85a07b88594d0487ebf5
parent964ff3f58c6a4727cbfab11aa9d08d5fdca96e79
feat: task function worker node affinity  (#2269)

* feat: task function worker node affinity

closes #778

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: cleanup worker node affinity namespace

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: validate worker nodes affinity

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: refine worke nodes affinity validation

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* fix: check worker node keys affinity is part of pool worker nodes

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* fix: fix default task function properties handling

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* test: improve task function objects coverage

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* test: cleanup tests

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* test: fix mismerge

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: silence linter

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: code reformatting

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: code formatting

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* chore: merge eslint-plugin-perfectionist reformatting

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* chore: reenabled lint-staged

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* chore: fix mismerge

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: code formatting

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* chore: fix mismerge

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* perf: lighter worker node keys validation on hot path

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: cleanups

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* refactor: early exit in worker choice strategies

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: align unique worker node affinity handling

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [autofix.ci] apply automated fixes

* refactor: remove uneeded comment

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* perf: reorder conditions

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: remove uneeded conditions

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* perf: add missing optimization

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* refactor: cleanup worker nodes array validation code

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
* fix: add defensive checks for worker node affinity in selection strategies

- Add bounded loops to prevent infinite loops when no valid worker in affinity set
- Add empty/single-key checks in all strategies
- Add comprehensive tests for workerNodeKeys affinity

* fix: add empty array guard and fix iteration in worker node affinity strategies

* refactor(selection-strategies): extract getSingleWorkerNodeKey helper to reduce duplication

Consolidate the early-return pattern for single-element workerNodeKeys
array into a reusable protected method in AbstractWorkerChoiceStrategy.
All 7 selection strategies now use this helper instead of duplicating
the same check and isWorkerNodeReady call.

* fix(selection-strategies): correct affinity handling and optimize to O(1) lookup

- Fix WeightedRoundRobin: check affinity membership before using workerNodeKey
- Fix InterleavedWeightedRR: add wrap-around to find valid workers behind current position
- Optimize: change checkWorkerNodeKeys() to return Set<number> for O(1) .has() lookup
- Update all 7 strategies to use Set.has() instead of Array.includes()

* refactor(iwrr): remove unnecessary wrap-around logic

The retry mechanism in executeStrategy() already handles the case where
choose() returns undefined. Each call advances the position via
interleavedWeightedRoundRobinNextWorkerNodeId(), eventually covering
the entire (roundId, workerNodeId) space.

Removed the interleavedWeightedRoundRobinChoose() helper method and
inlined the logic directly in choose() for simplicity.

* refactor(api): rename workerNodes to workerNodeKeys for clarity

Rename the task function affinity property from 'workerNodes' to
'workerNodeKeys' to avoid confusion with pool.workerNodes (IWorkerNode[])
and align with internal naming conventions where keys refer to indices.

* test(selection-strategies): add workerNodeKeys affinity tests for all strategies

Add comprehensive tests for worker node keys affinity in choose() method:
- Empty workerNodeKeys returns undefined
- Single workerNodeKey returns that key if ready
- Multiple workerNodeKeys respects affinity constraint

Covers: RoundRobin, LeastUsed, LeastBusy, LeastElu, FairShare,
WeightedRoundRobin, InterleavedWeightedRoundRobin strategies

* test(pool): add workerNodeKeys validation and affinity tests

- Add checkValidWorkerNodes edge case tests
- Add addTaskFunction workerNodeKeys validation tests
- Add integration test for execute() workerNodeKeys affinity

* docs: improve workerNodeKeys JSDoc and error messages

- Expand JSDoc for workerNodeKeys parameter across selection strategies
- Improve error message clarity for invalid worker node keys
- Update tests to match improved error messages

* docs: harmonize workerNodeKeys JSDoc at getSingleWorkerNodeKey

* fix(weighted-round-robin): set nextWorkerNodeKey when staying on same worker

* fix(pool): check worker node affinity before returning dynamic worker

* docs: clarify worker node affinity validation behavior

* refactor: rename checkValidWorkerNodes to checkValidWorkerNodeKeys

Align function name, parameter name, and error messages with actual
semantics: the function validates worker node keys (numeric indices),
not worker node objects.

* refactor: eliminate unnecessary Set/Array allocations in worker node affinity hot paths

- Remove workerNodeKeys getter from IPool interface and AbstractPool
- Replace new Set(this.workerNodeKeys) validation with O(1) range checks
- Rename getTaskFunctionWorkerNodes() to getTaskFunctionWorkerNodeKeysSet()
  returning Set<number> directly from source
- Propagate Set<number> through execute() → executeStrategy() → choose()
  eliminating intermediate array-to-Set conversions
- Replace new Set(this.workerNodes.keys()) in sendTaskFunctionOperationToWorkers()
  with direct workerNodes.length/keys() usage

* fix(wrr): update state in single-element affinity shortcut

Reset nextWorkerNodeKey and workerNodeVirtualTaskExecutionTime in the
size === 1 path to maintain consistent round-robin state.

* fix: address Copilot review comments

- Check affinity before creating dynamic worker
- Add null to workerNodeKeys signature
- Fix anchor link and document length constraint

* chore: disable biome formatter in opencode

* perf(strategies): fast path for round-robin without affinity

* test: consolidate worker choice strategies test suite

* test: verify workerNodeKeys transmission from worker constructor

* fix: address Copilot review concerns for worker node affinity

- Snapshot workerNodeKeys in sendTaskFunctionOperationToWorkers to prevent race conditions
- Use ReadonlySet<number> for workerNodeKeysSet in selection strategies for type safety
- Compute workerNodeKeysSet lazily instead of storing in TaskFunctionProperties
- Allow null in workerNodeKeys type to explicitly disable affinity

* fix: improve error message grammar in addTaskFunction validation

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* fix: validate worker node keys against max pool size and fix single-key state update

- Validate workerNodeKeys against maximumNumberOfWorkers instead of current
  pool size, allowing dynamic pools to reference future worker indices
- Only update nextWorkerNodeKey in RR/WRR single-key path when worker is
  ready, preventing state inconsistency on retry
- Clarify TaskFunctionObject.workerNodeKeys doc regarding null vs undefined

* feat: create dynamic workers to satisfy worker node affinity constraints

In dynamic pools, when a task's workerNodeKeys affinity references
worker nodes beyond the current pool size, automatically create
dynamic workers up to the maximum pool size to satisfy the affinity.

* test: harmonize variable naming in dynamic worker affinity test

* refactor: consolidate worker node keys validation into helper and enforce error types in tests

* fix: remove unused null from workerNodeKeys type and restore test suite title

* docs: mention worker node affinity feature in README

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* refactor: use project max helper instead of Math.max

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
---------

Signed-off-by: Jérôme Benoit <jerome.benoit@piment-noir.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
27 files changed:
.opencode/opencode.jsonc
README.md
docs/api.md
src/pools/abstract-pool.ts
src/pools/selection-strategies/abstract-worker-choice-strategy.ts
src/pools/selection-strategies/fair-share-worker-choice-strategy.ts
src/pools/selection-strategies/interleaved-weighted-round-robin-worker-choice-strategy.ts
src/pools/selection-strategies/least-busy-worker-choice-strategy.ts
src/pools/selection-strategies/least-elu-worker-choice-strategy.ts
src/pools/selection-strategies/least-used-worker-choice-strategy.ts
src/pools/selection-strategies/round-robin-worker-choice-strategy.ts
src/pools/selection-strategies/selection-strategies-types.ts
src/pools/selection-strategies/weighted-round-robin-worker-choice-strategy.ts
src/pools/selection-strategies/worker-choice-strategies-context.ts
src/pools/utils.ts
src/utility-types.ts
src/utils.ts
src/worker/abstract-worker.ts
src/worker/task-functions.ts
src/worker/utils.ts
tests/pools/abstract-pool.test.mjs
tests/pools/selection-strategies/weighted-round-robin-worker-choice-strategy.test.mjs
tests/pools/selection-strategies/worker-choice-strategies-context.test.mjs
tests/pools/utils.test.mjs
tests/worker-files/cluster/testMultipleTaskFunctionsWorker.cjs
tests/worker-files/thread/testMultipleTaskFunctionsWorker.mjs
tests/worker-files/thread/testTaskFunctionObjectsWorker.mjs