chore: generate documentation
[poolifier.git] / README.md
CommitLineData
f2b5671a 1<div align="center">
0feeec4a 2 <img src="./images/logo.png" width="340px" height="266px"/>
ddaa3a76 3</div>
4
063f8c5b 5<h2 align="center">Node Thread Pool and Cluster Pool :arrow_double_up: :on:</h2>
1d4f79e7
APA
6
7<p align="center">
8 <a href="https://www.npmjs.com/package/poolifier">
9 <img alt="Weekly Downloads" src="https://img.shields.io/npm/dw/poolifier"></a>
d7757502 10 <a href="https://github.com/poolifier/poolifier/actions/workflows/ci.yml">
86f8562e 11 <img alt="Actions Status" src="https://github.com/poolifier/poolifier/actions/workflows/ci.yml/badge.svg"></a>
1d4f79e7
APA
12 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
13 <img alt="Quality Gate Status" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=alert_status"></a>
609edfec 14 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
22147b67 15 <img alt="Code Coverage" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=coverage"></a>
1d4f79e7
APA
16 <a href="https://standardjs.com">
17 <img alt="Javascript Standard Style Guide" src="https://img.shields.io/badge/code_style-standard-brightgreen.svg"></a>
18 <a href="https://gitter.im/poolifier/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge">
19 <img alt="Gitter chat" src="https://badges.gitter.im/poolifier/community.svg"></a>
e8a862ca
JB
20 <a href="https://opencollective.com/poolifier">
21 <img alt="Open Collective" src="https://opencollective.com/poolifier/tiers/badge.svg"></a>
8f8dc0e0
JB
22 <a href="https://badgen.net/badge/Dependabot/enabled/green?icon=dependabot">
23 <img alt="Dependabot" src="https://badgen.net/badge/Dependabot/enabled/green?icon=dependabot"></a>
1d4f79e7
APA
24 <a href="http://makeapullrequest.com">
25 <img alt="PR Welcome" src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square"></a>
26 <a href="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen">
27 <img alt="No dependencies" src="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen"></a>
1d4f79e7
APA
28</p>
29
74750c7f 30## Why Poolifier?
e76f5485 31
4dce520e 32Poolifier is used to perform CPU intensive and I/O intensive tasks on nodejs servers, it implements worker pools using [worker-threads](https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads) and cluster pools using [Node.js cluster](https://nodejs.org/api/cluster.html) modules.
50aa7901 33With poolifier you can improve your **performance** and resolve problems related to the event loop.
063f8c5b 34Moreover you can execute your tasks using an API designed to improve the **developer experience**.
0feeec4a 35Please consult our [general guidelines](#general-guidance).
e76f5485 36
64060857 37- Performance :racehorse: [benchmarks](./benchmarks/README.md)
f2b5671a 38- Security :bank: :cop: [![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=security_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier) [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=vulnerabilities)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
b1b24f66 39- Easy to use :couple:
82be29c0 40- Dynamic pool size :white_check_mark:
e42a859c 41- Easy switch from a pool to another :white_check_mark:
82be29c0
JB
42- No runtime dependencies :white_check_mark:
43- Proper async integration with node async hooks :white_check_mark:
44- Support for worker threads and cluster node modules :white_check_mark:
45- Support sync and async tasks :white_check_mark:
89a4abfd 46- Tasks distribution strategies :white_check_mark:
82be29c0
JB
47- General guidance on pools to use :white_check_mark:
48- Widely tested :white_check_mark:
49- Error handling out of the box :white_check_mark:
50- Active community :white_check_mark:
5fd75a2e 51- Code quality :octocat: [![Bugs](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=bugs)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
d28c7996
JB
52 [![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=code_smells)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
53 [![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
54 [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
55 [![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
56 [![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_index)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
ddaa3a76 57
50aa7901 58## Contents
e76f5485 59
34a572eb 60<h3 align="center">
50aa7901
S
61 <a href="#overview">Overview</a>
62 <span> · </span>
34a572eb 63 <a href="#installation">Installation</a>
64 <span> · </span>
65 <a href="#usage">Usage</a>
66 <span> · </span>
4f4ae1cb 67 <a href="#node-versions">Node versions</a>
50aa7901 68 <span> · </span>
34a572eb 69 <a href="#api">API</a>
70 <span> · </span>
e36cfb13 71 <a href="#general-guidance">General guidance</a>
48211d04 72 <span> · </span>
34a572eb 73 <a href="#contribute">Contribute</a>
74 <span> · </span>
50aa7901 75 <a href="#team">Team</a>
34a572eb 76 <span> · </span>
77 <a href="#license">License</a>
78</h3>
79
50aa7901 80## Overview
13031992 81
31b90205
JB
82Node pool contains two [worker-threads](https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads)/[cluster worker](https://nodejs.org/api/cluster.html#cluster_class_worker) pool implementations, you don't have to deal with worker-threads/cluster worker complexity.
83The first implementation is a static worker pool, with a defined number of workers that are started at creation time and will be reused.
84The second implementation is a dynamic worker pool with a number of worker started at creation time (these workers will be always active and reused) and other workers created when the load will increase (with an upper limit, these workers will be reused when active), the new created workers will be stopped after a configurable period of inactivity.
0feeec4a 85You have to implement your worker extending the ThreadWorker or ClusterWorker class.
50aa7901
S
86
87## Installation
88
89```shell
27c5fb75 90npm install poolifier --save
1a4ec243 91```
1a4ec243 92
50aa7901
S
93## Usage
94
31b90205 95You can implement a worker-threads worker in a simple way by extending the class ThreadWorker:
1a4ec243
APA
96
97```js
98'use strict'
d2eb4964 99const { ThreadWorker } = require('poolifier')
1a4ec243 100
78cea37e 101function yourFunction(data) {
106744f7 102 // this will be executed in the worker thread,
103 // the data will be received by using the execute method
104 return { ok: 1 }
105}
106
50aa7901 107module.exports = new ThreadWorker(yourFunction, {
124119cf 108 maxInactiveTime: 60000
50aa7901 109})
1a4ec243
APA
110```
111
0feeec4a 112Instantiate your pool based on your needs :
1a4ec243
APA
113
114```js
115'use strict'
aee46736 116const { DynamicThreadPool, FixedThreadPool, PoolEvents } = require('poolifier')
1a4ec243 117
31b90205 118// a fixed worker-threads pool
1a4ec243 119const pool = new FixedThreadPool(15,
106744f7 120 './yourWorker.js',
121 { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })
1a4ec243 122
aee46736 123pool.emitter.on(PoolEvents.busy, () => console.log('Pool is busy'))
164d950a 124
31b90205 125// or a dynamic worker-threads pool
1a4ec243 126const pool = new DynamicThreadPool(10, 100,
106744f7 127 './yourWorker.js',
128 { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })
129
aee46736
JB
130pool.emitter.on(PoolEvents.full, () => console.log('Pool is full'))
131pool.emitter.on(PoolEvents.busy, () => console.log('Pool is busy'))
1a4ec243
APA
132
133// the execute method signature is the same for both implementations,
134// so you can easy switch from one to another
135pool.execute({}).then(res => {
136 console.log(res)
777b7824 137}).catch ....
1a4ec243
APA
138
139```
140
17a9a094 141You can do the same with the classes ClusterWorker, FixedClusterPool and DynamicClusterPool.
31b90205 142
4abb4f36 143**See examples folder for more details (in particular if you want to use a pool with [multiple worker functions](./examples/multiFunctionExample.js))**.
1a4ec243 144
ec2ccfc8 145Remember that workers can only send and receive serializable data.
deb85c12 146
50aa7901 147## Node versions
34a572eb 148
9e45c2c4 149Node versions >= 16.14.x are supported.
1a4ec243 150
059ea823 151## [API](https://poolifier.github.io/poolifier/)
4680c1b2 152
5e5ea237 153### `PoolOptions`
50aa7901 154
70e9dd65 155An object with these properties:
50aa7901 156
46e857ca 157- `messageHandler` (optional) - A function that will listen for message event on each worker
f42ca86f
JB
158- `errorHandler` (optional) - A function that will listen for error event on each worker
159- `onlineHandler` (optional) - A function that will listen for online event on each worker
160- `exitHandler` (optional) - A function that will listen for exit event on each worker
46e857ca 161- `workerChoiceStrategy` (optional) - The worker choice strategy to use in this pool:
5519f8f3 162
b4db5969 163 - `WorkerChoiceStrategies.ROUND_ROBIN`: Submit tasks to worker in a round robin fashion
1c6fe997
JB
164 - `WorkerChoiceStrategies.LEAST_USED`: Submit tasks to the worker with the minimum number of executed, executing and queued tasks
165 - `WorkerChoiceStrategies.LEAST_BUSY`: Submit tasks to the worker with the minimum tasks total execution and wait time
a7bbf44a 166 - `WorkerChoiceStrategies.LEAST_ELU`: Submit tasks to the worker with the minimum event loop utilization (ELU) (experimental)
9e2ab226 167 - `WorkerChoiceStrategies.WEIGHTED_ROUND_ROBIN`: Submit tasks to worker by using a [weighted round robin scheduling algorithm](./src/pools/selection-strategies/README.md#weighted-round-robin) based on tasks execution time
129ab848
JB
168 - `WorkerChoiceStrategies.INTERLEAVED_WEIGHTED_ROUND_ROBIN`: Submit tasks to worker by using an [interleaved weighted round robin scheduling algorithm](./src/pools/selection-strategies/README.md#interleaved-weighted-round-robin) based on tasks execution time(experimental)
169 - `WorkerChoiceStrategies.FAIR_SHARE`: Submit tasks to worker by using a [fair share scheduling algorithm](./src/pools/selection-strategies/README.md#fair-share) based on tasks execution time (the default) or ELU active time
5519f8f3 170
9b2bdf94 171 `WorkerChoiceStrategies.WEIGHTED_ROUND_ROBIN`, `WorkerChoiceStrategies.INTERLEAVED_WEIGHTED_ROUND_ROBIN` and `WorkerChoiceStrategies.FAIR_SHARE` strategies are targeted to heavy and long tasks.
f42ca86f 172 Default: `WorkerChoiceStrategies.ROUND_ROBIN`
5519f8f3 173
da309861
JB
174- `workerChoiceStrategyOptions` (optional) - The worker choice strategy options object to use in this pool.
175 Properties:
176
9adcefab 177 - `measurement` (optional) - The measurement to use in worker choice strategies: `runTime`, `waitTime` or `elu`.
9e2ab226
JB
178 - `runTime` (optional) - Use the tasks [median](./src/pools/selection-strategies/README.md#median) runtime instead of the tasks average runtime in worker choice strategies.
179 - `waitTime` (optional) - Use the tasks [median](./src/pools/selection-strategies/README.md#median) wait time instead of the tasks average wait time in worker choice strategies.
180 - `elu` (optional) - Use the tasks [median](./src/pools/selection-strategies/README.md#median) ELU instead of the tasks average ELU in worker choice strategies.
129ab848 181 - `weights` (optional) - The worker weights to use in weighted round robin worker choice strategies: `{ 0: 200, 1: 300, ..., n: 100 }`.
7171d33f 182
9adcefab 183 Default: `{ runTime: { median: false }, waitTime: { median: false }, elu: { median: false } }`
da309861 184
1f68cede 185- `restartWorkerOnError` (optional) - Restart worker on uncaught error in this pool.
609154c5 186 Default: `true`
dfd00839 187- `enableEvents` (optional) - Events emission enablement in this pool.
609154c5 188 Default: `true`
dfd00839 189- `enableTasksQueue` (optional) - Tasks queue per worker enablement in this pool.
609154c5 190 Default: `false`
34a572eb 191
83d127b9 192- `tasksQueueOptions` (optional) - The worker tasks queue options object to use in this pool.
7171d33f
JB
193 Properties:
194
195 - `concurrency` (optional) - The maximum number of tasks that can be executed concurrently on a worker.
196
dfd00839 197 Default: `{ concurrency: 1 }`
7171d33f 198
92e7183f 199#### `ThreadPoolOptions extends PoolOptions`
70e9dd65 200
129ab848 201- `workerOptions` (optional) - An object with the worker options to pass to worker. See [worker_threads](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options) for more details.
70e9dd65 202
92e7183f 203#### `ClusterPoolOptions extends PoolOptions`
70e9dd65 204
129ab848 205- `env` (optional) - An object with the environment variables to pass to worker. See [cluster](https://nodejs.org/api/cluster.html#cluster_cluster_fork_env) for more details.
70e9dd65 206
430745c9 207- `settings` (optional) - An object with the cluster settings. See [cluster](https://nodejs.org/api/cluster.html#cluster_cluster_settings) for more details.
70e9dd65
JB
208
209### `pool = new FixedThreadPool/FixedClusterPool(numberOfThreads/numberOfWorkers, filePath, opts)`
210
211`numberOfThreads/numberOfWorkers` (mandatory) Number of workers for this pool
212`filePath` (mandatory) Path to a file with a worker implementation
213`opts` (optional) An object with the pool options properties described above
214
31b90205 215### `pool = new DynamicThreadPool/DynamicClusterPool(min, max, filePath, opts)`
50aa7901 216
31b90205
JB
217`min` (mandatory) Same as FixedThreadPool/FixedClusterPool numberOfThreads/numberOfWorkers, this number of workers will be always active
218`max` (mandatory) Max number of workers that this pool can contain, the new created workers will die after a threshold (default is 1 minute, you can override it in your worker implementation).
70e9dd65
JB
219`filePath` (mandatory) Path to a file with a worker implementation
220`opts` (optional) An object with the pool options properties described above
34a572eb 221
70e9dd65 222### `pool.execute(data, name)`
50aa7901 223
0feeec4a 224`data` (optional) An object that you want to pass to your worker implementation
609154c5 225`name` (optional) A string with the task function name that you want to execute on the worker. Default: `'default'`
0feeec4a 226This method is available on both pool implementations and returns a promise.
34a572eb 227
228### `pool.destroy()`
34a572eb 229
50aa7901
S
230Destroy method is available on both pool implementations.
231This method will call the terminate method on each worker.
34a572eb 232
31b90205 233### `class YourWorker extends ThreadWorker/ClusterWorker`
50aa7901 234
70e9dd65 235`taskFunctions` (mandatory) The task function or task functions object that you want to execute on the worker
50aa7901
S
236`opts` (optional) An object with these properties:
237
dfd00839 238- `maxInactiveTime` (optional) - Max time to wait tasks to work on in milliseconds, after this period the new worker will die.
3832ad95
S
239 The last active time of your worker unit will be updated when a task is submitted to a worker or when a worker terminate a task.
240 If `killBehavior` is set to `KillBehaviors.HARD` this value represents also the timeout for the tasks that you submit to the pool, when this timeout expires your tasks is interrupted and the worker is killed if is not part of the minimum size of the pool.
241 If `killBehavior` is set to `KillBehaviors.SOFT` your tasks have no timeout and your workers will not be terminated until your task is completed.
609154c5 242 Default: `60000`
15eacd5d 243
dfd00839 244- `killBehavior` (optional) - Dictates if your async unit (worker/process) will be deleted in case that a task is active on it.
1c6fe997
JB
245 **KillBehaviors.SOFT**: If `currentTime - lastActiveTime` is greater than `maxInactiveTime` but a task is still executing, then the worker **won't** be deleted.
246 **KillBehaviors.HARD**: If `currentTime - lastActiveTime` is greater than `maxInactiveTime` but a task is still executing, then the worker will be deleted.
3832ad95 247 This option only apply to the newly created workers.
f42ca86f 248 Default: `KillBehaviors.SOFT`
50aa7901 249
e36cfb13 250## General guidance
34a572eb 251
31b90205 252Performance is one of the main target of these worker pool implementations, we want to have a strong focus on this.
e36cfb13
APA
253We already have a bench folder where you can find some comparisons.
254
255### Internal Node.js thread pool
256
257Before to jump into each poolifier pool type, let highlight that **Node.js comes with a thread pool already**, the libuv thread pool where some particular tasks already run by default.
258Please take a look at [which tasks run on the libuv thread pool](https://nodejs.org/en/docs/guides/dont-block-the-event-loop/#what-code-runs-on-the-worker-pool).
259
e843b904 260**If your task runs on libuv thread pool**, you can try to:
e36cfb13 261
0feeec4a 262- Tune the libuv thread pool size setting the [UV_THREADPOOL_SIZE](https://nodejs.org/api/cli.html#cli_uv_threadpool_size_size).
e36cfb13
APA
263
264and/or
265
0feeec4a 266- Use poolifier cluster pool that spawning child processes will also increase the number of libuv threads since that any new child process comes with a separated libuv thread pool. **More threads does not mean more fast, so please tune your application**.
e36cfb13
APA
267
268### Cluster vs Threads worker pools
269
270**If your task does not run into libuv thread pool** and is CPU intensive then poolifier **thread pools** (FixedThreadPool and DynamicThreadPool) are suggested to run CPU intensive tasks, you can still run I/O intensive tasks into thread pools, but performance enhancement is expected to be minimal.
063f8c5b 271Thread pools are built on top of Node.js [worker-threads](https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads) module.
272
e36cfb13
APA
273**If your task does not run into libuv thread pool** and is I/O intensive then poolifier **cluster pools** (FixedClusterPool and DynamicClusterPool) are suggested to run I/O intensive tasks, again you can still run CPU intensive tasks into cluster pools, but performance enhancement is expected to be minimal.
274Consider that by default Node.js already has great performance for I/O tasks (asynchronous I/O).
4a6952ff 275Cluster pools are built on top of Node.js [cluster](https://nodejs.org/api/cluster.html) module.
75876e4c 276
e36cfb13 277If your task contains code that runs on libuv plus code that is CPU intensive or I/O intensive you either split it either combine more strategies (i.e. tune the number of libuv threads and use cluster/thread pools).
0feeec4a 278But in general, **always profile your application**.
e36cfb13
APA
279
280### Fixed vs Dynamic pools
063f8c5b 281
31b90205 282To choose your pool consider that with a FixedThreadPool/FixedClusterPool or a DynamicThreadPool/DynamicClusterPool (in this case is important the min parameter passed to the constructor) your application memory footprint will increase.
063f8c5b 283Increasing the memory footprint, your application will be ready to accept more tasks, but during idle time your application will consume more memory.
afe102b2 284One good choice from poolifier team point of view is to profile your application using fixed or dynamic worker pool, and to see your application metrics when you increase/decrease the num of workers.
31b90205 285For example you could keep the memory footprint low choosing a DynamicThreadPool/DynamicClusterPool with 5 workers, and allow to create new workers until 50/100 when needed, this is the advantage to use the DynamicThreadPool/DynamicClusterPool.
0feeec4a 286But in general, **always profile your application**.
48211d04 287
50aa7901 288## Contribute
34a572eb 289
91633990 290Choose your task here [2.6.x](https://github.com/orgs/poolifier/projects/1), propose an idea, a fix, an improvement.
1a4ec243 291
0feeec4a
JB
292See [CONTRIBUTING](CONTRIBUTING.md) guidelines.
293
50aa7901 294## Team
1a4ec243 295
39deb558 296<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
39deb558 297
298**Creator/Owner:**
39deb558 299
50aa7901
S
300- [**Alessandro Pio Ardizio**](https://github.com/pioardi)
301
302**_Contributors_**
303
304- [**Shinigami92**](https://github.com/Shinigami92)
305- [**Jérôme Benoit**](https://github.com/jerome-benoit)
39deb558 306
50aa7901 307## License
1a4ec243 308
9507c1d4 309[MIT](./LICENSE)