refactor: more object destructuration cleanup in examples
[poolifier.git] / README.md
CommitLineData
f2b5671a 1<div align="center">
0feeec4a 2 <img src="./images/logo.png" width="340px" height="266px"/>
ddaa3a76 3</div>
4
3eed0bf4 5<h1 align="center">Node.js Worker_Threads and Cluster Worker Pool</h1>
1d4f79e7
APA
6
7<p align="center">
23fc6cf8
JB
8 <a href="https://github.com/poolifier/poolifier/graphs/commit-activity">
9 <img alt="GitHub commit activity (master)" src="https://img.shields.io/github/commit-activity/m/poolifier/poolifier/master"></a>
1d4f79e7
APA
10 <a href="https://www.npmjs.com/package/poolifier">
11 <img alt="Weekly Downloads" src="https://img.shields.io/npm/dw/poolifier"></a>
d7757502 12 <a href="https://github.com/poolifier/poolifier/actions/workflows/ci.yml">
86f8562e 13 <img alt="Actions Status" src="https://github.com/poolifier/poolifier/actions/workflows/ci.yml/badge.svg"></a>
609edfec 14 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
22147b67 15 <img alt="Code Coverage" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=coverage"></a>
8076b106
JB
16 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
17 <img alt="Quality Gate Status" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=alert_status"></a>
1d4f79e7
APA
18 <a href="https://standardjs.com">
19 <img alt="Javascript Standard Style Guide" src="https://img.shields.io/badge/code_style-standard-brightgreen.svg"></a>
20 <a href="https://gitter.im/poolifier/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge">
21 <img alt="Gitter chat" src="https://badges.gitter.im/poolifier/community.svg"></a>
e8a862ca
JB
22 <a href="https://opencollective.com/poolifier">
23 <img alt="Open Collective" src="https://opencollective.com/poolifier/tiers/badge.svg"></a>
1d4f79e7
APA
24 <a href="http://makeapullrequest.com">
25 <img alt="PR Welcome" src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square"></a>
26 <a href="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen">
27 <img alt="No dependencies" src="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen"></a>
1d4f79e7
APA
28</p>
29
74750c7f 30## Why Poolifier?
e76f5485 31
6ec11f90 32Poolifier is used to perform CPU and/or I/O intensive tasks on Node.js servers, it implements worker pools using [worker_threads](https://nodejs.org/api/worker_threads.html) and [cluster](https://nodejs.org/api/cluster.html) Node.js modules.
50aa7901 33With poolifier you can improve your **performance** and resolve problems related to the event loop.
063f8c5b 34Moreover you can execute your tasks using an API designed to improve the **developer experience**.
b492f20e 35Please consult our [general guidelines](#general-guidelines).
e76f5485 36
bd2ff6e0
JB
37- Easy to use :white_check_mark:
38- Performance [benchmarks](./benchmarks/README.md) :white_check_mark:
03babf6f 39- Fixed and dynamic pool size :white_check_mark:
6ec11f90 40- Easy switch from a pool type to another :white_check_mark:
82be29c0 41- No runtime dependencies :white_check_mark:
d0ed34c9 42- Proper integration with Node.js [async_hooks](https://nodejs.org/api/async_hooks.html) :white_check_mark:
aad54ad2 43- Support for CommonJS, ESM, and TypeScript :white_check_mark:
6ec11f90 44- Support for [worker_threads](https://nodejs.org/api/worker_threads.html) and [cluster](https://nodejs.org/api/cluster.html) Node.js modules :white_check_mark:
aad54ad2
JB
45- Support for multiple task functions :white_check_mark:
46- Support for sync and async task functions :white_check_mark:
89a4abfd 47- Tasks distribution strategies :white_check_mark:
b492f20e 48- General guidelines on pool choice :white_check_mark:
82be29c0 49- Error handling out of the box :white_check_mark:
6ec11f90 50- Widely tested :white_check_mark:
82be29c0 51- Active community :white_check_mark:
bd2ff6e0 52- Code quality [![Bugs](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=bugs)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
d28c7996
JB
53 [![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=code_smells)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
54 [![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
55 [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
56 [![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
57 [![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_index)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
bd2ff6e0 58- Code security [![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=security_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier) [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=vulnerabilities)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
ddaa3a76 59
4f8e444d
JB
60## Table of contents
61
62- [Overview](#overview)
63- [Installation](#installation)
64- [Usage](#usage)
65- [Node versions](#node-versions)
66- [API](#api)
67- [General guidelines](#general-guidelines)
68- [Worker choice strategies](#worker-choice-strategies)
69- [Contribute](#contribute)
70- [Team](#team)
71- [License](#license)
34a572eb 72
50aa7901 73## Overview
13031992 74
22c0d9a2 75Poolifier contains two [worker_threads](https://nodejs.org/api/worker_threads.html#class-worker)/[cluster](https://nodejs.org/api/cluster.html#cluster_class_worker) worker pool implementations, you don't have to deal with [worker_threads](https://nodejs.org/api/worker_threads.html)/[cluster](https://nodejs.org/api/cluster.html) complexity.
03babf6f 76The first implementation is a fixed worker pool, with a defined number of workers that are started at creation time and will be reused.
56bbcb6b 77The second implementation is a dynamic worker pool, with a number of worker started at creation time (these workers will be always active and reused) and other workers created when the load will increase (with an upper limit, these workers will be reused when active), the new created workers will be stopped after a configurable period of inactivity.
22c0d9a2 78You have to implement your worker by extending the _ThreadWorker_ or _ClusterWorker_ class.
50aa7901
S
79
80## Installation
81
82```shell
27c5fb75 83npm install poolifier --save
1a4ec243 84```
1a4ec243 85
50aa7901
S
86## Usage
87
6ec11f90 88You can implement a [worker_threads](https://nodejs.org/api/worker_threads.html#class-worker) worker in a simple way by extending the class _ThreadWorker_:
1a4ec243
APA
89
90```js
91'use strict'
d2eb4964 92const { ThreadWorker } = require('poolifier')
1a4ec243 93
78cea37e 94function yourFunction(data) {
106744f7 95 // this will be executed in the worker thread,
96 // the data will be received by using the execute method
97 return { ok: 1 }
98}
99
50aa7901 100module.exports = new ThreadWorker(yourFunction, {
124119cf 101 maxInactiveTime: 60000
50aa7901 102})
1a4ec243
APA
103```
104
0feeec4a 105Instantiate your pool based on your needs :
1a4ec243
APA
106
107```js
108'use strict'
6961ca9a 109const { DynamicThreadPool, FixedThreadPool, PoolEvents, availableParallelism } = require('poolifier')
1a4ec243 110
2fbe1783 111// a fixed worker_threads pool
6961ca9a 112const pool = new FixedThreadPool(availableParallelism(), './yourWorker.js', {
0eee77cb 113 errorHandler: (e) => console.error(e),
53795b86
JB
114 onlineHandler: () => console.info('worker is online')
115})
1a4ec243 116
2431bdb4 117pool.emitter.on(PoolEvents.ready, () => console.info('Pool is ready'))
53795b86 118pool.emitter.on(PoolEvents.busy, () => console.info('Pool is busy'))
164d950a 119
2fbe1783 120// or a dynamic worker_threads pool
31a7d5be 121const pool = new DynamicThreadPool(Math.floor(availableParallelism() / 2), availableParallelism(), './yourWorker.js', {
0eee77cb 122 errorHandler: (e) => console.error(e),
53795b86
JB
123 onlineHandler: () => console.info('worker is online')
124})
106744f7 125
53795b86 126pool.emitter.on(PoolEvents.full, () => console.info('Pool is full'))
2431bdb4 127pool.emitter.on(PoolEvents.ready, () => console.info('Pool is ready'))
53795b86 128pool.emitter.on(PoolEvents.busy, () => console.info('Pool is busy'))
1a4ec243
APA
129
130// the execute method signature is the same for both implementations,
131// so you can easy switch from one to another
8351c9af 132pool
8923de44 133 .execute()
0eee77cb 134 .then((res) => {
8351c9af
JB
135 console.info(res)
136 })
0eee77cb 137 .catch((err) => {
8351c9af
JB
138 console.error(err)
139 })
1a4ec243
APA
140```
141
6ec11f90 142You can do the same with the classes _ClusterWorker_, _FixedClusterPool_ and _DynamicClusterPool_.
31b90205 143
b0834e9d 144**See [examples](./examples/) for more details**:
b2b02b3d 145
7a923e5b 146- [Javascript](./examples/javascript/)
b2b02b3d 147- [Typescript](./examples/typescript/)
5ba0cd3c 148 - [HTTP client pool](./examples/typescript/http-client-pool/)
9aef1431 149 - [SMTP client pool](./examples/typescript/smtp-client-pool/)
fac9ce96 150 - [HTTP server pool](./examples/typescript/http-server-pool/)
d2bc8d80 151 - [Express worker_threads pool](./examples/typescript/http-server-pool/express-worker_threads/)
15a9c195 152 - [Express cluster pool](./examples/typescript/http-server-pool/express-cluster/)
d0ed34c9 153 - [Express hybrid pool](./examples/typescript/http-server-pool/express-hybrid/)
d2bc8d80
JB
154 - [Fastify worker_threads pool](./examples/typescript/http-server-pool/fastify-worker_threads/)
155 - [Fastify cluster pool](./examples/typescript/http-server-pool/fastify-cluster/)
3b311539 156 - [Fastify hybrid pool](./examples/typescript/http-server-pool/fastify-hybrid/)
9e623684 157 - [WebSocket server pool](./examples/typescript/websocket-server-pool/)
d2bc8d80 158 - [ws worker_threads pool](./examples/typescript/websocket-server-pool/ws-worker_threads/)
72855e92 159 - [ws cluster pool](./examples/typescript/websocket-server-pool/ws-cluster/)
02999424 160 - [ws hybrid pool](./examples/typescript/websocket-server-pool/ws-hybrid/)
1a4ec243 161
6677a3d3 162Remember that workers can only send and receive structured-cloneable data.
deb85c12 163
50aa7901 164## Node versions
34a572eb 165
9e45c2c4 166Node versions >= 16.14.x are supported.
1a4ec243 167
b492f20e 168## [API](./docs/api.md)
4680c1b2 169
4f8e444d
JB
170## [General guidelines](./docs/general-guidelines.md)
171
172## [Worker choice strategies](./docs/worker-choice-strategies.md)
48211d04 173
50aa7901 174## Contribute
34a572eb 175
91633990 176Choose your task here [2.6.x](https://github.com/orgs/poolifier/projects/1), propose an idea, a fix, an improvement.
1a4ec243 177
0feeec4a
JB
178See [CONTRIBUTING](CONTRIBUTING.md) guidelines.
179
50aa7901 180## Team
1a4ec243 181
39deb558 182**Creator/Owner:**
39deb558 183
50aa7901
S
184- [**Alessandro Pio Ardizio**](https://github.com/pioardi)
185
186**_Contributors_**
187
188- [**Shinigami92**](https://github.com/Shinigami92)
189- [**Jérôme Benoit**](https://github.com/jerome-benoit)
39deb558 190
50aa7901 191## License
1a4ec243 192
9507c1d4 193[MIT](./LICENSE)