build(deps-dev): apply updates
[poolifier.git] / README.md
CommitLineData
f2b5671a 1<div align="center">
0feeec4a 2 <img src="./images/logo.png" width="340px" height="266px"/>
ddaa3a76 3</div>
4
3eed0bf4 5<h1 align="center">Node.js Worker_Threads and Cluster Worker Pool</h1>
1d4f79e7
APA
6
7<p align="center">
23fc6cf8
JB
8 <a href="https://github.com/poolifier/poolifier/graphs/commit-activity">
9 <img alt="GitHub commit activity (master)" src="https://img.shields.io/github/commit-activity/m/poolifier/poolifier/master"></a>
1d4f79e7
APA
10 <a href="https://www.npmjs.com/package/poolifier">
11 <img alt="Weekly Downloads" src="https://img.shields.io/npm/dw/poolifier"></a>
d7757502 12 <a href="https://github.com/poolifier/poolifier/actions/workflows/ci.yml">
86f8562e 13 <img alt="Actions Status" src="https://github.com/poolifier/poolifier/actions/workflows/ci.yml/badge.svg"></a>
609edfec 14 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
22147b67 15 <img alt="Code Coverage" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=coverage"></a>
8076b106
JB
16 <a href="https://sonarcloud.io/dashboard?id=pioardi_poolifier">
17 <img alt="Quality Gate Status" src="https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=alert_status"></a>
1d4f79e7
APA
18 <a href="https://standardjs.com">
19 <img alt="Javascript Standard Style Guide" src="https://img.shields.io/badge/code_style-standard-brightgreen.svg"></a>
20 <a href="https://gitter.im/poolifier/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge">
21 <img alt="Gitter chat" src="https://badges.gitter.im/poolifier/community.svg"></a>
e8a862ca
JB
22 <a href="https://opencollective.com/poolifier">
23 <img alt="Open Collective" src="https://opencollective.com/poolifier/tiers/badge.svg"></a>
1d4f79e7
APA
24 <a href="http://makeapullrequest.com">
25 <img alt="PR Welcome" src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square"></a>
26 <a href="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen">
27 <img alt="No dependencies" src="https://img.shields.io/static/v1?label=dependencies&message=no%20dependencies&color=brightgreen"></a>
1d4f79e7
APA
28</p>
29
74750c7f 30## Why Poolifier?
e76f5485 31
6ec11f90 32Poolifier is used to perform CPU and/or I/O intensive tasks on Node.js servers, it implements worker pools using [worker_threads](https://nodejs.org/api/worker_threads.html) and [cluster](https://nodejs.org/api/cluster.html) Node.js modules.
50aa7901 33With poolifier you can improve your **performance** and resolve problems related to the event loop.
063f8c5b 34Moreover you can execute your tasks using an API designed to improve the **developer experience**.
b492f20e 35Please consult our [general guidelines](#general-guidelines).
e76f5485 36
bd2ff6e0
JB
37- Easy to use :white_check_mark:
38- Performance [benchmarks](./benchmarks/README.md) :white_check_mark:
03babf6f 39- Fixed and dynamic pool size :white_check_mark:
6ec11f90 40- Easy switch from a pool type to another :white_check_mark:
82be29c0 41- No runtime dependencies :white_check_mark:
cb954063 42- Proper integration with node [async_hooks](https://nodejs.org/api/async_hooks.html) :white_check_mark:
bd2ff6e0 43- Support CommonJS, ESM, and TypeScript :white_check_mark:
6ec11f90 44- Support for [worker_threads](https://nodejs.org/api/worker_threads.html) and [cluster](https://nodejs.org/api/cluster.html) Node.js modules :white_check_mark:
82ea6492
JB
45- Support multiple task functions :white_check_mark:
46- Support sync and async task functions :white_check_mark:
89a4abfd 47- Tasks distribution strategies :white_check_mark:
b492f20e 48- General guidelines on pool choice :white_check_mark:
82be29c0 49- Error handling out of the box :white_check_mark:
6ec11f90 50- Widely tested :white_check_mark:
82be29c0 51- Active community :white_check_mark:
bd2ff6e0 52- Code quality [![Bugs](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=bugs)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
d28c7996
JB
53 [![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=code_smells)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
54 [![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=duplicated_lines_density)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
55 [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
56 [![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=reliability_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
57 [![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=sqale_index)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
bd2ff6e0 58- Code security [![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=security_rating)](https://sonarcloud.io/dashboard?id=pioardi_poolifier) [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=pioardi_poolifier&metric=vulnerabilities)](https://sonarcloud.io/dashboard?id=pioardi_poolifier)
ddaa3a76 59
4f8e444d
JB
60## Table of contents
61
62- [Overview](#overview)
63- [Installation](#installation)
64- [Usage](#usage)
65- [Node versions](#node-versions)
66- [API](#api)
67- [General guidelines](#general-guidelines)
68- [Worker choice strategies](#worker-choice-strategies)
69- [Contribute](#contribute)
70- [Team](#team)
71- [License](#license)
34a572eb 72
50aa7901 73## Overview
13031992 74
22c0d9a2 75Poolifier contains two [worker_threads](https://nodejs.org/api/worker_threads.html#class-worker)/[cluster](https://nodejs.org/api/cluster.html#cluster_class_worker) worker pool implementations, you don't have to deal with [worker_threads](https://nodejs.org/api/worker_threads.html)/[cluster](https://nodejs.org/api/cluster.html) complexity.
03babf6f 76The first implementation is a fixed worker pool, with a defined number of workers that are started at creation time and will be reused.
56bbcb6b 77The second implementation is a dynamic worker pool, with a number of worker started at creation time (these workers will be always active and reused) and other workers created when the load will increase (with an upper limit, these workers will be reused when active), the new created workers will be stopped after a configurable period of inactivity.
22c0d9a2 78You have to implement your worker by extending the _ThreadWorker_ or _ClusterWorker_ class.
50aa7901
S
79
80## Installation
81
82```shell
27c5fb75 83npm install poolifier --save
1a4ec243 84```
1a4ec243 85
50aa7901
S
86## Usage
87
6ec11f90 88You can implement a [worker_threads](https://nodejs.org/api/worker_threads.html#class-worker) worker in a simple way by extending the class _ThreadWorker_:
1a4ec243
APA
89
90```js
91'use strict'
d2eb4964 92const { ThreadWorker } = require('poolifier')
1a4ec243 93
78cea37e 94function yourFunction(data) {
106744f7 95 // this will be executed in the worker thread,
96 // the data will be received by using the execute method
97 return { ok: 1 }
98}
99
50aa7901 100module.exports = new ThreadWorker(yourFunction, {
124119cf 101 maxInactiveTime: 60000
50aa7901 102})
1a4ec243
APA
103```
104
0feeec4a 105Instantiate your pool based on your needs :
1a4ec243
APA
106
107```js
108'use strict'
6961ca9a 109const { DynamicThreadPool, FixedThreadPool, PoolEvents, availableParallelism } = require('poolifier')
1a4ec243 110
2fbe1783 111// a fixed worker_threads pool
6961ca9a 112const pool = new FixedThreadPool(availableParallelism(), './yourWorker.js', {
53795b86
JB
113 errorHandler: e => console.error(e),
114 onlineHandler: () => console.info('worker is online')
115})
1a4ec243 116
2431bdb4 117pool.emitter.on(PoolEvents.ready, () => console.info('Pool is ready'))
53795b86 118pool.emitter.on(PoolEvents.busy, () => console.info('Pool is busy'))
164d950a 119
2fbe1783 120// or a dynamic worker_threads pool
31a7d5be 121const pool = new DynamicThreadPool(Math.floor(availableParallelism() / 2), availableParallelism(), './yourWorker.js', {
53795b86
JB
122 errorHandler: e => console.error(e),
123 onlineHandler: () => console.info('worker is online')
124})
106744f7 125
53795b86 126pool.emitter.on(PoolEvents.full, () => console.info('Pool is full'))
2431bdb4 127pool.emitter.on(PoolEvents.ready, () => console.info('Pool is ready'))
53795b86 128pool.emitter.on(PoolEvents.busy, () => console.info('Pool is busy'))
1a4ec243
APA
129
130// the execute method signature is the same for both implementations,
131// so you can easy switch from one to another
8351c9af 132pool
8923de44 133 .execute()
8351c9af
JB
134 .then(res => {
135 console.info(res)
136 })
137 .catch(err => {
138 console.error(err)
139 })
1a4ec243
APA
140```
141
6ec11f90 142You can do the same with the classes _ClusterWorker_, _FixedClusterPool_ and _DynamicClusterPool_.
31b90205 143
b2b02b3d
JB
144**See [examples](./examples/) folder for more details**:
145
7a923e5b 146- [Javascript](./examples/javascript/)
b2b02b3d 147- [Typescript](./examples/typescript/)
5ba0cd3c 148 - [HTTP client pool](./examples/typescript/http-client-pool/)
fac9ce96
JB
149 - [HTTP server pool](./examples/typescript/http-server-pool/)
150 - [Express](./examples/typescript/http-server-pool/express/)
a8706532 151 - [Fastify](./examples/typescript/http-server-pool/fastify/)
219d4044
JB
152 - [Websocket server pool](./examples/typescript/websocket-server-pool/)
153 - [ws](./examples/typescript/websocket-server-pool/ws/)
1a4ec243 154
6677a3d3 155Remember that workers can only send and receive structured-cloneable data.
deb85c12 156
50aa7901 157## Node versions
34a572eb 158
9e45c2c4 159Node versions >= 16.14.x are supported.
1a4ec243 160
b492f20e 161## [API](./docs/api.md)
4680c1b2 162
4f8e444d
JB
163## [General guidelines](./docs/general-guidelines.md)
164
165## [Worker choice strategies](./docs/worker-choice-strategies.md)
48211d04 166
50aa7901 167## Contribute
34a572eb 168
91633990 169Choose your task here [2.6.x](https://github.com/orgs/poolifier/projects/1), propose an idea, a fix, an improvement.
1a4ec243 170
0feeec4a
JB
171See [CONTRIBUTING](CONTRIBUTING.md) guidelines.
172
50aa7901 173## Team
1a4ec243 174
39deb558 175**Creator/Owner:**
39deb558 176
50aa7901
S
177- [**Alessandro Pio Ardizio**](https://github.com/pioardi)
178
179**_Contributors_**
180
181- [**Shinigami92**](https://github.com/Shinigami92)
182- [**Jérôme Benoit**](https://github.com/jerome-benoit)
39deb558 183
50aa7901 184## License
1a4ec243 185
9507c1d4 186[MIT](./LICENSE)