Webpack4 thread-loader bring little optimization for babel-loader. Is something wrong, or why?

As mentioned in Webpack docs and many information that introducing how to improve webpack build speed, thread-loader can obviously improve the build speed of babel-loader.

But when I try it in my project, it seems like that thread-loader does not bring any optimization for babel-loader.

I use speed-measure-webpack-plugin to measure build speed, and there are the code and result of my test.

my device: Macbook Pro (15-inch, 2018, 6-core, 12-thread)

version information:

{ "webpack": "^4.41.2", "thread-loader": "^3.0.1", "babel-loader": "^8.2.2", "webpack-preprocessor-loader": "^1.1.3", } 

Before using thread-loader for babel-loader, code of webpack.config.js:

// thread-loader rule: { test: /\.(jsx?)$/, exclude: [ /node_modules/, ], use: [ { loader: 'babel-loader', options: { babelrc: true, cacheDirectory: true, rootMode: 'upward', } }, { loader: 'webpack-preprocessor-loader', options: { params: env } }] } 

Time cost:

 SMP ⏱ Loaders thread-loader, and babel-loader, and webpack-preprocessor-loader took 15.71 secs module count = 315 

After:

// thread-loader warmup: const threadLoader = require('thread-loader'); const threadLoaderOptions = { poolTimeout: 2000, name: 'thread-loader-for-babel' } threadLoader.warmup(threadLoaderOptions, [ 'webpack-preprocessor-loader', 'babel-loader' ]); // thread-loader rule: { test: /\.(jsx?)$/, exclude: [ /node_modules/, ], use: [ { loader: 'thread-loader', options: threadLoaderOptions }, { loader: 'babel-loader', options: { babelrc: true, cacheDirectory: true, rootMode: 'upward', } }, { loader: 'webpack-preprocessor-loader', options: { params: env } }] } 

Time cost:

 SMP ⏱ Loaders thread-loader, and babel-loader, and webpack-preprocessor-loader took 14.64 secs module count = 313 

15.71s VS 14.64s, this effect is not obvious.

thread-loader change babel-loader work from serial execution to parallel execution, theoretically in my device with 6 cores and 12 threads, even if you factor in the worker startup time and other factors, there will be at least several times improvement, but it doesn't.

Is the thread-loader used correctly? (It should be correct, I have tried to modify the source code of thread-loader and log to ensure that multiple workers are running) If it is used correctly, what is the reason for the effect not obvious?

I have a guess. For modules that hit the same loader, webpack 4 has built-in multi-thread processing, that is, webpack has already done thread-loader work. So thread-loader has little effect. But this guess needs to go deep into the webpack 4 source code to verify.

(I will continue to study, but it will take some time. If there are anyone that have been studied, please share it)

thanks in advance. :)

1 Answer

Possible reasons as to why it's not as fast, is these reasons (found on the documentation):

Loaders running in a worker pool are limited. Examples: Loaders cannot emit files. Loaders cannot use custom loader API (i. e. by plugins). Loaders cannot access the webpack options.