[Question] Next.js compatibility

[aczekajski]

Hi!
Just wanted to ask if there are any known cases of this plugin being successfully used with next.js? 😊

Asking because I've seen cases where next.js might encounter problems when used with webpack plugins.

[asos-tomp]

See this project, which has supposedly found a way with a slight tweak: https://github.com/dlehmhus/next-with-linaria/blob/main/src/plugins/webpack-virtual-modules/index.ts

...Would be great to understand if the plugin itself can be updated to support NextJs (or ultimately any multi-compiler setup, I guess)

[larixer]

@asos-tomp They use older version of this plugin (commit id: aa8a334) plus literally the only change below:

   public writeModule(filePath: string, contents: string): void {
+    // next-with-linaria patch, if not initialized yet, add to static modules
     if (!this._compiler) {
-      throw new Error(`Plugin has not been initialized`);
+      if (!this._staticModules) {
+        this._staticModules = {};
+      }
+      this._staticModules[filePath] = contents;
+      return;
     }

[asos-tomp]

Ah OK - the suggestion here is that its keeping multiple compiler instances in sync, perhaps somewhere else in the repo.

you actually can use webpack-virtual-modules with Next.js 13, but you have to create a new instance of webpack-virtual-modules for every webpack instances that Next.js launches and then keep them in sync. That is basically how next-with-linaria works. (It is definitely hacky, but it works)

[asos-tomp]

I've found that disabling the SWC compiler (which can be achieved by using a .babelrc file thus):

{
  "presets": ["next/babel"]
}

...causes virtual modules to be accepted by NextJs. Not ideal, but hopefully an issue can be raised with Next to resolve this.

[larixer]

@asos-tomp Could you create a small demo repo on GitHub which showcases the problem with next.js and this plugin?

[asos-tomp]

Frankly, I now can't see an issue.

See: https://stackblitz.com/edit/nextjs-4byas6 and https://stackblitz.com/edit/nextjs-fnmbmj (no .babelrc)

This appears to have a virtual module loaded successfully, even using the SWC compiler. Unclear why I am having no joy in my project, but I will go back to basics!

Thanks for the prompt...

[asos-tomp]

In-fact, if I download the repo and run locally, I get:

Error: Failed to read source code from <local path>/app/test-module.js

Caused by:
    No such file or directory (os error 2)

Import trace for requested module:
./app/test-module.js
./app/page.tsx

...however, if I add a .babelrc:

{
  "presets": ["next/babel"]
}

...thus producing:

Disabled SWC as replacement for Babel because of custom Babel configuration ".babelrc" https://nextjs.org/docs/messages/swc-disabled

...then re-run npm run dev, all is well.

Is there a chance SWC doesn't actually run on StackBlitz, either way?

[larixer]

@asos-tomp Thanks, I can repro. I will take a look

[larixer]

@asos-tomp In the case of your demo repo the SWC is a native compiler that reads file system directly and not through the built-in webpack file system access and caching mechanisms. Unfortunately for this kind of setup the virtual modules are just impossible / should be accomplished through other operation system dependent means. It is possible to add multi compiler support to webpack-virtual-modules, so that the plugin kept virtual modules in sync across different compilers, but it won't work for your demo.

As you already figured out, if SWC or other native compilers are not used the plugin can be relatively easily adapted for multi compiler setup the way you do in your demo repo.

[asos-tomp]

Thanks so much for your time looking into this.

I fear that the purpose of SWC is speed/efficiency, perhaps forgoing some of the filesystem flexibilities innate in WebPack. This may scupper any attempt at virtualisation.

[larixer]

@asos-tomp Btw, you can try to workaround this:
vercel/next.js#31682

If you make sure that next-swc-loader is not the last in the loader list in webpack config, theoretically, all the file reading should go through webpack in this case, no native reads should happen and virtual modules should work.

[asos-tomp]

@asos-tomp Btw, you can try to workaround this: vercel/next.js#31682

If you make sure that next-swc-loader is not the last in the loader list in webpack config, theoretically, all the file reading should go through webpack in this case, no native reads should happen and virtual modules should work.

Ah great idea, I will give that a go!

[asos-tomp]

I can confirm that injecting a loader into the compilation for the virtual modules defuses the attempt to by SWC to load the module directly from disk, thus preventing error.

./loader.cjs:

module.exports = function (source) {
  this.callback(null, source);
};

plugin:

    compiler.hooks.compilation.tap(PLUGIN_NAME, (compilation) => {
      NormalModule.getCompilationHooks(compilation).beforeLoaders.tap(
        PLUGIN_NAME,
        (loaders, normalModule) => {
          if (aFunctionToIdentifyTheModuleAsVirtual(normalModule)) {
            loaders.push({
              loader: new URL(import.meta.resolve("./loader.cjs")).pathname
            });
          }
        }
      );
    });

[larixer]

@asos-tomp Great find! Thank you very much for your efforts to investigate this issue and finding a working workaround!

[asos-tomp]

N.B. For those battling to get NormalModule.getCompilationHooks to work, understand that NextJs has a "pre-compiled" bundle of webpack so:

import { NormalModule } from "webpack";

...isn't going to work - it will complain:

"The 'compilation' argument must be an instance of Compilation"

...due to the check here for a matching webpack runtime.

Instead you must alias to:

import NormalModule from "next/dist/compiled/webpack/NormalModule.js";