Vite Dev Server Transforms

3 April 2022

While creating vite-plugin-rss, I ran into a peculiar issue of serving an in memory XML file using Vite’s dev server.

For a little background, vite-plugin-rss has two modes, 'define' and 'meta', of which only 'meta' is currently relevant. In 'meta' mode, the plugin uses Rollup’s inter-plugin communication to pass RSS data from another plugin’s load or tranform step into vite-plugin-rss. Another plugin can add meta data to a module which vite-plugin-rss can read at the buildEnd phase to generate RSS XML items.

// ...
buildEnd() {
  if (!items && opts.mode === "meta") {
    const moduleIds = Array.from(this.getModuleIds()); // <- get the module ids
    const moduleInfo = moduleIds.map((id) => this.getModuleInfo(id)); // <- find the module info from the ids
    items = moduleInfo // <- get our RSS items
      .filter((module): module is ModuleInfo => !!module?.meta.rssItem)
      .map((module) => module.meta.rssItem);
  }

  // generate the RSS XML
  const renderedXML = createRssFeed(opts.channel, items, fileName);

  // add it as an emitted asset
  this.emitFile({
    fileName: fileName,
    name: fileName,
    source: renderedXML,
    type: "asset",
  });
}

Our issue arises when we want to generate our RSS XML file during development. I did a little experimentation and found that the buildEnd phase is not run when the dev server is started. In fact, a lot of Rollup’s plugin phases are not run. Only the load and tranform phases are run when using Vite’s dev server.

For a little background, we should discuss how the Vite dev server works. Vite utilizes native browser ES Module support to do minimal transforms before serving files. Raw ESM files (or modules) can be sent to the browser without having to generate an entire bundle during development. This provides faster hot module reload times and an overall better developer experience.

Now, let’s dig into the source code for Vite’s dev server. The Vite dev server uses a variety of middleware to serve different kinds of files.

// Internal middlewares ------------------------------------------------------

// request timer
if (process.env.DEBUG) {
  middlewares.use(timeMiddleware(root));
}

// cors (enabled by default)
const { cors } = serverConfig;
if (cors !== false) {
  middlewares.use(corsMiddleware(typeof cors === "boolean" ? {} : cors));
}

// proxy
const { proxy } = serverConfig;
if (proxy) {
  middlewares.use(proxyMiddleware(httpServer, config));
}

// base
if (config.base !== "/") {
  middlewares.use(baseMiddleware(server));
}

// open in editor support
middlewares.use("/__open-in-editor", launchEditorMiddleware());

// hmr reconnect ping
// Keep the named function. The name is visible in debug logs via `DEBUG=connect:dispatcher ...`
middlewares.use("/__vite_ping", function viteHMRPingMiddleware(_, res) {
  res.end("pong");
});

// serve static files under /public
// this applies before the transform middleware so that these files are served
// as-is without transforms.
if (config.publicDir) {
  middlewares.use(servePublicMiddleware(config.publicDir));
}

// main transform middleware
middlewares.use(transformMiddleware(server));

// etc...

For example, there is middleware for configuring CORS or serving static files from the public directory.

The middleware we will focus on is the transformMiddlware. This is where the module transformation happens. When a transformation happens, Vite needs to know what a module’s dependencies are and transform those modules as well.

Modules are tracked in a ModuleGraph which acts as a dependency graph and cache for a given project.

export class ModuleGraph {
  urlToModuleMap = new Map<string, ModuleNode>()
  idToModuleMap = new Map<string, ModuleNode>()
  // a single file may corresponds to multiple modules with different queries
  fileToModulesMap = new Map<string, Set<ModuleNode>>()
  safeModulesPath = new Set<string>()

// ...

The ModuleGraph tracks each module as a ModuleNode which records meta data about the last transformation time, id, file type, and more.

export class ModuleNode {
  /**
   * Public served url path, starts with /
   */
  url: string;
  /**
   * Resolved file system path + query
   */
  id: string | null = null;
  file: string | null = null;
  type: "js" | "css";
  info?: ModuleInfo;
  meta?: Record<string, any>;
  importers = new Set<ModuleNode>();
  importedModules = new Set<ModuleNode>();
  acceptedHmrDeps = new Set<ModuleNode>();
  isSelfAccepting = false;
  transformResult: TransformResult | null = null;
  ssrTransformResult: TransformResult | null = null;
  ssrModule: Record<string, any> | null = null;
  lastHMRTimestamp = 0;
  lastInvalidationTimestamp = 0;

  constructor(url: string) {
    this.url = url;
    this.type = isDirectCSSRequest(url) ? "css" : "js";
  }
}

When a file is requested by the client, the dev server will go to look up the file in the module graph. Then, Vite will transform the requested modules using a plugin container.

The plugin container is a way to run just the Rollup load and transform phases on a given module. The code was adapted from the Rollup plugin container created by wmr.

Okay, so now we have a rough idea of what’s going on in the dev server. As for my RSS XML file problem, I need to some how extract module info from the load and transform steps for use in the configureServer phase.

Luckily, we can get all this information from server configuration object. The server provides references to both server.pluginContainer and server.moduleInfo. From these two objects, we can get the same information that we were originally able to get in the buildEnd phase.

configureServer(server) {
  // serve feed.xml on dev server
  server.middlewares.use((req, res, next) => {
    if (
      typeof req.url === "string" &&
      new RegExp(`${fileName}$`).test(req.url)
    ) {
      if (!items && opts.mode === "meta") {
        const devServerModuleIds = Array.from(
          server.moduleGraph.idToModuleMap.keys() // <- get the module ids
        );
        const moduleInfo = devServerModuleIds.map((id) =>
          server.pluginContainer.getModuleInfo(id) // <- find the module info from the ids
        );

        items = moduleInfo // <- get our RSS items
          .filter((module): module is ModuleInfo => !!module?.meta.rssItem)
          .map((module) => module.meta.rssItem);
      }

      const renderedXML = createRssFeed(opts.channel, items ?? [], fileName);

      const fileContent = Buffer.from(renderedXML, "utf8");
      const readStream = new stream.PassThrough();
      readStream.end(fileContent);
      res.writeHead(200, {
        "Content-Type": "text/xml",
      });
      readStream.pipe(res);
      return;
    }
    next();
  });
}

Now we can serve the same RSS XML file using both the dev server and in production builds. This is especially helpful for testing the RSS XML output as I can point a local RSS Reader at the dev server’s URL to test the validity of the generated XML.