Making a Telescope.nvim Extension

30 August 2022

github

At Gatsby we use TypeScript across the stack in a Yarn (Berry) PnP monorepo. There are over 50 packages at this point and being able to navigate through these packages quickly is important for my workflow.

On the command line, I have various Bash functions to fly around. For example, the completion scripts I have setup allow me to quickly move between internal packages.

# cd into repo/packages or any subdirectory

cdgp () {
  local gitroot=`git rev-parse --show-toplevel`
  cd "$gitroot/packages/$1"
}
_cdgp_completion()
{
  local gitroot=`git rev-parse --show-toplevel`
  COMPREPLY="$(ls $gitroot/packages/)"
}
complete -F _cdgp_completion cdgp

In contracts, for Neovim, I have been painstakingly using :cd ../../packages or some derivative to navigate. This gets slow and cumbersome if my current working directory is too deeply nested.

I need something to move my cwd around and Telescope.nvim happens to be just the ticket. Telescope.nvim is a modern extendable fuzzy finder plugin for Neovim. The idea is to create a plugin that can find every package in the workspace, show them in the Telescope picker, and then change my cwd on selection.

Often times, with personal projects, I bite off more than I can chew. I set lofty goals and then give up when I feel like progress is too slow. So to get started quickly, I would only support the bare necessities:

Only Yarn Berry Support
Everything lives in my dotfiles
No file preview is necessary, just a picker

Setting up these ground rules helped me avoid getting blocked too early.

Getting Workspace Packages

Before I can get into creating a plugin, I need to figure out a way to extract a list of workspace packages. Yarn Berry comes with the handy yarn workspaces list command which gives an easily parsable output.

~/code/yarn-v2-workspace ❯ yarn workspaces list
➤ YN0000: .
➤ YN0000: packages/a
➤ YN0000: packages/b
➤ YN0000: Done in 0s 3ms

But we can make it much easier to parse with the --json flag.

~/code/yarn-v2-workspace ❯ yarn workspaces list --json
{"location":".","name":"yarn-v2-workspace"}
{"location":"packages/a","name":"a"}
{"location":"packages/b","name":"b"}

Parsing JSON

Yarn’s JSON output is pretty good but we don’t have a way to parse JSON natively in Lua-JIT. Luckily, there’s an open source module called json.lua that can do this for us. If this were JavaScript, I would have probably gone ahead at this point and npm installed this package, but this isn’t JavaScript.

Most Neovim plugins focus on being self contained without any external dependencies that aren’t other Neovim plugins. Lua does have a package manager called luarocks but I don’t want consumers of this plugin to have to download additional dependencies. So, I’m left with two choices:

Use the module as a git sub-module
Copy and paste

Looking at the commit history of json.lua, there haven’t been any updates in about 2 years. It might be better to use a git sub-module but I don’t think this repo will get any major updates any time soon. The entire module is a single file so I decided to copy and paste it into my repo (keeping relevant license information and attributions).

Understanding Telescope

Whenever I start a new project or learn a new stack, I start by reading. Today’s reading material happens to be the wonderful Telescope developer documentation. I’ve found that reading docs and fully understanding how something works can save a lot of time, stack overlow searches, and struggles later on.

Telescope’s API is fairly simple. There are pickers out of which you can pick items. Pickers contain a list of items supplied by a finder. Pickers also do sorting which can be configured by a sorter. When an item is selected, there is an event that triggers an action which does something with the chosen item. Telescope comes with some good defaults for finders, sorters, and actions. I didn’t have to deviate too much from these defaults.

This is what I ended up with after story time (i.e. documentation reading time):

pickers.new(opts, {
  prompt_title = "Node Workspaces - " .. package_manager,
  -- simple default finder
  finder = finders.new_table {
    results = workspace_keys,
  },
  -- simple default sorter
  sorter = conf.generic_sorter(opts),
  -- a little more complexity in the action mapping
  -- but it's pretty much just taking the selected item
  -- and looking up it's directory from a map
  attach_mappings = function(prompt_bufnr, map)
    actions.select_default:replace(function()
      actions.close(prompt_bufnr)
      local selection = action_state.get_selected_entry()
      local key = selection[1]
      local dir = workspaces[key]

      local path = vim.fs.normalize(workspace_root .. "/" .. dir)
      -- this is where we change the cwd
      vim.api.nvim_set_current_dir(path)
    end)
    return true
  end,
  }):find()

We now have something that works! It does everything I initially set out to do:

Only Yarn Berry Support
Everything lives in my dotfiles
No file preview is necessary, just a picker

Just pretend the screenshot doesn’t say ‘pnpm’. I forgot to take a picture at this point.

Building Out Compatability

With a feeling of accomplishment, I could either stop here or make the plugin more robust. So I brought forward a new set of goals that had been the in back of my mind:

Support for all major package managers (npm, yarn, pnpm, and yarn-berry)
Extract the extension into its own repository for distribution
Show a preview of the package’s package.json

Detecting the Package Manager

First things first, we need to figure out what package manager is being used. I had a general idea of how to detect a package manager, but Google helped me solidify my fuzziness. A quick Google search for “How to detect JavaScript package manager” led me to the detect-package-manager module. Sure this thing is written in JavaScript but it’s already worked out all the logic I need. And yep, lines 46-50 tell me everything I need to know:

// ...
  return Promise.all([
    pathExists(resolve(cwd, "yarn.lock")),
    pathExists(resolve(cwd, "package-lock.json")),
    pathExists(resolve(cwd, "pnpm-lock.yaml")),
  ]).then(([isYarn, isNpm, isPnpm]) => {
// ...

And re-implementing that logic in Lua:

--- detect which node package manager is being used
---@param root_path string path to workspace root
---@param package_json table parsed package.json
---@return string one of 'npm', 'yarn', or 'pnpm'
local function detect_package_manager(root_path, package_json)
    local res = "npm"
    if file_exists(root_path .. "/yarn.lock") then
        res = "yarn"
    elseif file_exists(root_path .. "/pnpm-lock.yaml") then
        res = "pnpm"
    end

    if res == "yarn" then
        -- check packagejson for packageManager to see if yarn berry
        if
            package_json.packageManager ~= nil
            and package_json.packageManager[6] ~= "1"
        then
            res = "yarn-berry"
        end
    end

    return res
end

Parsing Workspace Output

All of the major pacakage managers (npm, pnpm, yarn, and yarn-berry) all support workspaces, but they all do workspace package listing a little differently.

# npm - gives proper JSON output
# caveat: must be executed from the root of the workspace
npm list -json -depth 1 -omit=div

# pnpm - gives proper JSON output
pnpm ls --json -r

# yarn v1 - gives proper JSON output
# caveat: must remove first and last lines of output for proper JSON
yarn workspaces info

# yarn-berry - gives proper JSON output on multiple lines
yarn workspaces list --json

I won’t delve into the details, but I had to figure out ways to parse the different JSON outputs for each package manager.

Using a Previewer

We still have yet to cover the telescope previewer. This is the thing that shows a preview of the item when your cursor is over an item. For example, for file pickers, it will show a preview of the file.

I was stumped for a bit on how to get a previewer to work but I remembered to turn back to the trusty documentation. And there it is, with the entry_maker field. With entry-maker we can continue to use the default file previewer to show the selected package’s package.json.

Putting it all together, along with a few refactors:

pickers.new(opts, {
  prompt_title = "Node Workspaces - " .. package_manager,
  -- default finder using `entry_maker`
  finder = finders.new_table {
    results = workspaces,
    entry_maker = function(entry)
      return {
        -- add the path field for the previewer to read
        path = vim.fs.normalize(entry[2] .. "/package.json"),
        value = entry[2],
        display = entry[1],
        ordinal = entry[1],
      }
    end,
  },
  -- default sorter
  sorter = conf.generic_sorter(opts),
  -- default file previwer
  previewer = conf.file_previewer(opts),
  -- mapping to change our cwd
  attach_mappings = function(prompt_bufnr, _)
    actions.select_default:replace(function()
      actions.close(prompt_bufnr)
      local selection = action_state.get_selected_entry()
      vim.api.nvim_set_current_dir(selection.value)
    end)
  return true
  end,
})
:find()

As you can see, we’re still just using a lot of Telescope’s default configurations. The end result looks something like this:

Overall, Telescope.nvim has a really well built API. The default finder, sorter, and previewers were more than enough for this extension. The documentation is well written and easy to follow. Anything not included in the guide can be found in the :help docs or by using other extensions as examples.

I didn’t allow myself to get caught up with lofty goals and focused on getting a quick win. I used open source code to both save time and avoid re-inventing the wheel. Then I came back to flesh out my ideas and add greater usability.

You can see the completed plugins source code on GitHub.

Update

9/5/2022

I’ve discovered that Neovim has builtin JSON serialization. I’ve removed json.lua and replaced it with this builtin function. It pays to read the docs!