🎁 Package management

Pluto's built-in package management

Starting with version 0.15, Pluto has a built-in package manager, which means:

1. 🎈 Packages are automatically installed when you use import or using.
2. 🎈 Your package environment is stored inside the notebook file. When someone else opens your notebook with Pluto, the exact same package environment will be used, and packages will work on their computer.

🙋 These two features are designed to make it easy to write and share reproducible notebooks.

Basic usage

Installing and using packages

Pluto will automatically install or remove packages while you work on your notebook. When you import a new package, Pluto will install it:

🙋 Most packages will write installation instructions in their documentation: like "Run julia> ] install Example to install Example". If you are using Pluto, you can skip these instructions, and import the package directly, using import Example or using Example.

Logs

Installing packages can take some time, especially when starting Julia for the first time. Click on the status mark next to a package to view the installation progress. You can click on the icon to view the logs.

Removing packages

When you delete the code that imports a package, it will be uninstalled from the package environment. It is recommended to restart the notebook process afterwards to get a fresh start.

Updating packages

You can search for and install any available updates by clicking on the icon. A backup of your notebook file will be created in the same folder as your notebook, in case the new versions do not work as expected.

Good to know

Based on Pkg

Pluto's package management is a wrapper around Pkg.jl, Julia's built-in package manager. Packages are installed from the General registry.

🙋 You can discover all available packages on juliahub.com.

Isolated package environment

Every notebook runs in its own isolated package environment. This ensures that your notebook code will not be influenced by packages installed elsewhere. (The LOAD_PATH is set to ["@", "@stdlib"].)

Notebook file

Pluto stores the contents of Project.toml and Manifest.toml directly in the notebook file. For forwards-backwards compatibility, this is done using two extra "cells" at the bottom of the file, containing the two files as string literals. For example, here is a notebook that imports HypertextLiteral and PlutoUI: fonsp/Pluto file format demo.jl.

🙋 We encourage you to open Pluto, import some packages and look at the file!

Fully reproducible environments

When someone else opens your notebook for the first time, Pluto will install all required packages based on the information in the Manifest.toml file. This means that the exact same versions of all packages you used in the notebook will be installed, ensuring your work is fully reproducible when shared with others.

Compatibility across Pluto versions

When opening an old Pluto notebook that does not have embedded project files, Pluto will generate them as if you typed those imports for the first time. If a call to Pkg.activate is made, the notebook will run in 'backwards compatibility mode', using the same environment and behaviour as old Pluto versions.

Compatibility across Julia versions

The Manifest.toml is designed to be (generally) backwards compatible: you can upgrade Julia and use an old manifest. However, the Manifest.toml is not always forwards compatible: a manifest generated generated with a newer version of Julia might not run on older versions.

Pluto will always try to load the embedded manifest, and if it fails, it will discard the manifest (leaving only Project.toml) and try again. This is one reason why Pluto automatically adds [compat] ranges for each package in the Project.toml.

Package installations are shared between environments

In Julia, packages are always installed globally, while environments (including notebooks) only store version information, not the package code itself. This means that multiple notebooks that use the same version of Plots will not lead to more disk usage or precompile time. (This is different from venv in Python/pip, or node_modules in NodeJS/npm, where every environment contains copies of package code by default.) Julia's package manager is designed to work well with many package environments on the same computer.

Of course, if different notebooks use different version of the same package, and you work on both in parallel, then source files for both versions will exist on your computer. A small price to pay for simpler reproducibility!

In Julia 1.5 and 1.6, you can clean up package files that have not been used in a while using import Pkg; Pkg.gc(). In Julia 1.7 and up, this will be done automatically.

Custom registries

Pkg.jl supports additional private or public registries, which can be added in the Julia REPL with ] registry add https://github.com/myuser/MyRegistry.git, and this is also supported by Pluto's package managemer.

However, note that registries are not stored in the Project.toml/Manifest.toml files, which means that other people can only open your notebook if they added the custom registry before doing so. Alternatively, you can use a "Pkg cell" (more on this later) where you add the registry before adding packages.

Advanced: set up an environment with Pkg.activate

Pluto's package manager is enabled for all users, for ease of use and to promote reproducibility in scientific computing. There is no option to disable the behaviour globally (for your entire Pluto session). Instead, Pluto will detect notebooks that use Pkg.activate to set up an environment explicitly, and uses the old behaviour for those notebooks.

The philosophy here is that everyone should have a reproducible package environment by default, without having to do anything. This takes priority over other use cases, and hence not using the built-in package manager requires some extra work.

Any notebook that calls Pkg.activate will not use Pluto's package management, and run in 'backwards compatibility mode'. The Pkg.activate call should be placed directly in your notebook code: it is detecting using the same syntax analysis used for reactivity.

Pattern: The "global environment"

If you do not intend on sharing a notebook file and you want to use your global package environment (called (v1.6) or similar, the one you get when you launch the Julia REPL), then you can call Pkg.activate() without any arguments.

🙋 If you are developing a package, then activating your global environment is an easy way to test your local version in Pluto.

This "global environment" pattern can be placed at the top of a notebook:

begin
    import Pkg
    # careful: this is _not_ a reproducible environment
    # activate the global environment
    Pkg.activate()

    using Plots, PlutoUI, LinearAlgebra
end

When running this in Pluto (try it out!), you will notice that the status marks next to packages disappear, and Pluto is running in 'backwards compatibility mode'. Packages will no longer be installed or removed automatically, you have to use the Pkg REPL to do this yourself.

Pattern: The "shared environment"

If you have multiple notebooks in a repository and you want to use share a Pkg environment between them, then you can call Pkg.activate(path_to_environment). You can use @__DIR__ to get the path of the notebook's folder, and joinpath(@__DIR__, "..") to get its parent, joinpath(@__DIR__, "..", "..") for the parent's parent, etc.

The function Base.current_project() can be used to automatically find the closest parent directory that contains a Project.toml, in most cases this is what you want.

For example, if your project looks like this:

my_project/
    data/
        ...
    notebooks/
        Interesting analysis.jl
        ...
    Project.toml
    Manifest.toml
    ...

then the "shared environment" pattern can be placed at the top of a notebook:

begin
    import Pkg
    # activate the shared project environment
    Pkg.activate(Base.current_project())
    # instantiate, i.e. make sure that all packages are downloaded
    Pkg.instantiate()

    using Plots, PlutoUI, LinearAlgebra
end

When running this in Pluto, you will notice that the status marks next to packages disappear, and Pluto is running in 'backwards compatibility mode'. Packages will no longer be installed or removed automatically, you have to use the Pkg REPL to do this yourself.

Pattern: The "Pkg cell"

When adding packages, Pluto's default package management will always install the latest version from the registry. If you need to install a specific version or branch of a package, or a package is not registered, you can use a "Pkg cell".

A common pattern is a so-called "Pkg cell", placed at the top of a notebook:

begin
    import Pkg
    # activate a temporary environment
    Pkg.activate(mktempdir())
    Pkg.add([
        Pkg.PackageSpec(name="Plots", version="1"),
        Pkg.PackageSpec(name="PlutoUI", version="0.7"),
    ])
    using Plots, PlutoUI, LinearAlgebra
end

This will 1) activate a temporary environment using Pkg.activate, 2) add the required packages, 3) import them with using. When running this in Pluto (try it out!), you will notice that the status marks next to packages disappear, and Pluto is running in 'backwards compatibility mode'. Packages will no longer be installed or removed automatically, you have to use the Pkg REPL to do this yourself.

Placing all code in a single begin block ensures that the lines will run in the correct order.

🙋 You can use this helper tool to generate a "Pkg cell" automatically!

Advanced: edit the notebook environment

Pluto.jl includes a helper function Pluto.activate_notebook_environment that activates a notebook Pkg environment in the REPL:

julia> import Pluto

julia> Pluto.activate_notebook_environment("~/Documents/hello.jl")

julia> ]

(hello.jl) > status

After activating a notebook environment, you can use the Pkg REPL to view or modify the embedded environment. Changes from either side are synchronised (i.e. Pkg REPL changes are written to the notebook, editing the notebook updates the Pkg REPL env). Watch the demo video:

Schermopname.2021-03-17.om.15.19.47.mov