⚠️ envd is still under heavy development, and subject to change. it is not feature-complete or production-ready.

envd (ɪnˈvdɪ) provides an alternative to Docker for AI/ML applications.

🐍 Escape Dockerfile Hell - Develop with Python, save time on writing Dockerfiles, bash scripts, and Kubernetes YAML manifests

⏱️ Save you plenty of time - Build the environment up to 6x faster.

☁️ Local & cloud - envd images are OCI compatible, integrate with Docker and Kubernetes seamlessly.

🔁 Repeatable builds & reproducible results - You can reproduce the same environment on your laptop, public cloud VMs, or Docker containers, without any changes in setup.

Why use envd?

Environments built with envd provide the following features out-of-the-box:

🐍 Life is short, use Python[^1]

Development environments are full of Dockerfiles, bash scripts, Kubernetes YAML manifests, and many other clunky files that are always breaking. envd builds are isolated and clean. You can develop with Python, save time on writing Bash / Makefile / Dockerfile / ...

[^1]: The build language is starlark, which is a dialect of Python.

⏱️ 6x faster build

envd adopts a multi-level cache mechanism to accelerate the building process. For example, the PyPI cache is shared across builds and thus the package will be cached if it has been downloaded before. It saves plenty of time, especially when you update the environment by trial and error.[^2]

[^2]: Docker without buildkit

☁️ Local & cloud native

Who should use envd?

We’re focused on helping data scientists and teams that develop AI/ML models. And they may suffer from:

• building the development environments with Python/R/Julia, CUDA, Docker, SSH, and so on. Do you have a complicated Dockerfile or build script that sets up all your dev environments, but is always breaking?
• Updating the environment. Do you always need to ask infrastructure engineers how to add a new Python/R/Julia package in the Dockerfile?
• Managing environments and machines. Do you always forget which machines are used for the specific project, because you handle multiple projects concurrently?

Talk with us

💬 Interested in talking with us about your experience building or managing AI/ML applications?

Set up a time to chat!

Getting Started 🚀

Requirements

• Docker (20.10.0 or above)

Install and bootstrap envd

envd can be installed with pip (only support Python3). After the installation, please run envd bootstrap to bootstrap.

pip3 install --pre --upgrade envd
envd bootstrap

You can add --dockerhub-mirror or -m flag when running envd bootstrap, to configure the mirror for docker.io registry:

envd bootstrap --dockerhub-mirror https://docker.mirrors.sjtug.sjtu.edu.cn

Create an envd environment

Please clone the envd-quick-start:

git clone https://github.com/tensorchord/envd-quick-start.git

The build manifest build.envd looks like:

def build():
    base(os="ubuntu20.04", language="python3")
    # Configure the pip index if needed.
    # config.pip_index(url = "https://pypi.tuna.tsinghua.edu.cn/simple")
    install.python_packages(name = [
        "numpy",
    ])
    shell("zsh")

Note that we use Python here as an example but please check out examples for other languages such as R and Julia here.

Then please run the command below to set up a new environment:

cd envd-quick-start && envd up

$ cd envd-quick-start && envd up
[+] ⌚ parse build.envd and download/cache dependencies 2.8s ✅ (finished)
 => download oh-my-zsh                                                    2.8s
[+] 🐋 build envd environment 18.3s (25/25) ✅ (finished)
 => create apt source dir                                                 0.0s
 => local://cache-dir                                                     0.1s
 => => transferring cache-dir: 5.12MB                                     0.1s
...
 => pip install numpy                                                    13.0s
 => copy /oh-my-zsh /home/envd/.oh-my-zsh                                 0.1s
 => mkfile /home/envd/install.sh                                          0.0s
 => install oh-my-zsh                                                     0.1s
 => mkfile /home/envd/.zshrc                                              0.0s
 => install shell                                                         0.0s
 => install PyPI packages                                                 0.0s
 => merging all components into one                                       0.3s
 => => merging                                                            0.3s
 => mkfile /home/envd/.gitconfig                                          0.0s
 => exporting to oci image format                                         2.4s
 => => exporting layers                                                   2.0s
 => => exporting manifest sha256:7dbe9494d2a7a39af16d514b997a5a8f08b637f  0.0s
 => => exporting config sha256:1da06b907d53cf8a7312c138c3221e590dedc2717  0.0s
 => => sending tarball                                                    0.4s
envd-quick-start via Py v3.9.13 via 🅒 envd 
⬢ [envd]❯ # You are in the container-based environment!

Set up Jupyter notebook

Please edit the build.envd to enable jupyter notebook:

def build():
    base(os="ubuntu20.04", language="python3")
    # Configure the pip index if needed.
    # config.pip_index(url = "https://pypi.tuna.tsinghua.edu.cn/simple")
    install.python_packages(name = [
        "numpy",
    ])
    shell("zsh")
    config.jupyter()

You can get the endpoint of the running Jupyter notebook via envd envs ls.

$ envd up --detach
$ envd envs ls
NAME                    JUPYTER                 SSH TARGET              CONTEXT                                 IMAGE                   GPU     CUDA    CUDNN   STATUS          CONTAINER ID
envd-quick-start        http://localhost:42779   envd-quick-start.envd   /home/gaocegege/code/envd-quick-start   envd-quick-start:dev    false   <none>  <none>  Up 54 seconds   bd3f6a729e94

More on documentation 📝

See envd documentation.

Roadmap 🗂️

Please checkout ROADMAP.

Contribute 😊

We welcome all kinds of contributions from the open-source community, individuals, and partners.

• Join our discord community!
• To build from the source, please read our contributing documentation and development tutorial.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Friends A. 📖 🎨	Aaron Sun 📓 💻	Aka.Fido 📦 📖 💻	Bingyi Sun 💻	Ce Gao 💻 📖 🎨 📆	Guangyang Li 💻	Gui-Yue 💻
Haiker Sun 💻	Ikko Ashimine 💻	Jian Zeng 🎨 🤔 🔬	Jinjing Zhou 🐛 💻 🎨 📖	Jun 📦 💻	Keming 💻 📖 🤔 🚇	Kevin Su 💻
Ling Jin 🐛 🚇	Manjusaka 💻	Nino 🎨	Pengyu Wang 📖	Sepush 📖	Siyuan Wang 💻 🚇 🚧	Wei Zhang 💻
Xu Jin 💻	Xuanwo 💬 🎨 🤔 👀	Yuan Tang 💻 🎨 📖 🤔	Yuchen Cheng 🐛 🚇 🚧 🔧	Yunchuan Zheng 💻	Zheming Li 💻	Zhenguo.Li 💻 📖
Zhenzhen Zhao 🚇 📓 💻	Zhizhen He 💻 📖	jimoosciuc 📓	kenwoodjw 💻	nullday 🤔 💻	wyq 🐛 🎨 💻	xiangtianyu 📖
xing0821 🤔 📓 💻	zhyon404 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

License 📋

Apache 2.0