Environments

When you run your experiments in Neptune cloud (with neptune send or in a Jupyter notebook), you’re able to specify which machine learning libraries should be available. You can do it using environments (docker images in which your experiment runs).

You are able to select an environment via an --environment CLI aragument, or from the UI, when starting a notebook.

Environments are related to worker types - the physical or virtual machines that your experiments run on. Choosing environment that requires a GPU (they are named gpu-*) implies you need a worker with GPU.

Let’s say you want to run an experiment that utilizes Keras in Python 3 on a worker with GPU. The following command will execute your code within an environment that provides Keras and Python 3 on a worker with GPU.

neptune send --environment keras-2.2-gpu-py3 --worker m-k80

Available Workers

See neptune.ml for details on available worker types along with pricing.

Preemptible Workers

Neptune provides preemptible workers which are highly affordable but potentially short-lived compute instances. Preemptible workers offer the same machine types as regular worker types for up to 5 times cheaper than regular price but they can be preempted at any point of experiment’s execution and cannot run for longer than 24 hours. For detailed pricing visit neptune.ml

These types of workers are suitable for experiments which can snapshot theirs execution state frequently and continue execution from previously snapshotted state.

Preempted experiment will end up in preempted state.

To run experiment on preemptible s-k80 worker use --worker s-k80-preemptible option in neptune send command.

Tip

You can use experiment’s output as input in another experiment. Simply add --input /SHORT_EXP_ID/output in new experiment’s inputs.

Available Environments

Note

All our environments come with some pre-installed libraries:

dlib, gym, h5py, ipykernel, matplotlib, nltk, numpy, opencv-python, pandas, pillow, scikit-learn, scikit-image, scipy, sklearn, spacy, sympy, Cython, theano 0.9, tensorflow 1.9, keras 2.2, pytorch 0.4.0, cupy-cuda90, chainer

This list is later adjusted for specific environments

Name CPU/GPU Python Version Additionally Installed Libraries
base CPU Python 2.7
base-gpu GPU Python 2.7
base-gpu-py3 GPU Python 3.6
base-cpu-py3 CPU Python 3.6
tensorflow-1.9 CPU Python 2.7 tensorflow 1.9
tensorflow-1.9-gpu GPU Python 2.7 tensorflow 1.9
tensorflow-1.9-gpu-py3 GPU Python 3.6 tensorflow 1.9
tensorflow-1.9-cpu-py3 CPU Python 3.6 tensorflow 1.9
tensorflow-1.6 CPU Python 2.7 tensorflow 1.6
tensorflow-1.6-gpu GPU Python 2.7 tensorflow 1.6
tensorflow-1.6-gpu-py3 GPU Python 3.6 tensorflow 1.6
tensorflow-1.6-cpu-py3 CPU Python 3.6 tensorflow 1.6
tensorflow-1.5 CPU Python 2.7 tensorflow 1.5
tensorflow-1.5-gpu GPU Python 2.7 tensorflow 1.5
tensorflow-1.5-gpu-py3 GPU Python 3.6 tensorflow 1.5
tensorflow-1.5-cpu-py3 CPU Python 3.6 tensorflow 1.5
keras-2.2 CPU Python 2.7 keras 2.2, tensorflow 1.9
keras-2.2-gpu GPU Python 2.7 keras 2.2, tensorflow 1.9
keras-2.2-gpu-py3 GPU Python 3.6 keras 2.2, tensorflow 1.9
keras-2.2-cpu-py3 CPU Python 3.6 keras 2.2, tensorflow 1.9
keras-2.1.4 CPU Python 2.7 keras 2.1.4, tensorflow 1.9
keras-2.1.4-gpu GPU Python 2.7 keras 2.1.4, tensorflow 1.9
keras-2.1.4-gpu-py3 GPU Python 3.6 keras 2.1.4, tensorflow 1.9
keras-2.1.4-cpu-py3 CPU Python 3.6 keras 2.1.4, tensorflow 1.9
keras-2.0 CPU Python 2.7 keras 2.0, tensorflow 1.9
keras-2.0-gpu GPU Python 2.7 keras 2.0, tensorflow 1.9
keras-2.0-gpu-py3 GPU Python 3.6 keras 2.0, tensorflow 1.9
keras-2.0-cpu-py3 CPU Python 3.6 keras 2.0, tensorflow 1.9
pytorch-0.4.0 CPU Python 2.7 pytorch 0.4.0
pytorch-0.4.0-gpu GPU Python 2.7 pytorch 0.4.0
pytorch-0.4.0-gpu-py3 GPU Python 3.6 pytorch 0.4.0
pytorch-0.4.0-cpu-py3 CPU Python 3.6 pytorch 0.4.0
pytorch-0.3.1 CPU Python 2.7 pytorch 0.3.1
pytorch-0.3.1-gpu GPU Python 2.7 pytorch 0.3.1
pytorch-0.3.1-gpu-py3 GPU Python 3.6 pytorch 0.3.1
pytorch-0.3.1-cpu-py3 CPU Python 3.6 pytorch 0.3.1