Neural Style Transfer

Estimated time: 20 minutes

In this part of our introduction you will learn how to:

  • change hyperparameters of your experiment in runtime;
  • track command line arguments;
  • download your experiments’ output;
  • turn your selfie into a painting.

Neural style transfer is an algorithm that passes a style of an image on to other images.

transferred style

Begin by entering the example’s directory:

cd ../3-neural-style-transfer

Data Upload

This example needs a pre-trained model and some images. They are located are in neptune-examples/data directory. Upload them to your Neptune cloud storage using the following commands (assuming you’re in the experiment’s directory):

neptune data upload -r ../data/images
neptune data upload ../data/vgg16_for_neural-style.ckpt model.ckpt

The first command uploads whole images directory to the cloud storage. The second command uploads the pre-trained neural network and names it model.ckpt.

Running the Experiment

Our implementation in the main.py file uses Keras library and requires a GPU to run.

Just like in the previous example, specify a type of machine to run the experiment on and the compute environment for your code:

neptune send \
  --environment keras-2.2-gpu \
  --worker s-k80 \
  --input images \
  --input model.ckpt

Open the Channels tab. You will see the image turning from a photo to an artwork, step by step.

channels screenshot

Tweaking Experiments in Runtime

Using Neptune’s action mechanism you can tweak your experiments while they’re running. It can be handy when you want to manually change some hyperparameters or save your model in a particular moment.

In the main.py, we declared an action: a python function that changes the content_style_balance hyperparameter.

content_style_balance = params.content_style_balance
def _change_content_style_balance_handler(csb):
    global content_style_balance
    content_style_balance = csb
    return True
ctx.register_action(
    name='Change content/style balance',
    handler=_change_content_style_balance_handler)

Now open the Actions tab and invoke it! Just type a new value for Change content/style balance, say 10 (or, if you want to go the other way - 0.1), and click Run.

actions screenshot

Observe what happens both with the image and with charts plotting cost functions.

Tracking command line arguments

Neptune CLI works well with command line arguments in your own code. In our example, you can use --content and --style arguments to change paths to the processed images.

Go ahead and check how van Gogh would have painted Mona Lisa, just type:

neptune send \
    --environment keras-2.2-gpu \
    --worker s-k80 \
    --input images \
    --input model.ckpt \
    main.py --content %mona_lisa.jpg --style %starry_night.jpg

The % characters are not passed to your script. It is a way to tell Neptune that a particular command line argument should be tracked (as Neptune’s parameter). Now, you can view your experiment’s parameters in the Properties tab or as columns in Experiment List view of your project.

You can run several experiments simultaneously. Each time you send a new experiment to Neptune, it begins, even if the others are not yet completed. Open the Experiment List view to see the list of waiting, running and completed experiments.

experiments screenshot

Download your experiments’ outputs

If you want to download your output images, you can do it from the browser. Go to the experiment view, click Browse Files on the left panel, open the output directory and download ‘final.jpg’.

download experiment output

Turn your selfie into a painting ;)

Maybe you are tempted to turn your selfie into a painting?

Firstly upload it to Neptune storage:

neptune data upload my_selfie.jpg images

Then start a new experiment:

neptune send \
    --environment keras-2.2-gpu \
    --worker s-k80 \
    --input images \
    --input model.ckpt \
    main.py --content %my_selfie.jpg --style %picasso.jpg

Voilà! Piotr's selfie

Next steps

Now you’re ready to use Neptune with your own experiments. If you want to use more advanced features of Neptune, take a look at the Advanced Topics section.