How to run grid search?

Problem

I would like to run grid search over my hyperparameters. How do I do that?

Solution

In order to run grid search over hyperparameter space you need to:

Step 1

Define your hyperparameter space

Go to the neptune.yaml and define parameter grid, over which you would like to run your search

1
2
3
parameters:
   filter_nr: [32, 128, 512]
   dropout: [0.0, 0.2, 0.5]

Step 2

Define a metric which you would like to optimize

Go to the neptune.yaml and define your metric channel and the goal of the experiment (maximize/minimize)

1
2
3
metric:
  channel: 'auc'
  goal: maximize

Step 3

Initialize Neptune Context in your script and load parameters

In the main.py

1
2
3
4
5
6
7
8
import neptune

ctx = neptune.Context()
filter_nr = ctx.params.filter_nr
dropout = ctx.params.dropout
...
auc_score = roc_auc_score(y_true, y_pred)
ctx.channel_send('auc', auc_score)

Step 4

Run your script

1
neptune run --config neptune.yaml main.py

Important Note

If you are running a grid search from a directory where you would like to continue developing, remember to specify the --snapshot option: neptune run --snapshot main.py

Step 5

Explore Grid of experiments

Now your experiment will be visible as ...-GRP-... experiment

image

When you click on the ...-GRP-... you can explore hyperparameter space, and sort by metric.

image

You can also see what where the hyperparameters chosen to be grid searched over for this experiment by going to the summary section.

image

See also