How to log keras metrics?

Problem

I have a training script written in keras. How do I adjust it to log metrics to Neptune?

Solution

Say your training script looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
import keras
from keras import backend as K 

mnist = keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = keras.models.Sequential([
  keras.layers.Flatten(),
  keras.layers.Dense(512, activation=K.relu),
  keras.layers.Dropout(0.2),
  keras.layers.Dense(10, activation=K.softmax)
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

Now there are two options:

Use automatic integration

All you need to do to adjust it is add the following snippet at the top:

1
2
3
4
import neptune

ctx = neptune.Context()
ctx.integrate_with_keras()

Write custom Callback

For more advanced metrics or logging images, you may want to create your own custom callback:

Step 1

Extend Keras.callback class

1
2
3
4
5
6
from keras.callbacks import Callback

class NeptuneMonitor(Callback):
    def on_epoch_end(self, epoch, logs={}):
        innovative_metric = logs['acc'] - 2 * logs['loss']
        ctx.channel_send('innovative_metric', epoch, innovative_metric)

Step 2

Instantiate it and add it to your callbacks list:

1
2
neptune_monitor = NeptuneMonitor()
model.fit(x_train, y_train, epochs=5, callbakcs=[neptune_monitor])

All your metrics are now logged to Neptune:

image

See also