How to log lightGBM metrics?

Problem

I have a training script written in lightGBM. How do I adjust it to log metrics to Neptune?

Solution

Say your training script looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import lightgbm as lgb
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_wine

data = load_wine()

X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.1)
lgb_train = lgb.Dataset(X_train, y_train)
lgb_eval = lgb.Dataset(X_test, y_test, reference=lgb_train)

params = {
    'boosting_type': 'gbdt',
    'objective': 'multiclass',
    'num_class': 3, 
    'num_leaves': 31,
    'learning_rate': 0.05,
    'feature_fraction': 0.9,
}

gbm = lgb.train(params,
                lgb_train,
                num_boost_round=500,
                valid_sets=[lgb_train, lgb_eval],
                valid_names=['train','valid'],
                )

Now, you need to use lightGBM callbacks to pass log metrics to Neptune:

Step 1

Implement your custom callback:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import neptune

neptune.init('your_API_token', 'username/my_project')
neptune.create_experiment()

def neptune_monitor():
    def callback(env):
        for name, loss_name, loss_value, _ in env.evaluation_result_list:
            neptune.send_metric('{}_{}'.format(name, loss_name), x=env.iteration, y=loss_value)
    return callback

Step 2

Pass it to lgb.train object via callbacks parameter:

1
2
3
4
5
6
7
gbm = lgb.train(params,
                lgb_train,
                num_boost_round=500,
                valid_sets=[lgb_train, lgb_eval],
                valid_names=['train','valid'],
                callbacks=[neptune_monitor()],
                )

All your metrics are now logged to Neptune:

image

See also