Skip to content

Step 2 - Training a TensorFlow model

Alright, so as you know, the heavy lifting is over, and building models is just a walk in the park once the data is tip top. So, straight from the Datasets view we can select our dataset and click “Create Model” which will look something like this:

CreateModel

If you are a Data Scientist, and have read this far, you probably agree that using a DNN to fit a model around this dataset is a tad too much. But who has time for that kind of philosophical thinking! I will use a 120–120–4 DNN with CategoricalCrossEntropy in the Keras API. All this I will run on a GPU instance with 30 epochs. And this is all the code we need:

import tensorflow as tf
from mlops.training.tensorflow import TensorFlowTrainer


def my_network(mlops, labels):
    inputs = tf.keras.Input(shape=(mlops.train_data.shape[1],))
    hidden_1 = tf.keras.layers.Dense(120, activation="relu")(inputs)
    hidden_2 = tf.keras.layers.Dense(120, activation="relu")(hidden_1)
    outputs = tf.keras.layers.Dense(labels)(hidden_2)
    optimizer = tf.keras.optimizers.Nadam(
        mlops.hyperparameters["learning_rate"])
    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    model.compile(
        optimizer=optimizer,
        loss=mlops.loss,
        metrics=mlops.metrics,
    )
    return model


if __name__ == "__main__":
    mlops = TensorFlowTrainer()
    mlops.read_data(number_of_label_cols=4, label_pos='end')
    model = my_network(mlops, 4)
    model.fit(
        mlops.train_data,
        mlops.train_labels.astype('float64'),
        batch_size=mlops.hyperparameters["batch_size"],
        epochs=mlops.hyperparameters["epochs"],
        validation_data=(mlops.val_data, mlops.val_labels.astype('float64')),
        callbacks=[mlops.callback],
        shuffle=True
    )
    mlops.predict(model)
    mlops.save_model(model)

As you can see, most important things are baked into the MLOps SDK after submitting through the console. You access all the hyperparameters through mlops.hyperparameters. The best thing is, if I wanted to run hyperparameter optimization on this dataset, the total lines of code I would have to change would be zero. This allows you to experiment at a fast pace, doing many iterations on smaller datasets, and then quickly scale it up once you are ready to build your production model. As well, using the mlops.callback class, we automatically feed all metrics back to the console so that you and your colleagues can collaborate and iterate together.

Result

Ooooh! Fantastic results. A stunning accuracy of 25 % on a 4 class problem. But what could have gone wrong? Must have been those lazy data engineers. Oh wait…