Step 2 - Training a TensorFlow model¶
Alright, so as you know, the heavy lifting is over, and building models is just a walk in the park once the data is tip top. So, straight from the Datasets view we can select our dataset and click “Create Model” which will look something like this:
If you are a Data Scientist, and have read this far, you probably agree that using a DNN to fit a model around this dataset is a tad too much. But who has time for that kind of philosophical thinking! I will use a 120–120–4 DNN with
CategoricalCrossEntropy in the Keras API. All this I will run on a GPU instance with 30 epochs. And this is all the code we need:
import tensorflow as tf from mlops.training.tensorflow import TensorFlowTrainer def my_network(mlops, labels): inputs = tf.keras.Input(shape=(mlops.train_data.shape,)) hidden_1 = tf.keras.layers.Dense(120, activation="relu")(inputs) hidden_2 = tf.keras.layers.Dense(120, activation="relu")(hidden_1) outputs = tf.keras.layers.Dense(labels)(hidden_2) optimizer = tf.keras.optimizers.Nadam( mlops.hyperparameters["learning_rate"]) model = tf.keras.Model(inputs=inputs, outputs=outputs) model.compile( optimizer=optimizer, loss=mlops.loss, metrics=mlops.metrics, ) return model if __name__ == "__main__": mlops = TensorFlowTrainer() mlops.read_data(number_of_label_cols=4, label_pos='end') model = my_network(mlops, 4) model.fit( mlops.train_data, mlops.train_labels.astype('float64'), batch_size=mlops.hyperparameters["batch_size"], epochs=mlops.hyperparameters["epochs"], validation_data=(mlops.val_data, mlops.val_labels.astype('float64')), callbacks=[mlops.callback], shuffle=True ) mlops.predict(model) mlops.save_model(model)
As you can see, most important things are baked into the MLOps SDK after submitting through the console. You access all the hyperparameters through
mlops.hyperparameters. The best thing is, if I wanted to run hyperparameter optimization on this dataset, the total lines of code I would have to change would be zero. This allows you to experiment at a fast pace, doing many iterations on smaller datasets, and then quickly scale it up once you are ready to build your production model. As well, using the mlops.callback class, we automatically feed all metrics back to the console so that you and your colleagues can collaborate and iterate together.
Ooooh! Fantastic results. A stunning accuracy of 25 % on a 4 class problem. But what could have gone wrong? Must have been those lazy data engineers. Oh wait…