Why MLOps?

Most data-driven companies fail to respond to the market fast enough. Not only due to unnecessary steps or approvals or manual work but also because Data Science itself is a complex process. It involves a set of phases and each phase can contribute to the success or failure of the project significantly. 

Often, the analysis, predictions, and outcome of a Data Sc. project are outdated quite fast based on ever-changing market situations. Therefore, applying the same principles, the same machine learning model, and going through the same data science workflow manually would be a significant effort for Data scientists. To reduce the efforts and improve productivity,  the whole ML process is automated with the concept of a pipeline.

Machine Learning Operations or MLOps is generally a streamlined process of performing Machine Learning that preprocesses the data, trains a model, deploys it in production, monitors the performance of the model, and re-trains the model when required ( e.g. new datasets, new features, etc.). 

In a nutshell, MLOps bring together the concept of DevOps and apply it to Machine Learning. It helps bridge the gap between Data Scientists and IT operations.

There are primarily 4 stages:

Training Model

Understanding the business and objective of the analysis is quite essential for any ML project. Think about why you are doing the analysis, how the business or organization can benefit from that, what would define the success of the project and can you measure that. Most companies possess a set of KPIs that are primary indicators for measurement.

Therefore, requirements analysis, gathering the right data, analyzing the data quality, and pre-processing are prior steps before you build your ML model. 

Once the data is cleaned, identify the ML algorithm that suits the problem (Supervised / Unsupervised). Apply it to the data, develop the model, and fine-tune it. 

All these steps will be part of the pipeline that would enable us to perform continuous training ( at regular intervals). This is also called Auto shadow/ Auto ML where most machine learning tasks are automated, including hyperparameter tuning, feature selection, and model architecture selection.

This would reduce the chance of model decay (ML models lose relevance over time due to shifts in data distribution.)

Deployment

The trained model now is ready to be deployed in the production or test environment. Verify the trained model includes all necessary artifacts ( file, codes, configurations, etc.) and then package the model with its dependencies into a container (can be a Docker image). Containerization ensures the consistency of the model across different platforms and makes it easier to scale and deploy. 

Next is Infrastructure - Kubernetes is a popular choice in deploying machine learning models. By containerizing (e.g., Docker) the model and its dependencies, Kubernetes ensures consistency and portability across different environments. It also helps scale the number of replicas of the model based on incoming workload, improves reliability by reducing downtime, and also helps in rollbacks to previous versions ( if required).

Last but not least is maintaining a version control system (CI/ CD pipeline) to keep track of data, models, and artifacts. Git is a popular tool in this case.

Monitoring 

This tracks model performance metrics - Accuracy, Precision or mean absolute error or RMSE to assess the model's overall performance. This help identifies how well the model in performing with the external data and whether it meets the desired performance goals (mostly tracked by KPIs).

• Kubernetes cluster monitoring helps track CPU, memory utilization, node health, and network performance.

• Container monitoring identifies resource bottlenecks or potential issues by tracking how often the container restarts.

• Model application logs checking to gain insights about model errors and behaviors.

• Alert and notification services.

• Model decay and data drift.

• Dashboard and visual exploration - Grafana is a popular tool.

• Security threats and unauthorized accesses.

Analyzing Results

All of the outputs of the monitoring stage are analyzed to make a decision. 

• Checking data drift and model drift refers to the changes in data distribution and model decay. If the model decay is higher than the threshold, model re-training is required.

• Cost analysis is often done to identify the scope of cost reductions. Many times, resources are not utilized fully which increases the costs unnecessarily.

• A/B testing to compare different model versions and algorithms.

• Feedback analysis focused on the communication between data scientists, developers, and operations teams. It helps collaborate to maintain continuous improvements.