MLflow and Azure Machine Learning
APPLIES TO:
Azure CLI ml extension v2 (current)
Python SDK azure-ai-ml v2 (current)
MLflow is an open-source framework that's designed to manage the complete machine learning lifecycle. Its ability to train and serve models on different platforms allows you to use a consistent set of tools regardless of where your experiments are running: locally on your computer, on a remote compute target, on a virtual machine, or on an Azure Machine Learning compute instance.
Tip
Azure Machine Learning workspaces are MLflow-compatible, which means you can use Azure Machine Learning workspaces in the same way that you use an MLflow tracking server. Such compatibility has the following advantages:
- You can use Azure Machine Learning workspaces as your tracking server for any experiment you're running with MLflow, whether it runs on Azure Machine Learning or not. You only need to configure MLflow to point to the workspace where the tracking should happen.
- You can run any training routine that uses MLflow in Azure Machine Learning without changes. MLflow also supports model management and model deployment capabilities.
Tracking with MLflow
Azure Machine Learning uses MLflow Tracking for metric logging and artifact storage for your experiments, whether you created the experiments via the Azure Machine Learning Python SDK, the Azure Machine Learning CLI, or Azure Machine Learning studio. We recommend using MLflow for tracking experiments. To get started, see Log metrics, parameters, and files with MLflow.
Note
Unlike the Azure Machine Learning SDK v1, there's no logging functionality in the SDK v2. We recommend that you use MLflow for logging.
With MLflow Tracking, you can connect Azure Machine Learning as the back end of your MLflow experiments. The workspace provides a centralized, secure, and scalable location to store training metrics and models.
Capabilities include:
- Track machine learning experiments and models running locally or in the cloud with MLflow in Azure Machine Learning.
- Track Azure Databricks machine learning experiments with MLflow in Azure Machine Learning.
- Track Azure Synapse Analytics machine learning experiments with MLflow in Azure Machine Learning.
You can also use MLflow to Query & compare experiments and runs with MLflow.
Important
- MLflow in R support is limited to tracking experiment's metrics, parameters and models on Azure Machine Learning jobs. Interactive training on RStudio, Posit (formerly RStudio Workbench) or Jupyter Notebooks with R kernels is not supported. Model management and registration is not supported using the MLflow R SDK. As an alternative, use Azure ML CLI or Azure ML studio for model registration and management. View the following R example about using the MLflow tracking client with Azure Machine Learning.
- MLflow in Java support is limited to tracking experiment's metrics and parameters on Azure Machine Learning jobs. Artifacts and models can't be tracked using the MLflow Java SDK. As an alternative, use the
Outputsfolder in jobs along with the methodmlflow.save_modelto save models (or artifacts) you want to capture. View the following Java example about using the MLflow tracking client with the Azure Machine Learning.
Example notebooks
- Training and tracking an XGBoost classifier with MLflow: Demonstrates how to track experiments by using MLflow, log models, and combine multiple flavors into pipelines.
- Training and tracking an XGBoost classifier with MLflow using service principal authentication: Demonstrates how to track experiments by using MLflow from compute that's running outside Azure Machine Learning. It shows how to authenticate against Azure Machine Learning services by using a service principal.
- Hyper-parameter optimization using Hyperopt and nested runs in MLflow: Demonstrates how to use child runs in MLflow to do hyper-parameter optimization for models by using the popular library Hyperopt. It shows how to transfer metrics, parameters, and artifacts from child runs to parent runs.
- Logging models with MLflow: Demonstrates how to use the concept of models instead of artifacts with MLflow, including how to construct custom models.
- Manage runs and experiments with MLflow: Demonstrates how to query experiments, runs, metrics, parameters, and artifacts from Azure Machine Learning by using MLflow.
Model registries with MLflow
Azure Machine Learning supports MLflow for model management. This support represents a convenient way to support the entire model lifecycle for users who are familiar with the MLflow client.
To learn more about how to manage models by using the MLflow API in Azure Machine Learning, view Manage model registries in Azure Machine Learning with MLflow.
Example notebooks
- Manage model registries with MLflow: Demonstrates how to manage models in registries by using MLflow.
Model deployments of MLflow models
You can deploy MLflow models to Azure Machine Learning so that you can apply the model management capabilities and no-code deployment offering in Azure Machine Learning. Azure Machine Learning supports deploying models to both real-time and batch endpoints. You can use the azureml-mlflow MLflow plug-in, the Azure Machine Learning CLI v2, and the user interface in Azure Machine Learning studio.
Learn more at Deploy MLflow models to Azure Machine Learning.
Example notebooks
- Deploy MLflow to Online Endpoints: Demonstrates how to deploy models in MLflow format to online endpoints using MLflow SDK.
- Deploy MLflow to Online Endpoints with safe rollout: Demonstrates how to deploy models in MLflow format to online endpoints using MLflow SDK with progressive rollout of models and the deployment of multiple model's versions in the same endpoint.
- Deploy MLflow to web services (V1): Demonstrates how to deploy models in MLflow format to web services (ACI/AKS v1) using MLflow SDK.
- Deploying models trained in Azure Databricks to Azure Machine Learning with MLflow: Demonstrates how to train models in Azure Databricks and deploy them in Azure ML. It also includes how to handle cases where you also want to track the experiments with the MLflow instance in Azure Databricks.
Training MLflow projects (preview)
You can submit training jobs to Azure Machine Learning by using MLflow projects (preview). You can submit jobs locally with Azure Machine Learning tracking or migrate your jobs to the cloud via Azure Machine Learning compute.
Learn more at Train machine learning models with MLflow projects and Azure Machine Learning.
Example notebooks
MLflow SDK, Azure Machine Learning v2, and Azure Machine Learning studio capabilities
The following table shows which operations are supported by each of the tools available in the machine learning lifecycle.
| Feature | MLflow SDK | Azure Machine Learning v2 (CLI/SDK) | Azure Machine Learning studio |
|---|---|---|---|
| Track and log metrics, parameters, and models | ✓ | ||
| Retrieve metrics, parameters, and models | ✓ | 1 | ✓ |
| Submit training jobs with MLflow projects | ✓ 2 | ||
| Submit training jobs with inputs and outputs | ✓ | ✓ | |
| Submit training jobs by using machine learning pipelines | ✓ | ✓ | |
| Manage experiments and runs | ✓ | ✓ | ✓ |
| Manage MLflow models | ✓3 | ✓ | ✓ |
| Manage non-MLflow models | ✓ | ✓ | |
| Deploy MLflow models to Azure Machine Learning (Online & Batch) | ✓4 | ✓ | ✓ |
| Deploy non-MLflow models to Azure Machine Learning | ✓ | ✓ |
Note
- 1 Only artifacts and models can be downloaded.
- 2 On preview.
- 3 Some operations may not be supported. View Manage model registries in Azure Machine Learning with MLflow for details.
- 4 Deployment of MLflow models to batch inference by using the MLflow SDK is not possible at the moment. View Deploy MLflow models to Azure Machine Learning for details.
Next steps
- Track machine learning experiments and models running locally or in the cloud with MLflow in Azure Machine Learning.
Feedback
Submit and view feedback for