Setup of an Azure workspace¶

Impressions

1. Introduction ¶

This notebook is the first of a series (starting with "2x_") that leverage the Azure Machine Learning Service. Azure ML, as we also call it, is a service that allows us to train, deploy, automate, and manage machine learning models, at scale, in the cloud.

In this tutorial, we will set up an Azure ML workspace. Such resource organizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows. In particular, an Azure ML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, deployment, inferencing, and the monitoring of deployed models.

2. Pre-requisites¶

For this and all the other "2x_" notebooks to run properly on our machine, we need access to the Azure platform.

Unless we already have one, we should first:

3. Azure workspace ¶

In the deployment tutorials present in this repository (numbered "21_" to "23_"), we use the Azure ML SDK. It allows us to access our Azure resources programmatically. As we are running our notebooks in the "cvbp" conda environment, the SDK should already be installed on our machine. Let's check which version of the Azure SDK we are working with.

In [11]:

# For automatic reloading of modified libraries
%reload_ext autoreload
%autoreload 2

import sys
sys.path.extend(["..", "../.."]) # to access the utils_cv library
# Azure
import azureml.core
from azureml.core import Workspace

# Check core SDK version number
print(f"Azure ML SDK Version: {azureml.core.VERSION}")

Azure ML SDK Version: 1.0.48

To create or access an Azure ML Workspace, you will need the following information:

subscription ID: the ID of the Azure subscription we are using
resource group: the name of the resource group in which our workspace resides
workspace region: the geographical area in which our workspace resides (e.g. "eastus2" -- other examples are available here -- note the lack of spaces)
workspace name: the name of the workspace we want to create or retrieve.

Note: As with other Azure services, there are limits on certain resources like cluster size associated with the Azure Machine Learning service. Please read this article on the default limits and how to request more quota.

If you have a workspace created already, you need to get your subscription and workspace information. You can find the values for those by visiting your workspace in the Azure portal. If you don't have a workspace, the create workspace command in the next section will create a resource group and a workspace using the names you provide.

Replace the values in the following cell with your information.

In [12]:

subscription_id = "YOUR_SUBSCRIPTION_ID"
resource_group = "YOUR_RESOURCE_GROUP_NAME"  
workspace_name = "YOUR_WORKSPACE_NAME"  
workspace_region = "YOUR_WORKSPACE_REGION" #Possible values eastus, eastus2 and so on.

Get or Create the workspace This cell will create an AML workspace for you in a subscription, if one does not exist already, provided you have the correct permissions.

This will fail when:

You do not have permission to create a workspace in the resource group You do not have permission to create a resource group if it's non-existing. You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources. If this cell succeeds, you're done configuring AML!

In [13]:

# A util method that creates a workspace or retrieves one if it exists, also takes care of Azure Authentication
from utils_cv.common.azureml import get_or_create_workspace

ws = get_or_create_workspace(
        subscription_id,
        resource_group,
        workspace_name,
        workspace_region)

WARNING - Warning: Falling back to use azure cli login credentials.
If you run your code in unattended mode, i.e., where you can't give a user input, then we recommend to use ServicePrincipalAuthentication or MsiAuthentication.
Please refer to aka.ms/aml-notebook-auth for different authentication mechanisms in azureml-sdk.
ERROR - get_workspace error using subscription_id=2ad17db4-e26d-4c9e-999e-adae9182530c, resource_group_name=amlnotebookrg, workspace_name=amlnotebookws

Creating new workspace

UserWarning: The resource group doesn't exist or was not provided. AzureML SDK is creating a resource group=amlnotebookrg in location=eastus using subscription=2ad17db4-e26d-4c9e-999e-adae9182530c.

Deploying AppInsights with name amlnotebinsights0ca90703.
Deployed AppInsights with name amlnotebinsights0ca90703. Took  seconds.
Deploying KeyVault with name amlnotebkeyvaultb03f393a.
Deploying StorageAccount with name amlnotebstorage914c3d2fd.
Deployed StorageAccount with name amlnotebstorage914c3d2fd. Took  seconds.
Deployed KeyVault with name amlnotebkeyvaultb03f393a. Took  seconds.
Deploying Workspace with name amlnotebookws.
Deployed Workspace with name amlnotebookws. Took  seconds.

Let's check that the workspace is properly loaded

In [14]:

# Print the workspace attributes
print(f'Workspace name: {ws.name}\n \
      Workspace region: {ws.location}\n \
      Subscription id: {ws.subscription_id}\n \
      Resource group: {ws.resource_group}')

Workspace name: amlnotebookws
       Workspace region: eastus
       Resource group: amlnotebookrg

We can also see this workspace on the Azure portal by sequentially clicking on:

Resource groups, and clicking the one we referenced above

Workspace_name

For more details on the setup of a workspace and other Azure resources, we can refer to this configuration notebook.

Creating a workspace automatically adds associated resources:

A container registry, which can host Docker images, and gets lazily created
A storage account, in which output files get stored
Application Insights, which allows us to monitor the health of and traffic to a web service, for instance
A key vault, which stores our credentials.

Such resources, when first created, cost less than a penny per day. To get a better sense of pricing, we can refer to this calculator. We can also navigate to the Cost Management + Billing pane on the portal, click on our subscription ID, and click on the Cost Analysis tab to check our credit usage.

We will continue using our workspace in the next notebooks, so will keep it available. However, if we needed to delete it, we would run the command below.

Let's write the workspace configuration for the rest of the notebooks to connect to the workspace.

In [15]:

ws.write_config()

In [16]:

# ws.delete(delete_dependent_resources=True)
# This deletes our workspace, the container registry, the account storage, Application Insights and the key vault

4. Next steps¶

In this notebook, we created a new workspace, and stored configuration information in a ./aml_config/config.json file. Using this file we can load as workspace using from_config() method ws = Workspace.from_config().

In the next notebook, we will learn how to deploy a trained model as a web service on Azure.