23 Azure Machine Learning Interview Questions (ANSWERED)

To create a machine learning pipeline in Azure, you have several options:

• Azure Machine Learning SDK: provides Python libraries that allow you to create, manage, and run machine learning pipelines. You can use the SDK to define the steps of your pipeline, specify dependencies between steps, and execute the pipeline. The SDK supports both code-based and visual pipeline authoring.

• Azure Machine Learning Designer: is a visual interface in Azure Machine Learning Studio that allows you to create and deploy machine learning models without writing code. You can use the Designer to drag and drop modules, connect them to create a pipeline and configure the parameters of each module. This option is suitable for users who prefer a visual approach to pipeline creation.

• Azure Data Factory: is a cloud-based data integration service that can also be used to create machine learning pipelines. It provides a visual interface for building data pipelines that can include machine learning tasks. You can use Data Factory to orchestrate the execution of your machine learning pipeline and integrate it with other data processing tasks.

There are 3 different ways to create pipelines in Azure Machine Learning: using the Python SDK, the Command Line Interface (CLI), or with Azure Machine Learning studio. In each one, the high-level overview steps to create the pipeline are similar:

1. Define your pipeline steps in components: a component is a self-contained piece of code that does one step in a machine learning pipeline, such as data preprocessing, feature engineering, model training, or model evaluation. For each component, you need to prepare the following:

   • Prepare the execution logic.
   • Define the interface of the component (the input and output of each component),
   • Add other metadata of the component, including run-time environment, the command to run the component, etc.

2. Register component for reuse and sharing: While some components are specific to a particular pipeline, the real benefit of components comes from reuse and sharing. Register a component in your Machine Learning workspace to make it available for reuse. Registered components support automatic versioning so you can update the component but assure that pipelines that require an older version will continue to work.

3. Connect to the workspace: finally you can connect the whole pipeline to the environment, it could be an Azure Machine Learning environment(curated or custom registered), docker image or conda environment to then submit it to the workspace.

The Scoring Script is a Python file (.py) that contains the logic about how to run the model and read the input data submitted by the batch deployment executor. Each model deployment provides the scoring script (allow with any other dependency required) at creation time.

The scoring script must contain two methods:

• The init method: which is used for any costly or common preparation. For example, can be used to load the model into memory. This function is called once at the beginning of the entire batch job.

• The run method: Such method is called once per each mini_batch generated for your input data and it should return a Pandas DataFrame or an array/list. Each returned output element indicates one successful run of an input element in the input mini_batch.

Azure Machine Learning environments are an encapsulation of the environment where your machine learning training happens. They specify the Python packages, environment variables, and software settings around your training and scoring scripts. They also specify runtimes (Python, Spark, or Docker).

Environments can broadly be divided into three categories:

• Curated environments: are provided by Azure Machine Learning and are available in the workspace by default. Intended to be used as is, they contain collections of Python packages and settings to get started with various machine learning frameworks. These pre-created environments also allow for faster deployment time.

• User-managed environments: here you're responsible for setting up your environment and installing every package that your training script needs on the compute target.

• System-managed environments: are used when you want conda to manage the Python environment for you.

Data Asset and Datastore are the two key components in Azure ML workspace, to connect the actual sources of data with the machine learning authoring tools.

• Data asset is a reference to the collection of related data sources, used in various authoring mechanisms like Automated ML, Notebooks, Designer, and Experiments. The data referenced in any Data asset can come from a wide variety of Data Sources like local or web files, Azure blob storage and datastores, Azure Open Datasets, Azure Data Lake, and an array of other data sources.

• Datastore facilitates the connection between Data assets and the various sources of data, by securely storing the connection and authentication information. With Datastores in the picture, machine learning professionals no longer need to provide credentials and data source connection details in their scripts, pipelines, or any other authoring tools.

The figure below shows how Data Asset encompasses a logical representation of the actual sources of data, while the Datastore safeguards the connection details to the actual sources of data by keeping the credentials in a separate secure location (represented by the lock icon).

In general, it's a good engineering practice to build a machine learning pipeline to split a complete machine learning task into a multi-step workflow. Such that, everyone can work on the specific step independently. In Azure Machine Learning, a component represents one reusable step in a pipeline and is designed to help improve the productivity of pipeline building. Specifically, the use of components offers the following benefits:

• Well-defined interface: Components require a well-defined interface (input and output). The interface allows the user to build steps and connect steps easily. The interface also hides the complex logic of a step and removes the burden of understanding how the step is implemented.

• Share and reuse: As the building blocks of a pipeline, components can be easily shared and reused across pipelines, workspaces, and subscriptions. Components built by one team can be discovered and used by another team.

• Version control: Components are versioned. The component producers can keep improving components and publish new versions. Consumers can use specific component versions in their pipelines. This gives them compatibility and reproducibility.

In general, pipeline components are similar to pipeline jobs because they both contain a group of jobs/components.

Here are some main differences you need to be aware of:

• A pipeline job is an independently executable workflow of a complete machine learning task, such as data preparation, model training, and model deployment. A pipeline job is submitted to Azure Machine Learning for execution and can be scheduled or triggered manually. It provides a way to automate and manage the entire machine learning workflow.

• Now, when developing a complex machine learning pipeline, it's common to have sub-pipelines that use multi-step to perform tasks such as data preprocessing and model training. These sub-pipelines can be developed and tested standalone. Pipeline component groups multi-step as a component that can be used as a single step to create complex pipelines.

In summary, a pipeline job represents the entire workflow of a machine learning pipeline, while a pipeline component represents an individual step or task within that pipeline.

A Batch Deployment is a set of compute resources hosting the model that does the actual batch scoring. To create a batch model deployment, you'll need to do the following items:

1. Create a batch endpoint: Such batch endpoint is an HTTPS endpoint that clients can call to trigger inference over large volumes of data.

2. Register the model in the workspace: This will allow us to track the model version.

3. Create the scoring script: Batch deployments require a scoring script that indicates how a given model should be executed and how input data must be processed. Batch Endpoints support scripts created in Python.

4. Create an environment where your batch deployment will run: Such an environment needs to include all the packages plus any dependency your code requires for running.

5. Create a deployment definition: this allows us to configure the key aspects such as the endpoint_name, the model, environment, compute target, and more.

6. Submit the deployment: this can be done using Azure CLI, Python or Azure Machine Learning Studio.

To define a job in Azure Machine Learning, you can create a YAML file. Whether you want to run one script as a command job or multiple scripts sequentially as a pipeline. For both command and pipeline jobs, you'll need to create a YAML file, which details:

• Which scripts to run.
• What the inputs and outputs are for each script.
• The compute that will be used to run the scripts.
• The environment that needs to be installed on the compute to run the scripts.

The variation of the training data and actual data is what is referred to as data drift and should be tracked and monitored regularly. Data drift leads to model performance degradation over time, and so needs to be monitored. The best case is to set up monitoring and alerts to understand when your deployed model differs too much from the training data and so needs to be retrained.

Computing the data drift in Azure Machine Learning requires the following steps:

1. Define the target and baseline datasets: the target dataset is usually the training dataset, and the baseline dataset is usually constructed from the inputs of the scoring service. These datasets must contain a column that represents the date and time of each observation.

2. Create and set up dataset monitor: It can be created by using either the Python SDK or Azure Machine Learning studio. The dataset monitor runs at a set frequency (daily, weekly, monthly) intervals and it analyzes new data available in the target dataset since its last run.

   Here you can configure the monitor according to requirements specifying drift_threshold or even a feature_list to be considered. For example, suppose you want to monitor for three specific features ['a', 'b', 'c'], to measure drift on a monthly cadence with a delay of 24 hours, and you want to an alert is when the target dataset drifts more than 25% from the baseline data. In Python SDK the previous is set with:

   from azureml.datadrift import DataDriftDetector
   
   ws = Workspace.from_config()
   alert_config = AlertConfiguration(email_addresses=['<insert email address>'])
   
   monitor = DataDriftDetector.create_from_datasets(ws,
       "data-drift-monitor",
       ds_baseline,
       ds_target,
       compute_target=compute_target,
       frequency='Month',
       feature_list=['a', 'b', 'c'],
       alert_config=alert_config,
       drift_threshold=0.25,
       latency=24)

Depending on the resource that you select, the metrics that you see will be different. Metrics are scoped differently for online endpoints and online deployments.

The metrics at endpoint scope are

• Request Latency.
• Request Latency P50, P90 and P95 (Request latency at the 50th, 90th, and 95th percentile, respectively).
• Requests per minute.
• New connections per second.
• Active connection count.
• Network bytes.

Split on the following dimensions:

• Deployment.
• Status Code and Status Code Class.

For example, you can split along the deployment dimension to compare the request latency of different deployments under an endpoint.

The metrics at deployment scope are:

• CPU Utilization Percentage.
• Deployment Capacity (the number of instances of the requested instance type)
• Disk Utilization, and Memory Utilization Percentage.
• GPU Memory Utilization and GPU Utilization (only applicable to GPU instances).

You can split metrics on the Instance Id dimension. For instance, you can compare CPU and/or memory utilization between different instances for an online deployment.

In Azure Machine Learning, there are several training methods available for training machine learning models. Some of these training methods include:

• The SDK for Python: The Python SDK provides several ways to train models, each with different capabilities:

  • command() in which you submit a command() that includes a training script, environment, and compute information.
  • Automated machine learning allows you to train models without extensive data science or programming knowledge.
  • Machine learning pipelines which can be used by the previously mentioned training methods. Pipelines are more about creating a workflow, so they encompass more than just the training of models.

• Designer: It allows you to train models using a drag and drop web-based UI. You can use Python code as part of the design, or train models without writing any code.

• Azure CLI: often used for scripting and automating tasks. For example, once you've created a training script or pipeline, you might use the Azure CLI to start a training job on a schedule or when the data files used for training are updated.

Independent of the use case, there are similar preparation steps required for putting an ML model into production:

• First, the trained model needs to be registered in the model registry. This will allow us to track the model version and binaries and fetch a specific version of the model in deployment.

• Second, we need to specify the deployment assets (for example, the environment, libraries, assets, and scoring file). These assets define exactly how the model is loaded and initialized, how user input is parsed, how the model is executed, and how the output is passed back to the user.

• Finally, we need to choose a compute target to run the model, for example, Azure Kubernetes Service (AKS) or Azure Container Instances (ACI), or one of the many other Azure compute services.

There are 3 available options for building a data ingestion pipeline with Azure Data Factory. Such Azure Data Factory pipelines is used to ingest data for use with Azure Machine Learning. Data Factory allows you to easily extract, transform, and load (ETL) data. Once the data has been transformed and loaded into storage, it can be used to train your machine learning models in Azure Machine Learning.

• Data Factory + Azure Functions: allows you to run small pieces of code (functions) without worrying about application infrastructure. In this option, the data is processed with custom Python code wrapped into an Azure Function.

• Data Factory + custom component: In this option, the data is processed with custom Python code wrapped into an executable. This approach is a better fit for large data or heavy algorithms that process significant amounts of data than the previous technique.

• Data Factory + Azure Databricks notebook: In this technique, the data transformation is performed by a Python notebook, running on an Azure Databricks cluster. This is probably, the most common approach that uses the full power of an Azure Databricks service. It's designed for distributed data processing at scale.

Azure Machine Learning supports multiple types of computing for experimentation, training, and deployment. You can select the most appropriate type of computing target for your particular needs.

• Compute instance: Behaves similarly to a virtual machine and is primarily used to run notebooks. It's ideal for experimentation.

• Compute clusters: Multi-node clusters of virtual machines that automatically scale up or down to meet demand. A cost-effective way to run scripts that need to process large volumes of data. Clusters also allow you to use parallel processing to distribute the workload and reduce the time it takes to run a script.

• Kubernetes clusters: Cluster based on Kubernetes technology, giving you more control over how the compute is configured and managed. You can attach your self-managed Azure Kubernetes (AKS) cluster for cloud computing, or an Arc Kubernetes cluster for on-premises workloads.

• Attached compute: If you already use an Azure-based compute environment for data science, such as an Azure virtual machine or an Azure Databricks cluster, you can attach it to your Azure Machine Learning workspace and use it as a compute target for certain types of workload.

1. Data Distribution:

   • Azure ML Notebooks are good when you are training with limited data on a single machine. Though Azure ML provides training clusters, the data distribution among the nodes is to be handled in the code.
   • Azure Databricks with its Resilient Distributed Datasets (RDDs) are designed to handle data distributed on multiple nodes.This is advantageous when your data size is huge.
   • When your data size is small and can fit in a scaled-up single machine/ you are using pandas dataframe, then the use of Azure databricks is an overkill.

2. Data Cleaning:

   • Databricks can support a lot of file formats natively and querying and cleaning huge datasets are easy.
   • In AzureML notebooks the previous has has to be handled custom. It can be done with an aml notebooks but cleaning and writing to stores has to be handled.

3. Distributing the training:

   • Databricks provides inbuilt ML algorithms that can act on chunks of data on that node and coordinate with other nodes.
   • In AzureML notebooks this can be done tensorflow, horovod, etc.

• An endpoint is a stable and durable URL that can be used to request or invoke a model. You provide the required inputs to the endpoint and get the outputs back. An endpoint provides: a stable and durable URL (like endpoint-name.region.inference.ml.azure.com), an authentication mechanism, and an authorization mechanism.
• A deployment is a set of resources and computes required for hosting the model or component that does the actual inferencing.

• A single endpoint can contain multiple deployments. These deployments can host independent assets and consume different resources based on the needs of the assets.
• To function properly, each endpoint must have at least one deployment. Endpoints and deployments are independent Azure Resource Manager resources that appear in the Azure portal.
• To illustrate the difference, the following diagram shows an endpoint that has two deployments, blue and green. The blue deployment uses VMs with a CPU SKU, and runs version 1 of a model. The green deployment uses VMs with a GPU SKU, and runs version 2 of the model. The endpoint is configured to route 90% of incoming traffic to the blue deployment, while the green deployment receives the remaining 10%.

Azure Machine Learning allows you to implement online endpoints and batch endpoints.

• Online endpoints are designed for real-time inference—when you invoke the endpoint, the results are returned in the endpoint's response.
• Batch endpoints, on the other hand, are designed for long-running batch inference. Each time you invoke a batch endpoint you generate a batch job that performs the actual work.

Is recommended to use online endpoints when:

• You have low-latency requirements.
• Your model can answer the request in a relatively short amount of time.
• Your model's inputs fit on the HTTP payload of the request.
• You need to scale up in terms of number of requests.

Is recommended to use batch endpoint when:

• You have expensive models or pipelines that require a longer time to run.
• You want to operationalize machine learning pipelines and reuse components.
• You need to perform inference over large amounts of data that are distributed in multiple files.
• You don't have low latency requirements.
• Your model's inputs are stored in a storage account or an Azure Machine Learning data asset.
• You can take advantage of parallelization.