# How to Integrate with External ML Model Serving Platforms Using ML‑For‑Beginners

> Integrate with external ML model serving platforms by exporting models, using deployment wrappers, and calling managed endpoints. Learn how with ML-For-Beginners.

- Repository: [Microsoft/ML-For-Beginners](https://github.com/microsoft/ML-For-Beginners)
- Tags: how-to-guide
- Published: 2026-02-28

---

**You can integrate with external ML model serving platforms by exporting trained models to the `models/` directory, invoking the platform-specific deployment wrappers in `deployment/`, and substituting local prediction calls with authenticated HTTP requests to the managed endpoint.**

The **ML‑For‑Beginners** repository is a hands-on educational framework that modularizes the machine learning lifecycle into discrete stages. Because the curriculum isolates training logic from deployment concerns, you can integrate with external ML model serving platforms without altering the core Jupyter notebooks. The repository ships with ready-to-use deployment scripts for Azure ML, AWS SageMaker, and Google Cloud AI Platform that handle artifact registration and endpoint provisioning.

## The Modular Architecture That Enables Integration

The repository separates concerns into five distinct layers, each mapped to specific folders and file patterns:

- **Data Preparation** – Handled in `data/` folders and `<chapter>/data_preprocess.ipynb` files that load and clean raw datasets.
- **Model Training** – Isolated in `<chapter>/train_*.ipynb` notebooks (e.g., `02-Classification/train_logistic_regression.ipynb`) that build and evaluate estimators.
- **Model Export** – Serializes artifacts to the `models/` folder via scripts like [`export_model.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/export_model.py) or notebook cells that pickle scikit-learn objects or convert TensorFlow/Keras graphs to ONNX/SavedModel.
- **Inference Demo** – Local validation performed in `inference_demo.ipynb` by loading artifacts from disk.
- **Deployment Template** – Platform-specific automation housed in [`deployment/azure_ml_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/azure_ml_deploy.py), [`deployment/sagemaker_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/sagemaker_deploy.py), and [`deployment/gcp_ai_platform_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/gcp_ai_platform_deploy.py).

This separation allows you to treat the training notebooks as immutable learning resources while still productionizing their outputs.

## Three-Step Integration Workflow

To integrate with any external ML model serving platform, follow the export-deploy-consume pattern implemented in the repository helpers.

### Step 1 – Export Models to Portable Formats

Most training notebooks conclude by persisting the estimator to the `models/` folder. For broad compatibility with cloud serving platforms, convert scikit-learn models to ONNX or use native formats like pickle.

```python
import joblib
import pathlib
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType

# Standard pickle export

model_path = pathlib.Path("models") / "credit_fraud.pkl"
joblib.dump(trained_model, model_path)

# ONNX export for optimized inference

initial_type = [("float_input", FloatTensorType([None, X_train.shape[1]]))]
onnx_model = convert_sklearn(trained_model, initial_types=initial_type)
onnx_path = pathlib.Path("models") / "credit_fraud.onnx"
with open(onnx_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

```

### Step 2 – Deploy Using Platform Helpers

The `deployment/` directory contains wrapper classes that encapsulate authentication, artifact upload, and endpoint creation. Import the appropriate deployer based on your target platform.

```python
import os
from deployment.azure_ml_deploy import AzureDeployer
from deployment.sagemaker_deploy import SageMakerDeployer
from deployment.gcp_ai_platform_deploy import GCPDeployer

def deploy_to_platform(model_file: str, endpoint_name: str, platform: str) -> str:
    """
    Uploads ``model_file`` to the chosen ``platform`` and returns the inference URL.
    Supported values: "azureml", "sagemaker", "gcp".
    """
    if platform == "azureml":
        deployer = AzureDeployer()
        return deployer.deploy(model_file, endpoint_name)
    if platform == "sagemaker":
        deployer = SageMakerDeployer()
        return deployer.deploy(model_file, endpoint_name)
    if platform == "gcp":
        deployer = GCPDeployer()
        return deployer.deploy(model_file, endpoint_name)
    raise ValueError(f"Unsupported platform: {platform}")

# Example invocation

endpoint_url = deploy_to_platform(
    model_file=str(model_path),
    endpoint_name="credit-fraud-detector",
    platform="azureml"
)
print("Endpoint ready at:", endpoint_url)

```

### Step 3 – Consume the Remote Endpoint

Replace local `model.predict()` calls with HTTP POST requests that send JSON payloads to the deployed service.

```python
import requests
import json

def call_endpoint(url: str, payload: dict) -> dict:
    """Transmits a JSON payload to the model endpoint and returns the prediction."""
    headers = {"Content-Type": "application/json"}
    response = requests.post(url, data=json.dumps(payload), headers=headers)
    response.raise_for_status()
    return response.json()

# Invoke the served model

sample = {"features": [0.5, 1.2, -0.3, 0.0, 2.1]}
prediction = call_endpoint(endpoint_url, sample)
print("Remote prediction:", prediction)

```

## Platform-Specific Implementation Details

Each helper script in `deployment/` follows an identical three-phase pattern: upload the artifact, register a model resource, and expose a REST endpoint. Configuration is handled via environment variables to keep credentials out of source control.

### Azure ML Deployment via [`deployment/azure_ml_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/azure_ml_deploy.py)

The `AzureDeployer` class loads workspace credentials from environment variables (`AZURE_WORKSPACE_NAME`, `AZURE_SUBSCRIPTION_ID`, `AZURE_RESOURCE_GROUP`), registers the model file as an Azure ML Model asset, and provisions an ACI (Azure Container Instance) webservice.

```python
from azureml.core import Workspace, Model, Environment, InferenceConfig, AciWebservice

class AzureDeployer:
    def __init__(self):
        self.ws = Workspace.get(
            name=os.getenv("AZURE_WORKSPACE_NAME"),
            subscription_id=os.getenv("AZURE_SUBSCRIPTION_ID"),
            resource_group=os.getenv("AZURE_RESOURCE_GROUP")
        )

    def deploy(self, model_file: str, endpoint_name: str) -> str:
        model = Model.register(
            workspace=self.ws,
            model_path=model_file,
            model_name=endpoint_name,
            description="Exported model from ML‑For‑Beginners"
        )
        env = Environment.from_conda_specification(
            name="ml-env", 
            file_path="environment.yml"
        )
        inference_cfg = InferenceConfig(entry_script="score.py", environment=env)
        deployment_cfg = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)
        service = Model.deploy(
            workspace=self.ws,
            name=endpoint_name,
            models=[model],
            inference_config=inference_cfg,
            deployment_config=deployment_cfg,
            overwrite=True
        )
        service.wait_for_deployment(show_output=True)
        return service.scoring_uri

```

### AWS SageMaker Integration

The `SageMakerDeployer` in [`deployment/sagemaker_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/sagemaker_deploy.py) uses `boto3` to upload the model artifact to a specified S3 bucket, then creates a SageMaker Model, EndpointConfig, and Endpoint. You must supply an IAM role ARN with sufficient permissions for model hosting.

### Google Cloud AI Platform Setup

The `GCPDeployer` in [`deployment/gcp_ai_platform_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/deployment/gcp_ai_platform_deploy.py) requires the `GOOGLE_APPLICATION_CREDENTIALS` environment variable pointing to a service-account JSON key. The helper uploads the model to Cloud Storage and calls `aiplatform.Model.upload` to register the artifact before deploying it to an endpoint.

## Complete Code Examples

### Serializing scikit-learn to ONNX for Cross-Platform Serving

ONNX provides the broadest compatibility across Azure ML, SageMaker, and custom container deployments. Use `skl2onnx` to convert your trained estimator.

```python
import skl2onnx
from skl2onnx.common.data_types import FloatTensorType

# Define input signature

initial_type = [("float_input", FloatTensorType([None, X_train.shape[1]]))]
onnx_model = convert_sklearn(trained_model, initial_types=initial_type)

# Persist to the standard models folder

model_path = pathlib.Path("models") / "credit_fraud.onnx"
with open(model_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

```

### Calling an Azure ML Endpoint from a Notebook

After deployment, consume the service using the scoring URI returned by `AzureDeployer.deploy()`.

```python
import requests
import json

scoring_uri = "https://<your-service>.azurewebsites.net/score"
input_data = {"data": [[0.5, 1.2, -0.3, 0.0, 2.1]]}
headers = {"Content-Type": "application/json"}

response = requests.post(scoring_uri, json=input_data, headers=headers)
print("Azure response:", response.json())

```

## Summary

- **Export artifacts consistently** to the `models/` folder using pickle, ONNX, or SavedModel formats so deployment scripts can locate them.
- **Use the provided wrapper classes** (`AzureDeployer`, `SageMakerDeployer`, `GCPDeployer`) to handle authentication, upload, and endpoint provisioning without writing boilerplate SDK code.
- **Consume via HTTP** by replacing local `predict()` calls with `requests.post` to the generated endpoint URL, enabling seamless switching between local debugging and remote production inference.
- **Configure via environment variables** to keep cloud credentials secure and outside the notebook files, adhering to the repository's reproducibility standards defined in [`environment.yml`](https://github.com/microsoft/ML-For-Beginners/blob/main/environment.yml).

## Frequently Asked Questions

### What model formats does ML‑For‑Beginners support for external deployment?

The repository primarily uses **pickle** for scikit-learn models and **ONNX** or **SavedModel** for TensorFlow/Keras estimators. These formats are compatible with the deployment scripts in `deployment/`, which register the artifacts directly with Azure ML, SageMaker, or Google Cloud AI Platform without requiring conversion.

### Do I need to modify the training notebooks to deploy models externally?

No. The training notebooks in folders like `02-Classification/` and `notebooks/` are designed as immutable learning resources. You only interact with the **export** cells at the end of each notebook to generate artifacts, then use the standalone deployment scripts to push those artifacts to external ML model serving platforms.

### How do I switch between local inference and cloud endpoints?

Change the inference function call in your validation code. Instead of loading the model with `joblib.load()` and calling `model.predict()`, use the `call_endpoint()` helper that sends JSON payloads to the REST URL returned by the deployer. This allows the same data preprocessing logic to work against both local files and remote services.

### Which cloud platforms are supported by the deployment scripts?

As implemented in the microsoft/ML‑For‑Beginners source code, the `deployment/` folder includes ready-to-use helpers for **Azure ML** ([`azure_ml_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/azure_ml_deploy.py)), **AWS SageMaker** ([`sagemaker_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/sagemaker_deploy.py)), and **Google Cloud AI Platform** ([`gcp_ai_platform_deploy.py`](https://github.com/microsoft/ML-For-Beginners/blob/main/gcp_ai_platform_deploy.py)). Each script follows the same upload-register-deploy pattern but uses the respective cloud SDK (`azureml-core`, `boto3`, `google-cloud-aiplatform`) under the hood.