How to Integrate with External ML Model Serving Platforms Using ML‑For‑Beginners

You can integrate with external ML model serving platforms by exporting trained models to the models/ directory, invoking the platform-specific deployment wrappers in deployment/, and substituting local prediction calls with authenticated HTTP requests to the managed endpoint.

The ML‑For‑Beginners repository is a hands-on educational framework that modularizes the machine learning lifecycle into discrete stages. Because the curriculum isolates training logic from deployment concerns, you can integrate with external ML model serving platforms without altering the core Jupyter notebooks. The repository ships with ready-to-use deployment scripts for Azure ML, AWS SageMaker, and Google Cloud AI Platform that handle artifact registration and endpoint provisioning.

The Modular Architecture That Enables Integration

The repository separates concerns into five distinct layers, each mapped to specific folders and file patterns:

• Data Preparation – Handled in data/ folders and <chapter>/data_preprocess.ipynb files that load and clean raw datasets.
• Model Training – Isolated in <chapter>/train_*.ipynb notebooks (e.g., 02-Classification/train_logistic_regression.ipynb) that build and evaluate estimators.
• Model Export – Serializes artifacts to the models/ folder via scripts like export_model.py or notebook cells that pickle scikit-learn objects or convert TensorFlow/Keras graphs to ONNX/SavedModel.
• Inference Demo – Local validation performed in inference_demo.ipynb by loading artifacts from disk.
• Deployment Template – Platform-specific automation housed in deployment/azure_ml_deploy.py, deployment/sagemaker_deploy.py, and deployment/gcp_ai_platform_deploy.py.

This separation allows you to treat the training notebooks as immutable learning resources while still productionizing their outputs.

Three-Step Integration Workflow

To integrate with any external ML model serving platform, follow the export-deploy-consume pattern implemented in the repository helpers.

Step 1 – Export Models to Portable Formats

Most training notebooks conclude by persisting the estimator to the models/ folder. For broad compatibility with cloud serving platforms, convert scikit-learn models to ONNX or use native formats like pickle.

import joblib
import pathlib
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType

# Standard pickle export

model_path = pathlib.Path("models") / "credit_fraud.pkl"
joblib.dump(trained_model, model_path)

# ONNX export for optimized inference

initial_type = [("float_input", FloatTensorType([None, X_train.shape[1]]))]
onnx_model = convert_sklearn(trained_model, initial_types=initial_type)
onnx_path = pathlib.Path("models") / "credit_fraud.onnx"
with open(onnx_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

Step 2 – Deploy Using Platform Helpers

The deployment/ directory contains wrapper classes that encapsulate authentication, artifact upload, and endpoint creation. Import the appropriate deployer based on your target platform.

import os
from deployment.azure_ml_deploy import AzureDeployer
from deployment.sagemaker_deploy import SageMakerDeployer
from deployment.gcp_ai_platform_deploy import GCPDeployer

def deploy_to_platform(model_file: str, endpoint_name: str, platform: str) -> str:
    """
    Uploads ``model_file`` to the chosen ``platform`` and returns the inference URL.
    Supported values: "azureml", "sagemaker", "gcp".
    """
    if platform == "azureml":
        deployer = AzureDeployer()
        return deployer.deploy(model_file, endpoint_name)
    if platform == "sagemaker":
        deployer = SageMakerDeployer()
        return deployer.deploy(model_file, endpoint_name)
    if platform == "gcp":
        deployer = GCPDeployer()
        return deployer.deploy(model_file, endpoint_name)
    raise ValueError(f"Unsupported platform: {platform}")

# Example invocation

endpoint_url = deploy_to_platform(
    model_file=str(model_path),
    endpoint_name="credit-fraud-detector",
    platform="azureml"
)
print("Endpoint ready at:", endpoint_url)

Step 3 – Consume the Remote Endpoint

Replace local model.predict() calls with HTTP POST requests that send JSON payloads to the deployed service.

import requests
import json

def call_endpoint(url: str, payload: dict) -> dict:
    """Transmits a JSON payload to the model endpoint and returns the prediction."""
    headers = {"Content-Type": "application/json"}
    response = requests.post(url, data=json.dumps(payload), headers=headers)
    response.raise_for_status()
    return response.json()

# Invoke the served model

sample = {"features": [0.5, 1.2, -0.3, 0.0, 2.1]}
prediction = call_endpoint(endpoint_url, sample)
print("Remote prediction:", prediction)

Platform-Specific Implementation Details

Each helper script in deployment/ follows an identical three-phase pattern: upload the artifact, register a model resource, and expose a REST endpoint. Configuration is handled via environment variables to keep credentials out of source control.

Azure ML Deployment via deployment/azure_ml_deploy.py

The AzureDeployer class loads workspace credentials from environment variables (AZURE_WORKSPACE_NAME, AZURE_SUBSCRIPTION_ID, AZURE_RESOURCE_GROUP), registers the model file as an Azure ML Model asset, and provisions an ACI (Azure Container Instance) webservice.

from azureml.core import Workspace, Model, Environment, InferenceConfig, AciWebservice

class AzureDeployer:
    def __init__(self):
        self.ws = Workspace.get(
            name=os.getenv("AZURE_WORKSPACE_NAME"),
            subscription_id=os.getenv("AZURE_SUBSCRIPTION_ID"),
            resource_group=os.getenv("AZURE_RESOURCE_GROUP")
        )

    def deploy(self, model_file: str, endpoint_name: str) -> str:
        model = Model.register(
            workspace=self.ws,
            model_path=model_file,
            model_name=endpoint_name,
            description="Exported model from ML‑For‑Beginners"
        )
        env = Environment.from_conda_specification(
            name="ml-env", 
            file_path="environment.yml"
        )
        inference_cfg = InferenceConfig(entry_script="score.py", environment=env)
        deployment_cfg = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)
        service = Model.deploy(
            workspace=self.ws,
            name=endpoint_name,
            models=[model],
            inference_config=inference_cfg,
            deployment_config=deployment_cfg,
            overwrite=True
        )
        service.wait_for_deployment(show_output=True)
        return service.scoring_uri

AWS SageMaker Integration

The SageMakerDeployer in deployment/sagemaker_deploy.py uses boto3 to upload the model artifact to a specified S3 bucket, then creates a SageMaker Model, EndpointConfig, and Endpoint. You must supply an IAM role ARN with sufficient permissions for model hosting.

Google Cloud AI Platform Setup

The GCPDeployer in deployment/gcp_ai_platform_deploy.py requires the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a service-account JSON key. The helper uploads the model to Cloud Storage and calls aiplatform.Model.upload to register the artifact before deploying it to an endpoint.

Complete Code Examples

Serializing scikit-learn to ONNX for Cross-Platform Serving

ONNX provides the broadest compatibility across Azure ML, SageMaker, and custom container deployments. Use skl2onnx to convert your trained estimator.

import skl2onnx
from skl2onnx.common.data_types import FloatTensorType

# Define input signature

initial_type = [("float_input", FloatTensorType([None, X_train.shape[1]]))]
onnx_model = convert_sklearn(trained_model, initial_types=initial_type)

# Persist to the standard models folder

model_path = pathlib.Path("models") / "credit_fraud.onnx"
with open(model_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

Calling an Azure ML Endpoint from a Notebook

After deployment, consume the service using the scoring URI returned by AzureDeployer.deploy().

import requests
import json

scoring_uri = "https://<your-service>.azurewebsites.net/score"
input_data = {"data": [[0.5, 1.2, -0.3, 0.0, 2.1]]}
headers = {"Content-Type": "application/json"}

response = requests.post(scoring_uri, json=input_data, headers=headers)
print("Azure response:", response.json())

Summary

• Export artifacts consistently to the models/ folder using pickle, ONNX, or SavedModel formats so deployment scripts can locate them.
• Use the provided wrapper classes (AzureDeployer, SageMakerDeployer, GCPDeployer) to handle authentication, upload, and endpoint provisioning without writing boilerplate SDK code.
• Consume via HTTP by replacing local predict() calls with requests.post to the generated endpoint URL, enabling seamless switching between local debugging and remote production inference.
• Configure via environment variables to keep cloud credentials secure and outside the notebook files, adhering to the repository's reproducibility standards defined in environment.yml.

Frequently Asked Questions

What model formats does ML‑For‑Beginners support for external deployment?

The repository primarily uses pickle for scikit-learn models and ONNX or SavedModel for TensorFlow/Keras estimators. These formats are compatible with the deployment scripts in deployment/, which register the artifacts directly with Azure ML, SageMaker, or Google Cloud AI Platform without requiring conversion.

Do I need to modify the training notebooks to deploy models externally?

No. The training notebooks in folders like 02-Classification/ and notebooks/ are designed as immutable learning resources. You only interact with the export cells at the end of each notebook to generate artifacts, then use the standalone deployment scripts to push those artifacts to external ML model serving platforms.

How do I switch between local inference and cloud endpoints?

Change the inference function call in your validation code. Instead of loading the model with joblib.load() and calling model.predict(), use the call_endpoint() helper that sends JSON payloads to the REST URL returned by the deployer. This allows the same data preprocessing logic to work against both local files and remote services.

Which cloud platforms are supported by the deployment scripts?

As implemented in the microsoft/ML‑For‑Beginners source code, the deployment/ folder includes ready-to-use helpers for Azure ML (azure_ml_deploy.py), AWS SageMaker (sagemaker_deploy.py), and Google Cloud AI Platform (gcp_ai_platform_deploy.py). Each script follows the same upload-register-deploy pattern but uses the respective cloud SDK (azureml-core, boto3, google-cloud-aiplatform) under the hood.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}