How to Integrate Trained ML Models with a Flask Web App

You can integrate trained ML models with Flask by serializing the model using joblib, loading it once at application startup to avoid request latency, and exposing REST endpoints that accept JSON payloads and return predictions as JSON responses.

The microsoft/ML-For-Beginners repository demonstrates a production-ready pattern to integrate trained ML models with Flask web applications. This architecture cleanly separates the model training phase from the serving layer, enabling you to deploy scikit-learn models as scalable REST APIs. By following the file structure and loading patterns in modules/06_deploy_model/, you can serve any serialized model with minimal boilerplate.

Architecture Overview

The repository organizes the integration into three distinct layers. This separation ensures that data scientists can retrain models without touching web code, and developers can upgrade the API without risking model integrity.

Layer	Responsibility	Key Source Files
Model Training	Jupyter notebooks train and serialize the model using joblib or pickle.	notebooks/06_Train_Model.ipynb
Model Packaging	The artifact model.pkl resides in a dedicated model/ directory that the Flask app imports at runtime.	modules/06_deploy_model/model/model.pkl
Flask Serving	A lightweight Flask server loads the serialized model once at startup, defines inference endpoints, and returns JSON responses.	modules/06_deploy_model/app.pymodules/06_deploy_model/requirements.txt

Training and Serializing the Model

Before integration, you must persist the trained model to disk. In notebooks/06_Train_Model.ipynb, the training workflow concludes with serialization:

import joblib
from sklearn.ensemble import RandomForestClassifier

# Train your model

model = RandomForestClassifier()
model.fit(X_train, y_train)

# Serialize to the model directory

joblib.dump(model, "model/model.pkl")

Using joblib is preferred over standard pickle for scikit-learn models because it efficiently handles NumPy arrays and large numeric data. The resulting model.pkl file becomes the immutable artifact that the Flask application consumes.

Loading the Model in Flask

The critical performance optimization occurs in modules/06_deploy_model/app.py: the model loads once at application startup, not inside the request handler. This prevents expensive I/O and deserialization from blocking every API call.

import os
from flask import Flask, request, jsonify
import joblib
import numpy as np

app = Flask(__name__)

# Load the serialized model once when the worker starts

MODEL_PATH = os.path.join(os.path.dirname(__file__), "model", "model.pkl")
model = joblib.load(MODEL_PATH)

By assigning model at the module level, you ensure it resides in memory for the process lifetime. This pattern works reliably with WSGI servers like Gunicorn because each worker process loads its own copy of the model during initialization.

Creating Prediction Endpoints

With the model loaded, you define REST endpoints that accept input data, run inference, and return predictions. The repository implements a /predict endpoint that expects a JSON payload containing a features key:

@app.route("/predict", methods=["POST"])
def predict():
    """
    Expects JSON: {"features": [[f1, f2, ...], [f1, f2, ...]]}
    Returns JSON: {"predictions": [0, 1, ...]}
    """
    try:
        payload = request.get_json(force=True)
        X = np.array(payload["features"])
        preds = model.predict(X)
        return jsonify({"predictions": preds.tolist()})
    except (KeyError, ValueError) as exc:
        return jsonify({"error": str(exc)}), 400
    except Exception as exc:
        # Log the full traceback in production logs

        return jsonify({"error": "internal server error"}), 500

The endpoint validates input by converting the JSON list to a NumPy array, catching dimension mismatches or missing keys to return appropriate HTTP 400 errors. For production health monitoring, the repository also includes a /health endpoint that returns a simple status check used by CI/CD pipelines and load balancers.

Containerizing for Production

The modules/06_deploy_model/ directory includes a Dockerfile that containerizes the Flask service for cloud deployment. Containerization ensures that the model runtime environment matches the training environment, preventing dependency conflicts.

# Dockerfile

FROM python:3.11-slim

WORKDIR /app

# Install dependencies

COPY modules/06_deploy_model/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the Flask app and the trained model

COPY modules/06_deploy_model/app.py .
COPY modules/06_deploy_model/model/ model/

EXPOSE 5000
CMD ["python", "app.py"]

The requirements.txt pins specific versions of Flask, scikit-learn, joblib, and NumPy to ensure reproducibility. When building the image, the model.pkl artifact is baked into the container layer, allowing the service to start instantly without downloading assets at runtime.

Security and Error Handling Best Practices

The repository demonstrates several production safeguards in app.py that you should replicate when you integrate trained ML models with Flask.

• Input Validation: The /predict endpoint explicitly checks for the features key and validates that the data can be converted to a numeric array, returning HTTP 400 for malformed requests.
• CORS Configuration: The app initializes flask_cors.CORS(app) to permit cross-origin requests from browser-based frontends, essential for full-stack applications.
• Error Handling: A generic @app.errorhandler(Exception) catches unhandled errors, logs the full traceback server-side, and returns a sanitized JSON error message to the client to prevent information leakage.

Summary

To integrate trained ML models with a Flask web app, follow the three-layer pattern demonstrated in the microsoft/ML-For-Beginners repository:

• Serialize the model using joblib.dump() in your training notebook to create a portable model.pkl artifact.
• Load once at startup by calling joblib.load() at the module level in app.py, ensuring the model resides in memory for all requests.
• Expose REST endpoints that accept JSON input, convert it to NumPy arrays, call model.predict(), and return JSON responses with proper error handling and CORS support.
• Containerize using the provided Dockerfile to deploy the model consistently across development and production environments.

Frequently Asked Questions

How do I prevent the model from reloading on every API request?

Load the model at the module level in your Flask application, outside of any request handler function. In modules/06_deploy_model/app.py, the code calls model = joblib.load(MODEL_PATH) immediately after creating the Flask app instance. This executes once when the Gunicorn or Flask worker process starts, keeping the deserialized model in memory for the process lifetime and eliminating I/O overhead per request.

What format should I use to serialize the trained model?

Use joblib instead of standard pickle for scikit-learn models, as demonstrated in notebooks/06_Train_Model.ipynb. The joblib.dump(trained_model, "model/model.pkl") method efficiently handles large NumPy arrays and numeric data common in machine learning workflows. For deep learning frameworks like TensorFlow or PyTorch, use their native serialization formats (SavedModel or state_dict) and load them similarly in Flask.

How do I handle CORS and security when exposing the model API?

Enable CORS by initializing flask_cors.CORS(app) in app.py to allow browser-based frontends to call the API from different origins. Implement input validation by checking that the JSON payload contains the expected features key and that values are numeric arrays, returning HTTP 400 for malformed data. Add a generic error handler that logs stack traces server-side while returning sanitized JSON error messages to clients, preventing sensitive environment details from leaking to end users.

Can I deploy this Flask model service to the cloud?

Yes, the repository includes a Dockerfile in modules/06_deploy_model/ that containerizes the Flask app and the model.pkl artifact. Build the image with docker build -t ml-flask-app . and deploy it to managed services like Azure App Service, AWS Elastic Beanstalk, or Google Cloud Run. The container exposes port 5000 and runs the app with python app.py, making it compatible with any platform that supports Docker containers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}