Deploying machine learning models with Flask and Docker

Getting a model to good accuracy is the part most tutorials cover. Getting it to answer an HTTP request reliably, in an environment you did not train it in, is the part that stalls projects. This walkthrough covers the path from a trained model to a containerised service.

Prerequisites

• Working Python (3.7+)
• Familiarity with machine learning basics
• Docker installed

1. Serialise the model

Before anything else, get the trained model out of memory and onto disk in a form you can load back:

import joblib
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier()
model.fit(X_train, y_train)

joblib.dump(model, "model.joblib")

Pin the versions of the libraries used to train. A model pickled under one scikit-learn version and loaded under another is a class of bug you do not want to debug in production.

2. Wrap it in an API

Flask is enough for a prediction endpoint:

from flask import Flask, request, jsonify
import joblib

app = Flask(__name__)
model = joblib.load("model.joblib")

@app.route("/predict", methods=["POST"])
def predict():
    data = request.json
    prediction = model.predict([data["features"]])
    return jsonify({"prediction": prediction.tolist()})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

Load the model once at import, not per request. Deserialising on every call is the most common reason a prediction endpoint is slow for no good reason.

3. Containerise

Docker is what makes “works on my machine” stop being a sentence anyone says:

FROM python:3.8-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 5000
CMD ["python", "app.py"]

For anything beyond a demo, put a production WSGI server in front — Gunicorn with a few workers — rather than Flask’s development server.

4. Practices worth keeping

• Validate the input. Check shape, type and range before the model ever sees the payload.
• Handle errors deliberately. Return a useful status code and message; never let a stack trace reach the caller.
• Version code and models together. A prediction you cannot reproduce is a prediction you cannot defend.
• Monitor. Track latency, error rate and the distribution of incoming features. Drift shows up in the inputs before it shows up in the accuracy.

5. Common pitfalls

• Missing data handled differently at inference than during training
• No error handling around deserialisation or prediction
• No logging, so failures are invisible until someone complains
• Single-process serving under concurrent load
• Endpoints exposed without authentication or rate limiting

Closing

Deployment is where a model stops being an experiment. Serialise deliberately, validate the input, containerise the environment, and put enough monitoring in place that you find out about problems before your users do.