Standard ML Implementation: What I Learned Building Production Systems at Scale

Standard ML Implementation: What I Learned Building Production Systems at Scale

I walked into a client meeting in March 2026 absolutely certain I was going to pitch a pure Rust data pipeline. Three hours later, I left with a mandate to rebuild their entire ML inference stack using Standard ML implementation patterns. Not the language Standard ML (though that's a fine thing). The standard — as in "what should every machine learning implementation look like in production."

Turns out, most people have been doing it wrong.

Here's the thing about production ML: 80% of implementations fail within six months. Not because the models are bad. Because the infrastructure around them is held together with duct tape and hope. I've watched companies burn through $2M in compute credits because someone thought "we'll figure out the engineering later."

I'm Nishaant Dixit, founder of SIVARO. We've been building data infrastructure and production AI systems since 2018. My team has shipped systems processing 200K events per second. We've made every mistake. We've fixed most of them.

This guide is what I wish someone had handed me in 2021.

What "Standard ML Implementation" Actually Means

Standard ML implementation isn't about a specific language or framework. It's a discipline. A set of patterns that separate projects that ship from projects that stall.

Think of it like this: building a CRUD app in 2026 is boring but predictable. You know the stack. You know the patterns. ML implementation should feel the same way. It doesn't — yet. But it's getting there.

The core pieces:

• Model development — what you do in notebooks (chaos, mostly)
• Training infrastructure — where the compute happens
• Serving infrastructure — where predictions happen in real-time
• Monitoring and observability — where you catch the drift before your users do
• Data pipelines — the plumbing nobody wants to talk about

Most people focus on the model. They shouldn't.

The ORM Debate (and What It Taught Me About ML Infrastructure)

I need to tell you a story about ORMs. Stay with me — this connects.

In 2024, my team was building a data pipeline for a fintech client. We needed to pull transaction data from a PostgreSQL database. The engineer on the project — smart kid, MIT grad — wanted to use raw SQL. "ORMs are overrated," he said. "When to use them, and when to lose them," quoting this exact article.

I get the argument. ORMs can be slow. They hide complexity. They create magic that breaks at 2 AM.

But here's what that engineer missed: standardizing the interface matters more than peak performance in most ML implementations. We weren't running one query. We were running 12,000 queries per second across six services. Raw SQL works great until you have to change a column name in production and hunt down 47 inline queries across your codebase.

ORMs are the cigarettes of the data engineering world — that's the counter-argument. And it's not wrong. ORMs are addictive. They're comfortable. They let you pretend the database doesn't matter.

But here's what I've learned after shipping 14 production ML systems: standardization beats optimization in every early-stage implementation.

Now, ORMs Are Awesome makes a compelling case. The author's right about one thing: abstraction layers let you move faster during development. But the real insight? You need both approaches. A Standard ML implementation uses ORMs for the 80% case and raw SQL for the 20% that matters.

We ended up using SQLAlchemy for our fintech pipeline. But we also wrote raw SQL for the three hot-path queries that accounted for 65% of the load. The ORM handled everything else. Raw SQL or ORMs? Why ORMs are a preferred choice — turns out the answer is "both, with intent."

The Five Pillars of Standard ML Implementation

1. Data Pipelines Need Contracts, Not Hope

I can't tell you how many ML implementations I've seen fail because someone thought "we'll just clean the data in the notebook."

No. Absolutely not.

Your data pipeline needs contracts. Explicit, typed, versioned contracts. You're building a data infrastructure company — SIVARO's whole existence is based on this premise.

Here's what a minimal data contract looks like in practice:

python
from dataclasses import dataclass, field
from typing import Optional, List
from datetime import datetime

@dataclass
class TransactionFeature:
    user_id: str
    amount_cents: int  # never use floats for money
    timestamp: datetime
    merchant_category: str
    is_international: bool
    device_fingerprint: Optional[str] = None
    
    def validate(self) -> bool:
        if self.amount_cents <= 0:
            raise ValueError(f"Negative amount: {self.amount_cents}")
        if not self.user_id:
            raise ValueError("Empty user ID")
        return True

This isn't sexy. It's boring. That's the point.

2. Training Infrastructure Is a Sunk Cost — Own It

Most companies treat training infrastructure like a college student treats laundry: they'll deal with it when they run out of clean underwear.

I consulted for a Series B startup in early 2025. They had 12 engineers spending 40% of their time managing training infrastructure. Twelve engineers. On infrastructure that could be solved with three config files and a managed service.

The problem was they'd built their own training orchestrator. From scratch. Because "our needs are unique."

They weren't.

A Standard ML implementation uses off-the-shelf training infrastructure with custom components only where differentiation matters. For 90% of companies, that means:

• A managed GPU cluster (AWS SageMaker, GCP Vertex AI, or Azure ML)
• A job queue (Celery/RabbitMQ or equivalent)
• Experiment tracking (MLflow, Weights & Biases, or Neptune)
• Artifact storage (S3/GCS with versioning)

Stop building the infrastructure. Build the product.

Here's a training pipeline pattern we use at SIVARO:

python
import mlflow
from typing import Dict, Any
import torch

class StandardTrainingPipeline:
    def __init__(self, config: Dict[str, Any]):
        self.config = config
        mlflow.set_experiment(config["experiment_name"])
        
    def train(self, train_data, val_data):
        with mlflow.start_run() as run:
            mlflow.log_params(self.config)
            
            model = self._build_model()
            for epoch in range(self.config["epochs"]):
                train_loss = self._train_epoch(model, train_data)
                val_loss = self._validate(model, val_data)
                mlflow.log_metrics({
                    "train_loss": train_loss,
                    "val_loss": val_loss
                }, step=epoch)
            
            mlflow.pytorch.log_model(model, "model")
            return run.info.run_id

Notice what's missing: error handling for infrastructure failures. That's because the managed service handles it. I'm not paying engineers to reimplement retry logic for spot instance terminations.

3. Serving Infrastructure — Latency Is a Feature

I built a recommendation system in 2023 that had a 40ms P99 latency target. We hit 38ms. The client was thrilled.

Six months later, their traffic doubled. P99 jumped to 210ms. Nobody noticed because nobody was monitoring.

Standard ML implementation demands serving infrastructure that treats latency as a core feature, not an afterthought. You need:

• Model serving with autoscaling — not just running a Flask app on a VM
• Cold start management — models don't load instantly
• Request batching — GPUs are expensive; don't waste them
• Fallback strategies — when the model goes down, what happens?

This is where most implementations fail. They treat model serving like API development. It's not. Models are stateful. They need memory. They need warmup. They crash differently than web servers.

Here's a serving pattern that's served me well:

python
from fastapi import FastAPI, BackgroundTasks
from pydantic import BaseModel
import asyncio
import torch

app = FastAPI()
model_lock = asyncio.Lock()
loaded_model = None

class PredictionRequest(BaseModel):
    features: list[float]
    request_id: str

class PredictionResponse(BaseModel):
    prediction: float
    request_id: str
    model_version: str

@app.on_event("startup")
async def load_model():
    global loaded_model
    # Heavy lifting happens at startup, not during requests
    loaded_model = torch.jit.load("model.pt")
    loaded_model.eval()
    loaded_model.share_memory()  # critical for multiprocessing

@app.post("/predict", response_model=PredictionResponse)
async def predict(request: PredictionRequest):
    # Non-blocking inference with timeout
    try:
        async with asyncio.timeout(0.100):  # 100ms timeout
            tensor = torch.tensor(request.features)
            with torch.no_grad():
                result = loaded_model(tensor)
            return PredictionResponse(
                prediction=result.item(),
                request_id=request.request_id,
                model_version="v2.1.4"
            )
    except asyncio.TimeoutError:
        # Fallback: return cached prediction or default
        return PredictionResponse(
            prediction=0.5,
            request_id=request.request_id,
            model_version="fallback"
        )

Three things to notice here:

1. Model loading happens at startup, not on first request
2. We have a hard timeout
3. There's a fallback response

This is Standard ML implementation. Not clever. Not magical. Just defensive.

4. Monitoring — You're Blind Without It

I worked with an e-commerce company in late 2025. Their ML team had shipped a pricing model that was losing them $40K/week. For six weeks. Nobody caught it because they only monitored model accuracy, not business impact.

Standard ML implementation monitors three things:

• Model metrics — accuracy, precision, recall, drift
• System metrics — latency, throughput, error rates
• Business metrics — revenue, conversion, user engagement

You need all three. Two isn't enough. One is negligent.

Here's what our monitoring stack looks like:

python
import prometheus_client
from prometheus_client import Histogram, Counter, Gauge

# Model performance
prediction_latency = Histogram(
    'model_prediction_latency_seconds',
    'Prediction latency in seconds',
    buckets=[0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5]
)

prediction_counter = Counter(
    'model_predictions_total',
    'Total predictions',
    ['model_version', 'status']  # success/fail/timeout
)

# Data drift
feature_distribution = Gauge(
    'feature_mean_value',
    'Running mean of feature values',
    ['feature_name']
)

# Business impact (logged separately to a BI tool)
revenue_impact = Gauge(
    'model_revenue_impact_dollars',
    'Estimated revenue impact from model decisions',
    ['model_name']
)

You don't need Datadog or New Relic for this. A Prometheus server and a Grafana dashboard cost $0 to set up. The data is free. The only thing you're paying for is the attention to set it up.

5. The Feedback Loop — Close It or Die

Here's the pattern I see most often:

1. Data scientist trains model
2. Model gets deployed
3. Model serves predictions
4. Everyone moves on to the next project

No feedback loop. No way to know if predictions were correct. No way to retrain.

This is the single biggest failure mode in ML implementation.

Standard ML implementation closes the loop. Every prediction gets logged. Every outcome that can be observed gets attributed back to the model. Drift gets detected. Retraining gets triggered.

python
class FeedbackTracker:
    def __init__(self, db_connection):
        self.db = db_connection
        
    def log_prediction(self, prediction_id, features, prediction, model_version):
        self.db.execute(
            "INSERT INTO predictions VALUES (?, ?, ?, ?, datetime('now'))",
            (prediction_id, features, prediction, model_version)
        )
    
    def log_outcome(self, prediction_id, actual_outcome):
        self.db.execute(
            "UPDATE predictions SET actual_outcome = ? WHERE id = ?",
            (actual_outcome, prediction_id)
        )
    
    def get_drift_metrics(self, window_hours=24):
        # Compare recent predictions to historical distribution
        recent = self.db.execute(
            "SELECT AVG(prediction) FROM predictions WHERE timestamp > datetime('now', ?)",
            (f'-{window_hours} hours',)
        ).fetchone()
        historical = self.db.execute(
            "SELECT AVG(prediction) FROM predictions"
        ).fetchone()
        return abs(recent[0] - historical[0])

This isn't hard. It's just discipline.

The Infrastructure You Actually Need

Let me save you some money.

You don't need:

• Kubernetes for your first ML implementation
• A dedicated MLOps team
• Custom hardware
• 10 different cloud services

You do need:

• A feature store (even if it's just a PostgreSQL table with good indexing)
• A model registry (MLflow is fine)
• A prediction log (your database)
• A monitoring dashboard (Grafana)
• A retraining trigger (a cron job or a webhook)

That's it. Add complexity when you have evidence you need it.

I learned this the hard way. In 2022, I advised a healthcare startup that spent $180K on an ML infrastructure stack before they had a single model in production. They burned through their runway on infrastructure that never got used. The company folded six months later.

When Standard ML Implementation Breaks

Nothing is perfect. Standard ML implementation has failure modes.

First: it assumes stable data. If your data distribution shifts dramatically (like COVID did for retail models in 2020), your contracts and pipelines break. You need anomaly detection on top of everything.

Second: it assumes engineering maturity. Standard ML implementation requires your team to care about infrastructure. If your culture is "move fast and break things," this approach will feel bureaucratic.

Third: it's not optimized for research. If you're doing cutting-edge ML research (not application), you need more flexibility. Standard ML implementation is for shipping, not discovering.

The Tiny-C Reference Manual Lesson

Here's something interesting. I was reading the Tiny-C reference manual programming documentation recently — it's a minimal C compiler designed for educational purposes. The author's philosophy stuck with me: "A complete implementation in 500 lines is more instructive than a partial implementation in 5000."

That's Standard ML implementation. Not the most features. Not the most optimized. The most complete.

I've applied this to how we build at SIVARO. We don't try to do everything. We try to do the essential things completely. Our data pipelines process 200K events/second not because they're clever, but because they're simple. They don't fail because there's nothing to fail.

Where We're Going

July 2026, and the landscape is shifting. OfficeCLI AI agents office files is a real thing now — we're seeing AI systems that can autonomously process and generate office documents. That requires Standard ML implementation that handles unstructured data reliably. The patterns I've described apply there too, just with different data types.

The difference between companies that succeed with ML in 2026 and those that don't isn't model architecture. It's implementation architecture. It's the boring stuff: data contracts, monitoring, feedback loops.

I've seen it a hundred times. Two teams. Same model architecture. One ships in two months. The other is still stuck in infrastructure hell a year later. The difference? The first team understood that Standard ML implementation isn't about the model. It's about everything around the model.

Build the boring stuff. Ship the interesting stuff.

---

FAQ

What's the difference between Standard ML implementation and MLOps?

MLOps is the practice of managing ML lifecycle. Standard ML implementation is the specific technical patterns you apply to make that lifecycle work. Think of it as MLOps with an opinionated technical approach.

Should I use managed services or build my own infrastructure?

Start with managed services. I've never seen a company under 50 engineers that should build their own ML infrastructure. The exceptions are companies doing proprietary hardware or novel model architectures.

How do I handle model versioning in production?

Use a model registry. MLflow is the default choice in 2026, but even a versioned S3 bucket with a metadata file works. The key is that every prediction is traceable to a specific model version.

What's the minimal monitoring setup I need?

Three things: a latency histogram, an error counter, and a drift detector. You can build this with Prometheus and a database in a day. Don't overthink it.

How often should I retrain models?

Depends on your data drift. Monitor drift continuously, retrain when it crosses a threshold you've set based on business impact. For most applications, that's weekly to monthly.

Can Standard ML implementation work for real-time systems?

Yes, with modifications. You'll need faster serving infrastructure, streaming data pipelines (Kafka instead of batch), and lower-latency monitoring. The principles stay the same.

What's the biggest mistake companies make?

Trying to do too much too fast. They build for scale they don't have yet. Standard ML implementation starts simple and layers on complexity only when there's evidence it's needed.

---

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.