Flama: a Python framework that serves any model, from scikit-learn to LLMs, as an API in two commands

[u/p3rdy]

I maintain Flama, an open source (Apache-2.0) Python framework I started in 2018 to get trained models out of notebooks and behind an API without writing a server around them.

Save a model with its metrics into a single file:

import flama
from sklearn.datasets import load_iris
from sklearn.neural_network import MLPClassifier

X, y = load_iris(return_X_y=True)
model = MLPClassifier(max_iter=2000).fit(X, y)
flama.dump(model, family="ml", path="model.flm", metrics={"accuracy": model.score(X, y)})

Then serve it:

pip install "flama[pydantic]"
flama serve --model file=model.flm,url=/model,name=iris

That's a running API with POST /model/predict/, input validation and interactive docs at /docs/.

LLMs work the same way: flama get downloads a model from Hugging Face into a .flm, and flama serve puts it behind OpenAI, Anthropic and Ollama compatible endpoints plus a built-in chat UI, on MLX (Apple Silicon) or vLLM (Linux with CUDA). Existing clients only need a base URL:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/openai/v1", api_key="local")

When a one-liner isn't enough, models become resources in a full ASGI app next to your own routes, with type hints driving validation, OpenAPI docs and dependency injection, and the hot paths (routing, JSON, request parsing) compiled in Rust and shipped as plain wheels.

Right now I'm working on the next release: Python 3.15 support and per-framework extras, so you only install the ML libraries you serve.

Repo: https://github.com/vortico/flama

Docs: https://flama.dev

Happy to answer questions, and if it looks useful, a star on GitHub helps more people find it.