r/Python • u/lovespeedtest • 16d ago
Discussion Pushing ThreadingHTTPServer past toy use: long-lived streams, a semaphore cap, and when to give up a
I’ve been running a small home media server for about a year, written entirely with the standard library: http.server.ThreadingHTTPServer, http.client, urllib, threading, queue. No framework and no third-party packages. It’s about 3k lines and serves a handful of my own devices (phone, TV, laptop, tablet). I picked stdlib-only on purpose so it would run anywhere with a bare Python and nothing to upgrade, but the server has grown into things the stdlib server probably wasn’t meant for, and I’d like a reality check from people who’ve been here.
What the traffic looks like
Mostly short JSON requests (library, history, search), which are fine.
Byte-range proxying of large media files, so connections stay open for minutes.
Live HLS: for each live stream a client watches, I run one “watcher” thread that follows the upstream playlist, plus 4 worker threads pulling segments ahead of the player through a
queue.PriorityQueue, with athreading.Conditionto wake request handlers waiting for a segment. A TV can sit on one of these for hours.
What I’ve done so far
Capping concurrency, because thread-per-connection with no limit worried me:
class Server(ThreadingHTTPServer):
daemon_threads = True
slots = threading.BoundedSemaphore(MAX_CONN) if MAX_CONN else None
def process_request(self, request, client_address):
# don't stall the accept loop: if no slot frees up quickly, answer 503 and close
if self.slots and not self.slots.acquire(timeout=0.2):
request.sendall(b"HTTP/1.1 503 Service Unavailable\r\nRetry-After: 10\r\n"
b"Connection: close\r\nContent-Length: 0\r\n\r\n")
self.shutdown_request(request)
return
super().process_request(request, client_address)
def process_request_thread(self, request, client_address):
try:
super().process_request_thread(request, client_address)
finally:
if self.slots:
self.slots.release()Also:
A small keep-alive pool of
http.client.HTTPSConnectionobjects per upstream host (up to 8 idle per host, behind a lock), because opening a fresh TLS connection per media segment was the slowest part.State lives in JSON files, written to
file.tmpand thenos.replace()so a crash mid-write can’t corrupt anything. Every read-modify-write goes through one module-levelthreading.Lock.
What I’m unsure about
Is the semaphore in
process_requestthe right place? It counts connections, not work. A client holding one hours-long live connection costs the same slot as a 5 ms JSON call. Would two pools be saner, one for streaming paths and one for everything else? Is that possible without rewriting the server class?Clients that disappear. When a TV is switched off mid-stream, sometimes I get
BrokenPipeError/ConnectionResetErrorright away, and sometimes the handler thread sits inwfile.write()until the socket timeout. What do people set for timeouts on long-lived responses, so dead clients are dropped without killing slow-but-alive ones?Background threads per stream. Five threads per live stream is fine for 1–2 streams, but it feels wasteful. Is anyone running a single asyncio loop in a side thread just for the upstream fetching, while keeping the threaded HTTP front end? Or does mixing the two models end up worse than picking one?
The single lock for JSON state. It’s simple and correct, but every library write serializes behind it. At this scale I doubt it matters. At what point did you move to
sqlite3, which is still stdlib, and was it worth it?The honest question: at what point is stdlib-only a liability rather than a virtue? I know aiohttp/Starlette would make the streaming parts nicer. I’m trying to work out whether I’ve hit that point or am just bothered by how the code looks.
I’m not chasing scale. This will never serve more than ~10 clients. I mostly want to know which of these are real problems and which are fine at this size. War stories welcome.
11
u/clcodingupdates 16d ago
Don't optimize for hypothetical scale. Measure thread count, memory usage, connection latency, and streaming failures first. If those numbers are healthy, your current architecture may be perfectly adequate.
2
u/lovespeedtest 16d ago
Good point. I'll add some basic numbers (threads, memory, failed streams) before touching anything.
9
u/larsga 16d ago
So, to summarize, you have no problems, and you want input on how to solve these problems you don't have?
Sounds to me like this is fine.
5
1
u/lovespeedtest 16d ago
Ha, fair. More "is this going to bite me later" than an actual problem. Sounds like the answer is no.
5
u/Khavel_dev 16d ago
The stdlib server was never designed for long-lived connections. Your semaphore cap is smart, but the core issue is that each idle connection ties up a full OS thread with its stack. On a home server with maybe 6 devices you're fine. But if each HLS stream spawns 1 watcher + 4 workers + the client handler, that's 6 threads per stream, and the ceiling comes fast.
For what you're doing (lots of waiting, little compute), asyncio is a way better fit. One event loop handles hundreds of idle connections without a thread per connection. The migration is annoying because everything becomes async/await, but the concurrency model actually matches your workload.
If you're allergic to asyncio, the selectors module is stdlib and lets you build a basic event loop that does I/O multiplexing without threads. Less ergonomic but still solves the thread exhaustion problem.
0
u/lovespeedtest 16d ago
Fair point, and you're mostly right. The watcher + 4 workers are per live stream (shared if several clients watch the same one), so it's ~5 threads per stream plus one per client. Fine at home scale, but I'll probably cut the workers per stream before thinking about asyncio.
10
u/wRAR_ 16d ago
I’d like a reality check from people who’ve been here.
You can't distinguish that from AI responses, so...
0
16d ago
[deleted]
7
u/spinachspelunking 16d ago
So write it in your first language and let Reddit translate it? Or pass it through a translator? Heck, I'd rather read broken english than whatever this slop is.
4
1
u/AleDev99 16d ago
At ~10 clients most of this is fine; the two things I'd actually change are timeouts and the single semaphore. For dead clients, set timeout = 30 on your handler class (StreamRequestHandler applies it to the socket) and write media in smallish chunks, so a TV that vanished raises socket.timeout on the next write instead of hanging, while a slow-but-alive client still finishes each chunk well inside 30s. For slots you don't need to rewrite the server: keep a generous global cap, and acquire a second semaphore inside the handler after you've parsed the path, one for streaming routes and one for everything else, so an hours-long HLS session never blocks a 5 ms JSON call. I'd leave the per-stream threads alone until they actually cause trouble, because an asyncio side loop next to a threaded front end works but means debugging two concurrency models. sqlite3 becomes worth it the first time you want a query instead of loading the whole JSON file, and WAL mode gives you concurrent readers for free.
1
u/lovespeedtest 16d ago
This is really helpful, thanks. The handler timeout plus a second semaphore for streaming routes is exactly the kind of small change I was hoping for. Going to try that first and leave threads alone.
1
u/AleDev99 16d ago
Glad it helps! One small tip: set the handler timeout a bit above your slowest normal request so you only cut off the truly stuck ones, and log when the streaming semaphore is full. That tells you quickly if you need a higher limit. Good luck, curious to hear how it goes.
2
u/yvrelna 16d ago
Anything under a single household would be sufficiently served with the standard library.
Five threads are practically nothing for a modern CPU. We're not living in the 1960s.
sqlite3
I generally just default to sqlite3 for local data operations, not because of scalability concerns, but because the database transactions paradigm is mentally much easier to deal with than dealing with manual locks and concurrency. But that's probably also because I came from a professional web development background and has extensive experience working with serious databases, while relatively less on local locks.
3
u/WasteFeedbackrdhvv 15d ago
Respect for staying stdlib-only with ThreadingHTTPServer, BoundedSemaphore capping, and five threads per live HLS stream
-1
u/Sachu_James 12d ago
The part I would look at first is the four workers per stream. If the watcher already puts segments into the queue and the handlers wait on the Condition, the workers look like an extra hop, since each handler thread is already a consumer. Dropping to one watcher per stream and letting handlers read straight from the queue would cut the thread count without changing the streaming behaviour, and it keeps the per stream state in one place.
20
u/Rudd-X 16d ago
Your design seems safe and it works. Kudos for creating your media server!