r/java • u/elliotbarlas • Jul 25 '22
Latest version of Microhttp, an event-driven, zero-dependency, pure-Java web server with 500 LOC, capable of 1,000,000+ requests per second on commodity EC2 hardware.
https://github.com/ebarlas/microhttp40
u/FirstAd9893 Jul 25 '22
As currently implemented, I wouldn't recommend using this except when all clients can be trusted. The request headers are stored in an unbounded list, and so searching for headers has an O(n) cost. This makes writing DOS attacks too easy.
The server should at the very least have a header filter to avoid parsing and storing headers that won't be examined.
22
u/elliotbarlas Jul 25 '22
Microhttp also doesn't provide TLS, HTTP/2, etc. You have to look at it in the context of a broader system. In a deployment setting, Microhttp would sit behind a reverse proxy or load balancer, as mentioned in the intro of the GitHub repo. Header policies could live there.
Having said that, adding
maxHeaderCountandmaxHeaderSizetoOptionsis quite a reasonable ask.10
u/FirstAd9893 Jul 25 '22
I think it's important to measure performance in the context of a broader system, when all the necessary security measures are in place. What then is the overall cost, and what are the recommendations?
19
u/elliotbarlas Jul 25 '22 edited Jul 25 '22
In my experience, a reverse proxy or load balancer network appliance is used to terminate TLS and to route requests in a content-aware way.
There are both hardware and software network appliances deployed on-premise and in the cloud. Netscaler, HAProxy, Nginx, ELB, ALB, etc.
Microhttp is a reflection of that deployment landscape. And the benchmark numbers simply show the potential of that one component on its own.
In any production setting, system-wide tests should absolutely be conducted! No disagreement there.
7
u/0x256 Jul 25 '22
Or send many request in parallel with huge content-lenght values, each sending bytes one by one, stopping one short and waiting for the timeout. Tons of allocations and array-copys, everything held in-memory. It's easy to be fast if you only implement a fraction of the protocol and handle only the good path. Nice educational piece of software and interesting to read, but competing against actually useable http implementations in a benchmark seems a bit unfair and misleading.
7
u/elliotbarlas Jul 25 '22
Slow requests are a universal challenge for web servers. There is no silver bullet.
Microhttp handles them with minimal resource utilization. No dedicated thread. Byte buffer that expands multiplicatively (1024, 2048, 4096, etc). Maximum request size limit. Maximum time to receive request. Simple, efficient timeout scheduler.
Do you have an alternative solution in mind for slow requests?
6
u/0x256 Jul 25 '22 edited Jul 26 '22
Not only slow clients, but request body handling in general seems to be problematic as it is currently implemented:
The whole body is buffeted in memory, limiting uploads to 1-2gb and wasting memory where the request could be processed as a stream.
No body streaming means protocol upgrades (e.g. websockets) are impossible to implement on top of the current API.
The body is buffered even if the application does not expect a body and could answer with an error immediately.
100‐continue (a mandatory http/1.1 flow) is not implemented, causing valid clients to stall and timeout.
The framework should not read the request body unless the application explicitly requests it, treat it as an error (413) if there is a body the application did not expect, and allow reading data in chunks of suitable size with application defined timeouts per chunk.
9
u/elliotbarlas Jul 26 '22 edited Jul 26 '22
This falls squarely outside of the Microhttp domain. Microhttp discretizes all requests and responses. Streaming is not supported. It's tailored for small payloads. All stated up front in the docs.
This covers the vast majority of web services and REST APIs that I build.
This also reduces complexity tremendously and contributes greatly to the small code footprint.
9
u/0x256 Jul 26 '22
Then you should document more clearly that this is an incomplete HTTP 1.1 implementation and should not be used in internet-facing applications. At least 100‐continue should be implemented though, as a reverse proxy like nginx won't protect you from that.
7
u/elliotbarlas Jul 26 '22
Thanks for the recommendation. I'll clarify in docs that Expect-100 is not currently supported. I'll also reevaluate adding support. It ought to be straightforward.
2
u/agentoutlier Jul 26 '22
To be fair OOB both Undertow and I believe Netty do not support expect 100. Jetty does.
Some frameworks You need to configure it or use some add on classes.
I imagine this library could do similar.
2
u/0x256 Jul 26 '22 edited Jul 26 '22
I doubt that, and for undertow I know you are wrong. 100-continue is not a SHOULD feature, if a client uses it the server MUST handle it or the request will run into a timeout. For highly configurable 'know-what-you-are-doing' libraries like Undertow or Netty this may be pluggable, but it is supported or that would be a serious bug.
Edit: You probably mean that it is not enabled or magically handled by default in Undertow. Yes, because undertow and netty implement almost anything in 'handlers' you have to manually add and configure. The idea of 100-continue is that the application can decide whether it wants to answer immediately, or needs the offered request body for a result. An example would be a client that wants to upload a file, but only if it is not already present on the server. With 100-continue, this can be implemented in a single atomic request without wasting bandwidth in the case the file is not needed. Undertow lets you handle that explicitly (with
HttpContinueAcceptingHandlerand a Predicate). Other frameworks may automatically send the 100-continue response as soon as the application tries to read from the request body. In any way, all complete HTTP/1.1 implementations must somehow support that.1
u/agentoutlier Jul 27 '22
Dude I know how important it is. I filed a bug on at least two frameworks that didn’t have it turned on:
https://github.com/jooby-project/jooby/issues/2363
So I’m well aware.
The other one was a netty lib that I just can’t find the bug for.
All I was saying that if the damn author wants to keep the core code base small they can do similar.
But yeah thanks for the mansplaining of HTTP and spec semantics. You might have as well included the link for me because I’m too dense: https://datatracker.ietf.org/doc/html/rfc2119
2
u/0x256 Jul 27 '22
This is a public discussion, so please don't feel personally attacked just because I explained something you might already know.
I explained the 100-continue flow to show that it might be a good idea to pass the final decision to the application and that application developers might benefit from this protocol feature, if it is exposed to them. This in turn explains why low-level libraries like Undertow or Netty let you control their behavior in such a great detail and not just hide the complexity and automatically handle everything for you. Undertow still fully supports 100-continue, and Netty does too. That was my point. Do you disagree?
→ More replies (0)
15
u/ZeroGainZ Jul 25 '22
I'm sure this is cool, but I never quite understood why libraries advertise their LOC. Especially for something like java. Anyways, I'll give it a peek 👍
13
u/elliotbarlas Jul 25 '22
I tend to agree, but Microhttp is specifically a response to commonly used frameworks with overwhelming complexity, so I wanted to indicate up front how small it is.
8
u/FirstAd9893 Jul 25 '22
More lines of code typically means more bugs. Advertising a low LOC is an indirect way of claiming that the software likely has fewer bugs than a similar project of the same age.
9
u/elliotbarlas Jul 25 '22
Comprehensibility is one of the main objectives. Simply put, one can spend a few hours and develop an understanding of this library. The same just isn't possible for most Java web servers or frameworks.
13
u/RockleyBob Jul 26 '22
What I saw:
500 LOC
What I read:
The opposite of Spring
I think that’s the gist. No magic.
1
u/portmapreduction Jul 26 '22
It's a better descriptor versus saying your library is 'lightweight' which usually translates to something like 'it fits my use case exactly since I wrote it with no fluff, thus it is lightweight'.
2
Jul 27 '22
Neat, are there plans for adding websockets?
1
u/elliotbarlas Jul 27 '22
Thanks. No plans to add support for WS. Like a few of the other exclusions, that would increase the scope considerably. But neat idea to explore in a fork.
4
u/portmapreduction Jul 26 '22
Pretty impressive given it's size. Kind of cheating with all the records that would be 100 lines a pop with autogen'd tostring/equals/hashcode (kidding, it's actually a good advertisement for records). I've tended to use netty for these kinds of tasks but it's nice to see someone use nio itself instead.
1
2
0
Jul 26 '22
[deleted]
4
u/elliotbarlas Jul 26 '22
Microhttp is both event-driven and multi-threaded. By default, the concurrency level is the number of CPUs.
Each thread waits for network I/O events using an NIO selector.
A particular connection is managed by a single thread and selector. No locks or synchronized blocks.
1
u/brunocborges Jul 26 '22
I'd be curious to see how Microhttp would work on Kubernetes with CPU limits applied.
5
u/pointy_pirate Jul 26 '22
Java has a lot of NIO frameworks, yes they are less popular (especially in the enterprise world) but NIO serves some specific use cases that are very much at home in the Java ecosystem.
12
u/elliotbarlas Jul 25 '22
Microhttp is registered with TechEmpower Framework Benchmarks. Latest continuous run results:
2,163,060 req/sec - plaintext test
1,268,535 req/sec - JSON test
284,587 req/sec - DB query test
https://www.techempower.com/benchmarks/#section=test&runid=c72182f8-476d-4920-a264-26f615d693d3&test=plaintext
The TechEmpower benchmark repo is here:
https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Java/microhttp