r/java • u/speakjava • Oct 26 '20
Why we chose Java for our High-Frequency Trading application
https://medium.com/@jadsarmo/why-we-chose-java-for-our-high-frequency-trading-application-600f7c04da948
Oct 26 '20
ZGC and Shenandoah may or will take care of GC pause times.
As others have said, Graal will take care of them warmup time (i.e. remove the overhead of running in the interpreter).
With both of these in place, I am not sure how much better Azul Zing is compared to OpenJDK. Perhaps, the article is moot.
2
u/speakjava Oct 26 '20
Graal can eliminate warmup time through ahead-of-time compilation, which is great for certain situations like 'serverless' computing (a really bad name IMO).
However, it is important to understand that the code generated will be significantly less optimised than that generated using a JIT through adaptive compilation. Even with Graal's profile guided optimization (PGO) you won't get as efficient code. Java can dynamically load classes at runtime that, by definition, limit the method inlining that can be performed by static compilation. In addition, speculative optimisations, that can deliver significant performance improvements are heavily restricted with AOT compilation.
Zing replaces the 20+ year old C2 JIT in Hotspot with Falcon that uses the LLVM compiler back-end for native code generation. The resulting native code is more heavily optimised for many applications.
Full disclosure, I work for Azul.
14
u/cl4es Oct 26 '20
When C2 was first released is completely irrelevant. What matters is whether it's actively maintained, improved and developed alongside other runtime features, be it ZGC, Loom, Valhalla...
And C2 is all of that. (It's also 100% open source.)
So can we please refrain from calling a technology that is successfully powering a huge number of production systems across the globe "legacy" and "old" just because you are investing in and selling a competing technology?
Thanks!
(I work for Oracle on the OpenJDK - C2 included.)
2
u/speakjava Oct 28 '20
That's a fair comment.
If the age of a technology had an impact then I'd be in the wrong job promoting Java, since it's over 25 years old :-).
The primary reason for replacing C2 with Falcon was the modular nature of the LLVM compiler design (on which it's based), making it much easier to add new optimisations and features.
5
u/cl4es Oct 28 '20
C2 surely have some challenges, and it's tempting to replace it - Oracle's Graal shows that Azul is definitely not alone in thinking C2 might need to be replaced (at some point).
But C2 remains the default JIT of the OpenJDK and I think it will remain so in the mainstream until some party contributes a fully open source alternative that is demonstrably superior in every way that matters. Which includes performance, maintainability and cohesion with the rest of the project.
I sadly don't see this happening any time soon, so I am and have been advocating that anyone invested in OpenJDK should also commit and contribute to the development of C2. Which doesn't necessarily preclude also investing in current and future alternatives, be it Falcon or Graal or something else.
5
4
u/vips7L Oct 26 '20
Can you use ZGC or Shenandoah in graal? Maybe in VM mode but I don't think you can with native image.
3
Oct 26 '20
Graal still needs a GC so why not ZGC or Shenandoah? If it isn't available today, I do not see why it can't be done in the future.
2
3
Oct 26 '20
Most exchanges where latency matters the most have a very well defined trading window where all sorts of JVM warm ups can be done to get your applications trading-ready. I'm not sure if the JIIT compiler may actually be an advantage here as you can give it data that matches real-time conditions and allow it to tune before market open.
0
Oct 26 '20
[deleted]
2
Oct 26 '20
One mode of Graal VM can be used to optimize *all* of the bytecode into machine instructions. The result is a binary executable that runs on the machine and JIT does not exist. So, basically as soon as the OS loads the executable, the code is optimized and the first call to any method runs optimized machine instructions. No need to load from cache.
One of the drawbacks is that Graal VM does not know how the application is going to be used. This means it has to guess as to which way a branch will be taken. If it guesses wrong, then the final performance will be less than what JIT could do. To overcome this, one can produce a profiling executable, run it, generate a profile and feed the profile back into Graal. Graal then produces the optimal executable for that workload. Yes, it is extra steps but if you are looking to trim microseconds, then any step is worth it.
12
u/alex_tracer Oct 26 '20 edited Oct 26 '20
I wonder why they do not mention Shenandoah GC that present in OpenJDK or ZGC in Oracle's version.
1
0
Oct 26 '20
[deleted]
10
u/alex_tracer Oct 26 '20
Probably because it's not battle tested for low-latency applications.
Wait, you really believe in that?
The main reason of creation of Shenandoah GC is reduction of GC pauses. It shows low (~1ms) pauses for huge (like 100GB) heaps (vs 4Gb in article above). And it already widely used in production by some quite big companies (1000+ hosts).
So as for me,the article above should be considered to be an advertisement. If they were really looking for a solution for the latency problem they would consider other options ( Shenandoah, ZGC ).
haven't seen any stats on it
That's why I've put a link to Shenandoah page. It has stats :)
1
u/pmarschall Oct 27 '20
different memory semantics in G1
You should get the JMM memory semantics. Do you mean the old-gen write barrier?
23
u/cogman10 Oct 26 '20
I read this and, honestly, the argument for using Java here seems really weak.
With a small team, limited resources, and a job market scarce in skilled developers, Java meant we could quickly add software improvements as the Java ecosystem has quicker time-to-market than C derivatives. An improvement can be discussed in the morning, and be implemented, tested and released in production in the afternoon.
Nothing about C/C++/Rust (other than an arguably scarce job market) make it impossible, or indeed hard, to do a "Idea in the morning, deploy in afternoon" workflow. Any CI/CD pipeline can accomplish this.
The needed attributes here (very low pause time, very fast startup) scream "use C/C++/Rust". Particularly because they are already trading off throughput using C4 to escape pause time problems.
20
u/tetrarkanoid Oct 26 '20
GC'd languages are more productive than languages that require manual memory management.
5
3
u/pikob Oct 26 '20
But, if you use RAII everywhere, and use a consistent approach to handling allocated days, it's not that much of an issue.
For me, it's compile times, better intellisense and package management.
2
u/ObscureCulturalMeme Oct 26 '20
use a consistent approach to handling allocated days
Was that an autocorrect gotcha?
I don't know how many days I've been allocated, but looking back over my life I must admit I've been woefully inconsistent in handling them. That's usually not been the fault of the JVM, however, barring one particularly frustrating debugging session.
2
u/vytah Oct 27 '20
That never happens outside of some simpler projects.
Any time your object graph stops being a tree, trivial ownership rules stop working.
1
3
Oct 26 '20
fwiw - nasdaq's market tech group offers exchange operators matching technology that I recall hearing is mostly written in Java.
2
Oct 27 '20
Except you missed the most important attribute, stability. They say at the start they absolutely cannot afford ANY down time due to bugs, which is much easier to guarantee in a GC language.
1
u/grimonce Oct 27 '20
Java is C derivative, wth? It is a C-like language.
1
u/DannyB2 Oct 28 '20
Java syntax looks superficially like C. The similarity ends there.
The semantics of the languages, and the libraries are different.
3
Oct 26 '20
[removed] — view removed comment
-2
u/cogman10 Oct 26 '20
why would anyone risk the memory unsafe code of C just to save a small ( compared to a developer salary ) amount of $ on RAM?
The win from C for HFT isn't ram saved. The win from C is startup performance, runtime performance, and consistency. You can always guarantee a 1ms response time. You won't be interrupted by a GC ever. Further, if you are in a critical portion of code, you can avoid touching the heap all together in those cases.
You pick C because you need high performance, low startup time, and consistent performance. All things this article calls out as issues they've ran into using java.
15
u/AndiiKaa Oct 26 '20
Just want to emphasise, that you can‘t guarantee the 1ms response time with c alone. If you want guaranteed response times you need a real time operating system in the first place.
2
u/DannyB2 Oct 28 '20 edited Oct 28 '20
Startup time should not be a consideration. You start up once and then run for a very very long time. You want high performance over a long time with very short or no GC pauses. Depending on your requirements. Use a modern GC if 1 ms pauses are acceptable.
If you can't have GC, then design a Java program like a Java game. Allocate all your structures up front. Then enter the main loop where you do the work and use all of the pre-allocated structures. No new memory allocation should be done. Thus no GC happens.
Memory cost is a one time capital cost. And it is very low. An extra 64 GB of ram added to a production server is way less than the cost of a developer. Optimizing for developer time is highly important. Things have changed since the mainframe daze when machines were expensive and programmers were cheap. Now the cost of a developer (with benefits) for one year can buy an awful lot of hardware.
You can never buy back time. So optimize for time at the cost of memory. You can always buy more memory. And a few hundred extra GB of heap is cheap, cheap cheap!
There is also a thing called Opportunity Cost. If I can beat my C++ using competitor to market by six months to a year, and only for the cost of an extra 64 GB of ram in the production server, my manager and I will laugh all the way to the bank.
1
u/helikal Oct 26 '20
If you have to first launch the program when it matters, you loose regardless of programming language. If you compare C and warmed-up Java programs, then a Java program can be just as responsive as a C program.
-5
u/reqdk Oct 26 '20
On top of all that you mentioned, I wonder if the folks that talk about "memory unsafe code" of C and C++ are even proficient in the language or have any pride in their job beyond the level of catering to the lowest common denominator all the time. The C/C++ programmers I know would give you real judgemental looks if you mentioned "memory unsafe" as a reason not to use the language. Something something git gud something something stop hitting yourself.
6
u/MentallyWill Oct 26 '20
This comment shows that you've been fortunate enough in your career to only have worked with people who were good C/C++ developers. Take it from someone who is, to quote you, "proficient in the language" and takes "pride in their job", not everyone knows how to proficiently and safely use C/C++ and I'm pretty sure if you had to conduct code reviews for one of those people regularly you'd understand where the "memory unsafe" people are coming from.
Believe you me I wish it were as simple as telling them "something something git gud something something stop hitting yourself" but since that answer doesn't fly in a real company then until you invest the time training those people or play politics to get them fired maybe don't jump straight to assuming the "memory unsafe" people don't know what they're talking about?
0
u/reqdk Oct 27 '20 edited Oct 27 '20
I haven't. I've had more than enough devs in multiple languages that wrote bad code. I've let them go and fast. I've also been around for long enough to encounter my fair share of politics. I don't believe in solving HR problems with technical decisions. If I ran a game dev company tomorrow and made the decision to not use C++ because of potential memory unsafe code, I'd be running a real risk of being out of a job myself soon.
Every single project that I've been on that catered to the lowest common denominator like that out of "reddit wisdom" has been utter crap to work in. They've also all been web dev projects and software engineering is far more than web dev.
0
3
u/ivan0x32 Oct 27 '20
TL;DR: Because Azul ZingTM will solve all of your problems! We promise, I mean Azul promises, not we, who definitely did not just advertise Azul Zing.
2
2
u/frankreyes Oct 27 '20
I'd like to see how much more memory you need to allocate for Zing vs Hotspot in order to get those numbers. There's this paper [PDF] which claims that it takes 6x more memory for performance to be comparable to non-GC programs, as a function of the useful memory vs the reserved memory. Also, there is this long but interesting rant about the topic: Why mobile web apps are slow.
I haven't found any results yet but there's a hint here, which agrees with the previous research:
One of the benefits of Zing's heap management behavior is that larger Java heaps result in better GC efficiency (often much better than HotSpot), without the typical higher-pause-time downsides often found with other JVMs. As a result, we often recommend systematically starting high, then walking down to determine the proper heap size that provides sufficient headroom to allow improved CPU efficiency for Java applications.
https://docs.azul.com/zing/19.09.0.0/SysReqs_MemoryRequirements.htm
2
u/aelfric5578 Oct 26 '20
I've never worked on HFT or anything with such stringent latency requirements, but I wonder if they would be better off with something like GraalVM to get better performance.
12
u/pjmlp Oct 26 '20
What GraaVM offers in performance has been available in commercial JDKs since around 2000.
In fact as MaximeVM became GraalVM, it got several improvements out of J/Rockit.
Its great contributions is being a meta-circular JIT and offering Java developers free beer AOT that gcj never could achieve.
2
u/coder111 Oct 26 '20
Oh man, thanks for reminding me of GCJ- I'm still a bit sorry it died.
By the way, can GraalVM build Swing/AWT apps yet?
3
u/vips7L Oct 26 '20
Probably not. It would help with the startup but iirc the GC in substrate vm isn't as good as what's available in hotspot.
1
1
u/Fury9999 Oct 27 '20
Can anyone inform me of the drawbacks? I am wary anytime I see heaps of praise with no mention of tradeoff, limitations, drawbacks, etc. Surely everyone would use it if there were none? Thanks
4
u/cheeseman1957 Oct 27 '20 edited Oct 28 '20
If you’re asking about drawbacks for Azul’s Zing which the author seems to almost be selling in the article, there are a few that come to mind: it’s expensive (I just signed a purchase order for a renewal and you’re looking at a few thousand per year per machine). Not horribly expensive, but it can add up. Zing also does poorly with high throughput. We’ve found better performance with the hotspot jvm in some use cases where faster throughput is preferred even with the occasional small GC (as opposed to slower throughput without the occasional outlier). Source: also work at a HFT firm and use Java for much of the platform.
1
1
Oct 27 '20
In my admittedly limited experience, the biggest factor in GC time is is unnecessary assignment in functions and bloated class definitions. I've seen massive reduction in GC (and overall memory use) by simply refactoring functions to use lambda generators instead of constructing new classes and replacing assignment blocks with stream operations. Fewer assignments and smaller objects = less memory used and less frequent GC. I think there are also optimisation gains from using chained functions and lambda objects instead of custom classes as well, but don't quote me.
-4
u/dumdedums Oct 26 '20
They spend the beginning talking about how microseconds matter, and how people customize kernels for faster access to network cards, but doesn't adding the JVM kind of negate it all in a way? Like no matter which one you use, it would have a higher latency, defeating the point, right?
7
u/tetrarkanoid Oct 26 '20
Network latencies being orders of magnitude higher than CPU register/cache accesses, the raw execution speed of C/C++ might not matter enough to make a difference. Atleast, that's what the author seems to be making a case for. I don't know if I'm convinced.
4
u/sternone_2 Oct 26 '20
the tcpi/ip stack in linux is really shitty code and very slow
i have done speed test and even windows was faster
the first thing you do in hft is remove that part from the linux kernel
0
Oct 26 '20
[deleted]
5
u/cantstopthemoonlight Oct 26 '20
Further more Java can do runtime based optimizations that the c++ static optimizer can’t do. For example both optimizers and inline methods. But In Java if you have an if branch that checks for null and it is never null the jvm and remove the if check and essentially replace it with a try-catch exception.
2
56
u/[deleted] Oct 26 '20
Is this an add fort Azul Zing ?