5
u/joinr Sep 17 '20 edited Sep 18 '20
I went to reproduce stuff more deliberately, and decided to eliminate the effects of lein and other config issues. I also included u/bsless remarks about the parallel gc. My revised take is that the performance hit is about ~5% in this micro bench for slamming seqs without BiasedLocking (aligned with u/alexdmiller assumption regarding impact). The far more significant effect is as u/bsless hinted, ParallelGC indicates about a 2x performance benefit. Also, the expectation of JDK 15 being faster now holds when ParallelGC is used, regardless of BiasedLocking.
Edit: These numbers are based on a faulty assumption that LongRange is actually coercible via seq. These are not lazy seqs
| OS | java.runtime.version | java.vendor | args | Execution Time Mean |
|---|---|---|---|---|
| Windows 10 | 1.8.0_222-b10 | AdoptOpenJDK | 712.892795 µs | |
| Windows 10 | 15+36-1562 | Oracle Corporation | 1.303109 ms | |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseBiasedLocking | 1.247518 ms |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseParallelGC | 644.126121 µs |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseBiasedLocking -XX:+UseParallelGC | 636.434973 µs |
Edit new numbers with actual lazy seqs reflect a similar story, with minor changes:
| OS | java.runtime.version | java.vendor | args | Execution Time Mean |
|---|---|---|---|---|
| Windows 10 | 1.8.0_222-b10 | AdoptOpenJDK | 8.217622 ms | |
| Windows 10 | 15+36-1562 | Oracle Corporation | 15.559892 ms | |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseBiasedLocking | 15.606927 ms |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseParallelGC | 8.548820 ms |
| Windows 10 | 15+36-1562 | Oracle Corporation | -XX:+UseBiasedLocking -XX:+UseParallelGC | 8.619192 ms |
JDK 8 slightly dominates in the legit lazyseq numbers, and adding biased locking to JDK 15 incurs a tiny performance hit for some reason.
These were run in an "unclean" environment, e.g. browser open with multiple tabs (tends to muck with absolute performance of JVM in my experience), but they were consistently unclean :). I didn't mess with heap settings either. I think the comparison is decent; if not, folks are welcome to pipe up.
3
u/pron98 Sep 17 '20
Very happy to see that the impact is so small. I wonder if this might be due to the increased inlining in JDK 15 that might eliminate the lock altogether in some places in this benchmark.
2
u/alexdmiller Sep 17 '20 edited Sep 17 '20
This particular benchmark code doesn't actually use seqs so, while interesting, is probably not relevant to the original question. This range call produces a LongRange instance which can both act as a seq and a self-reducible coll. The seq call just returns the same object, and reduce is called on the LongRange and turned into a tight loop (time probably dominated by boxed long addition) - see https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LongRange.java#L218-L227.
Due to the self-reduce, I think the only objects getting created/destroyed in this code are the Longs. You maybe seeing GC initialization effects more than actual GC effects given how you're running this test. (Maybe also interesting re heap settings: https://kstefanj.github.io/2020/04/16/g1-ootb-performance.html)
Also keep in mind that the most common seq types (map, filter, etc) are allocated in 32 element chunks, and I believe no synchronization is required as you seq through each (immutable) chunk, so there's a lot less synchronizing in practice than you might think. (But that code is pretty twisty to follow.)
2
u/joinr Sep 18 '20 edited Sep 18 '20
Ah, I wish I had revisited the LongRange code; I was under impression there would be an explicit coercion via
seq, which is why I tried to avoid directly reducing it, since I knew it was internally reducible. I updated the benchmark to use an explicit lazy seq viatakeanditerateinstead ofrange. The results are more or less consistent, with some minor surprise in that re-enabling biased locking seems to incur a minor perf hit (like 0.3%) on JDK 15. Parallel GC still has about 2x performance impact, and now JDK 8 slightly dominates JDK 15 in general. Interesting. I edited the original post to u/pron98 to reflect the new numbers.
19
4
u/pron98 Sep 16 '20
It will be interesting to see the effect of this JEP (Disable and Deprecate Biased Locking) on Clojure performance. Might require changes in the Clojure implementation of some data structures.