r/Clojure • • Sep 16 '20

JDK 15 Released

https://jdk.java.net/15/
28 Upvotes

20 comments sorted by

4

u/pron98 Sep 16 '20

It will be interesting to see the effect of this JEP (Disable and Deprecate Biased Locking) on Clojure performance. Might require changes in the Clojure implementation of some data structures.

2

u/alexdmiller Sep 16 '20

I am interested as well, but I don't have any reason to believe it will require any changes. The Clojure data structures were initially created before biased locking existed and I don't know of anything that relies specifically on those features.

1

u/pron98 Sep 16 '20

I was mostly thinking about the use of synchronized in LazySeq.

1

u/alexdmiller Sep 16 '20

Yeah, like I said, this usage pre-dates the existence of biased locking in Java (which was added around Java 6 iirc). My assumption would be that this is not a bottleneck in Clojure (where most seq access is uncontended) and thus you're unlikely to see any significant impact. But if someone sees otherwise, I would certainly like to hear about it!

2

u/pron98 Sep 16 '20 edited Sep 16 '20

Uncontended use (more precisely, single-threaded use) is precisely what biased locking was designed to help, and its disabling is likely to affect. I am not saying there's likely going to be a pronounced impact on Clojure, but uncontended synchronized is exactly what requires re-evaluation after this change, which will add a(n uncontended) CAS.

3

u/joinr Sep 16 '20
(ns biasedlockingtest.core
  (:require [criterium.core :as c]))


(defn seq-smash []
  (reduce + (seq (range 100000))))

openjdk version "1.8.0_222"

OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)

OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)

biasedlockingtest.core=> (c/quick-bench (seq-smash))
Evaluation count : 1038 in 6 samples of 173 calls.
Execution time mean : 595.515765 µs
Execution time std-deviation : 12.253500 µs
Execution time lower quantile : 586.219618 µs ( 2.5%)
Execution time upper quantile : 613.448159 µs (97.5%)
Overhead used : 1.774535 ns

openjdk version "15" 2020-09-15

OpenJDK Runtime Environment (build 15+36-1562)

OpenJDK 64-Bit Server VM (build 15+36-1562, mixed mode, sharing)

biasedlockingtest.core=> (c/quick-bench (seq-smash))
Evaluation count : 492 in 6 samples of 82 calls.
Execution time mean : 1.243192 ms
Execution time std-deviation : 35.091776 µs
Execution time lower quantile : 1.201389 ms ( 2.5%)
Execution time upper quantile : 1.274804 ms (97.5%)
Overhead used : 8.127900 ns
nil

Some overhead down in the micros. I don't know if this can be extrapolated to say "there's a 2x performance hit on seq traversal in JDK 15 without biased locking" or not though. I just tested a larger app and did not notice any regression though (maybe I'm not smashing seq's enough to matter).

1

u/pron98 Sep 16 '20

JDK 8 is practically stone-age technology, and too much of HotSpot has changed since to make any comparison useful. It would be best to compare 15 with biased locking vs. 15 without.

Anyway, a big performance impact for completely uncontended monitors is expected. The assumption is that such uses today are rare, and that uncontended CASes are overall cheap enough for this not to matter for real workloads. But because Clojure uses synchronized in a very common data structure is a matter of some concern, so some evaluation may be in order.

7

u/joinr Sep 16 '20 edited Sep 17 '20

JDK 8 is practically stone-age technology

There appear to be millions upon millions of cave people about for one reason or another.

It would be best to compare 15 with biased locking vs. 15 without.

-XX:+UseBiasedLocking

OpenJDK 64-Bit Server VM warning: Option UseBiasedLocking was deprecated in
version 15.0 and will likely be removed in a future release.

biasedlockingtest.core=> (c/quick-bench (seq-smash))
Evaluation count : 882 in 6 samples of 147 calls.
Execution time mean : 690.804281 µs
Execution time std-deviation : 11.418663 µs
Execution time lower quantile : 679.968000 µs ( 2.5%)
Execution time upper quantile : 707.770636 µs (97.5%)
Overhead used : 2.071022 ns
nil

edit the above benchmark may have been a goof still running on Java 8 (hence similar numbers). A better reproducible benchmarking method indicates this is likely erroneous, possibly a lame copy/paste error or misconfiguration of lein. Unbiased locking is more in the 1.2x ms range..

Evaluation count : 480 in 6 samples of 80 calls.
             Execution time mean : 1.284481 ms
    Execution time std-deviation : 41.078023 ╡s
   Execution time lower quantile : 1.232725 ms ( 2.5%)
   Execution time upper quantile : 1.331263 ms (97.5%)
                   Overhead used : 8.336353 ns

Stone age is somehow marginally faster; fascinating.

edit the java 8 version is using the ParallelGC vs G1GC by default. This ends up being a significant factor in performance

1

u/pron98 Sep 16 '20 edited Sep 16 '20

There appear to be millions upon millions of cave people about for one reason or another.

Hey, we still use hammers, don't we? :)

Stone age is somehow marginally faster; fascinating.

I wasn't trying to denigrate JDK 8, just to point out that it's very old and a lot has changed in OpenJDK since then, so to isolate causes there should be an apples-to-apples comparison. There are many reasons why some microbenchmarks would be faster on 8, and especially biased locking; in the years since, we've put a lot into the VM -- to the point that companies upgrading from 8 to 14 report very significant savings -- but we weren't optimising for this microbenchmark. I recall a famous comparison by someone about how computers from the eighties are faster than contemporary ones on some benchmarks; that doesn't mean they're more advanced, just that the focus is on other metrics.

Anyway, I think the effect of biased locking is clear, so now it's up to the Clojure implementers to evaluate whether or not this has any significant impact on real applications, and if so to consider changing the implementation.

2

u/joinr Sep 16 '20

I wasn't trying to denigrate JDK 8

I didn't catch that; I was remarking with Vulcan-like passion on the results. Based on other benchmarks with substantial VM jumps toward more modern releases, I would expect the opposite (typically you seem to get performance for free; almost like the early halcyon days of Moore's law, where stuff just ran faster every year).

It would definitely be worthwhile to have better data to make judgements from though.

→ More replies (0)

1

u/bsless Sep 16 '20

You should see extra speedup with -XX:+UseParallelGC, the default since Java 9 has been G1, while in 8 it's Parallel

2

u/alexdmiller Sep 17 '20

My point is really that seqs were designed before biased locking, and I do not recall seeing a huge boost for Clojure when biased locking was added (I'm sure it had benefit, I just don't remember anyone noticing it particularly). Without evidence to the contrary, I'm going to assume the removal of biased locking is probably also not something most people will notice. The JVM taketh with one hand and giveth with another.

Does the removal also include lock coarsening/elision? Not clear to me how those relate.

3

u/pron98 Sep 17 '20 edited Sep 17 '20

No, it shouldn't affect lock coarsening and elision, and, indeed, the reason I think the effect is small in the benchmark someone here did might well be due to those optimisations, aided by inlining, which has improved in JDK 15.

I merely meant to bring this to your attention out of an abundance of caution. Lock biasing was precisely intended as an optimisation in cases where synchronized methods/blocks experience single-threaded use. This should be quite rare in modern Java (language) programs, but I remembered that it might be common in Clojure. If it doesn't have a big effect that's good news.

5

u/joinr Sep 17 '20 edited Sep 18 '20

Better benchmarks

I went to reproduce stuff more deliberately, and decided to eliminate the effects of lein and other config issues. I also included u/bsless remarks about the parallel gc. My revised take is that the performance hit is about ~5% in this micro bench for slamming seqs without BiasedLocking (aligned with u/alexdmiller assumption regarding impact). The far more significant effect is as u/bsless hinted, ParallelGC indicates about a 2x performance benefit. Also, the expectation of JDK 15 being faster now holds when ParallelGC is used, regardless of BiasedLocking.

Edit: These numbers are based on a faulty assumption that LongRange is actually coercible via seq. These are not lazy seqs

OSjava.runtime.versionjava.vendorargsExecution Time Mean
Windows 101.8.0_222-b10AdoptOpenJDK712.892795 µs
Windows 1015+36-1562Oracle Corporation1.303109 ms
Windows 1015+36-1562Oracle Corporation-XX:+UseBiasedLocking1.247518 ms
Windows 1015+36-1562Oracle Corporation-XX:+UseParallelGC644.126121 µs
Windows 1015+36-1562Oracle Corporation-XX:+UseBiasedLocking -XX:+UseParallelGC636.434973 µs

Edit new numbers with actual lazy seqs reflect a similar story, with minor changes:

OSjava.runtime.versionjava.vendorargsExecution Time Mean
Windows 101.8.0_222-b10AdoptOpenJDK8.217622 ms
Windows 1015+36-1562Oracle Corporation15.559892 ms
Windows 1015+36-1562Oracle Corporation-XX:+UseBiasedLocking15.606927 ms
Windows 1015+36-1562Oracle Corporation-XX:+UseParallelGC8.548820 ms
Windows 1015+36-1562Oracle Corporation-XX:+UseBiasedLocking -XX:+UseParallelGC8.619192 ms

JDK 8 slightly dominates in the legit lazyseq numbers, and adding biased locking to JDK 15 incurs a tiny performance hit for some reason.

These were run in an "unclean" environment, e.g. browser open with multiple tabs (tends to muck with absolute performance of JVM in my experience), but they were consistently unclean :). I didn't mess with heap settings either. I think the comparison is decent; if not, folks are welcome to pipe up.

3

u/pron98 Sep 17 '20

Very happy to see that the impact is so small. I wonder if this might be due to the increased inlining in JDK 15 that might eliminate the lock altogether in some places in this benchmark.

2

u/alexdmiller Sep 17 '20 edited Sep 17 '20

This particular benchmark code doesn't actually use seqs so, while interesting, is probably not relevant to the original question. This range call produces a LongRange instance which can both act as a seq and a self-reducible coll. The seq call just returns the same object, and reduce is called on the LongRange and turned into a tight loop (time probably dominated by boxed long addition) - see https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LongRange.java#L218-L227.

Due to the self-reduce, I think the only objects getting created/destroyed in this code are the Longs. You maybe seeing GC initialization effects more than actual GC effects given how you're running this test. (Maybe also interesting re heap settings: https://kstefanj.github.io/2020/04/16/g1-ootb-performance.html)

Also keep in mind that the most common seq types (map, filter, etc) are allocated in 32 element chunks, and I believe no synchronization is required as you seq through each (immutable) chunk, so there's a lot less synchronizing in practice than you might think. (But that code is pretty twisty to follow.)

2

u/joinr Sep 18 '20 edited Sep 18 '20

Ah, I wish I had revisited the LongRange code; I was under impression there would be an explicit coercion via seq, which is why I tried to avoid directly reducing it, since I knew it was internally reducible. I updated the benchmark to use an explicit lazy seq via take and iterate instead of range. The results are more or less consistent, with some minor surprise in that re-enabling biased locking seems to incur a minor perf hit (like 0.3%) on JDK 15. Parallel GC still has about 2x performance impact, and now JDK 8 slightly dominates JDK 15 in general. Interesting. I edited the original post to u/pron98 to reflect the new numbers.

19

u/alexdmiller Sep 16 '20

Just FYI, Clojure works just fine with Java 15. :)

5

u/emaphis Sep 16 '20

Clojure is working fine with Java 16 EA so far.