r/java • • Mar 21 '21

Java lambdas can deadlock parallel class initialization

https://www.youtube.com/watch?v=_2hZf1kw4A4
230 Upvotes

46 comments sorted by

View all comments

Show parent comments

38

u/[deleted] Mar 21 '21

Whilst interesting; who would do such a thing? Having a heavy static class initialiser using a parallel stream like that.. ouch.

35

u/AndrewHaley13 Mar 21 '21

Well, this kind of edge case really needs to be fixed. It's far from obvious that lambda streams shouldn't be used in static initializers.

73

u/murkaje Mar 21 '21

There's nothing to fix as it's documented behavior. What needs to be improved is educating newer java developers that static initializer is not some nice place to dump code, but a critical part of class loading and initialization flow and that before classes are fully initialized many restrictions are placed on them and the performance is poor.

The topic keeps being brought up, most recent case was around a performance regression: https://cl4es.github.io/2019/02/21/Cljinit-Woes.html

So keep static fields for objects you know and won't be doing heavy computation like initializing empty collections or ones with few elements. Most of the time the initialization doesn't need to happen before the first use of said member, so better put it in some lazy-init container or create one. Perhaps Lazy Static Final Fields will bring something simpler to use.

9

u/[deleted] Mar 21 '21

Your reply is great.

3

u/sadfsdffsdafsdfsdf Mar 22 '21

There's nothing to fix as it's documented behavior.

While I believe you, could you however be so kind and point to that documentation? I would be really curious about that.

1

u/slaymaker1907 Mar 22 '21

I disagree that this should be seen as acceptable because while it would make sense for there to be performance problems, this seems like a correctness issue.

8

u/DasBrain Mar 21 '21

¯\(ツ)/¯

JDK-8215634

Happens if you fix bugs so OpenJDK actually implements the spec.
Also see apangin's great answer on Stack Overflow.

7

u/[deleted] Mar 21 '21

It’s obvious that you don’t put any kind of heavy computation into a static initialiser. Anything that uses parallel stream would be by definition heavy.

What would be your proposed fix?

23

u/[deleted] Mar 21 '21

You can't judge how heavy a computation is by looking at the code in your static initializer. You might just do "new Foo()" and THAT inside uses parallel streams for some reason.

If they wanted to restrict calling arbitrary code... they should've restricted calling arbitrary code.

11

u/sweetno Mar 21 '21

Well, when you program, you don't just slap constructor and method invocations here and there. More often than not you get a rough idea what are the runtime characteristics of the used APIs: memory usage and algorithm complexity. Say, Java doesn't have a single best-for-all Set implementation, you need to pick the one that suits your case.

2

u/Muoniurn Mar 22 '21

I didn’t test it, but I believe that there is no problem with calling another class that creates threads. The problem here is about calling the same class’s static methods from another thread from the static initializer. Which you can’t do by a trivial new Foo(), only by this already red flag raising line in the video. (You can fall into this trap by new Foo() that awaits a lambda though, I believe . But I would yet again question that code)

Nonetheless, a compiler warning would be appreciated if it can prove in this specific case that a deadlock can arise.

-6

u/[deleted] Mar 21 '21

If you put parallel in your stream for a non-heavy computation, wtf are you doing.

Its clearly an abuse here. Sure they've found an interesting edge case. But it makes sense, the class isn't initialised until the static fields are done. Java's model is providing a strong guarantee that this cannot overlap by two threads, ergo using a parallel stream is a dumb idea.

Now the lambda's abstraction is broken by this, but then to fix, you'd need to put non-capturing lambda's somewhere else.

2

u/[deleted] Mar 21 '21

You didn't get the point I was making. When you call a method, you don't always know what it does. Are YOU PERSONALLY to blame for every problem caused by code you call?

9

u/[deleted] Mar 21 '21

You don’t always, but you should know enough about it before calling. Even so, calling complex code in a class initialisation is just bad.

This code is literally someone who knew how to creat a deadlock saying hey it’s a deadlock. Interesting because it’s hidden, but practically not something you’d do.

2

u/Roachmeister Mar 21 '21

This code is literally someone who knew how to creat a deadlock saying hey it’s a deadlock.

No it isn't. He specifically said in the video that he was stumped for several hours trying to figure out why it wasn't running.

→ More replies (5)

8

u/r0estir0bbe Mar 21 '21

Why not? That seems like a valid use case to me, e.g. for computing specific constants - not great, but not terrible?

6

u/[deleted] Mar 21 '21

If they are constant why are you computing them?

Even if you need to compute them once don’t do it in an initialiser like that. This is why streams can be bad, a simple line initialisation is actually a bigger computation; and just because you can write it on a line doesn’t mean you should.

There’s other ways to achieve a similar result.

5

u/john16384 Mar 21 '21

A constant can be a huge lookup table.

11

u/[deleted] Mar 21 '21

Then that would be a result of a calculation and therefore would be calculated.

There’s a difference between initialising a class and doing a tonne of calculations.

0

u/john16384 Mar 22 '21

You asked why a constant may require calculation. I gave an example where this can be the case. Just because a constant needs sone calculation doesn't make it any less a constant.

There are examples in compression and encryption where these constants can't even be pre-computed and loaded from say a database, but we don't even have to look further than Java's Pattern#compile method for constants that are almost always static and require some significant calculation.

I am not saying though that multiple threads should be allowed, but I can't help but wonder how much stuff would break if Pattern#compile did this.

3

u/[deleted] Mar 22 '21

Your mixing terminology though. A physical constant is often only calculated once, because it never changes. Sounds like your taking about invariants, which are constant over a certain portion of calculations.

Really though, you shouldn’t do such things in a static initialiser, because that’s not where large calculations should go, and they are part of a class initialisation phase lower than user level.

I wonder if the obsession with immutable classes is why many don’t see an issue with this.

1

u/sweetno Mar 21 '21

Huge lookup tables with constants are better be read from a file/DB. If reading/computing them is long, the program should ensure feedback for the user: logging info messages/displaying them in the GUI. Users don't like when programs consume system resources without visible effects. Static initializers are not good places to implement this sort of feedback.

1

u/DuncanIdahos9thGhola Mar 21 '21

The issue is that when you use the old way of Thread and Runnable you get a inner class. The lambda makes a static inner class. Different byte code - so to the lesson I guess is don't assume these are equivalent - even though IDEs usually give you that option to convert from one form to the other.

6

u/[deleted] Mar 21 '21

I think the lesson is to not do something weird with a class initialiser like that. Basically this is saying please refer to me before I’m in initialised.

An inner class would be illegal since it’s a static context.

It’s an interesting result, but that’s about it.

→ More replies (1)
→ More replies (3)