r/java • u/daviddel • Sep 23 '22
Virtual Threads: New Foundations for High-Scale Java Applications - Brian Goetz
https://www.infoq.com/articles/java-virtual-threads/22
9
9
u/ReasonableClick5403 Sep 24 '22
If you are interested in Java, I have found that nearly any talk by Goetz is worth watching. Great to see him keeping it going.
37
u/Joram2 Sep 23 '22
Great write up by Goetz and great work Java team. Java really leapfrogged every other mainstream language here.
Java's virtual threads are much better than async/await which is what C#, Kotlin, and also JavaScript/TypeScript/Node use (Goetz didn't mention JavaScript).
Java's virtual threads are much better than Reactive/async/Futures like what Scala has.
Golang (and Erlang) had virtual threads much earlier, but the Structured Concurrency API, and the Extent Local Variable API is nicer than what Golang offers.
20
u/RandomName8 Sep 23 '22
Scala gets this just as much as Java. Let's not mistake what is getting Loom, it's the JVM, not the Java language.
That said, while Futures are instantly replaced by fibers, reactive-stream libraries are not, as they provide so much more that is too damn hard to manually do, even with fibers.
5
u/_INTER_ Sep 23 '22
All the better, it benefits all JVM languages. Truely a marvelous feat by the engineering team.
Additionally Akka changed license recently. So I hope more movement away from the reactive style which I welcome.
5
u/RandomName8 Sep 23 '22
So I hope more movement away from the reactive style which I welcome.
I, too, prefer what I like better đ.
Joking aside, for me it's not much about what style I like, but about not having unbounded processes with no backpressure support. Fibres (in any language) still make backpressure and bounding a process a silent, implicit contract. Hard to ensure across a codebase and easy to break. The one good thing about reactive-libraries is that they throw this in your face in a way that you can't "just forget" about it.
Handling limited resources with fibers is not going to be any easier than with futures. I fear it's even going to make the problem worse because it's going to encourage a mutable and imperative approach towards gatekeeping these resources, which in turn will lead to subtle bugs and hard to reason about systems.
Of course, in the end, time will tell if it ends up being this way or not. What I can say for sure, is that having truly native fibers enable certain things that before were just not doable, and that's great!
1
u/emaphis Sep 24 '22
You can abate your fears. While Java is adding virtual threads, it's also adding immutable algebraic data types through project Amber.
2
u/Joram2 Sep 23 '22
I presume only the JVM version of Scala can use Loom but scala-native (https://scala-native.org/) and scala-js (https://www.scala-js.org/) can't use Loom. Similarly with Kotlin, the JVM Kotlin gets Loom, but the native and Javascript backends do not.
The Kotlin devs advocate their async/await stuff as a big feature of the language. The Scala devs advocated their Reactive async future library as a better alternative to async/await.
while Futures are instantly replaced by fibers...
The new Project Loom Structured Concurrency API still uses simple Futures. The more elaborate reactive future libraries, like Completable Future and Scala's reactive future libraries are generally what you wouldn't use if switching to virtual threads.
reactive-stream libraries are not, as they provide so much more that is too damn hard to manually do, even with fibers.
Things like Akka, RxJava, and Vert.x, and Helidon? All of those have functionality that is useful even with virtual threads, but without a major virtual thread centric redesign, like what Heilidon is doing with Nima, I suspect most new projects will stop using those frameworks.
3
u/RandomName8 Sep 24 '22
I presume only the JVM version of Scala can use Loom but scala-native
Sure, but then again scala is 99.999% the JVM: up until scala 3 scala.js wasn't even supported by the scala team, it was all a community "plugin" if you will. Scala-native is still in that position (and probably will be forever).
The Kotlin devs advocate their async/await stuff as a big feature of the language. The Scala devs advocated their Reactive async future library as a better alternative to async/await.
Very different stories to me. You can say "The kotlin devs" since it's easy to see that that's JetBrains, but who are "The scala devs"? EPFL (Odersky's team)? they don't advertise reactive libraries, just async using Futures (I just checked scala's site to confirm, didn't see anything obvious other than mentions to Futures).
Similarly with Kotlin, the JVM Kotlin gets Loom, but the native and Javascript backends do not.
Since neither Kotlin nor Scala control the jvm, they do intend to ship their own "custom runtime" features in the platforms that they design from the ground up (fibers are on the plate for both). But again it's widely different when talking about a product, like Kotlin, and community and committee driven project, like Scala (with all the goods and bads both approaches have).
The new Project Loom Structured Concurrency API still uses simple Futures. The more elaborate reactive future libraries, like Completable Future and Scala's reactive future libraries are generally what you wouldn't use if switching to virtual threads.
This sentence confuses me, it seems you're saying one thing and then the opposite. I think we are saying the same thing, not sure.
Things like Akka, RxJava, and Vert.x, and Helidon? All of those have functionality that is useful even with virtual threads, but without a major virtual thread centric redesign, like what Heilidon is doing with Nima, I suspect most new projects will stop using those frameworks.
Fair. I don't know what's going to happen but I don't think those libraries need a major redesign. Honestly the one big thing they stand to gain from fibers is stacktraces without all the heroics they do today. In any case, no library that's about concurrency and parallelism will ignore Loom for sure, they'll adapt.
1
u/Joram2 Sep 24 '22
but who are "The scala devs"? EPFL (Odersky's team)? they don't advertise reactive libraries, just async using Futures (I just checked scala's site to confirm, didn't see anything obvious other than mentions to Futures).
I was thinking of this, where the official Scala site explored adding async+await, and choose to go with a future library instead: https://docs.scala-lang.org/sips/async.html
This sentence confuses me, it seems you're saying one thing and then the opposite.
The new Structured Concurrency API recommended for virtual threads uses java.util.concurrent.Future. Look at the code examples in the official JEP:
I don't think those libraries need a major redesign
They definitely do. Large parts of Vert.x and Akka are designed around non-blocking async.
19
u/mahmoudimus Sep 23 '22
Also the way it was rolled out in a backwards compatible way is really an excellent feat of engineering. Kudos all around to the Java architects and leaders working on this to get it across the line. Some exciting things happening in the JVM space lately!
1
u/ReasonableClick5403 Sep 24 '22
Extent Local Variable
is this meant as a replacement for ThreadLocal?
I am also wondering about the implications of Loom for MDC/log context propagation.
1
u/Joram2 Sep 24 '22
is this meant as a replacement for ThreadLocal?
Extent Local Variables are intended to replace most uses of ThreadLocal, but not all cases. Extent Local Variables are not tied to a thread, they are scoped to a block of code and all functions called from that block.
Golang uses Context for this purpose.
1
u/ReasonableClick5403 Sep 24 '22
Go's Context are completely manual and must be explicitly passed as a normal value, so I wouldn't necessarily equal the two mechanisms. If a library you use in go does not pass a context, there is absolutely no way to ensure context propagation.
1
u/Joram2 Sep 24 '22
That's true. But Go's Context is serving the types of scenarios that Extent Local Variables are intended to serve. Like adding login information to an http request.
1
u/Bright_Ad_2431 Sep 24 '22
Kotlin is not async/await based, it uses coroutines, which are similar to goroutines, but they also enforce Structured Concurrency
1
u/Joram2 Sep 25 '22
https://kotlinlang.org/docs/composing-suspending-functions.html#structured-concurrency-with-async
Kotlin uses the async + await in their examples. I knew Kotlin has their version of Structured Concurrency.
1
u/Bright_Ad_2431 Sep 25 '22
from the link you just posted:
"This programming style with async functions is provided here only for illustration, because it is a popular style in other programming languages. Using this style with Kotlin coroutines is strongly discouraged for the reasons explained below."
I can't see how Loom is better than Kotlin's coroutine, they look very much alike from the dev perspective
1
u/Joram2 Sep 25 '22
The "Structured concurrency with asyncï»ż" section that I linked is the recommended piece. That does use async/await keywords. The "Async-style functions" section is the one that is "strongly discouraged".
I just watched a Roman Elizarov talk on Structured Concurrency. The Kotlin devs are super smart. A coroutine is basically just another word for a virtual thread. Kotlin has had both structured concurrency and virtual threads (they call them coroutines) for years.
The downsides of Kotlin are you have to declare some functions suspend functions, you have to remember to call them with runBlocking rather than just calling them like normal functions, and you have to be careful not to other JVM functions like Thread.sleep that block the OS thread.
With virtual threads, all the JVM standard libraries are updated so that things like locks, sleep, file + network IO block virtual threads, not OS threads. Java virtual threads don't require suspend functions vs regular functions.
1
u/Bright_Ad_2431 Sep 25 '22
Yep, exactly. The good thing is that project Loom will also be beneficial to Kotlin, so their coroutines will probably change a bit, like getting rid of suspend and runBlocking as they don't make sense anymore.
1
u/Joram2 Sep 26 '22
I imagine new JVM-only Kotlin apps would want to use JDK virtual threads and the JDK Structured Concurrency API and not a Kotlin specific API.
Kotlin probably wants a concurrency API that supports the Native + JavaScript backends as well as the JVM one. Also, Kotlin will need to support existing versions of Java for a while that don't have virtual threads. And even Java 19, this stuff is still in preview/incubator mode, it will take several more JDK releases for everything to be final.
12
u/nimtiazm Sep 23 '22
One of the advantages of pooling threads is to circumvent the âexpensive creationâ part like jdbc connections et al. How does creating a virtualthread-per-task with a semaphore address that?
27
u/Joram2 Sep 23 '22
If an application benefitted from pooling jdbc connections with OS threads, I presume switching to virtual threads doesn't impact that, and the application would continue to benefit by pooling jdbc connections.
9
u/TheCountRushmore Sep 23 '22
Right.
While creating the virtual thread is cheap. The networking round trip and setup to the DB server is not.
23
u/taftster Sep 23 '22
Virtual threads arenât expensive. You donât need to pool them.
8
u/nimtiazm Sep 23 '22
Yeah thatâs clear. Creating an expensive connection in a new virtual thread every single time is the question.
8
u/kiteboarderni Sep 23 '22
They are not saying that. You would still use a connection pool like hikari to execute the query for example from the virtual thread. No need to create the connection there.
3
u/hippydipster Sep 23 '22
One advantage is you could write your pool perhaps more simply, and just make some blocking methods to get a connection. With virtual threads, you can just go ahead and create a virtual thread and call the blocking call and wait for it to return with a connection. You're' not holding up a real thread.
2
u/taftster Sep 23 '22
Iâm following your question now. But others have answered decently.
Virtual threads wonât help the mechanics of setting up connections to a remote database (as in your example). Like the literal TCP socket wonât benefit necessarily.
So connection pools will still be needed. But an underlying thread pool and/or asynchronous socket manager wonât be. When a client is matched with a connection object, the thread that itâs operating on might be a virtual thread under the hood and might have been newly created at the time itâs asked for.
1
8
u/lbalazscs Sep 23 '22
If you are asking how to use the Semaphore class, here is a tutorial: https://www.baeldung.com/java-semaphore
If you're asking how performance/scalability will increase if you can only use a few database connections, well, it won't necessarily increase. Virtual threads are useful if you can take advantage of Little's Law (by creating many of them), and you may not be able to do that in a simple CRUD application. The question is how many independent I/O tasks you have.
4
u/mauganra_it Sep 23 '22
You don't even really need semaphores. Connection pools already act as limiters.
Things change if there is something like pgbouncer between the application and the database. That way, connection setup should become very cheap, and virtual threads become more effective.
10
u/kiteboarderni Sep 24 '22 edited Sep 24 '22
I wonder how many millions of $ of public cloud computing costs will reduce from the work of /u/pron98 when people can use far less resources for their scalability concerns by using loom :)
3
u/sureshg Sep 24 '22
Yes, if they can do the same for Project Leyden (Creating static image without much configuration), jvm would be the best platform for cloud and serverless.
2
Sep 24 '22
You used an "r" instead of a "u". I got a 404 and I thought he deleted his Reddit from people pestering him with pings too much lol.
1
0
Sep 26 '22
[deleted]
1
u/kiteboarderni Sep 26 '22
Funny I thought we were in the java subreddit.
1
Sep 26 '22
Funny, I thought developers can be language agnostic and choose the right tool for the right job
1
u/kiteboarderni Sep 26 '22
True we should just build everything in node!
1
Sep 26 '22
Well if cloud cost is your primary concern rust and go will improve your cost efficiency in that regard way more than green threads.
1
u/kiteboarderni Sep 26 '22
Ya people should just throw away their millions of lines of java code and move to rust instead of switching to newer jdks I suppose.
1
3
u/dolle Sep 24 '22
One thing that is still not clear to me: can the user control the thread pool that the virtual threads get mounted on, and how the virtual threads are scheduled? Or is there just a default, fixed thread pool that all virtual threads use?
When debugging concurrent code that doesn't have timing behavior, it would be nice if one could fix the thread pool to a single thread to get deterministic behavior of the virtual threads, for example. If you can further control the scheduling of virtual threads onto the single thread, then you could do randomized testing of concurrent code which would give a seed that could be used to reproduce the exact interleaving that made the test fail.
3
u/lbalazscs Sep 24 '22
The JDK's virtual thread scheduler is a work-stealing ForkJoinPool that operates in FIFO mode. The parallelism of the scheduler is the number of platform threads available for the purpose of scheduling virtual threads. By default it is equal to the number of available processors, but it can be tuned with the system property jdk.virtualThreadScheduler.parallelism. Note that this ForkJoinPool is distinct from the common pool which is used, for example, in the implementation of parallel streams, and which operates in LIFO mode.
1
2
u/Void_mgn Sep 26 '22
From use of async await in js I am very happy this direction was chosen for java.
1
u/Illustriously_Naive Sep 24 '22
As someone who has 0 knowledge on threads and concurrency, how does one get started so that they can understand this?
5
Sep 24 '22
JEP Cafe videos on the Java YouTube channel.
Virtual threads: https://www.youtube.com/watch?v=lKSSBvRDmTg
Virtual threads 2: https://www.youtube.com/watch?v=UVoGE0GZZPI
Structured concurrency: https://www.youtube.com/watch?v=2nOj8MKHvmw1
1
u/vips7L Sep 24 '22
Java Concurrency in Practice if you want a better standing around concurrency.
1
1
1
15
u/oparisy Sep 23 '22 edited Sep 25 '22
Loom at last! đ€©