r/programming • u/KingStannis2020 • Apr 14 '21
[RFC] Rust support for Linux Kernel
https://lkml.org/lkml/2021/4/14/102342
u/LambdaJon Apr 14 '21
Is there actually a means of catching such allocation failure in the Rust language? I realize this might be a different answer for KMD vs UMD, but Iāve often wondered the same when doing things like pushing to a vec or cloning something.
30
u/KingStannis2020 Apr 14 '21
As some of the other comments / emails mention, essentially the APIs to do so just need to be fleshed out. Most collections have
try_*()methods that return an error on allocation failure.12
u/The-Best-Taylor Apr 14 '21
It would be great if we could add a lint to disallow using the panicking versions. But I don't know the feasibility of that.
30
u/bloody-albatross Apr 14 '21
It would be great if we could add a lint to disallow using the panicking versions.
I imagine more something like a compiler switch where the panicking versions aren't even compiled. Something like
#[cfg(panic)]on panicking functions and exclude that configuration for the kernel or something.26
u/steveklabnik1 Apr 14 '21
That is what is being suggested in thread sorta kinda; basically there would be a config flag that would be set that would remove those APIs in the first place.
1
u/matthieum Apr 15 '21
Combined with static-analysis to ensure that all potentially panicking functions are flagged -- rather than rely on humans identifying them all faithfully, across refactorings -- it would be a great addition.
13
u/merlinsbeers Apr 15 '21
Or a version of Rust just for use in the kernel...
Kernel-rust... K-rust... Krust.
10
u/NihilistDandy Apr 15 '21
This will call for a new Krusty Krab logo. I hope that's not already a thing. š
1
3
u/LambdaJon Apr 15 '21
Ah, I think Iāve not seen the try_*() methods in example codes and such, but that makes a lot of sense. Itās nice to be able to have the choice between a version that panics vs explicit error, from my experience itās often context dependent which one you want.
8
u/myrrlyn Apr 15 '21
they're still unstable while the specifics of allocator fallibility get hashed out
89
u/Rusky Apr 14 '21
Rust-the-language doesn't say anything about allocation, basically the same way as C or C++. All the allocation failure policy comes from the standard library (specifically the
alloccrate), which it sounds like they don't plan to use in the kernel at all.At that point, it doesn't really matter if or how you catch allocation failure from things like
Vec::push, because kernel Rust code will be using kernel APIs for allocation instead. And since those signal failure with their return value, Rust code can handle allocation failure the same way as the rest of the kernel- though probably with some wrappers andResultto make it more idiomatic.7
u/Kered13 Apr 14 '21
I'm not very familiar with Rust, but can't panics be caught? So even a panicing allocator should be fine, just have something to catch it and convert it to an error code before it exits Rust code?
21
16
u/edzorg Apr 14 '21
So besides perhaps some syntactic familiarity for Rust programmers, what are the benefits we're getting?
59
u/steveklabnik1 Apr 14 '21
The "
## Goals" and "## Why Rust?" parts of the link should explain that.16
1
u/matthieum Apr 15 '21
I would note that fallible allocation APIs have been in the Rust project's plans for a while now.
As you mentioned they're strictly library issues, not language issues, but it doesn't mean people are not gunning for it.
For example, with regard to
Vec, the idea would be to have atry_pushmethod which can return aResultif no memory can be allocated and let the caller handle that.One possibly new demand would be to flag the possibly panicking APIs to catch their use at compile-time. I don't think I've seen this proposed seriously before, but I can definitely see the appeal. The cheap way would be to make a linter and maintain either a list of functions that don't panic or a list of functions that panic. More involved would be to use static analysis across the various libraries to identify automatically all those functions -- possibly combined with explicitly marking them in the source code so that only the Rust CI has to perform the analysis to check the manual annotations; everybody else can just rely on said annotations.
I'm pretty sure there's enough enthusiasm for seeing Rust in the kernel that the Rust community will be more than happy to help making Rust suitable.
5
u/flatfinger Apr 14 '21
Although the Standard defines a means by which C implementations can allow programs to catch and reliably recover from allocation failures, the standard library is too weak to allow reliably recovery in many execution environments. A proper robust allocation library should either support multiple independent heaps, or a means of reserving and sub-allocating storage, such that if a reservation request succeeds, any sub-allocation requests issued against the reservation whose total number and size don't exceed what was reserved would be guaranteed not to fail.
Having a library issue a panic when an allocation request fails is hardly worse than having the allocation request return a pointer that will appear valid but may or may not actually be usable.
255
u/journalctl Apr 14 '21
Even if Rust doesn't end up being accepted in Linux, I think the feedback the Linux community provides will be positive for other bare-metal Rust use cases.
8
59
u/koreth Apr 15 '21
This is a pretty positive development, and I say that as someone who thinks Rust is overhyped at the moment. Device driver code is exactly the kind of place where Rust's design tradeoffs are a good fit for the problem. ("Design tradeoffs" is not me disparaging Rust. All programming languages have design tradeoffs.)
36
u/shinyquagsire23 Apr 15 '21
I picked it up for a small hypervisor I've been writing and my general thoughts so far have been like,
MMIO has a ton of boilerplate, and I basically have to blanket unsafe on anything touching hardware, which, ehhhhh, I guessss
a lot of libraries require std, which is not portable at the moment, and no-std libraries are kinda fragmented
children structs not being able to reference their parents leads to weird patterns (ie, a USB endpoint belongs to a bus, but a borrowed endpoint cannot call parent functions, so the bus ends up having to have all endpoint functionality)
enums/traits/inheritance requires a lot of weird boilerplate, enough that I ended up writing a small code generator for syscalls
immutable by default vibes well with my Verilog mindset, I feel more confident that the compiler will be smart when I do intermediate expressions
I'd say tho that the bonuses have outweighed the headaches? Porting to Rust unearthed some subtle bugs in my C code with casting, and I would much rather an array index failure throw an assertion I can fix vs weird unstable behavior. I think Linux drivers in Rust will be great for standardizing a more embedded-oriented subset of Rust+crates where allocation can be less panic-oriented.
18
u/NotTheHead Apr 15 '21
I basically have to blanket unsafe on anything touching hardware, which, ehhhhh, I guessss
Yeah, that kinda makes sense to me, even if it's inconvenient.
5
u/BobHogan Apr 15 '21
Yea, rust's philosophy on that is that it can't guarantee that the hardware will work as expected, so interacting with the hardware is inherently unsafe. But unsafe in rust doesn't mean that the code is actually not safe to use, it just means that rust can't provide any of its safety guarantees on that code
1
u/crusoe Apr 15 '21
Yes, and you put the unsafe all there in well labeled places you can test and valgrind the hell out of.
2
u/i-can-sleep-for-days Apr 15 '21
But you should be testing all the code right? It's not like if it is safe you don't need to have good testing. It just handles the dumb cases like input is null etc, but lots of other errors you still need to test for (logical errors, general stupidity).
So it really should be, valgrind and test the crap out of your code, period.
2
u/jl2352 Apr 15 '21
But you should be testing all the code right?
In theory, of course you should. In practice you always end up having a trade off. You just don't have time to write a substantial number of test cases for every part. Sometimes that results in testing the same thing, multiple times, in multiple places. Sometimes that's a good idea, and sometimes it's just a waste of time.
It all depends on context.
5
u/smmalis37 Apr 15 '21
children structs not being able to reference their parents leads to weird patterns (ie, a USB endpoint belongs to a bus, but a borrowed endpoint cannot call parent functions, so the bus ends up having to have all endpoint functionality)
I'd need to see code to be sure, but there's probably something workable here. Whether its passing a ref to the parent into every method on the child or just a big refactoring. What I'm guessing you hit is a parent containing a child and a child containing a ref to the parent, which is essentially a self-referential struct, which yeah rust can't easily do yet (though there are some crates that can thanks to a little
unsafe)3
u/theXpanther Apr 15 '21
The main problem is that rust tries to prevent aliasing, this you can't have a mutable reference to a object and it's container at the same time
4
u/IceSentry Apr 15 '21
Out of curiosity, why couldn't you have used derive macros or macros in general to fix your boilerplate issue instead of having to write a code generator?
3
u/shinyquagsire23 Apr 15 '21
So I tried that but I guess the crux of it is like, I have 128 possible SVC async handler functions indexed by a u8, if I don't define an impl I want it to have a default, and I want the lookup to be O(1)/a jump table because SVCs get called a lot and context switches are already expensive. There's not really a good way I've seen to define an array of function ptrs based on which functions have an attribute macro unless Rust added like, some kinda deferred const array index setting or something? Because ultimately any macro will just spit out code in-place, can't store state between macros and output something later.
I think the other solution was to do an attribute macro which took the function and defined a const fn ptr in a specific linker section to kinda cheese a fake array of sorts? But then I'd need 128 sections in the linker script and a way to read the pointers out.
6
Apr 15 '21
Did you try to just write it normally and see what assembly gets spit out? I find that a lot of people optimize way too early without realizing the Rust compiler is actually pretty good at this. Iām not that familiar with your example but just from what youāre saying Iād define an enumeration for the cases that arenāt default handled, and then just match on it. I would expect that to be optimized down fairly well.
2
u/shinyquagsire23 Apr 15 '21
That's roughly what I ended up doing, I just also automated the matching and some struct defines because I didn't want to have to update 4+ spots for every handler I add. Which like, I'd say it's a fair enough thing to automate anyhow.
1
u/Plasma_000 Apr 15 '21
A lot of the more mature libraries require std by default but their std dependencies can be disabled through feature flags, so you might wanna check for that
1
u/ergzay Apr 16 '21
MMIO has a ton of boilerplate, and I basically have to blanket unsafe on anything touching hardware, which, ehhhhh, I guessss
What hardware are you touching? For a lot of hardware there's already libraries out there that wrap all that unsafe for you already.
2
u/shinyquagsire23 Apr 16 '21
Cortex-A57 on a Tegra X1 (Nintendo Switch), I saw a handful of Cortex-M libraries but mostly felt like I'd rather be 100% certain how I'm poking registers at least initially. The Cortex-M libraries seemed to be tied to their own interrupt handling stuff as well I think?
2
u/ergzay Apr 16 '21
I haven't looked at stuff for that processor myself, so can't help you too much.
4
u/B_M_Wilson Apr 15 '21
If this happens then I will finally learn Rust after holding out on C for so long
4
Apr 15 '21
What is rust?
18
u/duncanlock Apr 15 '21
Rust is a multi-paradigm programming language designed for performance and safety, especially safe concurrency. https://en.wikipedia.org/wiki/Rust_%28programming_language
2
u/obvious_apple Apr 15 '21
Does this mean if any mainstream rust module js used in the kernel from now on it can only be compiled in LLVM?
It seems hard because we just stepped away from the GCC monopoly.
19
u/leitimmel Apr 15 '21
There will be no forced Rust dependency for now. This is about adding support for Rust so people can write their own out-of-tree kernel modules.
2
u/ergzay Apr 16 '21
Pretty sure Linus said he wanted the kernel modules to be in-tree, just not enabled by default unless Rust is available.
2
u/Repulsive-Street-307 Apr 15 '21
This is a initiative by Google because they really really want to use rust in Android (presumably they're tired of being pwned by broken on purpose drivers and other shit). Sure eventually it may spread to the 'mothership'. But not before gcc has rust support is my guess. Or LLVM becomes 'the' blessed compiler, which would be a big political fight so it probably won't happen.
-115
u/OverjoyedBanana Apr 14 '21
Why not JavaScript ? I heard from friends that it's a popular language.
69
Apr 15 '21
I thought I'm the only person on Reddit who opens a topic, and just types some random words, so they can submit a comment.
Hello, friend.
→ More replies (2)10
15
-59
u/OverjoyedBanana Apr 14 '21
/s ? Are you all lacking the basic social cues parser here ?
57
u/JoJoJet- Apr 15 '21
Everyone can tell that you're being sarcastic. You're being downvoted because you're implying that rust is just a fad, and is only being considered because it's popular right now. You're ignoring everything that makes rust great, and all of the reasons why it's one of the few languages that could be used in the kernel alongside C.
→ More replies (2)-82
u/bumblebritches57 Apr 15 '21
literally nothing makes it great.
it is just a fad.
the borrow checker idea may have legs, but the language it's self is a dumpsterfire
12
u/DoktuhParadox Apr 15 '21
? Its syntax is modern and clean. I'm a Java developer, and writing Rust code is both easier and more fun in basically every situation. Just because industry props up shitty langs like Java doesn't mean we like them. "The borrow checker" isn't a feature or an "idea" they just added for funsies, it is a necessary consequence of ensuring ownership compliance at compile time.
14
Apr 15 '21
Wait, people hate Rust already? Or is the usual fringe fraction of every community that has this potent self-hatred for the thing they read and use every day?
→ More replies (25)17
u/futlapperl Apr 15 '21
It's weird with Rust. People who dislike it absolutely detest it and are very vocal about it, though they never seem to actually explain their reasoning. It's always just some comment about how it's a fad or how they don't like the syntax.
→ More replies (4)4
44
Apr 15 '21
[removed] ā view removed comment
→ More replies (18)24
u/hou32hou Apr 15 '21 edited Apr 15 '21
The part that Rust done right but Haskell did not is familiarity, the syntax of rust is so familiar to C-family language that itās much more easier to pick up than Haskell, although both of them shared similar concepts underneath (sum types, default immutability, trait/typeclass etc).
When I started learning Haskell, I spent way too much time parsing the code than actually understanding it (fortunately I managed to get pass it already), but when I learn Rust I just imported my JavaScript parser from my brain and it works most of the time, leaving me more time to actually understand the concept beneath (borrow checkers etc).
→ More replies (3)
-30
u/Dew_Cookie_3000 Apr 15 '21
Didn't he just tell them like three weeks ago we'll try it in drivers first and see how it works out? Where are the drivers they wrote and how did that work out? I knew they would do shit like this far too soon cos you give a bullshitter an inch and they take ten miles. Goddamn pushy obnoxious community of bullshitters.
Nice diplomatic Linus no good, bring back no filter Linus.
28
u/crusoe Apr 15 '21
Ugh drivers live in the kernel. What is announced now is no different than then. They're not taking commits for core kernel functionality in rust. They're allowing drivers to be written in it.
29
Apr 15 '21
new drinking game: take a shot every time this nitwit shows up in a /r/programming thread and finds some way to complain about Rust and/or it's community for no particular reason.
(don't actually play that drinking game you'll die from alcohol poisoning)
-18
u/Dew_Cookie_3000 Apr 15 '21
Not rust itself, its community. I don't have a problem with rust itself because I don't use it and don't need to use it, it's someone's else problem if they use it. It's just another language after all and no language is perfect. Its community though is pushy as hell and obnoxious, and they ruin this sub for me. It's not like I go to r/rust and pester people, I don't. This is a general programming sub and they need to respect that. Years ago there were similar problems here with haskell fanboys and before that it was ruby, and I shat on both till the hype died out and they behaved better. You don't see me shitting on haskell and ruby here now because there isn't a problem with either. Back then they used to come to java threads and shit on java and plug their language just like the rust fanboys now do with golang threads and others, that's if those threads even make it past being downvoted to zero in new submissions.
20
u/StillNoNumb Apr 15 '21
Honestly, I've been seeing way more people complaining about Rust community than Rust fanboys recently
4
u/DoktuhParadox Apr 15 '21
Me too. It's bizarre. Reminds me of the good old days where agleiva and his alt would troll each and every Java thread bitching about how C# was better. I've used Java for 10 years, and I agree. But so what? Complaining about a language isn't gonna stop others from using it when their whole codebase is already written...
5
u/StillNoNumb Apr 15 '21
Also, do you remember u/shevegen? Together they are this sub's legendary trolls haha
1
u/DoktuhParadox Apr 15 '21
I don't! What phenomenon is this?? At least I can read these comments, agleiva got suspended (and agleiv2 for ban evasion probably). Why do these people get so upset by seeing news about programming languages in a programming subreddit? I really don't like Go, but seeing it in the news doesn't make me angry. I truly don't understand.
3
u/Repulsive-Street-307 Apr 15 '21 edited Apr 15 '21
Probably has something with the rust main devs 'inclusiveness' and code of conduct. Ie: they don't allow griefers, fascists, trolls and other bad actors to ruin things for everyone in the name of 'free speech' and being 'anti-cancel culture'. Remember that we're living in a age of extreme propaganda. So you get a bunch of trolls targeting projects -and people- for various reason completely unrelated to the projects themselves (ie: Denuvan fork, Systemd haters, rust haters, etc, often obfuscating motives with 'it's not the philosophy of unix' and similar). In this case it's even completely irrelevant, because this is just google saying 'we want to use Rust in our fork and think it's a good idea, here are some proposed guidelines' and Linus going 'ok, to include your guidelines i'd have to mandate no panics and allocation failure by error return'.
Basically there is little chance of rust code getting upstream while gcc doesn't compile it.
16
Apr 15 '21
Its community though is pushy as hell and obnoxious, and they ruin this sub for me
hypocritical... let me just copy-paste some comments from your profile:
Great post. We need more of whatever this is and less Rust bullshit.
Never hire a rust douchebag. Never hire anyone with rust on their resume.
Great post. We need more HTTP posts and less Rust douchebags in this sub.
Great post. We need more Conway posts and less Rust spam in this sub.
Great submission. This sub needs more Go and less Rust.
Tldr: all other languages got it right. Rust got it wrong. Pathetic.
Now is a good time to ban rust pluggers from your website before they trash it like they trashed this subreddit
Lol twelve hours and zero vote score and zero comments. Nobody cares. Rust has peaked. Hype is dying. It is all downhill from here.
Rust community is full of bullshit and bullshitters
Use C# don't use Rust.
use Julia you don't use Rust
Use Zig don't use Rust.
This will expose all the rust millionaires.
Those were all from the first two pages. You're utterly obsessed and it just makes you look like, well not just look like, it straight-up makes you an asshole.
You can like and dislike whatever programming languages (or anything, really) you want, but that doesn't make it right to shit on people because they're enthusiastic about something that they like and you dislike.
4
u/DoktuhParadox Apr 15 '21
Its community though is pushy as hell and obnoxious, and they ruin this sub for me.
Rust is a programming language. It is becoming more popular. You don't like hearing about Rust because you're a sensitive weirdo? Fine, filter it out of your feed. You don't like this RFC? Fine, say why. But bitching about a language's community is such a waste of time. I can guarantee you and the other guy who's prowling this thread come off way more obnoxiously than any "Rust fanboy" could ever hope to, and I promise that Rust will continue to grow with or without you or /u/AntiRedditFa's incessant bitching in every single Rust thread.
-1
u/germandiago Apr 15 '21
I think Zig would be very good news but it is still far ahead because it is not 1.0 yet.
7
u/matthieum Apr 15 '21
This may be a tougher battle.
In general, advocating for another language is tough. This can be seen here; it's not good enough for the "new" language to be as good as the other! Given the additional hurdles of having multiple languages, the "new" language must significantly improve on the existing ones to be worth integrating.
Rust has an "easy" angle: safe Rust does not have Undefined Behavior. In code as critical as the kernel, that's a serious value proposition.
What would be the value proposition of Zig? If it's just "it's a wee bit cleaner", it's probably not going to cut it.
3
u/Repulsive-Street-307 Apr 15 '21
I'd be kind of seriously annoyed if 'rust has to have a way not to have any panic ever in kernel code' was the standard applied to Rust, and Zig was 'oh it's a better c, come on in!'.
1
u/germandiago Apr 15 '21 edited Apr 15 '21
For what I saw the error handling is quite better in Zig (as in impossible to ignore errors) and, without being Rust, it is quite safer to use correctly than C. Also, compile-time is quite better and it breathes a lot of the C simplicity of the sense of just being functions and structs. Also, it seems to be very easy to cross-compile without additional tooling, though I am not sure this is an advantage. Also checks can be enabled at run-time for things that cannot be detected at compile-time (of course, this has a performance hit). It has Integration with C libraries without FFI/bindings.
Anyway, here: https://ziglang.org/learn/overview/
The learning curve is also way lower for all people that currently use C, so it is a transition that would require less investment than using Rust.
3
u/CryZe92 Apr 15 '21
as in impossible to ignore errors
You can't really (accidentally) ignore errors in Rust either. How is this different?
1
u/germandiago Apr 15 '21
It is different because the learning curve is quite lower, because it is easy to compile, because it is faster to compile, because integration with C libraries is easier (can just call without wrapping).
Error handling might not be one of the differences then :D I just see Zig very compelling for a transition path from C, but I am not advocating for Zig or Rust. Obviously Rust is more ready-to-go right now. Zig is not even 1.0. But I would say it is the most promising language I have seen to replace C in a long time.
3
u/matthieum Apr 16 '21
I agree with everything you say.
However, having worked in the past in the integration of new languages in a company, my experience has been that the transition pain is more about organizational than technical issues, and given the pain, folks want a significant advantage out of it.
I do see Zig being used for libraries or small utilities that would traditionally be written in C. You don't feel the pain of lifetime handling as much in small projects, because the architecture fits in your head.
I am more dubious about a large C project choosing to move to Zig; even when mature. It solves a bunch of paper cuts, but leaves the greatest issues -- memory safety, type safety -- unaddressed, so given the pain of transitioning, if they do transition (and not all will choose to), I'd expect they'd look for something significantly better.
-42
Apr 15 '21
Well, I guess it's time to switch to OpenBSD
→ More replies (4)22
u/pure_x01 Apr 15 '21
Please do. I will stick with Linux but I love that there are choices so the more people who switch the better it is .
0
Apr 15 '21
[deleted]
19
u/_Sh3Rm4n Apr 15 '21
The linux kernel does not bootstrap the gcc compiler, so why should it bootstrap rustc?
200
u/KingStannis2020 Apr 14 '21
Linus' initial opinion: https://lkml.org/lkml/2021/4/14/1099