r/Compilers • • Jul 27 '22

A bit confuse

I would like to build a compiler but I feel confused. Writing a compiler is general for every cpu architecture or assembly language?

For example, I want to build a compiler for Arm, will it work for x64? Do I need to know every assembly language?

Thank you

2 Upvotes

16 comments sorted by

23

u/WittyStick Jul 27 '22 edited Jul 27 '22

Compiler writers will generally write their compiler to target an architecture-agnostic IR (intermediate representation), which then maps to each architecture separately. The amount of architecture-dependent code is then limited to the parts which deal with the IR lowering, and the higher-level parts of the compiler don't need to be concerned with architectural specifics.

LLVM for example, is a popular IR which many compiler writers target. It has already done the work of emitting architecture specific machine code, and has many optimizations built into it. A compiler writer who targets LLVM will not need to know much about any specific architecture.

Many compiler writers will just emit C code, and have an existing C compiler do the work of compiling to machine code. This is handy because there are already optimized C compilers for virtually every architecture already, and a large number of programmers are already familiar with it, which isn't necessarily the case for LLVM and other IRs, but LLVM is gradually becoming the norm to use for compiled languages.

8

u/elprophet Jul 27 '22

I ask this every few months, and every few months the answer is "The LLVM IR has too much churn", but, is there a resource that would be good for an upper level collegiate compilers course using LLVM IR as the target? And what about resources for common compilers topics using the LLVM IR like register coloring, optimization passes, etc?

5

u/fernando_quintao Jul 28 '22

Hi u/elprophet. We have put together an online course about LLVM here. I use LLVM in two project assignments in UFMG's static analysis course (syllabus here). The LLVM online course focus on the clang front end and on the opt middle end. We don't talk about generating a backend. The course was using LLVM 10.0, so some of the techniques in the videos might not work with LLVM 15.0.

2

u/WittyStick Jul 28 '22 edited Jul 28 '22

I'm not the person to ask and I don't really use LLVM. I followed the Kaleidescope tutorial (in OCaml) some time ago, and found it reasonable to use. No idea about today as the project has grown a lot since then.

My primary focus is dynamically-typed languages and interpretation, and I don't think LLVM is the right tool for the job, but if I were developing a statically typed language I would probably go for LLVM.

3

u/hermeticwalrus Jul 28 '22

I did the kaleidoscope tutorial (in c++) just under a year ago, and it got me up and running in LLVM pretty well

1

u/davew_haverford_edu Jul 31 '22

The expectation that you can benefit from the optimizations of a pre-existing C compiler is not universally realized when you compile to C. The C compiler will, of course, make its effort to optimize the code you've given it. But, sometimes compiler-generated C ends up being so obfuscated that the compiler can't be sure it's legal to optimize something that might have been evident in handwritten code (such as cases where you can easily determine that a vector instruction is legal).

9

u/[deleted] Jul 27 '22

Yep, this is why a lot of people will target something like LLVM (https://llvm.org/) or another language (like transpiling to C) so they can leverage those systems to handle platform targeting.

2

u/mamcx Jul 27 '22

A compiler NOT need to target a "CPU". You can make a compiler that generates HTML, for example.


In your case, even LLVM could be too complicated. The EASIEST thing you can do is compile to a language that is already done (ie: a "transpiler").

So, you can make a compiler in Rust that generate Rust. Or in Rust that generates C#, and so on.


If wanna be "close to the metal" but not too complicated to do, compiling to WASM (web assembly) could be more than good enough, and pretty efficient.

1

u/[deleted] Jul 28 '22

So knowing x64 assembly could I build a compiler for arm assembly?

3

u/FUZxxl Jul 28 '22

Sure, arm assembly is easy to pick up once you are familiar with how CPUs work in general.

2

u/mamcx Jul 28 '22

Yes. Your input -> output can be selected as you wish (how easy it will vary!)

You can check a sample on how to do it from scratch (in this case, using Forth that is trivial to bootstrap from assembly):

https://github.com/nornagon/jonesforth/blob/master/jonesforth.S

----

If wanna do a compiler. Check less prone to errors than that, I recommend instead that you use (as targets):

- Zig (instead of C)

- Web Assembly

- Your own bytecode

If wanna do compiler. Check https://craftinginterpreters.com for a great overview of the whole thing.

5

u/[deleted] Jul 28 '22

> Do I need to know every assembly language?

Yes, if the compilation goes all the way from source code to the native code of the CPU. Then you will need to understand all the CPUs you want to target, which is probably not going to be that many (eg. I only target one at a time).

You will also need to consider the target platform: x64 code will be somewhat different between Windows and Linux for example.

But you can also write a compiler which generates intermediate code, which is portable across targets. For example, by generating C code (unless you are planning to write a C compiler, then that doesn't really move you forward).

4

u/casino_r0yale Jul 28 '22

A compiler is any program that transforms one language to another. A Markdown compiler for example can be a program that creates HTML from Markdown source.

If you’re specifically targeting a particular CPU architecture, no your binaries for ARM will not work on x64. However, many languages that aim to be multi platform compile to an intermediate representation (one example being LLVM) that then has compilers for many target architectures.

2

u/[deleted] Jul 28 '22

Hi,

A compiler for ARM won't provide a code that works on x64 architecture. They have different architectures right. Think about Linux. They have their versions for ARM, x86, etc...

I would advise you to take a look at this book: "Compilers: Principles, Techniques, and Tools"

1

u/umlcat Jul 27 '22

Each hardware platform or architecture requires it's own code Generation, yet if you want to implement a compiler for different architectures there's parts that are the same, and the final modules are different.