Rust Compiler For Dummies

Setting Up the Stage:

As you may already know, all programs that are written are ultimately converted into binary instructions (those fancy zeros and ones). Why? Because the CPU cannot understand our source code, all it understands is 0s and 1s. Since we cannot write or interact with the memory by writing 0s and 1s, we have to create some abstraction for writing programs in a human readable fashion.

The thinnest abstraction layer for writing code to interact with memory is **Assembly**. It is a low-level language that helps us write human readable code, operates directly with memory, and provides fine-grained control over computer operations.
Although Assembly offers high efficiency and speed, it is not commonly used because it is more prone to memory issues and the complexity of writing code. So then how can we interact with memory safely?

The second level of abstraction provides us with **high-level programming languages** (like Rust, C, and Go). These languages are more human readable than assembly language, featuring syntactic sugar that hides much of the complexity and memory management from the programmer. With this, programmers can write programs with less complexity and fewer errors. Different languages handle memory allocation and deallocation in various ways, some manage it efficiently, while others do not, each with its own trade-offs. But wait a sec... haven't I just told you the CPU cannot understand anything other than 0s and 1s (binary)?

We cannot execute high-level programming language programs directly on the CPU, we must convert these syntactic sugar programs into binary code. This translation is done by **compilers**. We will focus on how Rust code is converted to binary code and what internal checks occur during compilation. We will peel layer by layer and understand each and every stage. With this in mind, let's get started 😄.

High-Level Overview of Compilation Layers

The code written by developers is human-readable, allowing others to easily read and understand it. However, a compiler cannot understand human-written code. We need to convert source code into binary code, which is the only format the CPU can understand.

Although there are several steps required to produce binary code, at a high level, these steps are divided into three main pillers: Frontend, Middle, and Backend. This breaks down the complex process of converting source code to machine code.

At the frontend, you have Rust code. At the backend, you have the binary machine code generated by LLVM (Low Level Virtual Machine) that runs directly on the target machine. In the middle, all the Rust-specific ownership and borrowing checks happen.

We will peel back each layer and understand how Rust compilation works. If we zoom in a little on the Rust compilation three pillers, this is what we get as you can see in the image below.

I don't expect you to know all the terms shown in the picture above, but don't worry by the end of this blog, you will understand all of them. Let's go step by step, peeling back each layer to understand what happens while compiling our code.

Keeping this big picture in mind, let's start peeling the Orange 🍊.

Layer One: Lexing, Parsing, and AST

Let's take an example as source code:

fn main() {
    // Let's do some investigation :)
    let some = String::from("chinna");
    println!("Say my name:");
    println!("{}", some);

    time_pass(&some);
}

fn time_pass(pass: &String) {
    println!("Time passing with this guy: {}", pass);
}

This step is the first one where compilation starts. The compiler first reads the .rs file as plain text, then breaks down this linear text into Tokens like fn, some, {—this is called Lexing.

Then the compiler converts these tokens into a tree-like structure called AST (Abstract Syntax Tree), and this AST still resembles the source code a lot, but it is in a tree-like structure. This is known as Parsing. You can see the AST version of the code example that we have taken here. Due to this, all the macros get expanded in this layer.

The AST captures all the syntactic code into a tree-like structure. You may ask, why do we need to do this?

Well, compilers cannot understand this linear source code directly. The source code is sugar-coated syntax designed for human readability, not for compilers. The Abstract Syntax Tree (AST) abstracts away certain details, it is a tree data structure that best represents the syntactic structure of the source code. You can learn more about AST on this Wikipedia page.

Layer Two: AST Lowering (HIR and THIR)

Well, after parsing the tokens and converting them into an AST, the next layer begins. At this point, the AST closely resembles the source code, which still contains a lot of syntactic sugar, such as for and match.

We need to peel away this syntactic sugar to simplify the AST. The result of this desugaring process is a form of the AST known as HIR (High-Level Intermediate Representation). HIR is still close to what the user originally wrote, but it removes syntactic sugar for example, converting a for loop into a loop with iteration logic. After removing all the fluff, HIR is now a more compiler-friendly abstraction representation of the AST.

This process of simplifying or transforming the AST by removing syntactic sugar is known as lowering, and by the way, you can check the HIR representation of the AST by running this command: rustc +nightly -Z unpretty=hir-tree src/main.rs.

By lowering the HIR further down and checking whether all types of the code are used correctly or not, like for example you cannot add an integer with a string, and of course you can do shit like this in JavaScript 🤡.

After doing all the checks for types now THIR(Typed High-Level Intermediate Representation) is represented in form AST. As the name might suggest, the THIR is a lowered version of the HIR where all the types have been filled in, which is possible after type checking has completed.

After lowering HIR to THIR and before MIR unsafety checker will walks through THIR representation and it checks every expression for unsafe operations (like raw pointer dereferences, calls to unsafe functions, static mut accesses, union field accesses) and verifies these only appear inside an unsafe context (unsafe blocks or functions).

Because unsafeck needs typed expressions, it’s placed after type checking and THIR construction but before MIR building. This positioning allows it to enforce Rust’s safety guarantees early and precisely, ensuring all unsafe code is properly enclosed in explicit unsafe contexts.

Layer Three: Middle Intermediate Representation (MIR)

After lowering, the HIR becomes a compiler-friendly abstraction of the AST. From here, the next layer of abstraction begins this is the heart of the Rust compiler. MIR (Middle Intermediate Representation) is the phase where many classic memory bugs (like race conditions and use-after-free errors) can be detected. If such bugs are found, the compiler will simply throw an error.

MIR represents your code as a Control Flow Graph (CFG). Think of this as a detailed flowchart. Every if, loop, and match is broken down into basic blocks and explicit "go-to" jumps between them.

To guarantee safety, the compiler can't just check the "happy path." It must analyze every possible path your code could take. What happens if this if is true? What if it's false? What if this loop runs zero times?

The CFG makes all these paths explicit. The borrow checker can systematically walk this flowchart, tracking the state of every variable (owned, borrowed, moved) through every possible branch and loop, ensuring no rule is ever violated. You can learn more about CFG at here and for in order to see the MIR version of our code example you can check here

Well long story short, MIR provides the structure for the compiler to exhaustively check every possible path and guarantee that Rust's safety invariants are met. This system is incredibly robust and is the foundation of Rust's safety.

The unsafe keyword creates a critical exception. It is a promise from the programmer that they will uphold Rust's safety rules manually. If that promise is broken, the compiler's foundational assumptions are invalidated. This can compromise the behavior of the entire program, no matter how "safe" the surrounding code appears.

Ultimately, a Rust program is considered sound if it is free from undefined behavior. Safe code that passes the compiler's checks achieves this soundness automatically.

Layer Four: LLVM Code Generation

We are getting into the final stages of compilation. After performing all the Rust borrow and ownership checks during the MIR phase, the compiler applies optimizations at the MIR level, such as removing dead code and simplifying control flow.

MIR is then translated into LLVM IR (Low Level Virtual Machine Intermediate Representation), a platform-independent intermediate representation used by the LLVM backend. LLVM IR is comparable to assembly, but it is a bit more high-level and human-readable.

The code-generation phase of the Rust compiler is mainly done by LLVM (Low Level Virtual Machine). LLVM is a collection of tools for building compilers, most notably used by the C/C++ compiler Clang. Finally, after all rigorous optimizations, LLVM IR is translated into machine code for the target platform (e.g., x86_64 or ARM64).Consequently, binaries compiled for one architecture cannot run on another without emulation or translation, which means you cannot run x86_64 binary code on an ARM64 processor or vice-versa.

References:

rust_compiler_video_by_daniel
stack_overflow_discussion
llvm_video
stack_overflow_discussion_of_llvm rust_book_on_undefined_behavior