Today's objectives
How to prepare for this lecture
Whenever a process is run, the operating system allocates memory to it. All memory has an address, ranging from 0x0 to either 2^32 0xffffffff or 2^64 (twice as many fs), depending on if it is a 32-bit or 64-bit architecture, respectively.
The operating system provides the abstraction to the process that the process owns the entire memory space.
<aside> 🎓 Abstractions are an incredibly important part of computer science; it is one of our most important, often used, and effective solutions to solving complex problems, so if you are unfamiliar with the concept listen up! They are a simplified interface to a more complex implementation. Consider your web browser: when you visit a website like this one, the abstraction is that you are directly connected to the machine hosting the website, while in reality, your packets are being sent through dozens of intermediate machines (routers and switches), which might be dropping, reordering, or messing with the data—but your computer and your browser hide that complexity and present you with the simple abstraction "enter data on one end, and data comes out the other end." Abstractions hide details of lower-level problems so that we can simplify and focus on the higher-level problems. Without this, we wouldn't be able to tackle the big problems of computer science... but they can also bring about security vulnerabilities...
</aside>
To a running process, it appears that they own every address from 0x0 to 0xffff.... Of course, in reality, there are multiple processes running at the same time (how else are you going to be listening to your low-fi hip hop study music while reading this?). If both processes want to write to the same exact address, say 0xcf159ab0 on a 32-bit architecture, then do they overwrite each other's data? Not at all—this would not only be a massive headache for developers, but it would mean that any data in one process on a machine (e.g., some ad running in a browser) would be able to see or manipulate data in any other process on that machine (e.g., your banking app). Operating systems provide isolation between processes, which means that a program cannot directly access another process's memory (unless explicit permission is given, such as shared memory).
This is possible because the addresses presented to the processes are not the real addresses of the memory: what one process refers to as 0xcf159ab0 might in reality be the address 0xdeadbeef on the actual memory hardware; and the other process's 0xcf159ab0 might have the physical address d00d1351. The memory addresses that processes see are called virtual addresses, and the real addresses in the physical RAM are called physical addresses. Implementing "virtual memory" is one of the critical tasks of operating systems. They do this by maintaining lookup tables for each process that "translate" between that process's virtual addresses and the physical addresses.
To see how different an abstraction can be from reality, note that it is possible that two process's memory could be interleaved in physical memory: one process could have one "page" of memory (typically 4KB of data on 32-bit architectures), and another process could have the very next physical page. This all seems innocuous, but it has led to some severe attacks.
<aside> 😈 The RowHammer Attack. The RowHammer attacks shows that, in popular hardware designs of RAM, it is possible to flip bits in memory without actually accessing those bits in memory. RAM is implemented in adjacent "rows" of data; researchers found that by repeatedly accessing one row (even just by reading the memory) over and over in rapid succession, it is possible to cause bit-flips on nearby rows. Because OSes can interleave different processes' memory in physical memory, this means that a malicious process can alter another process's data by legitimately accessing its own data. This can be a very difficult attack to detect, and thus defenses are now focusing on trying to prevent the attack with different designs of RAM.
</aside>
When a process is started, the operating system allocates virtual memory, and breaks this memory space (from 0x0 to 0xffff... into several broad "segments" of memory. You can see the slides for a representation of all of the different regions of memory.
The main takeaways that I want to emphasize here is that:
We will focus primarily on the stack, but today, many of the most interesting memory attacks take place in the heap. Many of the same principles apply.
You learned in CMSC 216 what the layout of a stack is: how variables are represented on the stack, what happens when you call and return from functions, and so on. You may have thought that those details did not matter, because they are generally hidden behind layers of abstraction that your compiler provides. From the programmer's perspective, a local variable of a function just sort of.. exists when it needs to and then.. ceases to exist when it doesn't. That is the "abstraction" the compiler provides. In reality, the compiler is dealing with the details of allocating and deallocating memory, and keeping tracking of where variables reside at any point in time.
<aside> 🔥 Layers of abstraction hide details—which is usually a good thing!—but when programmers do not use an abstraction perfectly, it can sometimes introduce "attack vectors" that adversaries can use. We must assume attackers know the details behind an abstraction, and thus we must know them as best we can, as well.
</aside>
We will start off our semester by peeling away this layer of abstraction and revisiting good old stack layouts. In so doing, we will learn the danger they can provide—and how to protect against it.