Before the questions, make sure you can: explain why an OS needs hardware support; distinguish hardware interrupts, traps (software interrupts) and exceptions, and describe the steps of interrupt handling, including priorities and nesting; compare programmed I/O, interrupt-driven I/O and DMA and compute the CPU overhead of each; explain device controllers and device drivers; explain dual-mode operation, the mode bit and privileged instructions; describe memory protection with base and limit registers and CPU protection with a timer; explain how a system call works step by step, name common Linux system calls (fork, execve, wait, exit, open, read, write, close, kill) and compute the saving from buffering; describe the boot sequence; and compare monolithic, layered, microkernel, hybrid and unikernel architectures and choose one for a given device.
An operating system cannot protect programs from each other, or itself from programs, using software alone: a buggy program could simply overwrite the kernel or never give the CPU back. The hardware provides the tools — interrupts, a privileged mode, memory-protection registers and a timer — and the OS builds everything on top of them. Every time an application wants something only the OS may do (read a file, send a packet, start a process), it must ask through a system call, a controlled door into the kernel.
Interrupts: how the world gets the CPU's attention
An interrupt is a signal that makes the CPU stop what it is doing, save its state, and run a special routine — an interrupt handler (interrupt service routine, ISR). Three sources:
| Kind | Cause | Example |
|---|---|---|
| Hardware interrupt | A device signals an event | Network card: packet arrived; disk: transfer done; timer tick |
| Trap (software interrupt) | A program deliberately asks the OS for service | A system call instruction (syscall on x86-64) |
| Exception (fault) | An error or special condition during an instruction | Division by zero, invalid memory access, page fault |
Handling an interrupt, step by step:
- The device raises an interrupt request (IRQ); the CPU finishes the current instruction.
- The CPU saves the program counter and status (on the kernel stack) and switches to kernel mode.
- It uses the interrupt number to find the handler's address in the interrupt vector table.
- The handler runs (usually briefly — longer work is deferred to later kernel threads).
- The saved state is restored and the interrupted program continues, never knowing it was paused.
Priorities and nesting. Interrupts have priorities. A high-priority interrupt (e.g., a power failure or a timer) may interrupt a lower-priority handler (nested interrupts); lower-priority ones wait until higher ones finish. This is preemptive priority scheduling in miniature — the same idea you will meet for processes in Chapter 4.
You are writing an essay (a program). The doorbell rings (hardware interrupt): you put a bookmark in (save state), answer the door (handler), then go back to the exact sentence. If the fire alarm goes off while you are at the door (higher priority), you deal with the fire first.
Three ways to do I/O
Devices are much slower than the CPU. How does the CPU move data to and from them?
- Programmed I/O (polling): the CPU repeatedly checks the device's status register and moves each piece of data itself. Simple, but the CPU is busy the whole time.
- Interrupt-driven I/O: the CPU starts the operation and does other work; the device interrupts when it is ready for the next piece. Good for slow devices (keyboard), but costly for fast ones — one interrupt per small piece.
- Direct memory access (DMA): a DMA controller moves whole blocks between the device and memory by itself and interrupts the CPU once per block.
Worked example. A 1 GHz CPU (10⁹ cycles per second) reads from a device at 4 MB/s (4,000,000 bytes/s).
- Polling, 4 bytes per poll, 400 cycles per poll: 1,000,000 polls/s × 400 = 400 million cycles/s → 40% of the CPU.
- Interrupt-driven, one interrupt per 4 bytes, 500 cycles per interrupt: 1,000,000 × 500 = 500 million cycles/s → 50% — worse than polling for this fast device!
- DMA, one interrupt per 4 KB block (4,096 bytes), 1,000 cycles to set up + 500 to handle the interrupt: 976.6 blocks/s × 1,500 = 1.46 million cycles/s → 0.15%.
For a keyboard (about 100 events/s), even polling costs only 100 × 400 = 40,000 cycles/s = 0.004% — but the CPU would have to wake up to check it constantly. Rule of thumb: interrupts for slow devices, DMA for fast ones.
Device controllers and drivers. Each device has a hardware controller with registers (status, command, data). The OS talks to it through a device driver — kernel code that knows that controller's registers and protocol, and offers a uniform interface (read, write, control) to the rest of the kernel. Drivers are the largest part of most kernels and the most common source of crashes.
Protection needs hardware
Dual-mode operation. The CPU has (at least) two modes, selected by a mode bit:
- Kernel mode (supervisor, privileged): all instructions allowed.
- User mode: privileged instructions are forbidden — for example, halting the CPU, changing the mode bit, accessing I/O ports directly, modifying memory-protection registers, disabling interrupts, or setting the timer.
If a user program tries a privileged instruction, the hardware raises an exception and the OS takes over (usually terminating the program). The only legal way from user mode to kernel mode is an interrupt, a trap (system call) or an exception — all of which jump to code chosen by the OS.
Memory protection. Each process may access only its own memory. The simplest hardware support is a pair of registers: base (first legal address) and limit (size). For every memory access in user mode the hardware checks base ≤ address < base + limit; otherwise → exception. Example: base = 30000, limit = 12000: addresses 30000–41999 are legal; 42000 traps. (Chapter 9 generalizes this with paging.)
CPU protection. A program could loop forever and never give the CPU back. The OS sets a hardware timer before giving the CPU to a program; when it expires, a timer interrupt returns control to the OS, which may switch to another program. Setting the timer is privileged. Timer interrupts are cheap: at 1,000 ticks/s and 2 µs per tick, the cost is 0.2% of the CPU (0.05% at 250 ticks/s).
System calls: the door into the kernel
A system call is the programming interface to OS services. What happens when a C or Java program writes to a file on Linux:
- The program calls a library function (
writein the C library, orFileOutputStream.writein Java, which eventually calls it). - The library puts the system-call number (on x86-64 Linux,
writeis 1) and the arguments in registers and executes thesyscallinstruction — a trap. - The CPU switches to kernel mode and jumps to the kernel's system-call entry point.
- The kernel looks up the number in the system-call table, checks the arguments (Is this file descriptor open? Is this buffer really in the caller's memory?) and runs the service.
- The result (or an error code such as
-EBADF) is returned, and the CPU switches back to user mode.
| Category | Linux examples |
|---|---|
| Process control | fork, execve, exit, wait4, kill (send a signal), getpid |
| File management | open, read, write, close, lseek, unlink |
| Device management | ioctl, read, write (devices look like files) |
| Information | getpid, uname, time |
| Communication | pipe, socket, send, recv, mmap (shared memory) |
| Protection | chmod, chown, setuid |
So the answer to "which system call sends a signal to a process on Linux?" is kill (the shell command kill is a user program that calls it).
System calls are expensive. A mode switch, argument checks and cache effects make a system call typically hundreds of times more expensive than an ordinary function call. Buffering reduces the number of calls. Example: writing 1,000,000 bytes, assuming 1 µs per system call plus 1 ns per byte copied:
| Buffer size | System calls | Time |
|---|---|---|
| 1 byte (unbuffered) | 1,000,000 | 1,001.0 ms |
| 512 bytes | 1,954 | 2.954 ms |
| 4,096 bytes | 245 | 1.245 ms |
| 65,536 bytes | 16 | 1.016 ms |
That is why BufferedWriter in Java and stdio in C exist: a 4 KB buffer makes this program about 800 times faster.
Booting
- Power on → firmware (BIOS or, today, UEFI) tests the hardware (POST) and finds a boot device.
- The boot loader (e.g., GRUB) loads the kernel image into memory.
- The kernel initializes memory management, drivers and the scheduler, mounts the root file system.
- It starts the first user process (PID 1,
systemdorinit), which starts all services and the login screen.
OS architectures
| Architecture | Idea | Pros | Cons | Examples |
|---|---|---|---|---|
| Monolithic | The whole OS (scheduler, memory, file systems, drivers) runs in kernel mode as one program | Fast (plain function calls inside the kernel) | A bug in any driver can crash everything; large | Linux, traditional UNIX (Linux adds loadable modules) |
| Layered | Each layer uses only the layer below | Clear structure, easier to verify | Hard to define layers; overhead | THE system (Dijkstra), parts of many OSs |
| Microkernel | Kernel keeps only the minimum (scheduling, memory, message passing); drivers and file systems run as user-mode servers | Reliable (a crashed driver can be restarted), secure | Message passing adds overhead | QNX (cars), MINIX 3, seL4 (formally verified) |
| Hybrid | Microkernel-like structure, but many services in kernel mode for speed | Compromise | Complexity | Windows NT family, macOS (XNU) |
| Exokernel / unikernel | Minimal kernel; the application is linked with only the OS parts it needs | Tiny, fast, small attack surface | Specialized; one application per image | Research and specialized cloud appliances |
| Virtual machine | A hypervisor gives each OS the illusion of its own hardware | Isolation, consolidation | Overhead (smaller with hardware support) | Cloud servers (Chapter 12) |
Microkernels pay a cost for message passing, but where reliability and safety matter more than peak speed, they win: QNX runs in cars and industrial controllers, and seL4 has a mathematical proof of correctness. The right architecture depends on the device's risks.
Modern Linux can run small, verified programs inside the kernel with eBPF — used by cloud companies for networking, security monitoring and observability without writing kernel modules. And tools such as strace (which system calls does my program make?) and perf (where does time go?) are everyday debugging tools built on the mechanisms in this chapter.
Key takeaways
- Interrupts (hardware), traps (system calls) and exceptions (faults) switch the CPU to kernel mode through the interrupt vector table; priorities allow nesting.
- I/O: polling wastes CPU, interrupts suit slow devices, DMA suits fast ones (one interrupt per block) — compute overhead as events/s × cycles/event ÷ CPU frequency.
- Dual mode + privileged instructions + base/limit registers + timer give the OS control and protection.
- A system call traps into the kernel with a number and arguments; Linux examples:
fork,execve,wait4,exit,open,read,write,close,kill. System calls are costly — buffer I/O. - Boot: firmware → boot loader → kernel → PID 1.
- Architectures: monolithic (fast), microkernel (reliable), hybrid, layered, unikernel, VM — choose by performance vs. reliability vs. size.
Ready? Close the notes and practise.
30 questions. Predict the output before you check — that is the skill the exam measures.