Operating System Concepts: Kernels, Processes, and Scheduling

Why Operating Systems Exist

The earliest computers had no operating system. A program ran on the bare hardware, controlled the machine completely, and ran to completion before the next program could be loaded — often by physically swapping cards or tape. This wasted the most expensive resource in the building: the processor sat idle whenever the program waited for slow input or output.

The operating system emerged to solve a resource-sharing problem. Its core job is to act as an intermediary between programs and hardware, doing two things at once:

• Abstraction — hide the messy, device-specific details of the hardware behind clean, uniform interfaces. A program writes to “a file,” not to specific magnetic sectors on a specific disk model.
• Arbitration — share finite hardware (CPU, memory, devices) among multiple programs safely and fairly, so that no program can monopolize the machine or corrupt another.

Almost every concept below is an instance of one of these two goals.

The Kernel and Protected Mode

At the center of an operating system sits the kernel — the privileged core that has direct, complete control of the hardware. The crucial hardware feature that makes a kernel meaningful is the processor’s distinction between privileged (kernel) mode and user mode.

In user mode, certain dangerous instructions — those that touch hardware directly, alter memory mappings, or talk to devices — are simply forbidden; attempting one traps to the kernel. Ordinary programs run in user mode. Only the kernel runs in privileged mode. This hardware-enforced boundary is what prevents a buggy or malicious program from taking over the machine, and it is the foundation of all operating-system security.

A program that needs the kernel to do something on its behalf — open a file, allocate memory, send data over the network — makes a system call, a controlled, deliberate gateway from user mode into kernel mode. The system-call interface is the true API of an operating system; everything else is a library built on top of it.

Processes and Threads

A process is the operating system’s abstraction of a running program: the program’s code, its memory, its open files, and its execution state, bundled together and isolated from every other process. Process isolation is what lets a crash in one program leave the others standing.

A thread is a unit of execution within a process. A single process can have many threads sharing the same memory, allowing one program to do several things at once — a concept developed in depth in Concurrency and Parallelism. Threads are cheaper than processes but more dangerous, because they share memory and can therefore corrupt each other.

The kernel maintains, for every process, a control block recording everything needed to pause it and resume it later. The act of saving one process’s state and loading another’s is the context switch — the fundamental operation that creates the illusion of many programs running at once on a processor that can truly run only a few things at a time.

The Scheduler

With more runnable processes than processors, something must decide which gets the CPU and for how long. That is the scheduler, and its policy shapes how the whole system feels.

The pivotal technique is preemptive multitasking: a hardware timer interrupts the running process at regular intervals (a time slice or quantum), returning control to the scheduler, which may then switch to another process. Preemption means no single program can hog the CPU by refusing to yield — the central weakness of the earlier cooperative multitasking model, where a single misbehaving program could freeze the entire machine (a flaw that plagued early Mac OS and Windows).

Schedulers must balance competing goals that cannot all be maximized at once:

• Throughput — total work completed.
• Latency / responsiveness — how quickly an interactive task reacts.
• Fairness — every process gets a reasonable share.

Real schedulers, like Linux’s Completely Fair Scheduler, use elaborate heuristics and priority systems to juggle these trade-offs, favoring interactive tasks for responsiveness while preventing background work from starving entirely.

Virtual Memory

Perhaps the most elegant operating-system abstraction is virtual memory. Each process is given the illusion of a large, private, contiguous address space, starting at address zero, regardless of how much physical RAM exists or what other processes are doing.

This is achieved through the MMU (Memory Management Unit), hardware that translates the virtual addresses a program uses into the physical addresses of actual RAM, using page tables maintained by the kernel. Memory is managed in fixed-size chunks called pages. Virtual memory delivers several wins at once:

• Isolation — a process literally cannot name another process’s memory; the addresses don’t map there.
• Overcommitment via paging — pages not currently in use can be written out to disk (“swapped”), letting the system run programs whose combined memory exceeds physical RAM. The cost is the dreaded performance collapse of thrashing when a system pages too aggressively.
• Demand loading and sharing — pages can be loaded only when first touched, and read-only pages (like shared library code) can be mapped into many processes at once.

Virtual memory was pioneered on the Atlas computer at the University of Manchester (1962) and refined throughout the 1960s, notably in Multics (see The Multics Story).

Concurrency Inside the Kernel

Because many processes interleave and devices interrupt at unpredictable times, the kernel is itself a deeply concurrent program, and it pioneered the synchronization primitives the rest of computing later adopted. Edsger Dijkstra’s semaphore (1965) — a counter with atomic increment and decrement used to coordinate access to shared resources — was invented for exactly this problem (see Edsger Dijkstra and Structured Programming). Mutexes, condition variables, and deadlock — the situation where processes wait on each other in a cycle and none can proceed — are all covered in Concurrency and Parallelism.

Device Management and I/O

The operating system also abstracts the chaotic world of hardware devices. Device drivers — kernel modules that translate generic requests into the specific commands a particular device understands — let the rest of the system treat a thousand different disk and network models through a handful of uniform interfaces. The Unix doctrine that “everything is a file,” letting programs read and write devices using the same calls they use for files, is the most influential expression of this idea (see The Unix Story). Beneath this sits the interrupt: rather than wasting the CPU polling a device to ask if it is ready, the device signals the processor when it has data, and the kernel responds. Interrupt-driven I/O is what allows the CPU to do useful work instead of waiting on slow hardware.

Kernel Architectures: Monolithic vs. Microkernel

A central, enduring design debate concerns how much code should run in the privileged kernel.

• Monolithic kernels put everything — scheduler, memory manager, file systems, drivers — in one large privileged program. Fast, because components call each other directly, but a bug anywhere can crash the whole system. Linux is the great monolithic kernel.
• Microkernels keep only the bare minimum in privileged mode (scheduling, memory, inter-process communication) and push file systems, drivers, and services out into ordinary user-mode processes. More robust and modular — a crashed driver need not crash the kernel — but historically slower due to the overhead of constant message-passing between components.

This debate produced one of computing’s most famous public arguments, the 1992 Tanenbaum–Torvalds debate, in which Andrew Tanenbaum (author of the MINIX microkernel) declared Linux’s monolithic design “obsolete.” Linux won the market decisively; microkernel ideas survived in macOS’s Mach foundation (see The macOS Lineage), QNX, seL4, and embedded systems where verifiable robustness matters more than raw speed.

Dead End: The Pure Microkernel Performance Promise

The pure microkernel was, for a time, the academically favored future. The first-generation microkernels of the late 1980s and early 1990s — Mach being the prominent example — promised that the modularity and robustness of moving services to user space would come at acceptable cost. In practice the performance penalty of the constant inter-process communication was severe enough that the pure model lost the mainstream argument; even systems built on Mach, like macOS, fold large amounts of traditionally microkernel-external code back into the kernel for speed. Later “second-generation” designs (L4 and its descendants) cut the IPC overhead dramatically and rehabilitated the approach in high-assurance niches, but the sweeping promise that microkernels would replace monolithic systems on the desktop and server never arrived. The lesson is recurrent in systems design: architectural elegance does not automatically beat the brute performance of fewer boundaries to cross.

📚 Sources

• Silberschatz, Galvin, Gagne — Operating System Concepts
• Tanenbaum & Bos — Modern Operating Systems
• Dijkstra — “Cooperating Sequential Processes” (1965)
• The Tanenbaum–Torvalds Debate (1992 Usenet archive)
• Kilburn et al. — “One-Level Storage System” (Atlas virtual memory, 1962)