In this blog, we will try to understand Windows OS architecture.To perform debugging operations, you must understand the binaries/executables/processes. To understand binaries/executables/processes, you must understand OS architecture.
Unlike *nix, Windows OS is very complex piece of software. This is by the fact that Microsoft developed various, different components (such as COM+, OLE, PowerShell etc.) communicating with each other via WinAPIs and similar stuff. Considering this, covering each and everything in one single blog post is not a good idea.
To keep this blogpost concise, we will understand basic component (such as Process, thread etc.). Then we will see the big picture. Other Microsoft technologies (such as .NET, Azure etc.) won’t be covered in this blogpost. As name suggests, this blog will be all theory. It may become boring, but please, bear with me!!!
Now to explain all these things in details, I’ll be using multiple resources. I’ll put all those in reference section in the end.
As scope is clear, we are ready to dive into Windows OS architecture.
Building blocks of Windows OS:
- Process: Process represents running instance of executable. Process manages/owns couple of things listed below.
- Executable binary: It contains initial code and data to execute code within process.
- Private virtual address space: Used as allocating memory for code within process on requirement basis.
- Primary token: This object stores multiple details regarding default security context of process and used as reference by threads within process.
- Private handle table: Table used for managing executive objects such as events, semaphores and files.
- Thread(s) for process execution
Processes are identified by unique process id. This ID remains constant till respective kernel process object exists. Now its important to note that every running instance of same binary will get separate process and process id.
All above can be summed up in next diagram.
- Virtual Memory: OS provides separate virtual, private, linear address space to every process. This space is used by processes to allocate memory to executables, threads, DLLs etc. Size of this address space depends upon how many bits process supports i.e. 32 or 64bit. 2(Bit) can provide supported total available VAS.
For 32bit processes on 32bit Windows, it is 4GB (2GB system processes + 2 GB user processes). This distribution can vary up to 1GB System process + 3GB for user process, if process is marked with LARGEADDRESSAWARE linker flag in header while compiling program in Visual Studio. For 32bit processes on 64bit Windows, limit goes up to 4GB user process, if process is marked with LARGEADDRESSAWARE linker flag in header while compiling program in Visual Studio. For 64bit processes, on Windows 10 64bit (our target system), it is 128TB system process + 128TB user process.
Now the reason to call memory scheme Virtual because the virtual address is not directly related physical address (i.e. actual address location in RAM and/or Storage). Virtual to physical mapping is done by memory manager. We will explore this in much detail in next blogpost.
- Threads: Threads execute code. As we have
seen earlier, thread is an object under process, and it use resources available
to process to perform tasks. Thread owns following information:
- Current access mode (User or Kernel)
- Execution context e.g. State of registers, page status etc.
- One or two stacks, for local variables allocation and call management
- Thread Local Storage array, data structure for thread data
- Base and current priority
- Processor affinity
Threads has three states as below:
- Running: Currently executing code on processor
- Ready: Waiting for availability of busy or unavailable processors for execution
- Waiting: Waiting for some event to occur (e.g. Processor trap). After the event, thread changes state to Ready.
We will explore this in next blogpost with practical example.
- System Calls: System call is piece of software shared between various subsystems. WinAPI and NTAPI contains such system calls. We will explore this in much detail in next blog with practical example. This will be important as we need this to look closely in programming tutorials.
Generalized OS Architecture:
Even though above diagram is shown as Windows Architecture, this is just speculated and simplified architecture. This is because the fact that Microsoft never revealed actual architecture. But don’t worry, above diagram is from Windows Internals book, which official Microsoft publication. Let’s try to understand each component one by one.
- Environment subsystem and subsystem DLLs: Environment subsystem is the interface between applications and some Windows executive system services. Subsystem does this with help of DLLs. Example of Environment subsystem is Windows subsystem, which has kernel32.dll, advapi32.dll, user32.dll etc. implementing Windows API functions. We will explore Windows subsystem in depth in future blog posts.
- NTDLL.dll: ntdll.dll is special system support library primarily for use of subsystem DLLs and native applications. This provide two types of functions.
- System service dispatch stubs to Windows executive system services: These functions provide interface to user mode system services. Example NtCreateFile function available in Notepad. Many of these functions for accessible via WinAPI. We will explore this in next blogpost with practical example.
- Internal support functions used by subsystems, subsystem DLLs, and other native images
- Windows Executive Layer: This is the only second lowest layer, sitting on top of Kernel (NTOSKRNL.exe). It has following components.
- Configuration manager: Implements and manages system registry.
- Process manager: Creates, terminates processes and threads. It deals with Windows Kernel to perform these activities.
- Security Reference Manager: Enforces security policies on local computer. It guards OS resources, performing runtime object protection and auditing.
- I/O manager: Implements device independent I/O. Responsible for dispatching to the appropriate device drivers for further processing.
- Plug-n-Play manager: Determines which driver is appropriate for device and loading it.
- Power manager: Power manager, with help of Processor power management and power management framework, co-ordinates power events and generate power management I/O notifications to device drivers.
- Windows Driver Model (WDM) Windows Management Instrumentation (WMI) routines: These routines enable device drivers to publish performance and configuration information. It also receives commands from user mode WMI service.
- Memory manager: Implements virtual memory. It also provides support to underlying cache manager. We will explore this in detail, during future blogposts.
- Async LPC facility: It passes messages between client process and server process on same computer. It is also used as local transport for RPC.
- Run-time library functions: This include string processing, arithmetic operations, data type conversion, and security structure processing.
- Executive support routines: Includes system memory allocation (paged and non-paged pool), interlocked memory access, special types of synchronization mechanisms.
- Kernel: Kernel provides some fundamental mechanisms such as thread-scheduling and sync services, used by Windows executive layer, and low-level hardware architecture dependent support such as interrupt and exception dispatching. Kernel is coded in C + some instructions in assembly. To keep things simple at low level, Kernel implements simple “kernel objects”, which in turn supports executive objects. For example, Control objects define how OS functions must be controlled. Dispatcher objects define synchronization capabilities of thread scheduling. We will (and must) explore Kernel in depth while looking into driver dev blog.
I have not covered each and everything in this post and cannot cover entire architecture in this humble blogpost. But I believe that I have covered enough details to get basic understanding of Windows architecture to move ahead with this journey. In next blogpost, we will use tools such as WinDBG and Sysinternal tools to dive into Windows architecture. Till then, Auf Wiedersehen!!!!