x86 Disassembly/Print Version

From Wikibooks, the open-content textbooks collection

< X86 Disassembly

Jump to: navigation, search

This is the print version of X86 Disassembly

If you print this page, choose "print preview" in your browser, or click printable version, you will see the page without this notice, without navigational elements to the left or top, and without the TOC boxes on each page.
Refresh this page to see the latest changes.
For more information about the print version, including how to create a truly printable PDF File, see Wikibooks:Print versions.

The Wikibook of

x86 Disassembly

Using C and Assembly Language

From Wikibooks: The Free Library

Introduction

x86 Disassembly

What Is This Book About?

This book is about the disassembly of x86 machine code into human-readable assembly, and the decompilation of x86 assembly code into human-readable C or C++ source code. Some topics covered will be common to all computer architectures, not just x86-compatible machines.

What Will This Book Cover?

This book is going to look in-depth at the disassembly and decompilation of x86 machine code and assembly code. We are going to look at the way programs are made using assemblers and compilers, and examine the way that assembly code is made from C or C++ source code. Using this knowledge, we will try to reverse the process. By examining common structures, such as data and control structures, we can find patterns that enable us to disassemble and decompile programs quickly.

Who Is This Book For?

This book is for readers at the undergraduate level with experience programming in x86 Assembly and C or C++. This book is not designed to teach assembly language programming, C or C++ programming, or compiler/assembler theory.

What Are The Prerequisites?

The reader should have a thorough understanding of x86 Assembly, C Programming, and possibly C++ Programming. This book is intended to increase the reader's understanding of the relationship between x86 machine code, x86 Assembly Language, and the C Programming Language. If you are not too familar with these topics, you may want to reread some of the above-mentioned books before continuing.

What is Disassembly?

Computer programs are written originally in a human readable code form, such as assembly language or a high-level language. These programs are then compiled into a binary format called machine code. This binary format is not directly readable or understandable by humans. Many programs, such as proprietary commercial programs, or very old legacy programs may not have the source code available to you.

Programs frequently perform tasks that need to be duplicated, or need to be made to interact with other programs. Without the source code and without adequate documentation, these tasks can be difficult to accomplish. This book outlines tools and techniques for attempting to convert the raw machine code of an executable file into equivalent code in assembly language and the high-level languages C and C++. With the high-level code to perform a particular task, several things become possible:

Programs can be ported to new computer platforms, by compiling the source code in a different environment.
The algorithm used by a program can be determined. This allows other programs to make use of the same algorithm, or for updated versions of a program to be rewritten without needing to track down old copies of the source code.
Security holes and vulnerabilities can be identified and patched by users without needing access to the original source code.
New interfaces can be implemented for old programs. New components can be built on top of old components to speed development time and reduce the need to rewrite large volumes of code.

Disassembling code has a large number of practical uses. One of the positive side effects of it is that the reader will gain a better understanding of the relation between machine code, assembly language, and high-level languages. Having a good knowledge of these topics will help programmers to produce code that is more efficient and more secure.

Tools

Assemblers and Compilers

x86 Disassembly

Assemblers

Wikipedia has related information at assembler.

Assemblers are significantly simpler than compilers, and are often implemented to simply translate the assembly code to binary machine code via one-to-one correspondence. Assemblers rarely optimize beyond choosing the shortest form of an instruction or filling delay slots.

Because assembly is such a simple process, disassembly can often be just as simple. Assembly instructions and machine code words have a one-to-one correspondence, so each machine code word will exactly map to one assembly instruction. However, disassembly has some other difficulties which cannot be accounted for using simple code-word lookups. We will introduce assemblers here, and talk about disassembly later.

Assembler Concepts

Assemblers, on a most basic level, translate assembly instructions into machine code bytes with a 1 to 1 correspondence. Assemblers also allow for named variables that get translated into hard-coded memory addresses. Assemblers also translate labels into their relative code addresses.

Assemblers, in general do not perform optimization to the code. The machine code that comes out of a assembler is equivalent to the assembly instructions that go into the assembler. Some assemblers have high-level capabilities in the form of Macros.

Some information about the program is lost during the assembly process. First and foremost, program data is stored in the same raw binary format as the machine code instructions. This means that it can be difficult to determine which parts of the program are actually instructions. Notice that you can disassemble raw data, but the resultant assembly code will be nonsensical. Second, textual information from the assembly source code file, such as variable names, label names, and code comments are all destroyed during assembly. When you disassemble the code, the instructions will be the same, but all the other helpful information will be lost. The code will be accurate, but more difficult to read.

Compilers, as we will see later, cause even more information to be lost, and decompiling is often so difficult and convoluted as to become nearly impossible to do accurately.

Intel Syntax Assemblers

Because of the pervasiveness of Intel-based IA-32 microprocessors in the home PC market, the majority of assembly work done (and the majority of assembly work considered in this wikibook) will be x86 based. Many of these assemblers (or new versions of them) can handle IA-64 code as well, although this wikibook will focus primarily on 32 bit code examples.

MASM

MASM is Microsoft's assembler, an abbreviation for "Macro Assembler." However, many people use it as an acronym for "Microsoft Assembler," and the difference isn't a problem at all. MASM has a powerful macro feature, and is capable of writing very low-level syntax, and pseudo-high-level code with its macro feature. MASM 6.15 is currently available as a free-download from Microsoft, and MASM 7.xx is currently available as part of the Microsoft platform DDK.

MASM writes in Intel Syntax.
MASM is used by Microsoft to implement some low-level portions of its Windows Operating systems.
MASM, contrary to popular belief, has been in constant development since 1980, and is upgraded on a needs-basis.
MASM has always been made compatible by Microsoft to the current platform, and executable file types.
MASM currently supports all Intel instruction sets, including SSE2.

Many users love MASM, but many more still dislike the fact that it isn't portable to other systems.

TASM

TASM, Borland's "Turbo Assembler," is a functional assembler from Borland that integrates seamlessly with Borland's other software development tools. Current release version is version 5.0. TASM syntax is very similar to MASM, although it has an "IDEAL" mode that many users prefer. TASM is not free.

NASM

NASM, the "Netwide Assembler," is a portable, retargetable assembler that works on both Windows and Linux. It supports a variety of Windows and Linux executable file formats, and even outputs pure binary. NASM comes with its own disassembler.

NASM is not as "mature" as either MASM or TASM, but is a) more portable then MASM, b) cheaper then TASM), and c) strives to be very user-friendly.

FASM

FASM, the "Flat Assembler" is an open source assembler that supports x86, and IA-64 Intel architectures.

(x86) AT&T Syntax Assemblers

AT&T syntax for x86 microprocessor assembly code is not as common as Intel-syntax, but the GNU GAS assembler uses it, and it is the de facto assembly standard on Linux.

GAS

Wikipedia has related information at GNU_Assembler.

The GNU Gas Assembler is the default back-end to the GNU GCC compiler suite. As such, GAS is as portable and retargetable as GCC is. However, GAS uses the AT&T syntax for its instructions, which some users find to be less readable than Intel syntax. As a result, assembly code written inline in a C code file for GCC must also be written in GAS syntax.

GAS is developed specifically to be used as the GCC backend. GCC always feeds GAS syntactically-correct code, so GAS often has minimal error checking.

GAS is available either a) in the GCC package, or b) in the GNU BinUtils package. [1]

Other Assemblers

HLA

Wikipedia has related information at High Level Assembly.

HLA, short for "High Level Assembler" is a project spearheaded by Randall Hyde to create an assembler with high-level syntax. HLA works as a front-end to other compilers such as MASM, NASM, and GAS. HLA supports "common" assembly language instructions, but also implements a series of higher-level constructs such as loops, if-then-else branching, and functions. HLA comes complete with a comprehensive standard library.

Since HLA works as a front-end to another assembler, the programmer must have another assembler installed to assemble programs with HLA. HLA code output therefore, is as good as the underlying assembler, but the code is much easier to write for the developer. The high-level components of HLA may make programs less efficient, but that cost is often far outweighed by the ease of writing the code. HLA high-level syntax is very similar in many respects to Pascal, (which in turn is itself similar in many respects to C), so many high-level programmers will immediately pick up many of the aspects of HLA.

Here is an example of some HLA code:

mov(src, dest);  //C++  style comments
 pop(eax);
 push(ebp);
 for(mov(0, ecx); ecx < 10; inc(ecx)) do
    mul(ecx);
 endfor;

Some disassemblers and debuggers can disassemble binary code into HLA-format, although none can faithfully recreate the HLA macros.

Compilers

Wikipedia has related information at compiler.

A compiler is a program that converts instructions from one language into equivalent instructions in another language. There is a common misconception that a compiler always directly converts a high level language into machine language, but this isn't always the case. Many compilers convert code into assembly language, and a few even convert code from one high level language into another. Common examples of compiled languages are: C/C++, Fortran, Ada, and Visual Basic. The figure below shows the common compile-time steps to building a program using the C programming language. The compiler produces object files which are linked to form the final executable:

For the purposes of this book, we will only be considering the case of a compiler that converts C or C++ into assembly code or machine language. Some compilers such as the Microsoft C compiler will compile C and C++ source code directly into machine code. GCC on the other hand will compile C and C++ into assembly language, and an assembler is used to convert that into the appropriate machine code. From the standpoint of a disassembler, it does not matter exactly how the original program was created. Notice also that it is not possible to exactly reproduce the C or C++ code used originally to create an executable. It is, however, possible to create code that compiles identically, or code that performs the same task.

C language statements do not share a one to one relationship with assembly language. Consider that the following C statements will typically all compile into the same assembly language code:

*arrayA = arrayB[x++];
 
*arrayA = arrayB[x]; x++;
 
arrayA[0] = arrayB[x++];
 
arrayA[0] = arrayB[x]; x++;

Also, consider how the following loop constructs perform identical tasks, and are likely to produce similar or even identical assembly language code:

for(;;) { ... }
 
while(1) { ... }
 
do { ... } while(1)

Common C/C++ Compilers

The purpose of this chapter is to list some of the most common C and C++ Compilers in use for developing production-level software. There are many many C compilers in the world, but the reverser doesn't need to consider all cases, especially when looking at professional software. This page will discuss each compiler's strengths and weaknesses, its availability (download sites or cost information), and it will also discuss how to generate an assembly listing file from each compiler.

Microsoft C Compiler

The Microsoft C compiler is available from Microsoft for free as part of the Windows Server 2003 SDK. It is the same compiler and library as is used in MS Visual Studio, but doesn't come with the fancy IDE. The MS C Compiler has a very good optimizing engine. It compiles C and C++, and has the option to compile C++ code into MSIL (the .NET bytecode).

Microsoft's compiler only supports Windows systems, and Intel-compatible 16/32/64 bit architectures.

The Microsoft C compiler is cl.exe and the linker is link.exe

Listing Files

In this wikibook, cl.exe is frequently used to produce assembly listing files of C source code. To produce an assembly listing file yourself, use the syntax:

cl.exe /Fa<assembly file name> <C source file>

The "/Fa" switch is the command-line option that tells the compiler to produce an assembly listing file.

For example, the following command line:

cl.exe /FaTest.asm Test.c

would produce an assembly listing file named "Test.asm" from the C source file "Test.c". Notice that there is no space between the /Fa switch and the name of the output file.

FSF GCC Compiler

This compiler is available for most systems and it is free. Many people use it exclusively so that they can support many platforms with just one compiler to deal with. The GNU GCC Compiler is the de facto standard compiler for Linux and Unix systems. It is retargetable, allowing for many input languages (C, C++, Obj-C, Ada, Fortran, etc...), and supporting multiple target OSes and architectures. It optimizes well, but has a non-aggressive IA-32 code generation engine.

The GCC frontend program is "gcc" ("gcc.exe" on Windows) and the associated linker is "ld" ("ld.exe" on Windows).

Listing Files

To produce an assembly listing file in GCC, use the following command line syntax:

gcc.exe -S <C sourcefile>.c

For example, the following commandline:

gcc.exe -S test.c

will produce an assembly listing file named "test.s". Assembly listing files generated by GCC will be in GAS format. GCC listing files are frequently not as well commented and laid-out as are the listing files for cl.exe.

Intel C Compiler

This compiler is used only for x86, x86-64, and IA-64 code. It is available for both Windows and Linux. The Intel C compiler was written by the people who invented the original x86 architecture: Intel. Intel's development tools generate code that is tuned to run on Intel microprocessors, and is intended to squeeze every last ounce of speed from an application. AMD IA-32 compatible processors are not guaranteed to get the same speed boosts because they have different internal architectures.

Metrowerks CodeWarrior

This compiler is commonly used for classic MacOS and for embedded systems. If you try to reverse-engineer a piece of consumer electronics, you may encounter code generated by Metrowerks CodeWarrior.

Green Hills Software Compiler

This compiler is commonly used for embedded systems. If you try to reverse-engineer a piece of consumer electronics, you may encounter code generated by Green Hills C/C++.

Disassemblers and Decompilers

x86 Disassembly

What is a Disassembler?

Wikipedia has related information at Disassembler.

In essence, a disassembler is the exact opposite of an assembler. Where an assembler converts code written in an assembly language into binary machine code, a disassembler reverses the process and attempts to recreate the assembly code from the binary machine code.

Since most assembly languages have a one-to-one correspondence with underlying machine instructions, the process of disassembly is relatively straight-forward, and a basic disassembler can often be implemented simply by reading in bytes, and performing a table lookup. Of course, disassembly has its own problems and pitfalls, and they are covered later in this chapter.

Many disassemblers have the option to output assembly language instructions in Intel, AT&T, or (occasionally) HLA syntax. Examples in this book will use Intel and AT&T syntax interchangably. We will typically not use HLA syntax for code examples, but that may change in the future.

x86 Disassemblers

Here we are going to list some commonly available disassembler tools. Notice that there are professional disassemblers (which cost money for a license) and there are freeware/shareware disassemblers. Each disassembler will have different features, so it is up to you as the reader to determine which tools you prefer to use.

Commercial Windows Disassemblers

IDA Pro: is a professional (read: expensive) disassembler that is extremely powerful, and has a whole slew of features. The downside to IDA Pro is that it costs $439 US for the standard single-user edition. As such, while it is certainly worth the price, this wikibook will not consider IDA Pro specifically because the price tag is exclusionary. Two freeware versions do exist; see below.; http://www.hex-rays.com/idapro/

PE Explorer: is a disassembler that "focuses on ease of use, clarity and navigation." It isn't as feature-filled as IDA Pro, but carries a smaller price tag to offset the missing functionality: $130; http://www.heaventools.com/PE_Explorer_disassembler.htm

W32DASM: W32DASM is an excellent 16/32 bit disassembler for Windows; http://members.cox.net/w32dasm/

Free Windows Disassemblers

IDA 3.7: This is a DOS GUI tool that behaves very much like IDA Pro, but is considerably more limited. It can disassemble code for the Z80, 6502, Intel 8051, Intel i860, and PDP-11 processors, as well as x86 instructions up to the 486.; http://www.simtel.net/product.php

IDA Pro Freeware 4.1: Behaves almost exactly like IDA Pro, but it only disassembles code for Intel x86 processors, and only runs on Windows. It can disassemble instructions for those processors available as of 2003.; http://www.themel.com/idafree.zip

IDA Pro Freeware 4.3: Better GUI than the previous version.; http://www.datarescue.be/idafreeware/freeida43.exe

BORG Disassembler: BORG is an excellent Win32 Disassembler with GUI.; http://www.caesum.com/

HT Editor: An analyzing disassembler for Intel x86 instructions. The latest version runs as a console GUI program on Windows, but there are versions compiled for Linux as well.; http://hte.sourceforge.net/

diStorm64: diStorm is an open source highly optimized stream disassembler library for 80x86 and AMD64.; http://ragestorm.net/distorm/

Linux Disassemblers

Bastard Disassembler: The Bastard disassembler is a powerful, scriptable disassembler for Linux and FreeBSD.; http://bastard.sourceforge.net/

ciasdis: The official name of ciasdis is computer_intelligence_assembler_disassembler. This Forth-based tool allows to incrementally and interactively build knowledge about a code body. It is unique that all diassembled code can be re-assembled to the exact same code. Processors are 8080, 6809, 8086, 80386, Pentium I en DEC Alpha. A scripting facility aids in analysing Elf and MSDOS headers and makes this tool extendable. The Pentium I ciasdis is available as a binary image, others are in source form, loadable onto lina Forth, available from the same site.; http://home.hccnet.nl/a.w.m.van.der.horst/ciasdis.html

objdump: comes standard, and is typically used for general inspection of binaries. Pay attention to the relocation option and the dynamic symbol table option.

gdb: comes standard, as a debugger, but is very often used for disassembly. If you have loose hex dump data that you wish to disassemble, simply enter it (interactively) over top of something else or compile it into a program as a string like so: char foo[] = {0x90, 0xcd, 0x80, 0x90, 0xcc, 0xf1, 0x90};

lida linux interactive disassembler: an interactive disassembler with some special functions like a crypto analyzer. Displays string data references, does code flow analysis, and does not rely on objdump. Utilizes the Bastard disassembly library for decoding single opcodes.; http://lida.sourceforge.net

ldasm: LDasm (Linux Disassembler) is a Perl/Tk-based GUI for objdump/binutils that tries to imitate the 'look and feel' of W32Dasm. It searches for cross-references (e.g. strings), converts the code from GAS to a MASM-like style, traces programs and much more. Comes along with PTrace, a process-flow-logger.; http://www.feedface.com/projects/ldasm.html

Disassembler Issues

As we have alluded to before, there are a number of issues and difficulties associated with the disassembly process. The two most important difficulties are the division between code and data, and the loss of text information.

Separating Code from Data

Since data and instructions are all stored in an executable as binary data, the obvious question arises: how can a disassembler tell code from data? Is any given byte a variable, or part of an instruction?

The problem wouldn't be as difficult if data were limited to the .data section of an executable (explained in a later chapter) and if executable code was limited to the .code section of an executable, but this is often not the case. Data may be inserted directly into the code section (e.g. jump address tables, constant strings), and executable code may be stored in the data section (although new systems are working to prevent this for security reasons).

Many interactive disassemblers will give the user the option to render segments of code as either code or data, but non-interactive disassemblers will make the separation automatically. Disassemblers often will provide the instruction AND the corresponding hex data on the same line, to reduce the need for decisions to be made about the nature of the code. Some disassemblers (e.g. ciasdis) will allow you to specify rules about whether to disassemble as data or code and invent label names, based on the content of the object under scrutiny. Scripting your own "crawler" in this way is more efficient; for large programs interactive disassembling may be unpractical to the point of being unfeasible.

The general problem of separating code from data in arbitrary executable programs is equivalent to the halting problem. As a consequence, it is not possible to write a disassembler that will correctly separate code and data for all possible input programs. Reverse engineering is full of such theoretical limitations, although by Rice's theorem all interesting questions about program properties are undecidable (so compilers and many other tools that deal with programs in any form run into such limits as well). In practice a combination of interactive and automatic analysis and perseverance can handle all but programs specifically designed to thwart reverse engineering, like using encryption and decrypting code just prior to use, and moving code around in memory.

Lost Information

All text-based identifiers, such as variable names, label names, and macros are removed by the assembly process. These identifiers, in addition to comments in the source file, help to make the code more readable to a human, and can also shed some clues on the purpose of the code. Without these comments and identifiers, it is harder to understand the purpose of the source code, and can be difficult to determine the algorithm being used by that code. When you combine this problem with the fact that the code you are trying to read may, in reality, be data (as outlined above), then it can be ever harder to determine what is going on.

Decompilers

Akin to Disassembly, Decompilers take the process a step further and actually try to reproduce the code in a high level language. Frequently, this high level language is C, because C is simple and primitive enough to facilitate the decompilation process. Decompilation does have its drawbacks, because lots of data and readability constructs are lost during the original compilation process, and they cannot be reproduced. Since the science of decompilation is still young, and results are "good" but not "great", this page will limit itself to a listing of decompilers, and a general (but brief) discussion of the possibilities of decompilation.

Decompilation: Is It Possible?

Wikipedia has related information at decompiler.

In the face of optimizing compilers, it is not uncommon to be asked "Is decompilation even possible?" To some degree, it usually is. Make no mistake, however: there are no perfectly operational decompilers (yet). At most, current Decompilers can be used as simply an aid for the reversing process, with lots of work from the reverser.

Common Decompilers

DCC Decompiler: Dcc is an excellent theoretical look at de-compilation, but currently it only supports small files.; http://www.itee.uq.edu.au/~cristina/dcc.html

Boomerang Decompiler Project: Boomerang Decompiler is an attempt to make a powerful, retargetable compiler. So far, it only decompiles into C with moderate success.; http://boomerang.sourceforge.net/

Reverse Engineering Compiler (REC): REC is a powerful "decompiler" that decompiles native assembly code into a C-like code representation. The code is half-way between assembly and C, but it is much more readable than the pure assembly is.; http://www.backerstreet.com/rec/rec.htm

ExeToC: ExeToC decompiler is an interactive decompiler that boasts pretty good results.; http://sourceforge.net/projects/exetoc

Analysis Tools

x86 Disassembly

Debuggers

Wikipedia has related information at debugger.

Debuggers are programs that allow the user to execute a compiled program one step at at time. You can see what instructions are executed in which order, and which sections of the program are treated as code and which are treated as data. Debuggers allow you to analyze the program while it is running, to help you get a better picture of what it is doing.

Advanced debuggers often contain at least a rudimentary disassembler, often times hex editing and reassembly features. Debuggers often allow the user to set "breakpoints" on instructions, function calls, and even memory locations.

A breakpoint is an instruction to the debugger that allows program execution to be halted when a certain condition is met. for instance, when a program accesses a certain variable, or calls a certain API function, the debugger can pause program execution.

Windows Debuggers

OllyDbg: OllyDbg is a powerful Windows debugger with a built-in disassembly and assembly engine. Has numerous other features including a 0$ price-tag. Very useful for patching, disassembling, and debugging.; http://www.ollydbg.de/

Wikipedia has related information at SoftICE.

SoftICE: A de facto standard for Windows debugging. SoftICE can be used for local kernel debugging, which is a feature that is very rare, and very valuable. SoftICE was taken off the market in April 2006.

Wikipedia has related information at Windbg.

WinDBG: WinDBG is a free piece of software from microsoft that can be used for local user-mode debugging, or even remote kernel-mode debugging. WinDBG is not the same as the better-known Visual Studio Debugger, but comes with a nifty GUI nonetheless. Comes in 32 and 64 bit versions.; http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx

IDA Pro: The multi-processor, multi-OS, interactive disassembler, by DataRescue.; http://www.datarescue.com

Linux Debuggers

gdb: the GNU debugger, comes with any normal Linux install. It is quite powerful and even somewhat programmable, though the raw user interface is harsh.

emacs: the GNU editor, can be used as a front-end to gdb. This provides a powerful hex editor and allows full scripting in a LISP-like language.

ddd: the Data Display Debugger. It's another front-end to gdb. This provides graphical representations of data structures. For example, a linked list will look just like a textbook illustration.

strace, ltrace, and xtrace: let you run a program while watching the actions it performs. With strace, you get a log of all the system calls being made. With ltrace, you get a log of all the library calls being made. With xtrace, you get a log of some of the funtion calls being made.

valgrind: executes a program under emulation, performing analysis according to one of the many plug-in modules as desired. You can write your own plug-in module as desired.

NLKD: A kernel debugger.; http://forge.novell.com/modules/xfmod/project/?nlkd

Debuggers for Other Systems

dbx: the standard Unix debugger on systems derived from AT&T Unix. It is often part of an optional development toolkit package which comes at an extra price. It uses an interactive command line interface.

ladebug: an enhanced debugger on Tru64 Unix systems from HP (originally Digital Equipment Corporation) that handles advanced functionality like threads better than dbx.

DTrace: an advanced tool on Solaris that provides functions like profiling and many others on the entire system, including the kernel.

mdb: The Modular Debugger (MDB) is a new general purpose debugging tool for the Solaris™ Operating Environment. Its primary feature is its extensibility. The Solaris Modular Debugger Guide describes how to use MDB to debug complex software systems, with a particular emphasis on the facilities available for debugging the Solaris kernel and associated device drivers and modules. It also includes a complete reference for and discussion of the MDB language syntax, debugger features, and MDB Module Programming API.

gdb: Comes standard, as a debugger, but is very often used for disassembly. If you have loose hex dump data that you wish to disassemble, simply enter it (interactively) over top of something else or compile it into a program as a string like so: char foo[] = {0x90, 0xcd, 0x80, 0x90, 0xcc, 0xf1, 0x90};

Debugger Techniques

Setting Breakpoints

As previously mentioned in the section on disassemblers, a 6-line C program doing something as simple as outputting "Hello, World!" turns into massive amounts of assembly code. Most people don't want to sift through the entire mess to find out the information they want. It can even be time consuming just to FIND the information one desires by just looking through. As an alternative, one can choose to set breakpoints to halt the program once it has reached a given point within the program.

For instance, let's say that in your program, you consistantly experience crashes at one particular section, immediately after closing a message box. You set a breakpoint on all calls to MessageBoxA. You run your program with the breakpoints, and it stops, ready to call MessageBoxA. Stepping line by line through the program and watching the stack, you see that a buffer overflow occurs shortly after.

Hex Editors

Wikipedia has related information at hex editor.

Hex editors, while not a very popular tool for reversing, are useful in that they can directly view and edit the binary of a source file. Also, hex editors are very useful when examining the structure of proprietary closed-format data files.

There are many many Hex Editors in existence, so this page will attempt to weed out some of the best, some of the most popular, or some of the most powerful.

Windows Hex Editors

Axe: suggested by the CVS one-time use camcorder hackers (discussed later).; http://www.jbrowse.com/products/axe/

HxD (Freeware): fast and powerful free hex, disk and RAM editor; http://mh-nexus.de/hxd/

Freeware Hex Editor XVI32: A freeware hex editor for windows.; http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm

Catch22 HexEdit: This is a powerful hex editor with a slew of features. Has an excellent data structure viewer.; http://www.catch22.net/software/hexedit.asp

BreakPoint Hex Workshop: An excellent and powerful hex-editor, its usefulness is restricted by the fact that it is not free like some of the other options.; http://www.bpsoft.com/

Tiny Hexer: free, does statistics.; http://www.mirkes.de/en/freeware/tinyhex.php

frhed - free hex editor: free, open source for Windows.; http://www.kibria.de/frhed.html

Cygnus Hex Editor FREE EDITION: A very fast and easy-to-use hex editor.; http://www.softcircuits.com/cygnus/fe/

Hexprobe Hex Editor: A professional hex editor designed to include all the power to deal with hex data, particularly helpful in the areas of hex-byte editing, byte-pattern analysis.; http://www.hexprobe.com/hexprobe/index.htm

UltraEdit32: A hex editor/text editor, won "Application of the Year" at 2005 Shareware Industry Awards Conference.; http://www.ultraedit.com/

ICY Hexplorer: A small, lightweight free hex file editor with some nifty features, such as pixel view, structures, and disassembling.; http://www.elektroda.net/download/file1000.html

WinHex: A powerful hex file and disk editor with advanced abilities for computer forensics and data recovery (also used by governments and military); http://www.x-ways.net/index-m.html

010 Editor: A very powerful and fast hex editor with extensive support for data structures and scripting. Can be used to edit drives and processes.; http://www.sweetscape.com/010editor/

A view of a small binary file in a 1Fh hex editor.

1Fh: A free binary/hex editor which is very fast even while working with large files. It's the only Windows hex editor that allows you to view files in byte code (all 256-characters).; http://www.4neurons.com/1Fh/

HexEdit: Powerful and easy to use binary file and disk editor. Free (source available) and shareware versions.; http://www.hexedit.com/

FlexHex: Provides full support for NTFS files which are based on a more complex model than FAT32 files. Specifically, FlexHex supports Sparse files and Alternate data streams of files on any NTFS volume. Can be used to edit OLE compound files, flash cards, and other types of physical drives.; http://www.heaventools.com/flexhex-hex-editor.htm

Linux Hex Editors

bvi: a typical three-pane hex editor, with a vi-like interface.

emacs: along with everything else, emacs obviously includes a hex editor.

xxd and any text editor: produce a hex dump with xxd, freely edit it in your favorite text editor, and then convert it back to a binary file with your changes included

GHex: Hex editor for GNOME.; http://directory.fsf.org/All_Packages_in_Directory/ghex.html

KHexEdit: Hex editor for KDE - a versatile and customizable hex editor. It displays data in hexadecimal, octal, binary and text mode.

http://home.online.no/~espensa/khexedit/index.html

BIEW: a viewer of binary files with built-in editor in binary, hexadecimal and disassembler modes. It uses native Intel syntax for disassembly. Highlight AVR/Java/Athlon64/Pentium 4/K7-Athlon disassembler, Russian codepages converter, full preview of formats - MZ, NE, PE, NLM, coff32, elf partial - a.out, LE, LX, PharLap; code navigator and more over.; http://biew.sourceforge.net/en/biew.html

hview: a curses based hex editor designed to work with large (600+MB) files with as quickly, and with little overhead, as possible.; http://tdistortion.esmartdesign.com/Zips/hview.tgz

HT Editor: A file editor/viewer/analyzer for executables. Its goal is to combine the low-level functionality of a debugger and the usability of IDEs.; http://hte.sourceforge.net/

HexCurse: An ncurses-based hex editor written in C that currently supports hex and decimal address output, jumping to specified file locations, searching, ASCII and EBCDIC output, bolded modifications, an undo command, quick keyboard shortcuts etc.; http://www.jewfish.net/description.php?title=HexCurse

hexedit: view and edit files in hexadecimal or in ASCII.; http://www.geocities.com/SiliconValley/Horizon/8726/hexedit.html

Data Workshop: an editor to view and modify binary data; provides different views which can be used to edit, analyze and export the binary data.; http://www.dataworkshop.de/index.html

VCHE: A hex editor which lets you see all 256 characters as found in video ROM, even control and extended ASCII, it uses the /dev/vcsa* devices to do it. It also could edit non-regular files, like hard disks, floppies, CDROMs, ZIPs, RAM, and almost any device. It comes with a ncurses and a raw version for people who work under X or remotely.; http://www.grigna.com/diego/linux/vche/

DHEX: DHEX is just another Hexeditor with a Diff-mode for ncurses. It makes heavy use of colors and is themeable.; http://www.dettus.net/dhex/

Hex Editors for Mac

HexEdit: A simple but reliable hex editor wher you to change highlight colours. There is also a port for Apple Classic users.; http://hexedit.sourceforge.net/

Hex Fiend: A very simple hex editor, but at least it works fine. It's only 346 KB to download and takes files as big as 116 GB.; http://ridiculousfish.com/hexfiend/

Other Tools for Windows

Resource Monitors

SysInternals Freeware: This page has a large number of excellent utilities, many of which are very useful to security experts, network administrators, and (most importantly to us) reversers. Specifically, check out Process Monitor, FileMon, TCPView, RegMon, and Process explorer.; http://www.microsoft.com/technet/sysinternals/default.mspx

API Monitors

SpyStudio Freeware: The Spy Studio software is a tool to hook into windows processes, log windows API call to DLLs, insert breakpoints and change parameters.; http://www.nektra.com/products/spystudio/

PE File Header dumpers

Dumpbin: Dumpbin is a program that previously used to be shipped with MS Visual Studio, but recently the functionality of Dumpbin has been incorporated into the Microsoft Linker, link.exe. to access dumpbin, pass /dump as the first parameter to link.exe:

link.exe /dump [options]

It is frequently useful to simply create a batch file that handles this conversion:

::dumpbin.bat
link.exe /dump %*

All examples in this wikibook that use dumpbin will call it in this manner.

Here is a list of usefull features of dumpbin [2]:

dumpbin /EXPORTS         displays a list of functions exported from a library
dumpbin /IMPORTS         displays a list of functions imported from other libraries
dumpbin /HEADERS         displays PE header information for the executable

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/_core_dumpbin_reference.asp

GNU Tools

The GNU packages have been ported to many platforms including Windows.

GNU BinUtils: The GNU BinUtils package contains several small utilties that are very useful in dealing with binary files. The most important programs in the list are the GNU objdump, readelf, GAS assembler, and the GNU linker, although the reverser might find more use in addr2line, c++filt, nm, and readelf.; http://www.gnu.org/software/binutils/
objdump: dumps out information about an executable including symbols and assembly. It comes standard. It can be made to support non-native binary formats.
readelf: like objdump, but more specialized for ELF executables.
size: lists the sizes of the segments
nm: lists the symbols in elf file

Other gnu tools

strings: lists the strings from
file: tells you what type of file it is
fold: folds the results of strings into something pageable

GNU Tools for dynamic reverse engineering

kill: can be used to halt a program - with the sig_stop signal
gdb: can be used to attach to a program
strace: trace system calls and signals

Other Tools for Linux

oprofile: can be used the find out what functions and data segments are used

subterfugue: is a tool for playing odd tricks on an executable as it runs. The tool is scriptable in python. The user can write scripts to take action on events that occur, such as changing the arguments to system calls.; http://subterfugue.org/

lizard: lets you run a program backwards.; http://lizard.sourceforge.net/index.html

dprobes: lets you work with both kernel and user code

biew: both hex editor and disassembler

ltrace: shows runtime library call information for dynamically linked executables

Platforms

Microsoft Windows

x86 Disassembly

Microsoft Windows

Wikipedia has related information at Windows.

The Windows operating system is a popular target for reverses for one simple reason: the OS itself (market share, known weaknesses), and most applications for it, are not Open Source or free. Most software on a Windows machine doesn't come bundled with its source code, and most pieces have inadequate, or non-existent documentation. Occasionally, the only way to know precisely what a piece of software does (or for that matter, to determine whether a given piece of software is malicious or legitimate) is to reverse it, and examine the results.

Windows Versions

Windows operating systems can be easily divided into 2 categories: Win9x, and WinNT.

Windows 9x

The Win9x kernel was originally written to span the 16bit - 32bit divide. Operating Systems based on the 9x kernel are: Windows 95, Windows 98, and Windows ME. Win9x Series operating systems are known to be prone to bugs and system instability. The actual OS itself was a 32 bit extension of MS-DOS, its predecessor. An important issue with the 9x line is that they were all based around using the ASCII format for storing strings, rather than unicode.

Development on the Win9x kernel ended with the release of Windows XP.

Windows NT

The WinNT kernel series was originally written as enterprise-level server and network software. WinNT stresses stability and security far more than Win9x kernels did (although it can be debated whether that stress was good enough). It also handles all string operations internally in unicode, giving more flexibility when using different languages. Operating Systems based on the WinNT kernel are: Windows NT (versions 3.1, 3.5, 3.51 and 4.0), Windows 2000 (NT 5.0), Windows XP (NT 5.1), Windows Server 2003 (NT 5.2), and Windows Vista (NT 6.0).

The Microsoft XBOX and and XBOX 360 also run a variant of NT, forked from Windows 2000. Most future Microsoft operating system products are based on NT in some shape or form.

Virtual Memory

32 bit WinNT allows for a maximum of 4Gb of virtual memory space, divided into "pages" that are 4096 bytes by default. Pages not in current use by the system or any of the applications may be written to a special section on the harddisk known as the "paging file." Use of the paging file may increase performance on some systems, although high latency for I/O to the HDD can actually reduce performance in some instances.

System Architecture

The Windows architecture is heavily layered. Function calls that a programmer makes may be redirected 3 times or more before any action is actually performed. There is an unignorable penalty for calling Win32 functions from a user-mode application. However, the upside is equally unignorable: code written in higher levels of the windows system is much easier to write. Complex operations that involve initializing multiple data structures and calling multiple sub-functions can be performed by calling only a single higher-level function.

The Win32 API comprises 3 modules: KERNEL, USER, and GDI. KERNEL is layered on top of NTDLL, and most calls to KERNEL functions are simply redirected into NTDLL function calls. USER and GDI are both based on WIN32K (a kernel-mode module, responsible for the Windows "look and feel"), although USER also makes many calls to the more-primitive functions in GDI. This and NTDLL both provide an interface to the Windows NT kernel, NTOSKRNL (see further below).

NTOSKRNL is also partially layered on HAL (Hardware Abstraction Layer), but this interaction will not be considered much in this book. The purpose of this layering is to allow processor variant issues (such as location of resources) to be made separate from the actual kernel itself. A slightly different system configuration thus requires just a different HAL module, rather than a completely different kernel module.

System calls and interrupts

After filtering through different layers of subroutines, most API calls require interaction with part of the operating system. Services are provided via 'software interrupts', traditionally through the "int 0x2e" instruction. This switches control of execution to the NT executive / kernel, where the request is handled. It should be pointed out here that the stack used in kernel mode is different from the user mode stack. This provides an added layer of protection between kernel and user. Once the function completes, control is returned back to the user application.

Both Intel and AMD provide an extra set of instructions to allow faster system calls, the "SYSENTER" instruction from Intel and the SYSCALL instruction from AMD.

Win32 API

Both WinNT and Win9x systems utilize the Win32 API. However, the WinNT version of the API has more functionality and security constructs, as well as unicode support. Most of the Win32 API can be broken down into 3 separate components, each performing a separate task.

kernel32.dll

Kernel32.dll, home of the KERNEL subsystem, is where non-graphical functions are implemented. Some of the APIs located in KERNEL are: The Heap API, the Virtual Memory API, File I/O API, the Thread API, the System Object Manager, and other similar system services. Most of the functionality of kernel32.dll is implemented in ntdll.dll, but in undocumented functions. Microsoft prefers to publish documentation for kernel32 and guarantee that these APIs will remain unchanged, and then put most of the work in other libraries, which are then not documented.

gdi32.dll

gdi32.dll is the library that implements the GDI subsystem, where primitive graphical operations are performed. GDI diverts most of its calls into WIN32K, but it does contain a manager for GDI objects, such as pens, brushes and device contexts. The GDI object manager and the KERNEL object manager are completely separate.

user32.dll

The USER subsystem is located in the user32.dll library file. This subsystem controls the creation and manipulation of USER objects, which are common screen items such as windows, menus, cursors, etc... USER will set up the objects to be drawn, but will perform the actual drawing by calling on GDI (which in turn will make many calls to WIN32K) or sometimes even calling WIN32K directly. USER utilizes the GDI Object Manager.

Native API

The native API, hereby referred to as the NTDLL subsystem, is a series of undocumented API function calls that handle most of the work performed by KERNEL. It has been speculated by many that by tapping into NTDLL directly, programs could be spared a certain amount of redirection, and have a performance increase. However, the NTDLL function calls and data structures are usually more complicated than the corresponding KERNEL functions and data structures, so gains are hard to measure. Also, without the added error checking, and the proper calls into kernel mode, the application risks producing errors that are crippling to the system. Microsoft also does not guarantee that the native API will remain the same between different versions, as Windows developers modify the software. This gives the risk of native API calls being removed or changed without warning, breaking software that utilizes it.

ntdll.dll

The NTDLL subsystem is located in ntdll.dll. This enigmatic library contains many API function calls, that all follow a particular naming scheme. Each function has a prefix: Ldr, Ki, Nt, Zw, Csr, dbg, etc... and all the functions that have a particular prefix all follow particular rules.

The "official" native API is usually limited only to functions whose prefix is Nt or Zw. These calls are in fact the same: the relevant Export entries map to the same address in memory. Thus there is not real difference, although the reason for the double-mapping results from ntdll's dual purpose: it is used to provide function calls in both kernel and user space. User applications are encouraged to use the Nt* calls, while kernel callers are supposed to use the Zw* calls. The origin of the prefix "Zw" is unknown; it is rumored that this prefix was chosen due to its having no significance at all.

In actual implementation, the Nt / Zw calls merely load two registers with values required to describe a native API call, and then execute a software interrupt.

Most of the other prefixes are obscure, but the known ones are:

RTL stands for "Run Time Library", calls which help functionality at runtime (such as RtlAllocateHeap)
CSR is for "Client Server Runtime", which represents the interface to the win32 subsystem located in csrss.exe
DBG functions are present to enable debugging routines and operations
LDR provides the ability to manipulate and retrieve data from shared libraries and other module resources

User Mode Versus Kernel Mode

Many of the other functions in NTDLL are usable, but not to application writers. Developers working on writing device drivers for Windows are frequently only allowed to use the Kernel-mode functions in NTDLL because device drivers operate at kernel-level. As such, Microsoft provides documentation on many of the APIs with prefixes other than Nt and Zw with the Microsoft Server 2003 Platform DDK. The DDK (Driver Development Kit) is available as a free download.

ntoskrnl.exe

This module is the Windows NT "'Executive'", providing all the functionality required by the native API, as well as the kernel itself, which is responsible for maintaining the machine state. By default, all interrupts and kernel calls are channeled through ntoskrnl in some way, making it the single most important program in Windows itself. Many of its functions are exported (all of which with various prefixes, a la NTDLL) for use by device drivers. It's not advised to try to call these routines from user mode, and the IMAGE_FILE_SYSTEM flag is set in the file's PE Header, preventing applications from trying this. Some functions from NTOSKRNL may be considered in later examples.

Win32K.sys

This module is the "Win32 Kernel" that sits on top of the lower-level, more primitive NTOSKRNL. WIN32K is responsible for the "look and feel" of windows, and many portions of this code have remained largely unchanged since the Win9x versions. This module provides many of the specific instructions that cause USER and GDI to act the way they do. It's responsible for translating the API calls from the USER and GDI libraries into the pictures you see on the monitor.

Win64 API

With the advent of 64-bit processors, 64-bit software is a necessity. As a result, the Win64 API was created to utilize the new hardware. It is important to note that the format of many of the function calls are identical in Win32 and Win64, except for the size of pointers, and other data types that are specific to 64-bit address space.

Differences

Windows Vista

Microsoft has released a new version of its Windows operation system, named "Windows Vista." Windows Vista may be better known by its development code-name "Longhorn." Microsoft claims that Vista has been written largely from the ground up, and therefore it can be assumed that there are fundamental differences between the Vista API and system architecture, and the APIs and architectures of previous Windows versions. Windows Vista was released January 30th, 2007.

Windows CE/Mobile, and other versions

Windows CE is the Microsoft offering on small devices. It largely uses the same Win32 API as the desktop systems, although it has a slightly different architecture. Some examples in this book may consider WinCE.

"Non-Executable Memory"

Recent windows service packs have attempted to implement a system known as "Non-executable memory" where certain pages can be marked as being "non-executable". The purpose of this system is to prevent some of the most common security holes by not allowing control to pass to code inserted into a memory buffer by an attacker. For instance, a shellcode loaded into an overflowed text buffer cannot be executed, stopping the attack in its tracks. The effectiveness of this mechanism is yet to be seen, however.

COM and Related Technologies

COM, and a whole slew of technologies that are either related to COM or are actually COM with a fancy name, is another factor to consider when reversing Windows binaries. COM, DCOM, COM+, ActiveX, OLE, MTS, and Windows DNA are all names for the same subject, or subjects, so similar that they may all be considered under the same heading. In short, COM is a method to export Object-Oriented Classes in a uniform, cross-platform and cross-language manner. In essence, COM is .NET, version 0 beta. Using COM, components written in many languages can export, import, instantiate, modify, and destroy objects defined in another file, most often a DLL. Although COM provides cross-platform (to some extent) and cross-language facilities, each COM object is compiled to a native binary, rather than an intermediate format such as Java or .NET. As a result, COM does not require a virtual machine to execute such objects.

This book will attempt to show some examples of COM files, and the reversing challenges associated with them, although the subject is very broad, and may elude the scope of this book (or at least the early sections of it). The discussion may be part of an "Advanced Topic" found in the later sections of this book.

Due to the way that COM works, a lot of the methods and data structures exported by a COM component are difficult to perceive by simply inspecting the executable file. Matters are made worse if the creating programmer has used a library such as ATL to simplify their programming experience. Unfortunately for a reverse engineer, this reduces the contents of an executable into a "Sea of bits", with pointers and data structures everywhere.

Remote Procedure Calls (RPC)

RPC is a generic term referring to techniques that allow a program running on one machine to make calls that actually execute on another machine. Typically, this is done by marshalling all of the data needed for the procedure including any state information stored on the first machine, and building it into a single data structure, which is then transmitted over some communications method to a second machine. This second machine then performs the requested action, and returns a data packet containing any results and potentially changed state information to the originating machine.

In Windows NT, RPC is typically handled by having two libraries that are similarly named, one which generates RPC requests and accepts RPC returns, as requested by a user-mode program, and one which responds to RPC requests and returns results via RPC. A classic example is the print spooler, which consists of two pieces: the RPC stub spoolss.dll, and the spooler proper and RPC service provider, spoolsv.exe. In most machines, which are stand-alone, it would seem that the use of two modules communicating by means of RPC is overkill; why not simply roll them into a single routine? In networked printing, though, this makes sense, as the RPC service provider can be resident physically on a distant machine, with the remote printer, and the local machine can control the printer on the remote machine in exactly the same way that it controls printers on the local machine.

Windows Executable Files

x86 Disassembly

MS-DOS COM Files

COM files are loaded into RAM exactly as they appear; no change is made at all from the harddisk image to RAM. This is possible due to the 20-bit memory model of the early x86 line. Two 16-bit registers would have to be set, one dividing the 1MB+64K memory space into 65536 'segments' and one specifying an offset from that. The segment register would be set by DOS and the COM file would be expected to respect this setting and not ever change the segment registers. The offset registers, however, were free game and served (for COM files) the same purpose as a modern 32-bit register. The downside was that the offset registers were only 16-bit and, therefore, since COM files could not change the segment registers, COM files were limited to using 64K of RAM. The good thing about this approach, however, was that no extra work was needed by DOS to load and run a COM file: just load the file, set the segment register, and jmp to it. (The programs could perform 'near' jumps by just giving an offset to jump too.)

COM files are loaded into RAM at offset $100. The space before that would be used for passing data to and from DOS (for example, the contents of the command line used to invoke the program).

Note that COM files, by definition, cannot be 32-bit. Windows provides support for COM files via a special CPU mode.

Notice that MS-DOS COM files (short for "command" files) are not the same as Component-Object Model files, which are an object-oriented library technology.

MS-DOS EXE Files

One way MS-DOS compilers got around the 64k memory limitation was with the introduction of memory models. The basic concept is to cleverly set different segment registers in the x86 CPU (CS, DS, ES, SS) to point to the same or different segments, thus allowing varying degrees of access to memory. Typical memory models were:

tiny: All memory access are 16-bit (never reload any segment register). Produces a .COM file instead of an .EXE file.
small: All memory access are 16-bit (never reload any segment register).
compact: accesses to the code segment reload the CS register, allowing 32-bit of code. Data accesses don't reload the DS, ES, SS registers, allowing 16-bit of data.
medium: accesses to the data segment reload the DS, ES, SS register, allowing 32-bit of data. Code accesses don't reload the CS register, allowing 16-bit of code.
large: both code and data accesses use the segment registers (CS for code, DS, ES, SS for data), allowing 32-bit of code and 32-bit of data.
huge: same as the large model, with additional arithmetic being generated by the compiler to allow access to arrays larger than 64k.

When looking at a COM file, one has to decide which memory model was used to build that file.

PE Files

Portable Executable file is the standard binary(EXE and DLL) file format on Windows NT, Windows 95 and Win32. The Win32 SDK has a file winnt.h, which declares various structs and variables used in the PE files. A DLL, imagehlp.dll also contains some functions for manipulating PE files. PE files are broken down into various sections that can be examined.

Relative Virtual Addressing (RVA)

In a Windows environment, executable modules can be loaded at any point in memory, and are expected to run without problem. To allow multiple programs to be loaded at seemingly random locations in memory, PE files have adopted a tool called RVA: Relative Virtual Addresses. RVA's assume that the "base address" of where a module is loaded into memory is not known at compile time. So, PE files describe the location of data in memory as an offset from the base address, wherever that may be in memory.

Some processor instructions require the code itself to directly identify where in memory some data is. This is not possible when the location of the module in memory is not known at compile time. The solution to this problem is described in the section on "Relocations".

It is important to remember that the addresses obtained from a disassembly of a module will not always match up to the addresses seen in a debugger as the program is running.

File Format

The PE portable executable file format includes a number of informational headers, and is arranged in the following format:

The basic format of a Microsoft PE file

MS-DOS header

Open any Win32 binary executable in a hex editor, and note what you see: The first 2 letters are always the letters "MZ". To some people, the first few bytes in a file that determine the type of file are called the "magic number," although this book will not use that term, because there is no rule that states that the "magic number" needs to be a single number. Instead, we will use the term "File ID Tag", or simply, File ID. Sometimes this is also known as File Signature.

After the File ID, the hex editor will show several bytes of either random-looking symbols, or whitespace, before the human-readable string "This program cannot be run in DOS mode".

What is this?

Hex Listing of an MS-DOS file header

What you are looking at is the MS-DOS header of the Win32 PE file. To ensure either a) backwards compatibility, or b) graceful decline of new file types, Microsoft has engineered a series of DOS instructions into the head of each PE file. When a 32-bit Windows file is run in a 16-bit DOS environment, the program will terminate immediately with the error message: "This program cannot be run in DOS mode".

The DOS header is also known by some as the EXE header. Here is the DOS header presented as a C data structure:

struct DOS_Header 
 {
     char signature[2] = "MZ";
     short lastsize;
     short nblocks;
     short nreloc;
     short hdrsize;
     short minalloc;
     short maxalloc;
     void *ss;
     void *sp;
     short checksum;
     void *ip;
     void *cs;
     short relocpos;
     short noverlay;
     short reserved1[4];
     short oem_id;
     short oem_info;
     short reserved2[10];
     long  e_lfanew;
 }

Immediately following the DOS Header will be the classic error message "This program cannot be run in DOS mode".

PE Header

At offset 60 from the beginning of the DOS header is a pointer to the Portable Executable (PE) File header (e_lfanew in MZ structure). DOS will print the error message and terminate, but Windows will follow this pointer to the next batch of information.

Hex Listing of a PE signature, and the pointer to it

The PE header consists only of a File ID signature, with the value "PE\0\0" where each '\0' character is an ASCII NUL character. This signature shows that a) this file is a legitimate PE file, and b) the byte order of the file. Byte order will not be considered in this chapter, and all PE files are assumed to be in "little endian" format.

The first big chunk of information lies in the COFF header, directly after the PE signature.

COFF Header

The COFF header is present in both COFF object files (before they are linked) and in PE files where it is known as the "File header". The COFF header has some information that is useful to an executable, and some information that is more useful to an object file.

Here is the COFF header, presented as a C data structure:

struct COFFHeader
 {
    short Machine;
    short NumberOfSections;
    long TimeDateStamp;
    long PointerToSymbolTable;
    long NumberOfSymbols;
    short SizeOfOptionalHeader;
    short Characteristics;
 }

Machine: This field determines what machine the file was compiled for. A hex value of 0x14C (332 in decimal) is the code for an Intel 80386.
NumberOfSections: The number of sections that are described at the end of the PE headers.
TimeDateStamp: 32 bit time at which this header was generated: is used in the process of "Binding", see below.
SizeOfOptionalHeader: this field shows how long the "PE Optional Header" is that follows the COFF header.
Characteristics: This is a field of bit flags, that show some characteristics of the file.

0x02 = Executable file
0x200 = file is non-relocatable (addresses are absolute, not RVA).
0x2000 = File is a DLL Library, not an EXE.

PE Optional Header

The "PE Optional Header" is not "optional" per se, because it is required in Executable files, but not in COFF object files. The Optional header includes lots and lots of information that can be used to pick apart the file structure, and obtain some useful information about it.

The PE Optional Header occurs directly after the COFF header, and some sources even show the two headers as being part of the same structure. This wikibook separates them out for convenience.

Here is the PE Optional Header presented as a C data structure:

struct PEOptHeader
 {
    short signature; //decimal number 267.
    char MajorLinkerVersion; 
    char MinorLinkerVersion;
    long SizeOfCode;
    long SizeOfInitializedData;
    long SizeOfUninitializedData;
    long AddressOfEntryPoint;  //The RVA of the code entry point
    long BaseOfCode;
    long BaseOfData;
    long ImageBase;
    long SectionAlignment;
    long FileAlignment;
    short MajorOSVersion;
    short MinorOSVersion;
    short MajorImageVersion;
    short MinorImageVersion;
    short MajorSubsystemVersion;
    short MinorSubsystemVersion;
    long Reserved;
    long SizeOfImage;
    long SizeOfHeaders;
    long Checksum;
    short Subsystem;
    short DLLCharacteristics;
    long SizeOfStackReserve;
    long SizeOfStackCommit;
    long SizeOfHeapReserve;
    long SizeOfHeapCommit;
    long LoaderFlags;
    long NumberOfRvaAndSizes;
    data_directory DataDirectory[16];     //Can have any number of elements, matching the number in NumberOfRvaAndSizes.
 }                                        //However, it is always 16 in PE files.

struct data_directory
 { 
    long VirtualAddress;
    long Size;
 }

Some of the important pieces of information:

MajorLinkerVersion, MinorLinkerVersion: The version, in x.y format of the linker used to create the PE.
AddressOfEntryPoint: The RVA of the entry point to the executable. Very important to know.
SizeOfCode: Size of the .text (.code) section
SizeOfInitializedData: Size of .data section
BaseOfCode: RVA of the .text section
BaseOfData: RVA of .data section
ImageBase: Preferred location in memory for the module to be based at
Checksum: Checksum of the file, only used to verify validity of modules being loaded into kernel space. The formula used to calculate PE file checksums is proprietary, although Microsoft provides API calls that can calculate the checksum for you.
Subsystem: the Windows subsystem that will be invoked to run the executable

1 = native
2 = Windows/GUI
3 = Windows non-GUI
5 = OS/2
7 = POSIX

DataDirectory: Possibly the most interesting member of this structure. Provides RVAs and sizes which locate various data structures, which are used for setting up the execution environment of a module. The details of what these structures do exist in other sections of this page, but the most interesting entries in DataDirectory are below:

IMAGE_DIRECTORY_ENTRY_EXPORT (0) : Location of the export directory
IMAGE_DIRECTORY_ENTRY_IMPORT (1) : Location of the import directory
IMAGE_DIRECTORY_ENTRY_RESOURCE (2) : Location of the resource directory
IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT (11) : Location of alternate import-binding directory

Code Sections

The PE Header defines the number of sections in the executable file. Each section definition is 40 bytes in length. Below is an example hex from a program I am writing:

2E746578_74000000_00100000_00100000_A8050000 .text
00040000_00000000_00000000_00000000_20000000
2E646174_61000000_00100000_00200000_86050000 .data
000A0000_00000000_00000000_00000000_40000000
2E627373_00000000_00200000_00300000_00000000 .bss
00000000_00000000_00000000_00000000_80000000

The structure of the section descriptor is as follows:

Offset Length   Purpose
------ -------  ------------------------------------------------------------------
 0x00   8 bytes Section Name - in the above example the names are .text .data .bss
 0x08   4 bytes Size of the section once it is loaded to memory
 0x0C   4 bytes RVA (location) of section once it is loaded to memory
 0x10   4 bytes Physical size of section on disk
 0x14   4 bytes Physical location of section on disk (from start of disk image)
 0x18  12 bytes Reserved (usually zero) (used in object formats)
 0x24   4 bytes Section flags

A PE loader will place the sections of the executable image at the locations specified by these section descriptors (relative to the base address) and usually the alignment is 0x1000, which matches the size of pages on the x86.

Common sections are:

.text/.code/CODE/TEXT - Contains executable code (machine instructions)
.testbss/TEXTBSS - Present if Incremental Linking is enabled
.data/.idata/DATA/IDATA - Contains initialised data
.bss/BSS - Contains uninitialised data

Section Flags

The section flags is a 32-bit bit field (each bit in the value represents a certain thing). Here are the constants defined in the WINNT.H file for the meaning of the flags:

#define IMAGE_SCN_TYPE_NO_PAD                0x00000008  // Reserved.
#define IMAGE_SCN_CNT_CODE                   0x00000020  // Section contains code.
#define IMAGE_SCN_CNT_INITIALIZED_DATA       0x00000040  // Section contains initialized data.
#define IMAGE_SCN_CNT_UNINITIALIZED_DATA     0x00000080  // Section contains uninitialized data.
#define IMAGE_SCN_LNK_OTHER                  0x00000100  // Reserved.
#define IMAGE_SCN_LNK_INFO                   0x00000200  // Section contains comments or some  other type of information.
#define IMAGE_SCN_LNK_REMOVE                 0x00000800  // Section contents will not become part of image.
#define IMAGE_SCN_LNK_COMDAT                 0x00001000  // Section contents comdat.
#define IMAGE_SCN_NO_DEFER_SPEC_EXC          0x00004000  // Reset speculative exceptions handling bits in the TLB entries for this section.
#define IMAGE_SCN_GPREL                      0x00008000  // Section content can be accessed relative to GP
#define IMAGE_SCN_MEM_FARDATA                0x00008000
#define IMAGE_SCN_MEM_PURGEABLE              0x00020000
#define IMAGE_SCN_MEM_16BIT                  0x00020000
#define IMAGE_SCN_MEM_LOCKED                 0x00040000
#define IMAGE_SCN_MEM_PRELOAD                0x00080000
#define IMAGE_SCN_ALIGN_1BYTES               0x00100000  //
#define IMAGE_SCN_ALIGN_2BYTES               0x00200000  //
#define IMAGE_SCN_ALIGN_4BYTES               0x00300000  //
#define IMAGE_SCN_ALIGN_8BYTES               0x00400000  //
#define IMAGE_SCN_ALIGN_16BYTES              0x00500000  // Default alignment if no others are specified.
#define IMAGE_SCN_ALIGN_32BYTES              0x00600000  //
#define IMAGE_SCN_ALIGN_64BYTES              0x00700000  //
#define IMAGE_SCN_ALIGN_128BYTES             0x00800000  //
#define IMAGE_SCN_ALIGN_256BYTES             0x00900000  //
#define IMAGE_SCN_ALIGN_512BYTES             0x00A00000  //
#define IMAGE_SCN_ALIGN_1024BYTES            0x00B00000  //
#define IMAGE_SCN_ALIGN_2048BYTES            0x00C00000  //
#define IMAGE_SCN_ALIGN_4096BYTES            0x00D00000  //
#define IMAGE_SCN_ALIGN_8192BYTES            0x00E00000  //
#define IMAGE_SCN_ALIGN_MASK                 0x00F00000
#define IMAGE_SCN_LNK_NRELOC_OVFL            0x01000000  // Section contains extended relocations.
#define IMAGE_SCN_MEM_DISCARDABLE            0x02000000  // Section can be discarded.
#define IMAGE_SCN_MEM_NOT_CACHED             0x04000000  // Section is not cachable.
#define IMAGE_SCN_MEM_NOT_PAGED              0x08000000  // Section is not pageable.
#define IMAGE_SCN_MEM_SHARED                 0x10000000  // Section is shareable.
#define IMAGE_SCN_MEM_EXECUTE                0x20000000  // Section is executable.
#define IMAGE_SCN_MEM_READ                   0x40000000  // Section is readable.
#define IMAGE_SCN_MEM_WRITE                  0x80000000  // Section is writeable.

Imports and Exports - Linking to other modules

What is linking?

Whenever a developer writes a program, there are a number of subroutines and functions which are expected to be implemented already, saving the writer the hassle of having to write out more code or work with complex data structures. Instead, the coder need only declare one call to the subroutine, and the linker will decide what happens next.

There are two types of linking that can be used: static and dynamic. Static uses a library of precompiled functions. This precompiled code can be inserted into the final executable to implement a function, saving the programmer a lot of time. In contrast, dynamic linking allows subroutine code to reside in a different file (or module), which is loaded at runtime by the operating system. This is also known as a "Dynamically linked library", or DLL. A library is a module containing a series of functions or values that can be exported. This is different from the term executable, which imports things from libraries to do what it wants. From here on, "module" means any file of PE format, and a "Library" is any module which exports and imports functions and values.

Dynamically linking has the following benefits:

It saves disk space, if more than one executable links to the library module
Allows instant updating of routines, without providing new executables for all applications
Can save space in memory by mapping the code of a library into more than one process
Increases abstraction of implementation. The method by which an action is achieved can be modified without the need for reprogramming of applications. This is extremely useful for backward compatibility with operating systems.

This section discusses how this is achieved using the PE file format. An important point to note at this point is that anything can be imported or exported between modules, including variables as well as subroutines.

Loading

The downside of dynamically linking modules together is that, at runtime, the software which is initialising an executable must link these modules together. For various reasons, you cannot declare that "The function in this dynamic library will always exist in memory here". If that memory address is unavailable or the library is updated, the function will no longer exist there, and the application trying to use it will break. Instead, each module (library or executable) must declare what functions or values it exports to other modules, and also what it wishes to import from other modules.

As said above, a module cannot declare where in memory it expects a function or value to be. Instead, it declared where in its own memory it expects to find a pointer to the value it wishes to import. This permits the module to address any imported value, wherever it turns up in memory.

Exports

Exports are functions and values in one module that have been declared to be shared with other modules. This is done through the use of the "Export Directory", which is used to translate between the name of an export (or "Ordinal", see below), and a location in memory where the code or data can be found. The start of the export directory is identified by the IMAGE_DIRECTORY_ENTRY_EXPORT entry of the resource directory. All export data must exist in the same section. The directory is headed by the following structure:

struct IMAGE_EXPORT_DIRECTORY {
        long Characteristics;
        long TimeDateStamp;
        short MajorVersion;
        short MinorVersion;
        long Name;
        long Base;
        long NumberOfFunctions;
        long NumberOfNames;
        long *AddressOfFunctions;
        long *AddressOfNames;
        long *AddressOfNameOrdinals;
}

The "Characteristics" value is generally unused, TimeDateStamp describes the time the export directory was generated, MajorVersion and MinorVersion should describe the version details of the directory, but their nature is undefined. These values have little or no impact on the actual exports themselves. The "Name" value is an RVA to a zero terminated ASCII string, the name of this library name, or module.

Names and Ordinals

Each exported value has both a name and an "ordinal" (a kind of index). The actual exports themselves are described through AddressOfFunctions, which is an RVA to an array of RVA's, each pointing to a different function or value to be exported. The size of this array is in the value NumberOfFunctions. Each of these functions has an ordinal. The "Base" value is used as the ordinal of the first export, and the next RVA in the array is Base+1, and so forth.

Each entry in the AddressOfFunctions array is identified by a name, found through the RVA AddressOfNames. The data where AddressOfNames points to is an array of RVA's, of the size NumberOfNames. Each RVA points to a zero terminated