\n\nThis page has been machine-translated from the original page.
\n
I have been reading xv6 OS, inspired by the book はじめてのOSコードリーディング ~UNIX V6で学ぶカーネルのしくみ.
\nI want to get better at reverse engineering and deepen my understanding of kernels and operating systems.
\n詳解 Linuxカーネル felt a bit heavy, so I was looking for somewhere lighter to start. I came across UNIX V6 — an OS with a total of around 10,000 lines of code, which is just barely comprehensible for a human — and became interested.
\nHowever, UNIX V6 itself does not run on x86 CPUs, so I decided to read the source code of kash1064/xv6-public: xv6 OS, a fork of xv6 OS — which is UNIX V6 adapted to run on x86 architecture.
\nContinuing from the previous article, I will keep reading the xv6 OS source code.
\nIn the previous article, I read through the xv6 OS bootstrap code and traced it up to the point just before the kernel body is loaded.
\nThis time, I will trace what actually happens when the kernel is loaded.
\n\nLet me first review where the kernel was being loaded during the bootstrap phase.
\nThe kernel was read into memory at address 0x10000, as shown below.
After that, the program headers were loaded and the entry() function was called, transferring control to the kernel.
void bootmain(void)\n{\n struct elfhdr *elf;\n struct proghdr *ph, *eph;\n void (*entry)(void);\n uchar* pa;\n\n elf = (struct elfhdr*)0x10000; // scratch space\n\n // Read 1st page off disk\n readseg((uchar*)elf, 4096, 0);\n\n // Is this an ELF executable?\n if(elf->magic != ELF_MAGIC)\n return; // let bootasm.S handle error\n\n // Load each program segment (ignores ph flags).\n ph = (struct proghdr*)((uchar*)elf + elf->phoff);\n eph = ph + elf->phnum;\n for(; ph < eph; ph++){\n pa = (uchar*)ph->paddr;\n readseg(pa, ph->filesz, ph->off);\n if(ph->memsz > ph->filesz)\n stosb(pa + ph->filesz, 0, ph->memsz - ph->filesz);\n }\n\n // Call the entry point from the ELF header.\n // Does not return!\n entry = (void(*)(void))(elf->entry);\n entry();\n}So this time I want to start by tracking down the entry() function.
Let me trace through how the kernel program is built.
\nThe final image file xv6.img is generated with the following commands.
xv6.img is produced by embedding bootblock and kernel into a 0x10000-byte blank area.
xv6.img: bootblock kernel\ndd if=/dev/zero of=xv6.img count=10000\ndd if=bootblock of=xv6.img conv=notrunc\ndd if=kernel of=xv6.img seek=1 conv=notruncWe already traced bootblock in the previous article, so this time let’s focus on kernel.
kernel: $(OBJS) entry.o entryother initcode kernel.ld\n$(LD) $(LDFLAGS) -T kernel.ld -o kernel entry.o $(OBJS) -b binary initcode entryother\n$(OBJDUMP) -S kernel > kernel.asm\n$(OBJDUMP) -t kernel | sed '1,/SYMBOL TABLE/d; s/ .* / /; /^$$/d' > kernel.symThe dependencies for kernel are $(OBJS) entry.o entryother initcode kernel.ld.
The list of $(OBJS) is quite long, so I will skip it. It includes kernel modules such as main.o.
The last two lines only output the binary’s disassembly and symbol information; the actual binary is built by the line $(LD) $(LDFLAGS) -T kernel.ld -o kernel entry.o $(OBJS) -b binary initcode entryother.
LD is used in the form $(TOOLPREFIX)ld, just like the GCC described in the previous article.
Since we are not cross-compiling this time, the plain ld command is used.
LD = $(TOOLPREFIX)ld\n\n# FreeBSD ld wants ``elf_i386_fbsd''\nLDFLAGS += -m $(shell $(LD) -V | grep elf_i386 2>/dev/null | head -n 1)LDFLAGS extracts elf_i386 from the output of ld -V and passes it as the -m elf_i386 option.
ld -V is the version-check command for ld with an option that lists the supported emulators.
The actual command executed at build time looks like this:
\nThe -T option, like the -c option, reads link commands from a linker script (kernel.ld).
The -b option specifies the binary format of the subsequently listed input object files; here binary is specified.
The initcode and entryother that follow are binaries assembled from assembly files.
ld -m elf_i386 -T kernel.ld -o kernel \\\nentry.o bio.o console.o exec.o file.o fs.o ide.o ioapic.o kalloc.o kbd.o lapic.o log.o main.o mp.o picirq.o pipe.o proc.o sleeplock.o spinlock.o string.o swtch.o syscall.o sysfile.o sysproc.o trapasm.o trap.o uart.o vectors.o vm.o \\\n-b binary initcode entryotherReference: LD, the GNU linker - Options
\nNext, let’s look inside the linker script kernel.ld.
First, what is a linker script? It is a file that specifies the memory layout of objects when the linker links object files to produce an executable.
\nNormally the linker’s built-in default linker script is used, so you do not need to specify one explicitly.
\nIncidentally, the default linker script built into the linker can be printed by running ld with the --verbose option.
However, for programs like an OS or embedded systems where the general-purpose OS management facilities are unavailable, a custom linker script must be configured.
\nReference: Scripts (LD)
\nReference: Basic Script Concepts (LD)
\nReference: GNU Cを使いこなそう | 株式会社コンピューテックス
\nThe full linker script used to build the xv6 OS kernel is as follows.
\n/* Simple linker script for the JOS kernel.\n See the GNU ld 'info' manual (\"info ld\") to learn the syntax. */\n\nOUTPUT_FORMAT(\"elf32-i386\", \"elf32-i386\", \"elf32-i386\")\nOUTPUT_ARCH(i386)\nENTRY(_start)\n\nSECTIONS\n{\n/* Link the kernel at this address: \".\" means the current address */\n /* Must be equal to KERNLINK */\n. = 0x80100000;\n\n.text : AT(0x100000) {\n*(.text .stub .text.* .gnu.linkonce.t.*)\n}\n\nPROVIDE(etext = .);/* Define the 'etext' symbol to this value */\n\n.rodata : {\n*(.rodata .rodata.* .gnu.linkonce.r.*)\n}\n\n/* Include debugging information in kernel memory */\n.stab : {\nPROVIDE(__STAB_BEGIN__ = .);\n*(.stab);\nPROVIDE(__STAB_END__ = .);\n}\n\n.stabstr : {\nPROVIDE(__STABSTR_BEGIN__ = .);\n*(.stabstr);\nPROVIDE(__STABSTR_END__ = .);\n}\n\n/* Adjust the address for the data segment to the next page */\n. = ALIGN(0x1000);\n\n/* Conventionally, Unix linkers provide pseudo-symbols\n * etext, edata, and end, at the end of the text, data, and bss.\n * For the kernel mapping, we need the address at the beginning\n * of the data section, but that's not one of the conventional\n * symbols, because the convention started before there was a\n * read-only rodata section between text and data. */\nPROVIDE(data = .);\n\n/* The data segment */\n.data : {\n*(.data)\n}\n\nPROVIDE(edata = .);\n\n.bss : {\n*(.bss)\n}\n\nPROVIDE(end = .);\n\n/DISCARD/ : {\n*(.eh_frame .note.GNU-stack)\n}\n}The minimum required element in a linker script is the SECTIONS element.
A MEMORY element is often defined, but it is not required.
The SECTIONS element defines sections and places them at arbitrary addresses.
Both physical addresses and virtual addresses can be defined for these addresses.
\nReference: Mastering GNU C | Computex Co., Ltd.
\nReference: リンカスクリプトの書き方
\nThe simplest linker script with only the SECTIONS element looks like the following example.
SECTIONS\n{\n . = 0x10000;\n .text : { *(.text) }\n . = 0x8000000;\n .data : { *(.data) }\n .bss : { *(.bss) }\n}Reference: Simple Example (LD)
\nIn the xv6 OS linker script, the following sections are defined:
\n.text : Where the executable binary is placed. Typically read/execute permission only..rodata : Where read-only data is placed..stab : Where an array of fixed-length structures called stabs is placed..stabstr : Where variable-length strings referenced from stabs are placed..data : Where readable and writable data is placed..bss : Where block starting symbols (objects that are declared but have not yet been assigned a value) are placed.Reference: STABS - Using Stabs in Their Own Sections
\nReference: STABS: Stab Section Basics
\nReference: .bss - Wikipedia
\nLet me now walk through the contents of the linker script in order.
\nLooking at the first lines of the linker script, three things are defined.
\n/* Simple linker script for the JOS kernel.\n See the GNU ld 'info' manual (\"info ld\") to learn the syntax. */\n\nOUTPUT_FORMAT(\"elf32-i386\", \"elf32-i386\", \"elf32-i386\")\nOUTPUT_ARCH(i386)\nENTRY(_start)OUTPUT_FORMAT defines the format of the output binary.
OUTPUT_ARCH specifies the architecture that the output binary targets.
ENTRY specifies the symbol name of the function that will be executed first.
Reference: Entry Point (LD)
\nThe _start specified here is defined in entry.S as follows.
# By convention, the _start symbol specifies the ELF entry point.\n# Since we haven't set up virtual memory yet, our entry point is\n# the physical address of 'entry'.\n.globl _start\n_start = V2P_WO(entry)\n\n# Entering xv6 on boot processor, with paging off.\n.globl entry\nentry:\n # Turn on page size extension for 4Mbyte pages\n movl %cr4, %eax\n orl $(CR4_PSE), %eax\n movl %eax, %cr4\n # Set page directory\n movl $(V2P_WO(entrypgdir)), %eax\n movl %eax, %cr3\n # Turn on paging.\n movl %cr0, %eax\n orl $(CR0_PG|CR0_WP), %eax\n movl %eax, %cr0\n\n # Set up the stack pointer.\n movl $(stack + KSTACKSIZE), %esp\n\n # Jump to main(), and switch to executing at\n # high addresses. The indirect call is needed because\n # the assembler produces a PC-relative instruction\n # for a direct jump.\n mov $main, %eax\n jmp *%eax\n\n.comm stack, KSTACKSIZEThe details of entry.S will be described later.
Reference: xv6: OSはどうメモリを参照、管理するのか（前編） - yohei.codes
\nFirst, let’s look at the part that defines the text section.
\n/* Link the kernel at this address: \".\" means the current address */\n/* Must be equal to KERNLINK */\n. = 0x80100000;\n\n.text : AT(0x100000) {\n*(.text .stub .text.* .gnu.linkonce.t.*)\n}\n\nPROVIDE(etext = .);/* Define the 'etext' symbol to this value */The first line, . = 0x80100000;, sets the value of the special symbol ..
This is used as the location counter.
\nSections defined afterward start at the address pointed to by the location counter.
\nWhen a section is defined, the location counter is incremented by that section’s size.
\nReference: Simple Example (LD)
\nIn xv6 OS, the initial value of the location counter is set to 0x80100000.
This means that the instruction addresses in the binary produced by the linker start from 0x80100000.
Section definitions use the following structure:
\nsection [address] [(type)] :\n [AT(lma)]\n [ALIGN(section_align) | ALIGN_WITH_INPUT]\n [SUBALIGN(subsection_align)]\n [constraint]\n {\n output-section-command\n output-section-command\n …\n } [>region] [AT>lma_region] [:phdr :phdr …] [=fillexp] [,]Reference: Output Section Description (LD)
\nAT(0x100000) defines the load address of the section as 0x100000.
Reference: Using LD, the GNU linker - Section Options
\nTo be honest, I had no idea at all what *(.text .stub .text.* .gnu.linkonce.t.*) was doing, but it appears to be a line that defines the contents of the section.
There are several ways to define the contents, but the basic form is filename(symbol).
Multiple definitions can span multiple lines.
\nWhen * is used in place of a filename, as in *(), all object files provided at link time are targeted.
In other words, *(.text .stub .text.* .gnu.linkonce.t.*) is an instruction telling the linker to place the data from the .text .stub .text.* .gnu.linkonce.t.* sections of each input object file into the .text section of the output executable.
Reference: Using LD, the GNU linker - Section Placement
\nFiles such as entry.o and bio.o given as linker inputs are all compiled as 32-bit ELF format, so each of them has a header and a .text section.
That is why the definition above is needed — to merge them all into a single executable.
\nOnce the .text section definition is complete, we need to define etext to mark the end of the segment.
PROVIDE(etext = .);/* Define the 'etext' symbol to this value */Reference: Man page of END
\nHere, PROVIDE is used to set etext at the current location.
PROVIDE is a directive that creates a symbol only when that symbol is undefined in the code.
Reference: PROVIDE (LD)
\nNext, the .rodata section is defined.
rodata stands for Read Only Data.
.rodata : {\n*(.rodata .rodata.* .gnu.linkonce.r.*)\n}The linker definition uses the same syntax as the .text section, so I will skip the explanation.
Next, the debug-purpose stab sections are defined.
/* Include debugging information in kernel memory */\n.stab : {\nPROVIDE(__STAB_BEGIN__ = .);\n*(.stab);\nPROVIDE(__STAB_END__ = .);\n}\n\n.stabstr : {\nPROVIDE(__STABSTR_BEGIN__ = .);\n*(.stabstr);\nPROVIDE(__STABSTR_END__ = .);\n}ld does not create sections whose defined contents turn out to be empty.
In the default code, each binary does not have a .stab section, so there was no .stab section in kernel either.
However, I confirmed that adding -gstabs to the gcc compile options defined in the Makefile causes a .stab section to be created, and that section then appears in the linked kernel as well.
The .data section holds readable and writable data.
/* Adjust the address for the data segment to the next page */\n. = ALIGN(0x1000);\n\n/* Conventionally, Unix linkers provide pseudo-symbols\n* etext, edata, and end, at the end of the text, data, and bss.\n* For the kernel mapping, we need the address at the beginning\n* of the data section, but that's not one of the conventional\n* symbols, because the convention started before there was a\n* read-only rodata section between text and data. */\nPROVIDE(data = .);\n\n/* The data segment */\n.data : {\n*(.data)\n}\n\nPROVIDE(edata = .);First, the line . = ALIGN(0x1000); aligns the current location to a 0x1000 boundary.
This line is not assigning a specific address to the location counter like . = 0x80100000; did earlier.
Instead, ALIGN aligns the current location to the boundary of the specified value, starting from the current location at the time ALIGN is executed.
Looking at the actual generated kernel binary, we can see that the binary data was continuous up to 0x80107aa9, and then the starting address of the .data section becomes 0x80108000.
$ objdump -D kernel | grep -5 \"Disassembly of section .data:\"\n80107aa6:67 6e outsb %ds:(%si),(%dx)\n80107aa8:65 gs\n80107aa9:64 fs\n...\n\nDisassembly of section .data:\n\n80108000 <ctlmap>:\n...\n80108010:11 17 adc %edx,(%edi)\n80108012:05 12 14 19 15 add $0x15191412,%eaxThis is the result of aligning to the 0x1000 boundary from the point where the current location counter had been incremented to 0x80107aaa.
Reference: Using LD, the GNU linker - Arithmetic Functions
\nThe rest of the definitions are the same as those already covered, so I will skip them.
\nThe .bss section is defined as follows.
This is the same as what we have already seen, so I will skip the explanation.
\nPROVIDE(edata = .);\n.bss : {\n*(.bss)\n}\nPROVIDE(end = .);Sections listed under /DISCARD/ are not linked into the generated object.
/DISCARD/ : {\n*(.eh_frame .note.GNU-stack)\n}.eh_frame is a section generated by gcc that stores information for obtaining a stack backtrace.
.note.GNU-stack is used in Linux object files to declare stack attributes.
Next, let’s look at the _start function that was defined as the kernel’s entry point at link time.
The entry.S file where _start is defined contains the following code.
# The xv6 kernel starts executing in this file. This file is linked with\n# the kernel C code, so it can refer to kernel symbols such as main().\n# The boot block (bootasm.S and bootmain.c) jumps to entry below.\n \n# Multiboot header, for multiboot boot loaders like GNU Grub.\n# http://www.gnu.org/software/grub/manual/multiboot/multiboot.html\n#\n# Using GRUB 2, you can boot xv6 from a file stored in a\n# Linux file system by copying kernel or kernelmemfs to /boot\n# and then adding this menu entry:\n#\n# menuentry "xv6" {\n# insmod ext2\n# set root='(hd0,msdos1)'\n# set kernel='/boot/kernel'\n# echo "Loading ${kernel}..."\n# multiboot ${kernel} ${kernel}\n# boot\n# }\n\n#include "asm.h"\n#include "memlayout.h"\n#include "mmu.h"\n#include "param.h"\n\n# Multiboot header. Data to direct multiboot loader.\n.p2align 2\n.text\n.globl multiboot_header\nmultiboot_header:\n #define magic 0x1badb002\n #define flags 0\n .long magic\n .long flags\n .long (-magic-flags)\n\n# By convention, the _start symbol specifies the ELF entry point.\n# Since we haven't set up virtual memory yet, our entry point is\n# the physical address of 'entry'.\n.globl _start\n_start = V2P_WO(entry)\n\n# Entering xv6 on boot processor, with paging off.\n.globl entry\nentry:\n # Turn on page size extension for 4Mbyte pages\n movl %cr4, %eax\n orl $(CR4_PSE), %eax\n movl %eax, %cr4\n # Set page directory\n movl $(V2P_WO(entrypgdir)), %eax\n movl %eax, %cr3\n # Turn on paging.\n movl %cr0, %eax\n orl $(CR0_PG|CR0_WP), %eax\n movl %eax, %cr0\n\n # Set up the stack pointer.\n movl $(stack + KSTACKSIZE), %esp\n\n # Jump to main(), and switch to executing at\n # high addresses. The indirect call is needed because\n # the assembler produces a PC-relative instruction\n # for a direct jump.\n mov $main, %eax\n jmp *%eax\n\n.comm stack, KSTACKSIZEReading through entry.S from the top, we find the following code.
First, .p2align 2 on the first line aligns the binary to a 4-byte boundary.
Reference: P2align (Using as)
\nReference: gcc - What does .p2align do in asm code? - Stack Overflow
\nImmediately under the .text directive, multiboot_header is defined.
Here, the multiboot header is defined to support the multiboot specification.
\n# Multiboot header. Data to direct multiboot loader.\n.p2align 2\n.text\n.globl multiboot_header\nmultiboot_header:\n #define magic 0x1badb002\n #define flags 0\n .long magic\n .long flags\n .long (-magic-flags)The multiboot specification standardizes how a bootloader loads an x86 operating system kernel.
\nIn the previous article, I examined the xv6 OS bootloader code; if you want to boot the xv6 OS kernel with GRUB, for example, the kernel must comply with this multiboot specification.
\nBootloaders such as GRUB are adopted as the standard in Linux systems and others. (GRUB2 is normally used.)
\nReference: マルチブート仕様
\nReference: GRUBでOSを起動する - OSのようなもの
\nReference: Trying to Run a Simple OS Kernel with GRUB - Momoiro Technology
\nI am planning to actually try booting the xv6 OS with GRUB after I have finished reading through the kernel, so I will move on without tracing the code in detail.
\nThe next piece of code is as follows.
\n# By convention, the _start symbol specifies the ELF entry point.\n# Since we haven't set up virtual memory yet, our entry point is\n# the physical address of 'entry'.\n.globl _start\n_start = V2P_WO(entry)The .globl directive is a declaration that makes a symbol accessible from all linked files.
_start is the symbol that was referenced as the entry point from the linker script and elsewhere, and this declaration enables it to be called from outside entry.S.
Reference: .globl - Google Search
\nNext, let’s look at the line _start = V2P_WO(entry).
V2P_WO is the following macro defined in memlayout.h.
// Memory layout\n\n#define EXTMEM 0x100000 // Start of extended memory\n#define PHYSTOP 0xE000000 // Top physical memory\n#define DEVSPACE 0xFE000000 // Other devices are at high addresses\n\n// Key addresses for address space layout (see kmap in vm.c for layout)\n#define KERNBASE 0x80000000 // First kernel virtual address\n#define KERNLINK (KERNBASE+EXTMEM) // Address where kernel is linked\n\n#define V2P(a) (((uint) (a)) - KERNBASE)\n#define P2V(a) ((void *)(((char *) (a)) + KERNBASE))\n\n#define V2P_WO(x) ((x) - KERNBASE) // same as V2P, but without casts\n#define P2V_WO(x) ((x) + KERNBASE) // same as P2V, but without castsThis is simply a macro that takes an address as an argument and subtracts KERNBASE, which is set to 0x80000000.
Originally, the linker had linked the kernel’s .text section using 0x80100000 as its base.
This is a mechanism that separates the virtual memory ranges for user mode and kernel mode, allowing the CPU to load the kernel’s virtual addresses via x86 CPU paging.
\nReference: xv6: OSはどうメモリを参照、管理するのか（前編） - yohei.codes
\nHowever, at the point where _start = V2P_WO(entry) is executed, virtual memory has not yet been configured on the kernel side, so 0x80100000 is subtracted to assign the entry point _start to a physical address.
Let’s continue tracing the rest of entry.S.
First, .globl entry makes entry a symbol accessible from outside.
What entry does can be summarized simply: it loads the kernel’s virtual address using paging.
When the entry label is called, the paging mechanism has not yet been enabled, so the first thing we do here is enable it.
# Entering xv6 on boot processor, with paging off.\n.globl entry\nentry:\n # Turn on page size extension for 4Mbyte pages\n movl %cr4, %eax\n orl $(CR4_PSE), %eax\n movl %eax, %cr4\n # Set page directory\n movl $(V2P_WO(entrypgdir)), %eax\n movl %eax, %cr3\n # Turn on paging.\n movl %cr0, %eax\n orl $(CR0_PG|CR0_WP), %eax\n movl %eax, %cr0\n\n # Set up the stack pointer.\n movl $(stack + KSTACKSIZE), %esp\n\n # Jump to main(), and switch to executing at\n # high addresses. The indirect call is needed because\n # the assembler produces a PC-relative instruction\n # for a direct jump.\n mov $main, %eax\n jmp *%eax\n\n.comm stack, KSTACKSIZEBefore tracing the code, let me briefly summarize what paging is.
\nPaging is a method of managing memory by dividing it into fixed-size chunks called pages.
\nThis allows the divided memory regions to be treated as a linear address space. It also allows auxiliary storage devices such as SSDs to provide a virtual page area, making it possible to handle more memory than the physical RAM capacity.
\nIn paging, writing a page from main memory to auxiliary storage is called a “page-out,” while writing a page back from auxiliary storage to main memory is called a “page-in” or “swap-in.”
\nThrough the paging mechanism, unused memory regions are saved to auxiliary storage via page-out.
\nThe next time that memory region is needed, the OS raises an exception called a “page fault” for the address that does not exist in physical memory, and an interrupt triggers a swap-in to write the page back to physical memory.
\nReference: x86_64アーキテクチャ - ばびろん’s すたっく メモリアクセス
\nReference: x86_64アーキテクチャ - ばびろん’s すたっく メモリアクセス(続き)
\nReference: What Is Paging? - e-Words IT Dictionary
\nTo enable the paging mechanism on an x86 CPU, the PG flag of CR0 (Control Register 0) must be set to 1.
Let’s look at the part that actually enables paging.
\nIn the previous article, I set the PE flag of CR0 (Control Register 0) when transitioning to protected mode — the approach here is almost identical.
entry:\n # Turn on page size extension for 4Mbyte pages\n movl %cr4, %eax\n orl $(CR4_PSE), %eax\n movl %eax, %cr4\n # Set page directory\n movl $(V2P_WO(entrypgdir)), %eax\n movl %eax, %cr3\n # Turn on paging.\n movl %cr0, %eax\n orl $(CR0_PG|CR0_WP), %eax\n movl %eax, %cr0The processing after # Turn on paging. at the end is where the PG flag of CR0 (Control Register 0) is set.
The constants used for the flag operations are each defined as follows.
\n// Control Register flags\n#define CR0_PE 0x00000001 // Protection Enable\n#define CR0_WP 0x00010000 // Write Protect\n#define CR0_PG 0x80000000 // Paging\n\n#define CR4_PSE 0x00000010 // Page size extensionFrom this we can see that not only the PG flag but also the WP flag is being set.
\nWhen the WP flag is set, the CPU can prevent ring-0 supervisor-level procedures from writing to read-only pages.
\nThis makes it easier to implement the copy-on-write mechanism when creating new processes in the OS.
\nI will write about this in a future article.
\nHowever, I do wonder why it is being set explicitly here, since the WP flag should be set by default on x86 CPUs.
\nReference: Control register - Wikipedia
\nReference: assembly - whats the purpose of x86 cr0 WP bit? - Stack Overflow
\nNext, let’s look at the following code, which comes slightly before the CR0 setup.
\n# Turn on page size extension for 4Mbyte pages\nmovl %cr4, %eax\norl $(CR4_PSE), %eax\nmovl %eax, %cr4Here, the PSE flag of CR4 (Control Register 4) is being set.
This flag controls the size of a single page.
\nWhen the PSE flag of CR4 is not set (the default), the page size is 4 KiB.
\nConversely, when the PSE flag is set, the page size is extended to 4 MiB.
\nReference: Control register - Wikipedia
\nI will cover the detailed background of why two page sizes exist in a separate article if the opportunity arises.
\nWe now know that xv6 OS uses a page size of 4 MiB.
\nFinally, let’s look at the following:
\n# Set page directory\nmovl $(V2P_WO(entrypgdir)), %eax\nmovl %eax, %cr3The paging mechanism in xv6 OS is enabled on the very next line, so paging is not yet active at this point.
\nTherefore, the $(V2P_WO(entrypgdir)) macro is used to convert the address of entrypgdir to a physical address before writing it to CR3.
CR3 is a register used when the paging mechanism is active; the x86 CPU uses it to reference the page directory and page table and convert linear addresses to physical addresses.
\nentrypgdir is a struct array defined in main.c.
// main.c\npde_t entrypgdir[]; // For entry.S\n\n// The boot page table used in entry.S and entryother.S.\n// Page directories (and page tables) must start on page boundaries,\n// hence the __aligned__ attribute.\n// PTE_PS in a page directory entry enables 4Mbyte pages.\n\n__attribute__((__aligned__(PGSIZE)))\npde_t entrypgdir[NPDENTRIES] = {\n // Map VA's [0, 4MB) to PA's [0, 4MB)\n [0] = (0) | PTE_P | PTE_W | PTE_PS,\n \n // Map VA's [KERNBASE, KERNBASE+4MB) to PA's [0, 4MB)\n [KERNBASE>>PDXSHIFT] = (0) | PTE_P | PTE_W | PTE_PS,\n};The array size is NPDENTRIES, which is defined in mmu.h as 1024:
// Page directory and page table constants.\n#define NPDENTRIES 1024 // # directory entries per page directory\n#define NPTENTRIES 1024 // # PTEs per page table\n#define PGSIZE 4096 // bytes mapped by a pageentrypgdir has two elements.
Honestly, I only have a rough sense of what is happening here, but it appears to simply initialize the page directory entries.
\nFirst, the line (0) | PTE_P | PTE_W | PTE_PS, which is common to both elements, defines the following:
0 — set all bits to 0PTE_P — set presentPTE_W — set read/writePTE_PS — set the 4 MiB page size bitThe first element, [0] = (0) | PTE_P | PTE_W | PTE_PS,, initializes the 0th page directory entry to this value.
The next element initializes the KERNBASE>>PDXSHIFT = 0x80000000 >> 22 = 512nd page directory entry to this value.
This initialization appears to be used when paging is subsequently enabled and execution transfers to the main function.
\nReference: xv6: OSはどうメモリを参照、管理するのか（前編） - yohei.codes
\nReference: what does this code mean in xv6 entrypgdir? - Stack Overflow
\nFinally, the stack pointer is set up before transferring to the main function.
\n# Set up the stack pointer.\nmovl $(stack + KSTACKSIZE), %espKSTACKSIZE is defined as 4096 in param.h.
#define NPROC 64 // maximum number of processes\n#define KSTACKSIZE 4096 // size of per-process kernel stack\n#define NCPU 8 // maximum number of CPUs\n#define NOFILE 16 // open files per process\n#define NFILE 100 // open files per system\n#define NINODE 50 // maximum number of active i-nodes\n#define NDEV 10 // maximum major device number\n#define ROOTDEV 1 // device number of file system root disk\n#define MAXARG 32 // max exec arguments\n#define MAXOPBLOCKS 10 // max # of blocks any FS op writes\n#define LOGSIZE (MAXOPBLOCKS*3) // max data blocks in on-disk log\n#define NBUF (MAXOPBLOCKS*3) // size of disk block cache\n#define FSSIZE 1000 // size of file system in blocksSetting the stack pointer is necessary to transfer to C code, but I honestly could not fully understand this part.
\nThe reason is that the variable stack is defined in main.c, and at this point no value has been stored in it yet.
As a result, it appears to be defined as a .comm symbol, intended to be redefined later.
Reference: c - assembly - mov unitialized variable? - Stack Overflow
\nQuite tricky…
\nThe series of processing that began during bootstrap finally comes to an end here, and execution transfers to the main.c function, which is the kernel body.
# Jump to main(), and switch to executing at\n# high addresses. The indirect call is needed because\n# the assembler produces a PC-relative instruction\n# for a direct jump.\nmov $main, %eax\njmp *%eax\n\n.comm stack, KSTACKSIZEThis has gotten quite long, so I will continue in the next article.
\nIn this article, I traced through the kernel program build process, the linker script, and the flow of execution at the entry point.
\nNext time, we should finally be able to trace the behavior of the kernel body itself.
\n