{"componentChunkName":"component---src-templates-post-template-js","path":"/diyos-mikanos-01-en","result":{"data":{"markdownRemark":{"id":"b7a23c63-d5f0-509c-b949-a0e4ed1a1a1e","html":"
\n

This page has been machine-translated from the original page.

\n
\n

I want to retry Introduction to OS Development from Scratch, which I bought as soon as it was released but left untouched because I couldn’t find the time and got discouraged by the difficulty.

\n

Basically, I plan to proceed mainly by typing in the code, and then write up articles about the points where I get stuck or where my understanding is still shallow.

\n

This time, I’ll cover environment setup through Chapter 2.\nBy the way, I’m skipping Chapter 1 because it’s mostly just doing the exercises.

\n\n

Table of Contents

\n\n

Environment setup

\n

I built the development environment with the following setup.

\n\n

The plan is to do development by connecting from VSCode to the Ubuntu instance running on Hyper-V via Remote SSH, and only log in to Ubuntu with the GUI when I need to check behavior in Qemu.

\n

Some of the QEMU commands explained in the book require 3D acceleration to be enabled.\nIf (qemu) gtk initialization failed occurs in a virtual desktop environment, check whether 3D acceleration is enabled.

\n

For the environment setup, I’m using the following GitHub repository published by the author as the base.

\n

Reference: Build and run scripts for MikanOS

\n

If your Ubuntu version is different, the steps may not work exactly as written and the amount of trouble increases, so if possible I think it’s best to prepare Ubuntu 20.04 in a virtual machine or WSL.

\n

I’ve summarized how to connect from VSCode to Ubuntu on a virtual machine via Remote SSH here.

\n

Reference: How to edit files directly on a virtual machine running on Hyper-V from VSCode - Yukituna’s Bookshelf

\n

Creating a UEFI application with EDK II

\n

Right from the beginning, it jumped into a technical area I knew absolutely nothing about.

\n

Reference: GitHub - tianocore/edk2: EDK II

\n

The .EFI file created here seems to be a UEFI application that is called from UEFI BIOS.\nMy understanding is that EDK II is something like an SDK for creating UEFI applications.

\n
struct MemoryMap {\n  UINTN buffer_size;\n  VOID* buffer;\n  UINTN map_size;\n  UINTN map_key;\n  UINTN descriptor_size;\n  UINT32 descriptor_version;\n};\n\nEFI_STATUS GetMemoryMap(struct MemoryMap* map) {\n  if (map->buffer == NULL) {\n    return EFI_BUFFER_TOO_SMALL;\n  }\n\n  map->map_size = map->buffer_size;\n  return gBS->GetMemoryMap(\n      &map->map_size,\n      (EFI_MEMORY_DESCRIPTOR*)map->buffer,\n      &map->map_key,\n      &map->descriptor_size,\n      &map->descriptor_version);\n}
\n

I didn’t really understand what it was doing, so for the time being I followed the code.\nHere, GetMemoryMap() stores the memory map in a MemoryMap structure named memmap.

\n

GetMemoryMap() uses gBS->GetMemoryMap() to obtain the memory map at the time the function is called.

\n

The second argument, given as (EFI_MEMORY_DESCRIPTOR*)map->buffer, is where the function stores the memory map it retrieves.\nThe stored data is defined as the EFI_MEMORY_DESCRIPTOR structure in edk2/MdePkg/Include/Uefi/UefiSpec.h.

\n
///\n/// Memory descriptor version number.\n///\n#define EFI_MEMORY_DESCRIPTOR_VERSION 1\n\n///\n/// Definition of an EFI memory descriptor.\n///\ntypedef struct {\n  ///\n  /// Type of the memory region.\n  /// Type EFI_MEMORY_TYPE is defined in the\n  /// AllocatePages() function description.\n  ///\n  UINT32                Type;\n  ///\n  /// Physical address of the first byte in the memory region. PhysicalStart must be\n  /// aligned on a 4 KiB boundary, and must not be above 0xfffffffffffff000. Type\n  /// EFI_PHYSICAL_ADDRESS is defined in the AllocatePages() function description\n  ///\n  EFI_PHYSICAL_ADDRESS  PhysicalStart;\n  ///\n  /// Virtual address of the first byte in the memory region.\n  /// VirtualStart must be aligned on a 4 KiB boundary,\n  /// and must not be above 0xfffffffffffff000.\n  ///\n  EFI_VIRTUAL_ADDRESS   VirtualStart;\n  ///\n  /// NumberOfPagesNumber of 4 KiB pages in the memory region.\n  /// NumberOfPages must not be 0, and must not be any value\n  /// that would represent a memory page with a start address,\n  /// either physical or virtual, above 0xfffffffffffff000.\n  ///\n  UINT64                NumberOfPages;\n  ///\n  /// Attributes of the memory region that describe the bit mask of capabilities\n  /// for that memory region, and not necessarily the current settings for that\n  /// memory region.\n  ///\n  UINT64                Attribute;\n} EFI_MEMORY_DESCRIPTOR;
\n

In the end, the memory map obtained here was saved by SaveMemoryMap() as a CSV file named memmmap.\nRun the following commands directly under edk2/Build/OSLoaderX64/DEBUG_CLANG38/X64/OSLoaderPkg/Loader/DEBUG, where the built image file is created, to inspect the contents of the image file.

\n
mkdir -o mnt\nsudo mount -o loop disk.img mnt\ncat mnt/memmap\n\n#//\n\nunmount mnt
\n

For now, if I can at least confirm the contents of this memory map, that means I’ve finished the material up through Chapter 2.

\n
Index, Type, Type(name), PhysicalStart, NumberOfPages, Attribute\n0, 3, EfiBootServicesCode, 00000000, 1, F\n1, 7, EfiConventionalMemory, 00001000, 9F, F\n2, 7, EfiConventionalMemory, 00100000, 700, F\n3, A, EfiACPIMemoryNVS, 00800000, 8, F\n4, 7, EfiConventionalMemory, 00808000, 8, F\n5, A, EfiACPIMemoryNVS, 00810000, F0, F\n6, 4, EfiBootServicesData, 00900000, B00, F\n7, 7, EfiConventionalMemory, 01400000, 3AB36, F\n8, 4, EfiBootServicesData, 3BF36000, 20, F\n9, 7, EfiConventionalMemory, 3BF56000, 270C, F\n10, 1, EfiLoaderCode, 3E662000, 2, F\n11, 4, EfiBootServicesData, 3E664000, 219, F\n12, 3, EfiBootServicesCode, 3E87D000, B7, F\n13, A, EfiACPIMemoryNVS, 3E934000, 12, F\n14, 0, EfiReservedMemoryType, 3E946000, 1C, F\n15, 3, EfiBootServicesCode, 3E962000, 10A, F\n16, 6, EfiRuntimeServicesData, 3EA6C000, 5, F\n17, 5, EfiRuntimeServicesCode, 3EA71000, 5, F\n18, 6, EfiRuntimeServicesData, 3EA76000, 5, F\n19, 5, EfiRuntimeServicesCode, 3EA7B000, 5, F\n20, 6, EfiRuntimeServicesData, 3EA80000, 5, F\n21, 5, EfiRuntimeServicesCode, 3EA85000, 7, F\n22, 6, EfiRuntimeServicesData, 3EA8C000, 8F, F\n23, 4, EfiBootServicesData, 3EB1B000, 4DA, F\n24, 7, EfiConventionalMemory, 3EFF5000, 4, F\n25, 4, EfiBootServicesData, 3EFF9000, 6, F\n26, 7, EfiConventionalMemory, 3EFFF000, 1, F\n27, 4, EfiBootServicesData, 3F000000, A1B, F\n28, 7, EfiConventionalMemory, 3FA1B000, 1, F\n29, 3, EfiBootServicesCode, 3FA1C000, 17F, F\n30, 5, EfiRuntimeServicesCode, 3FB9B000, 30, F\n31, 6, EfiRuntimeServicesData, 3FBCB000, 24, F\n32, 0, EfiReservedMemoryType, 3FBEF000, 4, F\n33, 9, EfiACPIReclaimMemory, 3FBF3000, 8, F\n34, A, EfiACPIMemoryNVS, 3FBFB000, 4, F\n35, 4, EfiBootServicesData, 3FBFF000, 201, F\n36, 7, EfiConventionalMemory, 3FE00000, 8D, F\n37, 4, EfiBootServicesData, 3FE8D000, 20, F\n38, 3, EfiBootServicesCode, 3FEAD000, 20, F\n39, 4, EfiBootServicesData, 3FECD000, 9, F\n40, 3, EfiBootServicesCode, 3FED6000, 1E, F\n41, 6, EfiRuntimeServicesData, 3FEF4000, 84, F\n42, A, EfiACPIMemoryNVS, 3FF78000, 88, F\n43, 6, EfiRuntimeServicesData, FFC00000, 400, 1
\n

About building the image

\n

I didn’t really understand the part where the created EFI file is built and then emulated in Qemu, because there was not much explanation in the book, so I wrote up my understanding here.

\n

In the book, the EFI file is built by using the build command directly under the edk2 directory.

\n

The build target information is written in edk2/Conf/target.txt.

\n

This time, I used a folder called OSLoaderPkg as the workspace for the UEFI application I created myself.

\n
#  PROPERTY              Type       Use         Description\n#  ----------------      --------   --------    -----------------------------------------------------------\n#  ACTIVE_PLATFORM       Filename   Recommended Specify the WORKSPACE relative Path and Filename\n#                                               of the platform description file that will be used for the\n#                                               build. This line is required if and only if the current\n#                                               working directory does not contain one or more description\n#                                               files.\nACTIVE_PLATFORM       = OSLoaderPkg/OSLoaderPkg.dsc
\n

Here, OSLoaderPkg.dsc is defined as follows.

\n
#@range_begin(defines)\n[Defines]\n  PLATFORM_NAME                  = OSLoaderPkg\n  PLATFORM_GUID                  = d3f11f4e-71e9-11e8-a7e1-33fd4f7d5a3e\n  PLATFORM_VERSION               = 0.1\n  DSC_SPECIFICATION              = 0x00010005\n  OUTPUT_DIRECTORY               = Build/OSLoader$(ARCH)\n  SUPPORTED_ARCHITECTURES        = X64\n  BUILD_TARGETS                  = DEBUG|RELEASE|NOOPT\n#@range_end(defines)\n\n#@range_begin(library_classes)\n[LibraryClasses]\n  UefiApplicationEntryPoint|MdePkg/Library/UefiApplicationEntryPoint/UefiApplicationEntryPoint.inf\n  UefiLib|MdePkg/Library/UefiLib/UefiLib.inf\n#@range_end(library_classes)\n\n  BaseLib|MdePkg/Library/BaseLib/BaseLib.inf\n  BaseMemoryLib|MdePkg/Library/BaseMemoryLib/BaseMemoryLib.inf\n  DebugLib|MdePkg/Library/BaseDebugLibNull/BaseDebugLibNull.inf\n  DevicePathLib|MdePkg/Library/UefiDevicePathLib/UefiDevicePathLib.inf\n  MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf\n  PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf\n  PrintLib|MdePkg/Library/BasePrintLib/BasePrintLib.inf\n  UefiBootServicesTableLib|MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf\n  UefiRuntimeServicesTableLib|MdePkg/Library/UefiRuntimeServicesTableLib/UefiRuntimeServicesTableLib.inf\n\n#@range_begin(components)\n[Components]\n  OSLoaderPkg/Loader.inf\n#@range_end(components)
\n

With this setting, the EFI file built by EDK II is placed in OUTPUT_DIRECTORY = Build/OSLoader$(ARCH), so it ends up being saved under edk2/Build/OSLoaderX64/DEBUG_CLANG38/X64/OSLoaderPkg/Loader/DEBUG/.

\n

Next, by running osbook/devtools/run_qemu.sh in the same directory as Loader.efi—edk2/Build/OSLoaderX64/DEBUG_CLANG38/X64/OSLoaderPkg/Loader/DEBUG/—the built EFI file is emulated in Qemu.

\n

The contents of osbook/devtools/run_qemu.sh looked like this. It calls make_image.sh using the execution directory information passed in as arguments.

\n
#!/bin/sh -ex\n\nif [ $# -lt 1 ]\nthen\n    echo \"Usage: $0 <.efi file> [another file]\"\n    exit 1\nfi\n\nDEVENV_DIR=$(dirname \"$0\")\nEFI_FILE=$1\nANOTHER_FILE=$2\nDISK_IMG=./disk.img\nMOUNT_POINT=./mnt\n\n$DEVENV_DIR/make_image.sh $DISK_IMG $MOUNT_POINT $EFI_FILE $ANOTHER_FILE\n$DEVENV_DIR/run_image.sh $DISK_IMG
\n

Let’s look at the contents of make_image.sh.

\n
#!/bin/sh -ex\n\nif [ $# -lt 3 ]\nthen\n    echo \"Usage: $0 <image name> <mount point> <.efi file> [another file]\"\n    exit 1\nfi\n\nDEVENV_DIR=$(dirname \"$0\")\nDISK_IMG=$1\nMOUNT_POINT=$2\nEFI_FILE=$3\nANOTHER_FILE=$4\n\nif [ ! -f $EFI_FILE ]\nthen\n    echo \"No such file: $EFI_FILE\"\n    exit 1\nfi\n\nrm -f $DISK_IMG\nqemu-img create -f raw $DISK_IMG 200M\nmkfs.fat -n 'JISAKU OS' -s 2 -f 2 -R 32 -F 32 $DISK_IMG\n\n$DEVENV_DIR/mount_image.sh $DISK_IMG $MOUNT_POINT\nsudo mkdir -p $MOUNT_POINT/EFI/BOOT\nsudo cp $EFI_FILE $MOUNT_POINT/EFI/BOOT/BOOTX64.EFI\nif [ \"$ANOTHER_FILE\" != \"\" ]\nthen\n    sudo cp $ANOTHER_FILE $MOUNT_POINT/\nfi\nsleep 0.5\nsudo umount $MOUNT_POINT
\n

Looking at this script, if disk.img already exists it removes it, then creates a 200 MB image file in raw format with qemu-img create -f raw $DISK_IMG 200M.

\n

Reference: QEMU-img

\n

Next, mkfs.fat formats it as an MS-DOS filesystem, mounts it under the mnt directory, and then copies the created EFI file to EFI/BOOT/BOOTX64.EFI.

\n

Reference: Ubuntu Manpage: mkfs.fat - create an MS-DOS filesystem under Linux

\n

This image file is then used, and run_image.sh emulates it with the qemu-system-x86_64 command.

\n
qemu-system-x86_64 \\\n    -m 1G \\\n    -drive if=pflash,format=raw,readonly,file=$DEVENV_DIR/OVMF_CODE.fd \\\n    -drive if=pflash,format=raw,file=$DEVENV_DIR/OVMF_VARS.fd \\\n    -drive if=ide,index=0,media=disk,format=raw,file=$DISK_IMG \\\n    -device nec-usb-xhci,id=xhci \\\n    -device usb-mouse -device usb-kbd \\\n    -monitor stdio \\\n    $QEMU_OPTS
\n

Summary

\n

This is just for my own study, so I’m skipping a detailed explanation.

\n

I’d like to keep going all the way to the end.

","fields":{"slug":"/diyos-mikanos-01-en","tagSlugs":["/tag/自作os/","/tag/みかん本/","/tag/os-en/","/tag/english/"]},"frontmatter":{"date":"2021-09-25","description":"I want to retry Introduction to OS Development from Scratch, which I bought as soon as it was released but left untouched because I couldn't find the time and got discouraged by the difficulty. This time I'll cover environment setup through Chapter 2.","tags":["自作OS","みかん本","OS (en)","English"],"title":"Notes from Working Through Introduction to OS Development from Scratch: Environment Setup Through Chapter 2","socialImage":{"publicURL":"/static/dc4d8b7f8795f3c3d3489d9957d155f2/no-image.png"}}}},"pageContext":{"slug":"/diyos-mikanos-01-en"}},"staticQueryHashes":["251939775","401334301","825871152"]}