Reading xv6OS Thoroughly to Fully Understand the Kernel - IOAPIC Edition -

This page has been machine-translated from the original page.

Inspired by An Introduction to OS Code Reading: Learning Kernel Internals with UNIX V6, I’m reading xv6 OS.

Because UNIX V6 itself does not run on x86 CPUs, I decided to read the source of kash1064/xv6-public: xv6 OS, a fork of the xv6 OS repository that makes UNIX V6 run on the x86 architecture.

In the previous article, I looked at the seginit function executed from main.

This time, I will trace the behavior of picinit and ioapicinit.

Table of Contents

• The picinit function

• The ioapicinit function

  • Accessing the memory-mapped IOAPIC register region
  • Reading and writing data via the Data register
  • Initializing the Redirection Table
• Summary
• Reference Books

The picinit function

The picinit function is defined in picirq.c.

It is a very small function, as shown below.

#include "types.h"
#include "x86.h"
#include "traps.h"

// I/O Addresses of the two programmable interrupt controllers
#define IO_PIC1         0x20    // Master (IRQs 0-7)
#define IO_PIC2         0xA0    // Slave (IRQs 8-15)

// Don't use the 8259A interrupt controllers.  Xv6 assumes SMP hardware.
void picinit(void)
{
  // mask all interrupts
  outb(IO_PIC1+1, 0xFF);
  outb(IO_PIC2+1, 0xFF);
}

//PAGEBREAK!
// Blank page.

outb is the following function defined in x86.h.

static inline void outb(ushort port, uchar data)
{
  asm volatile("out %0,%1" : : "a" (data), "d" (port));
}

It emits assembly that sends data to an arbitrary port.

Reference: assembly - what does “outb” in AT&T asm mean? - Stack Overflow

Here it masks the ports 0x21 and 0xA1, which are related to the 8259 PIC.

The 8259 PIC is an interrupt controller that accepts interrupt requests and sends them to the CPU, but xv6OS assumes SMP hardware and implements interrupts with APICs, so it has to be disabled.

Here, both 0x21 and 0xA1 are data ports of the 8259 PIC.

As described in the OSDev wiki below, when interrupts are implemented with the local APIC and IOAPIC, the 8259 PIC is disabled by writing 0xFF to these data ports.

Reference: 8259 PIC - OSDev Wiki

Reference: assembly - I/O Port Addressing - Stack Overflow

The ioapicinit function

The processing in picinit is now complete, and the PIC has been disabled.

Next, ioapicinit, which is defined in ioapic.c, is called from main.

This function initializes the IOAPIC.

void ioapicinit(void)
{
  int i, id, maxintr;

  ioapic = (volatile struct ioapic*)IOAPIC;
  maxintr = (ioapicread(REG_VER) >> 16) & 0xFF;
  id = ioapicread(REG_ID) >> 24;
  if(id != ioapicid) cprintf("ioapicinit: id isn't equal to ioapicid; not a MP\n");

  // Mark all interrupts edge-triggered, active high, disabled,
  // and not routed to any CPUs.
  for(i = 0; i <= maxintr; i++){
    ioapicwrite(REG_TABLE+2*i, INT_DISABLED | (T_IRQ0 + i));
    ioapicwrite(REG_TABLE+2*i+1, 0);
  }
}

Let’s look through the code in order.

Accessing the memory-mapped IOAPIC register region

In the line ioapic = (volatile struct ioapic*)IOAPIC;, the variable ioapic is set to point to an ioapic structure.

IOAPIC is defined as 0xFEC00000 in ioapic.c.

// ioapic.c
#define IOAPIC  0xFEC00000   // Default physical address of IO APIC

Because the default IOAPIC address is guaranteed to be 0xFEC00000, this address is specified as a fixed value.

Reference: x86 - About the IO-APIC 82093AA - Stack Overflow

The ioapic structure itself is very simple:

// IO APIC MMIO structure: write reg, then read or write data.
struct ioapic {
  uint reg;
  uint pad[3];
  uint data;
};

Reference: Memory Mapped Registers for Accessing IOAPIC Registers

Reference image: Intel 9 Series Chipset Platform Controller Hub Datasheet

The first member, reg, is used as an index, and data is then read or written through the data area.

I was curious about what values were actually stored in the ioapic region at boot, so I checked it in the debugger.

I looked at 20 bytes starting at 0xFEC00000, and at this point everything was zero.

$ x/5wx 0xfec00000
0xfec00000:0x000000000x000000000x000000000x00000000
0xfec00010:0x00000000

Reading and writing data via the Data register

Next, look at the following code.

maxintr = (ioapicread(REG_VER) >> 16) & 0xFF;
id = ioapicread(REG_ID) >> 24;
if(id != ioapicid) cprintf("ioapicinit: id isn't equal to ioapicid; not a MP\n");

The ioapicread function writes the received value to ioapic->reg, then reads and returns the value of data.

static uint ioapicread(int reg)
{
  ioapic->reg = reg;
  return ioapic->data;
}

In the IOAPIC, ioapic->reg is used as an index register, so data can be read and written indirectly.

The correspondence between index values and registers is as follows.

Reference image: Intel 9 Series Chipset Platform Controller Hub Datasheet

Reference: I/O APICについて - 睡分不足

The line maxintr = (ioapicread(REG_VER) >> 16) & 0xFF; gets version information by setting the index register to 1.

$ x/5wx 0xfec00000
0xfec00000:0x000000010x000000000x000000000x00000000
0xfec00010:0x00170020

maxintr becomes 0x17.

This seems to be retrieving the MRE field in bits 23:16 of the version register.

It looks like this value is the number of interrupt input pins minus one. (Perhaps this corresponds to the maximum number of interrupts that can be defined?)

Reference image: Intel 9 Series Chipset Platform Controller Hub Datasheet

In the next line, id was read as 0x0.

This is the value in bits 27:24 of the ID register, which represents the APIC ID. (Since the first one is 0, maybe the value is 0 on the first run?)

Reference image: Intel 9 Series Chipset Platform Controller Hub Datasheet

It then checks whether this value matches ioapicid.

As I saw in the multiprocessor article, ioapicid is the APIC number obtained from the MP configuration table.

Initializing the Redirection Table

Finally, ioapicwrite is used to disable all entries in the Redirection Table.

// Mark all interrupts edge-triggered, active high, disabled,
// and not routed to any CPUs.
for(i = 0; i <= maxintr; i++){
  ioapicwrite(REG_TABLE+2*i, INT_DISABLED | (T_IRQ0 + i));
  ioapicwrite(REG_TABLE+2*i+1, 0);
}

The ioapicwrite function writes data using the index and write value as follows.

static void ioapicwrite(int reg, uint data)
{
  ioapic->reg = reg;
  ioapic->data = data;
}

INT_DISABLED is defined as 0x00010000.

This sets bit 17 to 1.

Reference image: Intel 9 Series Chipset Platform Controller Hub Datasheet

This bit is the mask flag, and once it is set, interrupts are no longer delivered.

The value written in the low 8 bits is the interrupt vector.

I wonder why it disables them during initialization.

I assume it will become clear once I read more of the code, so I’ll keep going.

Summary

This time I read through picinit and ioapicinit.

Only ten functions executed from main remain unread, so the series is starting to hit the halfway point.

Next time I will read the consoleinit function.

Reference Books

• Build an OS in 30 Days!
• Introduction to OS Development from Zero
• An Introduction to OS Code Reading: Learning Kernel Internals with UNIX V6
• Detailed Linux Kernel
• Build and Understand an OS: Theory and Implementation for Running x86 Computers