Cosimulating: Using Bochs debugger to drive 2 simulators
Author: Kevin Lawton

# Welcome to the Bochs x86 PC Emulation Software Home Page!

# Cosimulating: using Bochs debugger to drive 2 simulators

## What is Cosimulation?

I'll add a discussion of cosimulation here soon.

## Cosimulation Programmatic Interface

This documentation is not yet complete, and this interface is subject to change.

### Callback Structure:

Upon startup, the cosimulation controller (debugger) will initialize each simulator (or emulator) by calling an init routine. There are 2 macro's in config.h (generated from config.h.in), which determine the names of the initialization routines, one for each simulator. These macro's are BX_SIM1_INIT and BX_SIM2_INIT. The first one likely can be kept as is. You will need to edit the second one, to be a routine in the second simulator which will receive a few parameters, and fill in a callback structure. This is the only routine in each simulator environment which is needed directly, as the callback routines which are filled in, are used subsequently. It should be of the type:

      void some_name_here(bx_dbg_callback_t *, int argc, char *argv[]);

The argc and argv parameters are as you might expect. They consist of any simulator specific command line options passed to the bochs executable at program invocation time. Since there can be more than one simulator, command line options are delimited by -sim1 and -sim2 as in:

      Usage: bochs [-rc path] [-sim1 ... ] [-sim2 ...]

The first parameter is the address of a callback structure, whose type bx_dbg_callback_t is defined in bx_debug/debug.h. Most fields are function pointers, though some are not. Below, is a description of each field in the callback structure.

      typedef struct {
        bx_bool (*setphymem)(Bit32u addr, unsigned len, Bit8u *buf);
        bx_bool (*getphymem)(Bit32u addr, unsigned len, Bit8u *buf);
        void    (*xlate_linear2phy)(Bit32u linear, Bit32u *phy, bx_bool *valid);
        bx_bool (*set_reg)(unsigned reg, Bit32u val);
        Bit32u  (*get_reg)(unsigned reg);
        bx_bool (*set_cpu)(bx_dbg_cpu_t *cpu);
        bx_bool (*get_cpu)(bx_dbg_cpu_t *cpu);
        unsigned       dirty_page_tbl_size;
        unsigned char *dirty_page_tbl;
        void    (*atexit)(void);
        unsigned (*query_pending)(void);
        void     (*execute)(void);
        void     (*take_irq)(void);
        void     (*take_dma)(void);
        void     (*reset_cpu)(unsigned source);
        void     (*init_mem)(int size_in_bytes);
        void     (*load_ROM)(const char *path, Bit32u romaddress);

        void     (*set_A20)(unsigned val);
        void     (*set_NMI)(unsigned val);
        void     (*set_RESET)(unsigned val);
        void     (*set_INTR)(unsigned val);
        void     (*force_interrupt)(unsigned vector);

      #if BX_INSTRUMENTATION
        void    (*instr_start)(void);
        void    (*instr_stop)(void);
        void    (*instr_reset)(void);
        void    (*instr_print)(void);
      #endif
      #if BX_USE_LOADER
        void    (*loader)(char *path);
      #endif
        } bx_dbg_callback_t;

bx_bool (*setphymem)(Bit32u addr, unsigned len, Bit8u *buf);

Set (write to) physical memory of simulator at address addr from the len bytes in buf. The bytes in buf should be copied to the simulator's physical memory byte-at-a-time with no concern for endian-ness. Return 1 if the write is OK, 0 if an error occurs.

bx_bool (*getphymem)(Bit32u addr, unsigned len, Bit8u *buf);

Get (read from) physical memory of simulator at address addr to the len bytes in buf. The bytes in buf should be copied from the simulator's physical memory byte-at-a-time with no concern for endian-ness. Return 1 if the read is OK, 0 if an error occurs.

void (*xlate_linear2phy)(Bit32u linear, Bit32u *phy, bx_bool *valid);

Translate a linear to a physical address, without generating an exception or updating the paging tables. The debugger passes the simulator the linear address. The simulator is expected to set phy to the corresponding physical address if available, and update valid, which should be set to 1 if the physical address was available, and 0 if not. It is possible, the address is not in the paging tables, and thus not available.

bx_bool (*set_reg)(unsigned reg, Bit32u val);

Set a specific CPU register as determined by reg in the simulator to val. The value of reg will be one of the defines, such as BX_DBG_REG_EAX defined in bx_debug/debug.h. The segment registers can only be set by this method in real mode. This function should return a 1 if the operation is successful, and 0 if not.

Bit32u (*get_reg)(unsigned reg);

Return the value of a specific CPU register in the simulator as determined by reg, whose value is of the same range as those passed into set_reg()

bx_bool (*set_cpu)(bx_dbg_cpu_t *cpu);

Set all the registers in the CPU simulator to those in the structure cpu. The bx_dbg_cpu_t structure is defined in bx_debug/debug.h.

bx_bool (*get_cpu)(bx_dbg_cpu_t *cpu);

Get values for all the registers in the CPU simulator, and place them in the structure cpu. The bx_dbg_cpu_t structure is defined in bx_debug/debug.h.

unsigned dirty_page_tbl_size;
unsigned char *dirty_page_tbl;

To keep track of what areas of memory have been written to, and allow the debugger to efficiently compare regions of memory in each simulator, each simulator should provide a dirty page table. It is expected that each byte in the array represents one 4K page of physical memory. A value of 1 represents that the page has been written to, since the last time it was cleared by the debugger, and a value of 0 represents that no write has occurred within that physical page.

You should fill in the field dirty_page_tbl with a pointer to an array of bytes. And dirty_page_tbl_size should be set to the size of the array in bytes. If possible, make the array big enough to span the largest amount of physical memory you might request. I chose to not make this size dynamic with the user's requested memory size, to eliminate an extra access via a pointer.

void (*atexit)(void);

Called when the debugger needs to terminate execution. This function should close files and do all necessary shutdown of the simulator. NOTE: Do not call exit() directly from the simulator, but instead call bx_dbg_exit(). As there may be multiple simulators, bx_dbg_exit() will handle invoking the atexit callback in each simulator first, and then it will call exit().

void (*execute)(void);

The debugger calls execute() in either slave or master simulator, commanding either to execute instructions until a guard is reached, in which case control should be returned back to the debugger.

void (*reset_cpu)(unsigned source);

This function is called once by the debugger to initialize the simulator's CPU upon program startup. Source will be either BX_RESET_HARDWARE or BX_RESET_SOFTWARE.

void (*init_mem)(int size_in_bytes);

This function is called once by the debugger to initialize the simulator's memory upon program startup. Size_in_bytes denotes the size of physical memory the user has requested, so that both simulators can use the same physical memory size. This would be an appropriate time to allocate memory for the simulator.

void (*load_ROM)(const char *path, Bit32u romaddress);

Commands the simulator to load a ROM image stored in the filename path into physical memory at address romaddress.

void (*set_A20)(unsigned val);

This function is called by the cosim controller to command either master or slave simulator to change it's A20 setting. If the value of val passed is 1, then the A20 address line is passed through and used. This will give normal addressing of the entire 32bit address space. If the value is 0, then A20 is masked out, and addressing at the 1Meg boundary will wrap, modeling 8086 addressing.

The cosim controller also commands the bx_pc_system class to maintain the same A20 state as passed in this function, so you may alternatively use values directly from that class, rather than keep your own A20 state. If so, set the value of set_A20 to NULL, and use the following members from bx_pc_system. In this case, set_A20 won't be called.

- bx_pc_system.enable_a20: Same value as passed to set_A20.
- bx_pc_system.a20_mask: Logical AND this with a physical address to obtain the address after application of the A20 gate.

void (*set_NMI)(unsigned val);

Not supported yet. Will tell the simulator that the value of the NMI pin, is currently val. Use an empty stub function for this for now.

void (*set_RESET)(unsigned val);

Not supported yet. Will tell the simulator that the value of the RESET pin, is currently val. Use an empty stub function for this for now.

void (*set_INTR)(unsigned val);

The INTR pin is driven by the device models. When the INTR pin is raised due to an interrupt request by the PIC, or lowered after the interrupt is acknowledged, this function is called to notify the simulator of the new status. Only the master simulator will receive notification, as INTR is always 0 for the slave simulator. Interrupts are forced in the slave simulator, synchronizing it to the execution path of the master, using force_interrupt() described below. A value in val of 0 indicates no interrupt is requested. A value of 1 indicates an interrupt request.

void (*force_interrupt)(unsigned vector);

In order for the debugger to force the slave simulator to take an interrupt at the same point as the master simulator, the interrupt vector is recorded when taken by the master simulator. The debugger commands the slave to the same point, and calls this routine, forcing the slave to take the given interrupt, vector.

void (*instr_start)(void); (Only defined if macro BX_INSTRUMENTATION is 1)

Called when the user types in "instrument start" at the debug prompt. The instrumentation package can use this function to do whatever is necessary to initialize the instrumentation package and/or command it to begin collecting data.

void (*instr_stop)(void); (Only defined if macro BX_INSTRUMENTATION is 1)

Called when the user types in "instrument stop" at the debug prompt. The instrumentation package can use this function to do whatever is necessary to temporarily or permanently stop the instrumentation package from collecting data.

void (*instr_reset)(void); (Only defined if macro BX_INSTRUMENTATION is 1)

Called when the user types in "instrument reset" at the debug prompt. The instrumentation package can use this function to command the instrumentation package to reset it's data collection mechanisms.

void (*instr_print)(void); (Only defined if macro BX_INSTRUMENTATION is 1)

Called when the user types in "instrument print" at the debug prompt. The instrumentation package can use this function to output it's collected data.

void (*loader)(char *path); (Only defined if macro BX_USE_LOADER is 1)

Called when the user types in "loader pathname" at the debug prompt. The idea is to allow a loader routine to read in a program which is specific to the OS you are running within the emulator, from a file on your native workstation, load it properly into simulator memory and run it on the simulator. This loader must be specific to the OS you are running within the simulator, and I do not provide one with bochs.

void (*take_irq)(void);
void (*take_dma)(void);
unsigned (*query_pending)(void);

These are vestiges of a past interface. They correspond to the "take irq", "take dma", and "query pending" commands, which you shouldn't use. They will be removed. Set these fields to NULL, or to empty stub functions.

### Debugger (Cosimulation Controller) Functions:

void bx_dbg_exit(int code)

When there is a situation in the simulator, where you need to terminate due to an unrecoverable error (panic), call bx_dbg_exit(). Among other things, this function will call the at_exit callback function in each simulator, and ultimately call the system exit() function.

Bit8u bx_dbg_IAC(void)

The simulator's CPU code should call this function when it is acknowledging an interrupt from the PIC via the INTR line. The interrupt vector number from the PIC is returned.

Bit32u bx_dbg_inp(Bit16u addr, unsigned len)

To read data from an IO device, the simulator should call this function. Pass in the IO address addr, and the size of the IO operation len.

void bx_dbg_outp(Bit16u addr, Bit32u value, unsigned len)

To write data to an IO device, the simulator should call this function. Pass in the IO address addr, and the size of the IO operation len.

Bit8u bx_dbg_ucmem_read(Bit32u addr)
void bx_dbg_ucmem_write(Bit32u addr, Bit8u value)

For memory read/write accesses which fall in the range of 0xA0000 to 0xBFFFF, the accesses should not be to directed to the simulator's memory, since these are UnCacheable MEMory addresses. The VGA adapter maps it's memory to this range. Instead, call these functions to perform reads/writes to memory accesses in this range. For bx_dbg_ucmem_read(), pass the physical address addr, and the value of the read is returned. For bx_dbg_ucmem_write(), pass the physical address addr and value value of the write.

void bx_dbg_async_pin_ack(unsigned what, bx_bool val)

In order for the master and slave simulators to accept changes in pins such as the A20 line, at the same point, the debugger provides a mechanism for pending the pin change, until it is acknowledged by the master simulator. The place where the change is ack'd, is recorded by the debugger. This information is used to run the slave simulator, forcing it to accept the changes at the same locale as did the master.

Initially, the IO devices call a function bx_dbg_async_pin_request(), not listed here, to record the pin change as pending. The pending status is recorded along with the guard information in bx_guard.async_changes_pending.which. This field contains a binary OR'd set of pending pin changes. Currently only A20 is supported, which is represented by the macro BX_DBG_ASYNC_PENDING_A20.

At a time prudent to your CPU simulator, check to see if there are any pending changes, that the CPU should acknowledge. If so, acknowledge them by calling bx_dbg_async_pin_ack(). The pending value of the A20 enable is stored in bx_guard.async_changes_pending.a20. Here is some sample code which performs this task, that you can insert into the appropriate place in your CPU simulator.

      if (bx_guard.async_changes_pending.which) {
        if (bx_guard.async_changes_pending.which & BX_DBG_ASYNC_PENDING_A20)
          bx_dbg_async_pin_ack(BX_DBG_ASYNC_PENDING_A20,
                               bx_guard.async_changes_pending.a20);
        // ...other checks here when they are supported
        }

The bx_dbg_async_pin_ack() function will in turn, invoke the set_A20() callback function in the master simulator, so you don't have to deal with updating local A20 state in your simulator here, as long as you handle it in set_A20(). Keep in mind, the slave simulator will never see the code inside this sample code if-construct, since changes are forced in the slave by the debugger at points where the master simulator acknowledged them, not as a direct effect of the IO devices.

### Guards:

Guards are a mechanism by which the debugger requests each simulator to stop execution and return control back to the debugger. The debugger runs each simulator for a particular number of instructions, or until certain events occur. Guards are set by the debugger, and it is up to each simulator to examine them upon and during execution of the execute() callback, and return control back to the debugger when the guard criteria are met.

Guard information set by the debugger is stored in global structure bx_guard of type bx_guard_t. For reference, it's declaration is shown here, followed by an explanation of the purpose of each field. Information about the guard encountered by the simulator, and which caused control to return to the debugger is stored in the global structure bx_guard_found[] of type bx_guard_found_t. This is actually an array of structures, where bx_guard_found[0] is the first simulator with ID 0, and bx_guard_found[1] is the second simulator with ID 1. This structure is also declared below, and the text explains the information which should be returned in this structure based on the guard encountered.

    typedef struct {
      unsigned long guard_for;

      // instruction address breakpoints
      struct {
    #if BX_DBG_SUPPORT_VIR_BPOINT
        unsigned num_virtual;
        struct {
          Bit32u cs;  // only use 16 bits
          Bit32u eip;
          unsigned bpoint_id;
          } vir[BX_DBG_MAX_VIR_BPOINTS];
    #endif

    #if BX_DBG_SUPPORT_LIN_BPOINT
        unsigned num_linear;
        struct {
          Bit32u addr;
          unsigned bpoint_id;
          } lin[BX_DBG_MAX_LIN_BPOINTS];
    #endif

    #if BX_DBG_SUPPORT_PHY_BPOINT
        unsigned num_physical;
        struct {
          Bit32u addr;
          unsigned bpoint_id;
          } phy[BX_DBG_MAX_PHY_BPOINTS];
    #endif
        } iaddr;

      bx_dbg_icount_t icount; // stop after completing this many instructions

      // user typed Ctrl-C, requesting simulator stop at next convient spot
      volatile bx_bool interrupt_requested;

      // booleans to control whether simulator should report events
      // to debug controller
      struct {
       bx_bool irq;
       bx_bool a20;
       bx_bool io;
       bx_bool ucmem;
       bx_bool dma;
       } report;

      struct {
        bx_bool irq;  // should process IRQs asynchronously
        bx_bool dma;  // should process DMAs asynchronously
        } async;

    #define BX_DBG_ASYNC_PENDING_A20   0x01
    #define BX_DBG_ASYNC_PENDING_RESET 0x02
    #define BX_DBG_ASYNC_PENDING_NMI   0x04

      // Asynchronous changes which are pending.  These are Q'd by
      // the debugger, as the master simulator is notified of a pending
      // async change.  At the simulator's next point, where it checks for
      // such events, it notifies the debugger with acknowlegement.  This
      // field contains a logically or'd list of all events which should
      // be checked, and ack'd.
      struct {
        unsigned which; // logical OR of above constants
        bx_bool a20;
        bx_bool reset;
        bx_bool nmi;
        } async_changes_pending;
      } bx_guard_t;

    typedef struct {
      unsigned long guard_found;
      unsigned iaddr_index;
      bx_dbg_icount_t icount; // number of completed instructions
      Bit32u   cs;     // cs:eip and linear addr of instruction at guard point
      Bit32u   eip;
      Bit32u   laddr;
      bx_bool  is_32bit_code; // CS seg size at guard point
      bx_bool  ctrl_c; // simulator stopped due to Ctrl-C request
      } bx_guard_found_t;

    extern bx_guard_t        bx_guard;
    extern bx_guard_found_t  bx_guard_found[];

### bx_guard_found[]:

It is the task of each simulator to update the bx_guard_found structure. There are some fields which are specific to the type of guard in question, and you should update those when a particular guard is encountered. Those fields are explained in more detail in the section relating to the specific guard. There are some fields which are updated for every case, no matter what the guard is. Below is a list and explanation of the usage of each field.

unsigned long guard_found; this should be filled in with the particular guard encountered, for example if an instruction count guard is hit, set this to BX_DBG_GUARD_ICOUNT.

unsigned iaddr_index; This field is updated, whenever a virtual/linear/physical instruction address guard is hit. It is the array index into the bx_guard.iaddr.vir[], bx_guard.iaddr.lin[], or bx_guard.iaddr.phy[] arrays, whichever is appropriate.

bx_dbg_icount_t icount; This contains the number of instructions which have been completely executed, when the guard was encountered.

Bit32u cs;
Bit32u eip;
Bit32u laddr;
bx_bool is_32bit_code; These all relate to the same instruction address. From the debugger's point of view, instruction addresses can be only at the beginning of the instruction. Once an instruction is completed, use the address of the next instruction. Set cs and eip to the instruction's address (CS:EIP). Set laddr to the instruction's corresponding linear address. Set is_32bit_code to the size (0=16bit, 1=32bit) of the code segment when the guard is encountered. This is used for disassembly.

bx_bool ctrl_c; To allow the user to interrupt a simulator from the debug prompt, the debugger traps Ctrl-C interrupts, and sets bx_guard.interrupt_requested. Your simulator can optionally look for this, provided that the BX_DBG_GUARD_CTRL_C bit is set in bx_guard.guard_for structure. If you chose to do so, you may look for this occurrance whenever is convenient. Set ctrl_c to 1 to signify this guard has occurred. Here' some sample code to demonstrate this:

        // convenient point to see if user typed Ctrl-C
        if (bx_guard.interrupt_requested && (bx_guard.guard_for & BX_DBG_GUARD_CTRL_C)) {
          bx_guard_found[BX_SIM_ID].guard_found = BX_DBG_GUARD_CTRL_C;
          return; // some mechanism to return control here
          }

### bx_guard:

unsigned long guard_for;

This is a binary OR'd list of guards the debugger is requesting each simulator to stop on. Only if the corresponding bit is set in this field, should the simulator examine the rest of the criteria for that guard. Currently, each simulator must be capable of recognizing the following guards, and returning to the debugger when they occur:

- BX_DBG_GUARD_ICOUNT: Instruction count.
- BX_DBG_GUARD_CTRL_C: User requested interrupt via Ctrl-C
- BX_DBG_GUARD_IADDR_VIR: Stop on this virtual instruction address
- BX_DBG_GUARD_IADDR_LIN: Stop on this linear instruction address
- BX_DBG_GUARD_IADDR_PHY: Stop on this physical instruction address

struct { .. } iaddr;

This structure holds the guard information for instruction address guards (breakpoints). Depending upon your selections, after editing config.h in the main directory (generated by running ./configure), certain types of instruction address guards are supported. Which ones, are determined by the BX_DBG_SUPPORT_VIR_BPOINT, BX_DBG_SUPPORT_LIN_BPOINT, and BX_DBG_SUPPORT_PHY_BPOINT macros.

If the guard_for field contains a set bit represented by BX_DBG_GUARD_IADDR_VIR, then the iaddr.num_virtual field holds the number of virtual instruction address guards to examine and compare to the current address. For each, you must examine the CS:EIP values stored in iaddr.vir[n], in the cs and eip subfields. If there is a match, record this in the guard found structure, and return control to the debugger:

    bx_guard_found[ID].guard_found = BX_DBG_GUARD_IADDR_VIR;
    bx_guard_found[ID].iaddr_index = n; // array index in bx_guard.iaddr.vir[]
    bx_guard_found[ID].icount = .. // number of completed instructions
    bx_guard_found[ID].cs = .. // CS selector value
    bx_guard_found[ID].eip = .. // EIP value
    bx_guard_found[ID].laddr = .. // linear address of CS:EIP
    bx_guard_found[ID].is_32bit_code = .. // 0=16bit code, 1=32bit code
    // return control here

If the guard_for field contains a set bit represented by BX_DBG_GUARD_IADDR_LIN, then the iaddr.num_linear field holds the number of linear instruction address guards to examine and compare to the current address. For each, you must examine the linear address values stored in iaddr.lin[n], in the addr subfield. If there is a match, record this in the guard found structure, and return control to the debugger:

    bx_guard_found[ID].guard_found = BX_DBG_GUARD_IADDR_LIN;
    bx_guard_found[ID].iaddr_index = n; // array index in bx_guard.iaddr.lin[]
    bx_guard_found[ID].icount = .. // number of completed instructions
    bx_guard_found[ID].cs = .. // CS selector value
    bx_guard_found[ID].eip = .. // EIP value
    bx_guard_found[ID].laddr = .. // linear address of CS:EIP
    bx_guard_found[ID].is_32bit_code = .. // 0=16bit code, 1=32bit code
    // return control here

If the guard_for field contains a set bit represented by BX_DBG_GUARD_IADDR_PHY, then the iaddr.num_physical field holds the number of physical instruction address guards to examine and compare to the current address. For each, you must examine the physical address values stored in iaddr.phy[n], in the addr subfield. If there is a match, record this in the guard found structure, and return control to the debugger:

    bx_guard_found[ID].guard_found = BX_DBG_GUARD_IADDR_PHY;
    bx_guard_found[ID].iaddr_index = n; // array index in bx_guard.iaddr.phy[]
    bx_guard_found[ID].icount = .. // number of completed instructions
    bx_guard_found[ID].cs = .. // CS selector value
    bx_guard_found[ID].eip = .. // EIP value
    bx_guard_found[ID].laddr = .. // linear address of CS:EIP
    bx_guard_found[ID].is_32bit_code = .. // 0=16bit code, 1=32bit code
    // return control here

volatile bx_bool interrupt_requested;

If the debugger has turned on the guard for a user interrupt, and the user has indeed requested one (Ctrl-C), the debugger will set this field to 1. The simulator should record this in the guard found information, and return control back to the debugger. Look above at the explanation for the bx_guard.interrupt_requested field for some sample code on how to do this.

struct { .. } async;

struct { .. } async_changes_pending;

Related Links:

- Debugger
- Instrumentation