% The file must begin with this \documentclass declaration. You can 
% give one of three different options which control how picky LaTeX 
% is when typesetting:
%
% galley - All ``this doesn't fit'' warnings are suppressed, and 
%          references are disabled (the key will be printed as a
%          reminder).  Use this mode while writing.
%
% proof -  All ``this doesn't fit'' warnings are active, as are
%          references.  Overfull hboxes make ugly black blobs in
%          the margin.  Use this mode to tidy up formatting after
%          you're done writing.  (Same as article's ``draft'' mode.)
%
% final -  As proof, but the ugly black blobs are turned off.  Use
%          this to render PDFs or PostScript to give to other people,
%          when you're completely done.  (As with article, this is the
%          default.)
%
% You can also use the leqno, fleqn, or openbib options to article.cls
% if you wish.  None of article's other options will work.

%%%
%%% PLEASE CHANGE 'galley' to 'final' BEFORE SUBMITTING.  THANKS!
%%% (to submit: "make clean" in the toplevel directory; tar and gzip *only* your directory;
%%% email the gzipped tarball to papers@linuxsymposium.org.)
%%%
\documentclass[final]{ols}

% These two packages allow easy handling of urls and identifiers per the example paper.
\usepackage{url}
\usepackage{zrl}

% The following package is not required, but is a handy way to put PDF and EPS graphics
% into your paper using the \includegraphics command.
\ifpdf
\usepackage[pdftex]{graphicx}
\else
\usepackage{graphicx}
\fi


% Here in the preamble, you may load additional packages, or
% define whatever macros you like, with the following exceptions:
%
% - Do not mess with the page layout, either by hand or with packages
%   (e.g., typearea, geometry).
% - Do not change the principal fonts, either by hand or with packages.
% - Do not use \pagestyle, or load any page header-related packages.
% - Do not redefine any commands having to do with article titles.
% - If you are using something that is not part of the standard
%   tetex-2 distribution, please make a note of whether it's on CTAN,
%   or include a copy with your submission.
%

% email address is split into macros
\providecommand{\XFthA}{thockin}
\providecommand{\XFthDomA}{google.com}

\begin{document}

% Mandatory: article title specification.
% Do not put line breaks or other clever formatting in \title or
% \shortauthor; these are moving arguments.

\title{Linux, Open Source, and System Bring-Up Tools}
\subtitle{How to make bring-up hurt less}  % Subtitle is optional.
\date{}             % You can put a fixed date in if you wish,
                    % allow LaTeX to use the date of typesetting,
                    % or use \date{} to have no date at all.
                    % Whatever you do, there will not be a date
                    % shown in the proceedings.

\shortauthor{Tim Hockin}  % Just you and your coauthors' names.
% for example, \shortauthor{A.N.\ Author and A.\ Nother}
% or perchance \shortauthor{Smith, Jones, Black, White, Gray, \& Greene}

\author{%  Authors, affiliations, and email addresses go here, like this:
Tim Hockin \\
{\itshape Google, Inc}\\
{\ttfamily\normalsize {\XFthA}{@}{\XFthDomA}}\\
} % end author section

\maketitle

\begin{abstract}
% Article abstract goes here.
\input{hockin-abstract.tex}
\end{abstract}

% Body of your article goes here.  You are mostly unrestricted in what
% LaTeX features you can use; however, the following will not work:
% \thispagestyle
% \marginpar
% table of contents
% list of figures / tables
% glossaries
% indices

\section*{Introduction}

  Sometimes you get lucky on a bring-up, and things just work the way they
  are supposed to.  More often than not, though, something goes wrong.
  Unfortunately, it's usually many ``somethings'' that go wrong.  When things
  do go wrong, someone has to figure out what happened and how to fix it.

  Platforms today are vastly more complicated than they were just a few
  years back.  Almost nothing works when the system powers on.  It all needs to
  be configured.  When the inevitable ``something'' goes wrong, determining
  the cause can be an overwhelming task.

  Of course, there are no magic bullets, but there are tools that can help to
  make some of these problems easier.

\section{Terminology}

  Before diving in, it's important that we are all speaking the same
  language:

  \textbf{Platform}: Sometimes used as a synonym for motherboard, a
  platform is really the combination of components that make up a
  computer system.  This includes the CPU or CPU family, the memory
  controller, the IO controller, the DRAM, the IO devices, and usually the
  system firmware.

  \textbf{Bring-up}: The process of evolving a platform from an expensive
  ``objet d'art'' into a fully operational computer system.  This process
  usually involves debugging and/or working around the hardware,
  configuring the system in BIOS, and hacking the drivers and kernel into
  shape.  It often includes superstitious rituals, cynical prayers, and
  lots of cussing.

  \textbf{BIOS}: Basic Input Output System.  The BIOS is the software that
  executes when a PC powers on, and is primarily responsible for
  configuring the hardware.

  \textbf{Device}: A piece of hardware that is logically self-contained.
  While a typical southbridge is a single chip, it is usually viewed as a
  collection of devices, such as disk controllers, network interfaces, and
  bridges.

  \textbf{Chipset}: A hardware chip or chips that provide the bulk of the
  IO on a platform.  Chipsets are typically tested and sold as a single
  unit.  These generally include a \textit{north bridge} which contains one or
  more memory controllers as well as high-speed IO bridges, and a \textit{south
  bridge} which contains lower-speed devices such as storage and legacy
  bus interfaces.

  \textbf{Register}: An addressable set of bits exposed by a device.  Most
  devices contain many registers.  Registers generally hold control and
  status bits for the device, and can be mapped into a multitude of
  address spaces such as PCI config space, memory, or IO space.

\section{Serial Console For The Unwashed Masses}

  An obvious place to start is to get the BIOS output as it boots.  Just
  about anyone who has ever booted up a PC has seen the BIOS output on the
  screen.  This is, however, not very useful.  Most servers do not have a
  monitor plugged in to them at all times.  VGA-capable chips, while not
  particularly high-tech, are not free to buy or run.  Why require one on
  every server?

  It is a sad fact that many platforms available today \textbf{still} do
  not have serial console support.  Those that do offer it usually offer
  it as an up-sell on the BIOS, and the implementation quality is often
  questionable.

  Some implementations provide side-band interfaces, which only get used to
  print certain information.  This is not particularly useful to anyone, and
  is fortunately not seen much any more.  Some implementations do what is
  called \textit{screen scraping} which depends on a real VGA device with
  real VGA memory to store the screen contents.  They periodically scan
  the VGA memory and send updates on the serial port.  Some
  implementations support text output but completely break down in the
  face of ``advanced'' features like cursor movement or color.

\subsection{Solving It Once And For All}

  In order to provide a consistent feature set, one Google engineer chose
  to solve this once and (hopefully) for all.  Thus was born
  \ident{SGABIOS} --- the \textit{Serial Graphics Adapter}.
  \ident{SGABIOS} is a stand-alone option ROM which can be loaded on a
  platform to provide serial console support.  It provides a robust set of
  VGA-compatible features which allow most BIOS output to be converted to
  serial-safe output.  It supports basic cursor movement, color, text
  input, and large serial consoles.

  The easiest way to use \ident{SGABIOS} is to make your BIOS load it as
  an option ROM.  You can try to convince your board vendor to include it
  as an option ROM in the BIOS build, or you can use tools (usually
  provided by the BIOS vendor) to load an option ROM into a BIOS image.
  If this is not an option for you, all is not lost.  There are
  commercially available add-in debug cards which have option-ROM sockets.
  In a pinch, many network and other cards have programmable ROMs which
  can be made to load an arbitrary option ROM.

  When started, \ident{SGABIOS} attempts to detect if there is a terminal
  attached.  If detected, \ident{SGABIOS} will adapt its internal
  structures to the detected terminal size.  \ident{SGABIOS} then traps
  \texttt{INT 10h}, the legacy ``print something'' BIOS function, and
  \texttt{INT 16h}, the legacy ``read keyboard'' function.  The final result
  is that any well-behaved BIOS, option ROM, or legacy OS will now be
  using the serial port transparently.  However, there are some badly
  behaved programs which attempt to write to VGA memory directly.
  \ident{SGABIOS} can not fix those applications.  Fortunately, this does
  not seem to be a very big problem.

  We have successfully run \ident{SGABIOS} with LILO and GRUB, as well as
  DOS.  It works wonderfully for the uses we have found, though it does
  have its limitations.  Some applications, such as LILO, query
  \texttt{INT 10h} for previously displayed data.  Because there is no VGA
  memory backing it, \ident{SGABIOS} only stores a small amount of the
  most recently printed output.   This has been good enough to handle the
  applications we have found to do this, but it does have the potential to
  fail.  As with so many things, it is a memory size vs. functionality
  tradeoff.

  You can find \ident{SGABIOS} at \url{http://sgabios.googlecode.com}.

\section{Simple Access To Registers}

  A recurring situation in my office is that you can boot, but something is
  not right.  You might have some ideas on what it could be, but you need
  to run some additional tests.  You need to modify some registers.

  You could have the board vendor build some test BIOSes with the various
  settings.  That's not going to be an effective, scalable, or timely
  solution.

  You could build a custom kernel which programs the desired changes; at
  least you control that part.  It's still a pretty heavy weight answer, and
  the hardware test team are not really kernel hackers.  This approach is
  better than the last one, but not good.

  One might ask ``Hold on, doesn't the kernel expose some APIs that let me
  fiddle with registers?''  Why yes, it does.  Now you only have to write
  some simple programs to do these tests.  But again, the test team are
  not really C programmers.  There must be something simpler.

\subsection{Introducing Iotools}

  A simple, scriptable interface to device registers allows anyone who can
  do basic programming to deal with this problem.  Almost anyone is now
  able to trivially read and write registers, thereby enabling a whole new
  debugging army.

  This is the goal of \ident{iotools}.  The \ident{iotools} package
  provides a suite of simple command line tools which enable various forms
  of register accesses.  They are mostly thin wrappers around Linux kernel
  interfaces such as sysfs and device nodes.  \ident{Iotools} also
  includes a number of simple logical operation tools, which make
  manipulating register data easier.

  The \ident{iotools} ``suite'' is actually a single binary, a la busybox.
  This allows for simple distribution and installation on target systems.
  The \ident{iotools} binary is less than 30 kilobytes in size when built
  with shared libraries.  Building it as a static binary obviously
  increases the size, depending on the libc it is linked against.  This
  should make \ident{iotools} suitable for use in most size-sensitive
  environments, such as flash or initramfs.

  A note of caution is warranted.  Writing to registers on a running system
  can crash the system.  You should always understand exactly what you are
  changing, and whether there might be a kernel driver managing those same
  registers.  Sometimes it is enough to simply unload a driver before making
  your changes.  Other times you just have to go for it.

\subsection{What's In Iotools?}

  At the time of writing, the \ident{iotools} suite includes tools to
  access the following register spaces:

    \begin{itemize}
    \item \textbf{PCI}: Read and write registers in PCI config space.
    This includes both traditional config space (256 bytes per device) and
    extended config space (4 Kbytes per device) for those devices which
    support it.  Access is provided by \texttt{sysfs} or \texttt{procfs}
    and is supported as 8 bit, 16 bit, and 32 bit operations.

    \item \textbf{IO}: Read and write registers in x86 IO ports.  This
    covers the 64 Kbyte space only.  Access is provided by \texttt{IN} and
    \texttt{OUT} instructions and is supported as 8 bit, 16 bit and 32 bit
    operations.

    \item \textbf{MMIO}: Read and write memory-mapped registers or
    physical memory.  This provides access to the entire 64 bit physical
    memory space via \texttt{/dev/mem}.  It supports 8 bit, 16 bit, and 32
    bit operations.

    \item \textbf{MSR}: Read and write x86 model specific registers on any
    CPU.  This provides access to the full 32 bit MSR space via
    \texttt{/dev/cpu/*/msr}.  It supports only 64 bit operations (all MSRs
    are 64 bits).

    \item \textbf{TSC}: Read the CPU timestamp counter on the current CPU.
    This is provided by the \texttt{RDTSC} instruction and is always a 64 bit
    operation.

    \item \textbf{CPUID}: Read data from the \texttt{CPUID} instruction on
    any CPU.  This provides access to the full 32 bit CPUID space via
    \texttt{/dev/cpu/*/cpuid}.

    \item \textbf{SMBus}: Read and write registers on \ident{SMBus}
    devices.  This is provided by the \texttt{/dev/i2c-*} drivers and
    supports 8 bit, 16 bit, and block operations.

    \item \textbf{CMOS}: Read and write legacy CMOS memory.  Most PCs have
    around 100 bytes of non-volatile memory that is accessed via the
    real-time clock.  Access is provided by the \texttt{/dev/nvram}
    driver, and only supports 8 bit operations.  This should be used with
    caution.  CMOS memory is often used by the system BIOS, and changing
    it can have unintended side-effects.
    \end{itemize}

  In addition to the register access tools, \ident{iotools} also includes
  several tools to perform logical operations on numbers.  These tools are
  important because they support 64 bit operations and treat all numbers
  as unsigned, which can be a problem in some shell environments.

    \begin{itemize}
    \item \textbf{AND}: Produce the logical AND of all arguments.

    \item \textbf{OR}: Produce the logical inclusive OR of all arguments.

    \item \textbf{XOR}: Produce the logical exclusive OR of all arguments.

    \item \textbf{NOT}: Produce the bitwise NOT of the argument.

    \item \textbf{SHL}: Shift the argument left by a specified number of bits,
      zero-filling at the right.

    \item \textbf{SHR}: Shift the argument right by a specified number of bits,
      zero-filling at the left (no sign extension).
    \end{itemize}

\subsection{A Simple Example}
\label{hockin-iotools-example}

  Suppose you need to test the behavior of enabling \ident{SERR} reporting
  on your platform.  This is controlled by bit 8 of the 16 bit register at
  offset 4 of each PCI device.  You could whip up a quick script:

  \begin{small}
  \begin{verbatim}
#!/bin/bash

function set_serr {
    # SERR is bit8 (0x100) of
    #     16 bit register 0x4
    OLD=$(pci_read16 $1 $2 $3 0x4)
    NEW=$(or $OLD 0x100)
    pci_write32 $1 $2 $3 4 $NEW
}

# hardcoded list of PCI addresses
set_serr 0 0 0
set_serr 0 0 1
set_serr 0 0 2
  \end{verbatim}
  \end{small}

  You can do better than this, though.  You can trivially make this script
  loop for each PCI device:

  \begin{small}
  \begin{verbatim}
#!/bin/bash

function set_serr {
  # SERR is bit8 (0x100) of
  #     16 bit register 0x4
  OLD=$(pci_read16 $1 $2 $3 0x4)
  NEW=$(or $OLD 0x100)
  pci_write32 $1 $2 $3 4 $NEW
}

# for each bus, dev, func
for B in $(seq 0 255); do
  for D in $(seq 0 31); do
    for F in $(seq 0 7); do
      pci_read32 $B $D $F 0 \
          >/dev/null 2>&1
      if [ $? != 0 ]; then
        # does not exist
        continue;
      fi
      set_serr $B $D $F
    done
  done
done
  \end{verbatim}
  \end{small}

  This version takes a bit longer to run, but works regardless of the
  devices in the system.  You can shorten the run time significantly by
  putting a sane upper bound on the number of buses.  Few systems have
  more than 20 or 30 buses, even in this era of point-to-point PCI
  Express buses.

  This is the sort of tool that someone with very basic shell scripting
  skills can produce in just a few minutes with \ident{iotools}.

  You can find \ident{iotools} at \url{http://iotools.googlecode.com}.

\section{Making It Simpler}

  The previous section shows just one example of the sorts of problems that
  arise during bring-up.  Frankly, it wasn't a particularly complicated
  problem, and the solution is bordering on real programming.  Worse than
  that, it requires that the person doing the work remember several
  ``magic''
  numbers.  Which register is this bit in?  How wide is that register?
  Which bit is it?  Taken further, the problem quickly becomes very
  difficult.

  Suppose you want to examine or configure something more complicated,
  like PCI Express advanced error reporting (AER).  AER is a
  \textit{capability} in PCI terminology.  That means that some devices
  will support it and some will not.  The only way to find out is to ask
  each device.  Further, each device might put the AER registers at a
  different offset in their PCI register set.  As if that is not enough,
  some devices have different AER register layouts, depending on what kind
  of device they are and which version of the the specification they
  support.

  Doing this in an \ident{iotools} script is certainly possible; it just
  isn't so simple anymore.  Google needed something that internalizes and
  hides even more of the details.  This gave rise to \ident{prettyprint}.

\subsection{An Unfortunate Name}

  The original goal of \ident{prettyprint} was this: to dump the state of
  all the registers in the system in a diff-friendly format.  This would
  allow us to use one of our favorite debug tools, which we call
  \textit{``Did you try rolling back the BIOS?''}  Boot with BIOS A,
  \texttt{prettyprint} the system.  Boot with BIOS B, \texttt{prettyprint}
  the system.  Then \texttt{diff} the results.

  Like the previous examples, there are other ways of doing this.  They
  all resulted in a screenful of numbers, followed by a few hours of
  digging through datasheets to find what each bit of each differing
  register means.  The only thing worse than going through this process
  and finding that the difference is undocumented is going through this
  process multiple times.

  Instead, \ident{prettyprint} attaches a datatype to field values,
  allowing it to produce output which is not only diff-friendly, but which
  is also human-friendly.

\subsection{Fundamentals Of Prettyprint}

  \ident{Prettyprint} has two fundamental constructs: \textit{registers} 
  and \textit{fields}.  In keeping with the common vernacular, a register
  is a single addressable set of bits.  Registers have a defined, fixed
  width, but they have no intrinsic meaning.

  Fields, on the other hand, are of arbitrary width and are the only
  entity with meaning.  Fields can be defined as a set of register bits
  (\textit{regbits}), constant bits, or even as procedures.  Every field
  has a datatype, and the result of reading a field is a value that can be
  evaluated against that datatype to produce a human readable string.

\subsection{The Power Of Fields}

  Let's look at a the simple example from section
  \ref{hockin-iotools-example}.  For each PCI device there is a 16 bit
  register at offset 4 called \ident{%command} (the \ident{%} is a
  convention to indicate a name is register).  For each PCI device there
  is also a field called \ident{serr}.  This field is exactly 1 bit wide,
  and is composed of bit 8 of \ident{%command}.  When accessing this
  field, one can interpret its value as a boolean, where a value of 1 =
  \textit{"yes"} and a value of 0 = \textit{"no"}.

  \begin{center}
  \includegraphics[width=0.8\columnwidth]{diagram1}
  \linebreak[3]
  \small{Figure 1: A simple field}
  \end{center}

  Now, when you dump the state of a device, you can see a line item that
  says \texttt{serr: yes}.

  This is vastly more useful than a hexadecimal number about which I have
  to remember that bit 8 being set means \ident{SERR} is enabled.  Even
  better, since I now have a system that understands \ident{serr}
  directly, I can write to it just as easily as I can read from it.

\subsection{Binding Fields To Devices}

  The previous example glossed over the details of ``for each PCI device''.
  This is a key aspect of \ident{prettyprint}'s power.  Registers are
  defined in an abstract way, divorced of exactly which device or access
  method they employ.  They simply have \textit{addresses}.  When it comes
  time to use these registers, \ident{prettyprint} passes control to the
  drivers which enable each class of device.  A \textit{binding} is used
  to map which abstract registers belong to which driver.

  When starting up, \ident{prettyprint} can find hardware devices in one
  of two ways.  Firstly, you can tell it where a device is found.  This is
  the only option for some devices, especially legacy devices.  For
  example, to tell \ident{prettyprint} about the serial port, you would
  have to tell it something to the effect of \textit{``there exists a
  serial port in IO space, at address 0x3f8''}.  In so doing, you have
  given \ident{prettyprint} enough information to bind the serial port
  registers and fields to a driver and address.

  Better still, you can let \ident{prettyprint} discover some devices.
  Many modern devices can be discovered either through the hardware
  itself, such as PCI, or through simple interfaces, such as ACPI.  In
  this case, the driver has a discovery routine which will find devices
  and bind them as it finds them.  This is how we are able to define
  things like \ident{serr} as something that exists ``for each PCI
  device''.

\subsection{About The Implementation}

  \ident{Prettyprint} is written in C++.  I can hear the cries of
  frustration already.  Why C++?  Because I thought that the problem
  decomposed nicely into an object-oriented model, and because I wanted to
  improve my C++.

  \ident{Prettyprint} has been designed from the start as a library to be
  used as a backend by various applications.  From state dumping utilities
  to interactive shells to \ident{FUSE} filesystems, anything is possible.

\subsection{Defining Registers And Fields}

  So how does one go about defining a device?  One of the things that the
  choice of C++ brought to the project was a way to manipulate the language
  syntax.  The end goal is to have an actual interpreted language which is
  used to define devices.  Until then, we have a set of C++ classes,
  functions, and templates which define a pseudo-language.

  This pseudo language is intended to make the definition of registers and
  fields as simple as possible.  Let's look at the \ident{SERR} example:

  \begin{small}
  \begin{verbatim}
REG16("%command", 0x04);
FIELD("serr", "yesno_t",
    BITS("%command", 8));
  \end{verbatim}
  \end{small}

  That's pretty straight-forward.  We define \ident{%command} as a 16 bit
  register at address 0x04.  We define \ident{serr} as a field with
  datatype \ident{yesno_t}, composed of bit 8 from \ident{%command}.

  Frequently, a field maps directly to a register.  To simplify this,
  \ident{prettyprint} understands \textit{regfields}.  For example, the
  PCI ``intpin'' field is the only consumer of the \ident{%intpin} register.

  \begin{center}
  \includegraphics[width=0.5\columnwidth]{diagram2}
  \linebreak[3]
  \small{Figure 2: A regfield}
  \end{center}

  We can express that as:

  \begin{small}
  \begin{verbatim}
REG8("%intpin", 0x3d);
FIELD("intpin", "int_t",
    BITS("%intpin", 7, 0));
  \end{verbatim}
  \end{small}

  Or we can take the equivalent regfield shortcut:

  \begin{small}
  \begin{verbatim}
REGFIELD8("intpin", 0x3d, "int_t");
  \end{verbatim}
  \end{small}

  Let's consider a more complicated example.  In a PCI-PCI bridge, there are
  several registers which control the address ranges which are decoded by
  the bridge.  They are implemented as 2 different registers, which combine to
  form a logical 64 bit address.  The low 20 bits of both the \ident{base}
  and \ident{limit} register are fixed to 0 and 1 respectively.

  \begin{center}
  \includegraphics[width=0.8\columnwidth]{diagram3}
  \linebreak[3]
  \small{Figure 3: Complex fields}
  \end{center}

  In \ident{prettyprint}, this is expressed as:

  \begin{small}
  \begin{verbatim}
REG16("%base_lo", 0x24);
REG32("%base_hi", 0x28);
REG16("%limit_lo", 0x26);
REG32("%limit_hi", 0x2c);

FIELD("base", "addr64_t",
      BITS("%base_hi", 31, 0) +
      BITS("%base_lo", 15, 4) +
      BITS("%0", 19, 0));
FIELD("limit", "addr64_t",
      BITS("%limit_hi", 31, 0) +
      BITS("%limit_lo", 15, 4) +
      BITS("%1", 19, 0));
  \end{verbatim}
  \end{small}

  Notice the use of \ident{%0} and \ident{%1} as registers.  These are the
  \textit{magic registers}.  When read, \ident{%0} always returns all
  logic 0 bits.  Likewise, \ident{%1} always returns all logic 1 bits.
  Also notice that the bits in a field are defined from most significant
  to least significant.  A field can be arbitrarily long, and can be
  composed of any number of regbits.

\subsection{Scopes And Paths}

  The examples so far have been relatively small.  In reality the
  \ident{%command} register has a number of fields that derive from it.
  All told, there are thousands of fields in each PCI device.
  \ident{prettyprint} provides \textit{scopes} as a mechanism for grouping
  related things together.

  Think of scopes like directories in a filesystem.  Each scope has a name
  and a set of contents.  A scope can contain registers, fields, or other
  scopes.  Like the filesystem metaphor, \ident{prettyprint} has paths.
  There is a conceptual \textit{root} of the path tree, and each register,
  field, and scope can be named by a unique path.  Also like a UNIX
  directory tree, path elements are seperated by a forward slash
  (\textit{/}), and two dots (\textit{..}) means the parent scope.

  The \ident{%command} register from our previous examples actually looks
  something like this:

  \begin{small}
  \begin{verbatim}
REG16("%command", 0x04);
OPEN_SCOPE("command");
    FIELD("io", "yesno_t",
          BITS("../%command", 0));
    FIELD("mem", "yesno_t",
          BITS("../%command", 1));
    FIELD("bm", "yesno_t",
          BITS("../%command", 2));
    FIELD("special", "yesno_t",
          BITS("../%command", 3));
    FIELD("mwinv", "yesno_t",
          BITS("../%command", 4));
    FIELD("vgasnoop", "yesno_t",
          BITS("../%command", 5));
    FIELD("perr", "yesno_t",
          BITS("../%command", 6));
    FIELD("step", "yesno_t",
          BITS("../%command", 7));
    FIELD("serr", "yesno_t",
          BITS("../%command", 8));
    FIELD("fbb", "yesno_t",
          BITS("../%command", 9));
    FIELD("intr", "yesno_t",
          BITS("../%command", 10));
CLOSE_SCOPE();
  \end{verbatim}
  \end{small}

\subsection{Datatypes}

  Each field can be evaluated against its datatype.  \ident{Prettyprint}
  defines a number of primitives:

  \begin{itemize}
  \item \textbf{int}: a decimal number
  \item \textbf{hex}: a hexadecimal number
  \item \textbf{enum}: an enumerated value
  \item \textbf{bool}: a binary enum
  \item \textbf{bitmask}: a set of name bits
  \end{itemize}

  These primitives are used to create several pre-defined datatypes:

  \begin{itemize}
  \item \textbf{int\_t}: a number
  \item \textbf{hex\_t}: a hexadecimal number
  \item \textbf{hex4\_t}: a 4 bit hexadecimal number
  \item \textbf{hex8\_t}: a 8 bit hexadecimal number
  \item \textbf{hex12\_t}: a 12 bit hexadecimal number
  \item \textbf{hex16\_t}: a 16 bit hexadecimal number
  \item \textbf{hex20\_t}: a 20 bit hexadecimal number
  \item \textbf{hex32\_t}: a 32 bit hexadecimal number
  \item \textbf{hex64\_t}: a 64 bit hexadecimal number
  \item \textbf{hex128\_t}: a 128 bit hexadecimal number
  \item \textbf{addr16\_t}: a 16 bit address
  \item \textbf{addr32\_t}: a 32 bit address
  \item \textbf{addr64\_t}: a 64 bit address
  \item \textbf{yesno\_t}: a boolean, 1 = "yes", 0 = "no"
  \item \textbf{truefalse\_t}: a boolean, 1 = "true", 0 = "false"
  \item \textbf{onoff\_t}: a boolean, 1 = "on", 0 = "off"
  \item \textbf{enabledisable\_t}: a boolean, 1 = "enabled", 0 = "disabled"
  \item \textbf{bitmask\_t}: a simple bitmask
  \end{itemize}

  Without doubt, any reasonably complex device will need to define its own
  datatypes.  \ident{Prettyprint} allows datatypes to be defined at any
  level of scope, and to be used in any scope below the definition -
  similar to C.

  \begin{itemize}
  \item \textbf{INT(name, units?)}:  define a new int type with optional units
  \item \textbf{HEX(name, width?, units?)}: define a new hex type with
  optional width and units
  \item \textbf{ENUM(name, KV(name, value), ...)}: define a new enum type
  with the specified named values
  \item \textbf{BOOL(name, true, false)}: define a new bool type with the
  specified true and false strings
  \item \textbf{BITMASK(name, KV(name, value), ...)}: define a new bitmask
  type with the specified named bits
  \end{itemize}

  Sometimes you want to define a new datatype for exactly one field.
  Rather than come up with a good name for it, each of the datatype
  definitions supports an \code{ANON_} prefix, which removes the name
  argument and produces an anonymous datatype.  For example, the previous
  PCI \ident{intpin} example used \ident{int_t} as the datatype.  In
  reality, we want an enumerated type.  This is the only field that will
  use this type, so we want to declare it anonymously:

  \begin{small}
  \begin{verbatim}
REGFIELD8("intpin", 0x3d, ANON_ENUM(
        KV("none", 0),
        KV("inta", 1),
        KV("intb", 2),
        KV("intc", 3),
        KV("intd", 4)));
  \end{verbatim}
  \end{small}

\subsection{Advanced Techniques}

  So far, we've seen how \ident{prettyprint} can be used to define simple
  registers and fields.  Unfortunately, few hardware devices are so
  simple.  Because the \ident{prettyprint} ``language'' is actually a
  dialect of C++, there is a lot of power at your fingertips.

  Hardware registers are at a premium.  Often the hardware will overload
  the meaning of some bits depending on the state of other bits.
  \ident{Prettyprint} supports the conditional definition of registers and
  fields.

  Let's look at another example.  In a PCI bridge's IO decode window, there
  is a \ident{width} field.  That field determines whether the high half of the
  \ident{base} field is valid.

  \begin{small}
  \begin{verbatim}
REG8("%base_lo", 0x1c);
REG16("%base_hi", 0x30);

FIELD("width", ANON_ENUM(
        KV("bits16", 0),
        KV("bits32", 1)),
    BITS("%base_lo", 3, 0));

if (FIELD_EQ("width", "bits16")) {
    FIELD("base", "addr16_t",
            BITS("%base_lo", 7, 4) +
            BITS("%0", 11, 0));
} else {  // bits32
    FIELD("base", "addr32_t",
            BITS("%base_hi", 15, 0) +
            BITS("%base_lo", 7, 4) +
            BITS("%0", 11, 0));
}
  \end{verbatim}
  \end{small}

  In this example you see the usage of \code{FIELD_EQ()}.  This performs a read
  of the \ident{width} field and compares the result against the value
  specified.  Comparisons can be done by string or by number, thanks to
  function overloading in C++.  The above example could have just as easily
  (though less maintainably) used:

  \begin{small}
  \begin{verbatim}
FIELD_EQ("width", 0)
  \end{verbatim}
  \end{small}

  The actual evaluation of the a comparison is done by the specific
  datatype, which is the only place that can actually determine what it
  means to compare values.  \ident{Prettyprint} supports the following
  comparison operations:

  \begin{itemize}
  \item \textbf{FIELD\_EQ}: the field is equal to the specified comparator
  \item \textbf{FIELD\_NE}: the field is not equal to the comparator
  \item \textbf{FIELD\_LT}: the field is less than the comparator
  \item \textbf{FIELD\_LE}: the field is less than or-equal-to the comparator
  \item \textbf{FIELD\_GT}: the field is greater than the comparator
  \item \textbf{FIELD\_GE}: the field is greater than or-equal-to the comparator
  \item \textbf{FIELD\_BOOL}: the field is boolean TRUE, equivalent to NE 0
  \item \textbf{FIELD\_AND}: the field matches the comparator
  \end{itemize}

  Again, because the \ident{prettyprint} ``language'' is really just C++,
  almost any native construct will work.  There are some limitations,
  though.

  To start with, C++ will not allow a \code{switch} statement on a non-integer
  value, so you can not switch on enumerated strings.  In the eventual
  \ident{prettyprint} language implementation, this will be supported.

  Secondly, control statements are evaluated just once, as the tree of
  registers and fields is being built.  Later changes to control bits do not
  change the tree structure.  This is something we want to enable in the
  \ident{prettyprint} language, but we do not have support for it yet.

\subsection{Discovering Specific Devices}

  Throughout these examples, we have looked at standard PCI fields and
  registers.  The PCI standard covers only a fraction of the available PCI
  register space.  Almost every PCI device defines its own non-standard
  register set.  What about those extra registers and fields?

  In the same way that \ident{prettyprint} can discover generic devices,
  such as PCI, it can also discover specific devices.  A device definition
  can register itself for discovery through a specific driver.  When the
  driver's discovery mechanism detects the registered device, as
  determined by a driver-specific signature, it invokes the specific
  device code, rather than the generic.

  For example, a device definition for an AMD Opteron might register
  with the PCI driver for the vendor and device pair \code{(0x1022,
  0x1100)}.  When the PCI driver finds that vendor and device pair, the
  Opteron-specific device code would be invoked, rather than the generic
  PCI device code.

  Rather than re-encoding the entire PCI specification, the generic PCI code
  can be invoked from the Opteron code.  This allows device code to extend
  standard devices with very little effort.

\subsection{The Directory Tree}

  The \ident{prettyprint} code is broken into four components.  The top level
  directory contains the core classes and functions that make up the
  \ident{prettyprint} library.  This includes pp\_register, pp\_field,
  pp\_datatype, etc.

  The \ident{drivers} subdirectory contains the driver modules.  Currently
  \ident{prettyprint} only supports Linux, though it would not be hard to
  add support for other operating systems.  At the time of writing, the
  drivers directory contains drivers for:

  \begin{itemize}
  \item \textbf{PCI}: via \texttt{/sys} or \texttt{/proc}
  \item \textbf{MEM}: via \texttt{/dev/mem}
  \item \textbf{IO}: via \texttt{IN} and \texttt{OUT} instructions
  \item \textbf{MSR}: via \texttt{/dev/cpu/*/msr}
  \item \textbf{CPUID}: via \texttt{/dev/cpu/*/cpuid}
  \end{itemize}

  The \ident{devices} subdirectory contains device code, written in the
  \ident{prettyprint} ``language''.  When we have a real language parser,
  this is the code that will be re-written in the new language.
  \ident{Prettyprint} currently has support for:

  \begin{itemize}
  \item \textbf{PCI}: most of the fields for generic PCI and PCI Express
  devices, including many capabilities.
  \item \textbf{CPUID}: very basic CPUID fields
  \end{itemize}

  Lastly, the \ident{examples} subdirectory contains example programs
  which use the \ident{prettyprint} library.

\subsection{FUSE And Prettyprint}

  One of the more exciting examples is the \ident{pp_fs} application.
  This is a \ident{FUSE} filesystem which allows direct access to
  registers and fields.

  Using \ident{pp_fs}, the example of setting \ident{SERR} on all devices
  becomes trivial:

  \begin{small}
  \begin{verbatim}
$ find /pp -wholename \*command\/serr \
   | while read X; do
     echo -n "$X: $(cat $X) -> "
     echo "yes" > $X
     cat $X
done
/pp/pci.0.0.0.2/command/serr: no -> yes
/pp/pci.0.0.1.0/command/serr: no -> yes
/pp/pci.0.0.0.1/command/serr: no -> yes
/pp/pci.0.0.0.0/command/serr: no -> yes
  \end{verbatim}
  \end{small}

\subsection{Current Status}

  The current examples demonstrate the capabilities
  of \ident{prettyprint}.  \ident{pp_discover} has already proven to be a
  useful tool at Google.  But there is still a lot of work to do in many
  areas.

  \ident{Prettyprint} is under active development.  A great way to get
  involved is to encode new hardware devices into the \ident{prettyprint}
  language.  The larger our device repository gets, the more useful it
  becomes.

  You can find \ident{prettyprint} at
  \url{http://prettyprint.googlecode.com}.

\section{Acknowledgements}

  Thanks to Nathan Laredo for pushing to make \ident{SGABIOS} a reality.

  Thanks to Aaron Durbin for busybox-ifying \ident{iotools} on a whim, and
  to all the Google platforms folks who have added tools to it.

  Thanks to Aaron Durbin, Mike Waychison, Jonathan Mayer, and Lesley
  Northam for all their help on \ident{prettyprint}.

  Thanks to Google for letting us hack on fun systems and release our work
  back to the world.

\end{document}

