CSL373 : Operating Systems : Tools

This page describes how to install the tools used in the Operating Systems class. The tools are already installed in the department cluster, so you will need to follow these instructions if you want to do assignments on other machines. We cannot guarantee that these tools will run on your computer, but they should run on recent versions of Linux.

It should be possible to get this development environment running under windows with the help of Cygwin. Install cygwin, and be sure to install the flex and bison packages (they are under the development header). Another alternative is to use virtualization software like VMware and install Linux in a VM.

Compiler Toolchain

Most modern Linuxes and BSDs have an ELF toolchain compatible with the labs. That is, the system-standard gcc, as, ld and objdump should just work. The lab makefile should automatically detect this. However, if your machine is in this camp and the makefile fails to detect this, you can override it by adding the following line to conf/env.mk:

GCCPREFIX=

If you are using something other than standard x86 Linux or BSD, you will need the GNU C compiler toolchain, configured and built as a cross-compiler for the target 'i386-csl373-elf', as well as the GNU debugger, configured for the i386-csl373-elf toolchain. You can download the specific versions we used via these links, although any recent versions of gcc, binutils, and GDB should work:

Once you've unpacked these archives, run the following commands as root:

# cd binutils-2.20.1
# ./configure --target=i386-csl373-elf --disable-nls
# make
# make install
# cd ../gcc-4.5.1
# ./configure --target=i386-csl373-elf --disable-nls --without-headers \
              --with-newlib --disable-threads --disable-shared \
              --disable-libmudflap --disable-libssp
# make
# make install
# cd ../gdb-6.8
# ./configure --target=i386-csl373-elf --program-prefix=i386-csl373-elf- \
              --disable-werror
# make
# make install

Then you'll have in /usr/local/bin a bunch of binaries with names like i386-csl373-elf-gcc. The lab makefile should detect this toolchain and use it in preference to your machine's default toolchain. If this doesn't work, there are instructions on how to override the toolchain inside the GNUmakefile in the labs.

Installing QEMU

QEMU is a modern and fast PC emulator. QEMU version 0.12.5 is set up on the department cluster for x86 machines. You can use GDB (or other graphical frontends) with QEMU for debugging. We highly recommend you use a patched version of QEMU instead of the stock version that may come with your distribution. The version installed on the deparment cluster is already patched. To build your own patched version of QEMU:
  1. Download and unpack the pre-patched 0.12.5 source tarball
  2. On Linux, you may need to install the SDL development libraries to get a graphical VGA window. On Debian/Ubuntu, this is the libsdl1.2-dev package.
  3. Configure the source code:
    Linux: ./configure [--prefix=PFX] [--target-list="i386-softmmu x86_64-softmmu"]
    OS X: ./configure --disable-sdl [--prefix=PFX] [--target-list="i386-softmmu x86_64-softmmu"]
    The prefix argument specifies where to install QEMU; without it QEMU will install to /usr/local by default. The target-list
  4. Run make && make install
Alternatively, you can download the official 0.12.5 source tarball from the QEMU homepage and apply the following patch series:
debug-seg Use DS-relative virtual addresses instead of linear addresses in the GDB stub.
info-mem Fix "info mem" in the QEMU monitor to not skip the last mapped memory range.
info-pg Add "info pg" in the QEMU monitor that prints the page table.
e100 Fixed bugs in QEMU's simulated E100 and adds E100 debugging.
pcap Adds packet capture support.
triple On triple fault, dump state and halt for inspection instead of resetting.


Installing Bochs

Bochs is an open-source IA-32 emulator, and is another alternative to run your OS. Bochs comes with a very useful internal debugger. See a local (and slightly augmented) copy of the documentation here. All additions in the local copy over the original documentation are in a different colour. The documentation is useful for trying the commands provided in the lecture notes and book draft. The original documentation can be found here (go to "User Guide" and then "Using Bochs internal debugger").

We highly recommend you use a patched version of Bochs instead of the stock version to allow reproducible executions. The version installed on the department cluster is already patched. There are two executables that need to be built: bochs (which is compiled with an internal debugger) and bochs-gdb (which is compiled with a gdb-stub for debugging with gdb). Both these executables have been pre-built on the department cluster. To build your own patched version of Bochs:

  1. Download and unpack the pre-patched 2.4.5 source tarball
  2. Configure the source code:
    ./configure [--prefix=/path/to/bochs-install] --enable-disasm --enable-smp --enable-debugger --enable-new-pit --enable-all-optimizations --enable-4meg-pages --disable-reset-on-triple-fault --with-all-libs --with-x --with-x11 --with-nogui
    The X, terminal, and "no GUI" interfaces can be configured, by passing --with-x --with-x11 --with-term --with-nogui to configure.
  3. Run make && make install
  4. This command builds the bochs binary with SMP support that can be used with both xv6 and pintos. This bochs binary supports an internal debugger shipped with bochs.
  5. To debug pintos using gdb, you will need to build another binary. To do this, configure the source code in the following way:
    ./configure [--prefix=/path/to/bochs-gdb-install] --enable-disasm --enable-gdb-stub --enable-new-pit --enable-all-optimizations --enable-4meg-pages --disable-reset-on-triple-fault --with-all-libs --with-x --with-x11 --with-nogui
    Use a different prefix /path/to/bochs-gdb-install for the install directory to avoid overwriting the previous installation.
  6. Run make && make install. This will create another binary called bochs in /path/to/bochs-gdb-install/bin directory. Go to this directory and rename the bochs executable to bochs-gdb using mv bochs bochs-gdb.
  7. Set your PATH variable to include both /path/to/bochs-install and /path/to/bochs-gdb-install directories.
If you would like to patch and build bochs from scratch (instead of using our custom tarball), you can download the official 2.4.5 source tarball from the Bochs homepage and apply the following patch series: These patches are designed for use with Bochs 2.4.5:
jitter Adds the "jitter" feature, in which timer interrupts are delivered at random (but reproducible) intervals.
att Sets the default disassembly mode to AT&T (also used in the handouts and class).

To apply all the patches, cd into the Bochs directory, then type:
  patch -p1 < bochs-2.4.5-jitter.patch
  patch -p1 < bochs-2.4.5-att.patch
You can use patch's --dry-run option if you want to test whether the patches would apply cleanly before trying to apply them.

xv6

Fetch xv6-rev3.tar.gz. Change the first line in xv6/Makefile to:
QEMU=/path/to/qemu-0.12.5-6828/i386-softmmu/qemu

Change the second line in xv6/Makefile to:
BOCHS=/path/to/bochs-install/bin/bochs
Go to the xv6 directory. Run
make qemu
to run xv6 inside Qemu.
Run
make bochs
to run xv6 inside Bochs. Remember to type ldsym "kernel.sym" at the prompt after running make bochs. This will load all the symbol to address mappings to help you in debugging.

Pintos

Fetch pintos.tar.gz and extract the source for Pintos into a directory named pintos/src, by executing
 
gunzip /path/to/pintos.tar.gz | tar x

Go to src/threads directory and type make. Go inside the newly created src/threads/build directory.
Before we proceed, ensure that bochs, qemu, and the pintos utility are in your path. Put bochs and bochs-gdb in your path using:

export PATH=/path/to/bochs-install/bin:/path/to/bochs-gdb-install/bin:$PATH
Note that /path/to/bochs-install and /path/to/bochs-gdb-install should be the same as you used in your bochs installation.
Put qemu in your path using:
export PATH=/path/to/qemu-0.12.5-6828/i386-softmmu:$PATH
/path/to/qemu-0.12.5-6828 should be the same as that used while installing Qemu.
If you are using an old pintos tarball, make the following modifications: These changes are also available as patches: pintos.script.patch, pintos.block_role_count.patch

Put the pintos utility in your path using:

cd /path/to/pintos
chmod +x src/utils/pintos
export PATH=/path/to/pintos/src/utils:$PATH
To ensure that your PATH environment variable is correctly set in each login session, you may want to add the following line to your .cshrc (or .bashrc) file:
export PATH=/path/to/pintos/src/utils:/path/to/qemu-0.12.5-6828/i386-softmmu/:/path/to/bochs-install/bin:/path/to/bochs-gdb-install/bin:$PATH
To run pintos, go to src/threads/build directory and type:
pintos --bochs --
to run the basic pintos OS inside bochs. Notice that the final "--" is required to let the script know that all arguments preceeding it are arguments to the script and not to the OS itself. Any arguments given after "--" are arguments to the OS. For example, the following command
pintos --bochs -- -q
passes the "-q" argument to the OS which causes it to quit after it's boot is finished.
To run Qemu instead of Bochs, use
pintos --qemu --
To run Bochs under gdb, use
pintos --gdb --
and then in another window, type
pintos-gdb kernel.o
where kernel.o is your compiled kernel from where gdb should load the symbol table.

Look at this page for more details on installing pintos utilities.

Other tools

A very good tool to help you browse your source code: cscope
Index your source code using ctags
Use cscope/ctags with vi: Vim/cscope tutorial, Vim/ctags tutorial

Subversion

Subversion (SVN) is installed on all machines in the department cluster. You can install it on your machine using "apt-get install subversion".
The Computer Services Center provides a subversion server. To create a repository called "pintos-repo":
$ ssh ssh1.iitd.ernet.in   # use your proxy password
$ mkdir svn
$ cd svn
$ svnadmin create pintos-repo
You should see a subdirectory called pintos-repo in the svn directory. Read pintos-repo/README.txt. Do not add, delete, or modify any files in this subdirectory. Edit the svn/authz file to provide read-write permissions to pintos-repo to yourself and your partners, and read permissions to sbansal. For example, if users cs1012345 and cs1067890 are partners and user cs1012345 is currently logged in, here is a sample authz file to use:
[/]
[pintos-repo:/]
cs1012345@IITD.ERNET.IN = rw
cs1067890@IITD.ERNET.IN = rw
sbansal@IITD.ERNET.IN = r
Notice that your username should be suffixed with @IITD.ERNET.IN and not cse.iitd.ernet.in. These statements provide read/write access to the group partners and read access to the instructor. Read access to the instructor is necessary for submission of assignments.

Set the directory permissions so that they look like the following:

The responsibility of maintaining the security of your repository lies with you. You will be penalized if your repository is found to be readable by other unauthorized people.

Go to one of the machines in the department cluster (or your local machine). Use the following command to checkout the files from your repository:

$ svn checkout https://svn.iitd.ernet.in/~cs1012345/pintos-repo
You will be prompted for your CSC password. Do not offer to save the password in plaintext format in your home directory as that is insecure. You can add the following line to ~/.subversion/servers to avoid getting prompted for saving passwords in future:
store-plaintext-passwords = no
This should create a directory called pintos-repo. Type the following commands to add the base pintos files to the repository:
$ cd pintos-repo/
$ cp -r /path/to/pintos/src .
$ svn add src
$ svn commit
"svn add" adds the files to your local checked-out copy of the repository. "svn commit" commits the changes in your local repository to the main repository. You will be prompted for your CSC proxy password each time you access the main repository. At this point, you have successfully setup your repository. Your partner can use svn checkout https://svn.iitd.ernet.in/~cs1012345/pintos-repo to checkout a copy of the code. Just like you, your partner is also authorized to make changes and commit them to the main repository.

If this does not work, carefully study the instructions given at the CSC Webpage on creating, accessing, and securing your repositories. Report any missing instructions that you find to the course staff so that we can fix the documentation.

Here are some more useful commands:

Version Control Software is used in almost all modern software development environments. Read this wikipedia article to know more about the theory. Here is a brief tutorial by the Computer Services Center on using the IITD Subversion repository. Here is a quick start document for basic SVN commands.