Advanced Operating Systems - Spring 2020

News

  • 2020-03-17: We will from now on use the Announcements Forum in Moodle for these sorts of updates.
  • 2020-03-14: There is now a Moodle for the course (see Resources).
  • 2020-03-06: We updated the book to correctly reflect that the root page table level on ARMv8 is L0 (see Resources).
  • 2020-02-28: The handout for the main project is out (see GitLab). We also updated the instructions for milestone 1 in the book (see Resources).
  • 2020-02-24: We updated the submission guidelines for milestone 0 (it should be on a branch called milestone0, but we will of course also accept it on master as previously stated, see Project Submission).
  • 2020-02-24: We updated the book with a more detailed description of the extra challenge in milestone 0.
  • 2020-02-22: We updated the book with more precise instructions on how to get started with the board and toolchain (see Resources).
  • 2020-02-21: We fixed the handout for milestone 0 (see GitLab).
  • 2020-02-21: We updated the book's introduction with information about the reset button (see Resources).
  • 2020-02-20: The handout for milestone 0 is out (see GitLab).
  • 2020-02-19: The book and other useful resources are available now (see Resources).

Overview

This course is intended to give students a thorough understanding of design and implementation issues for modern multicore operating systems.

We will cover key design issues in implementing an operating system, such as memory management, inter-core synchronization, scheduling, protection, inter-process communication, device drivers, and file systems, paying particular attention to system designs that differ from the traditional monolithic arrangements of Unix/Linux and Windows.

The course is structured around a significant project which builds up, over the course of the semester, a fairly complete, full-featured multicore operating system for the ARM-based PandaBoard hardware. The OS is based on the Barrelfish open-source multikernel developed at ETHZ in collaboration with Microsoft Research.

The goals of the course are:

  • Teach general operating systems principles, using a real research operating system to illustrate them and by reading the research papers which propose some of the ideas that the particular OS builds on.
  • Give a broader perspective on operating systems which do not look like Linux, Unix, or Windows.
  • Provide exposure to the practical experience of working on OS code on real "metal", including debugging, hardware access, etc. This kind of experience is hard to gain merely from reading books or papers.
  • Introduce a sense of the complexity of a real OS, rather than simplified teaching OSes often used in more basic courses.  

This course builds on the ETHZ undergraduate courses in Computer Architecture and Systems Programming (252-0061-00), the contents of which will be assumed knowledge. Proficiency in C programming is assumed.

Lectures will focus on project-related material (to provide more background, knowledge, and time in completing the practical work), and also the prior research ideas that have informed the design of the aspects of Barrelfish relevant to the project milestones. They will also provide comparisons between Barrelfish and other systems such as Unix, Windows, and microkernels like L4.

There is no textbook for this course, as no published book covers the material in sufficient depth. Instead, reading for the course will consist of research papers and system documentation; this will be posted on this web site in due course.


Schedule

Week Lecture Project work due (by book chapter) Due date*
 1  Introduction    -
 2    Chapter 2: Getting started  27th Feb
 3    -  -
 4    Chapter 3: Capabilities  12th Mar
 5    Chapter 4: Process Creation  19th Mar
 6  Self-Paging (Recording | Notes)  Chapter 5: Message Passing  26th Mar
 7  Q&A Session (Recording)  -  -

 8

   Chapter 6: Self-Paging  8th Apr
 9  Caches (Recording)  No classes  
 10  UMP (Recording | Slides)  Chapter 7: Booting and using the 2nd core  23rd Apr
 11    Chapter 8: Message Passing between cores  29th April
 12  Scheduling (Recording)  -  -
 13  Q&A Session (Recording)  -

 -

 14    -

 -

 15  Demo Session (Recording)  Chapters 9-13: Demo, integrated code, final report  29th May

 

*Project due date and time

Each project work package ("milestone") is due at 23:59 local time on the date indicated in the table above. The presentations are on the following morning. For the code submissions, we will accept git commits (see below for details) with commit date and time before 23:59 local time on the due date. So for milestone 0, we will accept commits created before Thursday, 27th of March 2019, 23:59 local time. The milestones in weeks 8 and 11 are due on a Wednesday and will be graded on a Thursday as the Fridays in those weeks are Good Friday and Labour Day respectively.


Resources

The AOS Book [pdf]. We will update the linked PDF throughout the course, so check back regularly if you choose to download and/or print the book.

The i.MX 8QXP Applications Processor Reference [pdf]

The Colibri iMX8X Module Datasheet [pdf]

The Aster Carrier Board Datasheet [pdf]

The ARMv8 Architecture Reference Manual [pdf]

SD Host Controller Specification [pdf]

FAT32 Specification [pdf]

Use your nethz credentials to access the protected resources.

Questions of general concern about the course or the project work can be directed to aos-students@lists.inf.ethz.ch (goes to all students and staff of the course). Administrative questions can be sent to aos-support@lists.inf.ethz.ch (course staff only). There is now also a Moodle Forum.

Getting the project handout

First use git to clone the handout repository that is available on D-INFK GitLab. If you do not have access to this repository, send us an email giving your nethz login, so we can give you read access to the repository.

$ git clone git@gitlab.inf.ethz.ch:COURSE-AOS2020/hand-out.git

To add your hand-in repository (see Project submission for details on how to get a group repository), you can tell git to use your group repository as the default remote ("origin") as follows:

$ cd /path/to/aos/code
$ git remote rename origin handout
$ git remote add origin
git@gitlab.inf.ethz.ch:COURSE-AOS2020/X-hand-in.git

Tips and tricks for Ubuntu

If you are using Ubuntu, you can make your life easier by installing custom udev rules to give unprivileged users access to the Colibri board's USB ports. Copy the following into /etc/udev/rules.d/60-colibri.rules

# USB OTG
SUBSYSTEM=="usb", ATTR{idVendor}=="0525", ATTR{idProduct}=="4026", MODE="0666"
# Serial USB device
SUBSYSTEM=="usb", ATTR{idVendor}=="0403", ATTR{idProduct}=="6001", MODE="0666"
# Serial TTY device
SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", MODE="0666"

Project submission

  • You will be using git to submit your project work.
  • For the initial individual milestones we provide everyone a hand-in repository. The repository URLs will follow the format gitlab.inf.ethz.ch/COURSE-AOS2020/nethz-hand-in where 'nethz' is your ETH username.
  • For the group milestones (including the final submission), each group will be provided a group git repository. The repository URLs will follow the format https://gitlab.inf.ethz.ch/COURSE-AOS2020/teamx-hand-in, where 'x' will be replaced by the group's 3-digit number as assigned at sign-up.
    We have set up a Doodle to help with group formation. Once you have formed a group, inform us by email to aos-support@lists.inf.ethz.ch or in the consultation session, so we can give you access to a group repository.
  • Each project milestone must be submitted by email to aos-support@lists.inf.ethz.ch and must include the full git SHA1 hash of the commit that constitutes the milestone submission. All submission commits (except for milestone 0) must be reachable from the master branch. Git tags or branches will not be accepted as milestone submissions. Make sure to clearly indicate the group for the milestone submissions.
    A commit's full git SHA1 hash can be generated using the command git rev-parse <rev>, where <rev> can be any of the options referring to a commit as discussed in the git documentation.
    Alternatively, most GUI git clients will show a commit's SHA1 hash somewhere. On GitLab there's an icon labelled "Copy commit SHA to clipboard" which also produces the right result.
  • Milestone submissions with a git commit date (not to be confused with author date) made after a milestone deadline will be marked late. The penalties for late submissions are: 25% reduction in points for one week late, 50% reduction in points for two weeks late. Submissions more than two weeks late will not be accepted.
  • There must be a single submission per milestone and group (except for milestones 0 and 1, see below). In particular, we will not accept multiple submissions for the final milestone where each group member will work on a different project.
  • For milestones 0 and 1 each group member has to publish their submission commit on their own submission repo on branches called milestone0 and master respectively. All other rules apply (late penalty, SHA1, ...).
  • The contents of the group repository must be self-contained, and must be buildable as described in section 3.6 of the course book.
  • The group will be graded by the code found in the git commit identified by the SHA1 commit hash provided as the milestone submission for the final milestone plus the written report which is to be handed in as PDF via email together with the SHA1 hash for the last milestone.
  • Attempting to edit git history, or otherwise circumventing the submission process is misconduct and will be penalised.

Examination

The final assessment will be a combination of project milestone grades, final project report grade, and grades derived from tests run on the final code submission. 

Course Hours

 Lecture:  Thursdays
 13:00 to 15:00, CAB G 51
 Project marking/consultation:

 Fridays

 10:00 to 12:00, CAB H 56
 10:00 to 12:00, CAB H 57

Staff

To help us load-balance questions, send all questions to aos-support@lists.inf.ethz.ch or if they are of general concern to aos-students@lists.inf.ethz.ch.

 


Acknowledgments

We thank Toradex and ARM for helping us to realize this course.