Courses

Data Stream Processing and Analytics - Spring 2019

Course Information

Code: 263-3826-00, 6 ECTS credits
Lecturer: Vasiliki Kalavri (kalavriv at inf dot ethz dot ch) CAB E 73.1
Language of instruction: English
Lectures: Mondays, 10-12, CHN E42
Exercises: Mondays, 13-15, CHN D46

What is this course about? - Lecture 0 is now available.


Overview

Modern data-driven applications require continuous, low-latency processing of large-scale, rapid data events such as videos, images, emails, chats, clicks, search queries, financial transactions, traffic records, sensor measurements, etc. Extracting knowledge from these data streams is particularly challenging due to their high speed and massive volume.

Distributed stream processing has recently become highly popular across industry and academia due to its capabilities to both improve established data processing tasks and to facilitate novel applications with real-time requirements. In this course, we will study the design and architecture of modern distributed streaming systems as well as fundamental algorithms for analyzing data streams.

Specifically, we will cover the following topics:

  • Distributed streaming systems design and architectures
  • Fault-tolerance and processing guarantees
  • State management
  • Windowing semantics and optimizations
  • Basic data stream mining algorithms (e.g. sampling, counting, filtering)
  • Query languages and libraries for stream processing (e.g. Complex Event Processing, online machine learning)
  • Streaming applications and use-cases 
  • Modern streaming systems: Apache Flink, Apache Beam, Apache Kafka, Timely dataflow

Course Program and Materials

Detailed lecture schedule

Recommended readings


Exercise Sessions

The exercise sessions will be a mixture of (1) reviews, discussions, and evaluation of research papers on data stream processing, and (2) programming assignments on implementing data stream mining algorithms and analysis tasks.


Examination

The course consists of lectures, exercises, and a final semester project. There will be no formal examination at the end of the course. Students are continuously graded based on their participation in class (10%), weekly assignments (50%), and semester project (40%).

 

Upload File: 

COMPUTER SYSTEMS - FALL 2018

What's new?

  • Exercise session registration is available.
  • No exercise session on September 21st. First exercise session will be on September 28th. 
  • Added mailing list for corrections, typos, and suggestion.
  • No lecture on September 28th. 
  • Bonus assignment guidelines accessible under course material. First deadline on November 1st. 
  • Small update to Bonus assignment guidelines. We will not discard any questions about the first half of the lecture for the second deadline, but we encourage you to submit a question that covers a topic of the second half. 
  • Chapter 14: Added definition (Def. 14.39) clarifying the difference between global and local clock skew.
  • Fixed Mistake in Virtual Machine & Network Stack exercise (last question). 
  • Uploaded some old exams for the Distrbuted Systems part (in course material).


Overview

This course is about real computer systems, and the principles on which they are designed and built. We cover both modern OSes and the large-scale distributed systems that power today's online services. We illustrate the ideas with real-world examples, but emphasize common theoretical results, practical tradeoffs, and design principles that apply across many different scales and technologies. 

Since this is a new course, we are still "debugging" it and if you find any typos, corrections or have suggenstions on how to improve the script you can send them to the following mailing list: compsys-errata@lists.inf.ethz.ch

Lecturer

Staff

  • Manuel Eichelberger (manuelei at ethz) ETZ G97 (En/De)
  • Roni Häcki (roni.haecki at inf) CAB E69 (En/De)
  • Vasiliki Kalavri (vasiliki.kalavri at inf) CAB E73.1 (En)

Hilfsassistenten

  • Daniel Gstöhl (gstoehld at student.ethz.ch) 
  • Jonas Gude (jgude at student.ethz.ch)
  • Jakob Meier (jakmeier at student.ethz.ch)
  • Claudio Ferrari (ferraric at student.ethz.ch)
  • Amray Schwabe (schwabea at student.ethz.ch)

Course Hours

Lecture

  • Mon 10-12h, CAB G  61
  • Fri 10-12h, CAB G 61

Exercise

Time Room Language TA
 Fri 13-15  CHN D 48  en/de  Daniel  Gstöhl
 Fri 13-15  ETZ F 91  en/de  Jonas Gude
 Fri 13-15  ETZ K 91  en/de  Jakob Meier
 Fri 13-15  HG D 3.1  en/de  Claudio Ferrari
 Fri 13-15  HG D 3.3  en/de  Amray Schwabe

 

 

Hardware Acceleration for Data Processing (HADP) - Fall 2018

 Course Material | Talks | Schedule | Seminar Hours | People


Overview

The seminar is intended to cover recent results in the increasingly important field of hardware acceleration for data science, both in dedicated machines or in data centers. The seminar aims at students interested in the system aspects of data processing who are willing to bridge the gap across traditional disciplines: machine learning, databases, systems, and computer architecture. The seminar should be of special interest to students interested in completing a master thesis or even a doctoral dissertation in related topics.

The seminar will start on September 18th with an overview of the general topics and the intended format of the seminar. Students are expected to present one paper in a 30 minute talk and complete a 4 page report on the main idea of the paper and how they relate to the other papers presented at the seminar and the discussions around those papers. The presentation will be given during the semester in the allocated time slot. The report is due on the last day of the semester.

Attendance to the seminar is mandatory to complete the credit requirements. Active participation is also expected, including having read every paper to be presented in advance and contributing to the questions and discussions of each paper during the seminar.


NEWS 

     1) The first introductory class will take place on 18th September 2018 at 13:15 in ML J 34.1 followed by the opening talk at 13:45.

    2) Selection of papers (3 papers max) and presentation dates (3 slots max) are expected to be ready by 25th September 2018. Please send your preferences to user_name[at]inf.ethz.ch, where username is amit[dot]kulkarni  

    3) A preliminary schedule for the presentations is available under Schedule section.

    4) The first seminar talk starts on 9th October 2018, 13:15. The presentation duration will be of 30 mins + 15 mins Q & A.

    5) Deadline for the report submission is on 11th January 2019.


Talks

Speaker Title Date/Time
Prof. Gustavo Alonso  Introduction to the seminar

 18/09/2018,

13:15

David Sidler

Accelerating String Matching Queries
in Hybrid CPU-FPGA Architectures

18/09/2018,

13:45

Dr. Tal Ben-Nun  Demystifying
Parallel and Distributed
Deep Learning

 25/09/2018,

13:15

Cedric Renggli  SparCML: High-Performance Sparse
Communication for Machine Learning

  25/09/2018,

14:15

Prof. Torsten Hoefler  How to survive in this seminar?

 02/10/2018,

13:15

Prof. Ce Zhang  System Relaxations for First Order Methods:
A 45 Minutes Crash Course 
02/10/2018,

14:15 

 


 Schedule

NAME PAPER DATE MENTOR
 Skanda Koppula  “ImageNet Training in Minutes” 9 Oct 
13:15 
 Prof. Torsten Hoefler
 Emanuele Esposito  “RAPID: In-Memory Analytical
Query Processing Engine with
Extreme Performance per Watt”
16
Oct
13:15 
 Prof. Gustavo Alonso
 Nikolas Göbel  “Ray: A Distributed Framework for
Emerging AI Applications”
16
Oct
14:15 
 Prof. Ce Zhang
 Gishor Sivanrupan “Live Video Analytics at Scale
with Approximation and Delay-Tolerance”
23
Oct
13:15 
Dr. Mushen Owaida
 Taha Shahroodi  “A Many-core Architecture for
In-Memory Data Processing”
30
Oct
13:15 
 Prof. Gustavo Alonso
 Nikita Lazarev  “ESE: Efficient Speech Recognition
Engine with Sparse LSTM on FPGA”
13 Nov
13:15 
 Dr. Mushen Owaida
 Thomas Lang  “UDP: A Programmable Accelerator
for Extract-Transform-Load Workloads and More”
20 Nov
13:15 
 Prof. Gustavo Alonso
 Alain Denzler  “Toward Standardized Near-Data
Processing with Unrestricted Data
Placement for GPUs”
20 Nov
14:15 
 Dr. Mushen Owaida
 Florian Tschopp  “GraphR: Accelerating Graph
Processing Using ReRAM”
27 Nov
13:15 
 Dr. Mushen Owaida
 Nicolas Winkler  “Caribou: Intelligent Distributed Storage” 27 Nov 
14:15 
 Prof. Gustavo Alonso
       
       

 Seminar Hours

Tuesdays, 13:00-15:00 in ML J 34.1


 People

Lecturers:

Facilitator / TA:

 

 

Advanced Systems Lab - Fall 2018

Course Organization & Materials

Find below the dates and details of tutorials (T) and exercises (E):

Date/Time Type Description Materials
 18. Sep.  The first tutorial session will take place on Sept. 18th in CAB G61  slides
 20. Sep.  The first exercise session will take place on Sept. 20th in CAB G61. It gives an overview of the project.  slides
 25. Sep.  The second exercise session will take place on Sept. 25th in CAB G61. It focuses on Microsoft Azure, Bash and Git.  azure, bash
 27. Sep. The third exercise session will take place in individual groups. Please look up your assigned group in the table below.   slides
 2. Oct.  Tutorial session on the life cycle of an experiment.   slides, throughput
 4. Oct. E  Exercise session on good and bad practices in Java middleware development.  slides
 9. Oct.  There will be no tutorial session on October 9th.  
 11. Oct.  The exercise session will cover GnuPlot, and Baseline experiments without the Middleware.  slides
 16. Oct.   Tutorial session on planning experiments.  slides
 18. Oct. The exercise session will cover good and bad practices when generating plots, and Baseline experiments with the Middleware.  slides
 23. Oct.  Tutorial session on queueing theory. slides
 25. Oct.  Exercise session on 2K experiments. slides
 30. Oct.  Tutorial session on System Analysis. slides
 1st. Nov.  E  Exercise session on Queueing Theory slides
 6th Nov. T   All the Tutorials are finished for this semester.  
 8th Nov E  Exercise session on Network of Queues  slides
 15th Nov. E  The rest of the exercise sessions will be Q/A. No material presented.  

 

Project Details

Project Description: [Project Description]

Report: [Report Outline (pdf), (tex)]

Programming: [Java Main Class] [ANT Build File] [Bash Script Examples]

Azure: [Education Hub] [VM TemplateCaution: When creating the VMs, the machines are started automatically. Stop them if you do not run experiments right away.
 

Project Deadline: December 17th, 17:00, 2018. 

NOTE: THE DEADLINE TO DE-REGISTER FROM THE COURSE IS ON 14th OCTOBER 2018.


Literature

"Art of Computer Systems Performance Analysis" - Raj Jain
John Wiley & Sons Inc; Auflage: 2 Rev ed. (21. September 2015)

"The Art of Computer Systems Performance Analysis" - Raj Jain
Wiley Professional Computing, 1991

From the 1st edition of particular relevance are the following chapters:

  • Chapters 1, 2, 3 (General introduction, Common terminology)
  • Chapters 4, 5, 6 (Workloads)
  • Chapter 10 (Data presentation)
  • Chapters 12, 13, 14 (Probability and statistics)
  • Chapters 16, 17, 18, 20, 21, 22 (Experimental design)
  • Chapters 30, 31, 32, 33, 36 (Queueing theory)

 


Lecturer

Gustavo Alonso

 


Course Hours

Tutorials: Tuesday, 17:00 – 19:00, CAB G 61.

Exercises: Thursday, 17:00 - 19:00

General Contact: sg-asl [at] lists.inf.ethz.ch


Exercise Sessions

Exercises sessions are held on Thursday from 17:00 - 19:00 in small groups. In the exercise sessions, we answer high-level questions related to the project and the report.

 Assistant
 Room  Email Last names assigned
 Michel Mueller  CHN D42  muellmic [at] inf.ethz.ch  A-C

 Muhsen Owaida

 CHN D44  mewaida [at] inf.ethz.ch  D-He
 Alba Ríos Rodríguez  CHN D46  rialba [at] student.ethz.ch  Ho-L
 David Sidler  CAB G56  dasidler [at] inf.ethz.ch  M-Sa
 Kaan Kara   CAB G52  kkara [at] inf.ethz.ch  Sc-Z

 


Office Hours

Office hours are indented to provide you advice that will help you to complete the project and the report. To make an appointment, contact your teaching assistant by email.

  • Make sure you come prepared with concrete and well formulated questions. If possible, include them in your email.
  • We will not complete the assignment for you and neither recommend nor make design decisions on your behalf.
  • We will not debug your code, provide technical support for your setup/scripts/data analysis, or give hints about whether what you have done so far is enough.
  • We will not grade your project in advance, so please avoid questions that try to determine whether what you have done is correct or sufficient for a passing grade.
Time
Assistant
 Friday, 9:00-10:00, 13:00-14:00  Michel Mueller
 Thursday, 15:00 - 17:00   Muhsen Owaida
 Thursday, 9:00-11:00  Alba Ríos Rodríguez
 Thursday, 16:00-17:00, Friday 09:00-10:00  David Sidler
 Friday, 9:00-10:00, 13:00-14:00  Kaan Kara

 


FAQ / Tips

Q: Which Java version am I allowed to use?

A: Use Java 8.

Q: With how many threads should I run memcached?

A: Run memcached with a single thread.

Q: Can I adapt the Ant file to include the log4j library?

A: Yes, you can alter the Ant file as long as we are still able to build your middleware from a clean checkout. Note: You cannot use external libraries other than log4j.

Q: Where can I see how much money I spend on Azure?

A: Go to aka.ms/startedu and click on the "Courses" tab. Next click on the course "ASL 2018", then you should be able to see your lab and the credit assigned and consumed so far.

Q: I think that I am getting charged by Azure despite shutting down my VMs?

A: If you use the Azure console interface you have to use the "deallocate" command to deallocate the VMs, if you only use the "stop" command the VMs are not deallocated you are still charged for them. If in doubt, check in the online interface if the VMs are "stopped (deallocated)".

Q: How do I construct the histograms?

A: To construct the histograms from the middleware, you need to record the response time of every request with the precision of 100us. Note that the bucket size is not necessarily 100us, it can be larger. However, you need to have at least 10 buckets per histogram. To construct the histograms on the clients side, use the CDF generated by memtier and decide on the bucket size yourself. BUT make sure all the histograms from both the clients and middleware HAS THE SAME BUCKET SIZE.

Q: I get the error "read error: Connection reset by peer". How to solve it?

A:

- Make sure your Middleware has started and ready to receive connection requests before you start your memtier clients.

- Increase the backlog queue size (this is the allowed number of new connections waiting to be accepted). See this thread.

- Check if the network thread can handle the amount of incoming connections and does not run out of file descriptors. 

Q: For Section 6 (2K Analysis), do I have to perform two separate analyses for throughput and response time?

A: Yes, you need solve the linear equation system separately for throughput and response time.

Q: For Section 7, which exact configuration should I use for network of queues?

A: You need to consider both 1 and 2 middleware configuration. You need to analyze both read only and write only workloads. You can set the number of worker threads to the constant that delivers highest throughput.

Q: It is asked in the report outline to consider under-saturated, saturated and over-saturated states of our experiment runs. What if we don't observe these states clearly?

A: You may or may not observe some of these states. You still need to argue which of these states the system is in throughout your experiments.

Q: How do I select the maximum throughput as it is asked in the report outline?

A: It is important when determining the maximum throughput of your system to take the response time in consideration and how it is affected by the increase in load. Clearly explain the reasoning behind the selection.

Q: Do I need to plot the interactive law verification in all my figures?

A: No, this is not necessary. However, it is necessary to verify it and state that it holds in the report.

Q: What do I have to consider when calculating the interactive law for the middleware?

A: When calculating the throughput based on the response time measured in the middlware you have to adapt either a) N (number of clients/requests) or b) Z (think time). a) When only considering the middleware and servers as the system, the number of requests in the system (N) is smaller than the number of clients. Given your middleware measurements, you can determine the number of requests in the system. For this approach the think time is still ~0. b) You can take the RTT between client and middleware as the think time (Z). The RTT can be measured for instance with ping. For this approach N is still the number of clients.

Errata / Updates

  • Project Description: Section 3 5. 2k Analysis. Changed parameters to be consistent with Report Outline.
  • Report Outline: Table in Section 6 changed the entries for "Instances of memtier per machine" and "Threads per memtier instance". Reason: typo
  • Report Outline: Section 3 & 3.1 & 3.2 changed first sentence from "Connect one load generator machine.." to "Connect three load generator machines..".
  • Report Outline: Section 5 clarified the use and meaning of the --ratio parameter.
  • Azure Slides: Slide 16 changed from "openjdk-7-jdk" to "openjdk-8-jdk"

 

Big Data for Engineers 2018 Schedule

Lecture

Date Topic Slides Material
 20.02 1. Introduction    
 27.02  2. Lessons learnt    
       
       
       
       
       
       
       
       
       
       

 

AOS - Fall 2017 - Book

We host the draft PDF of the AOS course book on this page. We will update the linked PDF throughout the course when additional chapters are ready, so check this page regularly if you choose to download and/or print the book.

News

  • 2017-09-21: Uploaded book draft with chapters 1 - 4.

Download

The AOS book [pdf]

 

 

Temporary Download for the week1 (milestone0) handout:

[tar.gz]

Unpack as follows:

$ mkdir bf_aos
$ cd bf_aos
$ tar xf /path/to/downloaded/aos2017_week1.tar.gz

Communication Networks Seminar

Overview

We will study recent advances in networking by reading and presenting research papers from top conferences in the area. Each session will include 2 research papers on a topic, at least one of which will be presented by a student. Both papers will be discussed in each session. Prior to each session, every student is expected to submit brief reviews for the session's papers here.

 

Objectives

  • To understand the state-of-the-art in the field
  • To learn to read, present and critique papers
  • To identify opportunities for new research

 

Expectations

  • Attend and actively participate in the entire seminar.
  • Present one of the seminar topics.
  • Submit brief reviews for each Tuesday's readings on the previous Friday.
  • You will be evaluated on all three factors above.

 

Resources

S. Keshav, How to Read a Paper.

 

Seminar Hours

Tuesday, 15:00-17:00 in CAB F 72. [Please note the change of venue.]

 

Staff

 

Schedule

 Date  Lecture  Presenters
 20.09  Introduction and overview  Timothy Roscoe
 Ankit Singla
 27.09  Data center network topology design (p1, p2)  Simon Kassing
 04.10  Data center network monitoring (p3, p4)  François Wirz
 11.10  Flow / packet scheduling - I (p5, p6)  Darko Makreshanski
 18.10  Flow / packet scheduling - II (p7, p8)  Reto Achermann
 25.10  Programmable packet processing (p9, p10)  Vojislav Dukic
 01.11   --- No seminar ---  
 08.11   --- No seminar ---  
 15.11  Network verification (p11, p12)  Zsolt István
 22.11  Congestion control (p13, p14)  Daniele Asoni
 29.11  Internet architecture - I (p15, p16)

 Antonios Karkatsoulis (p15)

 Simon Peyer (p16)

 06.12  Internet architecture - II (p17, p18)

 Melanie Hüsser (p17)

 Stefan Milosavljevic (p18)

 13.12  Industry experience (p19, p20)  Jonas Fernbach

 

 

Reading list

 

Case Studies from Practice

Exam review: 

  • Thursday 28. July 2016:  9:30-11:30 (CAB F 78)
  • Wednesday 3. August 2016: 14:00-16:00 (CAB F 78)

Exam: 

  • Final exam: 08 June 2016 at 10:00 in HG F 3.
  • Mid-term 21 March in HG G5 (45 minutes)

Course description

The course is designed to provide students with an understanding of “real-life” challenges taken from corporate settings and teach them how to address these.

Course Program and Materials 

Objective

By using case studies that are based on actual IT projects, students will learn how to deal with complex, not-straightforward problems. It will help them to apply their theoretical Computer Science background to real life situations and will teach them fundamental principles of IT management and challenges with IT in practice.
 

Content

The course "Case Studies from Practice" bridges the gap between computer science at ETH and the industry. In interactive case studies company representatives present the diversity of challenges in the daily routine and work of computer science graduates from ETH. As an integral part of the course, non-technical skills such as business problem-solving, communication and presentation, interviewing and relationship management are discussed and trained.

The course is particularly suitable for students who want to gain an overview of the different working areas and want to prepare for entrance into professional life.

Presenting companies so far include Deloitte (how to develop innovative technology solutions for a luxury retailer), Selfnation (lessons learned from a startup company), Credit Suisse (investment banking case), HP (business continuity management), 28msec (product pricing in a software startup company), Open Web Technology (developing a customer-centric digital strategy), Teralytics (Big Data in Practice), and Marc Brandis Strategic Consulting (various). Others will be confirmed as soon as possible.

 

Course Hours

Lectures

  • Mon, 12:00-14:00 ML F 36

Exercise

  • Mon,  14:00-15:00 ML F 36


Lecturer

Marc Brandis

Advanced Computer Networks Spring 2016

News

  • EXAM REVIEW: 
    • 16.09.2016 10:00-12:00 (CAB F 78)
    • 21.09.2016 14:00-16:00 (CAB F 78)
  • Exam: Friday 26.08.2016, 9-11am
  • Second project assignment online (Assignment 8).
  • First project assignment online (Assignment 7).
  • No class or exercise session next week (March 28 - April 1)
  • Slides for lecture 5 available
  • Solution for assignment 3 is available.
  • Assignment 4 is up
  • Slides for fourth lecture are up
  • Solutions for assignment 1 and 2 are available (login with your nethz account)
  • Assignment 3 is up
  • Slides for third lecture are up
  • Slides for second lecture are up
  • Slides for first lecture are up

 

Overview

This course covers a set of advanced topics in computer networks. The focus is on principles, architectures, and protocols used in modern data center networks. 

The goal of the course is to build on basic networking course material in providing an understanding of large, complex networked systems, and provide concrete experience of the challenges through a series of lab exercises.

Topics of this course include:

  • Networking Principles (naming, end-to-end protocol design, network state management, etc.)
  • Data center architectures (topology, addressing, etc.)
  • Data center network protocols (DCTCP, Infiniband, CEE, etc.)
  • End host architectures (U-Net, RDMA, Netmap, etc.)
  • Server and network virtualization
  • Software defined networking
  • Applications and application traffic

Assessment

The course consists of lectures, exercises, and a written examination. The end of semester exam will be 2 hours, with no supporting material allowed. The final assessment will be a combination of exercises and examination grades. The exam will account for 80% of the final grade, and the two programming assignments for 20%.

Staff

Course Hours

  • Lectures: Tue 13-15h, CAB G 51
  • Exercises: Thu 14-16h, CAB G 52

Additional material

Schedule

Lecture Date Topic Materials
L1  23.02  Network design principles I   Slides
L2  01.03  Network design principles II  Slides
L3  08.03   Data centre traffic patterns  Slides
L4  15.03   Data centre topologies I  Slides
L5  22.03   Data centre topologies II Slides
L6  05.04  Server Virtualization Slides
L7  12.04 Software Defined Networking Slides VN
Slides SDN 
L8 19.04  Data Center TCP Slides 
L9  26.04  Lossless Protocols Slides
L10  03.05  Endpoint Optimizations Slides 
L11  10.05 Network Function Virtualization  Slides 
L13  17.05 Load balancing   Slides
L14  24.05 Application empowerment   Slides
L15  31.05     

 

Assignments

Week no. Assign. Date Topic Materials
1 A1 25.02 Network Principles A1-pdf A1-slides A1-solution
2 A2 03.03 Fundamentals and principles A2-pdf A2-slides A2-solution
3 A3 10.03 Data Center Applications & Traffic A3-pdf A3-slides A3-solution
4 A4 17.03  Data Center Topologies A4-pdf A4-slides A4-solution
5  A5  24.03 Addressing & Routing  A5-pdf A5-slides A5-solution
6  A6 07.04  Server Virtualization A6-pdf A6-slides A6-solution
7  P1 14.04 Software Defined Networking A7-pdf A7-slides
 9  P1 21.04   A7W2-slides
10 P1 28.04   --
 11  - 05.05 no exercise (Ascension, public holiday)  --
 12  P1/P2 12.05  RDMA programming A8-pdf A8-handout A8-VM (from ETH network only)
 13  P2 19.05    
14  P2  26.05    
15  P2 02.06    
         

 

Reading list

Lecture 1

  • You Don't Know Jack about Network Performance. Kevin Fall, Steve McCanne. ACM Queue. [link]
  • Improving performance on the internet. Tom Leighton. ACM Communications. [link]
  • On the Naming and Binding of Network Destinations. Jerome Saltzer. [link]

Lecture 2 

  • The Design Philosopy of the DARPA Internet Protocol. David Clark. [link]
  • End-to-End Arguments in System Design. Jerome Saltzer, David Reed, David Clark. [link]
  • A Model, Analysis and Protocol Framework for Soft-State based Communication. Suchitra Raman, Steven McCanne. [link]
  • Transmission Control Protocol Specification. Jon Postel. [link]
  • Cross-Layer Visibility as a Service. R.R. Kompella, A. Greenberg, J. Rexford, A.C Snoeren, J. Yates. [link]

Lecture 3

  • Inside the Social Network’s (Datacenter) Network. A. Roy, et al. [link]
  • Sections 1 and 2 of: Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network. A. Singh, et al. [link]
  • Sections 1, 2, 3, and 4 of: The Nature of Datacenter Traffic: Measurements & Analysis. S. Kandula, et al. [link]

 Lecture 4

  • A Scalable, Commodity Data Center Network Architecture. M. Al-Fares, A. Loukissas, A. Vahdat [link]
  • Jellyfish: Networking Data Centers Randomly. A. Singla, C. Hong, L. Popa, P. B. Godfrey. [link]
  • Optional: Augmenting Data Center Networks with Multi-Gigabit Wireless Links. D. Halperin, et al. [link]
  • Optional: Slim Fly: A Cost Effective Low-Diameter Network Topology. M. Besta, T. Hoefler. [link]

Lecture 5

  • VL2: A Scalable and Flexible Data Center Network. A. Greenberg, et al. [link]
  • CONGA: Distributed Congestion-Aware Load Balancing for Datacenters. M. Alizadeh, et al. [link]
  • Use of BGP for routing in large-scale data centers. P. Lapukhov, et al. [link]
  • Optional: B4: Experience with a Globally-Deployed Software Defined WAN. S. Jain, et al. [link]
  • Optional: PortLand: A Scalable Fault-Tolerant Layer 2 Data Center Network Fabric. R. Niranjan Mysore, et al. [link]
  • Optional: Achieving High Utilization with Software-Driven WAN. C. Hong, et al. [link]

Lecture 6

  • I/O Virtualization. ACM Queue. [link]
  • kvm: the linux virtual machine monitor. [link]
  • virti-io: towards a de-facto standard for virtual I/O devices. [link]

Lecture 7

  • Open vSwitch [link]
  • Implementation Challenges for Software-Defined Networks [link]
  •  

Lecture 8

  • Safe and effective fine-grained TCP retransmissions for datacenter communication. [link]
  • Better never than late: meeting deadlines in datacenter networks. [link]
  • Deadline-aware datacenter TCP. [link]
  • Data Center TCP (DCTCP). [link]
  • Improving Data Center Performance and Robustness with Multipath TCP. [link]
  • Design, Implementation and Evaluation of Multipath TCP. [link]

Lecture 10

  • U-Net: a user-level network interface for parallel and distributed computing [link]
  • TCP offload is a dumb idea whose time has come [link]

Lecture 11

  • Research Directions in Network Service Chaining [link]
  • Languages for Software-Defined Network [link]
  • FlowVisor: A Network Virtualization Layer [link]

Lecture 12

 Lecture 13

  • Fastpass: A Centralized “Zero-Queue” Datacenter Network [link]
  • Enabling End-host Network Functions [link]



 

Advanced Computer Networks Spring 2016

What's new?

  •  

Overview

This course covers a set of advanced topics in computer networks. The focus is on principles, architectures, and protocols used in modern data center networks. 

The goal of the course is to build on basic networking course material in providing an understanding of large, complex networked systems, and provide concrete experience of the challenges through a series of lab exercises.

Topics of this course include:

  • Networking Principles (naming, end-to-end protocol design, network state management, etc.)
  • Data center architectures (topology, addressing, etc.)
  • Data center network protocols (DCTCP, Infiniband, CEE, etc.)
  • End host architectures (U-Net, RDMA, Netmap, etc.)
  • Virtual machine networking
  • Software defined networking

Assessment

The course consists of lectures, exercises, and a written examination. The end of semester exam will be 2 hours, with no supporting material allowed. The final assessment will be a combination of exercises and examination grades. The exam will account for 70% of the final grade, the classroom work for 30%.  It will include two programming assignments and 8 paper exercises given out during the semester.  

 

Staff

Course Hours

  • Lectures: Tue 13-15h, CAB G 51
  • Exercises: Thu 14-16h, CAB G 52

Additional material

 

Schedule

Week no. Lecture Date Topic Materials
         
         

 

Assignments

Week no. Assign. Date Topic Materials
1 A1 19 Feb Network design principles A1-pdf, Slides
2 A2 26 Feb Fundamentals and principles A2-pdf, Slides
3 A3 5 Mar Datacenter topologies  A3-pdf, Slides
4 A4 12 Mar 3-Tier Datacenter Design A4-pdf, Slides
5  A5  19 Mar New Datacenter Architecture A5-pdf, Slides
6  A6 26 Mar Data Center TCP A6-pdf, Slides
7  A7 2 Apr Flow control and DCTCP  A7-pdf, Slides
 9 P1 16 Apr Intro to RDMA programming p1-pdf, Handout, VM (from ETH network only)
10  - 23 Apr -No session-  --
 11  P1 30 Apr RDMA programming Q/A session  
 12 P2 7 May P1 demo, OpenFlow project p2-pdf
 13   14 May --  
14  Jeffrey C. Mogul  21 May  Guest  Lecture  
15   27 May P2 demo  
         

 

Reading list

Lecture 1

  • You Don't Know Jack about Network Performance. Kevin Fall, Steve McCanne. ACM Queue. [link]
  • Improving performance on the internet. Tom Leighton. ACM Communications. [link]
  • On the Naming and Binding of Network Destinations. Jerome Saltzer. [link]

Lecture 2 

  • The Design Philosopy of the DARPA Internet Protocol. David Clark. [link]
  • End-to-End Arguments in System Design. Jerome Saltzer, David Reed, David Clark. [link]
  • A Model, Analysis and Protocol Framework for Soft-State based Communication. Suchitra Raman, Steven McCanne. [link]
  • Transmission Control Protocol Specification. Jon Postel. [link]
  • Cross-Layer Visibility as a Service. R.R. Kompella, A. Greenberg, J. Rexford, A.C Snoeren, J. Yates. [link]
  • A Blueprint for Introducing Disruptive Technology into the Internet. Larry Peterson, Tom Anderson, David Culler and Timothy Roscoe. [link]
  • Overcoming the Internet Impasse through Virtualization. Tom Anderson, Larry Peterson, Scott Shenker, Jonathan Turner. [link]
  • Resilient Overlay Networks. David Anderson, Hari Balakrishnan, Frans Kaashoek, Robert Morris. [link]

Lecture 3

  • Parallel Computer Architecture: A Hardware/Software Approach. David E. Culler and Jaswinder Pal Singh  [link]

Lecture 4

  • The cost of a cloud: research problems in data center networks [link]
  • G.A.A. Santana, "Data Center Virtualization Fundamentals ," Cisco Press, 2013 , ISBN:1587143240

Lecture 5

  • Portland : A Scalable Fault - Tolerant Layer 2 Data Center Network Fabric [link]
  • VL2: Scalable and Flexible Data Center Network [link]
  • Facebook datacenter fabric [link]
  •  The Datacenter as a Computer: an introduction to the Design of Warehouse - Scale Machines, Second Edition. [link]

Lecture 6

  • The nature of data center traffic and analysis. [link]
  • Network traffic characteristics of data centers in the wild. [link]
  • Safe and effective fine-grained TCP retransmissions for datacenter communication. [link]
  • Better never than late: meeting deadlines in datacenter networks. [link]
  • Deadline-aware datacenter TCP. [link]
  • Data Center TCP (DCTCP). [link]
  • Improving Data Center Performance and Robustness with Multipath TCP. [link]
  • Design, Implementation and Evaluation of Multipath TCP. [link]

 Lecture 8

  • U-Net: a user-level network interface for parallel and distributed computing [link]
  • Arsenic: A user-accessible gigabit ethernet interface [link]
  • TCP offload is a dumb idea whose time has come [link]
  • FaRM: fast remote memory [link]

 Lecture 9 

  • OpenFlow: Enabling Innovation in Campus Networks [link]
  • Can the production network be the testbed [link]
  • OpenFlow Specification  [link

Lecture 11

  • TCP Splicing for Application Layer Proxy Performance [link]
  • Ananta: Cloud Scale Load Balancing [link]

 

Lecture 12

  • Managing Data Transfers in Computer Clusters with Orchestra [link]
  • Fastpass: A Centralized "Zero-Queue" Data Center Network [link]

 

Lecture 13

  • Inferring the Network Latency Requirements of Cloud Tenants [link]
  • Cicada: Introducing Predictive Guarantees for Cloud Networks [link]