Computer Science Department, University of Crete
HY-590.45. Modern Topics in Scalable Storage Systems

info | readings | syllabus | other resources

Course Staff

Name Email Office Hours
Instructor: Kostas Magoutis hy590-45@csd By appt./H-311
Teaching Assistant: Tian Changda csdp1340@csd By appt.

General Information

The course meets on Mon, Wed 6-8pm in H.208.

Announcements

1.3.2024 10:00: Your project proposals are due Wed 10/4 11:59pm.

27.2.2024 10:00: Paper assignments for student presentations are noted on the link below.

20.2.2024 10:00: You are welcome to go over the list of proposed papers for your presentations. Your selections of two papers per student are due by Monday 26/2.

10.2.2024 10:00: We will be using the AWS Academy cloud platform for course assignments, you may find our course page here

12.1.2024 10:00: The course will start on Monday 12/2

1.1.2024 10:00: You are welcome to get in touch with the instructor to discuss course-related issues

Course Description

The explosive growth of information processing services in recent years has created an unprecedented need for storage capacity. Scalable access to storage resources requires a class of distributed systems designed for fast, reliable, and uninterrupted access to storage media (e.g., magnetic disks and tapes) over high-speed networks. This course offers an introduction to scalable storage systems and examines existing design techniques as well as current research problems in the design and implementation of such systems, along with possible solutions.

Some of the advantages of the scalable storage model over direct-attached storage include expandable capacity and performance, as well as improved utilization and sharing of distributed storage resources. A number of challenges, however, are facing the scalable storage systems architect: First, it is the higher complexity (compared to direct-attached storage) due to the distributed nature of the scalable storage system. Administration, capacity planning, configuration, backup, and disaster recovery are complicated in large-scale scalable storage systems. Second, transferring data over the network requires stronger security and safety guarantees than when transferring them on the system I/O bus. In addition, it sometimes requires new, storage-specific network transport protocols. These and other challenges make scalable storage an exciting research area that has made significant advances in recent years.

The core part of the course focuses on the study of scalable storage systems with special emphasis on architectures, design principles for scalable performance, reliability, and availability, the management of data during their lifecycle, application-specific design concepts, ways to reduce implementation cost, storage system capacity planning, and storage outsourcing services.

This course is targeted for graduate students and advanced undergraduates and requires the undertaking of a research project. The topics of the research projects will be chosen with the help and guidance of the course staff.

Coursework

Prerequisites

Grading

The final grade depends on class participation, presentation of two research papers, and a research project.

Readings

 There are a number of paper readings that are available online. You are expected to read the papers before the beginning of each class.

There is no required textbook for this class. The following textbooks, however, are recommended readings:

Syllabus

Date Topic Readings, notes
Mon 12/2 Course overview -
Mon 19/2 Background -
Wed 21/2 Background II -
Mon 26/2 Extending file systems over the network Sandberg: Design and implementation of the Sun Network Filesystem
Wed 28/2 NFS (contd.) Macklem: Not Quite NFS, Soft Cache Consistency for NFS
Mon 4/3 Distributed coordination Ongaro: In Search of an Understandable Consensus Algorithm
Wed 6/3 Raft (contd.) Lecture slides, visualization
Mon 11/3 Distributed virtual disks Lee: Petal: Distributed virtual disks
Wed 13/3 Petal (contd.) Lecture slides
Mon 18/3 Clean Monday - no class -
Wed 20/3 Petal (contd.), Frangipani Lecture slides
Mon 25/3 National holiday - no class -
Wed 27/3 Related work presentations I 1315, 1327, 1334
Mon 1/4 No class today -
Wed 3/4 Related work presentations I 1339, 1344, 1346
Mon 8/4 Related work presentations I 1348, 1355, 1365, 1367
Wed 10/4 11:59pm Project proposals due -
Wed 10/4 Class rescheduled for Fri 12/4 -
Fri 12/4 Distributed file systems I Thekkath: Frangipani: A Scalable Distributed File System
Mon 15/4 Distributed file systems II Ghemawat: The Google File System
Wed 17/4 Google File System (contd.) Lecture slides
Mon 22/4 Instructor out of town, no class -
Wed 24/4 Tutorial on AWS Academy Learner Lab (TA) -
29/4-10/5 Easter recess -
Mon 13/5 Application-specific storage systems Saito: Manageability, availability and performance in Porcupine
Wed 15/5 Related work presentations II 1315, 1327, 1334
Mon 20/5 Related work presentations II 1339, 1344, 1346
Wed 22/5 Related work presentations II 1348, 1355, 1365, 1367

Projects HOWTO

Please note the following project guidelines:

Other Resources / Useful links