This year, Spring 2020, CS179 will be taught online, like the other Caltech classes, due to COVID-19.
The course should be "live" and nearly ready to go, starting on Monday, April 6, 2020. We won't be presenting video recordings or live lectures. The materials and slides are intended to be self-contained, found below.
The web page you're viewing is the main webpage for the course.
There *will* be some surveys posted on Piazza, on which day your homework assignments should be due, and also a survey about Zoom meeting scheduling, for interactive office hours with the TAs, with Zoom screen sharing. We're all in different timezones, so the times are listed in PDT, Caltech's timezone. (Please test out your Zoom accounts, ahead of time!)
Meanwhile, the website is still being set up for the online instruction this quarter.
Note that homework will start to be due sometime the second week, on or after Monday April 13, 2020, despite dates in the links. We will select the day after we get preferences from you in the survey, about a preferred day-of-week. In previous years, Wednesday 3pm PDT has been the customary time and date for HW submissions.
|Course Time and Place:|| The course time and place will be primarily online
through this website! Check for links!
In other years, ordinarily MWF 3:00-3:55 PM
|Announcements:|| There may be additional resources for the course on
A Piazza link has also been set up, CS179 Piazza, 2020.
Piazza will be the main forum for discussion, so it is important to make sure you are enrolled there!
The TAs and the CS179 students enrolled on Regis
as of 6pm PDT March 26 have been added to the Piazza list.
For office hours with the TAs, we will be using Zoom, which allows screen sharing and real-time discussion. The Zoom meeting links will be posted on Piazza. It will be good for you to set up your Zoom account or system now, if this hasn't been done already.
A survey for office hour preferences should soon be posted on Piazza, and another survey on preferred HW submission day.
You will be submitting your assignments on a GPU-enabled remote computer located at Caltech in the Barr lab.
The computer accounts have been set up now, and you should have received an email. Please test your accounts, and let us know if there is trouble!
The use of Graphics Processing Units for rendering is well
known, but their power for general parallel computation has only
recently been explored. Parallel algorithms running on GPUs can
often achieve up to 100x speedup over similar CPU algorithms, with
many existing applications for physics simulations, signal
processing, financial modeling, neural networks, and countless
This course will cover programming techniques for the GPU. The course will introduce NVIDIA's parallel computing language, CUDA. Beyond covering the CUDA programming model and syntax, the course will also discuss GPU architecture, high performance computing on GPUs, parallel algorithms, CUDA libraries, and applications of GPU computing.
Problem sets will cover performance optimization and specific GPU applications such as numerical mathematics, medical imaging, finance, and other fields.
This quarter we will also cover uses of the GPU in Machine Learning.
Labwork will require significant programming. A working knowledge of the C programming language will be necessary. Although CS 24 is not a prerequisite, it (or equivalent systems programming experience) is strongly recommended.
9 units; third term.
George Stathopoulos - firstname.lastname@example.org
Ethan Jaszewski - email@example.com
Alden Rogers- @firstname.lastname@example.org
|Supervising Professor:|| Professor
Al Barr - email@example.com
|Programming Guide and Textbook:|| Useful to consult! CUDA Programming Guide
Also perhaps? Programming Massively Parallel Processors (3rd Edtion) is recommended but not required. Amazon Link.
|Office Hours:|| We are still developing methods for "office
hours" for online instruction. We will post Zoom links for Zoom screen
sharing and interactive Zoom sessions with the TAs.
Students and TAs will be in multiple time zones, so it may be tricky to set up office hour times that are convenient.
A survey for office hour time preferences should soon be posted on Piazza.
|Piazza and HW Submission:||
|Grading policy:|| Here is the grading scheme for the class:
Homework extensions may be granted if the TAs see it as appropriate. E grades will not be granted except under extreme circumstances.
Please contact the Deans when
there are unusual or difficult circumstances, since the Deans can
give us more leeway than usual, if they feel your circumstance
Week 1 (Introduction), MWF 3pm PDT
Note that there are no "live" lectures. The course should be self-contained, below, and on Piazza.
Lecture 1 (Mon. 04/06): PPT PDF
(... The following are still being set up for 2020!) Lecture 2 (Wed. 04/08): PPT PDF
Lecture 3 (Fri. 04/10): PPT PDF
Week 2 (Shared Memory), MWF 3pm PDT
Lecture 4 (Mon. 04/13): PPT PDF
Lecture 5 (Wed. 04/15): PPT PDF
Lecture 6 (Fri. 04/17): PPT PDF
Week 3 (Reductions, FFT) MWF 3pm PDT
Lecture 7 (Mon. 04/20): PPT PDF
Lecture 8 (Wed. 04/22): PPT PDF
Lecture 9 (Fri. 04/24): PPT PDF
Week 4 (cuBLAS and Graphics) MWF 3pm PDT
Lecture 10 (Mon. 04/27): PPT PDF Google Doc
Lecture 11 (Wed. 04/29): cuBLAS example
Lecture 12 (Fri. 05/1): PPT PDF
Week 5 (Machine Learning and cuDNN I) MWF 3pm PDT
Lecture 13 (Mon. 05/04): PPT PDF
Lecture 14 (Wed. 05/06): PPT PDF
Lecture 15 (Fri. 05/08): PPT PDF
Week 6 (Machine Learning and cuDNN II) MWF 3pm PDT
Lecture 16 (Mon. 05/11): PPT PDF
Lecture 17 (Wed. 05/13): PPT PDF
Week 7 (Projects) MWF no class. Zoom discussions.
Week 8 (Projects) MWF no class. Zoom discussions.
Week 9 (Projects) MWF no class. Zoom discussions.
Week 10 (Projects) MWF no class. Zoom discussions.
Lab 1: assignment
text. Zipped UNIX files
needs to be updated slightly for 2020) Note: please follow the instructions in this text file instead of the one in the zipped folder
Lab 2: assignment text UNIX files (needs to be updated for 2020)
Lab 3: assignment text UNIX files (needs to be updated for 2020)
Lab 4: assignment text UNIX files (needs to be updated for 2020) Modified 4-27-19. Also, your project proposals will be due on Friday, May 1, so start working on those, and feel free to ask the TAs if you need guidance.
Lab 5: assignment text UNIX files (needs to be updated for 2020)
|CUDA Installation|| There is a Caltech computer to remotely log into, in
the Barr lab in Annenberg, where you should already have the computer
name, an account and
remote access to it.
For installing CUDA onto your own computer, you could also
consider this Guide
(updated 2019) DANGER! Especially for older non-Windows machines, make
a Clone of the whole computer system before attempting installation!
(updated 2019) DANGER! Especially for older non-Windows machines, make a Clone of the whole computer system before attempting installation!Don't try to install CUDA casually. You can easily lose your ability to log in and your entire laptop/desktop environment without this type of backup! The loss of a working computer environment can affect your other classes. With the clone backup, however, you should not lose too much time if there was a problem. But you should know how to clone your computer and then restore it!
Using the Barr machine in Annenberg will be a safer option.
FYI, Ubuntu 20.04 will be coming out shortly, and the installation process for CUDA will be a lot easier and safer, than on older Unix systems. We will try to create an instruction sheet, on how to do this.
To do the full partition backup, a suggested cloning tool is
where you can use these
Clonezilla instructions as a reminder.
Other cloning and burning tools are acceptable, if you have your own favorites.
Finally, use this code to retrieve your hardware info after you setup CUDA.
CUDA C Programming Guide
List of NVIDIA GPUs
Mapping from GPU name to Compute Capability
|Material from previous year(s):||
|Less useful, but cool resources:||
NVIDIA's Parallel Forall Blog
Videos from last several years of NVIDIA's conference on CUDA
How to Write Code the Compiler Can Actually Optimize (2015)
Excellent CPU optimization manuals
What Every Programmer Should Know About Memory
GPU focused systems guide to deep learning