GPU Programming

CS 179

This year, Spring 2020, CS179 is taught online, like the other Caltech classes, due to COVID-19.

The course is "live" and ready to go, starting on Monday, April 6, 2020. We won't be presenting video recordings or live lectures. The materials and slides are intended to be self-contained, found below.

The web page you're viewing is the main webpage for the course.

Due Dates & Office Hours

As of May 14, 2020, we will soon be transitioning to "appointment" style TA office hours. The purpose is for quick questions about your project, now that the assignments are mostly finished. Let a TA know by email that you will be wanting some help, ahead of time, for their regular office hours, or perhaps at other times.

Based on the 2020 Piazza surveys, homework sets are due Tuesdays 3 pm. The following dates should match the homework due dates on Piazza.

  • Lab 1 is due Tuesday, April 14, 2020.
  • Lab 2 is due Tuesday, April 21, 2020.
  • Lab 3 is due Tuesday, April 28, 2020.
  • Lab 4 AND Project Proposals are due Tuesday, May 5, 2020.
  • Labs 5 and 6. Lab 5 is due on May 12, 2020, while Lab 6 is optional, and can count for 10% extra credit, and is due May 19, 2020.
  • Final project deadlines. Final projects are due June 3 for seniors and grad students, and June 10 for everyone else. Note that the project proposal was due alongside Lab 4.

Office hours are on Zoom, every week. Times are in Pacific Time. See Piazza for Zoom links. As mentioned above, we may be transitioning to "appointment-style" office hours for your projects for quick questions as of May 14, since the homework assignments are mainly finished now.

Email a TA ahead of time to confirm, but they can likely be available at other times too, for quick questions to help you with your project, if you can set up a Zoom time with them in email.

  • Saturday 1-4pm -- Ethan
  • Sunday 1-4pm -- Alden
  • Monday 1-4pm -- George

Course Time and Place: The course time and place will be primarily online through this website! Check for links! In other years, ordinarily MWF 3:00-3:55 PM
 
Announcements: There may be additional resources for the course on Moodle, see CS179 2020 on Moodle

A Piazza link has been set up, CS179 Piazza, 2020.

Piazza will be the main forum for discussion, so it is important to make sure you are enrolled there!

For office hours with the TAs, we will be using Zoom, which allows screen sharing and real-time discussion. The Zoom meeting links will be posted on Piazza. It will be good for you to set up your Zoom account or system now, if this hasn't been done already.

You will be submitting your assignments on a GPU-enabled remote computer located at Caltech in the Barr lab.

The computer accounts have been set up now, and you should have received an email. Please test your accounts, and let us know if there is trouble!

Course Description: The use of Graphics Processing Units for rendering is well known, but their power for general parallel computation has only recently been explored. Parallel algorithms running on GPUs can often achieve up to 100x speedup over similar CPU algorithms, with many existing applications for physics simulations, signal processing, financial modeling, neural networks, and countless other fields.

This course covers programming techniques for the GPU. The course will introduce NVIDIA's parallel computing language, CUDA. Beyond covering the CUDA programming model and syntax, the course will also discuss GPU architecture, high performance computing on GPUs, parallel algorithms, CUDA libraries, and applications of GPU computing.

Problem sets cover performance optimization and a few specific example GPU applications such as numerical mathematics, medical imaging, finance, and other fields, ending with a 4-week project of the student's choice.

This quarter we will also cover uses of the GPU in Machine Learning.

Labwork will require significant programming. A working knowledge of the C programming language will be necessary. Although CS 24 is not a prerequisite, it (or equivalent systems programming experience) is strongly recommended.

9 units; third term.

   
Instructors/TAs: George Stathopoulos - gstathop@caltech.edu
Ethan Jaszewski - ejaszews@caltech.edu
Alden Rogers- @abrogers@caltech.edu

Supervising Professor: Professor Al Barr - barradmin@cms.caltech.edu

 
Programming Guide and Textbook: Useful to consult! CUDA Programming Guide
Also perhaps? Programming Massively Parallel Processors (3rd Edition) is recommended but not required. Amazon Link.

Office Hours: We have posted Zoom links for Zoom screen sharing and interactive Zoom sessions with the TAs.

Students and TAs are in multiple time zones, so it was tricky to set up office hour times that were convenient enough.

As of May 14, we will be transitioning more to "appointment-style" office hours, now that the assignments are mainly finished. The purpose is for quick questions to help you with your projects. Email a TA ahead of time, and other times can likely be available.

Again, the original Office hours were:

  • Saturday 1-4pm -- Ethan
  • Sunday 1-4pm -- Alden
  • Monday 1-4pm -- George
Zoom links are on Piazza.

 

Piazza and HW Submission:
  • Piazza Please ask through Piazza if you have a question/issue that likely affects other students. Piazza will be used to make announcements throughout the course.
  • Global TA email: cs179.ta@gmail.com.
    But use Piazza, generally, for questions on the assignments or the material. These may be of interest to other people. Send an email to the TAs if you have something that only affects you or your project group.
  • HW submission: You will be logging in remotely onto a Caltech computer in the Barr lab to submit your work. The TAs will log in there to test your work and grade it.
  • IMPORTANT: updated for 2020 Instead of emailing your lab solution to the TA email, please put a zip file of your solution in your home directory on the remote Barr-lab computer, with the name lab[N]_2020_submission.zip .

    See Piazza for more details!

Grading policy: Here is the grading scheme for the class:
  • 6 labs (60% of grade) (but in five weeks, due to COVID-19)
  • 4 week project (40% of grade)
All labs will be scored out of 100 and are weighted equally (meaning each lab is worth 10% of your grade). The final project can be completed individually or in a small team, as a pair.

Homework extensions may be granted if the TAs see it as appropriate. E grades will not be granted except under extreme circumstances.

Please contact the Deans when there are unusual or difficult circumstances, since the Deans can give us more leeway than usual, if they feel your circumstance warrants it.  

Note! To pass the course, a "sufficient" number of "good" assignments will need to be submitted and graded before Drop Day!!! That's May 20, 2020. If you're behind or not doing well, please drop the course before Drop Day!! We may send out a "scary note" in email near that time, to students who may be in trouble of not passing, and might need to drop.

"Lectures" Online: Week 1 (Introduction), MWF 3pm PDT
Note that there are no "live" lectures. The course is intended to be self-contained, below, and on Piazza.
Lecture 1 (Mon. 04/06): PPT PDF
(... The following are still being set up for 2020!)
Lecture 2 (Wed. 04/08): PPT PDF
Lecture 3 (Fri. 04/10): PPT PDF
Week 2 (Shared Memory), MWF 3pm PDT
Lecture 4 (Mon. 04/13): PPT PDF
Lecture 5 (Wed. 04/15): PPT PDF
Lecture 6 (Fri. 04/17): PPT PDF
Week 3 (Reductions, FFT) MWF 3pm PDT
Lecture 7 (Mon. 04/20): PPT PDF
Lecture 8 (Wed. 04/22): PPT PDF
Lecture 9 (Fri. 04/24): PPT PDF
Week 4 (cuBLAS and Graphics) MWF 3pm PDT
Lecture 10 (Mon. 04/27): PPT PDF Google Doc
Lecture 11 (Wed. 04/29): cuBLAS example
Lecture 12 (Fri. 05/1): PPT PDF
Week 5 (Machine Learning and cuDNN I) MWF 3pm PDT
Lecture 13 (Mon. 05/04): PPT PDF
Lecture 14 (Wed. 05/06): PPT PDF
Lecture 15 (Fri. 05/08): PPT PDF
Week 6 (Machine Learning and cuDNN II) MWF 3pm PDT
Lecture 16 (Mon. 05/11): PPT PDF
Lecture 17 (Wed. 05/13): PPT PDF
Week 7 (Projects) MWF no class. Zoom discussions.
Week 8 (Projects) MWF no class. Zoom discussions.
Week 9 (Projects) MWF no class. Zoom discussions.
Week 10 (Projects) MWF no class. Zoom discussions.

Assignments:
Project PROJECT INFO

CUDA Installation There is a Caltech computer to remotely log into, in the Barr lab in Annenberg, where you should already have the computer name, an account and remote access to it.

For installing CUDA onto your own computer, you could also consider this Guide

(In 2019) DANGER! Especially for older non-Windows machines, make a Clone of the whole computer system before attempting installation! Don't try to install CUDA casually. You can easily lose your ability to log in and your entire laptop/desktop environment without this type of backup! The loss of a working computer environment can affect your other classes. With the clone backup, however, you should not lose too much time if there was a problem. But you should know how to clone your computer and then restore it!

Using the Barr machine in Annenberg will be a safer option.

FYI, Ubuntu 20.04 will be coming out shortly, and the installation process for CUDA will be a lot easier and safer, than on older Unix systems. We will try to create an instruction sheet, on how to do this.

To do the full partition backup, a suggested cloning tool is Clonezilla, where you can use these Clonezilla instructions as a reminder.
An excellent USB "burning" tool (for making a Clonezilla drive or the CUDA boot drive) is Rufus, although it requires a Windows environment to run it.

Other cloning and burning tools are acceptable, if you have your own favorites.

Finally, use this code to retrieve your hardware info after you setup CUDA.

Resources: CUDA C Programming Guide
List of NVIDIA GPUs
Mapping from GPU name to Compute Capability

Material from previous year(s): 2015
2016
2017
2018
2019


Less useful, but cool resources: NVIDIA's Parallel Forall Blog
Videos from last several years of NVIDIA's conference on CUDA
How to Write Code the Compiler Can Actually Optimize (2015)
Excellent CPU optimization manuals
What Every Programmer Should Know About Memory
GPU focused systems guide to deep learning