GPU Programming

CS 179

Annenberg 105, MW(F) 3-3:55pm

This year, Spring 2022, CS179 is taught in person.

Some of this website still retains last year's information, and will be updated shortly.

Prof. Al Barr

Project information has been updated near end of web page. Also see CS179 Project Information

You will be receiving email invitations and Piazza "logistics" announcements.

See the course' Canvas Calendar to see links for TA office hours (TBA) and for uploading assignments (not onto Canvas) but onto a computer in the Barr lab.

FYI, last year's classes and recitations were recorded using Zoom, with links to be posted on Piazza in a "Zoom recordings" folder.

The web page you're viewing is the main webpage for the course.

A Piazza link has been set up, CS179 Piazza, 2022 Everyone enrolled in the course on Regis as of March 27, 2022 should have received an email invitation to join the Piazza discussion. If you added the course after that, please add yourself to Piazza.

The course is "live" and nearly ready to go. The lectures, recitations and other materials and slides are intended to be self-contained, listed below, for the first six weeks of the course. There are no lectures or assignments the last four weeks, which will let you focus on your GPU project.

Due Dates (adjusted for 2022, project proposal and lab 6) & Office Hours

Homework sets are due Tuesday, 3 pm, with the following due dates, to be uploaded onto a computer in the Barr lab.

  • Lab 1 is due Tuesday, April 5, 2022.
  • Lab 2 is due Tuesday, April 12, 2022.
  • Lab 3 is due Tuesday, April 19, 2022.
  • Lab 4 is due Tuesday, April 26, 2022.
  • Lab 5 AND Project Proposal are due on Tuesday May 3, 2022,
  • Note that the Project Proposal is now due alongside Lab 5.
  • See CS179 Project Information
  • Lab 6 is now due Tuesday May 10, 2022.
  • After six weeks, lectures and Recitations will cease, although office hours will continue, allowing you to focus on your approximately four-week projects. Office hours may change to "appointment style."
  • Final project deadlines. Final projects are due Friday June 3 for seniors and grad students, and June 10 for everyone else.
  • Note that we will need to submit your Senior and graduate student grades by Monday June 6, and other undergraduate grades by Wednesday June 15. Commencement for seniors is on Friday, June 10.

Most office hours will be live, TBA, but some will also on Zoom, every week. Times are in Pacific Time, TBA.

For some of the office hours with the TAs, we will using Zoom, which allows screen sharing and real-time discussion. The Zoom meeting links for the TA office hours will be posted on Piazza, but also can be "clicked" on, on the Canvas Calendar.

The TA Office hours take place:

  • TBA

For the later part of the course, we may be transitioning to "appointment-style" office hours for your projects for quick questions with the TAs, approximately mid May, since the homework assignments would mainly be finished then.

At that point, email a TA ahead of time to confirm, but they can likely be available at other times too, for quick questions to help you with your project, if you can set up a Zoom time with them in email.

Course Time and Place: The course time and place is Annenberg 105 MW(F) 3:00 - 3:55PM PDT
Announcements: There will be additional resources for the course on Canvas, such as the Canvas Calendar which will list Zoom links.

See CS179 2022 Syllabus and the 2022 CS179 Canvas Calendar that has the Zoom links.

A Piazza link has been set up, CS179 Piazza, 2022.

Piazza will be the main forum for discussion, so it is important to make sure you are enrolled there!

You will be submitting your assignments on a GPU-enabled remote computer located at Caltech in the Barr lab.

Important! Instead of using Canvas, or emailing your lab solution to a TA email like some assignments may accidentally ask, please put a zip file of your solution in your home directory on the remote Barr-lab computer, with the name lab[N]_2022_submission.zip .

See Piazza for more details!

The remote computer accounts have been set up. You will soon receive an email. Please test your accounts, and let us know if there is trouble!

Course Description: The use of Graphics Processing Units for rendering is well known, but their power for general parallel computation has only recently been explored. Parallel algorithms running on GPUs can often achieve up to 100x speedup over similar CPU algorithms, with many existing applications for physics simulations, signal processing, financial modeling, neural networks, and countless other fields.

This course covers programming techniques for the GPU. The course will introduce NVIDIA's parallel computing language, CUDA. Beyond covering the CUDA programming model and syntax, the course will also discuss GPU architecture, high performance computing on GPUs, parallel algorithms, CUDA libraries, and applications of GPU computing.

Problem sets cover performance optimization and a few specific example GPU applications such as numerical mathematics, medical imaging, finance, and other fields, ending with a 4-week project of the student's choice.

This quarter we will also cover uses of the GPU in Machine Learning.

Labwork will require significant programming. A working knowledge of the C programming language will be necessary. Although CS 24 is not a prerequisite, it (or equivalent systems programming experience) is strongly recommended.

9 units; third term.

   
Instructors/TAs: Thomas Barrett - (Head TA) tbarrett@caltech.edu
Michael Valverde - michael@caltech.edu
TBA

The TA Zoom Office Hour times will be listed soon, above, on this page.

Supervising Professor: Professor Al Barr - barradmin@cms.caltech.edu

 
Programming Guide and Textbook: Useful to consult! CUDA Programming Guide
Also perhaps? Programming Massively Parallel Processors (3rd Edition) is recommended but not required. Amazon Link.

Office Hours: We will post some Zoom links for Zoom screen sharing and interactive Zoom sessions with the TAs.
Piazza and HW Submission:
  • Piazza Please ask through Piazza if you have a question/issue that likely affects other students. Piazza will be used to make announcements throughout the course.
  • Global TA email: cs179.ta@gmail.com.
    But use Piazza, generally, for questions on the assignments or the material. These may be of interest to other people. Send an email to the TAs if you have something that only affects you or your project group.
  • HW submission: You will be logging in remotely onto a Caltech computer in the Barr lab to submit your work. The TAs will log in there to test your work and grade it.
  • IMPORTANT: updated for 2022 Instead of emailing your lab solution to the TA email, please put a zip file of your solution in your home directory on the remote Barr-lab computer, with the name lab[N]_2022_submission.zip .

    See Piazza for more details!

Grading policy: Here is the grading scheme for the class:
  • 6 labs (60% of grade)
  • 4 week project (40% of grade)
All labs will be scored out of 100 and are weighted equally (meaning each lab is worth 10% of your grade). The final project can be completed individually or in a small team, as a pair.

Homework extensions may be granted if the TAs see it as appropriate. E grades will not be granted except under extreme circumstances.

Mainly ask TA Thomas Barrett for extensions, not Prof. Barr!

Please contact the Deans when there are unusual or difficult circumstances, since the Deans can give us more leeway than usual, if they feel your circumstance warrants it.  

Note! To pass the course, a "sufficient" number of "good" assignments will need to be submitted and graded before Drop Day!!! That's May 20, 2022. If you're behind or not doing well, please drop the course before Drop Day!! We may send out a "scary note" in email near that time, to students who may be in trouble of not passing, and might need to drop.

"Lectures" Online: Week 1 (Introduction), MW(F) 3pm PDT
Lecture 1 (Mon. 03/29): PPT PDF
Lecture 2 (Wed. 03/31): PPT PDF
Lecture 3, Recitation (Fri. 04/02): PPT PDF

Week 2 (Shared Memory), MW(F) 3pm PDT
Lecture 4 (Mon. 04/05): PPT PDF
Lecture 5 (Wed. 04/07): PPT PDF
Lecture 6 Recitation (Fri. 04/09): PPT PDF

Week 3 (Reductions, FFT) MW(F) 3pm PDT
Lecture 7 (Mon. 04/12): PPT PDF
Lecture 8 (Wed. 04/14): PPT PDF
Lecture 9 (Fri. 04/16): PPT PDF

Week 4 (cuBLAS and Graphics) MW(F) 3pm PDT
Lecture 10 (Mon. 04/19): PPT PDF Google Doc
Lecture 11 (Wed. 04/21): cpp Text a Annotated cuBLAS example, also Bunny Point Alignment Derivation, PDF
Lecture 12 (Fri. 04/23): PPT PDF Recitation, cuBLAS, cuSolver, and Point Alignment

Week 5 (Machine Learning and cuDNN I) MW(F) 3pm PDT
Lecture 13 (Mon. 04/26): PPT PDF
Lecture 14 (Wed. 04/28): PPT PDF
Lecture 15 (Fri. 04/30): PPT PDF

Week 6 (Machine Learning and cuDNN II) MW(F) 3pm PDT
Lecture 16 (Mon. 05/03): PPT PDF
Lecture 17 (Wed. 05/05): PPT PDF
Lecture 18 (Fri. 05/07): PPT PDF

Week 7 (Projects) MW(F) no class. Zoom Lab hours.
Week 8 (Projects) MW(F) no class. Zoom Lab hours.
Week 9 (Projects) MW(F) no class. Zoom Lab hours.
Week 10 (Projects) MW(F) no class. Zoom Lab hours.

Assignments:
  • Lab 1: assignment text. Zipped UNIX files (no updating for 2022)
    Note: please follow the instructions in the above text file instead of the instructions in the zipped folder

  • Lab 2: assignment text UNIX files (no updating for 2022)

  • Lab 3: assignment text UNIX files (no updating for 2022)

  • Lab 4: assignment text UNIX files (no updating for 2022)
  • Lab 5: assignment text UNIX files (Lab 5 is due on May 3, 2022.) The Machine Learning Lab 6 assignment is also here.
    Also, your Project Proposals will be due now, on Tuesday, May 3, so start working ahead of time on those, and feel free to ask the TAs if you need guidance.

    Lab 6: (Whoops!) Had an "old" Lab 6 here, on PDE solving. Instead, use the Machine Learning Lab 6 "inside" of Lab 5! Lab 6 is due Tuesday May 10, 2022.

Project PROJECT INFO (Updated for 2022)

CUDA Installation There is a Caltech computer to remotely log into, in the Barr lab in Annenberg, where you should already have the computer name, an account and remote access to it.

For installing CUDA onto your own computer, you could also consider this Guide

(In 2019) DANGER! Especially for older non-Windows machines, make a Clone of the whole computer system before attempting installation! Don't try to install CUDA casually. You can easily lose your ability to log in and your entire laptop/desktop environment without this type of backup! The loss of a working computer environment can affect your other classes. With the clone backup, however, you should not lose too much time if there was a problem. But you should know how to clone your computer and then restore it!

Using the Barr machine in Annenberg will be a safer option.

FYI, Ubuntu 20.04 came out, and the installation process for CUDA is a lot easier and safer, than on older Unix systems. We will try to create an instruction sheet, on how to do this.

To do the full partition backup, a suggested cloning tool is Clonezilla, where you can use these Clonezilla instructions as a reminder.
An excellent USB "burning" tool (for making a Clonezilla drive or the CUDA boot drive) is Rufus, although it requires a Windows environment to run it.

Other cloning and burning tools are acceptable, if you have your own favorites.

Finally, use this code to retrieve your hardware info after you setup CUDA.

Resources: CUDA C Programming Guide
List of NVIDIA GPUs
Mapping from GPU name to Compute Capability

Material from previous year(s): 2015
2016
2017
2018
2019


Less useful, but cool resources: NVIDIA's Parallel Forall Blog
Videos from last several years of NVIDIA's conference on CUDA
How to Write Code the Compiler Can Actually Optimize (2015)
Excellent CPU optimization manuals
What Every Programmer Should Know About Memory
GPU focused systems guide to deep learning