ECSE 689

ECSE 689

Welcome to ECSE 689

This class explores the technical details behind image synthesis techniques used for realistic visual effects in modern feature films, games, and architectural & product visualizations. Starting with the fundamentals of radiometry, the study and measurement of electromagnetic radiation (like visible light), we present mathematical models of how light propagates in an environment to eventually form an image on a sensor (e.g., a camera's film or the human eye).

Students will code the numerical methods used in industry for visual effects: each student will be guided through the implementation of a renderer capable of generating realistic images based on the physics of light. You can refer to the topics we'll cover in this class, and the programming assignments, below.

Jorge Jimenez's work on realistic graphics in interactive game engines.
Portal 2: a Valve Software production with realistic lighting.

Radiometry & Physics-based Shading

What is light? What are the physical units and intuitive interpretations behind the quantities most commonly used to measure light? How does light interact with a surface? What causes the differences in the appearance of e.g., a metal and a plastic?

We'll begin by quickly establishing a common language to use when discussing the physics of light. The emphasis here, as with the entire course, will be to build an intuition of these concepts, rather than developing overly complex and unnecessarily nuanced mathematical constructs.

Light Transport & Shading Algorithms

Once we understand how light behaves, we'll be able to derive the rendering equation, which governs the energy equilibrium of visible light in an environment. By solving the rendering equation, we can generate realistic images of virtual worlds.

The course details two classes of numerical methods for solving the rendering equation: Monte Carlo and density-estimation techniques. These methods apply to other problems in engineering, physics, biology and statistics; as such, students will develop a very practical "numerical toolbox" for solving many problems they'll face throughout their careers.

Getting your Hands Dirty

After establishing a solid (but not overly complex) theoretical foundation based on intuition and simple numerical principles, the focus will shift to developing practical hands-on experience.

While the emphasis here will be on incrementally building a realistic rendering engine through a sequence of coding tasks, students will also develop the technical and soft skills necessary to succeed in an academic and industrial setting. There are no exams in this course. Students will be evaluated primarily on their ability to build and exercise practical technical skills.

Back to To

Course Topics

The course will combine slide-based lectures with take-home programming assignments. Students will be evaluated exclusively on the successful completion of programming tasks, however attendance and in-class participation tend to correlate with success on these assignments.

The course will follow the structure below, and course slides will be posted on the class' myCourses portal throughout the semester.

0. Introduction

  • 30 seconds on ray-tracing
  • A history of the theories of light
  • Emission processes
  • Radiometry
  • Lambert's Cosine Law

1. Appearance Modeling

  • BRDFs & their properties
  • Diffuse & specular reflection
  • Refraction and Fresnel equations
  • Polarization
  • The reflection equation

2. Microfacet BRDFs

  • Evolving past idealized scattering
  • Conductors vs. dielectrics
  • Parametric BRDFs
  • Microfacet theory & models
  • Data-driven BRDFs

3. Monte Carlo Methods

  • Statistics crash course
  • Monte Carlo integration
  • Sampling distributions
  • Case study: ambient occlusion
  • Importance sampling & stratification

4. Direct Illumination

  • Direct vs. indirect illumination
  • Solid angle vs. area parameterizations
  • Careful importance sampling
  • Modeling & sampling light sources
  • Multiple Importance Sampling

5. Advanced Monte Carlo

  • Advanced variance reduction
  • Control variates
  • Stratified sampling
  • Discrepancy & blue noise
  • Quasi-Monte Carlo

6. Path-space GI

  • Heckbert notation
  • Recursive ray-tracing
  • Path & light tracing
  • Bidirectional path tracing
  • Operator notation

7. Markov Chain MC

  • Paths as Markov Chains
  • Primary-space Metropolis Light Transport
  • Veach-style Metropolis Light Transport
  • Coherence in path space
  • Manifold Exploration

8. Many Light Methods

  • Bias & correlation
  • Virtual point light formulation
  • VPL generation & sampling
  • Singularities
  • Virtual sphere light & Lightcuts

9. Density Estimation

  • Complex light transport (e.g., SDS)
  • Photon mapping
  • Photon emission
  • Density estimation & final gathering
  • Progressive photon mapping

10. Volumetric Media

  • The Radiative Transport Equation
  • The Volume Rendering Equation
  • Biased ray-marching
  • Volumetric path tracing
  • Woodcook tracking

11. Advanced Volumes

  • Volume photon mapping
  • Beam radiance estimate
  • Photon beams
  • Volumetric virtual point lights
  • Virtual beam lights

12. Subsurface Scattering

  • High-order scattering
  • BSSRDF
  • The diffusion approximation
  • Searchlight problem
  • Multipole & advanced diffusion

13. Interactive Rendering

  • Trade-offs & approximation guidelines
  • Basis-space & precomputation methods
  • Screen-space techniques
  • Voxel-based approaches
  • Modern game rendering engines
Back to Top

Course Evaluation

Programming Assignments (90%)

This course has no exams and 90% of your evaluation is based on the successful completion of a series of programming tasks, listed below.

A bootstrapping assignment (worth 5% of your final grade) will help you setup your coding environment and generate your first image. In addition to this assignment #0, the next five programming assignment specifications are set in stone. Together, they combine for a total of 90% of the final score.

We provide you with a self-contained, open-source, extensible rendering engine; by extending only a few modules, you will be able to generate realistic images and implement the same algorithms used by the state-of-the-art in visual effects renderers. The difficulty of each task is roughly proportional to its weight, with the exception of the hacker points we list below: these are purposefully undervalued features meant for students who wish to go beyond the fundamental tasks outlined in the programming tasks.

Hacker Points (10%)

Students are encouraged to implement at least a couple of the (purposefully undervalued) tasks in the hacker points list. The instructor will update this list of tasks with new entries during the semester. The goal here is to provide a mix of easy-and-cool and difficult-and-ultra-cool features, all of which can immediately lead to interesting visual effects.

I will  also remain open to adding entries based on student requests: if you find a paper or feature that you want to interest, I'm happy to discuss possibly adding it to the list.

Course Evaluation Summary

Programming Task Weight Due Date
 Assignment 0
Setup & Basic Renders
5% September 27, 2017
Assignment 1
  Lights & Shadows
10% September 27, 2017
Assignment 2
MC Direct Lighting
15% October 11, 2017
Assignment 3
Advanced Monte Carlo
20% November 8, 2017
 Assignment 4
Path Tracing
20% November 29, 2017
Assignment 5
Photon Mapping
20% December 13, 2017
Hacker Points
[see details above]
up to 10% December 13, 2017

 

Class Participation

I want to promote an open environment of respectful discussion, curiosity, and collaboration towards building understanding. I expect students to ask meaningful and thought-provoking questions about the topics we discuss in class and their experiences implementing them. If a question sparks in-class discussion, that's even better!

Back to Top

Programming Tasks

The programming task handouts are linked below, each one detailing the required tasks as well as the submission procedure. Assignments will be handed in no later than 11:59pm EST on the due date listed in the handout for the assignments. Failure to submit a (valid) assignment on time will result in a score of 0 for that assignment. For some tasks, PBRT references are included after the task description, although students who attend the lectures and ask questions may be able to implement the homework without the need for external resources.

0. Setup + Basic Render

  • Setup your coding environment
  • Familiarize yourself with base code
  • Generate your first images
  • Test assignment submission process
  • Estimated Work: 1 day
  • Due September 27, 2017
  • Handout
  • Download Basecode

1. Lights & Shadows

  • Generate your first realistic image
  • Implement two simple light emitters
  • Implement diffuse reflection
  • Add shadows to your scene
  • Estimated Work: 3 days
  • Due September 27, 2017
  • Handout
  • Download Basecode

2. MC Direct Lighting

  • Sampling different distributions
  • Ambient occlusion
  • Sphere light sampling and shading
  • Estimated Work: 4.5 days
  • Due October 11, 2017
  • Handout
  • Download Basecode

3. Advanced MC

  • Glossy Phong BRDF
  • Phong BRDF importance sampling
  • Multiple importance sampling
  • Estimated Work: 6.5 days
  • Due November 8, 2017
  • Handout
  • Download Basecode

4. Path Tracing

  • Implicit Path Tracing
  • Explicit Path Tracing
  • Rectangular Lights
  • Estimated Work: 4.5 days
  • Due November 29, 2017
  • Handout
  • Download Basecode

5. Photon Mapping

  • Photon Generation/Shooting
  • Photon Density Estimation
  • Estimated Work: 4.5 days
  • Due December 13, 2017
  • Handout
  • Download Basecode

Once you've completed an assignment, you'll have to generate your submission document (a self-contained HTML file that you'll upload to myCourses). Each assignment outlines the process for generating this HTML file, and you can download the script here.

The assignment list will be updated throughout the semester.

Hacker Points - Feature List

As we progress through the semester, this list will be updated with additional extensions you can implement in your renderer that we know to be both fun and challenging. Each item's weight is listed in square brackets:


  • [1%] Render an interesting scene
  • [1 to 3%] Advanced BRDFs (with importance sampling, where possible)
    • Microfacet model w/ Beckmann or Phong distributions (iso- & anisotropic)
    • Dielectric BSDF (glass)
    • Disney BSDF
    • Rough dielectrics (brushed glass, Walter et al. 2007)
  • [1 to 3%] Advanced Emitters (with importance sampling, where possible)
    • Spotlight emitter
    • Other emitter shapes: cylinder, disk, and/or arbitrary mesh
    • Environment maps (use discrete PDFs and/or hierarchical sample warping)
  • [1%] Advanced Camera Models
    • Motion blur
    • Thin lens camera (depth of field)
    • Realistic camera
  • [1% to 5%] Advanced Surface Light Transport
    • Path tracing with multiple importance sampling
    • Progressive photon mapping
    • Stochastic Progressive photon mapping
    • Bidirectional path tracing
    • Bidirectional path tracing with multiple importance sampling
  • [3% to 10%] Volumetric Light Transport with Participating Media
    • Implicit volumetric path tracing
    • Volumetric path tracing with explicit single-scattering
    • Monte Carlo subsurface scattering
    • Diffusion approximations for BSSRDFs

The hacker points feature list will be updated throughout the semester.

Back to Top

How to Reach Me

One of the advantages of taking a graduate-level class is that it can afford a more personalized, interactive learning experience. It's important to me that students get the most out of my course: I want everyone to leave my class with stronger coding, analysis, presentation, and research skills than when they started. Moreover, I want students to develop these skills while having fun: one of the beautiful advantages of working on computer graphics is that the results of your hard work are very tangible.

If you have any concerns about my teaching style, the content of the class, or the structure of the course, please don't hesitate to submit a message using the anonymous form below. You'll also have an opportunity to formally evaluate me during the semester, and this will help to shape future iterations of the course, but I'm also interested in making adjustments during the semester in order to make the course more useful and pleasant for those taking it right now.

Send an anonymous message to the Professor



Back to Top