**ECSE 446/546: Realistic/Advanced Image Synthesis**
*_TinyRender Setup_*
_TinyRender_ is a minimalistic rendering engine written in C++11 that runs on Linux, macOS and Windows. The code we provide lays the foundations for the homework assignments. This codebase supports both real-time (online/interactive) and physically-based (offline) rendering paradigms.
While TinyRender provides significant scaffolding to simplify the development of full-fledged renderers, the code you will start with doesn't yet generate any impressive output: it loads a scene and either runs an placeholder OpenGL shader (in _online mode_) or saves a debug image as an OpenEXR file (in _offline mode_) --- all meaningful shading code is missing, and so these outputs consist of only black pixels. During the semester, you will be tasked with extending this system into a relatively complete renderer. The programming assignments will guide you through this incremental process.
(#) Core features
TinyRender provides many features that would be tedious to implement from scratch, including:
+ A [TOML](https://github.com/toml-lang/toml)-based scene file parser and loader
+ A [loader](https://github.com/syoyo/tinyobjloader) for Wavefront OBJ 3D geometry files
+ Support for saving HDR image outputs in the [OpenEXR](http://www.openexr.com) format
+ Basic linear algebra code for point/vector/normal/ray/bounding boxes
+ A pseudorandom number generator
+ An OpenGL library for real-time rendering ([SDL2](https://www.libsdl.org))
+ An optimized ray tracer with [bounding volume hierarchy builder](https://github.com/brandonpelfrey/Fast-BVH) and fast ray-triangle intersection routines
(##) Permissible reference sources
In addition to the optional PBRT course textbook, you should feel free to consult additional references when completing the homework assignments so long as you remember to cite them in an attached `readme.txt` file.
When asked to implement some feature, you should not copy or build atop existing implementations from other renderers. Doing so both hampers your learning (which will be tested on the final exam) and is unethical. We will obviously be on the lookout for this. That being said, referring to PBRT -- which includes a lot of implementation insights -- is perfectly reasonable. When in doubt about this rule or the propoer use of any references, ask the TAs.
(#) Instructions
Each of the five (5) assignments has a __strict deadline__ and submission instructions, both outlined in its handout. All the assignments share a common codebase, which we will release at the beginning of the semester.
We strongly urge students to start working on the assignments as early as possible. Building your own advanced renderer is often a very rewarding experience!
(#) Source code
Assignments are built incrementally and you will sometimes have to reuse parts of your code across assignments. A unified codebase is provided at the beginning of the semester, containing most of the pieces required to complete the tasks for all assignments. Some parts of the code may change throughout the semester -- e.g. due to unforeseen bugs, driver and OS compatibility issues, etc. -- which is why we are making the the code accessible through a Git repository. This will allow the TAs, if necessary, to seamlessly push modifications with minimal impact.
(##) Git prerequisites
If you are completely unfamiliar with version control using Git, we suggest reading the [GitLab documentation](https://docs.gitlab.com/ee/gitlab-basics/start-using-git.html) for a good overview. Only a small subset of Git commands will be used during the course, so beginner-level knowledge is sufficient. Version control commands can be executed either in the terminal or with an appropriate GUI. If you decide to use a GUI, we recommend [SmartGit](https://www.syntevo.com/smartgit/). Using the terminal is straightforward, and we will only provide instructions for this mode. On Windows, you will need to download [GitBash](https://gitforwindows.org/). For Linux / macOS, open up a terminal and Git should be installed by default. If not, you can obtain it [here](https://git-scm.com/downloads).
(##) Cloning the source code
Navigate to the directory you wish to clone our codebase repository into and run:
~~~ bash
git clone https://github.com/ecsex46/fall-2019.git
~~~
That's it, you're set for the semester! Note that this can take a few minutes since large scene files also need to be downloaded alongside the base code.
(##) Syncing after a push
In the event of a codebase patch by the TAs, we will make an announcement on __myCourses__ and ask that students sync the updated codebase version with their local codebase. These patches should _not_ affect any code students' have added to the base codebase, which means that _no conflicts_ should occur during the path (unless you've modified parts of the code you weren't instructed to). Pulling from the repository to obtain the updated codebase should therefore only require executing the following commands:
~~~ bash
git stash save
git pull
git stash pop
~~~
These commands should be called within the local directory where the repository was initially cloned. In short, these Git commands save your changes, synchronize your local tree with the (patched) master version, and then re-applies your changes. If done successfully, you should see which files were altered during the pull. If it's your first time using Git, we recommend duplicating your project directory as backup for extra safety; you can always revert back to it if something goes wrong.
(##) A note on public forks and plagiarism
While it is tempting to upload your personal rendering engine to the public domain, we ask that students _neither_ fork TinyRender nor upload any of its files online. In any cases of plagiarism, both the student who uploaded their files publicly __and__ students that plagiarize from these files will be at fault and will face disciplinary charges. Graders will automatically check your code against all other students' code -- both past and present -- as well as third-party external rendering codebases, in order to detect and flag any similiarities in code submitted for the assignments.
(#) Building the codebase
TinyRender has very few dependencies that require manual installation. It relies on [GLEW](http://glew.sourceforge.net/), [SDL2](https://www.libsdl.org/index.php) and [Boost](https://www.boost.org) (Mac-only). Additionnally, the [CMake](https://cmake.org) build system is required to compile and link TinyRender. We __strongly suggest__ using an IDE, such as [CLion](https://www.jetbrains.com/clion) (Linux / macOS) or [Visual Studio 2019](https://visualstudio.microsoft.com) (Windows 10), to complete the assignments. We provide building instructions for these two IDEs -- you are free to use any other IDE, however without our support. We do _not_ recommend using CLion on Windows as the real-time component of the assignments will not work out-of-the-box.
If you've never used these IDEs before, make sure to look up tutorials online on how to get setup. Most importantly, learning to use the IDE debugging tools can significantly improve your ability to debug your code. If you don't know how to use them, now's a good time to learn -- the TAs will provide some basic support here.
(##) Linux / macOS
(###) Installing dependencies on Linux
Installing the binary dependencies on Linux is straightforward, assuming you have a working [`gcc` compiler](https://gcc.gnu.org) which comes with most modern Linux distributions, such as Ubuntu and Arch Linux:
~~~ Bash
sudo apt-get update
sudo apt-get install libglew-dev libsdl2-dev cmake
~~~
(###) Installing dependencies on macOS
Begin by installing a reasonably up-to-date version of [XCode](https://developer.apple.com/xcode) (>= 8.0) along with the command-line tools. To check if the full XCode package is (already) installed, open a terminal and execute:
~~~ Bash
xcode-select -p
~~~
If you see:
~~~ Bash
/Applications/Xcode.app/Contents/Developer
~~~
you have everything set up. Next, install [Homebrew](https://brew.sh) by executing:
~~~ Bash
/usr/bin/ruby -e
"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
~~~
Then install the required packages, with:
~~~ Bash
brew install boost glew sdl2 cmake
~~~
You can verify that all packages were installed correctly by typing `brew list`.
!!! Note:
Switching macOS release mid-way through the semester __can cause compilation errors__ due to delays in porting dependencies to new OS versions. As such, we do _not_ recommend that you update your OS to [Catalina](https://www.apple.com/ca/macos/catalina-preview/) until the semester is over. If you do, TAs will _not_ assist you in fixing OS-related compatibility issues.
(###) Setting up TinyRender in CLion
Open CLion, click _Open..._ and select the `CMakeLists.txt` file at the root of the TinyRender source directory for the assignment. Select _Open as Project_ when prompted and let CLion load the project. CMake will ensure that you have all the dependencies installed to build TinyRender. If no errors appear, you should see `[Finished]` at the bottom of the CMake console, preceeded by a list of successful checks.
To build TinyRender, simply execute _Run_ -> _Build_. This will compile and link your program in _Debug_ mode. To create a _Release_ build, navigate to _File_ -> _Settings_ (_Preferences_ on macOS). In the tab _Build, Execution, Deployment_ -> _CMake_, click the + icon. You should now see _Debug_ and _Release_ under _Profiles_ (see figure below).
![Figure [clion-cmake]: Adding the _Release_ mode to a CMake build in CLion.](imgs/build_clion_cmake.png)
You can select your build profile in the top-right corner dropdown in CLion and click the button to launch the build.
(##) Windows
(###) Installing / modifying Visual Studio
To compile and run TinyRender on Windows 10, first install a copy of [Visual Studio 2019](https://visualstudio.microsoft.com) (Community edition will suffice). During the installation setup, you will be asked which workloads you wish to install. Select _Universal Windows Platform development_ and _Desktop development in C++_. This should automatically add the latest Windows 10 SDK (10.0.18362.0) to the installation, which you can confirm on the right-hand _Summary_ panel. Once complete, click _Install_. Note that Visual Studio is a pretty heavy software package and will require 10GB+ of disk space.
If you already have VS2019 installed, launch _Visual Studio Installer_ from Start menu and then click _Modify_. Make sure that you have the correct workloads installed along with the latest Windows 10 SDK. Click _Modify_ in the bottom-right corner and let the installation finish.
![Figure [vs-install]: Installing / modifying Visual Studio.](imgs/install_vs.PNG)
(###) Setting up TinyRender in Visual Studio
We provide a solution (`.sln`) file which you can directly open to create a TinyRender project in Visual Studio. To compile TinyRender, select your build profile at the top-left (_Release_ or _Debug_), make sure _Solution Platforms_ is set to x86 and click _Local Windows Debugger_ to launch the build.
(##) A note on Debug / Release modes
If you are not familiar with these two types of build profiles, _Release_ is compiled with a set of standard optimization options whereas _Debug_ includes debugging iterators and omits optimizations. You should always render your **final scenes** in _Release_ mode to accelerate the process. Developing in _Debug_ mode allows you to make use of the debugging tools and to leverage more verbose error messages. This will come in handy when tracking down bugs, but it comes at a significant performance reduction. In fact, _Debug_ mode automatically runs TinyRender on a single thread, while _Release_ mode defaults to multithreaded execution.
(#) High-level overview of the codebase
The TinyRender repository consists of codebase files and a few header-only dependency libraries that we briefly explain, below.
(##) Renderer structure
Directory / file | Description
------------|--------------
`externals/` | External dependency libraries (see below)
`src/` | Directory containing the main C++ source code
`data/` | Example scenes and test datasets to validate your implementations
`CMakeLists.txt` | CMake build file specifying how to compile and link TinyRender (only used on Linux / macOS)
(##) Main source code
Students only have to deal with the files in the `src` directory (and `data` for the scenes). The table below briefly outlines the content of each file and directory.
Directory / file | Description | Examples
------------|--------------|--
`main.cpp` | TinyRender's main program
`bsdfs/` | BSDFs directory | Diffuse, mirror, Phong
`integrators/` | Integrators directory | Ambient occlusion, path tracing
`shaders/` | Interactive shaders directory | Diffuse with point light
`renderpasses/` | Interactive render pass | Normal, direct
`core/` | Renderer core directory |
`core/accel.h` | Ray-scene intersection acceleration structure
`core/camera.h` | WASD camera for real-time rendering
`core/core.h` | C-like structures for all objects | Ray, camera, intersection
`core/integrator.cpp` | Integrator declaration file |
`core/integrator.h` | Integrator header file |
`core/math.h` | Rendering-specific mathematics | Sample warping functions
`core/platform.h` | Cross-platform type definitions and constants | $\pi$, $\epsilon$ for numerical errors
` renderer.cpp` | Renderer abstraction and structure implementations |
`core/renderer.h` | Renderer header file |
`core/renderpass.cpp` | Real-time rendering pipeline declaration file |
`core/renderpass.h` | Real-time rendering pipeline header file |
`core/utils.h` | Image saving methods
!!!Note:
Item marked in color are the ones you will modify throughout the semester; you won't have to modify any other files.
__You are not expected to understand every line of code in these files__ but you should be familiar with the overall structure of the renderer. All template files -- such as BRDFs, integrators and shaders -- are provided for every assignments at the start of the semester. This means that you will recycle the codebase throughout the course. This will allow you to spend more time focusing on the algorithms and less on actual software engineering.
(##) External libraries
Directory / file | Description
------------|--------------
`glm/` | Mathematics library for graphics
`bvh.h` | Bounding volume hierarchy builder
`cpptoml.h` | TOML file parser
`tinyobj.h` | Wavefront OBJ mesh loader
`tinyexr.h` | High dynamic range image format library
Let's have a brief overview of the most important dependencies.
(###) OpenGL Mathematics (GLM))
When developing any kind of graphics-related software, it's important to be familiar with core mathematics support libraries responsible for basic linear algebra types. TinyRender relies on [GLM](https://glm.g-truc.net/0.9.9/index.html) for this purpose, but we don't expect you to understand the inner workings of this library.
The main subset of types that you will most likely use are:
GLM type | TinyRender typedef | Object
-----|------|----
`glm::fvec2` | `v2f` | 2D float vector
`glm::fvec3` | `v3f` | 3D float vector
`glm::fvec4` | `v4f` | 4D float vector
`glm::mat4` | `mat4f` | 4×4 float matrix
where the number in the type indicates the dimension and the `f` stands for `float`. Whenever you need to perform an operation on a vector or matrix, you will have to call `glm::` with the corresponding operator. Below are some operations that you may eventually need:
Operation | Function
----------|---------
Dot product between vectors __u__ and __v__ | `glm::dot(u,v)`
Cross product between vectors __u__ and __v__ | `glm::cross(u,v)`
Component-wise absolute value of vector __v__ | `glm::abs(v)`
Normalization of vector __v__ | `glm::normalize(v)`
Euclidean norm of vector __v__ | `glm::l2Norm(v)` _or_ `glm::length2(v)`
Distance between points __p__ and __q__ | `glm::distance(p,q)`
Reflection of vector __v__ on surface with normal __n__ | `glm::reflect(v,n)`
For more information, you can consult [GLM's official documentation](https://glm.g-truc.net/0.9.9/api/index.html) or set up autocomplete in your IDE.
Other mathematical operators can be called via the [Standard C++ Library](https://en.cppreference.com/w/cpp/header) (`std`). For instance, absolute value $|x|$ can be obtained by calling `std::abs(x)` and exponentiation $x^n$ can be done with `std::pow(x,n)`.
(###) OpenEXR
[OpenEXR](http://www.openexr.com) is a standardized file format for storing high dynamic range (HDR) images. Unlike low dynamic range (LDR) image formats -- such as JPG and PNG -- HDR image pixel color values are not restricted to the interval [0,1]. This allows us to represent luminance ranges similar to the range we experience through the human visual system in the real-world. OpenEXR was originally developed by [Industrial Light and Magic](https://www.ilm.com/) and is now widely used in the movie and gaming industries. The directory `externals/tinyexr` contains a header-only implementation of this standard.
A word of caution: various tools for visualizing OpenEXR images exist, but not all really do what one would expect. The software EXR viewers listed below work correctly, whereas Preview on macOS (for instance) tonemaps EXR image output in an unexpected way:
+ [tev](https://github.com/Tom94/tev/releases) by Thomas Mueller
+ [HDRView](https://bitbucket.org/wkjarosz/HDRView) by Wojciech Jarosz
+ [HDRITools](https://bitbucket.org/edgarv/hdritools/overview) by Edgar Velázquez-Armendáriz
+ [Adobe Photoshop](https://www.adobe.com/products/photoshop.html) with the [OpenEXR plugin](http://www.openexr.com/photoshop_plugin.html)
(###) Simple DirectMedia Layer (SDL2))
[SDL2](https://www.libsdl.org/index.php) is a cross-platform development library designed to provide low-level access to audio, keyboard, mouse, joystick, and graphics hardware via OpenGL and Direct3D. In TinyRender, SDL2 is used for the real-time rendering parts of the assignments. Most of the OpenGL code will remain abstracted from you, however you will have to implement interactive rendering loops and vertex and fragment shaders in the assignments.
(##) TinyRender pipeline
At a high level, your renderer expects a scene description file as input, from which it parses all the information necessary to render the scene. This includes the position of the mesh object triangles, camera parameters, the type of rendering techniques to apply, and so on. Once parsed, this data is passed to the renderer which then executes either an offline or online/interactive _rendering loop_ -- which in turn will rely on offline integrator algorithms or interactive shaders -- in order to compute output images.
![Figure [pipeline]: TinyRender rendering pipeline.](imgs/diagram.png)
(#) Scene file format and parsing
Each assignment will ask that you render multiple scenes in order to demonstrate a working implementation of the required tasks. Scene data comprises mesh data and scene description files. We will provide this data, in addition to a configuration file used during assignment submission, in the following tree structure:
~~~
data/
scene_1/
mesh/
mesh_1.obj // Meshes
mesh_1.mtl // Materials (required)
tinyrender/
task_1_offline.toml // Offline scene
task_2_realtime.toml // Real-time scene
scene_2/
...
config.json // Configuration file for submission
~~~
You are free to inspect the meshes in a 3D modeling software suite, such as [Blender](https://www.blender.org) or [Autodesk Maya](https://www.autodesk.ca/en/products/maya/overview). Both of these software packages are available for free, but you will need to use your McGill email address if you wish to use Maya.
!!! Note:
The `.mtl` file is a [material library file](http://paulbourke.net/dataformats/mtl). This file format was originally designed for real-time graphics but `tinyobj.h` parses this information and assigns a material to each mesh in the scene. While dated, this format is convenient for many reasons -- most notably, it allows TinyRender to use a single material file for both offline and interactive subsystems. Do _not_ alter any of these mesh-related files unless you know what you're doing.
(###) Example scene file
The TOML scene description file contains all the metadata needed to render a scene. Below is an example of an TOML scene file for an offline rendering task:
~~~
# Offline rendering scene
[input]
objfile = "../mesh/mesh.obj"
[camera]
eye = [0.0, 1.0, -4.0]
at = [0.0, 0.0, 0.0]
up = [0.0, 1.0, 0.0]
fov = 30.0
[film]
width = 1920
height = 1080
[renderer]
realtime = false
type = "ao"
spp = 16
# Parameter = value (optional, depends on your integrator)
~~~
This file is the main input you pass to your renderer. The codebase can parse this information for you and store it in the `scene.config` object. In this example, TinyRender's parser would start by reading the OBJ meshes and their corresponding material files -- which need to share the same name. After which, TinyRender creates a pinhole camera at location `eye` = (0, 1, -4) looking at position `at` (both specified in _world_ coordinates). The `up`-vector of the camera is the _y_-axis, in this example. The field of view (FoV) is set to 30 degrees and the image plane is 1920×1080 pixels in resolution (full HD). Finally, the integrator used to compute pixel intensities is ambient occlusion (`ao`) and TinyRender is being told to trace 16 ray samples through each pixel (spp $\equiv$ sample per pixel).
An example of a TOML scene file for an interactive/online/real-time rendering task would have a similar structure, except that the renderer uses shaders and a different render loop logic. The only parameter you can provide to real-time renderers is the shader type. In the example below, the renderer will use screen space ambient occlusion (`ssao`) to render an interactive scene.
~~~
# Real-time rendering scene
# ...
[renderer]
realtime = true
type = "ssao"
# Parameter = value (optional, depends on your shaders)
~~~
Technically, you won't have to change anything in the TOML scene files. Both offline and real-time scenes that we require you to render will be provided. If you wish to play around by moving the camera or increasing the number of samples, you can substitute custom values in the corresponding fields.
!!! Note:
**Useful tip:** duplicate your scene files to quickly test your code. A lower film resolution and a smaller sampling rate will reduce the rendering time an allow you to more quickly test your code.
(##) Rendering a scene
(###) CLion
__To render a scene,__ you need to provide a TOML scene file as a program argument to TinyRender. In CLion, this is done by clicking on _Run_ -> _Edit Configurations..._ and passing the scene file path in _Program arguments_.
![Figure [clion-arg]: Passing a TOML scene file as an argument to TinyRender in CLion.](imgs/build_clion_argument.png)
(###) Visual Studio
In Visual Studio, go to _Project_ -> _Properties_ (Alt + Enter). Navigate to _Configuration_ -> _Debugging_ and enter the scene path as _Command Arguments_. The variable `$(SolutionDir)` is the path where your solution file is. You can ensure that the scene argument is shared across build modes by selecting _All Configurations_ in the top-left _Configuration_ dropdown.
![Figure [vs-arg]: Passing a TOML scene file as an argument to TinyRender in Visual Studio.](imgs/vs_argument.PNG)
(##) A note on scene file paths
The scene file parser will fail to load a scene if the associated path contains _spaces_ or _special characters_. That is, if you folder for the course is called `ECSE 446`, the parser will stop at `ECSE` as a space indicates an end-of-line. In general, you should avoid naming directories on your computer with spaces. There are two workarounds for this:
1. rename your folders so that the full path does not contain spaces (better option), or
2. use quotation marks for all scene files you pass as arguments.
The former is better practice. The latter with correctly evaluate the path before attempting to read it. Above all, _make sure the path to the TOML scene file is **valid**, otherwise the rendering will fail!_
(#) Grading
(##) Test and final scenes
Assignments will require you to demonstrate a complete implementation of the required tasks. Each assignment will include two sets of scenes: _test scenes_ include reference images to help you validate your implementation -- you do not have to submit these images -- and _final scenes_ are scenes for which you do _not_ know the reference output. You will be required to submit images for the _final scenes_ for grading. Note that getting the test scenes right does not necessarily mean you'll get a perfect score on the final scenes!
__All submitted assignment source code will be recompiled, and result regenerated, for grading.__ It is your responsibility to convince us that you have implemented the assignments correctly. __Code that does not compile may result in a score of zero.__ If your assignment does not compile at the moment of submitting but you do have some correct images from previous iterations, please submit these. In this case, also include a note -- either on myCourses or in a `readme.txt` file included in your submission -- to indicate (as precisely as possible) where you think the problem may lie.
(##) Configuration file
For your assignment to be graded, we require that you submit a valid `config.json` configuration file with their name and student ID. This file is parsed by a script that allows graders to directly compare your offline output with a reference solution. Below is an example of a JSON config file that would need to be submitted with an assignment (in this case, A1). Your configuration file should be similiar to the following; __only lines 2–4 should be modified.__
~~~ JSON linenumbers
{
"firstlast": "John Doe",
"id": 123456789,
"course": 446,
"assignment": 1,
"renders": [
{ "scene": "Sphere offline",
"render": "offline/sphere_normal_offline.exr"
},
{ "scene": "Dragon offline",
"render": "offline/dragon_normal_offline.exr"
},
{ "scene": "Sphere real-time",
"render": "realtime/sphere_normal_offline.exr"
},
{ "scene": "Dragon real-time",
"render": "realtime/dragon_normal_offline.exr"
},
]
}
~~~
(#) FAQ
(##) Admin
Q: _Do TAs have office hours?_
A: __No. If you want help from a TA, attend your tutorial. If you can't, come during the last 15 minutes of a different tutorial session.__
Q: _Can you tell me if my final images look correct?_
A: __No, but chances are if you think they look wrong, they are.__
Q: _Can I use the discussion board to ask students to validate my final images?_
A: __No, and this applies to code snippets, too. Any post containing final image renders or code will be removed.__
Q: _I started my rendering before the deadline but it's taking so long that it will not finish before midnight. Can I submit what I have now and submit my final image once it's done?_
A: __No, anything that gets submitted past the deadline will not be graded. The submission deadline is final.__
Q: _I'm very close to the full solution but I can't quite get it right and I need to submit. How much will I be penalized?_
A: __Submit what you have and make sure your code compiles. Attach a `readme.txt` file to your submission or use the myCourses submission textbox to explain what you think is wrong. The grader will take this into consideration.__
(##) Technical issues
Q: _My code produces different images on different computers. Which image do I submit?_
A: __Submit the one you think is the most correct. Graders will regenerate your images, regardless.__
Q: _I tried loading a scene but it failed... what gives?_
A: __Make sure the path to the TOML is valid. When in doubt, use an absolute path.__
Q: _I'm getting different program exit codes in different situations. What do they mean?_
A: __Exit codes are system specific but you should be getting 0 with the starter code. Use the debug tool to see where your program crashes. It is possible, albeit unlikely, that there remain some unresolve bugs in TinyRender.__
Q: _Well, I found a bug in the code... what do I do?_
A: __If you're comfortable with Git, you can submit a pull request online with the fix. If not, you can email the TAs or post on the discussion board directly.__
Q: _I updated my OS and now TinyRender does not compile anymore. What do I do?_
A: __We will not support OS-specific issues in this scenario. In general, these errors occur due to a compatibility issue with one of the dependencies.__
Q: _TAs pushed a fix which caused a conflict with my code. How do I proceed?_
A: __Inform the TA immediately -- this is not supposed to happen.__
_TinyRender_ is a minimalistic rendering engine written in C++11 that runs on Linux, macOS and Windows. The code we provide lays the foundations for the homework assignments. This codebase supports both real-time (online/interactive) and physically-based (offline) rendering paradigms.
While TinyRender provides significant scaffolding to simplify the development of full-fledged renderers, the code you will start with doesn't yet generate any impressive output: it loads a scene and either runs an placeholder OpenGL shader (in _online mode_) or saves a debug image as an OpenEXR file (in _offline mode_) --- all meaningful shading code is missing, and so these outputs consist of only black pixels. During the semester, you will be tasked with extending this system into a relatively complete renderer. The programming assignments will guide you through this incremental process.
(#) Core features
TinyRender provides many features that would be tedious to implement from scratch, including:
+ A [TOML](https://github.com/toml-lang/toml)-based scene file parser and loader
+ A [loader](https://github.com/syoyo/tinyobjloader) for Wavefront OBJ 3D geometry files
+ Support for saving HDR image outputs in the [OpenEXR](http://www.openexr.com) format
+ Basic linear algebra code for point/vector/normal/ray/bounding boxes
+ A pseudorandom number generator
+ An OpenGL library for real-time rendering ([SDL2](https://www.libsdl.org))
+ An optimized ray tracer with [bounding volume hierarchy builder](https://github.com/brandonpelfrey/Fast-BVH) and fast ray-triangle intersection routines
(##) Permissible reference sources
In addition to the optional PBRT course textbook, you should feel free to consult additional references when completing the homework assignments so long as you remember to cite them in an attached `readme.txt` file.
When asked to implement some feature, you should not copy or build atop existing implementations from other renderers. Doing so both hampers your learning (which will be tested on the final exam) and is unethical. We will obviously be on the lookout for this. That being said, referring to PBRT -- which includes a lot of implementation insights -- is perfectly reasonable. When in doubt about this rule or the propoer use of any references, ask the TAs.
(#) Instructions
Each of the five (5) assignments has a __strict deadline__ and submission instructions, both outlined in its handout. All the assignments share a common codebase, which we will release at the beginning of the semester.
We strongly urge students to start working on the assignments as early as possible. Building your own advanced renderer is often a very rewarding experience!
(#) Source code
Assignments are built incrementally and you will sometimes have to reuse parts of your code across assignments. A unified codebase is provided at the beginning of the semester, containing most of the pieces required to complete the tasks for all assignments. Some parts of the code may change throughout the semester -- e.g. due to unforeseen bugs, driver and OS compatibility issues, etc. -- which is why we are making the the code accessible through a Git repository. This will allow the TAs, if necessary, to seamlessly push modifications with minimal impact.
(##) Git prerequisites
If you are completely unfamiliar with version control using Git, we suggest reading the [GitLab documentation](https://docs.gitlab.com/ee/gitlab-basics/start-using-git.html) for a good overview. Only a small subset of Git commands will be used during the course, so beginner-level knowledge is sufficient. Version control commands can be executed either in the terminal or with an appropriate GUI. If you decide to use a GUI, we recommend [SmartGit](https://www.syntevo.com/smartgit/). Using the terminal is straightforward, and we will only provide instructions for this mode. On Windows, you will need to download [GitBash](https://gitforwindows.org/). For Linux / macOS, open up a terminal and Git should be installed by default. If not, you can obtain it [here](https://git-scm.com/downloads).
(##) Cloning the source code
Navigate to the directory you wish to clone our codebase repository into and run:
~~~ bash
git clone https://github.com/ecsex46/fall-2019.git
~~~
That's it, you're set for the semester! Note that this can take a few minutes since large scene files also need to be downloaded alongside the base code.
(##) Syncing after a push
In the event of a codebase patch by the TAs, we will make an announcement on __myCourses__ and ask that students sync the updated codebase version with their local codebase. These patches should _not_ affect any code students' have added to the base codebase, which means that _no conflicts_ should occur during the path (unless you've modified parts of the code you weren't instructed to). Pulling from the repository to obtain the updated codebase should therefore only require executing the following commands:
~~~ bash
git stash save
git pull
git stash pop
~~~
These commands should be called within the local directory where the repository was initially cloned. In short, these Git commands save your changes, synchronize your local tree with the (patched) master version, and then re-applies your changes. If done successfully, you should see which files were altered during the pull. If it's your first time using Git, we recommend duplicating your project directory as backup for extra safety; you can always revert back to it if something goes wrong.
(##) A note on public forks and plagiarism
While it is tempting to upload your personal rendering engine to the public domain, we ask that students _neither_ fork TinyRender nor upload any of its files online. In any cases of plagiarism, both the student who uploaded their files publicly __and__ students that plagiarize from these files will be at fault and will face disciplinary charges. Graders will automatically check your code against all other students' code -- both past and present -- as well as third-party external rendering codebases, in order to detect and flag any similiarities in code submitted for the assignments.
(#) Building the codebase
TinyRender has very few dependencies that require manual installation. It relies on [GLEW](http://glew.sourceforge.net/), [SDL2](https://www.libsdl.org/index.php) and [Boost](https://www.boost.org) (Mac-only). Additionnally, the [CMake](https://cmake.org) build system is required to compile and link TinyRender. We __strongly suggest__ using an IDE, such as [CLion](https://www.jetbrains.com/clion) (Linux / macOS) or [Visual Studio 2019](https://visualstudio.microsoft.com) (Windows 10), to complete the assignments. We provide building instructions for these two IDEs -- you are free to use any other IDE, however without our support. We do _not_ recommend using CLion on Windows as the real-time component of the assignments will not work out-of-the-box.
If you've never used these IDEs before, make sure to look up tutorials online on how to get setup. Most importantly, learning to use the IDE debugging tools can significantly improve your ability to debug your code. If you don't know how to use them, now's a good time to learn -- the TAs will provide some basic support here.
(##) Linux / macOS
(###) Installing dependencies on Linux
Installing the binary dependencies on Linux is straightforward, assuming you have a working [`gcc` compiler](https://gcc.gnu.org) which comes with most modern Linux distributions, such as Ubuntu and Arch Linux:
~~~ Bash
sudo apt-get update
sudo apt-get install libglew-dev libsdl2-dev cmake
~~~
(###) Installing dependencies on macOS
Begin by installing a reasonably up-to-date version of [XCode](https://developer.apple.com/xcode) (>= 8.0) along with the command-line tools. To check if the full XCode package is (already) installed, open a terminal and execute:
~~~ Bash
xcode-select -p
~~~
If you see:
~~~ Bash
/Applications/Xcode.app/Contents/Developer
~~~
you have everything set up. Next, install [Homebrew](https://brew.sh) by executing:
~~~ Bash
/usr/bin/ruby -e
"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
~~~
Then install the required packages, with:
~~~ Bash
brew install boost glew sdl2 cmake
~~~
You can verify that all packages were installed correctly by typing `brew list`.
!!! Note:
Switching macOS release mid-way through the semester __can cause compilation errors__ due to delays in porting dependencies to new OS versions. As such, we do _not_ recommend that you update your OS to [Catalina](https://www.apple.com/ca/macos/catalina-preview/) until the semester is over. If you do, TAs will _not_ assist you in fixing OS-related compatibility issues.
(###) Setting up TinyRender in CLion
Open CLion, click _Open..._ and select the `CMakeLists.txt` file at the root of the TinyRender source directory for the assignment. Select _Open as Project_ when prompted and let CLion load the project. CMake will ensure that you have all the dependencies installed to build TinyRender. If no errors appear, you should see `[Finished]` at the bottom of the CMake console, preceeded by a list of successful checks.
To build TinyRender, simply execute _Run_ -> _Build_. This will compile and link your program in _Debug_ mode. To create a _Release_ build, navigate to _File_ -> _Settings_ (_Preferences_ on macOS). In the tab _Build, Execution, Deployment_ -> _CMake_, click the + icon. You should now see _Debug_ and _Release_ under _Profiles_ (see figure below).
![Figure [clion-cmake]: Adding the _Release_ mode to a CMake build in CLion.](imgs/build_clion_cmake.png)
You can select your build profile in the top-right corner dropdown in CLion and click the button to launch the build.
(##) Windows
(###) Installing / modifying Visual Studio
To compile and run TinyRender on Windows 10, first install a copy of [Visual Studio 2019](https://visualstudio.microsoft.com) (Community edition will suffice). During the installation setup, you will be asked which workloads you wish to install. Select _Universal Windows Platform development_ and _Desktop development in C++_. This should automatically add the latest Windows 10 SDK (10.0.18362.0) to the installation, which you can confirm on the right-hand _Summary_ panel. Once complete, click _Install_. Note that Visual Studio is a pretty heavy software package and will require 10GB+ of disk space.
If you already have VS2019 installed, launch _Visual Studio Installer_ from Start menu and then click _Modify_. Make sure that you have the correct workloads installed along with the latest Windows 10 SDK. Click _Modify_ in the bottom-right corner and let the installation finish.
![Figure [vs-install]: Installing / modifying Visual Studio.](imgs/install_vs.PNG)
(###) Setting up TinyRender in Visual Studio
We provide a solution (`.sln`) file which you can directly open to create a TinyRender project in Visual Studio. To compile TinyRender, select your build profile at the top-left (_Release_ or _Debug_), make sure _Solution Platforms_ is set to x86 and click _Local Windows Debugger_ to launch the build.
(##) A note on Debug / Release modes
If you are not familiar with these two types of build profiles, _Release_ is compiled with a set of standard optimization options whereas _Debug_ includes debugging iterators and omits optimizations. You should always render your **final scenes** in _Release_ mode to accelerate the process. Developing in _Debug_ mode allows you to make use of the debugging tools and to leverage more verbose error messages. This will come in handy when tracking down bugs, but it comes at a significant performance reduction. In fact, _Debug_ mode automatically runs TinyRender on a single thread, while _Release_ mode defaults to multithreaded execution.
(#) High-level overview of the codebase
The TinyRender repository consists of codebase files and a few header-only dependency libraries that we briefly explain, below.
(##) Renderer structure
Directory / file | Description
------------|--------------
`externals/` | External dependency libraries (see below)
`src/` | Directory containing the main C++ source code
`data/` | Example scenes and test datasets to validate your implementations
`CMakeLists.txt` | CMake build file specifying how to compile and link TinyRender (only used on Linux / macOS)
(##) Main source code
Students only have to deal with the files in the `src` directory (and `data` for the scenes). The table below briefly outlines the content of each file and directory.
Directory / file | Description | Examples
------------|--------------|--
`main.cpp` | TinyRender's main program
`bsdfs/` | BSDFs directory | Diffuse, mirror, Phong
`integrators/` | Integrators directory | Ambient occlusion, path tracing
`shaders/` | Interactive shaders directory | Diffuse with point light
`renderpasses/` | Interactive render pass | Normal, direct
`core/` | Renderer core directory |
`core/accel.h` | Ray-scene intersection acceleration structure
`core/camera.h` | WASD camera for real-time rendering
`core/core.h` | C-like structures for all objects | Ray, camera, intersection
`core/integrator.cpp` | Integrator declaration file |
`core/integrator.h` | Integrator header file |
`core/math.h` | Rendering-specific mathematics | Sample warping functions
`core/platform.h` | Cross-platform type definitions and constants | $\pi$, $\epsilon$ for numerical errors
` renderer.cpp` | Renderer abstraction and structure implementations |
`core/renderer.h` | Renderer header file |
`core/renderpass.cpp` | Real-time rendering pipeline declaration file |
`core/renderpass.h` | Real-time rendering pipeline header file |
`core/utils.h` | Image saving methods
!!!Note:
Item marked in color are the ones you will modify throughout the semester; you won't have to modify any other files.
__You are not expected to understand every line of code in these files__ but you should be familiar with the overall structure of the renderer. All template files -- such as BRDFs, integrators and shaders -- are provided for every assignments at the start of the semester. This means that you will recycle the codebase throughout the course. This will allow you to spend more time focusing on the algorithms and less on actual software engineering.
(##) External libraries
Directory / file | Description
------------|--------------
`glm/` | Mathematics library for graphics
`bvh.h` | Bounding volume hierarchy builder
`cpptoml.h` | TOML file parser
`tinyobj.h` | Wavefront OBJ mesh loader
`tinyexr.h` | High dynamic range image format library
Let's have a brief overview of the most important dependencies.
(###) OpenGL Mathematics (GLM))
When developing any kind of graphics-related software, it's important to be familiar with core mathematics support libraries responsible for basic linear algebra types. TinyRender relies on [GLM](https://glm.g-truc.net/0.9.9/index.html) for this purpose, but we don't expect you to understand the inner workings of this library.
The main subset of types that you will most likely use are:
GLM type | TinyRender typedef | Object
-----|------|----
`glm::fvec2` | `v2f` | 2D float vector
`glm::fvec3` | `v3f` | 3D float vector
`glm::fvec4` | `v4f` | 4D float vector
`glm::mat4` | `mat4f` | 4×4 float matrix
where the number in the type indicates the dimension and the `f` stands for `float`. Whenever you need to perform an operation on a vector or matrix, you will have to call `glm::` with the corresponding operator. Below are some operations that you may eventually need:
Operation | Function
----------|---------
Dot product between vectors __u__ and __v__ | `glm::dot(u,v)`
Cross product between vectors __u__ and __v__ | `glm::cross(u,v)`
Component-wise absolute value of vector __v__ | `glm::abs(v)`
Normalization of vector __v__ | `glm::normalize(v)`
Euclidean norm of vector __v__ | `glm::l2Norm(v)` _or_ `glm::length2(v)`
Distance between points __p__ and __q__ | `glm::distance(p,q)`
Reflection of vector __v__ on surface with normal __n__ | `glm::reflect(v,n)`
For more information, you can consult [GLM's official documentation](https://glm.g-truc.net/0.9.9/api/index.html) or set up autocomplete in your IDE.
Other mathematical operators can be called via the [Standard C++ Library](https://en.cppreference.com/w/cpp/header) (`std`). For instance, absolute value $|x|$ can be obtained by calling `std::abs(x)` and exponentiation $x^n$ can be done with `std::pow(x,n)`.
(###) OpenEXR
[OpenEXR](http://www.openexr.com) is a standardized file format for storing high dynamic range (HDR) images. Unlike low dynamic range (LDR) image formats -- such as JPG and PNG -- HDR image pixel color values are not restricted to the interval [0,1]. This allows us to represent luminance ranges similar to the range we experience through the human visual system in the real-world. OpenEXR was originally developed by [Industrial Light and Magic](https://www.ilm.com/) and is now widely used in the movie and gaming industries. The directory `externals/tinyexr` contains a header-only implementation of this standard.
A word of caution: various tools for visualizing OpenEXR images exist, but not all really do what one would expect. The software EXR viewers listed below work correctly, whereas Preview on macOS (for instance) tonemaps EXR image output in an unexpected way:
+ [tev](https://github.com/Tom94/tev/releases) by Thomas Mueller
+ [HDRView](https://bitbucket.org/wkjarosz/HDRView) by Wojciech Jarosz
+ [HDRITools](https://bitbucket.org/edgarv/hdritools/overview) by Edgar Velázquez-Armendáriz
+ [Adobe Photoshop](https://www.adobe.com/products/photoshop.html) with the [OpenEXR plugin](http://www.openexr.com/photoshop_plugin.html)
(###) Simple DirectMedia Layer (SDL2))
[SDL2](https://www.libsdl.org/index.php) is a cross-platform development library designed to provide low-level access to audio, keyboard, mouse, joystick, and graphics hardware via OpenGL and Direct3D. In TinyRender, SDL2 is used for the real-time rendering parts of the assignments. Most of the OpenGL code will remain abstracted from you, however you will have to implement interactive rendering loops and vertex and fragment shaders in the assignments.
(##) TinyRender pipeline
At a high level, your renderer expects a scene description file as input, from which it parses all the information necessary to render the scene. This includes the position of the mesh object triangles, camera parameters, the type of rendering techniques to apply, and so on. Once parsed, this data is passed to the renderer which then executes either an offline or online/interactive _rendering loop_ -- which in turn will rely on offline integrator algorithms or interactive shaders -- in order to compute output images.
![Figure [pipeline]: TinyRender rendering pipeline.](imgs/diagram.png)
(#) Scene file format and parsing
Each assignment will ask that you render multiple scenes in order to demonstrate a working implementation of the required tasks. Scene data comprises mesh data and scene description files. We will provide this data, in addition to a configuration file used during assignment submission, in the following tree structure:
~~~
data/
scene_1/
mesh/
mesh_1.obj // Meshes
mesh_1.mtl // Materials (required)
tinyrender/
task_1_offline.toml // Offline scene
task_2_realtime.toml // Real-time scene
scene_2/
...
config.json // Configuration file for submission
~~~
You are free to inspect the meshes in a 3D modeling software suite, such as [Blender](https://www.blender.org) or [Autodesk Maya](https://www.autodesk.ca/en/products/maya/overview). Both of these software packages are available for free, but you will need to use your McGill email address if you wish to use Maya.
!!! Note:
The `.mtl` file is a [material library file](http://paulbourke.net/dataformats/mtl). This file format was originally designed for real-time graphics but `tinyobj.h` parses this information and assigns a material to each mesh in the scene. While dated, this format is convenient for many reasons -- most notably, it allows TinyRender to use a single material file for both offline and interactive subsystems. Do _not_ alter any of these mesh-related files unless you know what you're doing.
(###) Example scene file
The TOML scene description file contains all the metadata needed to render a scene. Below is an example of an TOML scene file for an offline rendering task:
~~~
# Offline rendering scene
[input]
objfile = "../mesh/mesh.obj"
[camera]
eye = [0.0, 1.0, -4.0]
at = [0.0, 0.0, 0.0]
up = [0.0, 1.0, 0.0]
fov = 30.0
[film]
width = 1920
height = 1080
[renderer]
realtime = false
type = "ao"
spp = 16
# Parameter = value (optional, depends on your integrator)
~~~
This file is the main input you pass to your renderer. The codebase can parse this information for you and store it in the `scene.config` object. In this example, TinyRender's parser would start by reading the OBJ meshes and their corresponding material files -- which need to share the same name. After which, TinyRender creates a pinhole camera at location `eye` = (0, 1, -4) looking at position `at` (both specified in _world_ coordinates). The `up`-vector of the camera is the _y_-axis, in this example. The field of view (FoV) is set to 30 degrees and the image plane is 1920×1080 pixels in resolution (full HD). Finally, the integrator used to compute pixel intensities is ambient occlusion (`ao`) and TinyRender is being told to trace 16 ray samples through each pixel (spp $\equiv$ sample per pixel).
An example of a TOML scene file for an interactive/online/real-time rendering task would have a similar structure, except that the renderer uses shaders and a different render loop logic. The only parameter you can provide to real-time renderers is the shader type. In the example below, the renderer will use screen space ambient occlusion (`ssao`) to render an interactive scene.
~~~
# Real-time rendering scene
# ...
[renderer]
realtime = true
type = "ssao"
# Parameter = value (optional, depends on your shaders)
~~~
Technically, you won't have to change anything in the TOML scene files. Both offline and real-time scenes that we require you to render will be provided. If you wish to play around by moving the camera or increasing the number of samples, you can substitute custom values in the corresponding fields.
!!! Note:
**Useful tip:** duplicate your scene files to quickly test your code. A lower film resolution and a smaller sampling rate will reduce the rendering time an allow you to more quickly test your code.
(##) Rendering a scene
(###) CLion
__To render a scene,__ you need to provide a TOML scene file as a program argument to TinyRender. In CLion, this is done by clicking on _Run_ -> _Edit Configurations..._ and passing the scene file path in _Program arguments_.
![Figure [clion-arg]: Passing a TOML scene file as an argument to TinyRender in CLion.](imgs/build_clion_argument.png)
(###) Visual Studio
In Visual Studio, go to _Project_ -> _Properties_ (Alt + Enter). Navigate to _Configuration_ -> _Debugging_ and enter the scene path as _Command Arguments_. The variable `$(SolutionDir)` is the path where your solution file is. You can ensure that the scene argument is shared across build modes by selecting _All Configurations_ in the top-left _Configuration_ dropdown.
![Figure [vs-arg]: Passing a TOML scene file as an argument to TinyRender in Visual Studio.](imgs/vs_argument.PNG)
(##) A note on scene file paths
The scene file parser will fail to load a scene if the associated path contains _spaces_ or _special characters_. That is, if you folder for the course is called `ECSE 446`, the parser will stop at `ECSE` as a space indicates an end-of-line. In general, you should avoid naming directories on your computer with spaces. There are two workarounds for this:
1. rename your folders so that the full path does not contain spaces (better option), or
2. use quotation marks for all scene files you pass as arguments.
The former is better practice. The latter with correctly evaluate the path before attempting to read it. Above all, _make sure the path to the TOML scene file is **valid**, otherwise the rendering will fail!_
(#) Grading
(##) Test and final scenes
Assignments will require you to demonstrate a complete implementation of the required tasks. Each assignment will include two sets of scenes: _test scenes_ include reference images to help you validate your implementation -- you do not have to submit these images -- and _final scenes_ are scenes for which you do _not_ know the reference output. You will be required to submit images for the _final scenes_ for grading. Note that getting the test scenes right does not necessarily mean you'll get a perfect score on the final scenes!
__All submitted assignment source code will be recompiled, and result regenerated, for grading.__ It is your responsibility to convince us that you have implemented the assignments correctly. __Code that does not compile may result in a score of zero.__ If your assignment does not compile at the moment of submitting but you do have some correct images from previous iterations, please submit these. In this case, also include a note -- either on myCourses or in a `readme.txt` file included in your submission -- to indicate (as precisely as possible) where you think the problem may lie.
(##) Configuration file
For your assignment to be graded, we require that you submit a valid `config.json` configuration file with their name and student ID. This file is parsed by a script that allows graders to directly compare your offline output with a reference solution. Below is an example of a JSON config file that would need to be submitted with an assignment (in this case, A1). Your configuration file should be similiar to the following; __only lines 2–4 should be modified.__
~~~ JSON linenumbers
{
"firstlast": "John Doe",
"id": 123456789,
"course": 446,
"assignment": 1,
"renders": [
{ "scene": "Sphere offline",
"render": "offline/sphere_normal_offline.exr"
},
{ "scene": "Dragon offline",
"render": "offline/dragon_normal_offline.exr"
},
{ "scene": "Sphere real-time",
"render": "realtime/sphere_normal_offline.exr"
},
{ "scene": "Dragon real-time",
"render": "realtime/dragon_normal_offline.exr"
},
]
}
~~~
(#) FAQ
(##) Admin
Q: _Do TAs have office hours?_