Assignment Goals

spherical light probe

The primary goals of this assignment are:

  1. Create a UE Plugin from scratch.
  2. Create a UE object importer.
  3. Get experience looking through other plugin code and engine source as a strategy for working with a large codebase.

For this assignment, the goal is to ultimately convert a high dynamic range light probe from a file in PFM format into a form we can use in the engine.

A light probe captures light from all directions, and can be used to make realistic lighting and reflection. You will find nine samples from Paul Debevec's light probe page in the hdrenv directory of your git repository. Light probes can be created using a capture device with special lenses, photos of a reflective sphere, or constructed from multiple photos. They are especially useful when trying to combine CGI elements with live action in film production, where it is common to include a reflective sphere somewhere in the shot that can be easily removed in postproduction.

Textures are often classified into low dynamic range (LDR) images or high dynamic range (HDR). In LDR, intensity in the range 0-1 is represented by a fixed-point data type like a byte. Anything too bright is clamped to 1. In HDR, intensity is represented in a floating point format, so the sky might be in the 100's and the sun on the order of 10,000, all of which would be clamped to 1 in an LDR texture. It is important that light probes be stored as floating-point high dynamic range images, since many lighting computations can scale the incoming light by relatively small factors, but the brightest spots should still show through.

The sphere map is a popular representation for light probes (as in the example). This can be captured by taking pictures of a reflective sphere (these samples were made from photos of a 3-inch ball bearing). To use you just need to figure out which point on the sphere would result in the desired reflection direction. For a reflection direction, R, and a sphere image taken looking down the X axis, the computation to figure out the texture coordinates for the reflection of R is:

0.5 - (0.5/PI) * acos(R.x) * normalize(R.yz)

Details

Create a project

  1. Create a Blank C++ project called assn5.
    • As usual, put the project at the top level of your git repository
  2. Create an "assn5" level and set it as your editor startup map.

Create a test material and object

  1. Create a new material.
    1. Set the Blend Mode to Translucent
    2. Hook a "Fresnel" node up to the Opacity
    3. Hook a "Texture Sample" node to the Emmissive Color, with the UVs input computed using the expression above from the reflection direction given by the "Reflection Vector" node.
    4. For testing, you can import one of the *_LDR.png sphere maps to use as your texture.
  2. Drag a copy of the SM_MatPreviewMesh_01 static mesh from the Engine content folder into your scene, and set the first of its three materials (Element 0) to your test material

Create your asset loader

  1. This time you will start from a Blank code plugin.
    • Edit the uplugin file to change it from a "Runtime" to "Editor" plugin
  2. Add an asset load class derived from UFactory to your plugin.
    • Add this in new header and C++ files, not in the same files as the ModuleInterface
    • After you create these files, choose "Tools" > "Refresh Visual Studio Projects" or "Tools" > "Refresh Xcode Project" in the Unreal Editor to update the Visual Studio or Xcode projects to know about the new files.
  3. Make your factory
    • To load from a file, you should override the UFactory::FactoryCreateFile or UFactory::FactoryCreateBinary function. Find examples in the engine and engine plugins.
    • To avoid undefined symbol errors, add the module for UFactory to your plugin's Build.cs.
    • Call Formats.Add in your class constructor to tell UE that you handle files with the pfm file extension. Note that the code that implements Formats.Add cannot handle extra spaces before the extension, or before or after the ";" that separates the extension from description.
    • At this point, if your importer creates and returns an empty UTexture2D object, you should be able to drag a pfm file into the Content window and get an empty texture object to confirm that your format registration code is correct.
    • Delete the old imported asset before trying to import again. Assuming you have not actually used it for anything (added it to any materials or as a property on another object), it is safe to "Force Delete" when removing the object.
  4. Parse the pfm header
    • Though our files are all color (PF), you should support both PF and Pf forms.
    • Our files do have at least one comment line in the header. Skip anything starting with a # to the end of the line.
    • Pay attention to the scale and endianness term in the header. The provided files are not the same endianness as either Intel or Apple CPUs, so you will need deal with swapping the bytes. You should do it only if necessary (so you should not the swap byte order if given a different file with scale -1).
    • The PFM format page is not very precise about what to do with the scale when it is not 1. Divide each texel by the absolute value of the scale as a normalization.
    • If things go wrong, use UE_LOG to send a message to the Editor Output window and return nullptr.
      • Make a new log category for your log messages instead of just using LogTemp
  5. Load the data into a UTexture2D
    • Create a UTexture2D with NewObject, using the InParent, Name, and Flags that were passed into your factory function.
    • Two texture source formats take 32-bit floats: TSF_R32F and TSF_RGBA32F. I recommend creating a temporary buffer to reorder the byte order of the data and, at least for RGB data, to add a 4th channel as (R,G,B,1).
    • PFM images start on the top row, while UE5 textures start on the bottom row, so you will also need to flip the image top to bottom during the load
    • For the GPU representation (specified by CompressionSettings), use either a 16 or 32-bit float format.
    • Texture->Source.Init() will initialize the size, format, and data. Use the width and height given in the file, 1 for the number of slices, and 1 for the number of mips. Look up "Source.Init(" for a number of examples of initializing a texture from data on the CPU.
    • Call Texture->UpdateResource() to update the texture object with the loaded data.
    • You can do this incrementally to test as you go: right dimensions with nullptr for the data, then set the TC format, then provide the data.

Grad Students

For rendering, graphics hardware has built-in support for cube textures, consisting of six square 2D textures looking out in the six directions: +X, -X, +Y, -Y, +Z, and -Z. The light probe page also provides each in a single image that contains all six faces in a cross like an unfolded cube (the *_cross.pfm files). To convert the unfolded cube format into the six faces, you will need to both extract the square sub-images and flip them around based on different assumptions for the axis directions. These are the directions for the unfolded cube, and for the six cube faces:

cube light probefolded cube layoutcube map layout

You will need some way to tell when to create a cube map instead of the regular texture. I recommend using the FactoryCreateFile function so you treat files that end with "_cross.pfm" differently from ones that don't.

Create a second reflective glass material using your cube texture. For a cube map, plug the reflection vector directly into the UVs input for the Texture Sample node, rather than using the sphere equation.

Submission

As always, for full credit, you must commit multiple times during your development.

Add an assn5.txt. Tell us what works and what doesn't, and anything else you think we should know for grading. For grad students, tell us what extensions you did. If you did any extensions for extra credit, tell us which ones. Tell us where to find the test files for any extensions.

For the base assignment, include a link a screen shots of the UE texture editor pane for your 2D texture object, with the image, size, and format all visible. Also include a link to a screen shot of the SM_MatPreviewMesh_01 object using your light probe texture.

For the grad extension, also include links to screen shots of each of the six faces (select at the top of the texture editor pane).

Push to your repository, and tag your final commit with an assn5 tag. Make sure the image links are included in the commit that you tag for us to grade.