Shading Pipeline Update

After about a week of trying to figure this out, some changes have been made to the pipeline itself. One of which is a completely new build of the Shrimp shader builder, this one called Prawn and is now hosted on GitHub. This build includes AOV parameters that are needed by RIBMosaic and the EXR driver, so before the shader even gets to Blender, we can see in the temp folder if the desired results are indeed rendering. Another thing that was figured out was that the shading models are confusing, the normal Diffuse and Specular blocks work fine, however models like Oren-Nayer does not render right. This was confusing, then by accident realized that with some hacking, the other model can simply be piped through Specular, or Diffuse. This turns that AOV parameter that is causing issues to be linked to the right one for RIBMosaic. Or one could simply do some work getting the Diffuse and Specular to look right by it self.

The next advancement is that since the shading pipeline was somewhat worked out, shading tasks have taken over and several shaders have been fixed, created and then brought into Blender and then added to the main scene files. Soon, the shading portion of Widow will be complete, since much of the shaders are already to go, the shading fragments have been seperated for nearly two years now by the Blender Material’s that Daniel had defined while making the geometry.

Brings me to another topic, the actual reason for the shading pipeline with all it’s code, software and links. In the commercial Renderman world, the tool SLIM has been the standard for shader creation, after all it is developed by Pixar and nearly every studio that uses Renderman, use SLIM. Problem in the open source world is that there is no such solid link between shader source code and the final asset, that is until now. Prawn is that attempt, to take an existing shader editor, modify it so that there is a solid link between it, RIBMosaic and Aqsis. This involved changing header code, changing each block that contained a header include, changed the source code to output the correct variables and finally to make sure that the AOV channels were being rendered correctly. While it is possible to create the shader source from scratch, via a text editor, the complexity involving what goes to where and why, not to mention that most people who would want to really use this software tend to be artists rather than programmers, well this was intended to fulfill that very spot.

So this is why the shading pipeline was so important to create, test and use, it really is a proof in concept that there does exist a tool that is nearly as powerful as SLIM. It is true that shader’s similar to the type used for the movie “Wall*E”, this shader was pretty much applied to everything in the movie, involves mainly writing the code from scratch in a professional studio.

However when used with SLIM it is possible to use Maya attributes for shader variables, thus making “new” shaders using a single shader, similar to the original Mosaic surface shader. In concept. SLIM is just far more advanced. Prawn, Shrimp, WidowShade, SLer, Shaderman…. all these tools at one point in time were or are being created to fulfill a need. The need to visually create shaders, since not all artists are programmers, artists visualize things better than reading text, so even though it is pre-defined blocks of code, an artist can create new shaders, even if they are basic without ever having to write it out. Could the shader be made more efficient? Most likely, however in consideration of the pipeline in this example, a lot of things were tied in just to make it possible to use a shader creation tool, so I could test out a new shader in minutes rather than hours just based on the templates and connecting new blocks to others. Typing it out in code would, in MY case, would take a considerable amount of time.

Prawn is only a modified Shrimp 2.0 program, in fact the original Shrimp source dates back to 1999 and only in the past couple years gotten a complete overhaul. Now Prawn is a fork off of that project and is intended to be part of the Widow Pipeline. While the code remains largely the same, there are plenty of differences, mainly to make this a plug n play system, so that anyone involved with making a pipeline could simply plop this into the system and go. While it is a Linux only build, literally all you need to do is compile it and start creating. After getting the required support libraries like FLTK and Scons of course.

Another update is that the pipeline itself will finally be undergoing the long awaited upgrade. Since Blender 2.57 is stable on my system , plus our chief modeler is more adept in the new Blender than the old, not to mention the amazing work Jeff Doyle is making on RIBMosaic, well it is time to really test out the new generation of Blender to Renderman.

Shader technical difficulties

Project Widow has had it’s fair share of technical difficulties, ranging from software bugs to networking problems and everything in between. In fact there has been so many technical problems that it is hard to remember exactly all of them. The newest one however seems to have crept into the shading pipeline, something that has been worked on for over a year in order to function as planned since the start of the production.

It is a very difficult process to figure out because there are different results each time, not sure where the problem lays at it’s root and let alone, how to fix this. So here is an attempt to document the steps needed to figure out the problem.

First off we need to explain what the shading pipeline is. The shading pipeline is the code and software that has been created and/or modified so that shaders can be visually created and then brought into Blender where Aqsis can render them. Sounds simple but it is not. The reason for the lack of simplicity is AOV, or Arbitrary Output Variables, or in laymans terms, the layers of images that contain a type of rendered output to be composited later on, so that we can make changes to these layers themselves so that we don’t have to re-render an entire sequence of frames just to fix the color of the specular reflection. Now in order to have this ability, one needs to declare these variables first. In Blender it functions the same way, so back in 2009 we decided to name these variables after the Blender ones, so that we could import the image sequence into Blender’s node compositor and have the outputs already to go without having to import each element. This is done via the MultiLayer OpenEXR format, one feature that was added to Aqsis because of this very project, it is also the same format used by Blender.

Last year there were attempts to modify a shader builder called ShaderLink, while this did prove successful the idea was to have AOV variable in the shader, however the code was not as robust as initially thought and then work went underway on the next contender, Shrimp. This actually had AOV built into the source code from the start, all one needed to do was name them as the same in RIBMosaic and Blender, for compatibility. The next problem was the include files, some of them were just too long to be used by Blender, the text editor would cut out the actual filename from the list and this include file happened to be the shading models file, essential to the functioning of each and every shader. So massive text editing was done to change each and every node and header file so that Shrimp, Blender, RIBMosaic and Aqsis could all be on the same page.

So that is the shading pipeline, a year long project in itself to make sure that each shader would not only function correctly in the final render but also later on in the composite stage, where each AOV element would be there for editing if need be.

Now there are problems, naturally.

Somewhere in the shading pipeline there is a bug and I cannot figure this one out, it’s weird. One of the problems was identified though, somehow Shrimp was reading the backup files that one of my text editors had made during the initial conversion from the Shrimp header library to the custom RSL Library header collection, which mainly involved changing all the nodes to look for the edited header files. So in the shader source created by Shrimp there would be both the Shrimp and RSL header file #include declarations. This was fixed simply by deleting the backup files, so now Shrimp is again running fine at the moment.

The next issue was even before this find, the fact that AOV’s were not being rendered the right way while using Blender and Aqsis. See in Shrimp, while constructing a shader visually, Piqsl would display the preview rendering as it is supposed to. The shader source would then be entered into the Blender text editor where then using RIBMosaic would add the shader fragment, then you apply the shader fragment to the Blender Material and in practice would function as you would expect. However the transition from Shrimp to RIBMosaic changed “something” and the render would come out looking NOTHING as it did while working on the shader in Shrimp. For instance, while building a rusty metal type of surface, I would apply the shader to a pipe, then preview the model in Aqsis, however as opposed to the shader preview in Shrimp, the rusty metal shader would render totally different. The weird part is that when using multiple AOV outputs, either by individual files per element, or via the MultiLayer OpenEXR file, the elements would be rendered correctly by themselves, just not combined. So in this instance I was trying to test out the color of the rust areas by changing the colors to neon reds, yellows and blues, combined the render would look nothing like it was supposed to, however opening up the AOV outputs in either GIMP or Cinepaint would show that they did in fact render the color to the Color AOV output file.

Rusty Shader, combined AOV

Color AOV

Confusion.

So after this I tried to see if I could just forget about AOV rendering, maybe if in the end it would just be easier to render the frames as they are, do a small amount of post work on the sequences, then render to video and be done with this short film finally. I opened up WidowShade, the shader editor that I modified from ShaderLink specifically for this short film. What happened was a disaster, the program fails all the time. Something is very wrong with my workstation. Even with the Aqsis binary path hardcoded into the program, WidowShade fails to render a preview, gives an error message and then that is it. So WidowShade is out of the question again, back to Shrimp where I still have no idea WHY the shader pipeline is failing.

This is critical because this is the WHOLE reason for this short film, to showcase the power of RSL and Renderman. Otherwise there is no point at all for this short film to exist at all.

You can imagine the frustration level over here.

Changes in lighting

In a series of changes in the short, one of the original ideas is coming back to the short film. In the beginning the outdoor environment was intended to be at night, then for some reason the time of day was changed to daylight and has stuck since. Well it has been decided to change it back to night and use a “streetlight” for illumination, in renders the sun shader has been giving the lighting a headache, the shadow map just does not cover enough of the set, no matter what geometry is placed in there. Spotlights have an angle, so one can easily see that the shadows do not behave like they would with the sun, multiple lights cause shadow overlap, global illumination cheats tend to help but in general it has been a challenge to perfect. So a quick choice to go back to night was made, simplify the lighting since that environment is seen only once and move on the next area.

The next change was the tunnel in Scene 008, the original test lighting rig would cause Aqsis to crawl on it’s knees, there are literally some 200 lights in it. This would have been the mostly resource hungry render job in the short, next to the the opening scene, thousands of minutes per frame and that was without shadow maps. Motion blur takes up a good chunk of time to render, even with the speed increases in Aqsis over the past 2 years, so imagine calculating the time to render a series of shots that could take thousands of minutes per frame, with many shots taking up hundreds of frames…. months of just render time if done on a single system.

So now the tunnel lighting is composed of point lights, with Inverse Square falloff set to a small distance from the source. This creates the illusion that they are bright but not overpowering the scene, otherwise the scene set with “standard” settings makes the scene look dull and lifeless, there is no contrast. There will be no shadow maps made for these lights, the camera will be moving too fast for anyone to notice plus the attention is on the train rather than the environment. If motion blur can be applied in post it would save thousands of hours of rendertime for this scene alone.