mr equivalent of brickmap?

[1armedScissor]

Can anyone tell me if there is a mentalray equivalent of renderman's brickmaps?

Thanks in advance everyone.

[bart]

Depends on how you are using brickmaps. And at what level are you talking about. For example, there are several mechanisms available for shader writers to make re-usable data caches of sorts. And there are some new techniques coming out in 3.7.

Aside from implementation, what kind of effect are you trying to achieve?

[1armedScissor]

Point Clouds.

[bart]

That is an implementation. What are you using the point clouds to do?

[1armedScissor]

for caching things like occlusion and indirect illumination

[haggi]

I think in 3.6 you have to do it yourself. You can render a lightmap and save the points into a pointcloud file. If you follow the pointCloud file format from pixar, you should be able to execute brickmake.

If you want to do it completly in mentalray you have to do the LOD or mipmapping youself because there is no such mechanism in mentalray. But it should be not too difficult to implement an octree which gives you different resolutions at different distances.

[1armedScissor]

Thanks Haggi, we'll give this a shot.

[bart]

In 3.7, there are particle maps, and there should be a particle map documentation that comes with that. But the tutorial doc for it might not be out yet. I'll keep you posted.

[caml]

We had some success with the light map to ptc approach. I did not put the tool together so I do not have all the details.

Basically 1 object + 1 shader + 1 point samples file (where to sample) = 1 ptc file

The main draw back was it was very similar to 2d texture baking and shared some of the same issues.

Imagine you have a car with hundreds of parts and you wish to bake occlusion to texture. You have to bake hundreds of maps and manage them. Also you end up with maps that are way too big (say a 2k map for a couple of bolts), and others that are too small (the same 2k map may not work for the hood of the car).

What we were really after is somthing that is more camera based. When mental ray writes an image it takes several samples per pixel and filters the results as an image. What we want to do is write out these samples prior to image filtering and store them at world space P and N of that sample. This is more or less what prman does with bake3d. This presents certain challenges of occlusion (to the eye, not ambient occlusion). There are a few different techniques you can use to get around this issue. There are a couple other minor issues such as radius size of each point sample but you can get around that aswell. We would be very keen to hear about any developments to write out this type of data to some sort of open format.

cl

[caml]

Hi Bart,

You mentioned particle data in your last post. This sounds like the type of thing we are after, for each shading sample evaluate some shader and return the color and position of that sample. I have found reference to it in the mrfm docs, but it seems to be more of an overview. Is there more documentation on this feature?

http://download.autodesk.com/us/maya/2009help/mr/manual/points.html

Thanks,

cl

[bart]

Using Maya 2009?

Did you check out the section further on down already?
http://download.autodesk.com/us/maya/2009help/mr/manual/map_mi.html
Also, the Map Data section underneath the Using and Writing Shaders > C++ Shader Interface Extension?

Also, there is a longer standing method of creating your own kd trees.

By the way, if you use the rasterizer, you can control samples collect (visibility samples in Maya) to have 1 sample per pixel, in the center of each pixel. Then, a box 1 1, would only show 1 sample per pixel.

You can also use sample passes if you just want the samples but at different supersample resolution. It's a pass statement in the camera block.

Do you have access to mental ray standalone?

Also, in the light map approach, why not bake them all to the same texture map? Any type of mapping can be used. Games folks often put multiple objects into the same texture map.

There actually may be several simpler ways of doing what you most recently described as needing, rather than learning a new map data api.

[caml]

Hi Bart,

Thanks for the info. I will forward your reply on to a couple of collegues who have a little more experience writing C++ mray shaders. Our overriding goal to to be able to exchange information between mental ray and prman. The simplest method is to do ndc lookups. In most cases this works better than you would expect. We would like to expand our toolset to handle more complex cases.

You mentioned samples passes. I think on the surface this is precisely what we are after. Store all the shading samples prior to pixel filtering. In prman this can be accomplished with their bake3d() command. You pass the shader the Position and Normal of the currently shaded point, and value pairs of the data you wish to store. As a reference, I believe gelato (I think some of the developers were absorbed by MI) also had the same thing, but called a spatial database instead of a point cloud. Unfortunately I think these samples files are binary and proprietary, and also contain a fixed set of variables. When I saw the map data/point data stuff in 3.7, my understanding was that this was much closer to the behaviour of bake3d. That is to say render the scene and shade normally; however, for each sample dump the position, normal and arbitrary data value to file. Once accomplished passing from one open format to another should be pretty straight forward.

On a side note, I do like the idea of forcing the renderer to do 1 sample per pixel. We did some tests with image files (with P, N and some arbitray color as exr channels) to ptc. One thing about the rasterizer, is that (I will admit not thoroughly tested, more of a general observation) it seems to take much longer than the equivalent scanline render.

We had some success with the lightmap approach baking directly to ptc. However there are some non trivial issues of always supplying a list of point to sample. We would rather have the renderer do that.

Again, all good info I will pass it along to some of the more technical guys.

Thanks,

cl

[bart]

Regarding rasterizer speed, keep in mind that you will now always be using a fixed sample grid, set by samples collect (in mental ray terminology; it takes on a variety of other terms inside each 3D application,eg, Visibility Samples in Maya).

With regular scanline and raytracing(primary ray), sampling is adaptive, meaning that it is possible in simple shaded scenes, there will be less than a sample per pixel.

So when you compare, you need to include the samples settings, and use a fairly representative scene for what you will be rendering. Simple scenes are misleading.

In motion, the rasterizer will probably show speed benefits, especially in more complex illumination or coloring, ie scenes demanding more samples.

[1armedScissor]

...We had some success with the lightmap approach baking directly to ptc. However there are some non trivial issues of always supplying a list of point to sample. We would rather have the renderer do that...
cl

What about doing it per vertex during the lightmap phase? That was how we were implementing it. We were also storing the results for the entire scene into one file instead of one per shader.

[jasonhuang1115]

Hi,
I am curious about what kind of features offered by 3D brickmap and 2D point cloud are superior than
mental ray's fg_map? I have seen several discussions and papers describing how great the point-based GI is and wondering if this is also a direction that mental ray is heading or if MR already had similar feature.

[1armedScissor]

Being able to sample the scene and store arbitrary information from the scene in point cloud gives you the opportunity to do much more than just GI.

[bart]

By the way, that is what GI is: photon energy and incoming direction stored at points in a photon map.

Particle maps in mental ray are a generalization of this so that a shader writer, or an OEM 3D application, can create any type of data to attach to points.

This allows any integrator of mental ray to use whatever kind of technique they wish to.

This also makes mental ray an excellent application for academic research in the area of new GI techniques. The customization for such is extremely flexible.

[jasonhuang1115]

Thank you, bart and 1armedScissor.
If I understand it correctly, point-based point cloud is implemented in Renderman so that one can minimize the usage of raytracing in production and still get occlusion, colorbleeding and other additional effects. As native raytracer for mental ray, is particle map an approach of saving and reusing data to avoid raytracing, too?

[bart]

Most modern GI techniques are variations on pre-calculation storage and re-use. Take FG for example. We store indirect diffuse illumination in the FG map. Because it is diffuse, we don't have to sample at every eye sample point. Instead we can control that point density with the FG control parameters. We also let the user control the illumination sampling above each FG point, ie how many FG rays are shot.

Particle maps provide a generalization of this concept.

By the way, we also have an Ambient Occlusion cache which can be used with the mib_fast_occlusion shader, or incorporated into other shaders if they use the appropriate shader calls. So if you want that softer feel for ambient occlusion, then you can choose to use the faster method.

When re-using stored data, your motion blurred scenes perform better as well.

In fact, if you take that generalization further, in training class I like to present the rasterization techniques as somewhat of a pre-stored shading sample that avoids dealing with time until re-use.

[1armedScissor]

Hi,
I am curious about what kind of features offered by 3D brickmap and 2D point cloud are superior than
mental ray's fg_map? I have seen several discussions and papers describing how great the point-based GI is and wondering if this is also a direction that mental ray is heading or if MR already had similar feature.

Depending on the data stored in the point cloud and how it's structure is formatted you can also use it to accelerate any raytracing, or even behave as a simplified geometry representation.

[caml]

Glad to see this thread still going. I switched companies and countries so I have been out of the loop. I will try to respond individually to some of the observations posted. I truly feel that solving this issue could really help mental ray make inroads with new users.


What about doing it per vertex during the lightmap phase? That was how we were implementing it. We were also storing the results for the entire scene into one file instead of one per shader.

Conceptually the idea is the same, but for film rez rendering, the point density is often not sufficient. Fine details tend to get discarded in vertex maps.

cheers,

cl

[caml]

Hi,
I am curious about what kind of features offered by 3D brickmap and 2D point cloud are superior than
mental ray's fg_map?

All of these items can be thought of as storage formats. Take an exr image, it has and arbitrary number of channels which contains users data, could be depth, uv, position, diffuse etc. We index the map by asking the value of pixel at a given row and column address in the image.

For example:

return the z depth at pixel 123, 234 of a 512x512 image

The 3d formats do the same thing, but store the data in x,y,z instead of just x and y. Amongst other things, you can store values for occluded samples not visible to the camera.

A brickmap is typically derived from a pointcloud, the same way an image is derived from pixel samples. It describes volumes rather than points. This allows it to do 2 specific things:

1. compression, think of when you crank up the compression on a jpeg.
2. mip mapping: a volume like a texture can have tiled lower rez versions of itself.

As with vertex mapping, you can lose detail with brick mapping, and encounter a few tell tale artifacts. In theory it is quite good, in practice there are still a few non trivial issues to sort out. The description above is a little bit of an oversimplification, google should yield a more detailed explanation.



I have seen several discussions and papers describing how great the point-based GI is and wondering if this is also a direction that mental ray is heading or if MR already had similar feature.

The point based stuff is interesting and generally means processing point data outside the renderer (although inside is also available). It is well suited to compliment renderers which have a ray tracing bottleneck. Mental ray is quite a quick ray tracer and I would view it as an alternative to pointbased GI.

cl

[caml]

Being able to sample the scene and store arbitrary information from the scene in point cloud gives you the opportunity to do much more than just GI.

This is the conclusion we arrived at.

[caml]

By the way, that is what GI is: photon energy and incoming direction stored at points in a photon map.

Particle maps in mental ray are a generalization of this so that a shader writer, or an OEM 3D application, can create any type of data to attach to points.

This allows any integrator of mental ray to use whatever kind of technique they wish to.

This also makes mental ray an excellent application for academic research in the area of new GI techniques. The customization for such is extremely flexible.

This feels like the scenario we are looking for, a generalized way of getting raw data in and out of mental ray. This would also potentially allow:

- processing or modifying baked data
- reading the processed data back into mental ray
- reading that data back into a custom renderer

[caml]

Regarding rasterizer speed, keep in mind that you will now always be using a fixed sample grid, set by samples collect (in mental ray terminology; it takes on a variety of other terms inside each 3D application,eg, Visibility Samples in Maya).

With regular scanline and raytracing(primary ray), sampling is adaptive, meaning that it is possible in simple shaded scenes, there will be less than a sample per pixel.

So when you compare, you need to include the samples settings, and use a fairly representative scene for what you will be rendering. Simple scenes are misleading.

In motion, the rasterizer will probably show speed benefits, especially in more complex illumination or coloring, ie scenes demanding more samples.

Hi Bart,

This is a good point. I think my tests were done on simple spheres. Incidentaly, this also serves as a pretty good analogy for why we prefer a point based approach to lightmap/textures:

If you have a simple scene with 1 piece of geometry and one piece of geometry such as a character, then it makes sense to bake effects such as occlusion to a texture map.

By comparison, if you have a very complex environment with thousands of of textures and pieces of geometry it can be quite complicated to bake and track these thousands of textures. Building in logic to determine the optimal resolution for each map, presents another challenge: how big is the object? how big is it on screen? etc.

At a certain point, screen/scene based sampling becomes much easier to manage. We piggy back off of the scene sampling from the renderer, to get our point distribution. In this way baking point clouds scales better with scene complexity. Similarly, one could bake photon hits to a texture for all pieces of geometry, and perform lookup from there, but it is much easier to think of it as a single 3d scene cache.

Let me know what you think about such an approach.

Thanks,

cl

[caml]

In fact, if you take that generalization further, in training class I like to present the rasterization techniques as somewhat of a pre-stored shading sample that avoids dealing with time until re-use.

This is the core of what we are trying to achieve:

just replace:
pre-stored shading sample that avoids dealing with time until re-use

with:
pre-stored shading sample that avoids dealing with time until re-use in a different renderer.

[1armedScissor]

Caml what about sticking with sampling during the lightmap phase but instead of sampling per vertex, sampling x # of points along the surface in mesh mode. Wouldn't this give you the density you are looking for in your point cloud?

[haggi]

I'm just starting to write a lightmap shader to generate a point cloud. Unfortunatly we still work with mays2008 and the builtin mentalray. This way we dont have access to the new map features.

What I want to archive is a point cloud with arbitrary data for every point. e.g. normal, color, coordinates, occlusion.

If I understand the manuals correctly, then I have to use a writable texture. The only way to write multiple data seems to be an exr file format (however I can access the different channels). Is there any other way to save my own data structure?

[haggi]

Is there a big difference in speed or memory if I use mi_eval_color() instead of calling a shader with mi_call_shader_x() in a lightmap shader?

I have to set pri, normal, point in the state variable. So I could call eval_color() without problems. Is there any difference?

[1armedScissor]

I'm just starting to write a lightmap shader to generate a point cloud. Unfortunatly we still work with mays2008 and the builtin mentalray. This way we dont have access to the new map features.

What I want to archive is a point cloud with arbitrary data for every point. e.g. normal, color, coordinates, occlusion.

If I understand the manuals correctly, then I have to use a writable texture. The only way to write multiple data seems to be an exr file format (however I can access the different channels). Is there any other way to save my own data structure?

Instead of using an existing file format why not create your own?

[haggi]

File format problem is solved. I thought that I'm restricted to the builtin lightmap edit functions to be sure that it works in a multithreaded environment. So I simply use thread local storage to use an array and write it in an exit function. But it seems that lightmap renders are singlethreaded anyway.

[1armedScissor]

File format problem is solved. I thought that I'm restricted to the builtin lightmap edit functions to be sure that it works in a multithreaded environment. So I simply use thread local storage to use an array and write it in an exit function. But it seems that lightmap renders are singlethreaded anyway.

Out of curiosity, how are you using the pointcloud Data that you're writing out Haggi?

[haggi]

I'm creating a volume shader where I can read in the pointcloud. If I create a brickmap from the data this could be very efficient for rendering a very big amount of "geometry".

[1armedScissor]

I'm creating a volume shader where I can read in the pointcloud. If I create a brickmap from the data this could be very efficient for rendering a very big amount of "geometry".

I ask because I've been trying the same for some time now. I can generate the point cloud easy enough but unless it's incredibly dense I'm having difficulty using it at render time. For any given sample in my regular render I've tried many different means of interpolating data from the point cloud for use in the render but any interpolation scheme I've used isn't good enough for a production quality render. I'm pretty certain that there must be a better way and I'm curious as to what it is.

[haggi]

Well, the first usage of this shader will be for plants and trees only for Backgrounds. I'll try to use an octree structure for a more efficient lookup and less aliasing problems. In my volume shader I'll try to render the informations maybe as spheres or as disks like prman. So for a tree several houndred thousend sample points should be enough I think.

[1armedScissor]

Well, the first usage of this shader will be for plants and trees only for Backgrounds. I'll try to use an octree structure for a more efficient lookup and less aliasing problems. In my volume shader I'll try to render the informations maybe as spheres or as disks like prman. So for a tree several houndred thousend sample points should be enough I think.

ahh, so is that how prman does it? I thought the way they did it was to make sure that there was at least 1 sample point for every micropolygon (probably way off base on this one). I've also started to think about possibly using some sort of splatting techniques.

[haggi]

Yes, if you render a closeup of a brickmap in renderman, you can see, that it consits of a lot of disks oriented to the camera. First I'll try to render spheres. Maybe I switch so disks later if I see any advantage.

[1armedScissor]

Thanks for the replies haggi!