Trouble getting to run a custom output shader

[winniban]

Dear Community,
i really need your help. I am currently trying to fix my problem for several days, reading tutorials and forum posts all the time. But i could not find any solutions.

I am at the beginning of writing my master thesis which includes writing some output shaders for mental ray.

I am using Maya 2012 64bit /windows 7

For the first step, i try to run a sample output shader from the mental ray help.

#include "shader.h"

struct depthfade {
        miScalar   tnear;   /* no fade closer than this */
        miScalar   tfar;    /* farther objects disappear */
    };

extern "C" DLLEXPORT int depthfade_version(void) { return(1); }

extern "C" DLLEXPORT miBoolean depthfade(void *result, miState *state, struct depthfade *paras)
{
        int         x, y;
        miColor     color;
        miScalar    depth, fade;
        miScalar    tnear, tfar;
        miImg_image *fb_color, *fb_depth;

        tnear    = *mi_eval_scalar(&paras->tnear);
        tfar     = *mi_eval_scalar(&paras->tfar);
        fb_color = mi_output_image_open(state, miRC_IMAGE_RGBA);
        fb_depth = mi_output_image_open(state, miRC_IMAGE_Z);

        for (y=0; y < state->camera->y_resolution; y++) {
                if (mi_par_aborted())
                        break;
                for (x=0; x<state->camera->x_resolution; x++) {
                        mi_img_get_color(fb_color, &color, x, y);
                        mi_img_get_depth(fb_depth, &depth, x, y);

                        if (depth >= tfar || depth == 0.0)
                                color.r=color.g=color.b=color.a = 0;
                        else if (depth > tnear) {
                                fade = (tfar - depth) / (tfar - tnear);
                                color.r *= fade;
                                color.g *= fade;
                                color.b *= fade;
                                color.a *= fade;
                        }
                        mi_img_put_color(fb_color, &color, x, y);
                }
        }
        mi_output_image_close(state, miRC_IMAGE_Z);
        mi_output_image_close(state, miRC_IMAGE_RGBA);

        return miTRUE;
    }

declare shader
    scalar "depthfade" (
		scalar "near" default 5,
		scalar "far" default 50
	)
	#: nodeid 1391649
	apply output
	version 1
end declare

I use Visual Studio 2005 and 2010.

After having setup the shader (which is correctly loaded in maya) and trying to render. Maya Crashes. "//error (mayatomr.output) while defining camera "camerashape1":undefined shader "depthfade1".

miCustomNodesBegin;
miCustomNodesFile "D:/customshader/include/depthfade.mi";
// parsing D:/customshader/include/depthfade.mi
miCustomNodesLibrary "D:/customshader/include/depthfade.mi";
(u"D:/customshader/include/depthfade.mi").replace("\\","/");
# Result: D:/customshader/include/depthfade.mi # 
(u"D:/customshader/include/depthfade.mi").replace("\\","/");
# Result: D:/customshader/include/depthfade.mi # 
(u"depthfade.mi").replace("\\","/");
# Result: depthfade.mi # 
// loading D:/customshader/include/depthfade.dll
// Result: D:/customshader/include/depthfade.dll // 
miCustomNodesIdAction;
// Result: 1 //

I also compiled a color Shader with the same Build Solution and compile flags (64bit)and it works perfectly. Thats why i dont think that i setup VS 2010 incorrectly.

Has somebody any idea?
Or could somebody please try to compile and run my code on his x64 machine?

Thank you very much.

winniban (from germany)

[steve]

Few hints,

1. Do you have z-depth output enabled in your scene ?

2. mental ray has deprecated the function mi_output_image_open() in favor of using named frame buffers. Try to use mi::shader::Edit_out_img class instead.

3. Where do you have your node ID from ? For experiments, you should start without assigning a specific ID but let Maya decide about upon loading your shader.

4. Maya2012 is on VisualStudio 2008, aka VC9. Should not be critical when using .dll's, but it introduces more complexity to system file dependencies.

Hope this helps, Steve.

[haggi]

The problem does not lie in the shader itself. If you export an mi file from maya, you will see that in the camera definition there is a reference to the ouput shader, but there is no instance of the output shader anywhere which can be referenced.

This I can read in the mi file:

output = "shaderGlow1:perspShape"
# mentalrayOutputPass1
output = "dummyOutput1"

And while "shaderGlow1erspShape" is defined somewhere above, the "dummyOutput1" is not. I suppose this is a serious bug in maya (maya2012 SAP here).

[maxt]

Yup, output shaders are not defined actually. Not funny, but you can do that yourself from with in a geometry shader.