Attached is one way of doing this.
It looks like you made some mistakes in your algorithm. 5 should be 0.5, and 2(a) should be 2(ab):
I've seen it written in this C notation:
if a is (<or = too) 5 c=2(a)
if a is > 5 1-c=2(1-a)(1-b)
In mental mill, I broke each bitmap into component colors to be processed separately. I created a Phenomenon to make the comparison of the float value against 0.5. I use Math_float_conditional to choose one result or another. The formulas are just multiplication, so there are several math_float_multiply nodes. Math_float_invert takes care of the 1-n operations. The two options are wired into the Conditional node.
a <= 0.5 ? 2 * a * b : 1 - ( 2 * (1 - a) * (1 - b) )
This Phenomenon is used once for each Red, Green, and Blue. The results are recombined into a color at the end with a Conversion_floats_to_color node.
Finally, the whole works is wrapped up as a Phenomenon, so you can add it to your toolbox for future use.
The shader is attached, along with a screen shot of the entire network. Take a look at the details and let me know if you have any questions.