Proposal: Normalized textures

For collaboration on developing the mod capabilities of VS; request new features, report bugs, or suggest improvements

Moderator: Mod Contributor

chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

@Safemode:
No, please! Let's get rid of spheremaps. There's nothing easier to using spheremaps at all. Cubemaps are supported in hardware. We can keep the spheremap code as a fallback for videocards that don't support cube maps, perhaps; but we should use cube maps for the reflective environment by default. Quality IS important. What's the point of giving chrystal domes high shininess if using spheremap is going to keep all those anisotropic blur and pixelation artifacts showing up?

OT:
Sorry I've been totally absorbed in the LaGrande work. Just for the curious, the Streaker is done:
http://wcjunction.com/phpBB2/viewtopic.php?p=8081#8081
and the the bumpmap ambient occlusion generator is almost there...
http://wcjunction.com/phpBB2/viewtopic.php?p=8133#8133
After that will be the rust generators...
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Sorry, Klauss; we posted at the same time it seems; missed your post.
Any reason why the env cubemap should be down-sampled? I'm thinking this assumes all shiny surfaces are spherical; but that's not the case: windows are often flat; you get the same scaling on reflection as you get by direct view.

By the way, I'm thinking seriously about using a single bit for alpha, and specifying the amount of alpha in xmesh. Heck; we only use alpha for glass, anyways; we don't need to add 8 million bits to the texture pack for a piece of glass; 1 million bits should be more than enough waste.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Usually the environment is downsampled since you seldom want mirror-like surfaces (and when you do, other techniques are way better). And doing so increases performance a big lot.

But also since, eventually, some day, with luck, you'll have dynamically rendered environment maps. Dynamically rendering several 2k x 2k x 6 is... well, expensive. When you're seldom using the extra resolution, it's simply dumb.

It's all about cost-benefit. It costs a lot to have full-res cubemaps, and provides little benefit.

Just a question to put things into perspective: will you often have fully flat windows/mirrors taking a significant portion of the viewscreen?
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Not a significant portion of the screen; but that's not too relevant, since the user's eyes and brains don't necessarily divide their attention equally across the screen. Reflections on a window are one of those things that tend to catch the eye. But if it's too expensive, so be it. I just had to ask.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Well, the resolution should be configurable anyway, so you could try higher resolutions. The real problem with high-resolution cubemaps is with dynamic content. Static content, while also less efficient if at high res (as always), and IMO not worth it, is certainly a different story. So it may be acceptable to you anyway.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

The size of the environment map could be configurable. Since we'd just be picking a mipmap, this should be no problem.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Been putting my ideas for a shader into a text file. This is out of the top of my head, except for two lines from Klauss for computing mipmap level from the "shininess" (which I've decided to call "gloss", which I think is a much better term for the concept). Haven't even looked at the current shaders; this is just the way I think the right shader should be. But expect mistakes, and take it as pseudocode, which it is.
Also, I've no idea where/how the tangent is used.

Code: Select all

//invariants:

float3 computed_ambient_color; //(see Note 1a)
float3 xmesh_diffuse_factor;
float3 xmesh_specular_factor;
float3 system_background_factor; //(see Note 1b)
float3 light; //color of the star //(see Note 1c)
float  xmesh_min_alpha;

//variants:

float3 L; //Light vector
float3 V; //View vector
float3 N; //Normal vector
float3 T; //Tangent vector
float damage;
float cloak;
//and the texture interpolants...

main()
{
  float4 temp;
  float3 diffuse_mtl_color;
  float3 specular_mtl_color;
  float3 emissive_mtl_color;
  float3 albedo_mtl_color;
  float3 mtl_normal; //mtl_normal.x = du; mtl_normal.y = dv; mtl_normal.z = z
  float3 normal;
  float3 R; //Reflected_vector.
  float  mtl_alpha;
  float  mtl_gloss;
  float  mtl_ambient_exposure; //the ao baking value
  float  ambient_diffuse_factor;
  float  ambient_specular_factor;
  float  ambient_contribution;
  float  diffuse_contribution;
  float  specular_contribution;
  float  tempf;
  float  NdotL;

  temp = normal_map_read();
  mtl_normal.x = 0.3333 * (temp.r+temp.g+temp.b) - 0.5;
  mtl_normal.y = temp.a - 0.5;
  mtl_normal.z = 0.25; //(see Note 2)
  mtl_normal.normalize();

  temp = glow_texture_read();
  emissive_mtl_color = temp.rgb;
  mtl_ambient_exposure = temp.a; //"ao bake"

  temp = diffuse_texture_read();
  diffuse_mtl_color = temp.rgb * xmesh_diffuse_factor;
  mtl_alpha = max( temp.a, xmesh_min_alpha ); //.a comes from 1 bit alpha

  temp = specular_texture_read();
  specular_mtl_color = temp.rgb * xmesh_specular_factor;
  mtl_gloss = temp.a;

  temp = damage_texture_read();
  damage_mtl_color = temp.rgb * xmesh_damage_factor;
  mtl_is_dielectric = temp.rgb; //.a comes from 1 bit alpha

  tempf = 1.0 - damage;
  diffuse_mtl_color  *= tempf;
  specular_mtl_color *= tempf;
  emissive_mtl_color *= tempf;
  mtl_gloss *= tempf;
  diffuse_mtl_color += (damage * damage_mtl_color);

  temp = detail_texture_read();
  diffuse_mtl_color *= temp.rgb; //(see Note 3)
  mtl_gloss *= temp.a * 256.0;

  //vectors and dots
  normal = wiggle(N by mtl_normal);
  LdotN = dot(L,normal);
  VdotN = dot(V,normal);
  R = 2.0 * VdotN * N - V;
  LdotR = dot(L,R);

  albedo_mtl_color = diffuse_mtl_color + specular_mtl_color;
  ambient_contribution = albedo_mtl_color * mtl_ambient_exposure * ambient_light_color;
  ambient_contribution += emissive_mtl_color;

  //Ambient factors (see Note 4):
  ambient_diffuse_factor = sqrt( mtl_ambient_exposure );
  tempf = max( mtl_ambient_exposure, VdotN );
  ambient_specular_factor = tempf * tempf;

  diffuse_mtl_color *= ambient_diffuse_factor;
  specular_mtl_color *= ambient_specular_factor;

  //Fresnel (see Note 5):
  tempf = 1.0 - VdotN; //or, 1 - cosine_of_view_to_normal_angle;
  tempf *= tempf;
  fresnel_reflection_factor = 0.0625 + 0.9375 * tempf;
  fresnel_reflection_factor *= mtl_is_dielectric; //antialiased boolean ;-)
  fresnel_transparency_factor = 1.0 - fresnel_reflection_factor;

  diffuse_mtl_color *= fresnel_transparency_factor; //(see Note 6)
  mtl_alpha *= fresnel_transparency_factor; //(see Note 7)
  specular_mtl_color *= fresnel_transparency_factor; //(see Note 8)
  specular_mtl_color += float3(fresnel_factor,fresnel_factor,fresnel_factor);//(see Note 9)

  diffuse_contribution = diffuse_mtl_color * light * max(LdotN, 0);

  tempf = max( LdotR, 0 );
  speculating_light = light * pow( tempf, gloss );
  mipmap_level = max( 0.0, 7.0 - log2( gloss + 1.0 ) );
  speculating_light += textureCubeLod( envMap, R, mipmap_level ).rgb * system_background_factor;
  specular_contribution = specular_mtl_color * speculating_light;

  temp = float3( ambient_contribution + diffuse_contribution + specular_contribution );
  temp.a = cloak * mtl_alpha;

  color = temp;
}


/* NOTES:

1a) "computed_ambient_color" comes from averaging the cubemaps' least mipmaps.
1b) "system_background_factor" is a color that multiplies the background, in
order to darken it, so that the texture can use the full color range.
1c) "light" is the color of the star; usually white, yellow, orange, red; it
should come from reading the least mipmap of the star's texture. There's no
need to use distance attenuation, since the eyes would adapt to the intensity
of light, which they cannot do with a monitor screen, since there's usually
other lights around. Besides, the precision of the RGB out is limited, so we
want to keep illumination more or less constant.

2) The reason we initialize z to 0.25 is a bit of a long story... I wanted to
get rid of the z component in order to maximize the usefulness of the dds rgb
channels, which is itself a bit of a long story. But if we only stored the dU
and dV, we'd have to square them, subtract them from 1, and take the square
root, in order to get z. So I had a brilliant idea: dU and dV are the sines of
the angles. What if we stored the tangents of the angles? Then we'd just have
to write 1 to z and renormalize! But there's one gottcha: tan(a)=1 when a is
a mere 45 degrees... So, I decided to store half the tangents, instead, which
limits the angle to arctan(2), or 63.434948822922010648427806279547 degrees.
Fair enough? Normalmaps are not intended to represent vertical walls, anyways.
But since I store half the tangents, I'd have to double them in the shader
prior to making z=1 and renormalizing. But doing all that is equivalent to
making z=0.5 and then renormalizing. Furthermore, another scaling by two would
be due, due to the fact that dU and dV's 0-1 range represents a -1 to 1 range.
So, make z=0.25, then normalize. Now you believe me... :)

3) Both the RGB and alpha channels of the detail texture are pretty near white
and maxed out. The point being, to apply detail texture with the least effort:
a simple multiplication. Detail's rgb modulate diffuse, while its alpha channel
modulates gloss. The purpose of this detail texture is simply to mask pixelation
by dithering. The point is not for it to be noticeable. Just tileable perlin
noise, but well formulated. There's no reason to control blending of the detail
texture, since it is mipmapped, and at longer ranges the mipmap minimizes out
to a single texel --i.e. a smooth, 98% white.

4) Ambient factors: We multiply diffuse color by the square root (gamma 0.5)
of the ambient occlusion as a compromise. On the one hand, modifying diffuse
color by ambient occlusion is highly incorrect and make ships look smokey, yet
everybody does that... The correct way to introduce ambient occlusion is in
the light map or "glow texture". But anyhow, having a half-strenght modulation
sort of helps the eyes forget that we don't have shadows.
The specular ambient factor is a bit harder to explain... One artifact of env
mapping is the lack of "specular self-occlusion". You look at the side of a
pipe that is reflecting the sky background where it shouldn't because it's in
a recessed place completely surrounded by ship on all sides! To help sweep this
artifact under the rug, I've been premultiplying the specular texture by the
square of the ao (gamma of 2), that is a darkened ao. The result is that only
parts of the ship that are well exposed to ambient light will have much of a
specularity. It works really well. Now I got a new idea I want to try, and
that is to make the specular darkening factor the greater of either the square
of the ao, or the square of VdotN. This is a bit of an "anti-fresnel". The
reason I think this should work is that if I'm looking at a point on the ship
whose normal is poking me in the eye, I KNOW there's nothing occluding the
reflection. Anyways, perhaps this needs to be implemented a bit more smoothly
than with a max function, but I'll think about it later.

5) A fairly good approximation of fresnel factor is (1-cos(view_to_normal))^2
See http://en.wikipedia.org/wiki/Fresnel_equations
the graph on the left. (Consider the average of Rp and Rs.) The graph is for
some hypothetical material with a base reflectivity of 4%. I'm using 6.25% for
the sake of gamma; --4% just wouldn't be visible, I don't think. Anyways, if
you average Rp and Rs in that graph, the curve is flat up to like 50 degrees,
and then starts to rise. Squaring 1-cos ought to come pretty damn close, and
it's damn easy to compute.

6) We multiply diffuse by fresnel transparency because to the extent that a
material speculates dielectrically, light fails to penetrate the dielectric
coating, and so we don't see the diffuse color under the coating. This only
applies to materials marked dielectric. Non-dielectrics have fresnel of zero,
and therefore we're multiplying diffuse by 1.

7) Needless to say, at an angle at which most light is reflected, little gets
through...

8) Now the flak begins to fly... Why on or off Earth am I multiplying spec
color by the goddam fresnel transparency? Keep your shirt on. What happens is
that dielectrically reflective materials don't have a specular color; well,
the specular color is white, for them; period. So, in the texture, you'd have
specular color as black, as it's not even used. Well, not necessarily true.
Take a glossy paint, for instance. There are reflections that happen off of
its glossy, dielectric surface. Those reflections are white, and fresnelized.
What about at angles where the fresnel reflection is less? Well, past the
dielectric surface, we got something else which has color. Usually purely
diffuse color; but there could be a specular color there too. Case in point,
metalized paints. You see the glossy, fresnelly finish, but inside there's
powdered metal that is highly specular and ultra-low shininess; and below
that there's a diffuse base color. So, eventually, I'm thinking, we could
have that whatever passes through our fresnel factor might get divided into
diffuse and specular components. But I haven't implemented this.
8) For now, specular color is just added to "white" * fresnel_factor, for
dielectrics.


THINGS I'D LOVE TO HAVE:
A) Klauss' GI
B) No compatibility concerns
C) Material "polish", namely a number from 0 to 1 that describes what % of the
surface is polished to its given gloss. Specular contribution would first be
computed for the given gloss, but the result is alpha blended with with a low
gloss specular computation. Nickel and stainless are like that. You can see
pretty sharp reflections blended with blurry reflections, together. And they
are both specular reflections; no diffuse color at all...

*/
EDIT:
I was just thinking, why do we have to have two separate sets of 6 textures? We could write a special shader for background that simply makes the reflection vector equal to the view vector. Heck, it takes so few instruction we could even add that to this shader, with a boolean invariant for "is_background". And we just use the cube map, at full res.
Last edited by chuck_starchaser on Tue Apr 01, 2008 4:16 am, edited 1 time in total.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

I would think the less diffuse a shiny object is, the closer to the 0 level mipmap we would want. We only have a given number of mipmaps, so it's just a matter of deciding what the cut-off points are. getting the right mipmap is as easy as asking GL to get you that number mipmap of that given texture. No problems there. (They're already loaded)
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:I would think the less diffuse a shiny object is, the closer to the 0 level mipmap we would want. We only have a given number of mipmaps, so it's just a matter of deciding what the cut-off points are. getting the right mipmap is as easy as asking GL to get you that number mipmap of that given texture. No problems there. (They're already loaded)
That's exactly what these two lines do:

Code: Select all

  mipmap_level = max( 0.0, 7.0 - log2( gloss + 1.0 ) );
  speculating_light += textureCubeLod( envMap, R, mipmap_level ).rgb * system_background_factor;
Just don't confuse the term "diffuse" with low shininess, though. Until recently I was assuming that shininess=1 meant the same thing as diffuse; but it doesn't: If you look at a sphere of a highly specular material but with the lowest shininess possible, like coated with aluminium powder or something, the brightest spot would still be where the surface normal is the median between your view vector and the light vector; wheras with diffuse color, the brightest spot is where the surface normal is aligned with the light vector.
Totally different animals.

BTW, I was editing my previous post when you posted. See the last bit.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Small change to my proposed shader. The glow map should have a gamma of 0.5. Reason being, in a glow map there's only a few spots where light is maxed out... the lights. The rest is pretty dark; --just the radiosity baking from those very lights. Here's an example of a glow map, and it includes ambient light baking (which my proposed shader eliminates the need for):

Image

But dark also implies inefficient use of the already scant bits per channel.

Gamma of 0.5 is equivalent to taking the square root of the per-channel intensity of each pixel, in normalized 0-1 range. So, full brightness being 1, square root is 1. Half brightness, 0.5, square root is 0.7172. So the texture is brightened in a non-linear way, better utilizing the numerical range. This is done off line (by the LaGrande Noodle) and then the dds is generated.

Once the shader gets an interpolated color for the glow map, for a given pixel, in floating point resolution, it merely needs to square it, to de-gamma. To square the values of rgb, it only needs to dot the incoming rgb vector by itself.

But in my texture packing, the alpha channel of the glow map has the ambient occlusion, and we need to square the ambient occlusion for modulating the specularity, anyways, so, two birds of a shot ;-)

Here's the updated draft:

Code: Select all

//invariants:

float3 computed_ambient_color; //(see Note 1a)
float3 xmesh_diffuse_factor;
float3 xmesh_specular_factor;
float3 system_background_factor; //(see Note 1b)
float3 light; //color of the star //(see Note 1c)
float  xmesh_min_alpha;

//variants:

float3 L; //Light vector
float3 V; //View vector
float3 N; //Normal vector
float3 T; //Tangent vector
float damage;
float cloak;
//and the texture interpolants...

main()
{
  float4 temp;
  float3 diffuse_mtl_color;
  float3 specular_mtl_color;
  float3 emissive_mtl_color;
  float3 albedo_mtl_color;
  float3 mtl_normal; //mtl_normal.x = du; mtl_normal.y = dv; mtl_normal.z = z
  float3 normal;
  float3 R; //Reflected_vector.
  float  mtl_alpha;
  float  mtl_gloss;
  float  mtl_ambient_exposure; //the ao baking value
  float  ambient_diffuse_factor;
  float  ambient_specular_factor;
  float  ambient_contribution;
  float  diffuse_contribution;
  float  specular_contribution;
  float  tempf;
  float  NdotL;

  temp = normal_map_read();
  mtl_normal.x = 0.3333 * (temp.r+temp.g+temp.b) - 0.5;
  mtl_normal.y = temp.a - 0.5;
  mtl_normal.z = 0.25; //(see Note 2)
  mtl_normal.normalize();

/***************start*of*changes***************/

  //vectors and dots (moved up here now)
  normal = wiggle(N by mtl_normal);
  LdotN = dot(L,normal);
  VdotN = dot(V,normal);
  R = 2.0 * VdotN * N - V;
  LdotR = dot(L,R);

  //Glow texture and ambient factors (see Note 4) (also moved up)
  temp = glow_texture_read();
  mtl_ambient_exposure = temp.a; //"ao bake"
  tempf = max( mtl_ambient_exposure, VdotN );
  temp.a = tempf;
  ambient_diffuse_factor = sqrt( mtl_ambient_exposure );
  temp = dot( temp, temp ); //de-gamma
  emissive_mtl_color = temp.rgb;
  ambient_specular_factor = temp.a; //the square of the ao

/*****************end*of*changes***************/

  temp = diffuse_texture_read();
  diffuse_mtl_color = temp.rgb * xmesh_diffuse_factor;
  mtl_alpha = max( temp.a, xmesh_min_alpha ); //.a comes from 1 bit alpha

  temp = specular_texture_read();
  specular_mtl_color = temp.rgb * xmesh_specular_factor;

/***************start*of*changes*2*************/
/* shininess (gloss) should also be encoded with gamma = 0.5 */
  mtl_gloss = temp.a * temp.a * 256.0; //changed!
/*****************end*of*changes*2*************/

  temp = damage_texture_read();
  damage_mtl_color = temp.rgb * xmesh_damage_factor;
  mtl_is_dielectric = temp.rgb; //.a comes from 1 bit alpha

  tempf = 1.0 - damage;
  diffuse_mtl_color  *= tempf;
  specular_mtl_color *= tempf;
  emissive_mtl_color *= tempf;
  mtl_gloss *= tempf;
  diffuse_mtl_color += (damage * damage_mtl_color);

  temp = detail_texture_read();
  diffuse_mtl_color *= temp.rgb; //(see Note 3)
  mtl_gloss *= temp.a;

  albedo_mtl_color = diffuse_mtl_color + specular_mtl_color;
  ambient_contribution = albedo_mtl_color * mtl_ambient_exposure * ambient_light_color;
  ambient_contribution += emissive_mtl_color;

  diffuse_mtl_color *= ambient_diffuse_factor;
  specular_mtl_color *= ambient_specular_factor;

  //Fresnel (see Note 5):
  tempf = 1.0 - VdotN; //or, 1 - cosine_of_view_to_normal_angle;
  tempf *= tempf;
  fresnel_reflection_factor = 0.0625 + 0.9375 * tempf;
  fresnel_reflection_factor *= mtl_is_dielectric; //antialiased boolean ;-)
  fresnel_transparency_factor = 1.0 - fresnel_reflection_factor;

  diffuse_mtl_color *= fresnel_transparency_factor; //(see Note 6)
  mtl_alpha *= fresnel_transparency_factor; //(see Note 7)
  specular_mtl_color *= fresnel_transparency_factor; //(see Note 8)
  specular_mtl_color += float3(fresnel_factor,fresnel_factor,fresnel_factor);//(see Note 9)

  diffuse_contribution = diffuse_mtl_color * light * max(LdotN, 0);

  tempf = max( LdotR, 0 );
  speculating_light = light * pow( tempf, gloss );
  mipmap_level = max( 0.0, 7.0 - log2( gloss + 1.0 ) );
  speculating_light += textureCubeLod( envMap, R, mipmap_level ).rgb * system_background_factor;
  specular_contribution = specular_mtl_color * speculating_light;

  temp = float3( ambient_contribution + diffuse_contribution + specular_contribution );
  temp.a = cloak * mtl_alpha;

  color = temp;
}
Probably the code will need to be rearranged to optimize it to separate dependencies, but for now I'm trying to keep it in an easier to understand, logical flow.

EDIT:
Made another change: Shininess ("gloss") should also be encoded with gamma = 0.5, as a change of gloss between 1 and 2 is a lot more
significant than a change between 250 and 251.
See "/***************start*of*changes*2*************/".

EDIT 2:
Klauss, for PRTP and PRTN, I would suggest we can make them half the resolution as the diffuse, specular, etc.; but WITH an alpha channel, and that we use the alpha channel for luma and the rgb for chroma. IOW, we scale rgb to make sure that r^2 + g^2 + b^2 = 1, and put the correction factor in the alpha channel.
White would then be 0.577, 0.577, 0.577, 1.0 (0.577 being sqrt(1/3)). Thus, 1.0 in alpha represents 1.732 (sqrt(3)).
Actually, it's a little more complicated than that, since allowing the alpha channel to go so low as to use less bits than rgb would defeat the purpose; and Blender nodes don't have like "if" statements, only math ops...
So we'd need a continuous function that yields alpha ~= sqrt( 1/3 ) * ( r^2+g^2+b^2) when the latter term is high, but that falls no lower than about 0.2 or something. I'll figure it out.
Heck, maybe just luma = sqrt((r^2+g^2+b^2)/3); then scale r, b and g by sqrt(1/3)/luma.
I can write a Blender noodle to do this encoding with my eyes closed, by now... ;-)

In fact, we can do the prt bakings at 4 or 8 times the resolution and then have LaGrande scale them down --at floating point precision per channel--, and also perhaps scale the rgb numbers it gets for prtn+prtp for a given texel so they add to the same number we got for the ao bake (assuming the ao bake is higher quality), then do this chroma/luma encoding calculation; and only then, finally, quantize back down to 8-bits per channel to save a png.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

The shader is too long to read now, maybe later :p

But the idea about gamma 0.5 is a very good (and standard) one - it's a form of HDR texture. Can't remember the name though.

The standard way to do it is transform the texture with:

Code: Select all

dest.rgb = src.rgb / max(src.r,src.g,src.b)
dest.a = max(src.r,src.g,src.b)
So reading would be:

Code: Select all

tex = texture2d(blah); tex.rgb *= tex.a;
There are other forms which have a larger range (lets you have textures with values above 1.0), but they are quite complex and interpolate poorly. Radiance coding is one such:

Code: Select all

dest.rgb = src.rgb / max(src.r, src.g, src.b);
dest.a = log(1+max(src.r, src.g, src.b));
But since reading would be nonlinear:

Code: Select all

tex = texture2d(blah); tex.rgb *= exp(tex.a)-1;
Then bilinear interpolation does weird things to it.

Also remember that all of those are dependant on shaders. If you want to support shaderless hardware, it's either having alternative versions, decoding on software (a complicated option but an option nonetheless) or using register combiners for the first option (it won't work for the second, and many shaderless hardware nowadays support those - even the old TNT2).

In fact, my plan for the long run was to use a variant of the first encoding (with a scaling factor to allow higher-than-1.0 values) for glowmaps alone in th engine. Furthermore, I'd render glowmaps to a same-format texture target and then postprocess to create HDR effects. I have the project on RenderMonkey working quite nicely, it's really cool.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Klauss, my formula is far superior to any of the standard methods.
Precisely, it avoids non-linearities that could result in poor interpolaltion. The function max(r,g,b) can make alpha jump around: Think of r, g and b changing smoothly like 3 sine waves at 120 degree phases. Then alpha would jump like the unfiltered wave of a 3-phase rectifier. So here you had these smooth, nice little waves rolling at a pleasant 1/5 of the sampling frequency... er... dot pitch, and now you got a folded alpha of triple the frequency to begin with, and with all sorts of subpixel harmonics... Hell! Even r, g and b now have the same high harmonics, since one or another of the three is always going to be crawling up-side-down on the 1.0 ceiling, then suddenly jumping below it, only to go up and hit it again.... I don't think anybody put any serious thought into that encoding method... It's BAD:
Look what happens also at very low light levels with the standard method 1 that you describe: You got r=0.004, g=0.008, b=0.012, so you make a = 0.012, pushing b to the top of the range, and r,g to 0.33 and 0.66; but now a is only using a bit and a half. You could say "so what?, I scale rgb exactly", but interpolation of a will kill you deader than a summer fly crash-landing on Titan during an ammonia volcano eruption event, because at bit and a half levels, having a 2 or 3 as the value of your neigbor's a can bias your a. Your a's tiny value's least distortion will now scale r, g and b like the proverbial tail wagging the dog. You haven't got rid of the rgb precision problem; you've only switched it around to a. And you might say that a has a bit more muscle than rgb put together, in dxt5, and that's true, but if we rely ONLY on a to carry all the weight, we miss out on a lot of potential range... Think about it.

My formula automatically balances the use of bits between r, g, b and a. In fact, half the luma is carried by rgb, and half by a; each being approximately the square root of the incoming luma.

Look at it again, man:

Encoding:
a = sqrt((r^2+g^2+b^2)/3);
temp = sqrt(1/3)/a;
r *= temp;
g *= temp;
b *= temp;


Decoding:
(rgb) *= (sqrt(3)*a);
(NOTE: sqrt(3) is a constant; doesn't have to be computed. :D)

FAR superior.


Having values above one is irrelevant. A representation system has a dynamic range, and that's what matters; not the absolute value.

Whether an image benefits from gamma 0.5 depends on the contents. If you had typical coloric contents all bunched up near the histogram's high end, you might benefit from a gamma of 2.
The reason I want to use gamma 0.5 for glow map and shininess is that shininess is more sensitive at the low end, and that the glow map only has a few lines at the top end of the histogram, then nothing for as far as you can see, then below 0.25 or so you got a jungle of lines again. Gamma .5 sacrifices resolution at the high end, which is okay for ship lights, as we always max them out, anyways; and spreads the low end's compressed jungle into a varied ecosystem :D.

And, yes, this IS shader-only stuff. Your GI is shader-only, remember? This "root luma to alpha" method is what I'm suggesting we use for prtp and prtn; nothing else.

EDIT:
Heck, and it's NOT shader-only stuff. Multiplication by alpha is pretty standard operation in ogl. All you have to do to display this encoding correctly, except for absolute brightness, is alpha-blend it onto black.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

chuck_starchaser wrote:My formula automatically balances the use of bits between r, g, b and a. In fact, half the luma is carried by rgb, and half by a; each being approximately the square root of the incoming luma.
I like that. I like it a lot. Good thinking.
Anyway, I wasn't storing plain max(r,g,b), i was storing its sqrt(). :p
I did say I used a variant ;)
chuck_starchaser wrote:Having values above one is irrelevant. A representation system has a dynamic range, and that's what matters; not the absolute value.
Man, it's not!
A representation that can't represent 2.0 will not represent overwhite, which is crucial for HDR effects. Overwhite will bleed a lot more creating halo effects that hint at its overwhiteness. While the final pixel's color is clamped to 1.0, the bleeding effect remains and is not attainable in any realistic way without a representation that allows for values above 1.0.
chuck_starchaser wrote:Whether an image benefits from gamma 0.5 depends on the contents.
Technically true, but don't forget that DXT5 has a much less damaging compression technique for alpha than for rgb. Hence, making rgb carry chrominance and alpha luminance does improve things almost always. There's a paper on that, you know.
chuck_starchaser wrote:And, yes, this IS shader-only stuff. Your GI is shader-only, remember? This "root luma to alpha" method is what I'm suggesting we use for prtp and prtn; nothing else.
Ok, yep, this is all very good for PRT. I was warning about the dangers of using it for glowmaps, which should be readable by shaderless techniques.
chuck_starchaser wrote: EDIT:
Heck, and it's NOT shader-only stuff. Multiplication by alpha is pretty standard operation in ogl. All you have to do to display this encoding correctly, except for absolute brightness, is alpha-blend it onto black.
Hm... maybe. Hm... playing a bit with blending modes. Maybe - cool.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

What happens, Klauss, is that when I talked about using gamma .5 and when I talked about using alpha for luma, I was talking about different things, for different purposes, for different textures, but you seem to treat these things as being related, somehow.
The biggest irony is that this idea I came up for spreading the luma between rgb and a, it turns out each gets sqrt(luma), which also happens to be gamma .5. But that's pure coincidence, I believe.

Here's a png I made for testing:

Image

And here it is encoded using my method:

Image

I get banding, like concentric rings, not sure why.
Well, I kind of do... But it's a long story, and I'm still thinking about it.
The vertical stripes are 3 waves of r, g and b at 120 degrees. The luma
is not changing horizontally.
I applied then a radial white to black gradient and blended it multiply.
So far so good.
So, the only luma change in the pic is the radial fade to black, from the center. It's not linear; more like sqrt(a tent). Anyhow, rather than alpha going down bit by bit, we get a cooperation between rgb and alpha, and I guess there's no guarantee that they are going to take neat turns going down... Have to think...

Here's the noodle, btw:

http://wcjunction.com/ftp/noodle.png

If you're wondering what those connections beteween the blue resolution texture are doing, the answer is essentially "nothing". What happens is that noodles have to begin from reading a texture. You can't have a math node with no input to it, just a literal 3 and a literal .5 and Power function just sitting there and expect it to give you the square root of 3. At least one input has to come from a texture. So the top left texture I use for setting the resolution. I set red to 0, green to .5 and blue to 1, and then I use the rgb channels when I need one of those values; that's all.

EDIT:
It occurred to me that perhaps the rings were due to the background of the VS board template being dark but not black; but I tested on a black background and the rings are still there...
What an irony... I started off with the idea of sacrificing chroma for higher luma precision, but in the end it seems I'm sacrificing luma for chroma.
On the other hand, Blender nodes work blindly. On a coded app I could explore a few values up rgb and down alpha, and viceversa, to try and find the best match to the incoming luma. Not that necessarily that would translate on a better dds result.
I guess that's the real question, though. Does this work better for a dds? My intuitive answer is it ought to, but we shall see...
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

I think I know how to fix it. Once I compute the gammafied rgb, I need to quantize them, somehow, and then recalculate alpha. But I don't think I can use Blender nodes to quantize...

This is it; much better idea: I think I'll need two separate Blender noodles: The first would be the one I did, no changes. Then I would take the output PNG, make a DDS, then load the DDS, convert it back to a PNG, and feed it, together with the original texture, to a second noodle that recalculates the precise alpha that the DDS-converted RGB values require to match the original's luma most closely; then combines the rgb of the first noodle's output and the new alpha, and writes a third PNG. This last PNG then I dds-compress.

I'll try it tonight.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Ya, good idea, that way you'll cancel quantization effects on the rgb. I'll let you work ;)
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
Post Reply