The Graphics Thread

Development directions, tasks, and features being actively implemented or pursued by the development team.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

I read somewhere that writing S3TC-compressed textures required a license.

I that's actually the case, then we should be really careful with how such a tool is written.

In any case, it shouldn't be overly hard to write such a tool with python & numpy. Numpy is really powerful for this kind of stuff. But it would take quite some time, and adding a GUI like CubeMapGen's even more so.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Hmmm....
Well, the worst that can happen, license-wise, is that we couldn't distribute the tool; right?
It's not like we cannot "write" s3tc textures; we do so when we use gimp or nvcompress.
Anyhow, I'd much prefer to use C++, myself, a language I know; and that we can later
optimize with XMM inline assembly, if we want to.
I like to know when I'm using a float, and when I'm using a double; I don't like at all this
business of automatic typing...
And forget the user interface for now; the work we want to do is batch work, anyways.
(I'd rather code this thing than have to manually go through all the backgrounds again...)
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: The Graphics Thread

Post by safemode »

the algorithm used in s3tc compression requires a license to compress (not decompress). This license is granted to nvidia and the likes (who i assume pay for it) and they grant use of their tool for free to anyone. So it's ok for us to use their tool to compress dds files, but we couldn't for instance, create our own tool that does the same thing.

Now, creating a tool that used one of these tools that has a license to generate the faces can be used to create a cubemap dds file just fine. So long as the algorithm remains in the program that has the license to use it ...

Basically, so long as the s3tc compression algorithm stays in the tools from nvidia or ati, everyone is golden. You can take it's output and create your own cubemaps all you want, you're not using the s3tc compression algorithm to do so. Just organizing it's output into a cubemap instead of a single file or whatever you want to do.

edit: such a program could be redistributable no problem. But you can't distribute the nvidia or ati tools obviously. So it would be like a frontend to these tools. Also note: The gimp plugin does not contain the compression algorithm. It uses the hardware to compress, and the hardware people license s3tc compression.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Good to know; thanks! We can definitely write temporary png's and call nvcompress on them with the nomips option.
Hallelluya! I don't have to write the compressor! :D
Coding begins...

EDIT:
Ehmm... What about reading the data from inside an nvcompressed dds and repackaging it?
Problem is, we need to put them all into a single dds cubemap file with like 60 layers in it.
IOW, the license is for the compression; not for the packaging, right? (I hope...)
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Code: Select all

#ifndef __FILTER_H__
#define __FILTER_H__


class filter
{
    mem_cubemap const & source_;
    mem_cubemap       & target_;
    radians             radius_;
    void init_constants();
    steradians          solid_angle_;
    float               shininess_;
    //etceteras...
public:
    virtual ~filter();
    filter
    (
        mem_cubemap const & source
      , mem_cubemap const & target
      , radians const & radius
    )
    : source_(source)
    , target_(target)
    , radius_(radius)
    {
        init_constants();
    }
    void do_it();
};


#endif

klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

BTW: scipy/numpy lets you know exactly what you're doing. It's used in scientific computing, where you can imagine chuck, knowing the amount of precision you have is paramount.

Code: Select all

>>> import numpy
>>> x = numpy.zeros(10000,'f')
>>> y = numpy.array(range(10000),'f')
>>> for i in xrange(10):
...   x += y * (x + y)
... 
>>> x[0:10]
array([  0.00000000e+00,   1.02300000e+03,   1.18096000e+05,
         3.14572500e+06,   3.90624960e+07,   3.02330880e+08,
         1.69485146e+09,   7.51619277e+09,   2.78942740e+10,
         8.99999990e+10], dtype=float32)
>>> x[9000:9010]
array([ Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf,  Inf], dtype=float32)
>>> 
'f' = float32
'd' = double (float64)
'i' = int32
'l' = long (int64)
etc...

That particular part of numpy is vector math. You have a lot of other things.

For instance, you have universal functions, which are functions that operate (natively) on a wide range of types (plain numbers, arrays, slices, etc...). With universal functions you can apply complex operations to vector data without having to do it step by step, with tons of intermediates.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
charlieg
Elite Mercenary
Elite Mercenary
Posts: 1329
Joined: Thu Mar 27, 2003 11:51 pm
Location: Manchester, UK
Contact:

Re: The Graphics Thread

Post by charlieg »

Just looking at this commit (part of r12709).

Isn't it unlikely for the compiler to be able to know to optimize this loop?

Code: Select all

for (size_t i=0; i < (sizeof(*a) / sizeof(*ia)); ++i) {
...
}
Given the content of the loop - it does not modify anything that affects the loop condition - shouldn't you be doing this?

Code: Select all

size_t limit = (sizeof(*a) / sizeof(*ia));
for (size_t i=0; i < limit; ++i) {
...
}
Only a minor optimization but these things can add up.
Free Gamer - free software games compendium and commentary!
FreeGameDev forum - open source game development community
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

"(sizeof(*a) / sizeof(*ia))" is a compile time constant, the compiler knows it.

Then, the loop is a standard for loop, with known (small, I think 12) loop count, ideal for loop unrolling.

I'll inspect the generated assembly and post it later.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
charlieg
Elite Mercenary
Elite Mercenary
Posts: 1329
Joined: Thu Mar 27, 2003 11:51 pm
Location: Manchester, UK
Contact:

Re: The Graphics Thread

Post by charlieg »

No need to prove anything. You obviously know how it works. ;)
Free Gamer - free software games compendium and commentary!
FreeGameDev forum - open source game development community
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: The Graphics Thread

Post by Deus Siddis »

Has anyone else noticed that in VS, there is not smooth shading across UV seams? Are there any ideas about what causes this?
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Deus Siddis wrote:Has anyone else noticed that in VS, there is not smooth shading across UV seams? Are there any ideas about what causes this?
I'm not sure what you refer to, Deus; a screenshot might help.
But let me say a couple of things in advance, in case they might hit the nail:
There is no way for the graphics to know where in the texture the neighboing texel to a texel on the edge of a UV island might be found, therefore there is no way to implement "across the seams smoothing". That's why the choice of what edges to mark as seams is rather crucial: Preferably edges that are already marked sharp; or that separate different materials; or otherwise edges in dark, hidden places, where the artifacts might be less noticeable.
Additionally, hardware filtering of the texture (trilinear, and specially anisotropic) has no way to know where the seams are, so it filters sometimes a whole track of texels centered on the very edge of a UV island --half inside, half outside--, which is why it is of paramount importance to leave some breathing room between islands, AT LEAST four texels, and all textures should extend the edge color around the islands; but many of the models in-game right now sport exactly zero space between UV islands, and have no edge dilation applied at all.
This is also the reason why, preferably, or, if at all possible, UV islands should be contained within power of two subdivisions of the texture space: Because mip-maps of the textures are reductions by powers of two, and if the islands are spread around kind of randomly, as the texels get bigger they begin to encompass color from more than one UV island. But presently no in-game models' UV unwraps pay any attention to this issue.
Finally, this is also why, when I'm working on a UV unwrap, I try to group my islands by common material, so that to whatever extent there may be inter-island color bleeds (by mip-map reduction, or by real time filtering) the color bled won't be too different from the color it bleeds into. But even grouping islands by material is not enough in terms of bleeding of AO (and PRT) bakes' information; so I try to pay attention to that issue as well. The more I manage to match material AND illumination conditions, the closer I can place islands to each other, and therefore the more efficiently I can use the texture space; but it's really hard to do a UV layout that satisfies so many conditions 100%, so sometimes I end up with a bit of bleed artifacts; but I often notice them and fix them; whereas, if I didn't pay as much attention to these issues, I'd end up with so many artifacts I'd probably give up rather than fix them.
But, in a nutshell, I think the problem you notice is a problem with the art, not with the graphics. There's nothing the graphics pipeline can do about UV islands touching each other, badly laid out, mixing islands of different colors together, badly chosen edges for seams, etceteras. And by the way, this is why I tell people to NOT use automatic unwrappers. Automatic unwrappers are written by programmers who have no personal experience whatsoever of the art pipeline, no knowledge of graphics at all; and whose only criteria, personal goal and source of pride is to optimize texture use % --by any means necessary... In other words, they SUCK.
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: The Graphics Thread

Post by Deus Siddis »

Yeah I've studied your UV tutorial on the wiki and try to abide by most of those rules as much as possible. Unfortunately this issue has nothing to do with textures and their bleeding. UV seams, which should only affect a texture, split vertices/edges to create hard edges in VS even when there is no texture applied at all--

Image

If I take that same model, delete the UV data and import it into VS again that hard edge there will be gone, because there won't be a UV seam there. And again, there is no texture applied to the pictured model.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Hmmm....


Hmmm....


That should AND shouldn't happen... I mean, it should happen a bit (and there's an art-side trick to solve it); but it
shouldn't look THAT bad even when the trick is not used, so... I'm not sure what to say.

BUT, let's talk about tricks:

No. First of all, even before any tricks: How did you do the seam and seam export?
If you have the mesh un-split, just with that edge marked as a seam, and you export to .obj WITH normals, I don't think
that should happen in the first place. At least it hasn't happened to me.
If you export without normals, maybe when mesher computes its own normals, finding the vertexes already split by the
seam, it might think it should compute separate normals; I'm not sure. I always export with normals, so I wouldn't have
experienced your problem.

If that doesn't work, (but it should work), then I'd use a trick: Instead of placing your seam on edges that separate polygons
at an angle, make a new cut that divides many polygons in half, so that the edges on this cut always separates polygons
that are co-planar. This would work no matter what.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

The exported mesh must have normal generation on.

For uv seams to look smooth, since they are seams, and are implemented by splitting vertices as smoothing seams are implemented, the normals have to be computed by blender and exported to the obj file. Then carried onto the bfxm file (some options given to mesher to not compute normals), and then the engine must not compute normals when it loads it either (some options given in the xmesh).

It's quite some work that noone does.

Besides, mesher could be buggy ;) (since now it computes binormals and tangens when computing normals, and those other two still need to be computed, only relative to the exported normal rather than the computed one - not sure that's so).

I remember I did that for some meshes that looked utterly bad... but certainly not all, and I remember commits on those meshes that undid my fixes (suckers).
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

klauss wrote:The exported mesh must have normal generation on.
I never heard of a mesh having "normal generation" on. You mean it has to have normals?
For uv seams to look smooth, since they are seams, and are implemented by splitting vertices as smoothing seams are implemented, the normals have to be computed by blender and exported to the obj file.
That's what I thought. In other words, exporting normals is not an option; it's mandatory.
Then carried onto the bfxm file (some options given to mesher to not compute normals),
You mean that without some special option, mesher will recompute normals, even if the .obj already has them? That is BAD. Recomputing normals should be a special option; not the opposite. So, what IS the option mesher needs?
and then the engine must not compute normals when it loads it either (some options given in the xmesh).
And what option(s) is/are that?
It's quite some work that noone does.
Because nobody knows anything about it. Xmesh and mesher are featuritis monstruosities. Too many options and options, and more options... Why can't we have a fixed way to do things that's simple and it works? Normals should go from .blend to .bfxm without any need for any options. There's no need to recalculate them anywhere.
Besides, mesher could be buggy ;)
Getting rid of 90% of the options would probably make the code simpler and easier to maintain. :D
(since now it computes binormals and tangens when computing normals, and those other two still need to be computed, only relative to the exported normal rather than the computed one - not sure that's so).
Wait! The original code I gave you for tangent and binormal also computed the normal, first of all; and I left it in because I had no idea where to get the normals from, IIRC.
I remember I did that for some meshes that looked utterly bad... but certainly not all, and I remember commits on those meshes that undid my fixes (suckers).
Damn!
So, could you tell me what the exact options for mesher and xmesh are needed for normals to proceed untouched from .blend to .bfxm?
charlieg
Elite Mercenary
Elite Mercenary
Posts: 1329
Joined: Thu Mar 27, 2003 11:51 pm
Location: Manchester, UK
Contact:

Re: The Graphics Thread

Post by charlieg »

Ah, at last, the fabled "James and the giant peach" mod that I have been hoping for all these years!
Deus Siddis wrote:Image
Free Gamer - free software games compendium and commentary!
FreeGameDev forum - open source game development community
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: The Graphics Thread

Post by Deus Siddis »

I export with normals too, and I just tried exporting again with vertex normals, in case I hadn't done that the first time but it makes no difference.

What apparently does make a difference though, I just discovered, is that if you export the mesh from obj directly to bfxm, this problem doesn't happen! It smoothly shades across the seam.

But if you convert from xmesh to bfxm, then you get a hard edge seam, no matter what you do or don't do. It doesn't matter if you modify the xmesh file (say to point to texture files) or don't, if you export every kind of normal from blender or don't, if you have textures applied or not.

Converting from xmesh to bfxm splits vertexes along UV seams, but direct conversion from obj to bfxm does not.


@charlieg
So then you'll definitely want to announce this new project on freegamer!
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

Deus Siddis wrote:Converting from xmesh to bfxm splits vertexes along UV seams, but direct conversion from obj to bfxm does not.
I remember debugging obj->bfxm a lot, it's the conversion that preserves the most indeed. We'll have to equally debug xmesh->bfxm I guess.

@chuck: in light of the above, it may be that preserving exported normals is the default after all, I don't remember after so long. But there is an option to compute normals, and there is a flag in xmesh files to tell the engine to compute them at load time. So, that's probably what's wrong with xmesh->bfxm (or, rather, bfxm->xmesh).
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

klauss wrote:
Deus Siddis wrote:Converting from xmesh to bfxm splits vertexes along UV seams, but direct conversion from obj to bfxm does not.
I remember debugging obj->bfxm a lot, it's the conversion that preserves the most indeed. We'll have to equally debug xmesh->bfxm I guess.
This should be a high priority, --make that a red alert priority. Any way I can help?
@chuck: in light of the above, it may be that preserving exported normals is the default after all, I don't remember after so long. But there is an option to compute normals, and there is a flag in xmesh files to tell the engine to compute them at load time. So, that's probably what's wrong with xmesh->bfxm (or, rather, bfxm->xmesh).
Okay, but so, what flag is that? I don't remember seeing anything about normals in xmesh; I could be wrong; let me see...

Code: Select all

<Mesh scale="1.000000" reverse="0" forcetexture="0" sharevert="0" polygonoffset="0.000000" blend="ONE INVSRCALPHA" technique="fireglass" alphatest="0.000000" >
<Material power="1000000.0" cullface="1" reflect="1" lighting="1" usenormals="1">
	<Ambient Red="0.000000" Green="0.000000" Blue="0.000000" Alpha="0.000000"/>
	<Diffuse Red="0.000000" Green="0.000000" Blue="0.000000" Alpha="0.100000"/>
	<Emissive Red="0.000000" Green="0.000000" Blue="0.000000" Alpha="0.000000"/>
	<Specular Red="1.000000" Green="1.000000" Blue="1.000000" Alpha="0.100000"/>
</Material>
I see nothing about normals.

EDIT:
LOL :lol: :roll:

Code: Select all

...usenormals="1">
Deus, could you check if that went to zero in your conversion?
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: The Graphics Thread

Post by Deus Siddis »

Checked, it never goes to zero, not after any conversion.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: The Graphics Thread

Post by klauss »

@chuck: you can help by testing.

What repeatable procedure looses information and what type of information?

That helps debugging by leaps and bounds.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: The Graphics Thread

Post by chuck_starchaser »

Yeah, Deus, could you post the mesh, and the steps to get no error and error, as well as the test mission file, units.csv data, and whatever else?
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: The Graphics Thread

Post by Deus Siddis »

Attached this particular mesh (one of the asteroid models), though the same problem occurs with every mesh I've tried. I didn't use any other resources, I just renamed the converted mesh (in bfxm format) to "llama" and dumped it in the llama's directory, within the unit directory. I used 0.5.0 version Vega Strike (I'm on windows) but with the new mesher I got from shenle.

Steps to get error: obj > bfxm > xmesh > bfxm

Steps to not get error: obj > bfxm
You do not have the required permissions to view the files attached to this post.
Post Reply