UnitConverter obj->bfxm

[pyramid]

UnitConverter
is an asset pipeline tool to convert models (exported from blender) from obj to VS bfxm format.

UnitConverter for Python 2 Source on GitHub
https://github.com/vegastrike/Tools/tre ... tConverter

UnitConverter for Python 3 Source on GitHub
https://github.com/vegastrike/Tools/tre ... Converter3

UnitConverter Manual on Vega-Strike Wiki
https://wiki.vega-strike.org/Developmen ... tConverter

UnitConverter Manual PDF with UI Screen Shots
https://github.com/vegastrike/Tools/blo ... Manual.pdf

UnitConverter on GitHub Wiki
https://github.com/vegastrike/Vega-Stri ... tconverter

UnitConverter Discussion Thread on Vega-Strike Forums
https://forums.vega-strike.org/viewtopi ... 20&t=11885


--- updated by pyramid on 2020-05-07 ---

* updated original post urls to new servers and code repository

--- original post by pyramid on 2008-08-09 01:09 ---

The purpose of this tool is to make artists' life easier when converting unit model wavefront obj files to vegastrike bfxm format and assist with compression of textures to dds format.

To obtain this tool, checkout the modtools/UnitConverter directory from the VS svn repository.

The tool is entirely written in python with tkinter (a library lacking a lot of required functionality) as the standard GUI library for VS support tools, which is integrated in both, the standard python distribution, and the python version included with VS. This means that one should be able to run the app without any additional python installation (though I'm not sure how).

Three main "screens" are selectable via the buttons on the left:

1) Model
Here you can specify the path to the obj and the mtl files. The directory of the obj file will become the working directory, means master textures need to reside there

2) Materials
For each material in the obj file you can specify master (input) textures. Then you can save a correct mtl file and convert the obj file to bfxm. Further there is a button that makes the texture compression for you. All you need to do is to copy the bfxm and texture files to the data/units directory of your vessel.

3) Config
Specify the paths for mesher and nvcompress. If any of the executables is in your system PATH you may just specify the name without the path.

Caution: This preview version 0.03 has been used to convert the Xuanzong model and hence, for the time being, is guaranteed to work *only* with this model obj & mtl files, *only* for diffuse maps, *only* with the current workflow, *only* up to the point the current mesher works, and *only* on my linux system.

The reason I am sharing a basically untested version is that in a couple of hours I'll be going on vacation.

* Comments and functionality suggestions are welcome (read to do in unitconverter.py first)
* Bug reports are not welcome (there'll be too many of them with this early version)

[loki1950]

Quick work pyramid Have fun on that vacation

I will be trying out your new beastie and hold onto any bugs till your return.


Enjoy the Choice

[chuck_starchaser]

Damn quick!
I seem to have the wrong version of Python; will have to dig into it.
In any case, we'll need to address the mesher problem of requiring one material per mesh, as having to reorganize materials at each export is a killer; it's right in the tweak/test loop, which is the most repeated loop in the entire modeling+texturing pipeline; though I guess this should be good enough for artists that work on textures exclusively by hand, rather than using bakes.
Have a great vacation!

[klauss]

I believe if we modify mesher to not crash if the mtl file is missing there will be no need to reorganize materials on export. Rather, you'd export each submesh to a different obj and tell the UnitConverter what textures to assign to each.

The unit converter will have to either create a temporary mtl file, pass in commandline arguments for setting the textures (will have to be implemented), or edit the xmesh. Either option is easy enough with python, so it should be a breeze.

I will have to write an obj->xmesh module for mesher though, since it would streamline the process a bit:

UnitConverter does obj->xmesh, parses the xmesh to gather info and show it to the user, for feedback and checking, lets the user edit the xmesh by setting things as textures, LODs and stuff like that (notice that it's not text editing, you just press buttons to assign LODs and textures), and then the UnitConverter uses the edited xmesh to build the final bfxm.

I might edit the unitconverter.py a bit myself to test some of it

[chuck_starchaser]

That's great! Sounds like a plan.

[ace123]

Awesome.
The mod tools should never crash!

We should allow the unit converter to start an instance of VS for testing purposes.

I would be interested in making the UnitConverter spit out a working units.csv as well so you can immediately test it in the engine (as well as place weapon mounts)

[pyramid]

All good things end. I'm back.

What klauss and ace123 are proposing is the way to go. For now I am contemplating the automation and simplification of textures to material assignment.

Taking the unit test model (motorcycle.blend) the blender export produces 2 obj files with the associated mtl files:
* test_bike.obj + mtl
* test_lights.obj + mtl

Editing the first file gives me a list of 13 materials:
* Chrome_tex_00001
* Gold_tex_00001
* BlackAnodize_tex_00001
* Rubber_tex_00001
* Bronze_tex_00001
* Stainless_tex_00001
* Leather_tex_00001
* Magnesium_tex_00001
* Zinc_tex_00001
* BluePlastic_tex_00001
* Steel_tex_00001
* CastAluminium_tex_00001
* BluePaint_tex_00001

Naturally, those materials will be represented by different areas overlapped in the 5 output textures.

In other cases (like the Xuanzong) there will be 2 submeshes joined in one obj + mtl files for which the user would want to assign different set of textures. This leads me to the conclusion that the interface for texture<->material assignment needs to be more flexible. My proposal is the following.

Provide a list of materials for each obj file from which the user can select one, multiple, or all materials, and for this selection assign the desired textures. Then select other materials and assign another set of textures. The selections and the corresponding textures will be saved in the configuration file so that the user is not required to finish all the work in one session and can continue after the instancing the program anew.

A version picturing the proposal will follow.

I have a lot of CineMut talk to follow up so not sure how the PRTn/p stuff fits into the UnitConverter.

[loki1950]

Welcome back pyramid we have had a bite for a maintainer for mesher http://vegastrike.sourceforge.net/forum ... hp?t=11872 so you might want to contact him as i imagine that you will be working closely in the future.

Enjoy the Choice

[chuck_starchaser]

All good things end. I'm back.

Welcome back!

Provide a list of materials for each obj file from which the user can select one, multiple, or all materials, and for this selection assign the desired textures. Then select other materials and assign another set of textures. The selections and the corresponding textures will be saved in the configuration file so that the user is not required to finish all the work in one session and can continue after the instancing the program anew.

Can't say I understand, but it sounds complicated. There's a much simpler way to go about it, which is also more intuitive: As a modeler, I can export one .obj for each mesh (and set of textures) I want in the final unit. Meshes are simply not related to materials at all. Just as I can use multiple materials in one mesh, I can use a material on several meshes. There's no relation at all.
Now, if what you're trying to do is interface with the version of mesher that assumes that material means mesh; that's going to be a mess. And I think Klauss already has a mesher version that doesn't; so better touch bases with him.

[pyramid]

Now, if what you're trying to do is interface with the version of mesher that assumes that material means mesh; that's going to be a mess. And I think Klauss already has a mesher version that doesn't; so better touch bases with him.

What I'm essentially trying to accomplish is an interface for all possible incarnations of model / textures.
So far I have come around this possibilities:
* 1-1 relationship: 1 obj file > 2 submeshes > 2 materials > 2 texture sets
* n-1 relationship: 1 obj file > 1 submesh > 13 materials > 1 texture set
What you are saying is that there is potentially an n-n relationship between obj files and texture sets, like:
* n obj files > n submeshes > n materials > n texture sets
This situation would make the interface much more complicated but I see what you mean by looking on the motorcycle test model. You have unwrapped the complete model into two texture sets (bike + lights). This gives me an n-n relationship. Still, I do not see the need to export each material into a separated mesh. In the motorcycle test model there are 13 materials (!) assigned to the entire bike mesh which supposedly represent one texture set.

I think before continuing hacking my program it is required to agree on the spec. I too new to this conversion stuff to be able understand the relationships in their entirety.

My current hierarchy relationship assumption is the following:

Code: Select all

blender file
|- submesh 1 = obj file 1
...|---material 1 = texture set 1
...|---material 2 = texture set 1
...|---material 3 = texture set 2
...|---material 4 = texture set 2
|- submesh 2 = obj file 2
...|---material 1 = texture set 1
...|---material 2 = texture set 1
...|---material 5 = texture set 3
...|---material 6 = texture set 3

How does the LOD fit into here? Is it associated with a submesh and has the same materials as the submesh but another texture set or is it just another submesh with its own materials?

[chuck_starchaser]

What I've been proposing is super-simple and natural. You're looking at the dark side of it, seeing it as an n-n relationship. The bright side of it is, it is NO relationship. Just ignore materials.

[chuck_starchaser]

What I've been proposing is super-simple and natural. You're looking at the dark side of it, seeing it as an n-n relationship. The bright side of it is, it is NO relationship. Just a matter of ignoring materials. When I set up materials, I'm talking to Blender; not to mesher. When I want to talk to mesher, I export separate obj's.

LOD's... I've never used them yet, but I think they work by naming convention, like 1_0, 2_0, 3_0 are top LoD of submeshes 1, 2 and 3; and 1_1 and 1_2 are the second and third LoD of submesh 1; or something like that.

[pyramid]

OK, what you are saying is that 1 obj shall correspond to 1 texture set. This means means also that we must assume that obj that require 2 different texture sets are wrongly exported, right?
I can live with that and this makes things much easier.

In the case of LODs I will assume that LOD meshes may have a texture set of their own, but also my inherit the texture set from the main mesh.

[chuck_starchaser]

You got it.

My understanding of LOD's is that currently they inherit the same textures as the top LOD by default; but good modeling discipline will be to specify smaller versions of the textures for them, just like mipmaps. In fact, this is sort of redundant with mipmaps, but it will reduce memory footprint enormously, as most ship models in a system are usually beyond visual range, and so loading big textures for them is very wasteful. Out of the 5 textures, the only one that is NOT simply reduced for lesser LOD's is the normalmap. Lower LOD normalmaps have to be baked individually.

EDIT:
I remember suggesting that reduced versions of the textures would be generated by default, if not provided; but I can't remember now why there was opposition to the idea.

BTW, you were asking about PRT's. They change nothing to the organization of things; just that instead of having just a glow map with the AO in alpha, we'll have one or two PRT textures. This is not decided yet, kind of hanging in the air until Klauss can resolve his videocard situation and resume work on GI and cubemaps.

[pyramid]

Great. For now I will leave the PRTs aside. With this I envision the following program workflow for the conversion:
* set model name
* indicate submesh(es) obj file(s)
* for each submesh set master texture files
* save and convert

[chuck_starchaser]

There's also the big question Klauss was posing to me by email: Should we have per-sub-mesh LoD's?
I said to him, at the time, that perhaps a good idea would be to reserve per-sub-LoD to sub-units,
but I'm not so sure anymore...
You'd better touch bases with Klauss. I'll send him an email pointing him to this thread.

[klauss]

I wouldn't worry with per-submesh LoD. I believe there's a way to achieve that effect with multiple bfxms and some units.csv editing without incurring the overhead of having subunits (subunits imply an extra AI and are really undesirable just for lodding).

[chuck_starchaser]

Alright, I bite ; what's the idea?

[klauss]

I don't know if it's implemented (perhaps what I saw is part of the "animated meshes" feature), but units can have multiple meshes. Each mesh has its own LOD, and is applied independently, so... there you have it.
All I have to figure out if that's only for animations and if so if it can be reused and adapted to work for lightweight subunits.

[chuck_starchaser]

Cool. How will you place them around? There's no location/orientation numbers for bfxm's, are there?, or will there be?
One set of prt's will work for any orientation?
I hope this gets implemented soon. When I redo the Clydesdale I'm hoping to be able to use and abuse sub-...whatevers
And, what ever happened to "instancing"? Maybe we could use it for submeshes that are too small to LoD but very numerous?

[pyramid]

For now I'll make it so that one LoD mesh represents all submeshes.

[chuck_starchaser]

I've no idea what you mean, but, AFAIK, bfxm allows per-sub-mesh LOD, and you use names like 1_0.obj and 3_2.obj to specify sub-mesh and LOD. I think the first number is submesh and the second LoD. The simplest course would be for the converter to reflect the current mesher functionality, I would suppose; minus the materials.

[Deus Siddis]

All I have to figure out if that's only for animations and if so if it can be reused and adapted to work for lightweight subunits.

VS supports 3D animation? Keyframed?

[pyramid]

UC version 0.04 is submitted to svn. It is tested in linux and preliminary tested on windows. It supports the workflow with a main mesh and a texture set. The new version was tested with the Tridacna vessel which comes with on ly one main mesh and diffuse, specular, glow, and normal textures.

[chuck_starchaser]

Cool!
What's the svn url?