UnitConverter obj->bfxm

A forum for the discussion and development of programs to assist working on or playing with the Vegastrike engine and data sets.
Post Reply
ace123
Lead Network Developer
Lead Network Developer
Posts: 2560
Joined: Sun Jan 12, 2003 9:13 am
Location: Palo Alto CA
Contact:

Post by ace123 »

The upgrade system works quite simply by taking the maximum of each ship stat and of the upgrade stats (of course with many exceptions)

I'm not sure which specific columns would change the thruster stats, but you should be able to open the CSV and make upgrades that only modify some columns.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Well, this is MORE OT, if that's even possible; but what Klauss started working on, at one time, IMO, would be the ticket: A sort of "language" for units.csv. I'm not sure the particulars Klauss was working on, but speaking out of my ass, I'd have a syntax such as,

Code: Select all

= 77
meaning that the upgrade changes the value of this column to 77

Code: Select all

+= 7
meaning that the upgrade increases this value by 7.0

Code: Select all

*= 1.77
meaning that the upgrade increases the value in this column by 77%

Code: Select all

^
to mean a toggling of a boolean value

Code: Select all

++
(optionally... as a shortcut for "+= 1.0")

Etceteras...
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

chuck_starchaser wrote:Well, this is MORE OT, if that's even possible; but what Klauss started working on, at one time, IMO, would be the ticket: A sort of "language" for units.csv. I'm not sure the particulars Klauss was working on, but speaking out of my ass, I'd have a syntax such as,

Code: Select all

= 77
meaning that the upgrade changes the value of this column to 77

Code: Select all

+= 7
meaning that the upgrade increases this value by 7.0

Code: Select all

*= 1.77
meaning that the upgrade increases the value in this column by 77%

Code: Select all

^
to mean a toggling of a boolean value

Code: Select all

++
(optionally... as a shortcut for "+= 1.0")

Etceteras...
Discussions of field-granularity, rather than line granularity modifiers have come up before, and I'm a long-time fan. The current set of line granularity modifiers (add, multiply, and replace) is insufficient and of insufficient granularity.

However, the addition of an additional subfield to every units.csv cell will be a tedious, if not necessarily intellectually challenging undertaking. Moreover, while I think it's a good time to consider the appropriate design of such a changeover, implementation should probably wait until some of the the code that applies the upgrades has been re-coded, if for no other reason than to allow for adaptation to any potential changes that may result from said rewrite.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

I'd be even more cautious than that. I'd say that the use of operators would have to remain optional for all time, and the behavior without them would have to remain unchanged from no matter what it is presently. Otherwise the change would break every mod to kingdom come.
However, where operators ARE used, they could be made to override any special behaviors that might presently be hard-coded.
Needless to say, all fields would have to change to text first, and adaptations be made to the code that reads units.csv, to convert from plain text; but there's no need for special tools to convert that, I don't think; just highlight all cells and change format to text and it's done.
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

chuck_starchaser wrote: Needless to say, all fields would have to change to text first, and adaptations be made to the code that reads units.csv, to convert from plain text; but there's no need for special tools to convert that, I don't think; just highlight all cells and change format to text and it's done.
?
Not sure I understood what you meant there. The units.csv is just text -- perhaps you were speaking of the difficulties of "=" in a spreadsheet?

So, a clean, backwards compatible way to do this is to just have a modifier column associated with every value column.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Possibly what I meant was that we'd have to add apostrophe in front of all cells, globally, and have the code ignore them.
JackS wrote:So, a clean, backwards compatible way to do this is to just have a modifier column associated with every value column.
OUCH; I hope not; units.csv is wider than my mind can fathom as it is.

But, what kindof backwards compatibility are you thinking about? I'm thinking about newer code being compatible with (able to read) older data; not the other way around. That is, modders wouldn't try to use *= syntax with obsolete engines, would they? What we want is that if, say, Mamiya Otaru shows up and wants to update PR's engine, the new engine will be able to read and understand his ancient units.csv. And all we need to do to ensure this is that if a cell has a numeric value without special operators then it is processed the way it is, and always has been, processed.
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

Looking forward toward tools such as the one pyramid has been building, I'm not overly concerned about the length of the line descriptor.

I was thinking of new engines reading old data - the units.csv code is all built around assumptions of default values. That's why we can have vast swaths of blank cells - there's nothing preventing us from having blank columns as well. It's actually a convenient organization to make certain we can support blank columns seamlessly, as we're definitely going to be adding additional stats here and there, so we might as well put the modifiers in new columns, vs. rewrite the parsing for existing ones.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Well, it seems to me you're saying that it's better to have complicated data than to have complicated code.
To me, the reverse is true, but I guess it's a matter of taste. :)
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

chuck_starchaser wrote:I guess it's a matter of taste.
Maybe more than taste it's a matter of requirements, or in real-world, the trade-off between various requirements, i.e. engine speed versus developer / modder / artist convenience.
In this particular case we are talking about more code/time required during the spawning of new ships (when reading stats from units.csv) and not per every frame. Personally, I'd rather advocate user convenience.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

UnitConverter v0.28 is out in the svn. To check out, please see my signature.

New features:
* supports adding models from bfxm. When adding a new model just point to either bfxm or obj file. It will convert a bfxm to obj and then you can continue the regular workflow. The textures used in the bfxm file will not yet be loaded (this is the next step).
* provides configurable units dir behavior: either 'units/Shipname' or 'units/vessels/Shipname'. As such it supports VS and Mods equally.

Manual/Documentation is still outstanding until the wiki problems have been solved. Instructions contained in the tool are updated though.

When testing the sounds column of units.csv it came to my attention that the sound file references stored there are not audible in the game. Is this enabled in the engine?
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

pyramid wrote:UnitConverter v0.28 is out in the svn. To check out, please see my signature.

New features:
* supports adding models from bfxm. When adding a new model just point to either bfxm or obj file. It will convert a bfxm to obj and then you can continue the regular workflow. The textures used in the bfxm file will not yet be loaded (this is the next step).
* provides configurable units dir behavior: either 'units/Shipname' or 'units/vessels/Shipname'. As such it supports VS and Mods equally.

Manual/Documentation is still outstanding until the wiki problems have been solved. Instructions contained in the tool are updated though.

When testing the sounds column of units.csv it came to my attention that the sound file references stored there are not audible in the game. Is this enabled in the engine?
Excellent! I'll be trying it again in a couple of days.
I'm still redoing the Hammer bakes.
This last one has been going since Friday. 16,000 rays per texel ambient occlusion (1024 rays, 4x antialiasing); 50% done...
Might sound excessive, but this is the Uniform AO, which I use to adjust the PRT's by, among other things.
Needs to be perfect.
xNormal is working very well now, after Santiago's last fixes.
Nate (CoffeBot) is working on retexturing the Tarsus for CineMut, and he's contributing Blender scripts to automate
some of the tasks, as he goes along. It's going to be a pretty sleek tool-chain... :D
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Sounds good. Just hope that there'll be a preview option for baking available at a later point in time. No point waiting 3 days to discover your mesh had a non-welded vertex. :wink:
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

There's no "preview option", per se, but you can set your parameters for a quick and dirty first job. I've already gone through many cycles of fixing stuff and this is final, final, final now. By the way, xNormal also has a CUDA option that has the ray collisions computed by the GPU. Only works with NVidia GPU's, and you need to install a beta driver from NVidia. I was fearful with messing with the driver, so I didn't do it. Supposedly it's more than 10 times faster that way.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

I'm running nVidia beta driver (linux) and so far had no problems at all. Too bad xNormal is for windows only. Setting lower parameters serves well for testing purposes. Looking forward to the first VS ship with CineMut support!

UnitConverter version 0.29 is out. It's mainly a bugfix for scaled helper objects. Previously, for some strange reason, I have set the center-up distance to ~2 (oh woe is me, the original helper object size) and of course you might want to scale the object because docking port size and now engine thruster diameter is read from the helper object. This means that every thruster can have a different diameter if required. See Entourage and Vigilance update in the latest svn.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

I'll try to come up with a colorful helper object tonight. We should produce a screenshot of a model in Blender with all the helper objects installed, to give a general idea.
I'm also going to have a pilot sub-unit to donate to Vegastrike, in case a decision is ever made to go with glass cockpits.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Unit Converter v0.30 is out with two main features and some improvements:
* Workspaces are now saved to separate config files for each workspace. This means you can work on a unit, then switch to another one and come back to the first one with all your settings remembered.
* There are two new config options. a) you can decide if you want to back up mtl and xmesh files before writing new ones, and b) you can disable/enable the creation of xmesh files by default.
* I have added an application icon, but tkinter seems to have trouble with that on linux. Until I know of a solution it will remain on windows only.
* There is a bug fix for shield mesh textures naming.
* Mount meshes will not be converted to bfxm and not copied to the data directory.
* Internally there are some improvements in configuration (ini) read/write.

My signature has a direct link to the UnitConverter.zip download.
That's for today.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Another good feature, when you have the time, would be a checkmark for each texture to enable/disable compression; --ON by default. PRT(s?) will need full color precision; --no dds; but could be packaged as .jpg's; probably will be; so if you check-off compression, UC could just rename the .jpg to .texture without dds'ing it.
By the same token, perhaps better names for the columns would be "input texture" and "output texture", rather than "master/compressed".
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Having PRT texture(s) means that you will require additional fields in UC, right? Would it not be better to have a global configuration option to enable/disable compression of PRTs or just disable it by default altogether without configuration?

The changed column naming will be available with the next version.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Indeed. Well, CineMut is not completely settled yet. Haven't decided yet whether to use 2 PRT's or a unified one. And it might evolve to having self-radiosity and self-reflection. For the latter I will need a way to bake a self-bake, and my only hope of that is if Santiago Orgaz, of xNormal fame, decides to add the feature to xNormal; but I haven't been too successful so far with my suggestion. And it only works in theory; I'd need one such bake to test the concept. So far, the CineMut texture pack looks like,

Texture 1:
Albedo (sum of diffuse and specular colors)

Texture 2:
Red represents the specular to diffuse balance
Green represents the dielectric coverage or blend ratio
Blue is the material's dielectric constant
Alpha is the shininess

Texture 3:
Red and Blue are damage dU/dV normalmap modifiers
Green is for sub-texel detail control
Alpha is damage obscurity

Texture 4:
RGB = Glow
Alpha = Cosine distribution ambient occlusion

Texture 5:
RGB = PRT (for now; it may be two textures later)
Alpha = Uniform distribution ambient occlusion

Texture 6:
RGB = Normalmap dU
Alpha = Normalmap dV

Texture 7:
To possibly come; will have "self-bake" for radiosity and self-reflections.

So, right now it's 6 textures; but it could grow to 7 if either I end up using 2 PRT's, OR using self-bake; OR even 8 textures if both.
Right now it's at 6, tho.

I guess you could make the PRT's not be compressed in code; but it would be better to have a compress flag; tell you why:
1) During texture development you might want to use un-compressed textures, to better see the results of changes you do, like varying the visibility of dirt streaks of scratches. You know that eventually, with compression, only a hint of them will be visible; but for initial tuning of individual effects you might not want to be looking at the ship through a distorting lense of texture compression.
2) IF/When the VS engine comes to support 3D bases, we'll probably not care for compression so much as for quality, as when you're at a base you don't need to worry about having 25 kinds of units loaded in the graphics card memory at the same time; so you have plenty of memory available.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Re: texture compression option.
That sounds very feasible that an artist might want to test the model with all uncompressed textures. I guess then a global option for enabling/disabling compression of all textures 1-6(or7) (except PRT) is the best way to go for now. Unless I misunderstood and you think it's of utmost importance to have the flag per texture.

Re: number of textures
Is mesher prepared to take more than 5 textures? I haven't found reference to encode beyond map_4. Neither looked into the code if we can simply continue with the count (map_5, map_6, ...)

*EDIT*
Actually tested with 9 textures and it seems we just need to continue counting :D. Great. The road is open for more textures.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Yeah, that sounds good. Maybe a checkbox like
[./] Bypass compression (ONLY FOR TESTING; MUST BE OFF FOR FINAL TEXTURES)
NOTE: Textures 6 onwards never compressed.

Made a mistake in the previous post; texture 5 is normalmap; texture 6 is PRT.

EDIT:
Actually, first texture is texture 0, isn't it?

Allright, take 2:

I tried to list possible future scenarios.
Scenario 1 is the current one: Unified, uncompressed PRT.
Scenario 2 would be using separate PRT's with luma in alpha; in which case they could be compressed.
Scenario 3 is having radiosity (using an UN-compressed "self-bake" texture)
Scenario 2+3 would be if we use separate PRT's AND self-bake.
If it's too complicated, just ignore it all. Current status quo is scenario 1.


Texture 0:
Albedo (sum of diffuse and specular colors)

Texture 1:
Red represents the specular to diffuse balance
Green represents the dielectric coverage or blend ratio
Blue is the material's dielectric constant
Alpha is the shininess

Texture 2:
Red and Blue are damage dU/dV normalmap modifiers
Green is for sub-texel detail control
Alpha is damage obscurity

Texture 3:
RGB = Glow
Alpha = Cosine distribution ambient occlusion

Texture 4:
RGB = Normalmap dU
Alpha = Normalmap dV

Texture 5:
Scenario 1: (STATUS QUO; would be UN-compressed)
RGB = PRT
Alpha = Uniform distribution ambient occlusion
Scenario 2: (would be compressed)
RGBA = PRTP (alpha used for luma; would be compressed in this case)

Texture 6:
Scenario 1:
Not used.
Scenario 2: (would be compressed)
RGBA = PRTN (alpha used for luma; would be compressed in this case)
Scenario 3: (would be UN-compressed)
To possibly come; will have "self-bake" for radiosity and self-reflections. (would be UN-compressed)

Texture 7:
Scenarios 2 + 3: (--i.e. if we have separate PRT's *AND* radiosity)
To possibly come; will have "self-bake" for radiosity and self-reflections. (would be UN-compressed)
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

As there are still some unknowns, I have selected scenario 1 for the latest UC. As PRT link will be now integrated into the bfxm, it should be removed from the techniques to avoid inconsistencies.

Unit Converter v0.31 comes mainly with support of the features discussed recently:
* changed texture headers to 'input/output textures'
* added support for texture 5 (PRT)
* no compression of PRT texture 5 ever
* get units.csv column entries based on column names (instead of fixed column positions)
* reads texture info from mtl file after conversion from bfxm file
* new config option for using input textures in bfxm (instead of compressed textures)
* minor visual reorganization (config options, textures)

There are also some minor internal fixes and improvements which can be looked up in the included release notes.

I have also added the ruler.blend file to the UC directory in svn.

Re: texture 5 (PRT)
I hope that is the intended behavior. In xmesh notation, the following output will be produced (here with texture compression enabled). Note texture5 remains uncompressed:

Code: Select all

texture="tex_000.texture"  texture1="tex_001.texture"  texture2="tex_002.texture"  texture3="tex_003.texture"  texture4="tex_004.texture"  texture5="moto_PRT.png"
Or in mtl notation:

Code: Select all

map_0 tex_000.texture
map_1 tex_001.texture
map_2 tex_002.texture
map_3 tex_003.texture
map_4 tex_004.texture
map_5 moto_PRT.png
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Looks perfect to me. 8) That's a TON of work you put in. :shock:
The ruler ship is about 90% unwrapped.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Glad to hear that. Indeed, yesterday was a productive day. I'll update the finished ruler ship in svn then.

What are you intending to do with the following line in cinemut_opaque.technique:

Code: Select all

<texture_unit src="file:moto_PRT.png" name="prtMap"/>
Can it be simply deleted once we can now put the PRT into the unit itself? Or are there other implications that are escaping me?
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Unit Converter v0.32 has proper support for integration of installations (bases):
* For new installations that are not yet in units.csv a default entry will be created on top of which you can then edit the different parameters.
* Testing of installations is now done automatically as a base via automatic editing of the Modelview.system and provided you set the "Unit type" as "Installation"

A preliminary module for editing system files is included in this version. Though it seems that the system files are not completely xml conform, it seems to work well with the Modelview.system, which is sufficient for this purpose.

Other minor handling changes and bug fixes are included.
The updated ruler ship (my favorite vessel :wink: ) was also committed for those who may see use in it (for I will test it as a space piano :idea:).
Post Reply