tools to fascilitate data file editing

[safemode]

We have things like units.csv, which i dont think is xml, and other files related to systems that are. Do we have tools to edit these in a graphical way? So adding units or editing can be done point and click easy, with bounds of legal values enforced right at the change point.

Think of a system editor that can show you where your planets etc will be positioned in 3d. Perhaps a GL app that opens up with select empty system or use a previously created one form the data4.x dir. Then is gives you a pane that is like gimp's only it has the various things you can have in a system, with the data4.x textures in use. You can drag and drop planets and moons and such in 3d space. Orbital paths would be automatically corrected so nothing collides over time. You can zoom around and work on your system and see how it will look in-game. Saving it generates the appropriate xml file. etc etc

I'd be interested in starting work on that via gtk2 and/or GL apps.

I was thinking of starting out with a units.csv editor with error handling. Perhaps even giving support for loading up of the artwork to help users distinguish what unit they're editing (reading the hud image).


The OpenGL system creator and editor would be really cool

Another editor would, and likely require proper xml-ification of data files in VS, would be a new config editor in-game.

python auto-generation via point and click i think is beyond the scope of VS. But if someone can think of a way of having a utility read in a campaign and editing it in some way, that would be cool.

[bgaskey]

I think you're right that the first necessity is a units.csv and maybe master_parts_list.csv editor to make the job manageable. I use a standard text editor in linux because its too slow to load etc in openoffice. I would still advocate splitting units.csv up. Its just too enormous, and some of the unit groups dont need many of the stats listed, leading to more confusion.

[safemode]

well, the editor would hide all un-used attributes.

It would basically be a point and click interface, looking nothing like how the flat text file looks.


The interface may be a list of units. You could double click a unit and bring up a pop-up window displaying it's attributes. You can add attributes via a pull-down or edit pre-existing attributes. The values you can enter can be error-checked and we could have intelligent suggestions / etc. Values can be auto-filled in or warned if some necessary info is missing.

Same deal for master parts list.

You'll have no idea what the actual files that you're editing / saving to even look like.

[bgaskey]

That would be great. With an editor like that bundled with svn, i guess there wouldn't be a need to break the files up.

My guess is it would be fairly easy to code as opposed to...say...refactoring vegastrike

And its a good start to the content creation toolkit being discussed right now in the feature forum.

[safemode]

refactoring VS is still necessary though. That's necessary for multi-threading VS in any meaningful way. There's no reason why we shouldn't be able to thread graphics rendering of the current frame while we set up the next. Basically right now, there is no way for VS to use 100% of the processing power of my computer, no matter what quality setting i use, and that's not the fault of not having enough to do. I'm GPU limited, and there is nothing i can do about it right now. With refactoring will come threading which means, the gpu wont hold up the rest of the processing.

Hopefully.


But yes, doing the editors for the csv files is much easier, and really has nothing to do with the refactoring work, so it shouldn't need to be redone anytime soon.

[bgaskey]

Not saying that refactoring isn't necessary. With multi core processors becoming common, apps need to support utilizing more than one core. Just saying that this will be a much more modest undertaking. That was just a random example to make a point.

Happy hacking

[klauss]

Just a note of something I recently noticed at work (a big lesson of life... mutltithreaded life), and thought I'd mention in case someone interested listens and ends up learning something very very useful:

With Hyper-threading, you can't easily get 200% power out of two threads. Because there's no guarantee the two threads will end on different processors, they may end on the same processor (only a different thread within that processor). In fact, if you do multithreading oblivious to HT you seldom get anything more than a 10% increase in thoughput from HT.

So, in order to make proper use of available power, one should pair threads using markedly different units of the processor. Since it's hard to control whether a thread will be using the float or the int unit (or which int unit), since many instructions execute on weird units (ie: multiplications usually execute on the fp unit), there's basically one single such choice of task-division at the developers' grasp: memory vs computing.

If you create threads in pairs, one that consumes a lot of CPU, ie: it is CPU bound, and one that handles massive amounts of memory, ie it is memory bound, then you'll get close to 200% thoughput on a single-core HT chip!

Ain't that cool?

You could then make a thread compute stuff on big huge arrays (ie: iterate physics), so that locality of reference is high and the thread becomes CPU bound, and then have another thread traverse tree-like structures (say octrees or whatever - ie: collision detection), do AI stuff, and the like. Since those structures are spread over memory, working with them tends to make them memory-bound. The memory thread will run at 100% speed because it will always be waiting for some data item to arrive, and the CPU thread won't interfere much because it will be hard at work computing stuff in registers.

I've tried it, it works. The trick is managing to divide tasks in such a way.

BTW: The engine in VS is currently memory bound IIRC. Any refactoring will require data structure changes to veer away from such an issue. It will be big.