units.csv editing...

A forum for the discussion and development of programs to assist working on or playing with the Vegastrike engine and data sets.
Post Reply
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

units.csv editing...

Post by JasonStiletto »

working on a units.csv editor, largely in a personal effort to make it easier to update master parts list vs base inventory loadouts, but, I'm working toward getting it to handle every field in the file.

screenshot1
screenshot2
screenshot3
screenshot4
screenshot5

If anyone is interested, I would be happy to throw the app out GPLed when I've got the code closer to where I want it. At the moment it's linux only, but Windows wouldn't even be a port, just a re-compile. ^^
Oblivion
Artisan Extraordinaire
Artisan Extraordinaire
Posts: 1269
Joined: Tue Mar 21, 2006 10:55 am
Location: Philippines

Post by Oblivion »

Wow :D :shock:

This would indeed simplify a lot of things.

Please please please include a pre-compiled windows exe for us evil microsoft users. lol :lol:

Can't wait for it to be finsihed... hehe
A Step Into Oblivion

Dreams of things that will never be,
Songs of thoughts only I can hear,
Leave me be to sleep forever,
To dream my dreams,
And sing my hymns,
Of things that will never be...
Zeog
ISO Party Member
ISO Party Member
Posts: 453
Joined: Fri Jun 03, 2005 10:30 am
Location: Europe

Post by Zeog »

We are in desperate need of such a thing!
Does it feature predefined collections of carried cargo, which can then be bought at the location, i.e. a quick way of entering lists as in screeshot5?
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

We are extremely appreciative of such efforts. The VS internal toolchain is ... let me be polite: underwhelming. One of the key reasons for this is that we're a bit low on people accustomed to working with GUIs (also, the first graphical tool that HellcatV made for VS way-back-when was not well received and very time consuming to create and maintain, so that rather soured him on prioritizing further such ventures - moreover, as understaffed as we are, it's a difficult call to pull someone off of doing engine work to go work on tools). Since the decision to move to OGRE, there's been an additional incentive to wait before going forward with more tool-work because of existing GUI packages that interface with OGRE, but, honestly, though there will probably be a great increase in tool attention after the OGRE release, we could really use more tools _now_ to accelerate getting _this_ release out :)

So, again, thank you. We can really use more folks like you.
Vector_Matt
Merchant
Merchant
Posts: 39
Joined: Fri Oct 06, 2006 1:43 pm
Location: Blasting Pirates, ludites, etc...

Post by Vector_Matt »

Oblivion wrote:Please please please include a pre-compiled windows exe for us evil microsoft users. lol :lol:
We, compiler impaired windows users, would indeed appriciate it greatly. :D

Great job, looks easy to use and sleek. :mrgreen:
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

Alright, I'll try to get a first release out within a week, windows and linux binaries, plus source. Database design and user interaction with said database has been my main job for the last 10 years, so I think it should be clean enough and easy to use. Will take feature requests when I drop the beta.

It doesn't at the moment feature predefined collections of cargo, but I'm working on building out cargo from incomplete tree nodes, and for something like you mentioned I could probably implement the feature in a day or so. In the languages I'm competent in, I'm a reasonably fast coder.

As I said, once I release the beta I'll take feature requests. If I can implement it (probably) and It doesn't make the program needlessly complicated (doing too much can make something unusable as easily as doing too little) I will implement it.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Sooo very cool.
There's already a CSV editor, but it's quite generic and not so user-friendly.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

added physics tab

TODO New Tabs: Radar(50%), Functionality, Extras

TODO Subqueries: Upgrades, Prohibited_Upgrades, Sub_Units, Sound, Light, Net_Comm, Dock, Cargo_Import (90%), Cargo, Explosion
TODO - validate record modification always takes place
TODO - Create new record
TODO - Create new record for subqueries
TODO - validate save feature
TODO - make data path picker work - no longer rely solely on heuristics.
TODO - validate all current fields are accessable
TODO - validate write still functions after field changes, read functions when fields missing
TODO - read master_part_list.csv

The fields in this table are endless... I had seen the csv editor, but I wanted more power. ^^
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

Latest Screenshot of units.csv editor.

TODO New Tabs: Extras

TODO - validate record modification always takes place
TODO - Create new record
TODO - Create new record for subqueries
TODO - make data path picker work - no longer rely solely on heuristics. (40%)
TODO - validate all current fields are accessable (80%)
TODO - validate write still functions after field changes, read functions when fields missing
TODO - read master_part_list.csv ...?

About ~25 hours programming time left before the beta, so unless I get suddenly busy, should be up by friday. Will probably post a preview by tuesday.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

When you finish it, if you agree, I wouldn't mind having it on SVN ;-)
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

klauss wrote:When you finish it, if you agree, I wouldn't mind having it on SVN ;-)
Ditto.
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

That would be fine. Since I'm releasing it GPL, my agreement was essentially implicit anyway. My personal long term hosting abilities aren't really that grand. ^^

Any information on these issues would be appreciated:

Hud_Functionality and Max_Hud_Functionality I'm currently showing as just the value in the field, so either 1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1 or an empty field. I believe I understand what you're doing with it.. but at the moment I don't really know how to break it down or if I should bother. This might concern other people more than it does me, but given a map of what it means it would be within the scope of what I'm trying to do to display it.

Shields: From what I can gather, the Top, Right, Front Shield is the front, the Bottom, Front Right and Left is the Right and Left, and the Top, Left, Back is the Back. Right now I'm displaying these by their fieldname, which means I'm showing several fields that are always empty - at least in Vegastrike. Should I move the fields to represent what they're used for, or will Vegastirke, or one of the other projects that uses it's engine, offer 8 emitter shields?

Likewise, Mesh, which is slightly more complex than a base string, I'm just displaying as a string, though I plan on revising this, so it displays as 3 fields- because my goal is simplicity.

For Directory the picker doesn't include ./ in front of the path, since I feel these are extraneous, and there are at least a few entries in that row that don't have it. I will ultimately have drop downs for everything for which a range of values can be determined, though this may not be in the first release. Is there some reason that the ./ should be in the path, or is it an artifact of how that field was populated?

Finally, are there any calculations that I can/should display to make life easier?
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

JasonStiletto wrote: Shields: From what I can gather, the Top, Right, Front Shield is the front, the Bottom, Front Right and Left is the Right and Left, and the Top, Left, Back is the Back. Right now I'm displaying these by their fieldname, which means I'm showing several fields that are always empty - at least in Vegastrike. Should I move the fields to represent what they're used for, or will Vegastirke, or one of the other projects that uses it's engine, offer 8 emitter shields?
So, the engine acts a bit hackishly here, as the number of emmitters is encoded by the number of non-empty fields, and the meanings for the fields changes based upon number of emmitters :-P. Some interface sugar that corresponds to 2,4,6,8 shield facings, with a selector for which number is desired (so as to be able to change the number associated with an entry) could be nice. The VS data set will eventually have ships with 8 shield facings - the capital vessels haven't been massaged much stat-wise of late.
JasonStiletto wrote: Finally, are there any calculations that I can/should display to make life easier?
Oh, sure. Thrust, Mass --> linear acceleration is a nice quickie that's fun for comparison. Were I to think more on it, I'm sure there are others that would come up, but they're in the same vein.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

JasonStiletto wrote:Hud_Functionality and Max_Hud_Functionality I'm currently showing as just the value in the field, so either 1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1 or an empty field.
Those are subsystem damage status, they're only required for savegames, so I would imagine a CSV editor could safely set them all to 1s. I don't think anyone needs to mess with it.
But... if you wish, I bet I can dig up their meanings.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

well, my ISP is apparently going to make hosting files a bit painful for a few days. This will probably go down sometime tonight, and I won't be able to host myself for 4 more days.

http://members.cox.net/david.mears/vsda ... ux-gtk.zip
http://members.cox.net/david.mears/vsdataboy-win32.zip

if that dies before you get to play with it, here's a magnet url that should work with the mainlined Bittorrent and Azureus.

magnet:?xt=urn:btih:7DJQYSMUFJXZZS7MUUJ5JUKOHJWDHAXB

this is not feature complete, even for a beta. In particular, saving hasn't been enabled yet, as it is not well tested. The next version will have that functionality, along with mote completely built out screensets. Various other things may or may not work as expected, and probably the best way to look at it is as an interactive screenshot.

never the less, it is slowly drawing toward where I want it to be. Comments welcome, but it's a bit early for bug reports. ^^ Comments are very welcome- I get bored and distracted easily and I've already been working on this thing for since the 27th of last month.


in any case, the code uses the heuristics from Vegastrike to find the data4x directory, but if you happen to have it somewhere strange... err, the best thing would probably be to just set the program in the data4 directory, or one level above or below it.

since it doesn't save currently I am reasonably confident that it can't do anything to your data. So no worries if you want to see what I'm doing.

being able to compile TO too OSes from one OS saves lots of time.
Zeog
ISO Party Member
ISO Party Member
Posts: 453
Joined: Fri Jun 03, 2005 10:30 am
Location: Europe

Post by Zeog »

As for saving files, may I add some pitfalls we've encountered earlier?
-- Not all string data is stored in quotation marks.
-- Please maintain empty lines (and try to avoid adding empty fields instead which show up as a line of commas). These are very usefull for hand editing.
-- We've had people adding extra empty fields to the end of each line which show up as trailing commas.

Those things don't break the data but every unnecessary change of format will be commited to the repository and marks the entire database line as changed which makes diffs practically useless. (People were going back and forth between storing strings with and without quotation marks due to using different tools. Therefore, the entire file was replace every time. It's very tiresome to find out what content they actually changed.)

Thank you much! :-)
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

the save routine re-opens the csv file, re-parses it on save and checks the value of the field stored internally vs the line it's outputting field by field. Only records it modifies should be touched at all. While this slows saving down, it carefully preserves the csv file exactly as is, including blank lines, which aren't saved, and empty fields, which may be "", or just the nothingness between two commas. In addition, the entire table is stored in ram while working with it, and fields that it doesn't know about aren't stored at all- but where they are in the file is. the parser simply rewrites whatever was in that location when saving. It won't add any trailing commas or anything like that.

I need to do a little more work since floats are rarely internally precise. I guess I can just figure out the precision of the stored number, try generating a string with the same precision and make sure it matches. Also, I wasn't aware of the comma gotcha, since my parser handles that on import and the export parser puts quotes around the things it thinks of as strings.

I will make the parser note quotes and if they aren't in the string, check the string it's writing for commas and write as is if there aren't any.

The biggest problem I may have with the save feature is that it currently won't do too well with appended records, which means I'll probably want to keep up with which records are actually "New" if any, and sort them into the output whenever I hit the end of a category.. or something.

anyway, the routine is already written, but as I don't fully trust it yet, I haven't enabled it.

I will diff the program output vs the backup file it generates (or just the svn diff command) before releasing one where it's turned on, to verify sanity.
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

I'm nearing release on the beta of the units editor, just finished the last major routine that needed to be added. writedat.. which is basically the reverse of the parser that lets me treat things like subunit as tables. Most of what is left is glue.

I am having trouble figuring out how to best populate the selector for prohibited upgrades..but that's really nothing too major.
JasonStiletto
Merchant
Merchant
Posts: 34
Joined: Sun Sep 24, 2006 8:17 pm
Contact:

Post by JasonStiletto »

Post Reply