Vega Strike Forums

Posted: **Sun Oct 29, 2006 7:44 pm**

updated 12-6-07
vs_validator-win32.zip
vs_validator-linux32.zip
vs_validator-source.zip (GPLicious and Pascal)

validates contents of units.csv against my understanding of them. Which isn't horrible.

current output:

Code: Select all

Finding data directory.
testing /home/stiletto/data4.x/units/units.csv
Using /home/stiletto/data4.x/
Parser Begins
Processing /home/stiletto/data4.x/master_part_list.csv no error checks - Done.
Reading /home/stiletto/data4.x/units/units.csv
Header Read
Unknown fields: 
 .. None.
Header Processed.
row 657 column 8 Hud_image "planet-ocean-hud.spr" file not found
row 697 column 8 Hud_image "Shipyard-hud.spr" file not found
row 713 column 2 Directory ./contraband directory does not exist.
row 714 column 2 Directory ./contraband directory does not exist.
row 715 column 2 Directory ./contraband directory does not exist.
row 718 cargo_import Cat Contraband/Luddite not found in master part list
row 727 column 8 Hud_image "base_locationmarker.spr" file not found
row 728 column 8 Hud_image "base_locationmarker.spr" file not found
row 729 column 8 Hud_image "base_locationmarker.spr" file not found
row 800 column 2 Directory ./h496 directory does not exist.
Database processed.
Max Fieldlengths (Str only)
---------------------------
cargo.Category: 45
cargo.description: 50
cargo.filename: 28
cargo_import.Cat: 49
light.meshfile: 18
mounts.SIZE: 50
mounts.type: 28
subunit.filename: 35
units.BSP_Mesh: 21
units.Cockpit: 18
units.Combat_Role: 12
units.Directory: 47
units.Explosion: 19
units.Hud_Functionality: 66
units.Hud_image: 36
units.Index: 39
units.Max_Hud_Functionality: 65
units.Mesh: 40
units.Name: 36
units.Object_Type: 19
units.STATUS: 33
units.Shield_Mesh: 23
upgrades.filename: 26

Posted: **Tue Oct 31, 2006 1:02 am**

ooooh, nice!

I am most appreciative.

Posted: **Tue Oct 31, 2006 2:11 am**

My data viewer doesn't like the borked records much. This way everyone wins. If, you know, the corrections are actually made. ^^

Posted: **Tue Oct 31, 2006 12:08 pm**

Does the validator check if the categories which are entered into the field for buyable cargo are actually present in the master parts list? It should display many errors for that as I haven't updated them yet. We had quite a lot of spelling errors there as well, so a find and replace strategy doesn't always work.

Thank you so much!

Posted: **Tue Oct 31, 2006 5:00 pm**

not as yet, it's no trouble to add it to the code base though. I'll release an expanded version today or tomorrow that checks the MPL and filenames are actually associated with files and the like, but for now the initial release already lists the most harmful problems of which there are several.

Posted: **Fri Nov 03, 2006 4:51 pm**

The version I'm working on now verifies cargo and cargo_import, cargo seems to pass okay but cargo import throws hundreds of errors. Most are, from what I can tell, valid errors, but

row 553 cargo_import Cat missions/Attack_Confed not found in master part list
row 553 cargo_import Cat missions/Attack_Luddite not found in master part list
row 554 cargo_import Cat missions/Rlaan_Milspec not found in master part list
row 555 cargo_import Cat missions/Rlaan not found in master part list
row 555 cargo_import Cat missions/Attack_Aera not found in master part list
row 555 cargo_import Cat missions/Attack_Luddite not found in master part list

Missions.. are.. cargo? This apparently deals with part of the program I don't understand yet.

After everything else I'm working a minor bug in my field size reporter, are there any other fields I should check specifically against data that isn't hard to get to? Suggestions welcome, precise suggestions more welcome.

Posted: **Fri Nov 03, 2006 7:14 pm**

The valid errors are due to my let's say laziness. We reorganized the cargo items and for testing purposes (pictures etc) I made all the categories available at the starting planet which is an ocean planet. The other sites still show the old category names which should therefore be detected. Good!

As for the missions. I'm not sure but I think that missions were introduced as a kind of cargo to rather "own" the mission, a hack once and now the standard.

Posted: **Fri Nov 03, 2006 7:34 pm**

if they're supposed to be there, so be it, but I don't know how to say they're valid, so.. erm.. so do you suppose I should just filter those results?

attached is the results from the latest build. I've got a plan whereby vsdataboy could flag everything invalid but implementation would probably take me about two weeks. For now I'm just going to worry about increasing core functionality and keep error reporting a separate package.

I'm going to try to get it to verify file existence for the endless meshes and what before I release the next verification tool. That should be able to do about 90% of what is needed. Range warnings could be put on individual fields too, but at the moment I don't see that as all that important. A correctly working tool should precede one that has flame stripes.

Posted: **Fri Nov 17, 2006 6:50 am**

Finally got some time allocated to play with this - but I'm encountering something odd. The magic number doesn't seem to be matching up; the executable within the zip in the posted link isn't being accepted as being a compatible Win binary. I DL'd it again on the odd chance of bit rot, but the problem persists. What zip utility did you use to make the zip (just to sanity check)?.

If you have the time, could you directly mail my ministerofinformation account at gmail a copy of the windows binary for the validator?

Posted: **Fri Nov 17, 2006 7:12 am**

apparently Info-ZIP from http://www.info-zip.org/ ..

Copyright (c) 1990-2006 Info-ZIP. All rights reserved.

probably version 2.32, or at least, that's what I've got installed today.

It was Cross compiled from Linux, and worked under Wine and the other puter. I'll verify that it runs under windows again tonight and e-mail you out a copy of the latest working version.

Posted: **Fri Nov 17, 2006 7:42 am**

thanks much.

Looked over the initial output from your validator.
Made what I believe to be fixes (albeit, with typo correction, there's always the chance of a .... typo) to most of the ones flagged in the first of the two attachments. That units.csv is checked in. (MPL compliance enforcement pass has NOT been done yet, so it'll still belch errors)

Two notes:

row 570 column 69 - Radar_Range 3.00E+038<> Integer?
That's not actually really an error, as the radar range is a float.

I haven't fixed the cargo import wierdness on what I presume to be the mineable asteroid chunks (lines 599-606). I want to double-check as to whether something "special" was done for that, or if those are just typos before I munge anything.

Looking good to me so far

Posted: **Fri Nov 17, 2006 8:00 am**

okidoke, one more update - the asteroid cargo bug should now be fixed in the units.csv in svn.

It was in the wrong category (should have been cargo entries, not cargo_import)

mystery solved (probably

)

Posted: **Sat Nov 18, 2006 12:04 am**

Strange...
I still get the asteroid chunks errors (lines 599--606) but the rest is gone which still should be there! SVN Revision 10783. I'm issuing the command for the vs_validator from the data directory itself. However, I have to supply the "./" argument otherwise it doesn't find the data directory... I have the feeling that it is not working the way it was intented to. Any ideas?

Code: Select all

Finding data directory.
Using ./
Parser Begins
Header Read
Unknown fields:
Index
Header Processed.
row 574 column 69 - Radar_Range 30000000000.0<> Integer?
row 577 column 69 - Radar_Range 30000000000.0<> Integer?
row 599 cargo missionCargo falsenot known boolean value.
row 599 cargo missionCargo falsenot known boolean value.
row 600 cargo missionCargo falsenot known boolean value.
row 600 cargo missionCargo falsenot known boolean value.
row 601 cargo missionCargo falsenot known boolean value.
row 601 cargo missionCargo falsenot known boolean value.
row 602 cargo missionCargo falsenot known boolean value.
row 602 cargo missionCargo falsenot known boolean value.
row 603 cargo missionCargo falsenot known boolean value.
row 603 cargo missionCargo falsenot known boolean value.
row 604 cargo missionCargo falsenot known boolean value.
row 604 cargo missionCargo falsenot known boolean value.
row 605 cargo missionCargo falsenot known boolean value.
row 605 cargo missionCargo falsenot known boolean value.
row 606 cargo missionCargo falsenot known boolean value.
row 606 cargo missionCargo falsenot known boolean value.
Database processed.

Posted: **Sat Nov 18, 2006 1:34 am**

jackS wrote:thanks much.

row 570 column 69 - Radar_Range 3.00E+038<> Integer?
That's not actually really an error, as the radar range is a float.

Changed units.inc to define that value as a float, so that line should validate properly now.

Posted: **Sat Nov 18, 2006 2:17 am**

Zeog wrote:Strange...
I still get the asteroid chunks errors (lines 599--606) but the rest is gone which still should be there! SVN Revision 10783. I'm issuing the command for the vs_validator from the data directory itself. However, I have to supply the "./" argument otherwise it doesn't find the data directory... I have the feeling that it is not working the way it was intended to. Any ideas?

it should find the data directory when run from the data directory. the latest version that i uploaded didn't have MPL compliance checking.

here's the newest code

vs_validator_linux.rar
vs_validator_win32.rar
vs_validator_source.rar (in pascal)

Edit: Version bump so that linux version walks search correctly

Posted: **Sun Nov 19, 2006 1:48 am**

updated x2, now does first generation checking to make sure meshes, directories, and what not are actually there.

vs_validator_linux.rar
vs_validator_win32.rar
vs_validator_source.rar (in pascal)

Posted: **Sun Nov 19, 2006 2:21 am**

This version seems to have some problems finding the shield meshes (as in, it appears convinced that it can't find any of them)

Also, some of the primitive meshes didn't look like they were being found - does the tool search the shared "<datadir>/meshes" directory?

Posted: **Sun Nov 19, 2006 3:09 am**

The only things with "Mesh" that aren't showing up for me are these:

Code: Select all

row 578 column 14 Mesh "landscape.bfxm" file not found
row 578 column 16 Rapid_Mesh "landscape_BSP.bfxm" file not found
row 578 column 17 BSP_Mesh "landscape_BSP.bfxm" file not found
row 579 column 14 Mesh "landscape.bfxm" file not found
row 579 column 16 Rapid_Mesh "landscape_BSP.bfxm" file not found
row 579 column 17 BSP_Mesh "landscape_BSP.bfxm" file not found
row 590 column 14 Mesh "" file not found

It may be a windows specific problem though. will test it on my windows box.

and as for searching the shared dir, err, no. I will see what I can do though.

EDIT: I freshened the code. I was trying to parse the other _Mesh like Mesh is parsed, so they were being read as empty strings that also should be files. The error was showing up under windows and not linux because adding nothing to a directory string gives a directory string, and a test if a file exists works for directories under linux but not windows - for directories.

anyway, fixed.

Edit2: Made it also check meshes subdirectory

Posted: **Sun Nov 19, 2006 6:51 am**

if you find errors in it's evaluations, let me know and I'll try to fix it.

Posted: **Sun Nov 19, 2006 7:18 am**

It finds all the meshes I expect it to find now

Great work!

Posted: **Tue Nov 28, 2006 9:01 pm**

I started to replace the failing cargo categories in units.csv. With the validator which is now working very well by the way I noticed that the cargo category validation needs to be a little relaxed.
An example: There exist items the following cargo categories in the master_part_list.csv:
Natural_Products/Food/Area
Natural_Products/Food/Confed
but in units.csv Natural_Products/Food is allowed as well is however currently reported as an error.

Posted: **Wed Nov 29, 2006 12:04 am**

Ah, I didn't realize. I'll modify it tonight, should be a new version by tomorrow sometime.

Posted: **Sat Feb 03, 2007 12:52 am**

Hey folks, I'm having some trouble with this validator. When I try to run it, I get an error stating that it is not a valid win32 application. Perhaps the archive is corrupted. Could you update it please?

Posted: **Sun Nov 18, 2007 7:15 am**

Sorry I disappeared, life exploded unexpectedly. Here is the current output and the freepascal source code for the validator. I'll see if I can make the data editor run again next week, coat it in a headly layer of GPL and upload it. I don't have a cross compiler going yet as I've just set pascal up on this computer.

Code: Select all

Finding data directory.
testing /home/stiletto/data4.x/units/units.csv
Using /home/stiletto/data4.x/
Parser Begins
Processing /home/stiletto/data4.x/master_part_list.csv no error checks - Done.
Reading /home/stiletto/data4.x/units/units.csv
Header Read
Unknown fields: 
 .. None.
Header Processed.
row 203 column 14 Mesh "dodo.bfxm" file not found
row 269 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 270 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 278 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 279 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 282 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 283 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 284 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 286 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 287 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 291 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 297 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 298 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 649 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 658 column 8 Hud_image "planet-ocean-hud.spr" file not found
row 669 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 670 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 671 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 672 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 673 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 674 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 676 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 677 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 681 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 682 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 683 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 685 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 686 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 687 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 689 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 690 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 691 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 692 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 693 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 698 column 8 Hud_image "Shipyard-hud.spr" file not found
row 701 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 702 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 703 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 704 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 705 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 713 column 2 Directory ./factions/upgrades/aera_mil_missions directory does not exist.
row 713 cargo_import Cat missions/Aera_Milspec not found in master part list
row 714 column 2 Directory ./factions/upgrades/aera_missions directory does not exist.
row 714 cargo_import Cat missions/Aera not found in master part list
row 714 cargo_import Cat missions/Attack_Confed not found in master part list
row 714 cargo_import Cat missions/Attack_Rlaan not found in master part list
row 715 column 2 Directory ./factions/upgrades/confed_mil_missions directory does not exist.
row 715 cargo_import Cat missions/Confed_Milspec not found in master part list
row 716 column 2 Directory ./factions/upgrades/confed_missions directory does not exist.
row 716 cargo_import Cat missions/Attack_Aera not found in master part list
row 716 cargo_import Cat missions/Attack_ISO not found in master part list
row 716 cargo_import Cat missions/Attack_Luddite not found in master part list
row 716 cargo_import Cat missions/Attack_Rlaan not found in master part list
row 716 cargo_import Cat missions/Confed not found in master part list
row 717 column 2 Directory ./factions/upgrades/iso_missions directory does not exist.
row 717 cargo_import Cat missions/Attack_Luddite not found in master part list
row 717 cargo_import Cat missions/ISO not found in master part list
row 718 column 2 Directory ./factions/upgrades/pirate_missions directory does not exist.
row 718 cargo_import Cat missions/Attack_Confed not found in master part list
row 718 cargo_import Cat missions/Attack_Luddite not found in master part list
row 718 cargo_import Cat missions/Pirate not found in master part list
row 719 column 2 Directory ./factions/upgrades/rlaan_mil_missions directory does not exist.
row 719 cargo_import Cat missions/Rlaan_Milspec not found in master part list
row 720 column 2 Directory ./factions/upgrades/rlaan_missions directory does not exist.
row 720 cargo_import Cat missions/Attack_Aera not found in master part list
row 720 cargo_import Cat missions/Attack_Luddite not found in master part list
row 720 cargo_import Cat missions/Rlaan not found in master part list
row 721 column 2 Directory ./factions/upgrades/standard_missions directory does not exist.
row 721 cargo_import Cat missions/Attack_Pirate not found in master part list
row 721 cargo_import Cat missions/Merchant not found in master part list
row 721 cargo_import Cat missions/Neutral not found in master part list
row 723 column 2 Directory ./contraband directory does not exist.
row 724 column 2 Directory ./contraband directory does not exist.
row 725 column 2 Directory ./contraband directory does not exist.
row 728 cargo_import Cat Contraband/Luddite not found in master part list
row 737 column 8 Hud_image "base_locationmarker.spr" file not found
row 738 column 8 Hud_image "base_locationmarker.spr" file not found
row 739 column 8 Hud_image "base_locationmarker.spr" file not found
row 803 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 807 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 808 cargo_import Cat upgrades/Hull_Upgrades not found in master part list
row 810 column 2 Directory ./h496 directory does not exist.
Database processed.
Max Fieldlengths (Str only)
---------------------------
cargo.Category: 45
cargo.description: 50
cargo.filename: 28
cargo_import.Cat: 49
light.meshfile: 18
mounts.SIZE: 50
mounts.type: 28
subunit.filename: 35
units.BSP_Mesh: 21
units.Cockpit: 18
units.Combat_Role: 12
units.Directory: 47
units.Explosion: 19
units.Hud_Functionality: 66
units.Hud_image: 36
units.Index: 39
units.Max_Hud_Functionality: 65
units.Mesh: 40
units.Name: 36
units.Object_Type: 19
units.STATUS: 33
units.Shield_Mesh: 23
upgrades.filename: 26

Posted: **Sun Nov 18, 2007 3:20 pm**

Thx Jason should make someone happy when we start combing the data tree prior to release.

Enjoy the Choice

Vega Strike Forums

new tool: vs_validator

new tool: vs_validator

Re: new tool: vs_validator