Base definition files

For collaboration on developing the mod capabilities of VS; request new features, report bugs, or suggest improvements

Moderator: Mod Contributor

Dilloh
Elite Hunter
Elite Hunter
Posts: 1149
Joined: Mon Aug 14, 2006 3:56 pm
Location: Black Forest, Germany

Post by Dilloh »

Now am I totally wrong or does the xml example by ace already include the possibility of 3D bases? I see bfxms being included, and a z-axis.

If so, I say :mrgreen: !
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

You're right that improperly indented XML is a pain to read. But, on the other hand, the goal is to never edit things by hand - XML is much easier to parse by tools, and so you'd be using tools to edit those files rather than hand-editing.

I imagine hand-editing won't ever be ruled out, sometimes it's useful being able to hand-edit some stuff if you're up to it. But the whole "content creation kit" movement's purpose is to minimize that. And XML allows machine editing rather well (as opposed to python code).
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
ace123
Lead Network Developer
Lead Network Developer
Posts: 2560
Joined: Sun Jan 12, 2003 9:13 am
Location: Palo Alto CA
Contact:

Post by ace123 »

Sorry, nothing in my exmaple is implemented yet.

I was merely suggesting how such a format could be designed, including possibilities for 3D bases.
And it sounds like to fix the lighting issues the best way is to make the whole scene 3D.

Moving to XML is a huge advantage. XML is a data format, while Python is a programming language.
If you include XSLT support, XML would arguably also be a programming language as well, but I don't think those are necessary.

The advantage of a structured data language is that it can be loaded and saved by an editor without losing track of the code that generated each item.
Last edited by ace123 on Sun May 11, 2008 8:35 pm, edited 1 time in total.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Gottcha. But those editing tools had better happen, then. Actually, the reason I was diving into .system files was precisely to make an editing tool, but was defeated. Now one of our PU guys (Gorbalad) might take a crack at it. We want to make a program or script that reads .system files and universe.xml and puts all the info into a collection of .csv files of various formats, though we could settle on some other representation. The point being, being able to easily edit maps, like via a spreadsheet, and then have scripts that update the .system and universe.xml files, as well as the map pages of the wcpedia wiki. Another thing we want to do is a program to adjust prices of commodities so that prices are higher the farther a base is from the nearest producing base, so that profits increase with distance. And we'll soon be opening the flood gates to the sol sector, in PU; which means new bases, new commodities... So we need to automate this process. So this program needs to do graph stuff --shortest distance. Anyways, I'm getting way off topic ... :oops:
But yeah, maybe if the devs could make a little library, just a couple of functions, that can parse the .system or .base file, then maybe 3rd parties could more easily contribute the editing tools.
ace123
Lead Network Developer
Lead Network Developer
Posts: 2560
Joined: Sun Jan 12, 2003 9:13 am
Location: Palo Alto CA
Contact:

Post by ace123 »

I'm interested how your CSV formats for .system files look.
One huge difference between CSV and XML is that XML is hierarchical (sector contains system which contains planets)

I can see how the Universe XML can be translated to CSV... that may be reasonable to try.

We need a tool that generates all the systems given a the description of the whole universe.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

ace123 wrote:I'm interested how your CSV formats for .system files look.
One huge difference between CSV and XML is that XML is hierarchical (sector contains system which contains planets)
I'm interested in it too; doesn't exist yet :) but like I said it would be a whole family of csv's, inter-referencing. One csv format type for each "class" (like system or star or planet or jump). It would be more logical to use a database; but I've never worked with databases.
We need a tool that generates all the systems given a the description of the whole universe.
We need something totally different; we don't want auto-generation, in fact; we want to enter the WC universe by hand, as there's tons of canon already dictating what each system should have in it; and even for systems that aren't pre-dictated by canon, we'd probably want to hand-taylor them to suit particular missions. But we need the data in a format that can be easily read, as we need to write code for economic analysis, as well as for updating (soon to be interactive-) wiki maps, not to speak of the .system and other game files.
In fact, the idea is to do what we did with units.csv, which we split into a multiplicity of csv files in their own repository, and each time we commit a change to one of those files, a script on the server puts them together into a new units.csv and commits it to trunk.
Similarly, we'd like to put the map into a separate repo, in csv format, and have a script in the server that runs after commits, updating .system files, as well as prices in units.csv, and wiki pages in wcpedia.com; --all in one shot, so we don't have to ensure synchronization manually.
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Post by jackS »

chuck_starchaser wrote:
We need a tool that generates all the systems given a the description of the whole universe.
We need something totally different; we don't want auto-generation, in fact; we want to enter the WC universe by hand, as there's tons of canon already dictating what each system should have in it; and even for systems that aren't pre-dictated by canon, we'd probably want to hand-taylor them to suit particular missions. But we need the data in a format that can be easily read, as we need to write code for economic analysis, as well as for updating (soon to be interactive-) wiki maps, not to speak of the .system and other game files.
In fact, the idea is to do what we did with units.csv, which we split into a multiplicity of csv files in their own repository, and each time we commit a change to one of those files, a script on the server puts them together into a new units.csv and commits it to trunk.
Similarly, we'd like to put the map into a separate repo, in csv format, and have a script in the server that runs after commits, updating .system files, as well as prices in units.csv, and wiki pages in wcpedia.com; --all in one shot, so we don't have to ensure synchronization manually.
Just as a possible clarification, re: autogeneration, I believe that ace123 was talking about our plans for developing a tool that will auto-generate all systems specified by a universe description file _offline_ rather than on-demand, and ostensibly checked in as part of any given data set. We are also planning to build a per-system editor. Once said systems existed, they would then each be amenable to editing as needed to reflect the particular known characteristics of the system. One could even imagine a fairly verbose set of optional inputs to the auto-generation tool to act as format extensions for the default universe file format.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Ok, I get it.
Still, for us the thing we need is translation, rather than generation. The WC universe is not amenable to autogeneration; whether on- or off-line. Then again, if the editing tool will have easy viewing, browsing and traversal of the universe, we could use it instead of a csv format. But the thing we need, in a nutshell, is a format we can keep on a separate svn repo, easy to edit and maintain, and a tool that can do exports to other formats, and other processings; and hopefully a Python script, so that it can run automatically on the server as a post-commit hook.
And just to clarify; I'm not asking for any of this; it will be our job. All I was suggesting is a small library, with a function or two, that can parse a .system file, or a .base file, and so on for the various other xml file formats used by the engine, to make it easier for people to implement editing tools, exporters and whatever. Just looking at trying to parse .system files looks daunting. You guys could probably make this little library with your eyes closed; but for us mortals it's deadly.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Just to show you, guys, that at PU we mean business...

Image

And this is NOT a model for the landing "room"; it's actually a low poly model
of it, that you'll see inside a few craters when you fly around the mining base.
http://wcjunction.com/phpBB2/viewtopic. ... 0225#10225

AND it's not finished.
AND it's not textured yet.

Ready when you are...
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Cool man!

That means 3D bases?

So very cool.

It almost makes me want to stop working on tangents and techniques to work on XGUI.py.

Haha... gotcha! (just kidding).
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Hahaha. We're actually readier for tangents; as we got normalmaps already that we can't use.
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Re: Base definition files

Post by pyramid »

The base interface is almost not documented at all in our wiki. Here's what I am missing:
* What exactly are the 3 vectors (Base.Ship(room,link, vecP, vecR, vecQ) representing?

EDIT: After some trial and error, what I found out:
* ''vectorP'' - position of the ship with (''centerx'', ''centerx'', ''scale'')
* ''vectorR'' - ship up orientation with up-vector coordinates (''x'', ''y'', ''z'')
* ''vectorQ'' - ship nose orientation with forward-vector coordinates (''x'', ''y'', ''z'')

Correct me if I'm wrong.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

That's right. I also had to deduce it by experimenting.
It is RIDICULOUSLY hard to edit those vectors, by the way.
Not only do they have to be normalized; they also must remain
perpendicular. It took me about a week of full time work to fix
them for all bases in PU, and they still aren't perfect.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

It might be that developers find the name alone very descriptive, since that's how most location/orientations in 3D are represented if not by position/quaternion.

But I can relate to modders: vecR? WTF?

Haha... yea, we need documentation, but more than documentation we need an editor. right?
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

One of the things I tried was using Blender to orient a vector and read the numbers and put them in, but this never worked.
Thinking back now, another thing I could have tried would have been to actually load the background into Blender as background, and a ship's mesh, then orient the ship until it was exactly right, and then deduce the vectors by subtracting coordinates, etceteras...
Hmm... Maybe you don't need to write an editor, after all...
Why didn't I think of that?
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

As far as documentation is concerned, it is always a good practice to document what you were unsuccessfully trying to find on the wiki and were forced to find out your own way. To show you that I am not just boasting (only a little :oops: ), have a look on the development wiki pages for stuff that i have been committing (and not only) over the last year and make a diff to one year ago.

(Almost) nobody likes documenting (actually I do cause then I don't need to remember everything). That's a fact. In any case, I can only encourage everybody to do so, if we ever hope to have major contributions flowing into the repository. Meanwhile I will sometimes put my scribe hat on. Content tools are a fine way to stimulate contribution, but they will never be able to cover all aspects of the game (e.g. cargo, upgrade, weapon images). Documentation is not only about nor for developers or modders, it's about giving equal opportunities to all members of the community to express themselves artistically and in useful ways. It's the ultimate expression of freedom. (Yes, documentation is a freedom tool).

All this blabber just to say that bases are half-way documented now:
http://vegastrike.sourceforge.net/wiki/ ... ackgrounds

Haven't yet done much editing of base backgrounds, so please let me know if you have more (or preferably, edit directly in the wiki).
pyramid
Expert Mercenary
Expert Mercenary
Posts: 988
Joined: Thu Jun 15, 2006 1:02 am
Location: Somewhere in the vastness of space
Contact:

Post by pyramid »

Exemplary! :wink:
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Thanks! Great job, by the way! I'll be glad to contribute to your documentation efforts; I just didn't know of their existence.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Something just came up, somewhat related to these base definition stuff.
Apparently we're missing some landing/take-off animations that were in the original Privateer. Gorbalad was suggesting we could use the 3D game engine itself to render them, but as Nate pointed out, there's probably no support in the engine for such things.
I was thinking, 3D bases could make use of animations (landing pad could be made into an elevator platform that takes your ship underground, after you land; some ships in the dealership room could sit on slowly rotating platforms, not to speak of people walking in the main concourse, etceteras).
So, why not have a "scene definition file", instead, which could handle 3D bases with animations, 2D images, movies, or 3D animations in space. It would be more complicated, for sure; but it would avoid having a scattering of different mechanisms to accomplish this or that. It would put a lot of code in the same place.
No rushing; I just had to write this before the wind blows the idea away.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Ya, animated stuff, good point. It would combine skeletal animation whenever supported with keyframe-based stuff if I read you correctly.

Ie: a skeletal animation defined in the ship's mesh deploys the landing gear as the ship approaches the landing pad through keyframed positions/orientations.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Exactly. When you consider the problem of animations at bases, and combine it with the problem of producing animations in space, the two come together. There's no point in trying to resolve them separately. There's just no reason that AI should be the *only* way to move ships in space. Scripted trajectories should be definable. IOW, the whole thing boils down to a need to extend the vegastrike "scene manager" to include other things besides ships and planets, such as walls; and to include other means of defining motion than just orbits and AI, such as keyframes. Then *ONE* scene manager will be able to produce normal space action, scripted animations of ships landing or taking off, ships or bases with moving parts, base interiors with people walking around, etceteras. IOW, rather than base definition files, what I think the right solution would be would be a scene manager, a la Ogre.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

what ship animation in space would require some hand of god manipulation?

basically, anything interactive with the user or a NPC can and should be operated by AI soley, and there's no need for the hand of god to complicate things by jumping in and changing something behind the AI's back so we have to then set it right afterwards.

i see global scene managers as manipulating those things that are completely eye candy, and totally un-affected by the user or any NPC's.



edit: Scripted movements of ships and such in the game would be definable by simple AI events. As we've been discussing in other threads. These events would be able to define activity on a minute scale or general commands. You could script a ship to leave a jump gate and head toward a base along a convoluted path given by a sequence of commands to proceed in this direction for N meters, then this direction, then that direction etc, controlling the speed, defensive posture, etc per leg etc.

But hey, maybe the scene manager makes more sense for pu. In any case, it's not like the scene manager would be used unless someone wanted it to be, and in VS, we're leaning towards making _nothing_ scripted as far as the user being able to predict what and where a unit will be and do in a given situation. That type of dynamic behavior is what i care about.
Last edited by safemode on Mon Jun 16, 2008 8:40 pm, edited 1 time in total.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Well, I was in fact thinking of eye candy: In the original privateer there were animations for when you jump, when you land, when you take off, when you destroy the steltek drone...
We could try and produce movies; but given that the user can own any of about a dozen ship models, that means that we'd need to have a dozen movies for each of those animations. But why? The 3D engine should be able to produce them in real time!
In fact, right now it does one animation: When you press A to autopilot there's a 2-seconds time of external camera and your ship passing by. I think it's a hack someone added. It would be great to have an official interface for such things.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

well, yea, a scene manager that manipulates the camera would be cool. An eye candy thing is fine. But what i'm concerned with is the scene manager _actually_ manipulating ships and things in the universe such that now the AI is left in a bad state because things were changed regarding it drastically without it's knowledge

Better for things like that would be feeding that data that the scene manager would have to be given to the AI's via the as-of-yet-unwritten event subsystem. Scripting their actions but within the rules of the game. This keeps the scene-manager cut-scene type state of what you're seeing always in sync with what the actual game state is, without hacking around anything.

The downside to that is you are working within the universe, so you can't generate "cut scenes" with 100% certainty of what will be displayed.

For instance, you may want to show a mini-movie of a massive attack on a base, depicting it's destruction. If we want to do this within the confines of the game engine and not via an avi, then the movie will be slightly different each time depending on how carefully the ai event scripts are written that get loaded to the units. It all depends on how controlled the environment is (no non-controlled units are allowed to interact :) ).

edit: I understand mods like PU have pre-defined cutscenes activated by various events in the campaigns. So something that acts like the hand of god and allows the game to render these scenes in real-time is ideal. I just wouldn't want such a thing to _replace_ the ability for VS to utilize the AI to do all of that and leave the "scene manager" to manipulating just the camera (and anything non-interactive). Mostly for the state issue, but partly because it will let us force things to be completely dynamic in VS, not relying on any invisible fingers.
Ed Sweetman endorses this message.
Post Reply