Reviving Vega Strike

Development directions, tasks, and features being actively implemented or pursued by the development team.
Post Reply
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

does it make mac binaries? and i should know this, but i wonder if libpython installs the interpreter or if those get packaged separately ...since that would effect unix peoples too then, as we currently dont require an independent python install, just the lib.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: Reviving Vega Strike

Post by klauss »

For mac and unix-like there's pythonfreeze.

Although I don't know how those binaries behave among distros and mac versions - we should try.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
Mr. Fahrenheit
Trader
Trader
Posts: 26
Joined: Mon Jan 15, 2007 4:06 pm

Re: Reviving Vega Strike

Post by Mr. Fahrenheit »

I hope this will be taken in the spirit intended (i.e., 'helpful'), but over the last couple of years it seems like one of the largest obstacles in the way of pushing out any new official releases has always revolved around the size of the download, and not wanting to use the SVN as a release mechanism.

Yet that is exactly where things stand right now.

I admire the tenacity of the group overall, and the good karma of not wanting to abuse source forge.

Surely some other project on SF has figured out a distribution mechanism. If nothing else, maybe just torrenting the release would work. It sure works for the innumerable movie/software/what-have-you pirates, and a lot of that stuff has *got* to be larger than VS. Or post a couple of questions around the 'net; find out how other projects in similar boats are handling things.

You guys have done a really nice job on a pretty complicated application, you shouldn't feel that you have to invent some cool new distribution system too; leverage the hive mind.

Anyhow, maybe this has all been chewed / is being chewed over elsewhere. But I saw the subject coming up Yet Again, and it's a shame because I'd bet that someone somewhere has cracked this nut. Possibly multiple someones, with multiple nuts. ....um, maybe I'd better stop typing now.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

using these other programs to create python bins wouldn't be a problem for the creation of the patchsets ...but I would be reluctant to put two or more projects as dependents just to be able to build the updater app.

Wouldn't it just make more sense to build the updater into vegastrike from the get-go as it already links to python and has hooks for that language in interpreting scripts? Just have the python script installed in /data that contains the code for the updater function.

This way we can have mods customize the updater app for their particular mod, changing the way it's rendered and path and function as well as determining download methods and such. The same should be done for vssetup. Making it a /data oriented subroutine, which is what it should be since it is dependent on /data. A mod would have different configuration routines it would want, different ways of showing that data to the user to be more in-tune with their mod. Similarly, any change to /data requires the config app should reflect that if needed, by having it be in /data and be a python script itself, the connection between the two becomes implicit.

I'm definitely leaning more towards a solution where the update app is vegastrike executed via the same type of python methods it uses to deal with missions and such files in /data. And i think vssetup should move towards the same. With their UI's being rendered in GL in-game and totally controlled by the python files associated with them.

as for torrent: without a decent way of incremental patches, it would mean total replacement each release. Aside from having to find a different server (wcjunction i guess) to be a seedbox, not everyone doesn't blink an eye at 700MB downloads. I know, crazy ..but they exist.

So right now we're mostly arguing over the details of implementing said incremental patch system. :)


edit: it's not a new distribution system we're talking about here either. It's a fairly old standard one. We all basically agreed on an incremental update system produced by an as-yet determined diff between release tags. This does not have to be tied to the project as it's part of package/release management and just needs to be howto'd basically in a way that the same output can be achieved on all archs we support. What we have yet to determine is the updater/patch aspect, which does get tied to vegastrike.
Ed Sweetman endorses this message.
Mr. Fahrenheit
Trader
Trader
Posts: 26
Joined: Mon Jan 15, 2007 4:06 pm

Re: Reviving Vega Strike

Post by Mr. Fahrenheit »

Fair enough, but figured I'd throw in a 2 cents. I know it is not a trivial sort of thing to try and do, patching folks' systems without damaging them -- god knows that there are 'professional' software houses that do a lousy enough job of it. Figured that something like torrents might take the load of the SF servers when new official releases come out, since right now, the number and types of changes seem to indicate a need for wholesale replacement anyway.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: Reviving Vega Strike

Post by klauss »

safemode wrote:using these other programs to create python bins wouldn't be a problem for the creation of the patchsets ...but I would be reluctant to put two or more projects as dependents just to be able to build the updater app.
That doesn't make sense... VS depends on python. Period.
Freeze.py at least is part of python's standard toolset, so you're not adding a dependency, you're using more of an existing dependency.

Besides... if a dependency saves you a lot of trouble by all means use it!.
safemode wrote:Wouldn't it just make more sense to build the updater into vegastrike from the get-go as it already links to python and has hooks for that language in interpreting scripts? Just have the python script installed in /data that contains the code for the updater function.
Perhaps vssetup, but I'm not sure about the updater.
Vssetup because settings are things a user expects to be able to do from inside the game... and not any other reason.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

in any case, it'll really be dependent on the person who takes up the actual coding.

We agree on that it should be python driven.
We agree It should be simple incremental updates.
We agree that it should at least support http (it should be able to GET and continue a download).
We agree that version information should be stored in a simple text file on the webserver
We also agree that it should handle multiple locations for the patches (or at least point to sf.net's mirror round robbin)


I dont think it can get any easier at making it cross platform other than putting it within vegastrike. But this is where we have yet to agree. I think by putting the updater in vegastrike, we guarantee that it runs on the system that vegastrike runs on. We also have the option of giving it a singular cross platform gui in-game if we want or just leave it as a cli option. It also gives us the option of runtime customizations, so mods can use these tools (i talk of vssetup here as well) so that they dont need to write their own. All vs would need is some callbacks for requesting network support (we have networking code already, so might as well try to re-use that) and we have FS support so there's that.. what's left is the GUI which we should or could have menu type callbacks for python to create a menu system (perhaps load it's own set of defined images and create a simple widget system for pulldowns and tick boxes and text inputs. etc. Though, the updater _could_ be done with little more than a confirmation from the user via a button press to update to the latest compatible version and no other Ui at all.

Either way, i dont see how it could get any easier other than from within VS. The FS and network backend are already there and any solution we have is going to need both of those.


I see vssetup being useful being internal to vegastrike not only for intuitive reasons, but because it creates a single interface on all archs, removes a dependency (we wont need gtk), and it allows mods to "skin" it so that it behaves and looks the way their mod is tailored to behave and look. It also lets vssetup do away with messy pseudo xml config crap that it currently deals with. We have a list of valid configuration directives in options.cpp/.h We then know what the config file can contain, and can parse it for correctness, completeness, and fill in missing configuration directives, as well as keep vssetup better in sync with vegastrike. Mods or us in /data land can set ranges for values to be given to the user via the python script that controls and generates the config menu screen. Since this would change depending on the mod.


I see the same for the updater, although it would be much less complicated than vssetup in regards to UI, as it technically needs none, but could have extended features added later that could be useful to users, like selecting mirrors or download methods as that stuff would be configurable and extendable via the python file. By being embedded with vegastrike, we get to use it's internal methods for fs and network and have the option of a GUI using the same interface as vssetup would use. Again, mods could be encouraged to use this as a general updater, completely configurable to adapt to their needs and yet requiring no additional dependencies or runtime bins.

Eventually it would be nice if we would have vegastrike installed separately from any sort of data, and mods wouldn't need to ship VS themselves, and everything would just work so long as you had a compatible version of VS installed, something that /data or the mod you are using would have a python file to check against (we would have to do a better job at making sure revisions to a minor do not change functionality at all). I'd really like to see that separation of game engine from game more in "vegastrike" than occurs now.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: Reviving Vega Strike

Post by klauss »

The issue I have with putting the updater in VS is the language.

Doing network stuff in C++ is painful. Python's library within VS is restricted, so you can't just "import urllib" (I think).

If you ever wrote networking code in C++ and python, and compare the two, you'll have to agree. Python is wonderful for that kind of stuff... wget in 1 line of code:

Code: Select all

open("google.html","w").write(urllib2.urlopen("http://www.google.com.ar").read())
Soo... much... beauty...

You could get that beauty from C++ of course. But you'll need dependencies. And I don't know any really good networking API for C++ btw.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

freeze is nifty, and i do like the way it works, I just want it to be moddable so that mods can create personalized update utilities without having to grab the /vegastrike source and compile things.

so, freeze generates a binary, but can this binary load python scripts at runtime like VS does? If so, then that's cool and i'm totally on board with this method. I'm sure i can customize cmake to use freeze on our special update.py and generate a bin (one that vegastrike --update could call even (perhaps we would want this as our preferred method, so that vegastrike can use it's builtin code to parse the config and or look up and find the /data dir and send that to the update bin along with it's own version and such relevant info). Once executed, it would import the python script in /data/updater that would provide the frontend to it's backend. It could be a simple cli or a full on gui frontend using python's tkInter. Mods may even include other python scripts to be imported as additional modules supporting/implementing cvs/svn/git or torrent clients. The backend basically only needs to provide the standard python networking and tkInter and file interfaces...which i think is default compiled in.

I like that idea a lot.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: Reviving Vega Strike

Post by klauss »

safemode wrote:so, freeze generates a binary, but can this binary load python scripts at runtime like VS does? If so, then that's cool and i'm totally on board with this method.
Python can eval(), can compile(), so it definitely can do:

Code: Select all

eval(open("scriptfile","r").read())
But I think we don't have to resort to that ugliness... I seem to recall that freeze did not impair python's import mechanism at all, so "import <lib>" did actually look for a file "lib.py" as usual.

Don't trust my word though, just try it.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

I am unlikely to be of much help in recreating vssetup or an update utility in python, as i don't really write in python at all, I just find it semi-hackable. Perl is my scripting language of choice. I can't stand white-space sensitive python.

I wish i did know python well though, as it seems like this should be a fairly easy task to at least get the update utility to work via command-line interface ... and the vssetup via tkInter should be quite simple compared to the gtk utility. (gtk i can write ....and i would tihnk tkInter would be recognizable).

but I am very much on board with this approach, ... so long as the UI and configuration is controlled by an external python script we can keep in /data and all the updater utility does is expose the network and provide some simple web oriented functions and expose the fs so we can patch /data and give us a means of creating a gui if we want to.
Ed Sweetman endorses this message.
JsnMtth
Bounty Hunter
Bounty Hunter
Posts: 174
Joined: Wed May 27, 2009 6:38 am
Location: Fresno, California - United States of America
Contact:

Re: Reviving Vega Strike

Post by JsnMtth »

Could the VS engine be run without ANY data?

If it can't it should, it should also be able to recover from mis-configuration is we're going to put vssetup in-game. It should be in-game, but it needs to have a safemode (no pun).

I would think that the update system should be able to recover from data corruption as well.

Also as far as the diff. Are we talking about a binary diff or a file diff?

Not to mention we'll need to distribute a public key and digitally sign the updates.

If they key accompanied the mirror list AND supported new data installations, then mods could be auto-installed too.
charlieg
Elite Mercenary
Elite Mercenary
Posts: 1329
Joined: Thu Mar 27, 2003 11:51 pm
Location: Manchester, UK
Contact:

Re: Reviving Vega Strike

Post by charlieg »

No diffs. Only file add/replace or delete. Keep it simple.
Free Gamer - free software games compendium and commentary!
FreeGameDev forum - open source game development community
snow_Cat
Confed Special Operative
Confed Special Operative
Posts: 349
Joined: Thu Jan 05, 2006 12:43 am
Location: /stray/
Contact:

Re: Reviving Vega Strike

Post by snow_Cat »

^ - -^ I remind you that I only got involved in this project after noticing a corrupted file.
^- - ^ If we rely upon Diffs in our patches/content updates then there will be a lot of corrupted files, many unique to one user, and many repairable only by a complete reinstall.

^ - -^ I don't think this will help us.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: Reviving Vega Strike

Post by safemode »

a script that describes what files get added or deleted or renamed is a diff. Just not a very versatile one. It describes the difference between what you have now and what you should have, only at the file level, rather than at the line level like our normal friendly "diff" app. That's all i was referring to.
Ed Sweetman endorses this message.
charlieg
Elite Mercenary
Elite Mercenary
Posts: 1329
Joined: Thu Mar 27, 2003 11:51 pm
Location: Manchester, UK
Contact:

Re: Reviving Vega Strike

Post by charlieg »

Sorry, I misread Jason's post anyway. A file diff, I think that's what I just described anyway.
Free Gamer - free software games compendium and commentary!
FreeGameDev forum - open source game development community
snow_Cat
Confed Special Operative
Confed Special Operative
Posts: 349
Joined: Thu Jan 05, 2006 12:43 am
Location: /stray/
Contact:

Re: Reviving Vega Strike

Post by snow_Cat »

Thanks for the clarification.
I have no objection replacing files rather than file-content.
JsnMtth
Bounty Hunter
Bounty Hunter
Posts: 174
Joined: Wed May 27, 2009 6:38 am
Location: Fresno, California - United States of America
Contact:

Re: Reviving Vega Strike

Post by JsnMtth »

@safemode
Sounds good to me. I don't think the extra complexity would be worth the binary diff savings.

@snow_Cat
We can checksum the data as part of the update. That will guarantee integrity, regardless of how we diff it.
Post Reply