releases and such

[safemode]

In linux, there is not much more to creating a pkged release than building the project, then running the appropriate pkg script through the appropriate pkg program. It may need to be tweaked due to new dependencies, a change in what binaries are actually built, but for the most part it doesn't need to be rewritten each release.

In win32, we have build scripts that are setup for VC++ and such. I'm not sure if we have anything to automate the packaging process for win32. If not then we should figure out if it's possible to make one because it shouldn't be a manual job each release.

In mac land, i'm not sure we even have a build script setup. If that's the case then why hasn't that been done? Otherwise it's also not sure whether it has a pkg script. Mac's should be easier to make install files for than win32, since Mac's idea of installation is simply copying what you pkged up to a directory.

All in all, what i'm asking is is why is it such an effort to physically create a release for all 3 OS's ? Ignoring the necessity to update documentation and release notes and such. How come it's not just a matter of "run build script", "run make pkg-$OS", "pkg data4.x", "put vegastrike pkg and data4.x pkg up online as new release"


by the way, how do we handle minor releases? do we re-package everything, only repackage the changed files? Even though only doing the changed files is more efficient, i think it'd end up being much harder to do than just pkging everything.

i'm just wondering because it doesn't seem possible that we dont have a way to automate the vast majority of this whole process by now. Other than some minor updates to some attributes for the pkg, and some clicks if you're in mac/windows, it should be all pretty much automatic. Again, i'm ignoring documentation/release note etc updates.

[safemode]

I started documenting all the code i've man-handled. The vast majority of the documenting is in headers as outlined in the code style wiki page.

Lots more needs to get documented.


I also got some optimizations along a fast path of opcode. Not big, but better than before.

[safemode]

artwork area is quickly shaping up mostly due to pyramid's work.
Most of the backend work is shaping up too, with only a few GL related issues remaining as "important" for 0.5

The vast majority of the "bugs" left are python oriented (short term anyway). Orbiting bug could be worked around by disabling orbiting for 0.5. Spawn location bugs for certain bases is a python issue. Multi-ship issues are python blockable for now. Segfaults seemingly from dyn_news appear to be from an assumption that doesn't exist (may be C++ originating). Mission issues in general and campaign issues. Upgrade issues. All of these seem to be python related bugs. Or at least their solutions could be made in python and subsequently fixed in updates to 0.5

Not all of these would be considered show stoppers for 0.5 but they could be addressed asap. not many devs work in the python code. I'm sure the mod community (what's left of it) would be better in that area.

[chuck_starchaser]

Just a comment on the release packaging problems. CharlieG will want to kill me for my saying this, but, what's really the benefit of a release, as in a monolithic installer? I know, some people out there WILL NOT use svn. Well, I used to be one of those people, until I got through a few times.

I think it's a question of marketing. You guys at Vegastrike don't "market" svn; --even to find where the instructions are for getting vs through svn is a challenge; they're sitting somewhere at the bottom of a wiki page.

The way we did it at PU is we "sold" svn to the masses. We put a clear announcement to the effect that our game was obtainable, and ONLY obtainable via svn, and we put clear, prominent and pictorial instructions on how to get PU:
http://wcjunction.com/phpBB2/viewtopic.php?t=274
We made it REALLY EASY to understand and follow; step by step.

And the result?

We've been averaging about 5 gigs a day for the past couple of months (since CharlieG announced us at Freegamer). That's about 12 full checkouts per day. Occasional peaks of up to 12 gigs in one day. No complaints.

Occasionally someone suggests that we should have a downloadable, but we argue back that we couldn't afford the bandwidth of having a new downloadable every week or so. Which is true, in fact. Finito.

Which doesn't mean you couldn't set a tag and call it "Release 0.5", if it helps the marketing; --as long as those who get 0.5 can upgrade to 0.51 via svn. Otherwise, not only is it a waste of bandwidth, but you'll have the eternal problem of people reporting bugs for this or that release, in addition to svn. (Heck, there's still people sometimes reporting 0.43 bugs at this forum...)

[safemode]

I agree with sticking to SVN, as it easily allows patching only changed files and seemlessly moving from one release to another.

I insist on the use of tags though. Having users using dev versions isn't acceptable and making sure everyone is on the same page is extremely convenient for gameplay (multiplayer would insist on this) and error tracking.

We can patch to the tag for bugfixes for a given version (and maintain multiple currently mainained versions) and those users would get updated on their next svn update.

It definitely makes more sense to go the svn route. You can easily have a data4.x module without /bin and a mac bin module, win32 bin module, and linux bins can be provided by distros. In each of these bin modules it pulls the data4.x bin so you end up with the whole thing but people dont have to download the bins for every OS whenever they pull data4.x

[charlieg]

Player retention is an important factor in the success and longevity of an open source game. Just because there's enough people that will use SVN meaning it's not necessary to make official releases in order to attract a small community, doesn't mean it's a good idea to shun the rest. People like oblivion would never have gotten so involved had they not been able to easily try a recent version of the game at the time. Most people won't want to get dirty with SVN, or learn how to compile (don't forget acquiring all the dependencies) just to try out Vega Strike.

[loki1950]

There are also distro's that will not touch svn releases on principal as they deem them unstable and not for inclusion in there stable repositories.

Enjoy the Choice

[safemode]

the apt part of that is recent. Building VS is one thing, packaging is another. We just dont have the time as a project to do the packaging. and really, SVN makes such packaging completely unecessary.

Have pkgs... but as a side-note, to the preferred method of svn rather than the other way around.

we're not saying to use SVN to compile VS. We're saying to use svn for the contents of the data4.x directory to download prebuilt binaries (which we can produce much easier than pkgs).

Basically data4.x is an unpackaged package. We're saying to use SVN for distributing that. Not for end users to downoad the source.



that being said, getting the tag of SVN for use in distro pkgs is absolutely no different from getting the tarball. Anyone who thinks it is is retarded beyond comprehension, and has no business pkging for a distribution anyway. It's the same code.

I would think pkg maintainers would be smart enough to know the difference from SVN head and getting the release tag.

I mean where do they think we got the tarball's code from? The Magic land of stable source? I doubt pkg maintainers are that stupid.

[chuck_starchaser]

Exactly as safemode says, CharlieG; downloading via SVN is not one bit harder than downloading an installable. It's only two clicks of a mouse. Well, typing the URL+"username"+"password" the first time. Tortoise is so easy to use it's even easier than a download. A download you still have to install it. The SVN way, there's not even any installation to do.
But I agree, once in a while, and for the sake of distro packaging perhaps, there may be a point in having a monolithic download. But it should be VERY secondary to the preferred way, to the point of having a sticky on the bug reports forum saying "Please, no bug reports about installable version such and so. Only bug reports wanted are those for the current, svn version."

What's really needed is a page with clear instructions on installing Tortoise and using it. You're welcome to copy PU's and modify it as necessary.
Privateer: Parallel Universe Download/Install instructions
PU for Linux - Compiling and Installing
It's just a matter of "marketing" the SVN way.
Need to help users break that association between SVN and "compiling sources".
Need to present it as The New and Better Way to Download
And, heck, we even have a
PU revision counter (changelog)
so people can check what's going on.

I'm telling you; nobody is complaining at the PU forum; users love being able to update every day.

[safemode]

Well, i differ from chuck in that i believe we should have a stringent differentiation on release tagged SVN and the HEAD.

I believe we could do things such that the primary way of getting VS is via the svn url to the current release tag. Bugfixes are made to this tag and bugfixes only until the next release is tagged from the HEAD.

so on and so forth.

I dont believe in pushing the HEAD on the common user, because i dont want to have to deal with the tons of bug reports relating to "my vs doesn't work etc etc" when i break the head... and breaking the head happens. It's par for the course when the only machine you've tested your changes on is your own. The head should be relegated to only those who know what they're doing use it. That minimizes the SNR of bug reports.


Keep the general populace on the release tag. The release tag is stable nobody should have any qualms about using it. It's the same damn code in a tarball.

[chuck_starchaser]

On the other hand, when you do break the head, it's good to have feedback ASAP. Right now, for example, our head IS broken. Well... for a couple of our users it is. These two people have missions they've completed but that keep showing up later as incomplete. We're looking into it. Suffice it to say, some bugs, like this one, may only show up for 1 person out of 100, and without a full user base on HEAD, you might not know there's a problem until much later.
How about a compromise?: Have tags, but only release them in case of emergencies, when HEAD breaks and the bug is elusive. Then you can say to people "We're stuck with a bug in HEAD, you can revert to tag such and so in the meantime, if you want."

[charlieg]

safemode: What you do is upload a source release to Sourceforge and ask the community to provide the builds.

[charlieg]

I'm telling you; nobody is complaining at the PU forum; users love being able to update every day.

You make the mistake of equating existing users with new users. I know it's an impossible thing to really prove, but I would expect the vast majority of people new to PU who look and see 'SVN' will just leave it and hope for a contained release some day in the future.

Sure, there are people who have the time to delve into things further. (Note: there's a difference between expected and actual time taken. Many people simply won't do something if they expect it to take any extra effort and/or time.) However most will look, see it's non-trivial (i.e. involves more than a standard installer) to try the game and simply go elsewhere. Sure, you may be pioneering a better distribution method but you're going to lose out on more players than you'd probably be comfortable with, were you ever to see tangible statistics.

[chuck_starchaser]

Maybe... Hard to tell, of course. Well, like I said, "once in a while"... Right now PU is in too high a state of flux to even think of a release, but maybe in a few months it will make sense to put one out to catch those who flee from svn.
Now that I think about it, we could make a trojan kind of release... A small downloadable that installs svn and downloads the rest; then autoupdates every time you launch the game...

[safemode]

The main problem that you're not seeing (because PU doesn't have it yet) is multiplayer. You wont be able to get a large multi-player userbase going when everyone is on a different day's revision. Some wont be compatible with others. It would be a complete mess.

It's better to keep the majority of users on the same stable version and let fringe users test the waters of the new development version. You may not get as many users giving feedback, but it's just more practical to do it that way.

You dont want to have to make posts and bulletins saying this day or that day's revision is broken ..that's stupid. Keep the dev stuff to devs. Dont make the users bounce around their revisions every other day of the week. Do you not see the damage control you'd have to go through everytime something bad happens in head? What happens when a distro pkgs it and sends it out and you find out "oh wait, that's wrong" and now all the users who downloaded that pkg have to get it again ....multiply that situation by a lot.

I'll tell you what would happen if we fed HEAD to the users. We'd all do our work in another branch (like i do with mine) and only commit changes to head that are deemed safe... Sounds pretty familiar, oh wait, it is. It's the same thing as having a tag and not using head.


You can't run a project in perpetual alpha status. That's what HEAD is. Either we'd end up releasing under a tag and keeping head for dev use or creating a new branch for dev use and using head as our "tagged running stable" which way makes more sense? If you said the latter i'll smack you.

[chuck_starchaser]

ROFL

Point well taken.

[safemode]

that last idea about a self downloading install would work nicely for making sure people are on the same revision (though i would prefer a revision of the tag (bugfix patches), rather than a revision of head).

you wouldn't even need a crazy installer or gui

find a simple console based svn. make a little win32 cmd script (and an analogue for mac) that simple asks if the installation dir (program\ files\vegastrike) or somesuch exists, if so we do an update on it. otherwise we co a copy.

Everything could be hard-coded, we'd release a new script each new release tag and viola.

All the user has to do is download our little pkg containing the script and svn. Double click on the script. Wait for the update to finish and done.

that's no harder than any normal installation program.

[chuck_starchaser]

Or you could have a special page for current tag, say,

Code: Select all

http://vegastrike.sourceforge.net/vs/current_tag.txt

(don't click it; doesn't exist)
So the client checks to see if it has changed, and if it has it does an svn switch to the new tag and updates.

[safemode]

that works too.


so the release schedule would be something like this after we create the "svn installer" which is downloaded once:

create a tag of current Head when head is sufficiently stable and all the roadmap issues are complete. This tag is the "release"

Post updates all over the site, pointing people to the new tagged release.

Update the current-version.txt file that the svn installer looks at when a user clicks it for updating their local copy.



That's it. You make two "svn installer"'s one for mac and one for windows... Done. No more worrying about pkging. No more re-downloading a bunch of files that haven't changed between revisions. Patching is easy as fixes to the data and VS are made to the release tag. Upgrading to the next release is easy.

What's not to like? the user doesn't ever even need to know he's using an svn client or have to know anything about svn. It's just like all those other installers that do basically the same thing, small initial installer download, then it greps an url and decides what to download based on user selections and other info.

I love the idea. It's simple. It's neat. And once you do the initial simple work of setting up the little script to execute the svn client, you dont have to do anything else.

You dont need to wait for people to get on board with the idea though. A proof of concept shouldn't be too hard to get working.

get this http://subversion.tigris.org/files/docu ... -1.4.6.zip

remove iconv and share
you can get rid of svnadmin, svnserve, probabaly the other exe's except svn too. Experiment ... all we need to be able to do is co, and update.

That would bring the svn installer to just under 2MB zipped up. You could have a self extracting zip auto-execute the script file after unpacking, further simplifying the process.

[safemode]

To get back on course as far as things to do prior to 0.5


I've decided that i can't abide by the VS configuration systems. I say systems becaues we have vs_config and it's GetVariable syntax, then we have xmlSupport and it's various type specific Get method syntax.

Then we have the default values needing to be carried over whenever we ask for a value.

it just feels wrong.


So i'm creating a new class, it has 2 functions. readConfig, where we read in the config file, and Defaults, where we reset all our variables to default settings.

This new class will contain every single option VS reads in from the config file (and uses). Default values are set in one place, at one time. Default values are stored in copies of the regular variables, prefixed with a D and are private members. The regular variables are public members, and are accessed directly.


The rest of the game wont need to call vs_config. Wont need to call xmlSupport ...wont need to worry about default values and wont have to set variables to hold these config values.


It's not done tonight... but I'm hoping to get it done sometime in the next couple days.



Note, this does not change config file syntax. It will, however, remove any un-used config variables no longer in use in VS.

[safemode]

Scratch that above post. Further investigation revealed that I was wrong about how the config system is setup.


Apparently it saves everything in strings, not knowing really what is what. It relies on the calling code to know what to look for and what it's values should be.

All it does is return a string, which is then passed through XMLSupport to convert to the correct datatype.


While this method is extensible, it makes hell for maintenance. Since the config system doesn't have a list of config directives, developers are left to comb the source and config files for existing or obscolete config variables.


I will change my class to modify VegaConfig.

I'll include a defaults.cpp that consists of a defaults function that sets every valid config parameter to it's default value. These are all going to be private data members that GetVariable accesses rather than getting a default in it's argument list. A default variable is retrieved from it's list by prefixing the config variable string the user passed to it with a D.

When the config file is read and loaded into VegaConfig, any config directive for which there is no default directive gets ignored. Then when GetVariable is called, any directives requested that are missing automatically uses the default value.

So the key difference i'll be making is creating a list of Default values embedded into VegaConfig ... which has the main purpose of centralizing a list of all valid config variables, and making it easy to add and remove "valid" directives ...since VegaConfig will ignore any directive requested or read from the config file for which there is no default variable.

[charlieg]

I don't think SF would be very fond of an SVN based installer... kinda defeats all their mirroring.

[safemode]

basically our big bandwidth hit comes from the first download, after that it's pretty much just a trickle. With pkgs it would be a hit everytime a release was made. And in any case, we've been using this method as means of distribution for almost 2 years (the last 0.4 release).

and pkgs would still exist, just not as the primary means of installation. And it would be cool to make them simply zipped up svn pulls, so that they could use the svn script installer (included) to update when new releases are made.

It'd be ideal if they offered a read only svn mirror

[chuck_starchaser]

Well, for PU we got a dedicated server now, though we haven't got it on-line yet. Lots of bandwidth (a terabyte per month IIRC); 160 gigs space; so if mirroring became a problem we could setup an svn mirror for vegastrike there, with a self-update cron thing.

BTW, Wolphin, a member of our forum, is looking into this installer thing. I was suggesting incorporating SVN as a library, but he seems very keen on it installing a separate SVN client if none is found already installed.

Would be nice to make the user.config separation from vegastrike.config, to go with this plan.

[chuck_starchaser]

Correction: 2 terabaytes per month bandwidth.