0.6 plans

Post by **chuck_starchaser** » Mon Apr 28, 2008 5:23 am

Ah, my apollogies. Yeah, it's too bad the STL keeps grabbing conceptual words left and right and giving them too specific meanings.

Post by **safemode** » Mon Apr 28, 2008 1:16 pm

Ok, so roadmaps are really pointless, so how are we going to demark minor revisions? I dont care really that everyone just does what they want, we really dont have enough people working on VS to tell people not to. But we need to figure out a way to say "This is version 0.5.1" without having to wait an entire year and 6 months after we decide to make a release to actually make the release. or at least tag something that _could_ simple be packaged up as a release if we wanted to.

I just dont see it happening very easily. Everyone's pet projects overlap in time of development, so at any moment we have new immature code unsuitable for release or perhaps even not even compileable on all systems.

I'm sorry if i'm coming off whiney or whatever. I'm just frustrated because we have absolutely no order in the project and everyone is afraid to enforce any order for fear of driving away the few develeopers we have that actually work on it. It seems more like we're all working on VS individually, rather than together, and to me, that's unfortunate, and to the users, that's gonna mean just another long wait until the code is synchronised and releasable. Maybe it's an impossibility with the size that we are, i dont know.

Post by **klauss** » Mon Apr 28, 2008 2:14 pm

What prompted that outburst of frustration?

Maybe that I just commited stuff that would go on 0.5.6?

Sorry, but that little project had already started before the roadmap existed, and it's required by the next release of GG and, AFAIK, all the bullet points for 0.5.1 are taken already:

1. Make a high quality graphics texture pack. This shouldn't require a change to code, except in vssetup, and the only change to data4.x would possibly be a config file change, though it's not necessary.

You're already on it I think?

2. continued updates to the shader and it's textures in units.

Chuck was working on some specs and I just started trying to implement some support in VS for those specs. IOW: chuck & I.

3. End User tools for helping with content creation. These should be a part of vegastrike code-base, to keep things in sync and reduce maintenance.

We've been brainstorming on those with Darkmage. I'm trying to get him to write a full spec before doing anything. However, the task is big, I wouldn't make 0.5.1 wait for it.

4. Minor bug fixes.

I guess everyone else is working on this.

However, I feel a bit targeted by that post. I was pretty much ignoring the roadmap to concentrate on already-taken projects. I'll try to prioritize (I won't drop my pet projects

) according to the roadmap from now on.

Post by **safemode** » Mon Apr 28, 2008 2:35 pm

it wasn't targeted just at you, all last revision we had the same issue to get 0.5 out. It's mostly pent up from that.

And I'm not saying what you added conflicts or makes anything harder, but if you feel something needs to be changed with the roadmap, then i would think it's preferred to make a change to the roadmap, then do the committing. So in case anyone does object, they'd get a chance to. The roadmap is the way it is because nobody has spoken up to see about modifying it. Right now it's pretty much exactly as i whipped it up.

I'm not concerned so much with some work being done outside of a strict list, but I'm concerned that we will still have the ability to say "when X Y Z are met, we tag a release". X Y and Z can't be held up by other stuff in future milestones.

basically, if anyone has any changes they want to make with the order of some of the minor revisions, or any other changes, voice them. We'll mull them over and if everyone agrees it'll get changed. That was my point in making it and posting it here. I just had to start with something, hence the list as it is. The goal is not to do what i want or what I think we should do, it's just to make it clear when we are ready to declare a minor revision so we can definitely say "we've reached 0.6.0" without what happened with 0.5 to happen again.

Post by **klauss** » Mon Apr 28, 2008 2:41 pm

Well, I think 0.5.1 is ok as it is. GG guys are screaming for movie support, and everyone ever wanted it, so I've been working on it and it's not on the roadmap. But adding sound to movies has been difficult with the current audio API, so maybe we'll have to push up the refactoring of sound code.

But I'll try to stick to the current roadmap. It makes sense in its assignment of priorities. That means I'll have to hack a lot of stuff to make streaming work with current audio API, and if I just can't do it cleanly enough (I refuse to make a mess just to avoid pushing up the refactoring), then I'll move the refactoring of sound code to... 0.5.2?

Does that sound reasonable?

Post by **safemode** » Mon Apr 28, 2008 2:45 pm

btw this is in general, the point about all the objectives being taken already for the next minor revision doesn't mean to just work on something in the future and commit that to the current head. That could ruin the source state for the next minor release.

if everyone doesn't have something to do every minor-revision then we need to get together and discuss things. Not only should things be added or moved around to better make use of people, but not every objective is a one man mission.

Post by **safemode** » Mon Apr 28, 2008 2:59 pm

klauss wrote:Well, I think 0.5.1 is ok as it is. GG guys are screaming for movie support, and everyone ever wanted it, so I've been working on it and it's not on the roadmap. But adding sound to movies has been difficult with the current audio API, so maybe we'll have to push up the refactoring of sound code.

But I'll try to stick to the current roadmap. It makes sense in its assignment of priorities. That means I'll have to hack a lot of stuff to make streaming work with current audio API, and if I just can't do it cleanly enough (I refuse to make a mess just to avoid pushing up the refactoring), then I'll move the refactoring of sound code to... 0.5.2?

Does that sound reasonable?

I didn't realize the movie code was so incomplete when i made the list. I figured sound came with movies via ffmpeg, obviously that's not the case.

So our problem is people _need_ movie support, coding movie support requires redoing the audio code. I think i'll be busy with end user tools at least through 0.5.2, so we can swap ray collider, with audio refactor, since you'll probably be busy with the graphics refactoring so it doesn't make sense to have both of them in the same revision anyway.

So instead of ray collider, we do audio refactoring and ray collider work gets pushed back a revision.

Personally, I would do your refactoring of audio now. Then worry about movie code later and avoid further work de-hackifying everything.

Make audio refactoring 0.5.1, then proper movie support part of 0.5.2, or if it's easy enough, slip it in 0.5.1.

Post by **klauss** » Mon Apr 28, 2008 3:19 pm

Ok then... I don't need much convincing when it comes to avoiding hacks

The problem with audio is that, though ffmpeg does the decoding, it doesn't do the playing. OpenAL isn't made for streaming, though it's possible - just weird, and thus current music code (which ought to be doing streaming) just loads the entire track into RAM. We can't do that with movies, we need OpenAL to process it in chucks to be able to synchronize with video, and hence the trouble: all the API for sound in VS has been based on the idea that you load a sound once (in full) and play multiple times. Doing streaming needs a separate audio processing thread, and the whole API is not thread-safe (in fact, it's documentedly thread-unsafe). So you can see how doing it without rewriting the sound API could be a problem.

So... I'll move sound refactoring phase I* to 0.5.1, will get to it rightaway (in parallel with texture packing stuff), and add movies in 0.5.1 too (most of the code is ready, it's only waiting for streaming audio).

* Phase I would be basic functionality. Phase II would add mixer channels (a feature to get seamless blending with music, comm chatter and sfx), and Phase III would add environmental effects.

Post by **safemode** » Mon Apr 28, 2008 3:40 pm

I assume each of these phases is intended for a separate minor revision. In which case, phase 3 ought to be done by 0.5.3. Also, i just noticed i skipped 0.5.4 and if the wiki would load i'd be able to check if that reflects it or filled it in.

I have part of 0.5.5 set for cleaning things up for the refactoring (now of just graphics). 0.5.4 needs to be defined, and it'll likely be something like 0.5.5 and 0.5.3.

We'll likely be working over audio related bugs during that time. With any luck, the period between your audio work and the graphics work should yield less time spent trouble shooting audio issues while we do the graphics rewrite. Which will be huge.

It'd be really nice to use 0.5.4 and 0.5.5 to get as many graphics and gameplay related bugs squashed as possible, so that the graphics reworking can be fully regression tested.

Post by **klauss** » Mon Apr 28, 2008 4:27 pm

I'd leave phase III for much later, because I don't even have a specification. But I've been thinking about it the past few weeks. I'll see where to put it.

Post by **safemode** » Mon Apr 28, 2008 4:44 pm

ok, well, figure out where you want phase 2 and (i checked the wiki finally, 0.5.4 is there).

Also, I'd like to hear what's going on with darkmage (never seen that nick used on here before) and your work on the content creation tools. Was wondering if it was in any conflict with some of the tools I was going to work on, relating to a new config program and some other somewhat backend editors for csv and xml files.

charlieg · Post by **charlieg** » Mon Apr 28, 2008 9:39 pm

safemode: I think your problem is that organising volunteer-driven development isn't something you can define detailed roadmaps for. The contributor landscape just changes too rapidly for it to be effective and instead it ends up frustrating. I think what you need to do is, instead, only group features into roughly 3 different categories.

1. The next release of this phase (0.5.1) - features people are actively working on that can be considered applicable to the next stable version

2. The current release phase (0.5.x) - features that there is desire to push into the current stable version but either will not be done in time for the next release or do not have anybody actively working on them

3. The next release phase (0.6.x) - major features that will force the release of a new stable branch of the game (breaks save games, requires large changes etc) which may be active or a considerable part of upcoming development

And for the sake of appeasing blue skyers:

4. Future release phases (0.x) - major features people want but nobody is actively working on or interested in taking up in the next release phase

If you simplify the roadmap using the above layout, you'll find everything easier to organise IMHO. Features from 0.5.x may go into 0.5.n (n being the next release) if somebody takes them up and they are done quickly enough, otherwise you just have whatever people are working on and can focus on tidying them up and releasing without having to worry too much about what happens next.

Also get people to use branches for features that are not considered to be part of the next stable release. That way trunk is always there for testing. e.g. Klauss maybe should have used a branch for his PGG-related features.

Post by **pyramid** » Mon Apr 28, 2008 10:28 pm

The whole point of breaking the 0.5.x release phase into smaller parts is to show the dependencies in case there are any. It is is the case with the xml support for config files and vsseup and contributor tools. The latter will be much easier to code and maintain when xml support (plus the corresponding config variables support in code) is standardized.

It can also serve as a rough linearization of what features will be tackled right now, and which next with a finer granularity than just 0.x. Than will be a different rough roadmap which will be worked upon soon(er or later).

This approach does not invalidate neither pet projects, which are the basis of the whole open source community development, nor the reality of frequently changing contributors. With six active developers with svn commits, their focus should also be on coordination and thus stimulation of the community to contribute. Without clear alignment between the devs and coordination of necessary tasks, the releases will take what they took. This is something we are trying to change.

I strongly believe that it makes good to the project to state, discuss, and document things. Just that. If at the end a feature is documented as being thought for 0.5.6 or 0.5.9, while we are still hacking on 0.5.1, doesn't really change landscape. But it gives visibility to the devs and to the community about priorities of the devs and the contributors.

There is another point that hasn't been discussed yet at all. Managing contributions. We have seen in this release that models that were submitted long time ago, only got hooked to the game just before the release. This is devs responsibility and shall be handled differently in the future. How, is yet to bee discussed.

So basically our approach is not very different from your proposal, just that the 0.5.x group is a little bit stretched and the 0.x scopes are not yet organized.

Post by **safemode** » Wed Apr 30, 2008 12:55 am

I'm beginning some work on svn re-organization of bin and data

I've come to realize that we really dont play well with DOS type directory structures. While every sane OS uses / for directory delimiters, we all know windows does not.

We hard-code / all over vsfilesystem. This is wrong. We should have a macro defined as per defined WIN32, such that the dirctory lookups for the data dir are magically correct whether we use dos or unix directories.

If nobody disagree's, i'll have a patch to vsfilesystem and (eventually many other places i would take it) for supporting \ or / as dir delimiters. I'm quite surprised we dont do that now.

As it is, the win32 repo wont be usable until the win32 bins are recompiled i would think. Since we make dirs /, only the current dir or one behind is searched for the config (. , ..) since all other dirs are illegal.

linux and mac should be fine though without the patch, though i can't reach any of the mac bins, someone will have to commit them.

edit:

Mac bins should go into /mac/bin if there are more than one dir for bins for ppc and x86, then there should be two dirs in /mac one for each.

I'm tempted to ignore making a linux bin tree, mostly because unlike win32 and mac, there is no one linux OS to build for. Thus it seems like a maintenance nightmare to try to pretend like there is and put out "one" set of linux bins. That's the distros job.

Once these OS specific trees are working, /bin will be removed from data4.x

Post by **safemode** » Wed Apr 30, 2008 1:53 am

ok, maybe i'm missing something. but I thought in windows land, stdio functions resolved some\path\to\file not some/path/to/file I doubt it does both. So how is it that most of our stuff is setup with the paths hardcoded to the unix way? How is it that fopen opens anything in windows ? Is there something in the vc project files that is macro-ing fopen for win32 ? or am i just completely clueless about win32 development.

i can't find any wrapper that we use, and we dont use a macro to define the directory delimiter, So i'm quite confused right now on what is going on in the windows build.

EDIT: brain fart. apparently windows can use forward slashes. which means the win32 repo i created should work without modification.

Post by **safemode** » Wed Apr 30, 2008 2:37 am

The win32 bin distrib repository has been tested and works as expected. No source changes are necessary.

With that said, I will go ahead and remove /bin from data4.x tonight.

Those of you with data4.x already downloaded shouldn't need to fret.

simply telling win32 to download to the parent directory of where you have data4.x should result in the directory being detected as already updated.

If not, canceling once data4.x starts downloading and moving your complete data4.x dir into the win32 download dir should also work.

Post by **ace123** » Wed Apr 30, 2008 4:37 am

forward slashes work fine in the Posix functions, and I believe the Win32 API as well... I think the reason that you are confused is the built-in windows command line tools do not work with slashes (don't know why though).

Post by **safemode** » Wed Apr 30, 2008 7:35 am

ok, some svn changes underway here.

Basically, win32's binary repository works without any modification. svn co win32 and you're set. Data4.x will have /bin removed soon (within a couple days) and everyone needing it is expected to switch over to win32

The process of switching over to win32 is fairly easy. You have two options.

1. Pull win32 without grabbing external trees (option in client). Then move your data4.x dir into the parent directory next to /bin. Turn external trees back on and svn update should work seamlessly on both data4.x and the win32 bin dir.

2. Pull win32 and cancel the download when it starts downloading data4.x and move your full data4.x dir in it's place. svn update should result in a complete pull of the repo, with no further downloading.

Number 1 is preferred. Newbies wont have to worry about all that, just pulling win32 is sufficient.

Basically, all OS's have the support for their binary dir to be alongside data4.x or even in a subdir that's alongside data4.x, and obviously inside data4.x. Thus there is no reason why distribution needs to duplicate data4.x anymore. Hopefully soon with new installers and configuration programs, all of the complication and confusion involved with not wrapping everything up in a monolithic pkg will be completely wiped out. Though it's really not complicated at all.

The mac repo is waiting on some mac bins. It's layout is somewhat different but logistically the same. It'll still pull data4.x is an external repo, and it shouldn't require any source modification.

The lino repo is really a far distant concern for me. I personally am against one, but some people think it's worthwhile... In any case, it's layout will be similar to win32's.

Note, this is for binary users ... Those who want source should just get the vegastrike repo and data4.x manually. But note, that doesn't mean you can't share data4.x between your prebuilt bin repo and source compiled. Just watch the versions because in the future, on releases we will be tagging not only the bins to a revision, but the external pull of data4.x as well.

The other svn change is hqtextures. hqtextures is the beginnings of the high quality texture pack. In my branch is initial support (all that's remaining is removing on-the-fly compression in the graphics code).

Basically, you pull hqtextures and place it next to your data4.x dir. Then you goto your config file and add < var name=hqtextures value=hqtextures / > into the data section. value is configurable so that mods can make their hq texture pack named whatever they want.

This is done in such a way that we use mod mechanisms, but still allow the use of a mod (we dont take the place of an actual mod).

There are currently no textures in hqtextures but i tested it locally. Artists will have to decide amongst themselves what they want added to the hqtextures pack. I would be very careful of what is selected, as it wont be compressed at all.

Adding textures is easy, you follow the same dir path as the texture has in data4.x, just pretend hqtextures is data4.x. all the overriding is done seamlessly.

In the future the config editing and pulling of the hqtextures repo would be done by the config program and installer. Until those tools are created, you'll have to manually insert the variable and set it. I'll probably insert a commented line that you can just uncomment so nobody has to type in the entire line.

Post by **pyramid** » Wed Apr 30, 2008 2:31 pm

And while we are at reorganizing the repo, can't we also rename data4.x to data and patch the source where required?

Post by **safemode** » Wed Apr 30, 2008 2:38 pm

It's not hard to regexp the source for data4.x It's definitely not hard to move the repo to data rather than data4.x

Apparently what's hard is getting everyone to change their svn working dir's to a new source and target. That svn switch cmd is tricky < / sarcasm >

but yea, that's the argument you'll get against it, I tried it, lasted for about a day.

Edit: If we're going to rename the data4.x dir we need to do it soon, like before friday. I want to get the word out about win32 and the OS bin distrib dirs so I can remove the /bin dir from data4.x very early next week.

Also i'm thinking about moving the music dir out to reduce the download and filesystem size since it's pretty common for people to play without music.

I think it's important also to tell people who aren't interested in really developing the SVN, to use export, not checkout. This I believe makes it impossible to patch update, but you should still be able to pull changed files and have it overwrite current files. It should really reduce the footprint of VS on people's computers. Though that might have to wait until the new installer-update program is done.

Post by **pyramid** » Wed Apr 30, 2008 3:02 pm

The reply to this post sounds to me like an ok to move on with the renaming:
http://vegastrike.sourceforge.net/forum ... hp?p=97234

Post by **safemode** » Wed Apr 30, 2008 3:51 pm

yea, i was talking to ace last night while doint some commits

The main concern is making sure we dont have a lot of people who already have data4.x downloaded , getting stuck with the prospect of having to download it again.

We need to make sure and test that the svn changes we are making are able to be mirrored by someone sitting on the sidelines with their data4.x dir so they can end up with the new dir scheme. This is highly important.

A step by step guide to get from data4.x to /win32/data for example will be necessary.

Basically, the first step would be svn switching data4.x to data after the appropriate svn copy is made and changes to dir paths in VS.
Now the user can rename thier data4.x dir to data, as it points to the svn repo "data"
They would then be directed to download the OS bin distrib repo of their choice, with retrieving external repos turned off in their client.
Then they would be directed to move the data dir into the directory they placed that bin distrib repo. So inside you wil have /bin and /data. (mac will look a bit different but same idea)
They can then turn retrieving external repos back on.
svn update should yield a fully up to date pull without redownloading data.

Furthermore, inside that bin distrib dir they would also need (if they wanted) the hqtextures repo.

So for those users /win32 would look like : /bin , /data , /hqtextures .

running svn update in win32 (example) may only update bin and data however, since they are part of that repo, /hqtextures is a separate repo that has to be placed there, the user will have to update the hqtextures dir specifically like any other repo they are updating.

For those not using bin distrib repos, you can just download data4.x (soon to be data), and put hqtextures next to it if you want super high quality mode.

Please note, hqtextures is empty, but has been tested and works. Artists need to get together and figure out what they want to have in it. This is for them so we dont have to fight over dds anymore. Also note, the code is still in place to compress on the fly any jpegs.. Rather then pull that code out, I'll make it so everything is happy with the compression config var, so in vanilla VS, the config will turn compression off, dds will still work as normal, but non-dds wont get compressed. I can then remove the special case for png, too.

Directory structure in hqtextures should mirror the texture as it is in data4.x

It should only contain textures. Possibly if there is a desire for it, much higher polygon count meshes. We dont want to be behind crisis in system requirements do we? What we lack in obscene shader requirements we can make up with in trillions of triangles.

Post by **klauss** » Wed Apr 30, 2008 6:52 pm

What about:

Code: Select all

svn move data4.x data
svn propedit svn:externals .
> vi/emacs opens up, add one-liner:
data4.x http://svn.vegastrike.sourceforge.net/svnroot/vegastrike/trunk/data

You'd change data4.x into an external copy of data (data4.x then becomes identical to data, it gets updated when data gets updated).

Post by **safemode** » Wed Apr 30, 2008 8:00 pm

you cant see external pulls without a real directory. Most people dont pull /trunk, they pull data4.x or whatever. thus i dont think it would be possible with an externalized data4.x to pull data4.x ..... maybe i'm wrong, but if i'm to believe the physical tree, obviously i would not see such a repository in trunk.

Keep in mind though, the goal is move people to thier OS specific bin dir, and that is _going_ to mean users are going to have to change some paths and do a little work to not have to redownload data files. I outlined a step by step on how to do that. We just need to make it posted on the site.

The only way to get this done is to do it, and force everyone resisting to to follow kicking and screaming. In the end it's aesthetic in nature and provides no new feature of fixes any bugs. So if we're going to make this seemingly painful change over aesthetic needs, it should just be done fast to minimize the pain.

I suggest we move the data4.x repo over to data We update the external defs in win32 and mac to point to the new data dir. We then patch the vs source to point to data.

We make a big post on the front regarding the instructions of moving to the new repos names.

the end. We get this done by the weekend and do it all at once. Like pulling a band-aid, it's better to get it done quick, cuz either way it's gonna hurt.

Post by **safemode** » Thu May 01, 2008 12:53 am

In my tree i have removed the special case PNG code. I also removed the blacklist for DXT5, as the new nvidia drivers fix that issue.

Have a look over and what not. Later tonight or so i'll probably merge my tree with svn head and update the default vegastrike.config file. texture_compression will be turned off. That means that any files not DDS in the data4.x repository wont get compressed on the fly.

We need to comb through again and make sure _everything_ that can be dds in data4.x is dds. Unless there is a really really good reason not to do so. Remember, hqtextures exists now.

edit:

I combed through, nothing of importance is missed.

I also just synced my tree with the head.

Everything should be good and you should notice pretty much nothing different.