0.6 plans

Development directions, tasks, and features being actively implemented or pursued by the development team.
Post Reply
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

In addition to pulling game options into proper class, I'm going to go ahead and give GCC some help in auto vectorizing. My plan today.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote:In addition to pulling game options into proper class, I'm going to go ahead and give GCC some help in auto vectorizing. My plan today.
My plan regarding that, was to split physics state from units, and put them all tightly packed in vectors. Then units would reference the entries by iterator references, that would have a reference to the backing vector and an index into it. A simple freelist can handle insertions and deletions, and in general free space handling. The vectors would use a special aligned allocator (to be written, or maybe boost has one).

That would not only pack the floats and allow SIMD to be used optimally, but would also concentrate writes and reads in a packed memory area, so it would do wonders for performance. With better performance, we might be less aggressive in the allocation of long simulation steps, and get rid of many integration bugs.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

klauss wrote:
safemode wrote:Ok, so i'm cleaning up the optional variables... and finally consolidating _ALL_ of them into the vs_options class ...since that work was started and never completed years ago.
I saw the patches. It's not so nice to add extern blah game_options to every cpp that uses it, include the extern in options.h.

Additionally, I'm reworking particle systems so all the options about particle systems will probably collide with trunk. Again, not your problem, just mentioning I'll have to work extra to merge that.

Other than that, go ahead, it will probably make it easier to edit settings on-the-fly.
I started doing the externs per source for two reasons .. 1 less obfuscation over what game_options is and 2 i didn't want to mislead people into thinking that the options class implementation file declares game_options. It doesn't.

1 extra line of code per file that uses it isn't a big deal and there are no practical downsides.
Ed Sweetman endorses this message.
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: 0.6 plans

Post by Deus Siddis »

I have a couple more proposals for the road to v0.6 regarding units.

First is in addition to the .milspec ships I'd like to get rid of all the .something ships so that variation between ships of the same class is determined entirely by equipment (perhaps NPC ships can be spawned with dynamic equipment outfits even, in the distant future). I can then begin precisely balancing each class of ship without curve balls being thrown in by these "dot variants" with seemingly random differences to base mass and such. I think the change will only require modification of units.csv, master_parts_list.csv and faction_ships.py.

Second, I'd like to reorganize the order of columns in units.csv to be logical in some way. Right now values that work together like mass and acceleration, are separated by unrelated columns placed seemingly at random.

If I have your approval Klauss, I'll get started on the patches.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

The thing holding units.csv type files back is the fact that our method of reading them is via simple spreadsheet viewer like excel.

these files shouldn't ever be edited by people directly.
Then we can make them only contain fields that differ from the default values.

a simple python gui would be infinitely more powerful than spreadsheet viewing.
Ed Sweetman endorses this message.
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: 0.6 plans

Post by Deus Siddis »

safemode wrote: Then we can make them only contain fields that differ from the default values.
It's not just a matter of eyesores, the .variants add complexity without purpose. We have more than enough different ships to create variety and with different models that make each recognizable. And they can be modified by installing different upgrades. The dot variants add an excess layer on top of these which does nothing new but makes the game more confusing to player and developer alike.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote:I started doing the externs per source for two reasons .. 1 less obfuscation over what game_options is and 2 i didn't want to mislead people into thinking that the options class implementation file declares game_options. It doesn't.

1 extra line of code per file that uses it isn't a big deal and there are no practical downsides.
There are lots. Not the least of which, that it adds a lot of uncertainty about where it comes from. Not the least of which, is that it's redundant, and redundancy means more work in case you want to change something, and more opportunities for inconsistencies.

In essence, it's bad practice because it's troublesome. Rather than re-discovering why, I'd suggest you follow good practice.

If you want to make it explicit where the structure comes when it's used, put it in a namespace Options, and reference it as Options::game_options. That's clear indication of where it came, and does not result in any kind of inconsistency, uncertainty or coupling.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

Deus Siddis wrote:I have a couple more proposals for the road to v0.6 regarding units.

First is in addition to the .milspec ships I'd like to get rid of all the .something ships so that variation between ships of the same class is determined entirely by equipment (perhaps NPC ships can be spawned with dynamic equipment outfits even, in the distant future). I can then begin precisely balancing each class of ship without curve balls being thrown in by these "dot variants" with seemingly random differences to base mass and such. I think the change will only require modification of units.csv, master_parts_list.csv and faction_ships.py.
AFAIK, It already is dynamic. The .blank or .stock variants (can't remember which) are used to build randomly upgraded ships.
Deus Siddis wrote:Second, I'd like to reorganize the order of columns in units.csv to be logical in some way. Right now values that work together like mass and acceleration, are separated by unrelated columns placed seemingly at random.

If I have your approval Klauss, I'll get started on the patches.
I propose something better. Put in one CSV the columns you think are related stats that have to be edited as a group and can be edited as a group on a spreadsheet, and I'll get you split CSV support.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: 0.6 plans

Post by Deus Siddis »

klauss wrote: AFAIK, It already is dynamic. The .blank or .stock variants (can't remember which) are used to build randomly upgraded ships.
Looking at faction_ships.py it looks like there are .blank's but not .stock's in the faction spawn lists,. So what about getting rid of everything except the .blank's?

And what are the ships that have no extensions for? They seem to be some of the only ships that are consistently fully equipped, including a set outfit of weapons.
I propose something better. Put in one CSV the columns you think are related stats that have to be edited as a group and can be edited as a group on a spreadsheet, and I'll get you split CSV support.
A combination of both might be best of all. Some things may belong in the same file but be arranged in a sub-optimal order, I'll have to check. Also before I can propose something like that, I'd need to ask about the function of a long list of mystery columns.

Does changing the order of columns in units.csv break the game? I know changing the order or rows won't cause harm because those are key based, but columns...?
loki1950
The Shepherd
Posts: 5841
Joined: Fri May 13, 2005 8:37 pm
Location: Ottawa
Contact:

Re: 0.6 plans

Post by loki1950 »

Changing column order would break every Python/C++ function that reads the csv files as they expect the columns in a specific order It is a data structure so it must be designed in advance and not changed in any ad hoc manner at random.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo ideapad 320-15ARB Win 10/Mint 19.2
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

loki1950 wrote:Changing column order would break every Python/C++ function that reads the csv files as they expect the columns in a specific order It is a data structure so it must be designed in advance and not changed in any ad hoc manner at random.

Enjoy the Choice :)

1. Aren't the csv files read by the engine in a single class, translating what is in units.csv into the unit object's data structures.
2. now is the time that breaking and then fixing can be done safely. these are config files...not data structures. They should reflect what the engine supports, and that means they have to change with the engine. We should not make these files static. We shouldn't even be directly reading/writing to these files outside of the engine. There are fields in the units.csv that aren't used _ANYWHERE_ . that's gotta be fixed. And whatever may be written in python that reads it, will just have to be fixed as well.
Ed Sweetman endorses this message.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

klauss wrote:
safemode wrote:I started doing the externs per source for two reasons .. 1 less obfuscation over what game_options is and 2 i didn't want to mislead people into thinking that the options class implementation file declares game_options. It doesn't.

1 extra line of code per file that uses it isn't a big deal and there are no practical downsides.
There are lots. Not the least of which, that it adds a lot of uncertainty about where it comes from. Not the least of which, is that it's redundant, and redundancy means more work in case you want to change something, and more opportunities for inconsistencies.
How can you add uncertainty about where a class comes from when you have the class type right in the file it's being used rather than hidden away in a header ? Your argument there makes absolutely no sense. You want me to hide it in a header where the user can't see it in order to make it more certain. No.

This practice of putting externs in headers is a laziness practice and laziness only. The only time it makes sense to put an extern in a header is if the header actually uses the variables that are extern. game_options thus-far doesn't get used in any headers off the top of my head.
In essence, it's bad practice because it's troublesome. Rather than re-discovering why, I'd suggest you follow good practice.

If you want to make it explicit where the structure comes when it's used, put it in a namespace Options, and reference it as Options::game_options. That's clear indication of where it came, and does not result in any kind of inconsistency, uncertainty or coupling.
so instead of extern vs_options game_options in the source files that use it it would be much more clear to use
using Options::game_options;

I dont think either method is any more explicit. Both require equal amounts of work in each source file that uses game_options. One requires adding a namespace abstraction simply to wrap a global variable to describe where it comes from, the other method references the actual class but requires no added abstraction.

game_options is the least of the worries when it comes to vegastrike externs and #defines. Duplicate variables... Duplicate functionality in data structures.

lets take SIMULATION_ATOM for instance. Pretty basic variable in the game.
it's actually a define
#define SIMULATION_ATOM (simulation_atom_var)
Why? Why obfuscate what SIMULATION_ATOM is ? creating a define for this does absolutely nothing. I dont get it.

what is simulation_atom_var?
well it's an extern float as written directly above the define.

So why dont we just use simulation_atom_var? Or better yet, which is what i did. Just make SIMULATION_ATOM a goddamn float and extern it too.
Hey, look, now cout knows what variable type SIMULATION_ATOM is. Wonder how many other functions treated SIMULATION_ATOM like an int (which is what cout was doing).

This kind of stuff is all over the place.


edit:
I didn't mean creating another variable SIMULATION_ATOM when we already have simulation_atom_var existing is a better idea. I just didn't feel like looking further into why we needed two variables to reference the same thing at this time and changing all the code referencing one or the other.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

Deus Siddis wrote:Does changing the order of columns in units.csv break the game? I know changing the order or rows won't cause harm because those are key based, but columns...?
I can't speak for the tools, but the engine doesn't care about the ordering. It uses the header row to identify data point locations. So make sure to have the right header in place.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote: How can you add uncertainty about where a class comes from when you have the class type right in the file it's being used rather than hidden away in a header ? Your argument there makes absolutely no sense. You want me to hide it in a header where the user can't see it in order to make it more certain. No.

This practice of putting externs in headers is a laziness practice and laziness only. The only time it makes sense to put an extern in a header is if the header actually uses the variables that are extern. game_options thus-far doesn't get used in any headers off the top of my head.
Headers are a centralized place where to define interface. Either functions or structures can both be an interface. If you don't centralize, mismatching interface declarations can result in (binary, as in runtime or link-time) trouble. Compilers will usually tell you of such issues, but it's poor code the code that's open to them. You simply can't have that problem if you declare everything only one, in header files.

You can call it laziness. Programmers are lazy, and it's a good thing. It prevents us from reinventing wheels, from making the same mistakes over and over and over. Because we're lazy, we learn from others instead of rediscovering stuff ourselves.

You can regard that as a bad trait if you want. I don't.
safemode wrote:
In essence, it's bad practice because it's troublesome. Rather than re-discovering why, I'd suggest you follow good practice.

If you want to make it explicit where the structure comes when it's used, put it in a namespace Options, and reference it as Options::game_options. That's clear indication of where it came, and does not result in any kind of inconsistency, uncertainty or coupling.
so instead of extern vs_options game_options in the source files that use it it would be much more clear to use
using Options::game_options;

I dont think either method is any more explicit. Both require equal amounts of work in each source file that uses game_options. One requires adding a namespace abstraction simply to wrap a global variable to describe where it comes from, the other method references the actual class but requires no added abstraction.
The difference is conceptual. A namespace clearly groups some part of the program together. In this case, Options. If you follow some kind of common sense, you can tell immediately where the code implementing those options (or providing the storage for game_options in this case) is located, because, common sense or common coding guidelines will tell you, all things related to options are in options.cpp. Or options_generic.cpp. Or something including options in their name.

That's a big help. An extern declaration tells you nothing of the sort. More, an extern declaration guarantees you nothing - your in-cpp declaration could easily declare a different type, and produce runtime bugs, if not compile-time ones. That is a tell-tale sign of redundancy, and redundancy in code is wrong.
safemode wrote: game_options is the least of the worries when it comes to vegastrike externs and #defines. Duplicate variables... Duplicate functionality in data structures.
Of course. But you're adding more of the bad stuff, instead of removing. It's not desirable.
safemode wrote: lets take SIMULATION_ATOM for instance. Pretty basic variable in the game.
it's actually a define
#define SIMULATION_ATOM (simulation_atom_var)
Why? Why obfuscate what SIMULATION_ATOM is ? creating a define for this does absolutely nothing. I dont get it.
Nor do I. Presumably, the author thought it wise to wrap it in a define for some reason. Perhaps he was toying with alternative representations. Who knows.
safemode wrote: I didn't mean creating another variable SIMULATION_ATOM when we already have simulation_atom_var existing is a better idea. I just didn't feel like looking further into why we needed two variables to reference the same thing at this time and changing all the code referencing one or the other.
AFAIK, one is a timestep base, another is a multiplier of it, representing the time step currently being used on physics update calls (remember this one changes because we have a heterogeneous physics system where units belong to one of a number of "update bins").

Anyway... that's what I remember from the last time I looked into it. Naming is horrible, the whole architecture of it is horrible.

But that says nothing about whether including the extern declaration (or definition, I never remember which is which) for game_options on every cpp is a good idea or not.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

klauss wrote:
safemode wrote: How can you add uncertainty about where a class comes from when you have the class type right in the file it's being used rather than hidden away in a header ? Your argument there makes absolutely no sense. You want me to hide it in a header where the user can't see it in order to make it more certain. No.

This practice of putting externs in headers is a laziness practice and laziness only. The only time it makes sense to put an extern in a header is if the header actually uses the variables that are extern. game_options thus-far doesn't get used in any headers off the top of my head.
Headers are a centralized place where to define interface. Either functions or structures can both be an interface. If you don't centralize, mismatching interface declarations can result in (binary, as in runtime or link-time) trouble. Compilers will usually tell you of such issues, but it's poor code the code that's open to them. You simply can't have that problem if you declare everything only one, in header files.

You can call it laziness. Programmers are lazy, and it's a good thing. It prevents us from reinventing wheels, from making the same mistakes over and over and over. Because we're lazy, we learn from others instead of rediscovering stuff ourselves.

You can regard that as a bad trait if you want. I don't.
headers define an interface and global variables not necessary for that interface aren't part of that interface. game_options is an instance of the vs_options class. It has no more place in the options header than any instance of any other class does. It may be needed by many files, that just means it'll have to be included by those files because passing it to each function or class that uses it isn't practical by any means.

I see no reason here not to continue doing it the way it's been done. game_options is simply an instance, not an interface.

Maybe in some feature that doesn't exist yet, we want to read in a second config file without destroying the values we read in from the first one.. etc.


_anyways_

on issues that matter a bit more .... vectorization of high traffic functions. aka...the fun that is non-portable code.

Some easily helpful things that can be done is
1. tweaking the cpu opts flags to better and more correctly reflect current gcc opts available for a given arch.
2. Use __restrict__ on all non-overlapping pointer and reference arguments to functions. This can be ifdef'd to "" on compilers that dont support it with no further changes necessary.

That will help the auto-vectorization out without adding any invasive non-portable code.
The other things we can do are more involved.
Rewriting conditionals that follow if(something) var = that; else var = what; into non-conditional writes var = (something) ? that : what; The auto-vectorizer can work with the latter, but not with the former.
Everything else is pretty much rewriting loops to avoid conditionals and be more simple.

So in between getting bored with putting options in one place i'll be sprinkling that around and checking how it changes the build with -ftree-vectorizer-verbose ...
Ed Sweetman endorses this message.
Deus Siddis
Elite
Elite
Posts: 1363
Joined: Sat Aug 04, 2007 3:42 pm

Re: 0.6 plans

Post by Deus Siddis »

klauss wrote: I can't speak for the tools, but the engine doesn't care about the ordering. It uses the header row to identify data point locations. So make sure to have the right header in place.
Alright I'll do some column rearranging then.

So what do you say about getting rid of all ships that aren't .blanks?
safemode wrote: There are fields in the units.csv that aren't used _ANYWHERE_.
So then why don't we just delete these empty columns?
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

from what unitcsv 's class looked like, the engine is hardcoded to read in a certain number of fields. I didn't get too far into it but I didn't feel like tackling rewriting the unit.csv file to not have that field and updating any other data related files. I still have _lots_ of cleaning to do and breaking.

And i've been spending far too much time today trying to help the vectorizer. blah.
Ed Sweetman endorses this message.
log0

Re: 0.6 plans

Post by log0 »

safemode wrote:headers define an interface and global variables not necessary for that interface aren't part of that interface. game_options is an instance of the vs_options class. It has no more place in the options header than any instance of any other class does. It may be needed by many files, that just means it'll have to be included by those files because passing it to each function or class that uses it isn't practical by any means.

I see no reason here not to continue doing it the way it's been done. game_options is simply an instance, not an interface.
Are there meant to be more public accessible instances of vs_options class? How about providing an interface? Maybe something as simple as GetGameOptions() in const and (only if really required) non const variants? I wish all global/static data in vs would be accessed trough proper interfaces instead of that extern crap.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote:from what unitcsv 's class looked like, the engine is hardcoded to read in a certain number of fields. I didn't get too far into it but I didn't feel like tackling rewriting the unit.csv file to not have that field and updating any other data related files.
You probably just looked units_csv.cpp superficially. If you look closer, the mapping from column to field is actually initialized the first time it is required, it's all in some special macros, CSVTable::Init and CSVTable::SetupOptimizer.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote: headers define an interface and global variables not necessary for that interface aren't part of that interface.
We will strongly disagree there. There is no reason not to consider global variables an interface. A lousy interface, but an interface it is.
safemode wrote:I see no reason here not to continue doing it the way it's been done. game_options is simply an instance, not an interface.
You just haven't experienced the pain of having to grep for a function yet, because the code doesn't even hint where it is.

The reason may be as simple as: we had decided not to introduce any of such externs a while ago. You're pedaling in the wrong direction. If you want to start a new discussion, I'll consider it a waste of time, but you may. Without such a discussion, you're simply going against the flow. Big projects always require us to do something we don't like regarding style, because no two programmers have the same style. Accept it or discuss it, but don't force it.
safemode wrote:Some easily helpful things that can be done is
1. tweaking the cpu opts flags to better and more correctly reflect current gcc opts available for a given arch.
2. Use __restrict__ on all non-overlapping pointer and reference arguments to functions. This can be ifdef'd to "" on compilers that dont support it with no further changes necessary.
I don't think __restrict__ is all that necessary. GCC seems to do just fine without it, with unaided alias analysis. In any case, it might be a good idea, but it's hard actually knowing where to put it. And, since GCC already seems to understand much of VS code regarding aliasing, it might all be for nothing.
safemode wrote:That will help the auto-vectorization out without adding any invasive non-portable code.
The other things we can do are more involved.
Rewriting conditionals that follow if(something) var = that; else var = what; into non-conditional writes var = (something) ? that : what; The auto-vectorizer can work with the latter, but not with the former.
Everything else is pretty much rewriting loops to avoid conditionals and be more simple.
Alignment is also rather easy and "portable". In the sense, it can be made portable. Lots of examples lie within ffmpeg code, we could draw inspiration from it.
safemode wrote:So in between getting bored with putting options in one place i'll be sprinkling that around and checking how it changes the build with -ftree-vectorizer-verbose ...
Uuu... didn't know that one.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

klauss wrote:
safemode wrote: headers define an interface and global variables not necessary for that interface aren't part of that interface.
We will strongly disagree there. There is no reason not to consider global variables an interface. A lousy interface, but an interface it is.
safemode wrote:I see no reason here not to continue doing it the way it's been done. game_options is simply an instance, not an interface.
You just haven't experienced the pain of having to grep for a function yet, because the code doesn't even hint where it is.

The reason may be as simple as: we had decided not to introduce any of such externs a while ago. You're pedaling in the wrong direction. If you want to start a new discussion, I'll consider it a waste of time, but you may. Without such a discussion, you're simply going against the flow. Big projects always require us to do something we don't like regarding style, because no two programmers have the same style. Accept it or discuss it, but don't force it.
grepping for a function is all i do in VS. Source files tend to not include headers that they actually use functions from because headers they do include include the one in question ... Or there are headers being included that dont need to be.

My favorite though has to be macro'ing up variables and objects under different names.

In any case, vs_options isn't a function call. It's the actual object and it and all methods related to it are located 1 implementation file and 1 header, as opposed to many of Unit's methods and objects being spread across many files. The idea that tracking down what game_options is is difficult is bs. It couldn't get any more simpler than saying where does this tricky XMLSupport namespace come from ... i guess i'll try xml_support.h and the aptly named .cpp file ... It's not new what i'm doing with vs_options, it's the way vs_options has been used for years. Changing it would be doing something new.

Making vs_options into some kind of global avoiding interface almost certainly will introduce performance impacting methods meant to get and set instances of the class and I dont see the benefit outweighing that cost. I could see if it involved many global variables or the class itself needed or expected certain global variables, but it doesn't. VS only needs the one global variable. Unless we decide in the future that we want to hot swap configs without shutting down. Which once vs_options is complete, we can do. Since there will be no static variables relating to game options.
safemode wrote:Some easily helpful things that can be done is
1. tweaking the cpu opts flags to better and more correctly reflect current gcc opts available for a given arch.
2. Use __restrict__ on all non-overlapping pointer and reference arguments to functions. This can be ifdef'd to "" on compilers that dont support it with no further changes necessary.
I don't think __restrict__ is all that necessary. GCC seems to do just fine without it, with unaided alias analysis. In any case, it might be a good idea, but it's hard actually knowing where to put it. And, since GCC already seems to understand much of VS code regarding aliasing, it might all be for nothing.
actually with gcc 4.7 even, not only is __restrict__ helpful in cases where the function operates on multiple pointer/references sent as arguments but it's not enough on it's own to get the job done. it will only create half vectorized code (not using restrict will usually create even worse code) .. You have to enforce the idea that a pointer/reference is aligned correctly. http://locklessinc.com/articles/vectorize/

The problem is VS's code paths aren't math bound in most cases. So many of the most expensive codepaths in the game simply can't be vectorized. Though we really could help things along by ensuring that the vector primatives are aligned and either single or double precision floating points and used as such throughout their activity in the game. Also, unrolling operations on them is actually worse when there is an alternative to looping operations on them when rocking vectorization.
safemode wrote:That will help the auto-vectorization out without adding any invasive non-portable code.
The other things we can do are more involved.
Rewriting conditionals that follow if(something) var = that; else var = what; into non-conditional writes var = (something) ? that : what; The auto-vectorizer can work with the latter, but not with the former.
Everything else is pretty much rewriting loops to avoid conditionals and be more simple.
Alignment is also rather easy and "portable". In the sense, it can be made portable. Lots of examples lie within ffmpeg code, we could draw inspiration from it.
It's possible that ffmpeg has some portable methods for aligning buffers, but i'm not so sure there is much that have to do with vector math. In either case, our ffmpeg does need updating and while that's getting done it would be the perfect time to look into stealing their alignment tricks.
safemode wrote:So in between getting bored with putting options in one place i'll be sprinkling that around and checking how it changes the build with -ftree-vectorizer-verbose ...
Uuu... didn't know that one.
set it =5 or so and rebuild. watch the sadness scroll by since probably 90% of the loops in the game can't be vectorized. The loops that can, are usually slower when vectorized than not. Almost all of them will result in vectorized and non-vectorized versions (hence binary bloat a bit).

What you want to see is iterations until profitability being less than profitability threshold. Usually it's 1 more or so. Luckily in many of the ones i looked at, i know for sure that it will iterate more than the threshold during gameplay.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote: In any case, vs_options isn't a function call. It's the actual object and it and all methods related to it are located 1 implementation file and 1 header, as opposed to many of Unit's methods and objects being spread across many files. The idea that tracking down what game_options is is difficult is bs. It couldn't get any more simpler than saying where does this tricky XMLSupport namespace come from ... i guess i'll try xml_support.h and the aptly named .cpp file ... It's not new what i'm doing with vs_options, it's the way vs_options has been used for years. Changing it would be doing something new.
I grant you that "options" is one case where there is little confusion. Still, the possibility of declaring it with the wrong type can arise, and confusion would ensue then. There's also the unbound extra work if you wanted to change its definition somehow, and had to grep all VS code to find all of its declarations.
I grant you it's a minor danger, but since we did have an agreement at one point to not introduce any more of those externs (and to in fact start removing them as the opportunity arises), why I ask not move that to the header where it would comply with our guidelines? Is there any benefit aside of you liking it?
safemode wrote:Making vs_options into some kind of global avoiding interface almost certainly will introduce performance impacting methods meant to get and set instances of the class and I dont see the benefit outweighing that cost.
First, I'm not advocating for so much. Only move the extern into the .h. But, for the record, inlining can completely remove the runtime overheads, so even in that case, I wouldn't worry about performance.
safemode wrote:
I don't think __restrict__ is all that necessary. GCC seems to do just fine without it, with unaided alias analysis. In any case, it might be a good idea, but it's hard actually knowing where to put it. And, since GCC already seems to understand much of VS code regarding aliasing, it might all be for nothing.
actually with gcc 4.7 even, not only is __restrict__ helpful in cases where the function operates on multiple pointer/references sent as arguments but it's not enough on it's own to get the job done. it will only create half vectorized code (not using restrict will usually create even worse code) .. You have to enforce the idea that a pointer/reference is aligned correctly. http://locklessinc.com/articles/vectorize/
Ok, just don't add it everywhere without checking usage.
safemode wrote:The problem is VS's code paths aren't math bound in most cases. So many of the most expensive codepaths in the game simply can't be vectorized. Though we really could help things along by ensuring that the vector primatives are aligned and either single or double precision floating points and used as such throughout their activity in the game. Also, unrolling operations on them is actually worse when there is an alternative to looping operations on them when rocking vectorization.
And those that are, require algorithmic vectorization. Ie: turn the problem into large loops over floats or small structures, yes. There you can make sure those arrays are aligned, and there you can profit with little work.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

I dislike having the extern for game_options in a header because then it makes it invisible. You now have this source file that uses a variable game_options and have no idea where to even look except to grep for game_options. At least if the extern is in the source file being used, you can grep it's type...which is far more likely to not steer you in a hundred different directions, especially if the type closely matches an actual filename.
Additionally i dislike having the extern in a header because then it means that every source file that includes that header has to link to the source file that defines that extern. Perhaps we want to make a modder tool and want to access all the game variables. By having the extern in a header we force the people to define that variable, or we define it ourselves in the implementation file for the class and then force them to use that particular name for their instance.
It makes things harder for no benefit.

I'd love for there not to be any need for it to be extern and global, even in namespaces. But hiding externs in headers doesn't help any. It only hurts. Ideally, game_options should be a public member variable within a base class that the game keeps alive for the entire run of the game and that all source dealing with the game would have access to without having to pull in every header in the source tree if it didn't need to.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: 0.6 plans

Post by klauss »

safemode wrote:I dislike having the extern for game_options in a header because then it makes it invisible. You now have this source file that uses a variable game_options and have no idea where to even look except to grep for game_options. At least if the extern is in the source file being used, you can grep it's type...which is far more likely to not steer you in a hundred different directions, especially if the type closely matches an actual filename.
You cite navigation. But forget about maintainance. If you have to change the variable's type, you have to change it in numerous places with your way. It was decided a while ago, that that was a greater evil.
safemode wrote:Additionally i dislike having the extern in a header because then it means that every source file that includes that header has to link to the source file that defines that extern.
It won't link to it unless it really uses it. Declarations in headers is a use case C++ has been greatly optimized for.
safemode wrote:Perhaps we want to make a modder tool and want to access all the game variables. By having the extern in a header we force the people to define that variable, or we define it ourselves in the implementation file for the class and then force them to use that particular name for their instance.
It makes things harder for no benefit.
They already have to use that name, they can't change the name under C++, not without major hacking that's never been guaranteed to work.
safemode wrote:I'd love for there not to be any need for it to be extern and global, even in namespaces. But hiding externs in headers doesn't help any. It only hurts. Ideally, game_options should be a public member variable within a base class that the game keeps alive for the entire run of the game and that all source dealing with the game would have access to without having to pull in every header in the source tree if it didn't need to.
Ok... cite someone agreeing with you. Right now, it sounds way too much that you're only trying to justify your way, I justify "mine", we go nowhere. I quote "mine" because it was a decision taken by many at some point you must have missed.

If it's just preference, play nice, do as we all do, put it in the header. If you have some other meaningful reason, cite sources, a link, something that explains it, and lets not waste any more time writing these useless posts.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Re: 0.6 plans

Post by safemode »

What are you talking about? vs_options has been done this way for _years_ ...it was never changed to a header accessed global variable. I'm not changing things, i'm staying consistent with the way the class has been used all along.

You quote maintenance but an extern in a header only helps the guy who is changing things underneath the people who use it, it doesn't help the people using it. That's like the argument of having all the headers the game uses in 1 header file and having all the source files just include that header rather than including the various individual headers that that particular source file requires. If that's the way you're leaning then we're never going to agree on what is more correct . But i can tell you that there are more than double the instances of extern being used in source files vs headers so if this was some agreed upon thing that went down in the couple years i was away then nobody bothered to actually get the work done.
Ed Sweetman endorses this message.
Post Reply