System Stars Re-Hash

Thinking about improving the Artwork in Vega Strike, or making your own Mod? Submit your question and ideas in this forum.

Moderator: pyramid

safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

ace123 wrote:I'm totally in favor of a good UI for mesher. That code is a mess and I already spent an hour or two to get rid of crashes when I gave it invalid arguments (I couldn't get the thing to work since it kept segfaulting.)

It should only take a day to hack one up in C/C++ using GTK.

If we want to go the route of Python, it might be possible to use Boost to export the conversion functions to Python... or we could make it spawn the other mesher program, but that's annoying since then we need a frontend and we aren't much better off--the idea is to only use the command line arguments for bulk conversions.

struct.pack/unpack is really powerful, so it would be possible to do Obj<->Bfxm in Python, but I really don't know if it's best to rewrite the Ogre conversion in python unless it's really simple to do... and if it depends on C++ code it may as well be entirely in C++.
I dont see why we need to modify the console utility at all (aside from bugfixes). it works. Why can't we just make a simple python Tkinter frontend that executes the console utility ? No need to export functions or do any of that mess. The end result is the same, it requires no additional compilation/dependencies.

Why would you want to do more work in having to create all new code, then have to debug it on all our OS's we support?


EDIT:

Now as for the shader. What i find most lacking from the current shader that nobody has really brought up is our inability to render planets realistically. How will the shader handle water? how will it handle different land types (ice, rock, folliage, lava, dusty, etc). What about gas planets?. How much of this is a result of the shader and how much of it is texture and how much of it is a lack or wrong GL effects usage? Planets tend to seem out of place when looking at the background and dont even get started on system stars. System Stars never look "right", some of that is GL effects issues and coding problems, lack of texturing and such. Not sure if the shader would have any effect on stars since they are so bright and you'd have to get so close to see any details.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:Now as for the shader. What i find most lacking from the current shader that nobody has really brought up is our inability to render planets realistically. How will the shader handle water? how will it handle different land types (ice, rock, folliage, lava, dusty, etc). What about gas planets?. How much of this is a result of the shader and how much of it is texture and how much of it is a lack or wrong GL effects usage?
Planets are Klauss' specialty. He already has a shader for them, I'm not sure at what stage of completion, but my guess would be around 99%.
This shader, CineMut Opaque, is only for ships and stations; --man-made stuff.
System Stars never look "right", some of that is GL effects issues and coding problems, lack of texturing and such. Not sure if the shader would have any effect on stars since they are so bright and you'd have to get so close to see any details.
Do you mean the main stars at systems? I totally agree they don't look right. I don't see why we need textures for them at all, really. In spite of sun-spots and whatnot, stars are so bright they overwhelm... make that burn... your eyes. Total saturation. Stars should look like white disks, regardless of whether they are orange stars or red stars or blue stars. The light they put out could have a color, but looking directly at them you'd see pure, saturated white.
Again, this shader will have nothing to do with stars. I don't even think you need a shader to paint a white sphere. I can't remember the details, but I did look at stars code once, and I think they had diffuse and specular colors and whatnot. Total non-sense, if so; all they need to do is emmit light; --glow color of 1.0, 1.0, 1.0, 1.0; black in diffuse; black in specular, no normal map, no damage map either ;-)

EDIT:
Speaking of sun-spots, people tend to think that sun-spots are phenomena on the surface of the sun. This is not true. They are caused by surface activity leading to huge masses of gas being expelled from the surface; but the surface phenomena are not visible to the naked eye. The faint dark spots we see are actually far from the Sun, flying towards us. The plasma clouds coming our way cool down and partially obscure the the light from the sun behind. They are like miniature nebulas hurtling towards us. That's why we know there's trouble coming when we see "sun spots".
They don't rotate with the sun, either. They fly in a straight line.
Best way to represent sun-spots, in a game where you can fly at high velocities, would be to have like 2 or 3 billboards that cover the star, and as you move around the star, tileable spotty textures scroll on these bill-boards at different speeds; to represent gas clouds at various distances from the star. Simplest solution, though, is not to even bother with sunspots; --they are not too visible anyways.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

Stars definitely need color, there is quite a variety of types of stars and some range from cool red to white hot blue to main sequence yellowish, later on we can even look into handling rarer stars like brown dwarfs and neutron etc. So simple white isn't enough.


The problem we have with stars is dealing with the corona. A simple white disk isn't appropriate for in-system stars when you start getting within 1au. Most stars would be too bright for you to discern between the surface of the star and it's coronal discharges, hence you would not see a clear sphere when looking at a star, and if you were close enough to the point where you would see detail in the corona of the star, it would be moving and dynamic. I dont think we're ready for that type of simulation.

Whatever we end up with, it just looks really wrong to have a discernable corona "halo" and surface when looking at a star.

Edit: also, there is something to be said for the flare effect most people are familiar with when looking at non-mercury level close images of the sun.
Ed Sweetman endorses this message.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

EDIT:
Speaking of sun-spots, people tend to think that sun-spots are phenomena on the surface of the sun. This is not true. They are caused by surface activity leading to huge masses of gas being expelled from the surface; but the surface phenomena are not visible to the naked eye. The faint dark spots we see are actually far from the Sun, flying towards us. The plasma clouds coming our way cool down and partially obscure the the light from the sun behind. They are like miniature nebulas hurtling towards us. That's why we know there's trouble coming when we see "sun spots".
They don't rotate with the sun, either. They fly in a straight line.
Best way to represent sun-spots, in a game where you can fly at high velocities, would be to have like 2 or 3 billboards that cover the star, and as you move around the star, tileable spotty textures scroll on these bill-boards at different speeds; to represent gas clouds at various distances from the star. Simplest solution, though, is not to even bother with sunspots; --they are not too visible anyways.
I dont know who told you about sunspots but i can find no article from nasa or any other scientific body that describes sunspots anywhere near what you have described them as. Which would be some type of plasma eclipse of the sun as it travels to earth.


_everywhere_ sunspots (visible and non) are described as "surface" elements that do rotate with the surface of the sun and can be seen angled and face on showing that it is rotating not simply flying away and crossing the visual path of earth via it's initial rotational inertia. There are even images of solar satellites peering into sunspots.

So i really dont know who confused the bejesus out of you :)

In any case, i dont really care about showing sun spots. I'm more concerned with the corona being represented and providing a believable star. Something you can see and think it's close, and as you get dangerously close, see and think that you're in deep trouble.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:Stars definitely need color, there is quite a variety of types of stars and some range from cool red to white hot blue to main sequence yellowish, later on we can even look into handling rarer stars like brown dwarfs and neutron etc. So simple white isn't enough.
Well, depends on the level of realism you want, and on whether you're using filters. Directly looking at a star would burn your eyes instantly, but even with heavy eye protection it would still overwhelm your receptors. Say you have a red star, where the emmitted light in the red span of the spectrum is twice as strong as green, which in turn is twice as strong as blue. But when you look at it, even with heavy eye protection, the red light is 100 times brighter than the maximum brightness your red receptors can assess. Then the green light is only 50 brighter than the range of your green receptors; and the blue light is a mere 25 times as bright as you blue receptors can size up. But in the final analysis, all of your receptors, --blue, green and red-- are maxed out; so the color you see is still white. That's why I'm saying, you can use light of the correct color for the star for the lighting, but looking at the star directly would probably look white. But again, with very heavy filtering you might bring the brighness down far enough to be able to see the actual color. But with such filtering, everything other than the star would look black.
No sky, no planets, no ships.
The problem we have with stars is dealing with the corona. A simple white disk isn't appropriate for in-system stars when you start getting within 1au. Most stars would be too bright for you to discern between the surface of the star and it's coronal discharges, hence you would not see a clear sphere when looking at a star, and if you were close enough to the point where you would see detail in the corona of the star, it would be moving and dynamic. I dont think we're ready for that type of simulation.
The "corona" we see from Earth is an artifact of atmospheric scattering. The real corona is so thin it's not worth bothering with, IMO. Safemode, stars look wrong precisely because of attempts at doing fancy stuff with them. A full-white glow sphere would look perfectly fine. Only thing to make sure is that the sphere uses a lot of subdivision, like 128 top to bottom and 256 around; to be sure it's really smooth along the periphery.
The other solution, perhaps even better, would be to make stars into billboards and apply a texture with a nice and smooth, well antialised white circle on black background as glow texture; black for diffuse and specular.
Whatever we end up with, it just looks really wrong to have a discernable corona "halo" and surface when looking at a star.
Exactly. It IS wrong.
Edit: also, there is something to be said for the flare effect most people are familiar with when looking at non-mercury level close images of the sun.
Not sure what you mean; what's "non-mercury level"?
If you're talking about lense-flares, those are pretty expensive, AFAIK.
Lots of re-draw.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

You may be blinded by an in-system star when looking directly at it, resulting in it looking white, but this blinding white-ness would fade surrounding details to black too, since the iris would have to contract to it's smallest (be it camera or eyes, and in-game we mostly deal with cameras). This, however is an artifact of a limitation in sight, not something that should be builtin to the star itself, hence it should have color for how it's light looks when less intense, such as reflections off surfaces a decent distance away, and the star, itself, for distances of around jupiter and further away.

"non-mercurian" was in reference to making out details of the sun if you could even look at the sun without going blind, due to being so close.

One instance is if you run the total_war campaign, you are set in the "atmosphere" of a star. This isn't rendered correctly at all How do we deal with this ? Do we make it realistic, and blanket everything in a blinding whiteness assuming the ships all dont burst into flames and everyone die of radiation poisoning? Or do we pretend like the ship's cameras can filter light levels such that the intense light of the sun is scaled down into the proper visual spectrum of the human eye for display in the ship's cockpit?


I'm not sure. All i know is that stars dont look right in game, from a distance any closer than from where it looks like a dot. It's not the existence of the corona that makes it look wrong, or how that corona changes if you have the sun targeted or not etc, it's in how there is a sharp easily discernable disc that represents the surface of the star. The star would be too bright for such details, it should look like a crazy blinding ever increasing ball of light. The "corona" effect is an artifact of the visual equipment trying to deal with the overload of light. It would not look like a clean sphere.

Either you agree that the star should blind you, and hence be white and hence have a flare effect when looking at it (the color of the star would still have to be a color so that reflections and light from the star at an appropriate distance would give the correct lighting), or you agree that the ships are equipped with filtering abilities that reduce the intensity of the star to acceptable levels making it possible to look at the star without being blinded. What that means for out of ship views is another issue.

I'm not talking about lense flare, i'm talking about our fake double drawn corona that can be seen on some stars (especially when they are targeted).
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:You may be blinded by an in-system star when looking directly at it, resulting in it looking white, but this blinding white-ness would fade surrounding details to black too, since the iris would have to contract to it's smallest (be it camera or eyes, and in-game we mostly deal with cameras). This, however is an artifact of a limitation in sight, not something that should be builtin to the star itself, hence it should have color for how it's light looks when less intense, such as reflections off surfaces a decent distance away, and the star, itself, for distances of around jupiter and further away.
You're right, but since computer screens lack the ability to blind you, we're basically having to compress like 10 orders of magnitude of light intensity visual range to a mere two and a half for monitors; in fact, we're having to compress more than 10 orders of magnitude precisely because looking at a star goes far beyond our visual range. I mean, if you'd be favorable to lense flares that are an artifact of lenses, why not be favorable to a much more natural artifact: the saturability of our own eyes?
But since I won't be able to convince you about white, may I suggest a compromise? Use the square root. That way there will be a hint of the official color of the star, while keeping some hint of the eye saturation effect. Thus, if the color of a star is 1.0, 0.81, 0.64 (red star) then use that color for the lighting in the system, but use its square root 1.0, 0.9 0.8 for the star itself.
"non-mercurian" was in reference to making out details of the sun if you could even look at the sun without going blind, due to being so close.
Gottcha.
One instance is if you run the total_war campaign, you are set in the "atmosphere" of a star. This isn't rendered correctly at all How do we deal with this ? Do we make it realistic, and blanket everything in a blinding whiteness assuming the ships all dont burst into flames and everyone die of radiation poisoning? Or do we pretend like the ship's cameras can filter light levels such that the intense light of the sun is scaled down into the proper visual spectrum of the human eye for display in the ship's cockpit?
Best way to deal with this would be to not go anywhere near a star. The sun, as seen from Earth, measures about 1 degree in diameter. If you were to get as close to the sun as from where it measures 10 degrees in diameter, you'd be getting 100 times as much heat, light and UV rays. You'd already die of overheating. And if you went as close as from where it measures 30 degrees across your whole ship would literally vaporize.
I'm not sure. All i know is that stars dont look right in game, from a distance any closer than from where it looks like a dot. It's not the existence of the corona that makes it look wrong, or how that corona changes if you have the sun targeted or not etc, it's in how there is a sharp easily discernable disc that represents the surface of the star. The star would be too bright for such details, it should look like a crazy blinding ever increasing ball of light. The "corona" effect is an artifact of the visual equipment trying to deal with the overload of light. It would not look like a clean sphere.
Exactly. If you really want lens flares and stuff, be my guest; but at its basic, a star would be a saturatingly emissive disc; nothing more. Anything else is a representation of a visual artifact, and the first step would be to de-confuse the issue by changing the name of it from "corona" to "lens flare", lest we forget its nature and purpose.
Either you agree that the star should blind you, and hence be white and hence have a flare effect when looking at it (the color of the star would still have to be a color so that reflections and light from the star at an appropriate distance would give the correct lighting), or you agree that the ships are equipped with filtering abilities that reduce the intensity of the star to acceptable levels making it possible to look at the star without being blinded. What that means for out of ship views is another issue.
Blinding and flare effects are separate things. To simulate blinding, at its simplest, you'd darken the whole image, the whole display, leaving the star white. At its most complicated, you could track the position of the star as it moves along the screen and give it a black trail, representing the areas of your retina that have been blinded. Not sure how easy would be to do this; probably not at all; and this would assume that your eyes are always focused on the center of the screen, which is not necessary, or even intuitive.
Lens flares are lense issues and follow optics rules I've never looked into. To do them right, we'd have to study the physics of real lens flares and model them in a special shader. Most games that have lens flares they are implemented so naively they look ridiculous.
I'm not talking about lense flare, i'm talking about our fake double drawn corona that can be seen on some stars (especially when they are targeted).
I know; that's in the texture. That's why I'm saying the starting point would be to either get rid of the texture or make it a white circle on a black background. Next step would be to introduce lense flares and/or blinding effects, to taste. But those "coronas" in the textures are most certainly not warranted.

EDIT:
Here you go:

Image

Or with a bit of blur, if you like:

Image

Or with alpha background, if more convenient:

Image

Or with alpha background AND blur:

Image

Take yer pick :)
Last edited by chuck_starchaser on Tue Jun 24, 2008 7:21 pm, edited 1 time in total.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

chuck_starchaser wrote: But since I won't be able to convince you about white, may I suggest a compromise? Use the square root. That way there will be a hint of the official color of the star, while keeping some hint of the eye saturation effect. Thus, if the color of a star is 1.0, 0.81, 0.64 (red star) then use that color for the lighting in the system, but use its square root 1.0, 0.9 0.8 for the star itself.
I have no problem with saturation when you're close enough for it to be saturating. You can be far enough from a star in a system where this does not occur. Also, you need the color of the star for reflections off surfaces even nearby as the reflections may not be intense enough to still overwhelm your sensitivity.

We're not talking about shades difference, that likely wont be noticable, but red giants are most definitely red and higher hot blue stars are definitely giving off blue light. The saturation effect should be a consequence of looking directly the star and be a consequence of intensity, not hard-coding the stars to just be white.

Best way to deal with this would be to not go anywhere near a star. The sun, as seen from Earth, measures about 1 degree in diameter. If you were to get as close to the sun as from where it measures 10 degrees in diameter, you'd be getting 100 times as much heat, light and UV rays. You'd already die of overheating. And if you went as close as from where it measures 30 degrees across you'd literally vaporize.
Then we need to fix things so that happens. I'd agree with that. But remember, not all stars are as hot as the sun, not all stars are the same size of the sun. You may be able to get _much_ closer to a waning red giant (well, maybe not physically as close, but as far as how big the star looks ).

Exactly. If you really want lens flares and stuff, be my guest; but at its basic, a star would be a saturatingly emissive disc; nothing more. Anything else is a representation of a visual artifact, and the first step would be to de-confuse the issue by changing the name of it from "corona" to "lens flare", lest we forget its nature and purpose.


The problem is, you can't represent the star like a saturating emissive disc because there's no way to make that visually accurate. There's way too much contrast information between the edge of the star and the surrounding space unless you're using filters, and whether it's the human eye or a camera, you will always get a "coronal effect" (lens flare always seems to connote a bunch of halos) without filters. That's why i said, either we admit that we are filtering the sun, and thus the idea that it's saturating out color data is moot, or we say there is no filtering and there is saturation, and we get visual artifacts.

The alternative I was suggesting to a "flare,coronal effect" was to give the star an atmosphere like planets, but make it diffuse and blend much better so that there is no border between the fade and surface of the star. We can still saturate out colors when looking directly at the star, and when very close to it, but we dont need to bother with any complicated flares.

The problem with faking stars with flat images or anything like that is that we would have to make absolutely sure that the user or no other ships would ever get near enough to a star for such cheating to be noticable.
It also makes eventually having more rarer and interesting types of stars totally impossible.

I much prefer we keep stars as "planets" of obscenely massive scale with the proper glow and specular attributes (no textures for most) and later on when we want to get a little crazy with some fun stars we can have texture ability still available to us. For instance. Making a star than can "flare up" in brightness as it rotates rapidly would be nifty. making a star with an accretion disk should be totally doable. Who wouldn't want to enter a system with a little neutron star pulsing bright every now and then with a dark disk of matter circling it in the not too far off distance.
Such a system may only be survivable by ships with heavy shields, but it could make for interesting strategic functions.

I'm gonna split this little tirade off to it's own topic since it has much less to do with the shader, than it has to do with underlying engine coding.
Last edited by safemode on Tue Jun 24, 2008 7:45 pm, edited 1 time in total.
Ed Sweetman endorses this message.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

I'm not saying the plain white disc method isn't truer to reality than others. I'm saying that if you're viewing the star you can't _not_ have visual artifacts that detract from the clean circle look. If you look at it with your eyes, you get a certain type of coronal effect, and when you view it through a camera lens you get a certain type of flare effect (lines extending vertically and maybe horizontally (spears)). Seeing it clean like that is gonna look more unrealistic than having the visual artifacts get used.

The rest of the scene around the star when looking at it face on ought to get much darker as well. If we're going to say that the star is saturating the camera feeding the ship and such, then light from objects all around the star have to appear darker.

so what we need is a way to deal with gradually decreasing intensity the further away we get from the star, allowing color to eventually show through for directly viewing the star. A decrease in intensity of the light as it reflects from surfaces, allowing the star's true color to show through earlier than as directly looking at it. Because the color has always been there, our instruments are just being saturated at the intensity it's receiving it at. Thus, fairly close objects to the sun would appear to have white light reflecting off it's surface and the star would appear white with no hint of color. The further away you got, the more color would begin to be visible fading from the white and reflected light would be even more pronounced in the star's true color. Eventually the intensity of the light from the star would be low enough so that it's not saturating our instruments at all and we'd be able to see it's true color, whatever that may be, directly.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:
chuck_starchaser wrote: But since I won't be able to convince you about white, may I suggest a compromise? Use the square root. That way there will be a hint of the official color of the star, while keeping some hint of the eye saturation effect. Thus, if the color of a star is 1.0, 0.81, 0.64 (red star) then use that color for the lighting in the system, but use its square root 1.0, 0.9 0.8 for the star itself.
I have no problem with saturation when you're close enough for it to be saturating. You can be far enough from a star in a system where this does not occur. Also, you need the color of the star for reflections off surfaces even nearby as the reflections may not be intense enough to still overwhelm your sensitivity.
Actually, saturation is almost invariant with distance, because although the total light reaching you is 1/4 if you double the distance, the apparent size of the star is also reduced, so that the brightness you see within the star itself is the same. This may not be very intuitive, but it's true. Now, if you're talking about stars at light-year distances, then the picture changes a bit: They are so tiny, at such distances that our eyes don't have enough resolution to see "inside the disk" of the star, plus atmospheric scattering, plus possible dimming by space hydrogen and dust between here and there. But at the distance of a local star, the brightness within its disc does not vary with distance, and saturates equally.
We're not talking about shades difference, that likely wont be noticable, but red giants are most definitely red and higher hot blue stars are definitely giving off blue light. The saturation effect should be a consequence of looking directly the star and be a consequence of intensity, not hard-coding the stars to just be white.
Sorry to contradict, but if you were to look directly at a star, red, blue or whatever, you'd see white. And the differences ARE shades, most of the time. Just look at the sky at night: Do you see any stars that are really red, or really blue? You can barely notice the color differences. So-called "red stars" are merely pinkish. Blue stars aren't even blue at all; the term merely denotes that they are whiter than the average white star. Red giants and brown dwarfs might be exceptionally redder, though. But brown dwarfs aren't too interesting, and red giants are pretty uncommon and short-lived; and in becoming red giants they consume their own planets; --not sure if we have planets around red giants... hope not.
Best way to deal with this would be to not go anywhere near a star. The sun, as seen from Earth, measures about 1 degree in diameter. If you were to get as close to the sun as from where it measures 10 degrees in diameter, you'd be getting 100 times as much heat, light and UV rays. You'd already die of overheating. And if you went as close as from where it measures 30 degrees across you'd literally vaporize.
Then we need to fix things so that happens. I'd agree with that. But remember, not all stars are as hot as the sun, not all stars are the same size of the sun. You may be able to get _much_ closer to a waning red giant (well, maybe not physically as close, but as far as how big the star looks ).
Certainly, but I'm rather skeptical you could get as close to any kind of star as from where it spans your windshield...
Exactly. If you really want lens flares and stuff, be my guest; but at its basic, a star would be a saturatingly emissive disc; nothing more. Anything else is a representation of a visual artifact, and the first step would be to de-confuse the issue by changing the name of it from "corona" to "lens flare", lest we forget its nature and purpose.
The problem is, you can't represent the star like a saturating emissive disc because there's no way to make that visually accurate.
That's true; but the most accurate you can get is with a circle as bright as the maximum intensity your monitor can deliver, which is white, by definition. This is a hard limit that no amount of lens flare, corona, blinding or magic can overcome; it's a limit we have to live with.
There's way too much contrast information between the edge of the star and the surrounding space unless you're using filters, and whether it's the human eye or a camera, you will always get a "coronal effect" (lens flare always seems to connote a bunch of halos) without filters.
Not two cameras will produce the same artifacts. It depends on the optics. The worst lens flare effects I've seen in games is when they try to put more than one effect, like a mixture of coronas, rays and false images. If we want some kind of lens flare we should study the theory of how they are produced, then settle on a particular type, design a shader that does that particular type of flare, --and does it well--, and use it consistently.
That's why i said, either we admit that we are filtering the sun, and thus the idea that it's saturating out color data is moot, or we say there is no saturation, and we get visual artifacts.
Sounds like a plan.
The alternative I was suggesting to a "flare,coronal effect" was to give the star an atmosphere like planets, but make it diffuse and blend much better so that there is no border between the fade and surface of the star. We can still saturate out colors when looking directly at the star, and when very close to it, but we dont need to bother with any complicated flares.
The only problem I see with that is that the atmospheres on planets look like crap. If anything, they've proven themselves NOT to work; so I'd be leery of duplicating them for stars. Atmospheres lose density as an exponential function of altitude, and there's basically nothing you can do to a sphere at an arbitrary distance from the surface to make it look like anything but a sphere at an arbitrary distance from the surface along the periphery of the planet (or star).
The problem with faking stars with flat images or anything like that is that we would have to make absolutely sure that the user or no other ships would ever get near enough to a star for such cheating to be noticable.
True; and I'd say the best solution would be to introduce some simple heat modeling and sound alarms in a ship if you get too close to a star, and to vaporize the player two seconds later.
It also makes eventually having more rarer and interesting types of stars totally impossible.
I'd say, quite the opposite: If we use a shader to compute atmospheric gradient and whatnot onto a billboard, we could go as sophisticated about it as we want; whereas if you have like a sphere overlaying a sphere, the math for texturing the atmo, specially towards the periphery, becomes intractable.
I much prefer we keep stars as "planets" of obscenely massive scale with the proper glow and specular attributes
Stars are perfectly black in diffuse and specular, as far as we are concerned. Hardly any light reaching a star would "bounce back"; it would be absorbed, and perhaps re-radiated; but the emissive intensity is so great that whatever light would be returned is insignificant by comparison. IOW, if you were to look at a binary star, you'd see no mutual illumination effects.
(no textures for most)
Or for any.
and later on when we want to get a little crazy with some fun stars we can have texture ability still available to us. For instance. Making a star than can "flare up" in brightness as it rotates rapidly would be nifty.
We can do this easier with a shader than with a texture, though.
making a star with an accretion disk should be totally doable.
Certainly. That would be interesting.
Who wouldn't want to enter a system with a little neutron star pulsing bright every now and then with a dark disk of matter circling it in the not too far off distance.
Or the other way around: A "dark star" (neutron star or black hole) with a sun-bright accretion disk.
Such a system may only be survivable by ships with heavy shields, but it could make for interesting strategic functions.
Watch out with those accretion disks, though... We detect neutron stars by the X-rays of their accretion disks; and we detect black holes by the Gamma ray emmisions of their accretion disks. Accretion disks are nothing to sneez at ;-)
Last edited by chuck_starchaser on Tue Jun 24, 2008 8:48 pm, edited 1 time in total.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:I'm not saying the plain white disc method isn't truer to reality than others. I'm saying that if you're viewing the star you can't _not_ have visual artifacts that detract from the clean circle look. If you look at it with your eyes, you get a certain type of coronal effect, and
What makes you so sure of that, Safemode? The corona effect we see down here is due to atmospheric scattering. I've never been in space, but I do believe you'd see the sun as a white circle with a very sharp edge, except for possible lens flare effects.
when you view it through a camera lens you get a certain type of flare effect (lines extending vertically and maybe horizontally (spears)). Seeing it clean like that is gonna look more unrealistic than having the visual artifacts get used.
It totally depends on the lenses and the optics. Lenses that are not perfectly smooth, or have water condensation on them, tend to produce rays. Other lens configuration produce ghost images.
The rest of the scene around the star when looking at it face on ought to get much darker as well. If we're going to say that the star is saturating the camera feeding the ship and such, then light from objects all around the star have to appear darker.
That's an eye effect: Your irises contract quickly to block the light, and so everything else appears darker. This should be VERY easy to implement.
so what we need is a way to deal with gradually decreasing intensity the further away we get from the star, allowing color to eventually show through for directly viewing the star.
No. Ditto; the brightness within the disk doesn't change with distance; only its apparent size does, and therefore the total light reaching you.
A decrease in intensity of the light as it reflects from surfaces, allowing the star's true color to show through earlier than as directly looking at it.
Surfaces should reflect the true color of the star; it's only when you look at the star directly that you should get saturation whitening.
Because the color has always been there, our instruments are just being saturated at the intensity it's receiving it at. Thus, fairly close objects to the sun would appear to have white light reflecting off it's surface and the star would appear white with no hint of color.
Well, this is true; you could get saturation with reflected light, as well; but this is already modeled anyways. Shaders do saturate.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

The thing is, I've never seen a non-flared/optically distorted view of the sun from space, from on earth, or anywhere. So despite it being a flaw in the optics, it's a flaw that exists whether you view it with your eyes, or via a camera, thus having the game not produce something would look "wrong" even if it's avoiding a flaw in our current means of producing optics.

I dont think the flat image for the star rather than the mesh and such will solve as many problems as it'll create. Most of the problems granted only occur when you get fairly close to it, but some may creep in if we want to do disks around stars, since the disk would have to rotate about something and really dim stars may not saturate or be radiating enough energy to keep ships from getting very close.


I dont see why we can't keep the sun a mesh, have a white texture for most/all non-interesting stars and a crazy intense colored glow texture to indicate the hue of the star, and then go on from there as far as shading and what not.

Now , the interesting stars could be textured a little more creatively, perhaps pulsing between very cool (maybe even non-saturated) to super bright, brown/red dwarfs, etc. Maybe lightly sprinkled throughout the universe to provide some interesting eye candy and show off how flexible the engine is in recreating star systems.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:The thing is, I've never seen a non-flared/optically distorted view of the sun from space, from on earth, or anywhere. So despite it being a flaw in the optics, it's a flaw that exists whether you view it with your eyes, or via a camera, thus having the game not produce something would look "wrong" even if it's avoiding a flaw in our current means of producing optics.
Yeah; I understand; I'm not saying "let's not flare it"; all I'm saying is that semantics ARE important; let's call a rose a rose, and speak of lens flare rather than of "corona". If we use the term "corona", then we don't know what we're talking about, and therefore won't even know what we're trying to achieve; whereas if we call it "lens flare", then we don't know what we're talking about either, but at least we can hope to find out. ;-)
I dont think the flat image for the star rather than the mesh and such will solve as many problems as it'll create.
I guess we're going to have to agree to disagree. Well, there's nothing wrong with having a white sphere for the star itself, but if you want any kind of "atmosphere", you're best off projecting it onto a billboard plane. Having a second sphere like we do now with some planets looks ridiculous. You can see the line of the second sphere, towards the horizon, and there's no way I can think of to get rid of it. Well, the whole idea of using a second sphere for atmo is mathematically absurd, and it looks it: as if the atmosphere was some kind of glass bubble. Totally unwarranted, since there's nothing it does for us that can't be done in the shader onto the planetary sphere itself; except at the periphery, where instead of helping it hinders the representation of gradual reduction of density with altitude.
I dont see why we can't keep the sun a mesh, have a white texture for most/all non-interesting stars and a crazy intense colored glow texture to indicate the hue of the star, and then go on from there as far as shading and what not.
Sure we can; it won't hurt anything to keep stars as meshes; I just got really scared when you mentioned atmospheres, tho. If you want some kind of atmosphere (thin outer layer) you're best off projecting that thin peripheral blur onto an intersecting billboard, rather than any kind of 3D mesh.
Now , the interesting stars could be textured a little more creatively, perhaps pulsing between very cool (maybe even non-saturated) to super bright, brown/red dwarfs, etc. Maybe lightly sprinkled throughout the universe to provide some interesting eye candy and show off how flexible the engine is in recreating star systems.
Frankly, I've no idea what pulsars might look like. Some have extremely short oscillation periods, like minutes; and then if such periods happen synchronously around the whole star, they would arrive at you at different phases depending on the distance between you and the various parts of a star. From the nearest surface you get an early signal. From the periphery of the star you get a delayed signal. So you might see cyclels of brightness starting at the center of the disk and spreading outwards like ripples. We could do that in a shader too.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

neutron stars are cool too. Besides the accretion disk, a neutron star is fairly dark visually, since it's burned out just about everything it can burn. What is neat about it is that it's gravity is great, but not black hole-esque, meaning the crap it pulls in to it stays on the surface of the star. This mass of material builds up until the pressure is great enough to begin fusing it. Hence, a chain reaction occurs on the surface of the star and it explodes. This could take years to centuries depending on how close the nearest star is. The explosion could be mediocre, or extremely brilliant.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:neutron stars are cool too. Besides the accretion disk, a neutron star is fairly dark visually, since it's burned out just about everything it can burn. What is neat about it is that it's gravity is great, but not black hole-esque, meaning the crap it pulls in to it stays on the surface of the star. This mass of material builds up until the pressure is great enough to begin fusing it. Hence, a chain reaction occurs on the surface of the star and it explodes. This could take years to centuries depending on how close the nearest star is. The explosion could be mediocre, or extremely brilliant.
Ehm... Neutron stars are far beyond "fusing". They are made of densely packed neutrons; so you can think of them as being a single atomic nucleus. Fusion implies lighter elements becoming heavier elements; but neutron stars can't "have elements", whether light or heavy, since their gravitation is so strong that any matter falling onto them gets smashed to a quark soup just by the force of the fall. I don't think neutron stars undergo any further evolutions beyond being neutron stars; unless they happen to acquire enough mass to become black holes.
http://en.wikipedia.org/wiki/Neutron_star
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

you are wrong there. Neutron stars dont break matter down to quark soup. Neutron stars are neutrons themselves, hence, their gravity can't break down subatomic particles or they, themselves would be such particles.

Neutron stars ciphon matter from nearby stars or their own exploded nebula caused by their going super-nova and such. This matter builds up on the surface of the star and does fuse when reaching a critical limit. This causes starbursts that put out amazing amounts of energy in a short time, sometimes lasting only a few seconds, sometimes lasting a day.

here's a little blurb about it http://imagine.gsfc.nasa.gov/docs/scien ... lsars.html

They dont get into the bursts though, just the pulsar aspect and hint at accretion disks.

http://csep10.phys.utk.edu/astr162/lect ... utron.html

that goes into detail on the bursts.

Wikipedia sucks :)


edit: Consequently, most of the energy is in xrays, but plenty would still be visible light to "pulse" the brightness of the star in-system. Much different from a pulsar, which has beams of intensely visible and invisible radiation rotating with the star.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

You know how I'd do stars? With HDR.

Stars wouldn't be rendered at the normal "0-1" plane, but on the HDR plane (a render texture). Then filtering gets applied to make a halo, and on the screen it goes.

But that needs sooo much coding that I don't think it will happen until we start using a library - say Ogre - to do all that coding.


BTW: Why do you say the wiki sucks? This page seems very clear about the matter, and similar to what you say. The surface are neutron-heavy (but not neutron-alone) nuclei, then things get more and more degenerate until you reach the core and "nobody knows".

But... they're not one big nuclei, quite clearly. Science magazines tend to promote that view of neutron stars however... I remember reading about that a lot of times, and had I not have read that wiki article before, a long while ago, I might still think like that.

BTW: Lets stop all this astrophycisist ramble (very interesting ramble, but follow my drift) and start looking at this as artists! Nobody, not even astrophycisists know how those things look up close. Let's get creative, following the realistic rule of: it's very bright, it will burn your wings if you get too close.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

The problem is with accurate portrayal of stars from an artistic standpoint is that they're all basically the same white sphere/ellipsoid objects visually. There's some color hue involved with how hot or cold the star is, but you dont see that till you're either very far away or off the light reflected from the surfaces of ships and such.

Basically, they're mostly not interesting at all visually, or in gameplay.

The only interesting ones, are the ones that dont behave like the rest. Neutron stars have significant accretion disks if they're nearby something, so this could have artistic merit. The star itself is interesting visually, as it's about the size of a base, and it's dynamic in that you can have it spinning and cause a pulsar, or you can have it bursting as it accumulates matter. You even get magnestars, which would offer some cool effects in-game such as scrambling instruments and such due to the insane magnetic field. They would also cause gamma bursts that we could make deadly to all but the most shielded vessels :)

Brown dwarfs are interesting as well, as they'd still be dangerous to get very near, but they're much more approachable than a normal star, and they can probably have a visible surface, since they no longer produce light of any significance.

Who knows how we'd do a black hole, but that would be an interesting system.

In addition to those, you got your red giants, and new stars.

That's about as interesting as it gets artistically, the vast majority of stars would just end up being white disks of varying size with some sort of color associated that starts becoming visible when saturation levels decrease sufficiently. Sadly boring, but that's how it is. Hence, my interest in the non-boring examples, though yes, such things would require code modification to handle the added effects, and i think such changes are needed.

We need to properly represent nebulas, this would be seen as the entire system you enter as being "foggy/dusty" of varying densities, not little units that look like crystals. Perhaps multiple colors given off by different elements in the plasma

We need to then be able to represent some sort of set of star effects. Flare ups would be nice, from a minor intensity to system killing :). That would involve both light and radiation.
Then we can have a property that dictates how intense a star's radiation and magnetism is. This could be given as a bitmask where the first half is how intense magnetism is, the second half how intense deadly radiation is. We wont care what radiation it is, just that it's deadly.

Gravity is the big killer though. Everything has to move to not get sucked in, and the rare stars are the worst (mostly). It would destroy the effectiveness of these systems to gameplay to make gravity overly realistic in strength and effect. So maybe we'll leave things to radiation and such to keep people from venturing too close to a star and just flying away without any apparent effect of gravity playing.

In any case, besides stars, our accretion disks are kinda lacking as well.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

LOL, I was joking when I said "quark soup". It's the Italian ancestry from my mother's side :D
I knew I was simplifying the matter, though I didn't know about fusion on neutron stars; that's new to me.

Anyways, Klauss, I agree this has veered a little (just a little) too much into science; but we shouldn't get "too artistic". Hey, if you could ask Picasso to paint a star, who the hell knows what he might come up with?; and it would be un-arguably artistic; but probably not what we want. This is a space sim engine for space sim games, so we're talking "art" in a narrower sense: namely Superrealism. Superrealists often hone their skills by working off photographs, for a few years, trying to make paintings that are indistinguishable from photos; and later on they begin to venture into making paintings entirely out of their imagination. We have little use for Cubists or Impressionists here. ;-) Knowledge of the subject matter is essential for the type of artists we have (or should have). Space artists find out all they can about an extrasolar planet before attempting to paint an "artist's concept" of it. That's the kind of artists we want.
If it were otherwise, there'd be a huge conflict: Imagine you make a shader for planets with clouds that cast shadows, but then some artist says "I want clouds that don't cast shadows for MY planet". The reason we can all work together is that we know what we want: We want stuff that looks as realistic as possible.
And there's a lot of room for "artistic input": There are races and factions that have well defined styles for ship-building; but within those guidelines there's plenty of liberty to explore. But the final result should look convincing, like the ships are made of recognizable or plausible materials and whatnot.
Same thing with stars. We only know what G-types look like up close; the rest we only got theories for; and a lot of room for imagination; but that doesn't detract from the fact that the more we think and research about the subject, the better our representations will be.

EDIT:
Safemode; why should stars be "interesting". Everything doesn't have to be interesting. I find stars interesting in many ways; but not necessarily in visual terms; and making them too sexy might detract from their intellectual appeal... :) Anyways, I don't think players are going to care too much about whether the edges of stars should look razor sharp or somewhat blurred, and if so how much. Most players will pay 100 times more attention to planets. However, I agree that stars should at least look plausibly correct, which currently they don't.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

safemode wrote:Basically, they're mostly not interesting at all visually, or in gameplay.
I wholehartedly disagree.
What are stars, if not lightsources?
What is a picture, if not a light map?
What is a picture then, without a star?

Stars are, thus, the source of all beauty. Not only because they've spawned all life in this universe (according to current scientific theories - ie: not teological theories or ancient scientific theories), but also because they fill the void with light, and light is what creates beautiful landscapes and beautiful starrise (ie: sunrise), and beautiful starsets.

I'm not (just) being poetical... I'm illustrating a point. Stars create beautiful lighting conditions. We therefore should concentrate on those lighting effects (I was kind of going there with the HDR comment BTW).

Light crossing an asteroid belt in the distance would be scattered much like it is through clouds (and planetary rings). Light in a nebula would create all kinds of beautiful shades and shadows. A planet, for instance, within a nebula (lets argue against the feassibility of that later) would project shadows. Light from a star hitting the ship's cockpit camera lens would reflect off interior elements thus creating lens flares (yep, that's how they happen - light gets reflected/refracted off lens edges or perhaps bleeds off into the tube's interior surface and wanders off in different directions than it should in an ideal lens system).

Light-emitting flares also have lots of artistic possibilities, not to mention superflares, which also have gameplay possibilities. Coronal mass ejections, although perhaps only in scripted missions since they're not common at all... so so so many things. Stars are alive, one could say, and are incredibly interesting if one can have an unglittering glimpse at them.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

hey i wish we could do dynamic stars and such, but the effects of mass ejections wouldn't be visible anywhere near the star's surface, it's too bright. You'd only see the plasma much further out as it's cooled and left the area of total light saturation around the star, and such ejections would be deadly explosions...

I think it would be awesome to get accretion disks rendered as they should. Same with nebulas. But the star itself has little to do with it outside of the fact that it's present and has a given color, intensity, and proximity. You're not going to see the effect of all the activity on a normal star's surface. It's all saturated out and so far away that even large activity isn't noticeable.

That's why i said, the only interesting stars as far as art and gameplay is concerned are the ones that are very rare and very dangerous or dead. Brown dwarfs are dead stars. Neutron stars and it's flavors are interesting, both in their lighting effects and in gameplay aspects.

The rest of what you're talking about really has little to do with star rendering and everything to do with getting asteroid belts/fields, dust clouds (accretion disks), and nebulas to be rendered correctly so that it reacts with our light source the way it should, and interacts with the ships the way it should.

If i thought we had the graphics power to handle simulating complex asteroid fields/belts and realistic accretion disks and free flowing responsive nebulas i'd be all about getting that stuff in the game. But i dont think we can handle those effects, while at the same time simulating everything else in the universe involving the units. Kinda makes you wonder what a space combat game would be like if it was first designed around creating an intensely beautiful and dynamic universe ....then begin to add ships and civilization.

ps. I think before we even consider being able to render such scenes alongside of the whole factions, economy, ships etc, we have to get the game threaded.

Since no one execution path takes that much time, it's very hard to decide what to thread to get any realistic gain in performance after all the locks and such. So i'm gonna propose we divide and conquer. Basically, each "system" runs on it's own thread. The way the locking would go would be that all systems will block until our current system's thread says it's ok to continue, this way jumps into our system, dont cause our system to hang when we dont want it to. Now, when we want to jump to another system, the "block" will occur when the screen is saturated so it wont be noticable. Then priorities change, but threads dont have to be created or joined. The systems are replaced and shuffled around, but they read data in from the disk on their own thread and simulate on their own asynchronously (until they want to jump) with our current system and others.

So a system per thread (not every system in the game, but all actively simulated systems, which i believe maxes at 8). What say you? We avoid the GL issues, we avoid thread locking (cept during jumps), and we can free up cpu for simulating the currently rendered system pretty significantly.
Ed Sweetman endorses this message.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

re threading, there are a few useful patterns for VS: double buffering and message passing.

The first one would mean creating entirely new render queues or unit lists each frame so that the other threads can go on reading the old ones (not necessarily copying all fields, the graphics system only needs mesh & matrix, etc, etc...).

The second one sends updates (eg matrix updates) to a shared, lockless queue. The hard part about the queue is handling redundant updates there, sadly enough :(

I'm already thinking on how to thread sound. It will be one of the first things to get threaded implementations (though the first functional version won't be so). We can use the lessons learned there to thread other subsystems.

re asteroid belts having nothing to do with stars... kind of, but not. One of the best ways to render scattered light is to render the star to a render texture and then filter that image before using it to render the objects that scatter light. Ie, the star itself is a white or very very bright disc, but then again any image is a bunch of pixels. After some convolution, that disc may become a beautiful part of the spacescape. That's why I say: the very first step is having the star rendered onto an HDR target so that the overly bright light can properly bleed through occluding objects. Like these shots
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

safemode wrote:I think it would be awesome to get accretion disks rendered as they should.
Indeed. One type of stellar object that is entirely missing from Vegastrike is young stars with large, dense, protoplanetary accretion disks. That's something I wouldn't even dare try make a shader for, but maybe Klauss would, as he's done self-shadowing for clouds. This would involve a lot of volumetric effect faking and self-shadowing. A protoplanetary disk would glow as bright as the star, in the proximity of it, and cool down outwards. Past the "dew-line", (about the distance of Mars's orbit for a star the size of the Sun), the albedo changes; --suddenly becomes brighter, as water there exists in the form of ice.
Same with nebulas.
Oh noooo! ...
Well, not only are all kinds of volumetric effects very difficult things to achieve in graphics, but nebulas don't exist. Well, not the way they are depicted in cheap sci fi's and games. Nebulas are many light years to hundreds of light years in size. If you were inside a nebula you wouldn't notice it. They only become visible when seen from far away, when your line of sight catches the full, light years thickness of it.
But the star itself has little to do with it outside of the fact that it's present and has a given color, intensity, and proximity. You're not going to see the effect of all the activity on a normal star's surface. It's all saturated out and so far away that even large activity isn't noticeable.
Exactly.
That's why i said, the only interesting stars as far as art and gameplay is concerned are the ones that are very rare and very dangerous or dead. Brown dwarfs are dead stars. Neutron stars and it's flavors are interesting, both in their lighting effects and in gameplay aspects.
Well, stars aren't too relevant to game play; they are just there. If planets are illuminated from one side, it's logical to expect to see a light source, and that's what the presence of the star fulfills; nothing much else. Neutron stars and black holes with incandescent accretion disks might look very nice; but a) there's only 2000 or so neutron stars in our galaxy, out of 100 billion stars; so Vegastrike's universe might qualify for ONE neutron star; don't start throwing them all over the place :D and 2) other than the fact they fry your brains in two seconds flat with x-rays from the accretion I don't see how they'd be more interesting to gameplay.
ps. I think before we even consider being able to render such scenes alongside of the whole factions, economy, ships etc, we have to get the game threaded.

Since no one execution path takes that much time, it's very hard to decide what to thread to get any realistic gain in performance after all the locks and such.
If properly done, the effort of multithreading the engine would promote encapsulation and reduction of dependencies. Even if there was no other benefit, multithreading would be a good pursuit just on the basis of the neatness it demands. But using locks is NOT neat. Well, some locks have to be there; but the best multithreading designs are those which avoid them as much as possible: NO shared data between threads (each thread is like a machine working with its own materials and products, totally independent of the rest); inter-thread communications implemented as a single message channel or fifo per any producer/consumer threads pair, using non-variable-length messages all of the same structure; like a conveyor belt taking product from one machine to the next, in an assembly line; like a pipeline. Not two threads should work on or off the same data. There was a webpage about multithreading for games, and the author had written a multithreaded game engine, to demo good multithreading techniques, which engine didn't have a single lock. Super fast. Message queues are inherently thread-safe AND fast; whereas the cost of using locks is enormous; not only because of the ton of code the lock takes to just pass through, but the fact that whenever a lock actually locks a thread, it forces a thread change, and thread changes imply tons of cache misses until the cache is completely refilled with data that pertains to the new thread. The idea is to make threads so independent that the only times you get thread switching is when the OS decides; or when all its work is done and it voluntarily gives up control.
So i'm gonna propose we divide and conquer. Basically, each "system" runs on it's own thread.
That's a text-book example of BAD use of multithreading. You don't want to divide the data between threads. You want to divide CODE between threads. If you divide data between threads, then each thread is as big as the whole engine.
If instead you divide the *code* between threads, then each thread is a smaller piece of the code, and in being so it incurs less misses in code cache. Data cache doesn't matter because you can prefetch it.

EDIT:
IOW: The trick is to divide up the code into sections that can function independently, except for a steady stream of data passed via a message queue. Each thread's state (the data it works with) is private and invisible from any other thread. To avoid threads "snooping" into each other's data at any cost. If the state of one thread MUST depend on some data belonging to another, that data should be passed as messages. NEVER read directly. If two threads need some common data, they each should get a private copy. The only data that meshes can share is read only data that no thread modifies, such that locks are not needed.
safemode
Developer
Developer
Posts: 2150
Joined: Mon Apr 23, 2007 1:17 am
Location: Pennsylvania
Contact:

Post by safemode »

That's my point. There is no part of the code that uses a lot of cpu. We're fast. But we can't scale for crap without threading out aspects of the game so that they can take place during the graphics drawing portion of our timeslice.

None of the individual functions are slow enough to make threading a viable use. None of the functions that accumulate a significant amount of time keep to their own objects/subsystem.

We need to run parallel to the graphics frame, so the graphics rendering frame _must_ be threaded. At the very least. We ought to be able to run parallel to the physics frame too, but there is no way to do that without sharing a _TON_ of data. My idea was to avoid data sharing by threading System object simulation ...but maybe that's impractical.
Ed Sweetman endorses this message.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Well, in fact, that sounds to me like the best idea you've come up with yet, for multithreading: Just separating the graphics. The graphics code is independent; and is a consumer only, receiving a steady stream of commands. Even if its code is a small part of the engine (code space wise), it probably represents a nice chunk of the processing time, and it probably incurs some hardware-related start/end of frame lockouts or waits that you don't want to propagate to the rest of the code. Sounds to me like a perfect first candidate for moving to a separate thread.

And I would say, this would be the right way to proceed for multithreading: Find a good candidate and separate just that, do a lot of testing, make it a feature for a new release. Once it's done and working 100% bug-free, consider a new section of code, such as the sound, and separate that into its own thread. And so on. But the first thread should be the easiest and most obvious candidate for separation, and I think the graphics should be it.
Something like physics or AI is very hard, due to a lot of dependencies, and such things should be separated last, if at all.
Post Reply