System Stars Re-Hash

[klauss]

Sound is easier chuck, and it's already being refactored. So it's a better candidate.

Actually, I'd separate concepts. If I've learned something by coding in python, is that it's tidier to write threads in concepts rather than anything else. When you have an aspect of the code that is inherently parallel, you thread it. Python has that nasty GIL that hinders all real multithreading, so the only reason to do multithreading if you're not doing heavy I/O is tidiness, threaded code is much easier to read (when carefully written) than singlethreaded code that has to multitask. Since each thread's code only has one task at hand and needs not worry with multiplexing concurrent tasks.

But then again for performance you have to thread for CPU units - ie, if you have two independent cores, you can spawn two threads to do one half of the same job if at all possible and it will accelerate things. If you have two virtual cores (hyper-threading) you can thread execution units: one thread consumes bandwidth, the other consumes ALU time, for instance.

Making a game engine thread as in the second paragraph is hard. Almost all tasks in a game are bandwidth-bound. One actually has to create ALU hogs on purpose to be able to fully utilize the CPU - how? if you know you have free ALU resources, you can do more computations somewhere. Ie: better sound effects, better physics interpolation, better deformable meshes, better particle systems, or something like that. Usual physics tasks are not ALU hogs, they use BSP trees or 3D hashes, accessibility graphs or huge AI structures, all incurring lots of cache misses and thus become bandwidth-bound.

So lets start by threading concepts I say, and optimize later.

[chuck_starchaser]

Sound is easier chuck, and it's already being refactored. So it's a better candidate.

Yeah, okay; same principles apply as for graphics; and it's probably smaller.

First step in moving to multithreading, btw, should be to make sure that all libraries are multithreaded. Otherwise any function with static variables that's called from more than one thread would likely crap out.

Python would probably have to be entirely contained in one thread.

[safemode]

Sound is easier to thread, but it's also _much_ less of an issue as far as cpu time. I dont run with sound or music at all when playing VS. Thus the performance problems i see will not benefit at all from threading sound. What i see is the game is waiting for the graphics draw to the tune of almost 50% of my cpu isn't getting used to simulate units. What this means is i have to do all simulation in the remaining 50% of my cpu and sometimes there's not enough time and i start seeing skipping frames or low framerates.

The _main_ underlying culprit in cpu usage is the graphics frame. The cpu just sits and has to wait. Sure, sound does that too, and both need to be done, but you wont see any real reduction in stuttering and reduction in time to load systems and such until you thread away graphics.

[safemode]

I would not bother with threading python. Keep python on the main process. Use Boost threads for whatever chunk of code we're trying to thread and make that the end of story.

The main problem with threading concepts is that concepts in VS share code all over the place. Nothing concept wise outside of discrete subsystems like sound and graphics, will be threadable without serious locking and issues with deadlocking. The other problem with concepts and threading out only aspects of the physics frame is that everything that goes on inside the physics frame is _extremely_ fast. The time it takes to push something to a thread and wake it up or then wait (if needed) for it's output later on is more time than just letting it execute serially. It's a lot more complicated that just looking at the code and thinking if "this can be done in parallel to that". You have to consider the total time what you're parallelizing takes and if it's even worth making parallel. Thread overhead is very real.

We're not going to end up with hundreds of threads or even a dozen, what we'll end up with is maybe 3 or 4 threads. 1 main process, 1 graphics, 1 sound, and maybe something else like AI.

I really dont think we'll be able to get any more in-depth than that without completely rewriting the game around a threading model and completely ditching normal python for stackless python so we can massively thread that too, and then rely mostly on the operating system for scheduling everything rather than our own internal scheduling mechanism. It would have been nice if threading was utilizing from the beginning, but that's not the case.

[chuck_starchaser]

Well, Klauss wasn't making claims about comparative usefulness; simply said it would be easier to start with sound; and I agree the second step in multithreading should be the easiest (first step being, checking the libraries in use); and although graphics sounds very promising, sound could be a perfect learning experience. In any case, I would insist that, first things first, making sure that thread-safe versions of libraries are used when possible; and, when not, that the fact is well documented somewhere, is quite a job; and it might magically fix problems too. Thread-safe libraries are often less buggy than single-threaded versions, as the latter are often older and not as carefully maintained. Any software for which no thread-safe version is available must be used in only one thread. So, identifying such libraries will help seeing more clearly where the cut lines ought to be when separating the code into threads.

BTW, persuading the OS kernel in scheduling you threads the way you want is not as difficult as it might sound; just the implementation is a bit different in that you cannot *tell* the OS how much time to give a thread, but you can make less urgent threads voluntarily give up control by blocking. In Windows, a thread simply executes a Sleep(0). Sleep(0) means "I'm done, for now; you can reschedule me at your convenience". So, the most time-critical thread can assess its own cpu needs, based on whether its own tasks are getting done in time; and if not it can message less important thread saying "can you give me more time please". The less critical threads, when they get this message, the try to do a minimum of work and execute a Sleep(0) ASAP.
You can even schedule the order of thread execution, by having threads assess whether or not it's their turn to execute, and executing a Sleep(1) right away if it's not.

[safemode]

we dont need to worry about thread safe libs, because we're still only using them from 1 process, that processing being the thread now, instead of the main process. It's not like we'll be using multiple threads with one library.

So in that sense, we should be perfectly fine. There wont be any other threads we have that will be talking to the audio libs. So it's the same as if it wasn't threaded.

Same with graphics rendering, everything involved with talking to GL will be confined to the graphics subsystem, and that will all run on the same thread.

Python is the only thing we really have to be concerned with from a threading standpoint, since it does have a threaded version and a non-threaded version. But we're not going to thread python across multiple threads, that would be a nightmare.

Allowing the OS to schedule our threads (on a scale diving deeper than i described) means we're at the mercy of each OS's schedular, and they may all behave very differently, hell, linux has multiple schedulers available and that's not even considering previous versions that people still use. Troubleshooting issues now becomes an exercise in kernel troubleshooting for every OS.

We should keep threading extremely simple, extremely limited and not try to push it because we haven't written VS from the ground up to use threads and it is gonna quickly get extremely ugly and hard to maintain if we try and retrofit too many places to be threaded. The gains diminish rapidly when you get beyond simply threading audio and graphics. They would diminish to the point where i really dont see how it would be worth it going beyond that except _maybe_ for AI functions beyond that of the individual unit.

[chuck_starchaser]

Good points; but don't forget that while sound libraries will be used only by the sound thread, and ogl by the graphics thread; there may be a number of low level libraries shared between threads. Boost is perhaps one of them, but boost is as thread-safe as it gets, for the most part, so, not much of a worry there. I'm not familiar with the code enough to come up with anything, but just for a general idea, if you log messages to stdout/stderr from more than one thread you could be in trouble; I'd make sure the audio and graphics threads don't even include the header files for logging messages. 3D vector math fuctions that might be called from the main thread and the graphics thread also come to mind; though I'm sure you're not planning to do math inside the graphics thread, but I mention it just in case.

[safemode]

Yea, it would be ideal for the graphics thread to simply pull some state variables from the unit or whatever we are drawing and play with GL. All the math should have been done by the physics frame. It should be analogous to playing sound in the sound thread. All the work is done by the decoder and once we are ready to push to the sound card we shouldn't have to do any more work. The physics thread is our decoder so once we're ready to push to the graphics card, our work should be done.


In any case, like klauss said, audio is the easiest, mostly because it's so completely optional, hence it lends itself to being threaded. But I really think we wont see any real gains until we do the graphics subsystem. I also think we need to fix some other things regarding gameplay before we worry about that other stuff.

When we talk about threading the graphics subsystem, we're really venturing into feature work when there is a TON of bugfix work that still needs to be done across the board. With all the work going into 0.5.1, we're probably going to have to make 0.5.2 a bugfix only release. A level of control unheard of in VS release history

[klauss]

In any case, like klauss said, audio is the easiest, mostly because it's so completely optional, hence it lends itself to being threaded.

Now, don't take it wrong, but...
Optional your ass!
Sorry, got carried away. To me sound is more important even than graphics. I'd happily have the sound thread eating 50% CPU time if it gives me an enjoyable experience.

Anyway, about libs being multithreaded as chuck says, he doesn't know how true it is. I was going to write the sound system threaded from the beginning until I realized: vsfilesystem isn't threadsafe!

So I got really depressed and then realized I should move on - make it single threaded and later worry about threading it.

[chuck_starchaser]

Doesn't boost have a thread-safe, portable filesystem? Why do we need a vsfilesystem?

[safemode]

vsfilesystem is used to grab files from various datadir directories that get defined by hard-coded paths and via the config file (once we find it).

I think you can bypass it via boost filesystem code once you read in the path to the datadir. or moddir (though, nobody uses the moddir anyway).

in any case, by optional, i meant you can have it completely disabled and you can still play the game. Meaning, it's not integral for the game to function. You can't play the game without visual rendering. You can't play the game without physics processing, You can't play the game without AI processing. You can play the game without audio.

Audio also lends itself to threading, the audio driver is basically a work queue getting data pushed to it from all angles, our audio subsystem would just be a workqueue getting commands pushed to it to play this or that data chunk from the game as needed. Does it need to be threaded, itself? No. We can make sure the only thing that talks to the audio thread is 1 other thread. Thus it doesn't need to be threaded internally, it just needs to be executed in it's own thread. Though it would be interesting to see how much of a strain it would be if the audio subsystem forked off it's actual play function into it's own thread, so we could begin playing a sound the moment we get the request without having to wait or halt a previous sound or mix the audio together in software in realtime.

In any case, i vote we take the simple route.

Work on trying to thread something, but ensure that that thing, itself, doesn't need to be thread-safe. That is, have everything that will run on it's own thread, have only 1 designated reader/writer.

If we can do that, we simplify a lot, and we can completly ignore the problems of thread-safeness.


edit; Sound may be important, but it's not something that will stop a player from being able to play the game. It's not a value judgment, it's just an observation of functionality. We dont currently make audio a necessary part of the game. It's just ear candy.

[klauss]

Problem is I alrady have too many tools that use vsfilesystem without regard for threadness. The XML parser, stream codecs, including video codecs, etc.

I believe the best approach would be to work a bit on it so that it is threadsafe. It's not much work - only shared structures need protection, so if we can do it in a tidy fashion it should benefit all parts of the code.

As you said, reading from a VSFile doesn't need a lock, only opening it. So I'm hoping adding thread safety to VSFile isn't much of an overhead.

[safemode]

all we need to do is make everything in vsfilesystem local to that object. Then we can just create a new vsfilesystem object for any thread that uses it.

The only thing we need to worry about is reading the config file and thus, waiting until it's read in order to use any variables it sets.

If the audio thread is being told to do something, it can be assumed that the config file has been read, so we shouldn't have to worry about any of that, just make the object that GetVariable reads static. So every vsfilesystem object has the same config object. (only one config file can be read per game, not per object).

[chuck_starchaser]

That's it; if the vsfilesystem doesn't use too much memory, you could wrap it into a singleton and instance it as a thread-local variable.