unit confusion

[safemode]

I think our main disagreement lies with the idea that you believe that the AI to game thread should be synchronized, via a message system. I was using locks in the beginning of my idea, to do the same thing, but without having to copy _all_ the data needed for reading and writing.

But my idea doesn't need to be synchronized, i was simply locking to keep from having a race during destruction. Then i realized that i could simply increment Ref whenever i push a command to the AI queue thread, and the processing of that would decrement ref. Negating any possibility of races on destruction.

So with no race on destruction, and since the AI thread doesn't need to be synchronized, we have no need for a two way IPC between the AI thread and the game process. We can safely share all the data required. We'd just have to make sure that everything the AI reads and writes from Unit is atomic or we have exclusive read and write access to.


In your method, every physics frame you would read your unit's message queue to process any previous AI thread output. You'd have to copy the data in that queue (_ALL_ of the "shared" data) you would then have to execute the physics frame then at the end, send a new AI command and copy all the new "shared" data to the AI Thread. If the AI thread doesn't finish by the time the unit is next processed in a physics frame, you have to wait an entire new physics frame for any changes to have an effect.

In my method, every scheduled frame , i execute the physics frame based on the current unit data and then send the AI thread a command. The AI thread then can make changes as it processes. Meaning if the next physics frame is processed before we finish completely, it will use any data currently processed at that time.

My method requires no expensive copying, no local queue to hold output from the AI thread messages, no "all or nothing" waiting for the AI thread's output and no locks.

In your method, your AI thread operates on a copy of the unit, and all it's changes have to be copied back to the real unit by the real unit. My AI thread gets told where to go and operates on the unit directly. The only caveat here is that I would have to make sure everything that the AI thread reads and writes is either atomic or not read/written to by anything else but the AI thread.

That's not as hard as it sounds considering all of this would be done after Unit is restructured. Long after. Considering we'd have to do a thorough check to make sure the game doesn't circumvent the AI and cheat anywhere, to do things a player would otherwise do. I'd like to see the AI take the place of a player completely, with the game simply directing the AI via the queue. I doubt it does anything as cleanly as that now.

[ace123]

You keep talking about requiring the threads to be synchronized.... but is there any reason not to do this over a network stack or FIFOs or other things designed for inter-process or inter-thread communication?

Just have it keep sending ship position and target information into the pipe, and keep receiving AI data out of it. If we are worried about uptodateness of the information, just discard all the way up to the most recent packet.

Might also be possible to do this with a synchronized vector that you copy information into, out of the units.

[safemode]

I keep talking about not needing to be synchronized. So i dont need all the provisions that needing to be synchronized entails.

[ace123]

I was really talking to chuck, should have been more specific about the "you".

I don't think we have to wait for the unit class to be restructured to do this.
any sort of synchronized vector separated would work equally well, which is what I believe that safemode was saying.

just copy the information over each code loop, and the other thread can read it every so often, when it's ready for new data, and push the AI commands onto another vector for the main engine to take data from.

One problem is that some AI classes like FlyByKeyboard will need to access a lot of features from the Unit, and it really should not be synchronized because of what it is.

So we will need to have a type of AI class that is computer controlled and can be completely independent. Maybe by making a ThreadedAI into a subclass of Order and have all the complicated AI scripts derive from that.

This should be doable without too much work but probably not for this upcoming release.

[chuck_starchaser]

Just have it keep sending ship position and target information into the pipe, and keep receiving AI data out of it.

That is EXACTLY what I was talking about, Ace.

[safemode]

What i was saying was that there is no need for the pipe. Just write it and forget it. It doesn't matter if the Unit is halfway through it's physics frame or just getting done it or starting it. It doesn't matter if the unit gets half of the info from the AI thread in one frame and the rest by the next. The AI thread should be 100% asynchronous. So we dont need pipes and we dont need to copy all kinds of data to keep it safe and we dont need to make the copying of output synchronized.


And i do think this is going to have to wait until after Unit gets re-organized, because it would be a cluster F to do it with Unit looking like it does now. Plus, the re-organization would push AI to a Ship class.

In any case, it wont get done unless someone actually works on it. and right now, the legality of the collision code is probably most important to be dealt with prior to release, and what aint broke is so far down the priority list it probably wont be really worked on until long after the release.

[safemode]

OK, back to the Unit re-organization for a bit.


I stripped out everything from Unit that i didn't think ought to be shared between Ships, Planets, missiles,enhancements, cargo etc.

Basically i'm left with :
Reference / unreference methods/data members
illegal constructors.
physics section with anything related to hulls/shields/damage removed.
network section
and misc section with only the name data members remaining.

I removed the collision section because it was too all over the place as far as code needed to make it work. It had stuff from GFX, stuff from Universe, stuff regarding Ships.

[chuck_starchaser]

WOW! You work fast!

Yeah, let's forget about multithreading for now.

I think the best way to refactor would be to first make sure the split is transparent and that everything is still working, if you haven't done so.
Illegal constructors would be a priority. With PU, there's a segmentation fault every time your ship is getting destroyed. I never die anymore: The game crashes a second before I do. Probably some resource allocated by a constructor but deallocated somewhere other than in the destructor.

[safemode]

PU with a new VS engine ? PU's python does things that are probably not valid anymore and the VS python hasn't done in a long time. I would look there for your segfault on dying. Since we dont see this problem in the regular VS game.

[safemode]

Oh, and the re-organization is going to be _extremely_ hard to be transparent. Because what we're doing is enforcing api's. No more using Unit* to do _everything_ under the sun. This is going to break a lot of code, and that's the point. So we can fix it to work the way it should.

We'll have levels of transparency, but for the most part, anything that is using Unit* to do non_generic unit type stuff, is not going to work, and we dont want it to. If we afford that type of stuff transparency, then we'll end up with something even more ugly than the current Unit class, and slower.

We can make function calls temporarily transparent for debugging purposes, and then go through the code and fix it so we dont need them anymore. but we wont be able to make direct access to data members that no longer exist in Unit* transparent without mucking up the whole idea of cleaning up Unit. We can virtual a function and wrap it around a #ifdef debug statement, but we can't virtual a data member.

[safemode]

blah. All of this virtualizing functions just to remain backwards compatible for the cross-over is driving me crazy. It's like double the work for no gain. I think i'd rather have the compile break and fix the pointers as needed.

I'm gonna get Unit the way i want it first, then work on Ship, and add any functions needed to any other types and once that's done, go through the compile errors making sure code that accesses a type-specific member uses a pointer of that type to do so, rather than generic unit pointers. I think that will be an over-all, easier thing to do.

I really can't wait to see how much of VS this breaks. shoot me now.

[loki1950]

I really can't wait to see how much of VS this breaks. shoot me now.

Which foot you seen to be doing a reasonable job of self-flagellation

Enjoy the Choice

[chuck_starchaser]

Oh, and the re-organization is going to be _extremely_ hard to be transparent. Because what we're doing is enforcing api's. No more using Unit* to do _everything_ under the sun. This is going to break a lot of code, and that's the point. So we can fix it to work the way it should.

Good point.

[safemode]

this is on hold for a bit. ..as hellcat has asked to help with collision retrofit.

[safemode]

i setup a wiki page for miscellaneous engine work
http://vegastrike.sourceforge.net/wiki/ ... ngine_Work
You can get to it from http://vegastrike.sourceforge.net/wiki/Development

Unit work info will go there. If anyone has any pet sections of the engine they are working on that doesn't fit anywhere else or is too small to warrant it's own page, add it there.

I hope to get back to unit after i'm done with opcode.