Speech and Sound

For collaboration between the different artists creating music and sound for vegastrike.
Post Reply
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Speech and Sound

Post by chuck_starchaser »

Update:
klauss and myself decided to tackle speech and sound for the Privateer/WCU mod, couple of months ago, where early results have been encouraging. Work is progressing slowly but surely, mostly at the planning level, for now; and will be apliccable to the VS engine and all mods that wish to take advantage of it.

Highlights (subject to change as time comes):

* Some sound files are 8-bits; klauss is trying to restore quality. Would be a nice surprise if someone has the masters saved somewhere.

* Comm voiceovers will sound like they come via digital radio and as if reproduced by a little speaker mounted on the dashboard.

* There will be different sound templates for the different ships one can buy. Perhaps not 100 templates at the beginning, but at least a few to choose from.

* We will NOT use the EAX extensions of OpenAL, as EAX is very limited in its reverb capabilities (only allows the timing of the first reflection to be adjusted, for example; the rest are canned). Instead, we will be using klauss's "convolution" program to pre-process in-flight sound files *when you buy a ship*, and save the processed sounds in a tree of folders under .vegastrike or .priv100.

* Five philosophies will used for in-flight sound:
1) Music: Little or no processing at all. No positioning, since the source of music should not be "located" in the cabin of the ship, but rather comes "from the game", or "from the world"; --as well as out of respect for the will of the artists.
2) Speech: Made to sound "canned": digitized, put through a little speaker which distorts a bit; but otherwise kept "raw", without reflections or reverb or positioning, for the sake of intelligibility.
Here's some samples already:

Now entering an automatic landing zone (male):
Original: http://www.geocities.com/hammerfall_pro ... _zone0.wav
Processed: http://www.geocities.com/hammerfall_pro ... g_zone.wav

Now entering an automatic landing zone (female):
Original: http://www.geocities.com/hammerfall_pro ... emale0.wav
Processed: http://www.geocities.com/hammerfall_pro ... female.wav

Will be done better; that was just using plain soundforge, without any of the better plug-in's.

3) In-cockpit sounds, such as buttons, beeps, seat squeeks, joystick end-of-range thumps, occasional rattles, fans humming, etc., will be 3D positional (OpenAL-wise) AND will feature carefully calculated timings for early reflections based on cockpit geometry, as well as late reverberations, also somewhat based on cockpit geometry.
4) Beyond cockpit sounds, like engines, reactor, transformers, pumps, cargo shift rumbles, weapons shooting, and incoming shots: All will be using convolutions to add "hull sound" coloration. This coloration will be extracted from weapon hit files, probably, by de-convolution, and then added to engine sounds, etc, by convolution.
5.a) Sounds from space (a): Vaccum has no air, by definition, therefore sound doesn't exist in the void (that's why astronouts in EVA must use radio to speak to one another). However, realism in this sense would be kind of boring, so we decided that the best solution would be to have the computer re-create sounds, from a library. Thus, you'll hear the engines of ships passing by, and the sound of their hull when you score hits, BUT, they will *clearly* sound synthetic, and as coming from the same little speaker through which the comms come. Actually, each radar type will have its own set of synthetic sounds, and speaker qualities, some will sound better, some worse; some directional, some not.
5.b) Sounds from space (b): Some people don't care about realism, and so want to hear realistic sounds (pun intended): reverberations, distance, muffling, etc. (not that you'd hear much through foot-thick chrystal, or that you'd roll down the windows, but anyhow...). So, as an option, one will be able to select this mode of operation. (Planned for implementation after 5.a)
5.c) Sounds from space (c): Some people do care about realism a lot, and would like to know what it's like in space. So, for them, the easiest implementation of all: no sounds from space.

And a possible 6th philosophy:
6) Synthetic speech: This would be good for two things: Messages from the ship's computer itself, as well as simulated translation of foreign speech. Thus, if we wish for the aera to sound like giant squids in mating season, no problem, we record them and play their weird speech faded out, and overlay the sound of the English (or your favorite language) text-to-speech synthesis.

* Two modes will be used (selectable by user in setup):
A) Headphones mode
B) Stereo speakers mode
And in order to support these two modes, the preprocessed files will probably be 4-track: 2 left channels and 2 right channels, representing two head rotations: 45 degrees to the left, and 45 degrees to the right, in headphone mode (pure), so that various head rotation angles can be computed at run-time by simple interpolation.
In stereo speaker mode, an additional processing step at runtime will subtract a signal from each channel, which is a slightly delayed and muffled version of the other channel, so as to compensate crossover audibility (left ear hearing right speaker, and right ear hearing left speaker), so as to have a bit of "sound holography" -style channel separation enhancement. This can only be done on a per sound file basis, as there is no way to read back output from the sound card and re-inject with any time accuracy, though.

Work to be done:
* Software wise, we pretty much know what we need to do and are working on it.
* Sound-files-wise it's a different story. The Privateer/WCU mod has a fairly complete set of comm voice-overs. Other mods I'm not sure. Vegastrike, the game, I can't remember if it has any. Would be nice to start building speech/sound libraries for each mod. The libraries perhaps should be kept in CVS, storage limits notwithstanding.
I can make some space available on my server, but I only got 2 gigs available. We must, somehow, set up parallel folder trees for "master" and "shipping" files, so that if later on we want to apply better processing or compression, or update encoding, we can go back to the masters and "remix" them. Right now, AFAIK, there are no masters anywhere; --just final, compressed files. I hope I'm wrong.
***** Update: klauss can offer tons of storage, but has no presence; I got presence but no storage; but he has an ftp account to my website, and so will you, if you need to upload sounds; so we'll do it like this: klauss keeps the sound library, both masters and compressed files; I'll probably keep only compressed files in my server, but temporarily keep masters there when working with them or for transfer back and forth. Anyone wishing copies of the masters will be able to obtain them from klauss through my server, one chunk of the total at a time. *****


Recording guidelines:
For those wishing to contribute content, here's some basic guidelines, do's and dont's:

* Ambient sound interference: Turn off computers, washroom fans, air conditioners, the TV next room; but don't shoot your neighbors, just wait till they are sleeping. Then record.

* Ambient reverberations: Only ok if recording sounds for bases, NOT for in-flight sounds. All sound masters for in-flight sound, including voiceovers, must be "flat", un-echoic; otherwise the reverbs present will clash with the cockpit reverb modelling. To avoid echos, surround your setup with sound absorbing panels (very cheap at home renovation stores, look like pressed wood panels, but are usually dark green, and are made of fibre-glass and something to keep it together; use gloves to handle, tho), or hang futons or matresses, or multiple layers of bed-covers tenting over, or wall yourself in by stacking sandbags or pillows all around... Please don't skip over this point.

* Microphones:
-- Directional is better than omni; the more directional the better, as it helps catch less echos.
-- Electret (condenser) microphones are fine, per se, but the interface of cheap ones often distorts. If at all possible, try to use professional quality mikes; otherwise use cheap singing mikes; but really try to avoid "computer" mikes. Good idea for a mike: Sennheiser ME66, for example; it is a condenser type mike, but it kicks major ass. Costs like $ 300 or 400 but IMO you get the quality of a $1000 mike, plus very acute directionality (it's a "boom" type mike: long, with many little slots along). The reason it's so cheap, I reckon, is that they sell many of them, since they are a popular sound upgrade for portable video cameras, and come with adaptors for such use. Response is very flat, almost as good as a laboratory mike.
Main problem with condenser mikes is that they boost base sounds at short range. Your mouth, or the sound source, should be at least one foot (30 cm) away from the mike.
-- Dynamic (coil) microphones are pricey and overrated, IMHO, but if you have one, good for you. Often the directionality of dynamic mikes is cardioid type. Some have a switch to select between omni, cardioid and hyper-cardioid. If your sound environment is anything but the best, in terms of quietness and surrounding sound absorbing material, set the mike to hyper-cardioid (and make sure it points to the sound source). But if your setup is excellent; --ultra quiet and as echo-free as on a flat grass field extending to the horizon, then set the mike to omni. They do sound best in this setting.

* Setup: Spend a few bucks and get a mike stand, if you don't have one. You can get one made in China for 30 bucks at a music store; or get a used one (as long as it stands, the rust doesn't matter). For recording voiceovers, place the mike at least 6 inches (15 cm) from your mouth (twice that if using a condenser mike). It should be at the level of your forhead and pointing slightly down, towards your mouth, so as to avoid your breath going into the mike and making wind-sounds. The mike stand should stand on the floor, preferably. If you have a short stand sitting on a desktop, make sure your computer is not on that very desk; --or turn it off; and don't even touch the desk during the recording, whether with your hands, feet or anything.
If using one of those newer recorders that save in MP3 format, set it to highest quality, of course. Ideally, our masters would be un-compressed, 24-bit precision, and 96 KHz or 192 KHz sampling; but 16-bit, 44KHz sampling at 320 kbps, or even 256 kbps should be good enough for speech masters.
If using magnetic tape: chromium and "metal" tapes are good; pretty much all else is garbage. When transferring the tapes from the recorder to the computer, make sure the recorder is running off batteries, NOT from a wall adaptor; and set your sound card for 24-bit, 96 KHz encoding, or what best you can get out of it.
If you have to choose between a cheap sound card mic's input and an average tape deck, by all means, choose the tape deck, and transfer to digital with the sound card's line input. Noise from tape masters can be "cleaned" by post-processing; but distortion (from computer-mikes' horrible connection interface) can't be got rid of.
During recordings and transfer, disable AGC (Automatic Gain Control), and manually ensure that level is high but well below "peaking" or "clipping"; --'well below' meaning 6 to 12 dB below.

* Mood:
If recording voiceovers, first play Vegastrike (or substitute your favorite mod) and/or read the history behind it, and get into the right mood to record the faction for which you will record. No sound quality or processing technique can even begin to compare to putting heart and soul into this kind of work. Listen to the recording objectively, and if you're not proud of it blind, do it again.
Announce your intention by posting in the right forum, if you wish to avoid duplicating someone else's work; but if more than one voiceover for a given comm is available (and good), we might use both by perhaps selecting at random at runtime. Also, for voiceovers, preferably get a partner and do male and female versions. Foreign accents are good, as long as the diction is clear. Practise before recording. Purse your lips: This is a secret well known by singers; --words come out clearer when your lips are pursed forward. And keep a jug of water and a glass near.
If you do comms for a particular faction, try to do all the comms for that faction in one recording session, and listen to the first takes between doing later ones, to make sure your tone and rhythm are consistent. If possible, do a second or third take of the whole set, then listen to them the next day, and pick the best take for each line.

* Little sounds:
We can use all kinds:
-- In your car (engine off):
Sound of gear shift stick hitting dead ends --good for joystick-hitting-dead-end sounds.
Air conditioning: Good for air conditioning sound :)
Window/seat-adjust motors --good for something probably.
-- In your home or office:
Seat squeak --good for seat squeek during high-G turns.
Computer fans noise: Good for fans for the electronics in the cockpit.
Clicks from switches of various kinds (make sure they are NOT mounted on a panel, as we'll probably want to add a panel patch for each ship, for consistency; wall mounted switches should be ok; but un-mounted switches you are holding in your hand would be best).
-- At the work-shop:
Electric screwdrivers at work
Removing and replacing covers
Pluggin and unplugging connectors
Circuit breakers
Welding, etc.; --all good for repair bot sounds.
-- In the back alley:
Stones hitting garbage container --good for hull-sound coloration extraction by de-convolution.

* Submitting:
Put all files you are submitting into a folder, with optional subfolders. Make sure you add a readme.txt file in each folder describing the files: Filename, date of recording, your name or handle, intended or envisioned use, encoding, and anything else of relevance. Then zip the root folder and upload. If you're going to delete the files from your PC, please wait until I've confirmed that I got them and they work. For car sounds, for example, state brand and model, so we can map expensive cars to expensive ships.. :)

Comments and suggestions welcome.
Chaos
Trader
Trader
Posts: 17
Joined: Mon May 16, 2005 8:27 pm
Location: France

Post by Chaos »

Nice post !

I would like to say 2 things:

1) It is possible to have sound in the cockpit from other ships and cosmic events. I already wrote to JackS about how justifying it, in a kind of a hard SF way. Here is in short what i suggested: The sound in the cockpit is generated by an Augmented Reality device wich role is to extract audio from optical domaine and help the pilot to have a 3D representation of his surrounding. This device is made of several parts, a radar to target external objects, some lasers+ and optical captors to mesurate micro-vibrations radiating from them, and a synthetiser that is able to filter and recreate plausible sounds from them.

2) As a consequence, it would be possible not only to have sounds in the cockpit (and at lightspeed btw), but to have real 3D sounds in the game as well. Let me explain. The device i am talking about require to have speakers all arounds you, so that sounds will come from front/rear but from top and bottom as well. I think that you may want to use a cheap 5.1 or 7.1 system for the game and use 2 of the channels to have sounds from up and down (you will have to position the top one firmly, for obvious reasons) . It would just require that an option is added to the game so that the 2 channels in questions are well affected to top and bottom. Then the VegaStriking experience would be very ... surrounding !! :-)
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

A system like that should cost 1 million credits, tho... :) Let's see,

10K creds:
I'm thinking that the cheapest radar you can buy in the game, and the one you start with, would have no sounds from space, at all. This would be best, first: as an educational first stage; and second: to make sure the player is aware that sounds from space are re-created.

20K creds:
This one would have comical sound samples coming out of the same speaker as the comms, and fading out during comms.

50K creds:
The third cheapest radar might have its own speaker, slightly larger, and feature distance cueing by volume control and spring-like reverb.

100K creds:
Next price up would be stereo and have a larger library of sounds.

200K creds:
Simulates sound as if your left and right windows were rolled down :)

500K creds:
Uses smart cabin reverb cancellation so that you hear "sounds from space" as if you were driving a convertible.

1M creds:
Features Gamma ray laser ranging of the surfaces of other ships to detect actual sound. Note that radar ranging would not work, as it's wave length only goes down to the milimeter range. We need to detect vibrations in the microns range.

2M creds:
Adds the feature of adaptively cancelling out hull reverberations of other ships, in order to make out some more subtle but important sounds, such as defective engines, pumps, and other devices. Integrates with the targeting system to provide statistically more accurate info on other ships' status.

5M creds:
Go ahead...
MamiyaOtaru
Privateer
Posts: 729
Joined: Tue Jan 07, 2003 8:32 am

Re: Speech and Sound

Post by MamiyaOtaru »

So you're hard at work removing features? I expect sound in my gameplay, even if I'm in space ;) Having sound in space, like 2d cockpits, is pretty much a part of privateer. WCU on the other hand is of course in many ways a clean break from the old, and an interesting place to experiment with upgradeable sound systems.
chuck_starchaser wrote:Update:
* Some sound files are 8-bits; klauss is trying to restore quality. Would be a nice surprise if someone has the masters saved somewhere.
Which ones specifically? You might find it hard to restore something that wasn't there though; it's not like we took out quality. Most of the sounds are just as they were originally. The original was a rather old game.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Re: Speech and Sound

Post by klauss »

MamiyaOtaru wrote:So you're hard at work removing features? I expect sound in my gameplay, even if I'm in space ;) Having sound in space, like 2d cockpits, is pretty much a part of privateer. WCU on the other hand is of course in many ways a clean break from the old, and an interesting place to experiment with upgradeable sound systems.
Hehe. Perhaps starting out without sound is a little too much. But the idea that better systems should sound better is nice, I think, even for PR. Remember that PR sounds are fully synthetic. You could realize, after playing much, why: you had a loussy sensor package, when you upgraded your sensor package (radar), you got a better "audible situation awareness" subsystem.
MamiyaOtaru wrote:Which ones specifically? You might find it hard to restore something that wasn't there though; it's not like we took out quality. Most of the sounds are just as they were originally. The original was a rather old game.
Again, hehe. I know that. And I know the only real way to solve it is to re-record things. But I was playing WCU and the bartender speech really left me almost deaf (I mind distortion a lot, since I do music recording besides the computer stuff). So I set out at enhancing those loussy originals. I have a program (actually, everyone has it since it's been uploaded somewhere, Orfeo) which has a nice little filter called "Harmonischer". (I didn't know how to call it). It "reinvents" the high end of the spectrum (which, in subsampled sounds, has to be deleted). I also have a special-purpose noise reduction filter for the aliasing in 8-bit recordings (sadly, Privateer's originals were not dithered, so it makes things harder). I doubt anyone could use those modules effectively since they're not well documented, and using them well requires quite some knowledge of the internals. Anyway, I'm trying to reach acceptable quality be reconstructing the high-end and removing some noise. After that, speech should sound less 8-bit/11Khz.
I guess chuck_starchaser meant that for original samples (the ones not taken from Privateer). But I think all those are already 16-bit/44Khz (or they don't need 44Khz, like the vdu sounds).

But there are some cases where I think there should be a master (I hope): IntroRF.wav is one of them.
It seems to have been derived from a compressed source (or was it compressed with ogg?). Anyway, compressed sounds make my work harder, since there's more to reconstruct. If anyone had the uncompressed ones (that is, forget about decompressing the compressed ones and uploading them, what would be nice is to have the ones that weren't compressed at all), it would be much better.

Again, since the only real solution is recording everything again, what I'm doing is just temporary stuff: so that sound is a little more high quality until everything is re-recorded. After all, it has to be re-recorded anyways for other reasons: it isn't quite right to be using Origin's content... or does PR have permission to do so?
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
John Cordell
Mercenary
Mercenary
Posts: 125
Joined: Sat Jul 10, 2004 9:43 am

Post by John Cordell »

Hehe this is quite funny, when i look at the Privateer project i see many things that still need to be done or redone but the last thing i would change or waste time on is the sound ;)
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

Well, someone has to "waste" time on it.
Preferrably, someone who doesn't think he's wasting it.
Like me.

:D
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: Speech and Sound

Post by chuck_starchaser »

MamiyaOtaru wrote:So you're hard at work removing features? I expect sound in my gameplay, even if I'm in space ;) Having sound in space, like 2d cockpits, is pretty much a part of privateer. WCU on the other hand is of course in many ways a clean break from the old, and an interesting place to experiment with upgradeable sound systems.
I wasn't even thinking when I said Privateer/WCU, I'm just used to uttering them together; and it's just an idea, anyways; what we'll come up with, really, is infrastructure that others can use as they see fit. The rest is just use suggestions.
[/quote]
jackS
Minister of Information
Minister of Information
Posts: 1895
Joined: Fri Jan 31, 2003 9:40 pm
Location: The land of tenure (and diaper changes)

Re: Speech and Sound

Post by jackS »

chuck_starchaser wrote: * Comm voiceovers will sound like they come via digital radio and as if reproduced by a little speaker mounted on the dashboard.
brief note: In the VS universe, dashboard speaker mounts would be considered vastly archaic. Intentional sound (communications, UI beeps, warnings, assistive positioning sounds for other vessels, etc) would be played (depending on the pilot's personal equipment preference/lifestyle choices) either through the headphones built into his helmet or through direct stimulation of the aural nerves. Unintentional sounds (hull stress, damage vibrations, engine vibrations, etc) would be both heard and played, with the dual hearing sources of the pilot's own ears and playback from external microphones on the helmet.
Duality
Daredevil Venturer
Daredevil Venturer
Posts: 583
Joined: Sun Feb 16, 2003 12:58 am
Location: West Coast of USA
Contact:

Post by Duality »

3) In-cockpit sounds, such as buttons, beeps, seat squeeks, joystick end-of-range thumps, occasional rattles, fans humming, etc., will be 3D positional (OpenAL-wise) AND will feature carefully calculated timings for early reflections based on cockpit geometry, as well as late reverberations, also somewhat based on cockpit geometry.
I wouldn't like this feature. For a person who wants to be in combat mode, he would not want to hear these type of sounds. I personally hate seat sqeeks the most.
4) Beyond cockpit sounds, like engines, reactor, transformers, pumps, cargo shift rumbles, weapons shooting, and incoming shots: All will be using convolutions to add "hull sound" coloration. This coloration will be extracted from weapon hit files, probably, by de-convolution, and then added to engine sounds, etc, by convolution.
Well actually, thats still too much exact for engines roaming, shield hitting and hull hitting. That would still be good though.
6) Synthetic speech: This would be good for two things: Messages from the ship's computer itself, as well as simulated translation of foreign speech. Thus, if we wish for the aera to sound like giant squids in mating season, no problem, we record them and play their weird speech faded out, and overlay the sound of the English (or your favorite language) text-to-speech synthesis.
As yes. That would be supercool.

As for in-game voice chat for vegastrike, microphones usually cost around $10. I got mine for the price similar to that.

Anyways, I downloaded a bunch of pointless synth programs so I don't know if I can be worth any help at all.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Duality wrote:
3) In-cockpit sounds, such as buttons, beeps, seat squeeks, joystick end-of-range thumps, occasional rattles, fans humming, etc., will be 3D positional (OpenAL-wise) AND will feature carefully calculated timings for early reflections based on cockpit geometry, as well as late reverberations, also somewhat based on cockpit geometry.
I wouldn't like this feature. For a person who wants to be in combat mode, he would not want to hear these type of sounds. I personally hate seat sqeeks the most.
I guarantee you, you won't even notice the seat squeeks unless you pay particular attention to them. I'm a big fan of subtlety. I like content that's rich, yet un-assuming. That's part of "realism", in the sense that real life has sensorial richness, but we don't pay much attention to it. I would not want it to be otherwise in the game. I wouldn't put squeeks every 5 seconds, just to make sure everybody notices them :). Klauss and I were talking about having radio static, by email, and sure we'll have it, but it will probably be a week of two playing the game before anybody even notices the effect. It will take a coincidence of a couple of rare events to bring it on. And there will be many sounds nobody will notice until probably a year down the road, either because of their rarity, or due to their very low volume, or due to masking by louder sounds during action. Switches and buttons will have their sounds, but you'll probably need a very quiet keyboard to hear them.. ;-)

4) Beyond cockpit sounds, like engines, reactor, transformers, pumps, cargo shift rumbles, weapons shooting, and incoming shots: All will be using convolutions to add "hull sound" coloration. This coloration will be extracted from weapon hit files, probably, by de-convolution, and then added to engine sounds, etc, by convolution.
Well actually, thats still too much exact for engines roaming, shield hitting and hull hitting. That would still be good though.
Better too much than too little; without hull coloration it's hard to believe the noise of afterburner actually comes from the engines, and not from a gas leak inside the cockpit. It needs hull sound coloration, and any coloration other than the sound of weapon hits on the hull, would make it sound like you're in two different ships. The consistency needs to be there; otherwise one has to force oneself to believe the reality of the environment. For a game to be immersive, everything should be convincing. Sound gets less conscious attention --but a lot more sub-conscious attention--, than graphics; at least it is so with me. Even with the cartoonish graphics of Diablo, say, I could totally get into the game, in good part thanks to the sounds and the music. Same with Starcraft, where the sounds weren't necessarily "realistic", but they were consistent, stereo-wise, and reverb-wise.
6) Synthetic speech: This would be good for two things: Messages from the ship's computer itself, as well as simulated translation of foreign speech. Thus, if we wish for the aera to sound like giant squids in mating season, no problem, we record them and play their weird speech faded out, and overlay the sound of the English (or your favorite language) text-to-speech synthesis.
As yes. That would be supercool.
As for in-game voice chat for vegastrike, microphones usually cost around $10. I got mine for the price similar to that.
Yes, we've got to have voice over udp for multiplayer. That's a must.
Anyways, I downloaded a bunch of pointless synth programs so I don't know if I can be worth any help at all.
Hey, absolutely! Put together a sampler to show how each of those synths does with some typical messages from the computer.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Re: Speech and Sound

Post by chuck_starchaser »

jackS wrote:
chuck_starchaser wrote: * Comm voiceovers will sound like they come via digital radio and as if reproduced by a little speaker mounted on the dashboard.
brief note: In the VS universe, dashboard speaker mounts would be considered vastly archaic. Intentional sound (communications, UI beeps, warnings, assistive positioning sounds for other vessels, etc) would be played (depending on the pilot's personal equipment preference/lifestyle choices) either through the headphones built into his helmet or through direct stimulation of the aural nerves. Unintentional sounds (hull stress, damage vibrations, engine vibrations, etc) would be both heard and played, with the dual hearing sources of the pilot's own ears and playback from external microphones on the helmet.
Yes, I agonized more than it probably shows about the issue of archaism. Ok, first of all, like I said to Mamiya Otaru, things I say here with regards to content, are mere suggestions, and some of them I'm attached to enough that I will try and stick them into the the distribution as a taste of what's possible, but I have no intention to impose my taste on others. Having said that, I did put a lot of thought into this matter. The problem is this: Surely there will be technologies far beyond our dreams by the third millenium, but the problem is we haven't got them yet. :)
No, well, the problem is that we're trying to convey information at many levels through a pair of cheap speakers (for a lot of people). And this is not easy.
Currently there's too much spectral uniformity and too little loudness balancing: Comms' voice-overs (speaking of Privateer/WCU) are sometimes too loud and booming, and yet hard to understand. This is partly because bass sounds in speech contribute nothing to intelligibility. Our brains have an incredible capacity to zoom into a sound in the middle of a flood of sounds. In a room full of people talking, one can concentrate on the conversation two people are having, a few meters away. But in order to do that, our brains single out the directionality and reverb pattern, as well as voice characteristics of those persons. Same is true in trying to convey info at different levels in a game sound environment: If the engines have a hull reverb coloration, you can concentrate your attention on the engines, even when louder sounds are present. If a comm voice sounds like it's coming through a tiny speaker on a dashboard, you'll probably be able to single out that characteristic sound, and comprehend the words, even in the midst of battle, with loud explosions and the sounds of weapons firing and transformers overloading going on.

There's a similar situation in the world of music recording: Everyone seems to agree that stereo is good, and the more "channel separation" the better. But the term is hard to find a definition for. In the old days, when stereophonic sound began, the tradition was to place two microphones in front of an orchestra, some distance apart. Ideally, a listener at home would have two speakers some distance apart, also, --namely, the same distance apart as the microphones were. But the logic of that was not too well adressed: At best one could expect the system would sound as if you were at a live performance, but behind a wall with two round holes on it, some distance apart, too. Would that sound very "stereophonic"?
The shit began to hit the fan, and the music industry had to act before it'd be too late and the whole world might wake up to the fact that stereophony was a sham.
So they began to single-out instruments for one channel or the other. Particularly with percussion.
Unfortunately for them, this solution was not fully satisfying: People did not want to be made so aware of the fact that there were "only" two channels. They wanted music to sound like it came from "everywhere".
More shams were added, like quadraphonic sound that never was.
Meanwhile, a new current started flowing: Headphone stereophony. Infinite channel separation, and let's see what can't be done with it: Using a pair of microphones inside a molded plastic head, resembling a human head, with realistically shaped ears, you could record sounds with accurate 3D positoning, up, down and all around; and get a vivid experience recreated through headphones costing a fraction of the price of stereo speakers. (Terrible for business.)
Only one problem: Recordings that isolate instruments to the right or left channel, sound horribly unrealistic through headphones. But recordings made through modelled human head microphones, which sound incredibly rich and positional through headphones, sound flat or "not stereo at all" when played through speakers.
So, if you were in the business of music recording, what would you do?
Well, what they do is about half and half: Some of the sounds are positoned accurately for headphones via the use of head microphones, while other instruments are isolated to one channel or the other, or use some other of those traditional dirty tricks for "channel separation enhancements". In other words, a potpurri of philosophies, to try and half-please everyone.

Same sort of difference here: not just in terms of stereophony, but first of all in terms of trying to avoid aural clashing between the sounds. We could argue that the radios and speakers of the third millenium will kick divine ass, and that sounds from other ships will be reconstructed more accurately than even most gifted ears could appreciate. But if *we* try to play every sound category at hi-fi quality, we'll end up with a muddy cacophony, where nothing will be distinguishable from anything else.
That's why klauss and I brainstormed so much to come up with a set of *different* philosophies for how to treat sounds in different categories. So, for instance, weapons sounds, cargo shifting sounds, engines, pumps and reactor, are in a highly reverberated, convoluted and muffled category. Cockpit sounds like switches and beeps, squeeks, rattles, fans and the occasional screw rolling on the floor will be highly "position accurate" (by finely crafted balancing, filtering and early reflections), so even though they will be low volume and unassuming, these little sounds will contribute all of the "3D soundscape consistent with the size and shape of cabin that I see around me", sensation. The comms speaker was originally in this category, but then it seemed to us more important to make the little speaker more consistent with the player's real world room's aural context. This came from two angles: a) we did not want to add early reflections to the comms, because to do so might lower the intelligibility of the words, and b) it was important for at least some sound to NOT be realistically modelled, in order to make the better modelled soundscape set noticeable by contrast. Thus, the comm speaker will use a cheap technique, akin to isolating it to one channel: It will play through the left channel, but a lower volume, mufffled version will come out of the right channel so as to not make it too unbelievable for the headphone listener.

Finally, and to get back to the justification for modelling a cheap, 2" speaker mounted on the dashboard: A) As mentioned earlier, it helps our cortexes' ability to "isolate" or "concentrate attention" on the speech. B) It helps justify removing bass sounds from speech, which are useless and clash with the sounds of explosions as well as with the music. C) Provides an anchor or frame of reference, from which we can distinguishe hi-fi sounds, even through non-hi-fi computer speakers; and D) Appeals to our real life familiarity with telephony, radio and intercomms.

I might even go as far as argue that E) people in the third millenium will prefer these second-millenium technologies, for the very reasons I've been giving: Hi-fi intercomms would probably be annoying and distracting, and their sounds get too easily confused with in-cockpit sounds, without any real benefit to the intelligibility of speech.
pifactorial
Bounty Hunter
Bounty Hunter
Posts: 170
Joined: Mon Feb 21, 2005 7:04 am
Location: Pomona College
Contact:

Post by pifactorial »

This audio project makes me salivate. If it's going to be as extensive as you're hyping, it'll put Vegastrike on the map. One question - when will we expect results? I'd love to contribute, but I'm technically on vacation :wink:.
klauss
Elite
Elite
Posts: 7243
Joined: Mon Apr 18, 2005 2:40 pm
Location: LS87, Buenos Aires, República Argentina

Post by klauss »

One question - when will we expect results? I'd love to contribute, but I'm technically on vacation
I got entangled with the graphics subsystem for two reasons: 1) I had to get acquainted with the code, and 2) I can't let a bug lay around when I've detected it (I had detected some, and I had to correct them), and it slowed me down on the sound project.

I'm finishing the .ETX processor (see the huge thread, Voice acting, sound editing, etc., for details).
It will take, if I get to it energetically (which I intend to do as soon as I send the last graphics-related patch to hellcatv, around this week), one or two weeks, since the parser should be simple enough. Designing environment files is another matter.

Parallel to that, I'll round up the integration of the event system and replacing the direct AUDPlay() calls with events to the ESD. All that using the standard sounds, nothing .ETX processed yet.

That, I don't know how long it will take.

But after those two things are done, we'll start having fun: the .py modules can call the shots about which .ETX files to use on which .WAV/.OGG files, and where to save them and to which events bind them, and with a lot of .py programming (or hardcoded stuff, I'm not sure yet, since this thing needs to be pretty efficient, and python is not), we'll have a working, renewed sound system.

I can't say times on that, sorry. But let's say that most of the grunt work is already done: fftw handles the ugly math, the rest I've already done in the past years, so all that work is done, all that remains to be done is to integrate it with VS. It's not easy, but I don't think it will take that much time.

About contribution: Where could you contribute? Content? Programming? (Please, know python, please please please). We really need content authors. Chuck and I can create some things, but it's still the most time consuming aspect of the sound project.
Oíd mortales, el grito sagrado...
Call me "Menes, lord of Cats"
Wing Commander Universe
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Yep, we're gonna need TONS of sounds to make a believable soundscape. And much of the programming won't be as exciting as it may sound... --lots of navigating folders, managing files, routing events, making persistent sound definitions... But the results will be exciting. But we need sounds, and we need voiceovers. Somebody please start organizing something to get voiceovers for VS and the other mods. I can't help with that; I used to have an Earthworks mike and stand, but got stolen it.
zaydana
Site Administrator
Site Administrator
Posts: 478
Joined: Thu Jan 02, 2003 10:05 am
Location: Perth, Western Australia
Contact:

Post by zaydana »

from a musician/artist's POV here... srry to throw another pineapple into the mix but where is game music going to come in? I'm all for the idea of starships having radios just like cars do... which means they are going to need to be fit in with all this other stuff. Anyways, point is, I don't think we'll be able to keep all our music compeltely seperate from the vegastrike.exe if we don't want it to clash with you guys' new soundscape.

I personally think a lot of the music in VS needs redoing, not because it isn't musically sound but because the syntheizers used in it sound like the equivalent of hard plastic. And i'm happy to do a lot of the music, I've already got a lot of content that could be touched up and put into the game. Problem is, music in VS needs to follow one of two philosophies:

1. Starships have radios. This type of music means we'd need a jukebox, a damn lot of music, possibility for people to add their own music. You could even add music to the shops at ports, let people buy new music. I think this would add immensely to the immersion of vegastrike, and would really fit in with the direction that things are going.
2. We go for the traditional idea of trying to make atmospheric music which reflects the mood of the game. This could work, but I don't know how it would fit in with your soudnscape. Also, there is a slight problem in making this soudn any good. Homeworld did it briliantly, but thats cos most of the time "music" consisted of a signle stringed instrument. Do we want that? or not...

Anyways, As you can tell, i'm more for radio galactic... but I think this is an issue that needs discussion. Either way will require a lot of work... no. 1 will need interface code and a bit of soundscape work on chuck and klauss' part, but no. 2 will need a hell of a lot of code to detect the "atmosphere" and then a lot of musical talent to make the different pieces work together.

I think if you guys put in so much work into the Sound Effects, and we leave music how it is, it will kinda be like our current situation with an awesome starscape and a god-awful HUD. The best time to do anything about it is now... so yeah. Comments/Ideas?
www2
Venturer
Venturer
Posts: 537
Joined: Sat May 14, 2005 10:51 am
Location: milkyway->the sol system->earth->Europe->The Nederland->Soud Holland->Leiden
Contact:

Post by www2 »

@zaydana
Mebey cane shouwecase/icecast use for the radiostasion (youre cane thuse withe in the game) and fiew radio stasion (in a playlist) in VS withe couple funny advertisement

And for youre mp3/ogg a ipod system.
cane download at a interslere webwinkel or a shop at a planet and downlad at a ficion p2p network for free.

www2

p.s sory for my English i speets beter then i write sory
and dis is my first post in dis forum.
All Your Base Are Belong To Us
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Good points, Zay. Currently, my thinking is to NOT position or reverb or filter music in any way, like I said in the first post. Part of it is I don't want to trample on the artists' intents. Part of it is I don't think of the music as coming from a stereo in the ship's cabin, but rather as "coming from the game itself". If I was flying a ship in hostile environments I probably wouldn't be playing CD's... ;-) And it would be hard to justify how the CD player automatically changes CD's (or switches stations) when combat starts. And then part of it is selfish: The problem with making 3D positioning and an aural environment believable is the fact that the player is in a room, in the real world, which room adds its own reberberations to the sound coming from the speakers. These reflections and reverbs conflict with the virtually generated ones; --that is, the auditory cortex has trouble separating the virtual 3D environment from the real one. But if part of the content coming through the speakers is NOT 3D positioned or "environmentalized", but rather "pure", then, for that part of the sound content, the player will "only" be able to hear the (real world-) room's aural characteristics. Having this dicotomy, the auditory part of the cortex can now better single out virtual versus real contexts, as if "by subtraction".
What I will do with music tracks, though, is make sure the levels are right, and that spectral balance isn't too lopsided, or interferes with other sounds too much. Dynamic adjustment of volume control, also, will probably be needed, so that when many sounds and comm voices are competing for the player's attention, the music doesn't become an annoyance.
So, I'm not sure I agree about being able to change CD's or switch radio stations, except perhaps for having juke boxes at bars... What I would like to see happen, if we do get a lot of music contributed, is styles of music for the different factions, so that when you're flying through Rlaan territory, you hear Rlaan music.

Are there any musicians around here interested in creating content in differen musical scales? I don't mean "scales" in the modern sense of "major, minor, gothic, jazz" whereby different bundles of notes are picked out of the 12-semi-tone, western scale (Pythagoread derived). I mean different tunings.

That would make THE difference! If you look at Diablo, for instance; the music of Diablo has a main track played on accoustic guitar; but if you tried to learn to play it on your guitar you'd be in trouble: The strings were tuned in a particular, non-western way. We could come up with new tunings, or revive ancient scales and assign them to the various factions. In ancient Greece there were 7 popular scales:

Dorian: (11)
- harmonic mode -- 11/11, 12/11, 13/11, 14/11,.. 16/11,.. 18/11,.. 20/11,.. 22/11 (or 2.0, the octave)
- modal mode ------ 11/11, 11/10, 11/9, 11/8, 11/7, 11/6.5, 11/6, 11/5.5 (or 2.0, the octave again)

Phrygian: (12)
- harmonic mode -- 12/12, 13/12, 14/12 .............. 22/12, 24/12
- modal mode ------ 12/12, 12/11, 12/10 .............. 12/6.5, 12/6

Lydian: (13)
- harmonic mode -- 13/13, 14/13, 15/13 .............. 24/13, 26/13
- modal mode ------ 13/13, 13/12, 13/11 .............. 13/7, 13/6.5

Hypo-Dorian: (14)
- harmonic mode -- 14/14, 16/14 .....
- modal mode ------ 14/14, 14/13

Hypo-Phrygian: (16)
- harmonic mode -- 16/16, 18/16, 20/16 .....
- modal mode ------ 16/16, 16/14, .....

Hypo-Lydian: (18)
- harmonic mode -- 18/18, 20/18, ........
- modal mode ------ 18/18, 18/16 ........

Mixolydian: (20)
- harmonic mode -- 20/20, 22/20, ........
- modal mode ------ 20/20, 20/18, ........

I jotted that down from my foggy recollection, from when I read "The Greek Aulos", can't remember the author; and "Genesis of a Music" by Harry Partch. I probably have big mistakes in it, so don't quote my numbers...

And there are many more ancient tunings to pick from. There was never a "pentatonic" scale, a red herring; and the term is pretty meaningless, anyways ("5 notes", yeah, but which?); but say for instance, classic Indian music came in, IIRC, NOT 7, but... 104 scales "sanctioned by the Vedas", which scales were collections out of a pool of ratios, or "shrutis", of which there were 66 in the octave. The Celtic scale and Celtic Harp were supressed, but I heard the original Celtic scale was similar to the ancient Hypo-Lydian from Greece.

Modern Arabic music is probably the only "pop" music in existence that still retains some of the original tunings, against the onslaught of the western, 12-note "McDonnald's" scale, I call it :)
And I think the origins of the Arabic scale are in the classical Turkish scale, but I could be wrong.
And there are many more: classic Chinese, Japanese, many African, many Precolumbian American scales, you name it...
pifactorial
Bounty Hunter
Bounty Hunter
Posts: 170
Joined: Mon Feb 21, 2005 7:04 am
Location: Pomona College
Contact:

Post by pifactorial »

chuck_starchaser wrote:Features Gamma ray laser ranging of the surfaces of other ships to detect actual sound
[nitpick]Using a "Gamma ray laser" to detect sound is like using a hydrogen bomb as a toothpick. Humans can only hear sound waves with wavelengths > 15mm. Therefore you would need a >= 40 GHz "laser" to detect all audible vibrations - this is on the border between radio and microwaves, where radar typically operates. A "Gamma ray laser" would be great for detecting vibrations of individual atoms, or as a weapon.[/nitpick]

As for my contribution, I could help with C++ programming and sounds (sorry, haven't learned Python yet, though), but I'm over 10,000 km from my computer, and only have enough computer access to check email, etc.

EDIT: Chuck, your music ideas sound great - everything sounds great! Mods, if this sound project is as big as it sounds, maybe a "Sound" forum ought to be created, alongside "Artwork".
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

Actually, the ratio of electromagnetic to accoustic wavelength is irrelevant. The reason I said Gamma ray laser is this: Sources of sounds inside other ships produce minute *amplitude* perturbations of the air inside the ship, which hit the walls and induce ever more minute vibrations in the material of the hull, which then reach outer surfaces. We're talking amplitudes (distance differences, or length between closest and furthest range of the vibrations) in the order of microns or amstrongs. So the laser sensor must keep track of the distance of one spot on the outer hull of a ship to sub-micron accuracy, plot that data in time, filter out the huge offsets introduced by the ship's linear motion towards or away from you, clean the signal, and convert it to audio. But to detect relative changes in distance in the order of amstrong, you need short wavelengths. The shorter the wavelength, the faster and more accurate the ranging, and therefore the better the sound quality.

Been thinking about setting up a sound forum at my website to connect with this; then I got carried away and started adding a lot of topics... no postings yet; it's sort of a "secret forum" for now.. ;-) but you can have a look at it http://www.deeplayer.com/VS/.
pifactorial
Bounty Hunter
Bounty Hunter
Posts: 170
Joined: Mon Feb 21, 2005 7:04 am
Location: Pomona College
Contact:

Post by pifactorial »

Good point, I forgot about amplitude. It's obvious that sound quality would improve with higher frequency EMR (it's also obvious that you know what you're talking about). I still think using gamma rays is excessive (this process you describe is often done using infrared light), but it's a really trivial issue not worthy of this forum :wink:.
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

You're right, it probably is excessive. Well, maybe the range of "sound reconstructor module" prices and qualities perhaps warrants extension, after all. Better models woould use shorter and shorter wavelenths to increase sound sensitivity and quality; and then the best of them all, at a whoping 10 million credits, say, could use multiple wavelength laser ranging, in order to dynamically pick the best wavelength for which interference is lowest, so that it can avoid, say, X-ray laser ranging data when the ship being monitored is passing over an X-ray galaxy or neutron star in the background.
zaydana
Site Administrator
Site Administrator
Posts: 478
Joined: Thu Jan 02, 2003 10:05 am
Location: Perth, Western Australia
Contact:

Post by zaydana »

not too much time on my hands, but remember that fictional movie called "farenheit 911"? how they were playing "the roof is on fire" (dunno name of the song) in the tank while going through iraq... I think we would listen to music in battle :) At least I would. Anyways... I know most of us agree with you on the whole soundscape thing, that will be awesome :) But maybe we should ask some non-musician non-technician non-coder people what they want? :P I think we both satisfy "technician" by the sound of it, but on coder/musician we are at oppisite sides and are probably destined to disagree :P
chuck_starchaser
Elite
Elite
Posts: 8014
Joined: Fri Sep 05, 2003 4:03 am
Location: Montreal
Contact:

Post by chuck_starchaser »

There doesn't need to be "disagreement". Different tastes, for sure. But look: How many gameplayers out there know that the 12-string guitar in the Diablo main theme is de-tuned? I'd be surprised if the numbers are as many as one in a thousand. But just about every gameplayer who has played Diablo will agree that the music in Diablo is one of the best music tracks in any game. Many even say it "makes" the game. But what they cannot tell you exactly is "why", and I would strongly suggest to you, 99% of it is due to the de-tuning of that guitar. And many people are nostalgic about the music tracks in games back in the days of Synchlair Spectrums and all that. And I venture to suggest that what they liked about them, but don't know it, is that the notes on those machines came "out of tune" due to technical limitations, and that that is what made the music sound more surreal in a subtle kind of way.

Same thing about accoustics in general: When I try to explain to people how my audio amplifier invention works to hyper-dampen a speaker, and about tri-amping, and about full-size bass-reflex speaker box design, it all goes over their heads; but when I play some music for them through my speaker amp they get goose-bumps and their faces turn pale... ;-)
pifactorial
Bounty Hunter
Bounty Hunter
Posts: 170
Joined: Mon Feb 21, 2005 7:04 am
Location: Pomona College
Contact:

Post by pifactorial »

chuck_starchaser wrote:When I try to explain to people how my audio amplifier invention works to hyper-dampen a speaker, and about tri-amping, and about full-size bass-reflex speaker box design, it all goes over their heads
Amen, brotha!

(Way over my head...) :lol:
Post Reply