bad_alloc crash

Halleck · Post by **Halleck** » Fri Jul 13, 2007 9:50 am

So I'm looking at some ships in test/modelview.mission. After VS has been running for a good several minutes, it quits without warning. This happens at random and I can't figure out any steps to reproduce or specific circumstances that cause this bug.

I'm running SVN HEAD (r11055)

Here's the tail of my stderr:

Code: Select all

SITUATION IS 2 force change 1 bool 1
SITUATION IS RESET TO 2
peace-away-frienly unknown
terminate called after throwing an instance of 'std::bad_alloc'
  what():  St9bad_alloc
Aborted

Safemode?

Post by **safemode** » Sat Jul 14, 2007 2:06 am

several new models were added, you made sure you're running the latest vegastrike if you're running the latest data4.x right?

Also, I've never seen this behavior before. And I've had vegastrike running for hours recently. The only real bug i've noticed is the stuck-on-you bug.

a backtrace could help... Though I wont have time to get to any developing until sunday most likely. I'm working tomorrow and I just worked 14 hours today.

EDIT: Oh, you compiled with no other compiler opts other than -O2 right? anything else leads to erradic behavior (at least on my system). I would also try a compile with -O0 just to be sure it's not something your specific compiler is doing.

Halleck · Post by **Halleck** » Sat Jul 14, 2007 6:42 am

Yeah I do have what's current because I'm the one that added those models.

I really don't know what you mean by those -O compiler options. I haven't modified my build files at all, and build like so:

Code: Select all

./bootstrap-sh && ./configure && make vegastrike

So I should be using whatever defaults are specified by configure/make. I don't even know how to change them even if I wanted to.

Post by **ace123** » Sat Jul 14, 2007 8:45 pm

Looks music related.
11066 should have fixed some race conditions but not all of them I guess.

You should at least be able to play without music (in vssetup)...

Halleck · Post by **Halleck** » Sun Jul 15, 2007 12:20 am

I've had it set to "no sound" for a while, including when I saw this error for the first time. (It's happened again several times but it's hard to track down.)

mduf · Post by **mduf** » Thu Jul 26, 2007 7:29 pm

It also happened to me but I don't think it's a bug of vegastrike (at least in my case). I ran top at the same time, it happened when I run out of memory, I saw that both my memory and my swap were full. Vegastrike try to allocate memory too load a new song, it can't be allocate because there's no enough memory so it throw a bad_alloc.

Here an example of the execution stack a the time it occure

Code: Select all

Core was generated by `vegastrike.11149'.
Program terminated with signal 6, Aborted.
#0  0xffffe410 in __kernel_vsyscall ()
(gdb) bt 20
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7919770 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xb791aef3 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0xb7b0a520 in __gnu_cxx::__verbose_terminate_handler ()
   from /usr/lib/libstdc++.so.6
#4  0xb7b07f55 in std::set_unexpected () from /usr/lib/libstdc++.so.6
#5  0xb7b07f92 in std::terminate () from /usr/lib/libstdc++.so.6
#6  0xb7b080ca in __cxa_throw () from /usr/lib/libstdc++.so.6
#7  0xb7b0850e in operator new () from /usr/lib/libstdc++.so.6
#8  0x0852fec5 in __gnu_cxx::new_allocator<char>::allocate (this=0xb0c68fd4, 
    __n=67071580)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/ext/new_allocator.h:88
#9  0x0852feea in std::_Vector_base<char, std::allocator<char> >::_M_allocate (
    this=0xb0c68fd4, __n=67071580)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:127
#10 0x08532969 in std::vector<char, std::allocator<char> >::_M_fill_insert (
    this=0xb0c68fd4, __position={_M_current = 0x9aebc736 ""}, __n=4096, 
    __x=@0xb0c68f60)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/vector.tcc:357
#11 0x08532b92 in std::vector<char, std::allocator<char> >::insert (
---Type <return> to continue, or q <return> to quit---
    this=0xb0c68fd4, __position={_M_current = 0x9aebc736 ""}, __n=4096, 
    __x=@0xb0c68f60)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:658
#12 0x08532c77 in std::vector<char, std::allocator<char> >::resize (
    this=0xb0c68fd4, __new_size=33539886, __x=0 '\0')
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:426
#13 0x089672f7 in ConvertFormat (ogg=@0xb0c69398) at src/aldrv/al_sound.cpp:403
#14 0x08967b93 in AUDLoadSoundFile (
    s=0x15124b38 "../music/militarybase_p.ogg", info=0x997a050)
    at src/aldrv/al_sound.cpp:478
#15 0x08a20e03 in Muzak::readerThread (input=0x998ede4)
    at src/cmd/music.cpp:392
#16 0xb7e3b504 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#17 0xb79bd51e in clone () from /lib/tls/i686/cmov/libc.so.6
(gdb)

It happened there, because that call need a lot of new memory, so most of the time it's at that moment it run out of memory. But on some few occasion Linux just killed it because the system memory was full.

So if your are like me and have you have an old machine, I think the cheapest solution is bigger swap.

mduf · Post by **mduf** » Thu Jul 26, 2007 8:38 pm

I got the same thing with No sound, it appear vegastrike load the music file anyway and just keep the volume at zero.

I got the same content for the stack. Again it looked like it happened when it run out of memory. But if I'm wrong about the memory thing, I hope the gdb output can help you.

mduf · Post by **mduf** » Thu Jul 26, 2007 9:21 pm

with more details (stack for all threads)

Code: Select all

Core was generated by `vegastrike.11149'.
Program terminated with signal 6, Aborted.
#0  0xffffe410 in __kernel_vsyscall ()
(gdb) thread apply all bt 20

Thread 3 (process 19894):
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7ebbbf6 in __nanosleep_nocancel ()
   from /lib/tls/i686/cmov/libpthread.so.0
#2  0xb7f21228 in SDL_Delay () from /usr/lib/libSDL-1.2.so.0
#3  0x08ab0857 in winsys_process_events () at src/gldrv/winsys.cpp:505
#4  0x08a91ad4 in GFXLoop (main_loop=0x8ab679a <bootstrap_first_loop()>)
    at src/gldrv/gl_init.cpp:506
#5  0x0884f67b in GameUniverse::Loop (this=0x997a0b0, 
    main_loop=0x8ab679a <bootstrap_first_loop()>) at src/universe.cpp:186
#6  0x08ab8240 in main (argc=1, argv=0xbfbae0b4) at src/main.cpp:332

Thread 2 (process 19898):
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7ebb01e in __lll_mutex_lock_wait ()
   from /lib/tls/i686/cmov/libpthread.so.0
#2  0xb7eb7673 in _L_mutex_lock_49 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0x959c250c in ?? ()
#4  0x00000011 in ?? ()
#5  0x08a20916 in Muzak::readerThread (input=0x99784a4)
    at src/cmd/music.cpp:364
#6  0xb7eb5504 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#7  0xb7a3751e in clone () from /lib/tls/i686/cmov/libc.so.6

Thread 1 (process 19897):
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7993770 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xb7994ef3 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0xb7b84520 in __gnu_cxx::__verbose_terminate_handler ()
   from /usr/lib/libstdc++.so.6
#4  0xb7b81f55 in std::set_unexpected () from /usr/lib/libstdc++.so.6
#5  0xb7b81f92 in std::terminate () from /usr/lib/libstdc++.so.6
#6  0xb7b820ca in __cxa_throw () from /usr/lib/libstdc++.so.6
#7  0xb7b8250e in operator new () from /usr/lib/libstdc++.so.6
#8  0x0852fec5 in __gnu_cxx::new_allocator<char>::allocate (this=0xb14e3fd4, 
    __n=67019356)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/ext/new_allocator.h:88
#9  0x0852feea in std::_Vector_base<char, std::allocator<char> >::_M_allocate (
    this=0xb14e3fd4, __n=67019356)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:127
#10 0x08532969 in std::vector<char, std::allocator<char> >::_M_fill_insert (
    this=0xb14e3fd4, __position={_M_current = 0x6bdc0136 ""}, __n=4096, 
    __x=@0xb14e3f60)
---Type <return> to continue, or q <return> to quit---
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/vector.tcc:357
#11 0x08532b92 in std::vector<char, std::allocator<char> >::insert (
    this=0xb14e3fd4, __position={_M_current = 0x6bdc0136 ""}, __n=4096, 
    __x=@0xb14e3f60)
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:658
#12 0x08532c77 in std::vector<char, std::allocator<char> >::resize (
    this=0xb14e3fd4, __new_size=33513774, __x=0 '\0')
    at /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:426
#13 0x089672f7 in ConvertFormat (ogg=@0xb14e4398) at src/aldrv/al_sound.cpp:403
#14 0x08967b93 in AUDLoadSoundFile (s=0x95929830 "../music/Machina.ogg", 
    info=0x9979d08) at src/aldrv/al_sound.cpp:478
#15 0x08a20e03 in Muzak::readerThread (input=0x9978534)
    at src/cmd/music.cpp:392
#16 0xb7eb5504 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#17 0xb7a3751e in clone () from /lib/tls/i686/cmov/libc.so.6
(gdb)

Post by **hellcatv** » Fri Jul 27, 2007 10:23 pm

it's allocating 33 megs here.... that doesn't sound like a lot to me...but what are your system specs...how much swap space do you really have free for instance?

mduf · Post by **mduf** » Mon Jul 30, 2007 3:05 pm

I should have said a very old system

athlon 600Mhz
384 MB Ram
~400 MB swap
with a GeForce 4

It was able to run previous svn version (those that still use the sound server)

When Vegastrike is running I have ~50 MB off free memory (swap + ram). When it load a new song it can fall to ~4 MB or sometime it does a bad_alloc.

With spec like that you can reproduce the std::bad_alloc thing in 5 or 10 minutes of play.

I didn't try with bigger with a bigger swap, but if I do I'll tell if it make things better.

Halleck · Post by **Halleck** » Wed Aug 15, 2007 4:33 am

This appears to have been fixed for me in 11256.

I observed the virtual memory used by VS climb over the course of several minutes from 700m to 2900m where it died with the bad_alloc message. This no longer happens as of 11256, where vmem is holding at about 250m according to top.

So see if this is fixed on your linux system.
Apparently this still happens on windows...

Post by **jackS** » Wed Aug 15, 2007 7:06 am

Fixed for windows now too. Am hunting a (minor) unrelated bug ATM. I'll commit a new binary sometime before I go to sleep.

mduf · Post by **mduf** » Wed Aug 15, 2007 9:46 pm

I tried 11261 and I still had a bad_alloc after few minutes of play. So I extended my swap to ~1.2GB and now everything is perfect, no bad_alloc anymore.

Post by **loki1950** » Wed Aug 15, 2007 10:00 pm

@mduf glad you found a fix has it affected the game experience do you get better frame rates and just smoother because is so i might bump up my swap from 1 GB although my system might not benefit from the tweak

Enjoy the Choice

Post by **jackS** » Wed Aug 15, 2007 10:22 pm

mduf wrote:I tried 11261 and I still had a bad_alloc after few minutes of play. So I extended my swap to ~1.2GB and now everything is perfect, no bad_alloc anymore.

What we fixed was the memory leak with music -- if you actually run out of memory, it'll still throw a bad_alloc when it tries to allocate a new music buffer.

charlieg · Post by **charlieg** » Wed Aug 15, 2007 10:29 pm

Sounds like VS needs a bit of profiling...

Post by **loki1950** » Wed Aug 15, 2007 10:33 pm

In hand charlieg check some of safemodes threads

Enjoy the Choice

Post by **jackS** » Wed Aug 15, 2007 10:59 pm

re: profiling -- Well, the nature of the music bug was actually a little more complicated than just a missing free/delete -- the delete was there, but we'd forgotten to decrement the reference count, so the delete wasn't actually occurring (Aldriver thought the buffer was still in use

)

Also, don't really need a profiler to notice a 20-80 MB/song leak

Still -- more profiling is always better.

The new version of VS has a much bigger memory footprint than old versions, just to begin with, so some memory issues (on machines with smaller amounts of memory) may be difficult to avoid in the short term.