Source Code Project Mantis - FSSCP
View Issue Details
0001951FSSCPmultiplayerpublic2009-07-05 16:302014-09-10 22:50
Assigned ToEchelon9 
PlatformOSOS Version
Product Version3.6.11 
Target VersionFixed in Version 
Summary0001951: Standalone crashes - Ship <ship>does not have subsystem <subsystem> linked into the model file
DescriptionBeen getting a bunch of new standalone crashes in the last couple of weeks. Looks like it's 2 different ones but they may be connected since they both crash at the same line sometimes. First one is a bunch of assertions that always seem to start when a Lilith model is loaded. This usually leads to the second one which is a crash but this crash also happens on it's own. Attaching logs
TagsNo tags attached.
related to 0002015resolved Echelon9 Assert: team != -1 in multi_respawn.cpp 
Attached Filesrar 1951.rar (46,094) 2009-07-05 16:32
log fs2_open_vp_1733.log (190,919) 2009-07-08 16:02
rar fs2_open_both.rar (6,320) 2009-07-09 00:57
log fs2_open-07-11-09.log (184,591) 2009-07-11 19:59
log fs2_open-07-18-09.log (88,164) 2009-07-19 03:20
jpg players1.jpg (168,365) 2009-08-15 17:51

jpg players2.jpg (184,912) 2009-08-15 17:52

txt lilith.txt (8,979) 2009-08-25 18:50
txt 1951.txt (25,778) 2010-07-08 15:50
rar 1951_logs.rar (27,379) 2010-07-09 18:47
rar 1951_3_6_15.rar (83,714) 2012-12-01 22:56
rar 1951_3_6_15_2.rar (97,436) 2012-12-13 12:29
? fubar_m18.fs2 (22,340) 2012-12-19 23:40
rar 1951_3_6_19.rar (96,842) 2013-03-28 22:57

2009-07-05 16:33   
Just a note that this was with retail FS2 data running on the standalone.
2009-07-08 16:03   
Attached additional log. First time seen on the MediaVP standalone. Slightly different set of problems but crashed at the same spot.
2009-07-09 00:15   
OK this has officially become an epidemic. Attached another log. This one shows something that I don't know if the others do or not. The Lilith model loads fine at least one time before it goes nuts and starts with the assertions on another load.
2009-07-11 19:58   
No crashes in today's session using the RC3 no warnings build. Quite a few asserts but they are all the same. This was with the MediaVP. Attaching log just in case.
2009-07-14 04:22   
Really need the callstack for this one.
The culprit is vm_project_point_onto_plane, which is only called from 3 places, but tracing through is going to be very hard without knowing which one is being called.
2009-07-19 03:22   
(Last edited: 2009-07-19 03:25)
Well it seems the big problems are well patched. Standalone did crash once in yesterdays games. I've seen the same crash before. Log attached. I asked them if someone dropped before the game started and was told that someone was lost in mission load. Same mission was played after restart with no crash.

2009-07-19 03:31   
I'd still like to see a callstack from when this occurs since it implies that a vector being passed to vm_project_point_onto_plane hasn't been normalised, which is really bad.
Are there any instructions for getting this to occur?
2009-07-19 03:46   
Didn't seen any nul vec3d errors today just the the one crash.
2009-07-25 18:19   
Still no luck on getting a debug trace. It's up for testing but since today was the normal Saturday games they were using 3.6.10. Did get another one of those dropped player crashes. Had the debug_filter turned on. I'd attach the file but it's 75meg and even zipped still over a meg. You can grab it here:
2009-07-26 22:08   
FUBAR: With this one, it might be worth the inconvenience of dropping some players in order to get a callstack - Asserts should never be ignored (who stuck Cmdline_nowarn into Assert?)
2009-07-27 00:32   
Already planned and the server to do that is up and running. Just waiting on our main tester to get unsick and give it a go. I didn't want to use the main Saturday games since you never know what version they are running, if they are using the MediaVPs or not, if they have any rogue tables, etc. That and there aren't that many multi players now. Hate turning new players off because of testing crashes.
2009-08-06 20:17   
Well we got to hits with the debugger yesterday. One as a client but it was a null vec3d so it's possibly related. The other was on the standalone and involved the player drop problem (still have the debugger open on this one). The think is it doesn't look like it was actually a player drop. I just happened to check on the standalone at the right time to actually see it happen. 6 IP addresses where showing as connected on the standalone but I could only see 3 players in the status window. Right after that it crashed. I asked and there were 6 or 7 people in the game. Looking at the multi.log earlier today I noticed that there was a FS2NetD refresh right about the time of the crash. This makes me wonder if it is that refresh that is causing the issue. It does cause problems for normal hosting such as the awarding of all medals in debrief bug.
2009-08-06 22:32   
OK I just saw this about to happen again. 3 players on server and even chatting but none showing in the player info pull down. Checked the debug log on the stnadalone and it had just done a FS2netD refresh. I hit reset all before it crashed and when the joined again the player info screen was fine. Definitely something going on here even if it's not related to the crash.
2009-08-15 17:55   
(Last edited: 2009-08-15 17:57)
Attached 2 screen shots of what appears to be the cause of the crash. You'll notice 7 IP connections showing but only 4 players. Seems that when one of those "ghost" players goes to start a mission or respawn it crashes.

Almost positive this is related to 1827

2009-08-25 18:50   
(Last edited: 2009-08-25 18:52)
While most of the stuff relating to this seems to be fixed or known I did get the Lilith problem again last night. We were discussing it last night but Wanderer had to leave and I felt the sudden need to get horizontal so we didn't make much progress. Took a second look at it today and it seems (big guess here)it's looking for an already cached subsystem at the wrong location. sip->subsystems[j].model_num is 515 and sip->model_num is 8583.

I'll attach the call stack and variables. I still have the debugger open so I can't get the fs2_open.log right now. Mission was cet_m03 which only has one Lilith. I'll be checking the SCP IRC channel periodically tonight so if you need more info and I'm logged in there just ask.

2010-04-05 12:55   
Changing the subject because this title could get confusing very soon.
2010-04-05 18:32   
Maybe we should change it to the actual error that is left.

Anyway think I had this happen again last night. Saw the server was crashed with a similar error but didn't have time to check it yet.
2010-04-05 19:03   
That or open a new bug that's not cluttered and close this one as fixed. If an issue was fixed this should probably be closed, it was never really supposed to be a parent type ticket.
2010-07-08 15:50   
Apparently not fixed. Just found a standalone crashed with it. Retail data. r6278.
2010-07-09 18:51   
Changed the topic to the error and uploaded the logs from yesterday's crash.
2010-07-10 08:03   
This latest example looks to be a Debug build correctly catching a model problem with a Warning() call. Please open a new case for this, but I'd say it is a model bug (although if with Retail data we will have to look into it).
2010-07-10 15:54   
Nope not a model bug. This happens on a reload of a model that is already loaded although it's usually the Lilith for some reason. Somewhere one of the indexes is getting changed or stepped on in memory.
2012-11-11 00:19   
Is this bug still a problem?
2012-11-11 11:23   
I have no clue at this point. The standalones were crashing so often with multiple issues I haven't tried to run one in over a year.
2012-11-11 13:07   
The model code has been changed a lot in the past two years and the cause of this bug may have been fixed. It would help to run a standalone to be sure.
2012-11-23 02:08   
Has this one resurfaced?
2012-11-23 12:17   
With all the new warnings and FS2netD server going down they haven't been up long enough to really test.
2012-12-01 21:35   
Unfortunately it still lives. Just got it with retail data loading the Gany.

I'll pack up the logs and attach them as soon as I get out of the debugger.
2012-12-02 00:13   
Is that the complete log? Because it doesn't display the expected -mod information at the beginning. Can you post the command line you used?
2012-12-02 00:38   
Standalone logs don't include that info for some reason.

Here it is. No mod was active. Retail data used.

C:\Games\FreeSpace2\fs2_open_3_6_15d_INF_SSE2.exe -nomotiondebris -missile_lighting -3dshockwave -post_process -3dwarp -warp_flash -snd_preload -standalone -fps -spec -glow -env
2012-12-13 12:29   
Another one this time with the Lilith again.
2012-12-19 23:23   
Can you attach the mission that was loaded?
2012-12-19 23:41   
Sure but it's not specific to that mission.
2012-12-20 02:02   
I've added some additional debug code. Please post the new warning message if it occurs again.
2013-01-04 10:07   
Has anyone tested the debug code I added? Does this problem still occur?
2013-01-04 13:52   
I've had the debug code up since it was added. Unfortunately I've only seen a few uses of the standalones in the last couple of weeks and most of the time it has been 1 player just sitting there waiting for other players not actually playing. Hoping things will pick back up a bit once people get tired of their xmas presents and go back to playing FS2.
2013-03-10 04:42   
I have not seen an occurrence of this in months. Last one I had I believe was shortly before commits 9423 and 9424. Possible plugging those memory leaks fixed this issue?
2013-03-10 15:23   
That's terrific news. I'm going to assume revision 9423 fixed this, as it's highly plausible that it's related.
2013-03-28 22:57   
Unfortunately this has occurred again. With the changes it hit an assert in model_get(). Attaching new log, stack, and variables.
2013-11-30 20:21   
Can we get clear steps for which mission will most typically trigger this bug?

I would really like to see an AddressSanitizer-enabled standalone server report this bug, as we will get a precise measure of any memory corruption going on.
2014-09-06 23:59   
Bump because we need steps to reproduce this.
2014-09-10 10:08   
I'll try to reproduce, but any other assistance reproducing appreciated.
2014-09-10 22:50   
Yeah, that comment was directed mainly at FUBAR.

Issue History
2009-07-05 16:30FUBAR-BDHRNew Issue
2009-07-05 16:32FUBAR-BDHRFile Added: 1951.rar
2009-07-05 16:33FUBAR-BDHRNote Added: 0011053
2009-07-08 16:02FUBAR-BDHRFile Added: fs2_open_vp_1733.log
2009-07-08 16:03FUBAR-BDHRNote Added: 0011062
2009-07-09 00:15FUBAR-BDHRNote Added: 0011063
2009-07-09 00:25chief1983Prioritynormal => high
2009-07-09 00:57FUBAR-BDHRFile Added: fs2_open_both.rar
2009-07-11 19:58FUBAR-BDHRNote Added: 0011075
2009-07-11 19:59FUBAR-BDHRFile Added: fs2_open-07-11-09.log
2009-07-14 04:22portej05Note Added: 0011083
2009-07-19 03:20FUBAR-BDHRFile Added: fs2_open-07-18-09.log
2009-07-19 03:22FUBAR-BDHRNote Added: 0011091
2009-07-19 03:25FUBAR-BDHRNote Edited: 0011091
2009-07-19 03:31portej05Note Added: 0011092
2009-07-19 03:46FUBAR-BDHRNote Added: 0011094
2009-07-25 18:19FUBAR-BDHRNote Added: 0011105
2009-07-26 22:08portej05Note Added: 0011106
2009-07-27 00:32FUBAR-BDHRNote Added: 0011108
2009-08-06 20:17FUBAR-BDHRNote Added: 0011126
2009-08-06 22:32FUBAR-BDHRNote Added: 0011127
2009-08-15 17:51FUBAR-BDHRFile Added: players1.jpg
2009-08-15 17:52FUBAR-BDHRFile Added: players2.jpg
2009-08-15 17:55FUBAR-BDHRNote Added: 0011136
2009-08-15 17:57FUBAR-BDHRNote Edited: 0011136
2009-08-25 18:50FUBAR-BDHRNote Added: 0011159
2009-08-25 18:50FUBAR-BDHRFile Added: lilith.txt
2009-08-25 18:52FUBAR-BDHRNote Edited: 0011159
2009-11-05 18:26chief1983Statusnew => acknowledged
2009-11-05 18:26chief1983Product Version3.6.9 => 3.6.11
2009-11-05 18:26chief1983Target Version => 3.6.12 RC1
2010-04-05 12:55chief1983Note Added: 0011856
2010-04-05 12:55chief1983SummaryRC3 standalone crashes => 3.6.10 RC3 standalone crashes
2010-04-05 18:32FUBAR-BDHRNote Added: 0011857
2010-04-05 19:03chief1983Note Added: 0011858
2010-07-08 15:50FUBAR-BDHRNote Added: 0012205
2010-07-08 15:50FUBAR-BDHRFile Added: 1951.txt
2010-07-09 18:47FUBAR-BDHRFile Added: 1951_logs.rar
2010-07-09 18:50FUBAR-BDHRSummary3.6.10 RC3 standalone crashes => Standalone crashes - Ship <ship>does not have subsystem <subsystem> linked into the model file
2010-07-09 18:51FUBAR-BDHRNote Added: 0012209
2010-07-10 08:03Echelon9Note Added: 0012214
2010-07-10 15:54FUBAR-BDHRNote Added: 0012224
2010-08-05 16:03chief1983Target Version3.6.12 RC1 => 3.7.2
2012-04-03 16:25Echelon9Relationship addedrelated to 0002015
2012-11-11 00:19Goober5000Note Added: 0014021
2012-11-11 11:23FUBAR-BDHRNote Added: 0014026
2012-11-11 13:07Goober5000Note Added: 0014029
2012-11-16 00:56Goober5000Target Version3.7.2 => 3.6.16
2012-11-23 02:08karajormaNote Added: 0014159
2012-11-23 12:17FUBAR-BDHRNote Added: 0014161
2012-12-01 21:35FUBAR-BDHRNote Added: 0014245
2012-12-01 22:56FUBAR-BDHRFile Added: 1951_3_6_15.rar
2012-12-02 00:13Goober5000Note Added: 0014250
2012-12-02 00:38FUBAR-BDHRNote Added: 0014251
2012-12-13 12:29FUBAR-BDHRFile Added: 1951_3_6_15_2.rar
2012-12-13 12:29FUBAR-BDHRNote Added: 0014434
2012-12-19 23:23Goober5000Note Added: 0014548
2012-12-19 23:40FUBAR-BDHRFile Added: fubar_m18.fs2
2012-12-19 23:41FUBAR-BDHRNote Added: 0014550
2012-12-20 02:02Goober5000Changeset attached => fs2open trunk r9455
2012-12-20 02:02Goober5000Note Added: 0014552
2013-01-04 10:07Goober5000Note Added: 0014614
2013-01-04 13:52FUBAR-BDHRNote Added: 0014615
2013-03-10 04:42FUBAR-BDHRNote Added: 0014761
2013-03-10 15:23Goober5000Note Added: 0014762
2013-03-10 15:23Goober5000Assigned To => Goober5000
2013-03-10 15:23Goober5000Statusacknowledged => resolved
2013-03-10 15:23Goober5000Resolutionopen => fixed
2013-03-28 22:57FUBAR-BDHRNote Added: 0014850
2013-03-28 22:57FUBAR-BDHRStatusresolved => feedback
2013-03-28 22:57FUBAR-BDHRResolutionfixed => reopened
2013-03-28 22:57FUBAR-BDHRFile Added: 1951_3_6_19.rar
2013-04-19 17:19chief1983Target Version3.6.16 => 3.7.0
2013-11-30 20:21Echelon9Note Added: 0015471
2014-09-06 23:57Goober5000Changeset attached => fs2open trunk r9423
2014-09-06 23:59Goober5000Note Added: 0016271
2014-09-06 23:59Goober5000Assigned ToGoober5000 => Echelon9
2014-09-06 23:59Goober5000Target Version3.7.0 =>
2014-09-10 10:08Echelon9Note Added: 0016273
2014-09-10 22:50Goober5000Note Added: 0016274