2018-03-17 22:36 EDT

View Issue Details Jump to Notes ] Related Changesets ]
IDProjectCategoryView StatusLast Update
0002503FSSCPPilot datapublic2013-01-20 04:26
Assigned Toniffiwan 
PrioritynormalSeverityminorReproducibilityunable to reproduce
Product VersionAntipodes 8 
Target Version3.7.0Fixed in Version 
Summary0002503: Pilot files "corrupted" with huge amount of data in red alert section
DescriptionAs per the following forum posts, there seems to be an intermittent bug that causes the new pilot files (i.e. per campaign file) to grow to ridiculous sizes. i.e. the red alert section contains over 50,000 (up to 7+ million!) wingman entries - considering that we're supposed to be limited to 400 (MAX_SHIPS) ship entries this doesn't make any sense.

This post indicates that the problem occurred during a normal play through of WingsOfDawn.


This post indicates that the problem occurred during a play through of the FS2 campaign which included switching between the fs2/blueplanet/blueplanet2 campaigns (i.e. a big no no for the old pilot code, but should be OK with the ant8 pilot code).


I've tried to reproduce the problem with just playing through WoD. I've played through the campaign up to M17 (which includes two red alert missions) by mostly using the cheats to pass through levels quickly, but I haven't been able to reproduce the issue so far.
Additional InformationI created this bug report to track all instances of the issue that I find.
TagsNo tags attached.
Attached Files
  • 7z file icon WOD Pilot Files.7z (55,145 bytes) 2011-09-18 00:38
  • 7z file icon niffi-ant8-pilot.7z (179,300 bytes) 2011-09-18 00:43
  • patch file icon mantis2503-1.patch (767 bytes) 2011-09-18 00:44 -
    Index: code/missionui/redalert.cpp
    --- code/missionui/redalert.cpp	(revision 7733)
    +++ code/missionui/redalert.cpp	(working copy)
    @@ -618,6 +618,8 @@
     		red_alert_store_subsys_status(&ras, shipp);
     		Red_alert_wingman_status.push_back( ras );
    +		// niffiwan: trying to track down red alert bug creating HUGE pilot files 
    +		Assert( (Red_alert_wingman_status.size() <= MAX_SHIPS) );
     	// store exited ships that did not die
    @@ -640,6 +642,8 @@
     			red_alert_store_subsys_status(&ras, NULL);
     			Red_alert_wingman_status.push_back( ras );
    +			// niffiwan: trying to track down red alert bug creating HUGE pilot files 
    +			Assert( (Red_alert_wingman_status.size() <= MAX_SHIPS) );
    patch file icon mantis2503-1.patch (767 bytes) 2011-09-18 00:44 +
  • 7z file icon MjnPlayer.7z (44,931 bytes) 2011-09-20 05:46
  • 7z file icon players.7z (72,201 bytes) 2013-01-07 22:12
  • 7z file icon players-rodo.7z (76,069 bytes) 2013-01-08 19:36
  • 7z file icon crybertrance_players_bp_bp2.7z (95,616 bytes) 2013-01-10 03:58
  • zip file icon RedAlertTest.zip (6,463 bytes) 2013-01-18 19:31
  • patch file icon mantis2503.patch (487 bytes) 2013-01-18 21:53 -
    diff --git a/code/pilotfile/csg.cpp b/code/pilotfile/csg.cpp
    index 01ad61f..d368584 100644
    --- a/code/pilotfile/csg.cpp
    +++ b/code/pilotfile/csg.cpp
    @@ -769,7 +769,9 @@ void pilotfile::csg_read_redalert()
    -	Red_alert_wingman_status.reserve( list_size );
    +	// about to read new redalert data so flush any existing data
    +	// otherwise wingman entries will multiply like rabbits
    +	Red_alert_wingman_status.clear();
     	cfread_string_len(t_string, MAX_FILENAME_LEN, cfp);
    patch file icon mantis2503.patch (487 bytes) 2013-01-18 21:53 +




niffiwan (developer)

Note how well the pilot files compress (1GB down to 180kb) because of all the repeated data in the offending pilot files!

I noted that pilotfile::csg_write_redalert writes the size of Red_alert_wingman_status and this size is way too big. Red_alert_wingman_status only has entries added to it in one place, red_alert_store_wingman_status. Therefore I've added some asserts to prevent there being more than MAX_SHIPS entries added to Red_alert_wingman_status.

The patch I created does not solve the root cause of the problem, however it should:

1) stop the problem from going "too far" when it does occur
2) give some more information into what is causing the problem (possibly either Ship_obj_list or Ships_exited)
3) make it very obvious to the player when it does occur, hopefully producing more bug reports. In theory, the problem could occur with a fairly small number red-alert wingman (say 500) and the player may not notice...



Echelon9 (developer)

I've committed the additional Assert() to Antipodes branch.


MjnMixael (manager)

At the time of my issue, I was playing through ST:R I got to a non-red alert mission. I played part of the mission and decided to exit out and alter some table options. Specifically I added $2D RADAR IMAGE to a few specialized ship entries. Upon trying to load the mission again, I would get a malloc failed.

Naturally I switched to debug and tried to load, but I couldn't get past pilot select. It died parsing the red-alert section of mike.str.csg.


niffiwan (developer)

MjnMixael - your pilot file was a little odd (well, more odd). The campaign progress was up to str14, however str02 was the last stored red-alert mission, not str10 as you would expect! The stored wingman also seemed to match str02, with alpha 1-3 & beta 1-3 (not 1-4 plus gamma/delta).

Did you play through without exiting the game from str02 (or str10) until exiting after playing part of str14?


MjnMixael (manager)

Last edited: 2011-09-21 09:55

Definitely not. I think during this session I played from str12.

EDIT: I remembered. On str10 I encountered this issue...


... and just continued without most of my wingmen and cheated through that next mission.


niffiwan (developer)

Last edited: 2013-01-07 22:13

View 4 revisions

:( looks like this is still occurring, reported by Phantom_Hoover on the new BP:AoA release...

edit: and another one, switch from AoA to WiH act3: http://www.hard-light.net/forums/index.php?topic=70746.msg1664219#msg1664219 (players.7z attachment)


rodo (reporter)

Here's the file, as requested by niffiwan.

I've tried to make some testing, but came to no real conclusion aside from this being a pilot code problem apparently.

Here's the details or walkthrough:

1- Had a ton of old and new pilot files on the pilot folders, apparently deleting the pilot directly from FS's interface will not delete the files on the disk, I suppose you already know this.

2- Started playing BP:AoA with the newly created WiH player, beated the game and proceded to start WiH:Part 1.

3- After beating the last mission of part 1 I exited and started playing with the command line options to change the enviromental lighting, tested the effects of said changes playing the last mission of part 1 once, then on the second test run I got the issue:

After selecting the player from the list at the start of FS the game would almost freeze and then jump back to reality, guess because of the massive file it's got to read, after that if I tried to continue the campaign, or play any mission from the list of available missions it would crash.

That's all I think, hope it helps.



niffiwan (developer)

Thanks for the info!

Just on the pilot files and deleting them from disk, FSO does delete the pilots from disk when you delete from the interface, however FSO also reconverts all your old-format pilots everytime you re-enter FSO. We need to do something else about that... perhaps rename the old pilots after they've been converted.


niffiwan (developer)

Last edited: 2013-01-12 04:16

View 3 revisions

Well this is annoying. The previously added assertions are not triggering when the problem occurs. Therefore maybe we're reading in dud data from a misaligned .csg.

An assertion on "list_size" when reading the .csg will probably trigger after the issue has occurred.

An assertion on "list_size" in pilotfile::csg_write_redalert() is guaranteed to corrupt the .csg when it triggers. Then again, the pilot is already being corrupted by this bug... ideally I'd like to limit the number of written entries to something like MAX_SHIPS, but also warn the player about it, and I don't think a proper warning can be generated in non-debug. Unless I setup some sort of delayed assertion to trigger after the .csg has finished being written...

edit: I'm a dope - add an assertion before you start writing any of the .csg :)

edit2: committed the assertion in r9497.


niffiwan (developer)

Last edited: 2013-01-13 17:51

View 2 revisions

I have an idea about the cause. When reading the red alert data from the pilot file, the existing in memory red alert data is not cleared. Therefore I think that we're getting the size of the red alert data section doubled each time the pilot file is read & written after any red alert mission is played. I haven't validated this, but this idea requires that the pilot file is both read & written between missions.

Also, I managed to reproduce the start of the issue by doing the following:
1) play BP:AoA journey of 1000 miles & complete it
2) check pilot file while game is in red alert briefing - 4x entries in red alert
3) play start of next red alert mission & exit mission & FSO
4) check pilot file - 8x entries in red alert, two (what look like) duplicate sets of 4x ships

I'm going to do some more testing & investigation of this idea next weekend (after my holidays)


Yarn (developer)

I have uploaded a campaign that should trigger this bug reliably after the first mission. Hope it helps!


niffiwan (developer)

Yarn: thanks for the test campaign, that was very useful :)

I have a solution - but I can't commit it until SVN is available again. In the meantime, any comments on the patch are welcome.


niffiwan (developer)

Fix committed to antipodes@9507.

+Related Changesets

-Issue History
Date Modified Username Field Change
2011-09-18 00:38 niffiwan New Issue
2011-09-18 00:38 niffiwan File Added: WOD Pilot Files.7z
2011-09-18 00:43 niffiwan File Added: niffi-ant8-pilot.7z
2011-09-18 00:44 niffiwan File Added: mantis2503-1.patch
2011-09-18 00:52 niffiwan Note Added: 0012833
2011-09-18 01:39 Echelon9 Note Added: 0012834
2011-09-20 05:46 niffiwan File Added: MjnPlayer.7z
2011-09-20 11:08 MjnMixael Note Added: 0012837
2011-09-21 06:13 niffiwan Note Added: 0012838
2011-09-21 08:34 MjnMixael Note Added: 0012840
2011-09-21 09:55 MjnMixael Note Edited: 0012840
2012-11-25 03:22 niffiwan Target Version => 3.7.0
2013-01-03 22:33 niffiwan Note Added: 0014612
2013-01-04 01:36 niffiwan Assigned To => niffiwan
2013-01-04 01:36 niffiwan Status new => assigned
2013-01-07 21:23 niffiwan Note Edited: 0014612 View Revisions
2013-01-07 21:25 niffiwan Note Edited: 0014612 View Revisions
2013-01-07 22:12 niffiwan File Added: players.7z
2013-01-07 22:13 niffiwan Note Edited: 0014612 View Revisions
2013-01-08 19:34 rodo Note Added: 0014621
2013-01-08 19:36 rodo File Added: players-rodo.7z
2013-01-08 19:46 niffiwan Note Added: 0014622
2013-01-10 03:58 niffiwan File Added: crybertrance_players_bp_bp2.7z
2013-01-10 05:10 niffiwan Note Added: 0014624
2013-01-10 15:54 niffiwan Note Edited: 0014624 View Revisions
2013-01-12 04:16 niffiwan Note Edited: 0014624 View Revisions
2013-01-13 17:16 niffiwan Note Added: 0014649
2013-01-13 17:51 niffiwan Note Edited: 0014649 View Revisions
2013-01-18 19:31 Yarn Note Added: 0014652
2013-01-18 19:31 Yarn File Added: RedAlertTest.zip
2013-01-18 21:53 niffiwan File Added: mantis2503.patch
2013-01-18 21:55 niffiwan Note Added: 0014653
2013-01-18 21:55 niffiwan Status assigned => code review
2013-01-20 04:26 niffiwan Changeset attached => fs2open antipodes r9507
2013-01-20 04:26 niffiwan Note Added: 0014657
2013-01-20 04:26 niffiwan Status code review => resolved
2013-01-20 04:26 niffiwan Resolution open => fixed
+Issue History