View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0002503 | FSSCP | Pilot data | public | 2011-09-18 04:38 | 2013-01-20 09:26 |
| Reporter | niffiwan | Assigned To | niffiwan | ||
| Priority | normal | Severity | minor | Reproducibility | unable to reproduce |
| Status | resolved | Resolution | fixed | ||
| Product Version | Antipodes 8 | ||||
| Target Version | 3.7.0 | ||||
| Summary | 0002503: Pilot files "corrupted" with huge amount of data in red alert section | ||||
| Description | As per the following forum posts, there seems to be an intermittent bug that causes the new pilot files (i.e. per campaign file) to grow to ridiculous sizes. i.e. the red alert section contains over 50,000 (up to 7+ million!) wingman entries - considering that we're supposed to be limited to 400 (MAX_SHIPS) ship entries this doesn't make any sense. This post indicates that the problem occurred during a normal play through of WingsOfDawn. http://www.hard-light.net/forums/index.php?topic=78286.0 This post indicates that the problem occurred during a play through of the FS2 campaign which included switching between the fs2/blueplanet/blueplanet2 campaigns (i.e. a big no no for the old pilot code, but should be OK with the ant8 pilot code). http://www.hard-light.net/forums/index.php?topic=77493.0 I've tried to reproduce the problem with just playing through WoD. I've played through the campaign up to M17 (which includes two red alert missions) by mostly using the cheats to pass through levels quickly, but I haven't been able to reproduce the issue so far. | ||||
| Additional Information | I created this bug report to track all instances of the issue that I find. | ||||
| Tags | No tags attached. | ||||
|
2011-09-18 04:38
|
|
|
2011-09-18 04:43
|
|
|
2011-09-18 04:44
|
mantis2503-1.patch (767 bytes)
Index: code/missionui/redalert.cpp =================================================================== --- code/missionui/redalert.cpp (revision 7733) +++ code/missionui/redalert.cpp (working copy) @@ -618,6 +618,8 @@ red_alert_store_subsys_status(&ras, shipp); Red_alert_wingman_status.push_back( ras ); + // niffiwan: trying to track down red alert bug creating HUGE pilot files + Assert( (Red_alert_wingman_status.size() <= MAX_SHIPS) ); } // store exited ships that did not die @@ -640,6 +642,8 @@ red_alert_store_subsys_status(&ras, NULL); Red_alert_wingman_status.push_back( ras ); + // niffiwan: trying to track down red alert bug creating HUGE pilot files + Assert( (Red_alert_wingman_status.size() <= MAX_SHIPS) ); } } |
|
|
Note how well the pilot files compress (1GB down to 180kb) because of all the repeated data in the offending pilot files! I noted that pilotfile::csg_write_redalert writes the size of Red_alert_wingman_status and this size is way too big. Red_alert_wingman_status only has entries added to it in one place, red_alert_store_wingman_status. Therefore I've added some asserts to prevent there being more than MAX_SHIPS entries added to Red_alert_wingman_status. The patch I created does not solve the root cause of the problem, however it should: 1) stop the problem from going "too far" when it does occur 2) give some more information into what is causing the problem (possibly either Ship_obj_list or Ships_exited) 3) make it very obvious to the player when it does occur, hopefully producing more bug reports. In theory, the problem could occur with a fairly small number red-alert wingman (say 500) and the player may not notice... Comments? |
|
|
I've committed the additional Assert() to Antipodes branch. |
|
2011-09-20 09:46
|
|
|
|
At the time of my issue, I was playing through ST:R I got to a non-red alert mission. I played part of the mission and decided to exit out and alter some table options. Specifically I added $2D RADAR IMAGE to a few specialized ship entries. Upon trying to load the mission again, I would get a malloc failed. Naturally I switched to debug and tried to load, but I couldn't get past pilot select. It died parsing the red-alert section of mike.str.csg. |
|
|
MjnMixael - your pilot file was a little odd (well, more odd). The campaign progress was up to str14, however str02 was the last stored red-alert mission, not str10 as you would expect! The stored wingman also seemed to match str02, with alpha 1-3 & beta 1-3 (not 1-4 plus gamma/delta). Did you play through without exiting the game from str02 (or str10) until exiting after playing part of str14? |
|
|
Definitely not. I think during this session I played from str12. EDIT: I remembered. On str10 I encountered this issue... http://www.hard-light.net/forums/index.php?topic=61281.msg1530478#msg1530478 ... and just continued without most of my wingmen and cheated through that next mission. |
|
|
:( looks like this is still occurring, reported by Phantom_Hoover on the new BP:AoA release... edit: and another one, switch from AoA to WiH act3: http://www.hard-light.net/forums/index.php?topic=70746.msg1664219#msg1664219 (players.7z attachment) |
|
|
|
|
|
Here's the file, as requested by niffiwan. I've tried to make some testing, but came to no real conclusion aside from this being a pilot code problem apparently. Here's the details or walkthrough: 1- Had a ton of old and new pilot files on the pilot folders, apparently deleting the pilot directly from FS's interface will not delete the files on the disk, I suppose you already know this. 2- Started playing BP:AoA with the newly created WiH player, beated the game and proceded to start WiH:Part 1. 3- After beating the last mission of part 1 I exited and started playing with the command line options to change the enviromental lighting, tested the effects of said changes playing the last mission of part 1 once, then on the second test run I got the issue: After selecting the player from the list at the start of FS the game would almost freeze and then jump back to reality, guess because of the massive file it's got to read, after that if I tried to continue the campaign, or play any mission from the list of available missions it would crash. That's all I think, hope it helps. Rodo. |
|
|
|
|
|
Thanks for the info! Just on the pilot files and deleting them from disk, FSO does delete the pilots from disk when you delete from the interface, however FSO also reconverts all your old-format pilots everytime you re-enter FSO. We need to do something else about that... perhaps rename the old pilots after they've been converted. |
|
|
|
|
|
Well this is annoying. The previously added assertions are not triggering when the problem occurs. Therefore maybe we're reading in dud data from a misaligned .csg. An assertion on "list_size" when reading the .csg will probably trigger after the issue has occurred. An assertion on "list_size" in pilotfile::csg_write_redalert() is guaranteed to corrupt the .csg when it triggers. Then again, the pilot is already being corrupted by this bug... ideally I'd like to limit the number of written entries to something like MAX_SHIPS, but also warn the player about it, and I don't think a proper warning can be generated in non-debug. Unless I setup some sort of delayed assertion to trigger after the .csg has finished being written... edit: I'm a dope - add an assertion before you start writing any of the .csg :) edit2: committed the assertion in r9497. |
|
|
I have an idea about the cause. When reading the red alert data from the pilot file, the existing in memory red alert data is not cleared. Therefore I think that we're getting the size of the red alert data section doubled each time the pilot file is read & written after any red alert mission is played. I haven't validated this, but this idea requires that the pilot file is both read & written between missions. Also, I managed to reproduce the start of the issue by doing the following: 1) play BP:AoA journey of 1000 miles & complete it 2) check pilot file while game is in red alert briefing - 4x entries in red alert 3) play start of next red alert mission & exit mission & FSO 4) check pilot file - 8x entries in red alert, two (what look like) duplicate sets of 4x ships I'm going to do some more testing & investigation of this idea next weekend (after my holidays) |
|
|
I have uploaded a campaign that should trigger this bug reliably after the first mission. Hope it helps! |
|
|
|
|
|
mantis2503.patch (487 bytes)
diff --git a/code/pilotfile/csg.cpp b/code/pilotfile/csg.cpp index 01ad61f..d368584 100644 --- a/code/pilotfile/csg.cpp +++ b/code/pilotfile/csg.cpp @@ -769,7 +769,9 @@ void pilotfile::csg_read_redalert() return; } - Red_alert_wingman_status.reserve( list_size ); + // about to read new redalert data so flush any existing data + // otherwise wingman entries will multiply like rabbits + Red_alert_wingman_status.clear(); cfread_string_len(t_string, MAX_FILENAME_LEN, cfp); |
|
|
Yarn: thanks for the test campaign, that was very useful :) I have a solution - but I can't commit it until SVN is available again. In the meantime, any comments on the patch are welcome. |
|
|
Fix committed to antipodes@9507. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2011-09-18 04:38 | niffiwan | New Issue | |
| 2011-09-18 04:38 | niffiwan | File Added: WOD Pilot Files.7z | |
| 2011-09-18 04:43 | niffiwan | File Added: niffi-ant8-pilot.7z | |
| 2011-09-18 04:44 | niffiwan | File Added: mantis2503-1.patch | |
| 2011-09-18 04:52 | niffiwan | Note Added: 0012833 | |
| 2011-09-18 05:39 | Echelon9 | Note Added: 0012834 | |
| 2011-09-20 09:46 | niffiwan | File Added: MjnPlayer.7z | |
| 2011-09-20 15:08 | MjnMixael | Note Added: 0012837 | |
| 2011-09-21 10:13 | niffiwan | Note Added: 0012838 | |
| 2011-09-21 12:34 | MjnMixael | Note Added: 0012840 | |
| 2011-09-21 13:55 | MjnMixael | Note Edited: 0012840 | |
| 2012-11-25 08:22 | niffiwan | Target Version | => 3.7.0 |
| 2013-01-04 03:33 | niffiwan | Note Added: 0014612 | |
| 2013-01-04 06:36 | niffiwan | Assigned To | => niffiwan |
| 2013-01-04 06:36 | niffiwan | Status | new => assigned |
| 2013-01-08 02:23 | niffiwan | Note Edited: 0014612 | |
| 2013-01-08 02:25 | niffiwan | Note Edited: 0014612 | |
| 2013-01-08 03:12 | niffiwan | File Added: players.7z | |
| 2013-01-08 03:13 | niffiwan | Note Edited: 0014612 | |
| 2013-01-09 00:34 | rodo | Note Added: 0014621 | |
| 2013-01-09 00:36 | rodo | File Added: players-rodo.7z | |
| 2013-01-09 00:46 | niffiwan | Note Added: 0014622 | |
| 2013-01-10 08:58 | niffiwan | File Added: crybertrance_players_bp_bp2.7z | |
| 2013-01-10 10:10 | niffiwan | Note Added: 0014624 | |
| 2013-01-10 20:54 | niffiwan | Note Edited: 0014624 | |
| 2013-01-12 09:16 | niffiwan | Note Edited: 0014624 | |
| 2013-01-13 22:16 | niffiwan | Note Added: 0014649 | |
| 2013-01-13 22:51 | niffiwan | Note Edited: 0014649 | |
| 2013-01-19 00:31 | Yarn | Note Added: 0014652 | |
| 2013-01-19 00:31 | Yarn | File Added: RedAlertTest.zip | |
| 2013-01-19 02:53 | niffiwan | File Added: mantis2503.patch | |
| 2013-01-19 02:55 | niffiwan | Note Added: 0014653 | |
| 2013-01-19 02:55 | niffiwan | Status | assigned => code review |
| 2013-01-20 09:26 | niffiwan | Changeset attached | => fs2open antipodes r9507 |
| 2013-01-20 09:26 | niffiwan | Note Added: 0014657 | |
| 2013-01-20 09:26 | niffiwan | Status | code review => resolved |
| 2013-01-20 09:26 | niffiwan | Resolution | open => fixed |
