View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0002971 | FSSCP | multiplayer | public | 2013-12-03 18:25 | 2015-05-01 06:33 |
| Reporter | chief1983 | Assigned To | MageKing17 | ||
| Priority | normal | Severity | crash | Reproducibility | sometimes |
| Status | resolved | Resolution | fixed | ||
| Platform | x86 | OS | Gentoo Linux | OS Version | 3.5.7 |
| Product Version | 3.7.1 | ||||
| Target Version | 3.7.4 | ||||
| Summary | 0002971: Standalone AddressSanitizer crash SEGV in hud_shield_quadrant_hit | ||||
| Description | ================================================================= ==18747== ERROR: AddressSanitizer: SEGV on unknown address 0x0000000c (pc 0x082fc5c5 sp 0xbfea81f0 bp 0xbfea8228 T0) AddressSanitizer can not provide additional info. #0 0x82fc5c4 in hud_shield_quadrant_hit(object*, int) /home/chief1983/fs2_open_build/code/hud/hudshield.cpp:573 (discriminator 1) 0000001 0x8a2e50b in ship_weapon_do_hit_stuff(object*, object*, vec3d*, vec3d*, int, int, vec3d) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:97 0000002 0x8a30565 in ship_weapon_check_collision(object*, object*, float, int*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:403 0000003 0x8a30eab in collide_ship_weapon(obj_pair*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:479 0000004 0x86bf397 in obj_collide_pair(object*, object*) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1645 0000005 0x86bca02 in obj_find_overlap_colliders(SCP_vector<int>*, SCP_vector<int>*, int, bool) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1265 0000006 0x86bc62f in obj_sort_and_collide() /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1230 0000007 0x86ceccf in obj_move_all(float) /home/chief1983/fs2_open_build/code/object/object.cpp:1502 0000008 0x805f16b in game_simulation_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4006 (discriminator 2) 0000009 0x80603e2 in game_frame(bool) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4399 0000010 0x806124c in game_do_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4814 #11 0x8063f5b in game_do_state(int) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:6494 0000012 0x81c54ee in gameseq_process_events() /home/chief1983/fs2_open_build/code/gamesequence/gamesequence.cpp:409 0000013 0x8065468 in game_main(char*) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7061 0000014 0x806589d in main /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7195 0000015 0xb5c03595 in ?? ??:0 ==18747== ABORTING | ||||
| Steps To Reproduce | How I have reproduced it: Connect two players to an AddressSanitizer-enabled Standalone Fire up Demon Dogfight Light Shoot around at the demon or the other player for a while. Took longer to create than previous ASan crashes but it happened to me within a minute still. | ||||
| Additional Information | Second time I reproduced it was the one that seemed to take longer, and had a different code path: #0 0x082fc660 in hud_shield_quadrant_hit (objp=0x940c8bc <Objects+508>, quadrant=-1) at hud/hudshield.cpp:575 0000001 0x086b45c3 in collide_ship_ship (pair=0xbfffe6d0) at object/collideshipship.cpp:1242 0000002 0x086bf398 in obj_collide_pair (A=0x940c8bc <Objects+508>, B=0x940c6c0 <Objects>) at object/objcollide.cpp:1645 0000003 0x086bca03 in obj_find_overlap_colliders (overlap_list_out=0xbfffe820, list=0xbfffe860, axis=2, collide=true) at object/objcollide.cpp:1265 0000004 0x086bc630 in obj_sort_and_collide () at object/objcollide.cpp:1230 0000005 0x086cecd0 in obj_move_all (frametime=0.0330047607) at object/object.cpp:1502 0000006 0x0805f16c in game_simulation_frame () at freespace2/freespace.cpp:4006 0000007 0x080603e3 in game_frame (paused=false) at freespace2/freespace.cpp:4399 0000008 0x0806124d in game_do_frame () at freespace2/freespace.cpp:4814 0000009 0x08063f5c in game_do_state (state=2) at freespace2/freespace.cpp:6494 0000010 0x081c54ef in gameseq_process_events () at gamesequence/gamesequence.cpp:409 #11 0x08065469 in game_main (cmdline=0xb6100610 "-standalone") at freespace2/freespace.cpp:7061 0000012 0x0806589e in main (argc=2, argv=0xbfffedb4) at freespace2/freespace.cpp:7195 Additional stack info from that crash: (frame 0) (gdb) info locals shi = 0x9181a48 <Shield_hit_data+40> num = 141278518 (gdb) frame 1 0000001 0x086b45c3 in collide_ship_ship (pair=0xbfffe6d0) at object/collideshipship.cpp:1242 (gdb) info locals dam2 = 3.17000103 quadrant_num = -1 damage = 6.8274188 a_override = false b_override = false hit = 1 LightOne = 0x940c8bc <Objects+508> light_sip = 0x99f02dc <Ship_info+109500> ship_ship_hit_info = {heavy = 0x940c6c0 <Objects>, light = 0x940c8bc <Objects+508>, heavy_collision_cm_pos = {{xyz = {x = 0, y = 0, z = 0}, a1d = {0, 0, 0}}}, light_collision_cm_pos = { {xyz = {x = -24.7636738, y = 252.959991, z = -178.660339}, a1d = {-24.7636738, 252.959991, -178.660339}}}, r_heavy = {{xyz = {x = -22.50424, y = 247.505051, z = -178.703674}, a1d = {-22 .50424, 247.505051, -178.703674}}}, r_light = {{xyz = {x = 2.25943375, y = -5.4549408, z = -0.0433349609}, a1d = {2.25943375, -5.4549408, -0.0433349609}}}, hit_pos = {{xyz = {x = -22.50 424, y = 247.505051, z = -178.703674}, a1d = {-22.50424, 247.505051, -178.703674}}}, collision_normal = {{xyz = {x = -0.382660955, y = 0.923859656, z = 0.00733927637}, a1d = {-0.3826609 55, 0.923859656, 0.00733927637}}}, hit_time = 0.39586851, impulse = 7192.90283, light_rel_vel = {{xyz = {x = -44.084198, y = -51.2995834, z = 21.8710117}, a1d = {-44.084198, -51.2995834 , 21.8710117}}}, collide_rotate = 1, submodel_num = 0, edge_hit = 0, submodel_rot_hit = 0, is_landing = false} world_hit_pos = {{xyz = {x = -37.1646347, y = 247.511673, z = 955.386658}, a1d = {-37.1646347, 247.511673, 955.386658}}} HeavyOne = 0x940c6c0 <Objects> player_involved = 0 dist = 309.667145 A = 0x940c8bc <Objects+508> B = 0x940c6c0 <Objects> (gdb) frame 2 0000002 0x086bf398 in obj_collide_pair (A=0x940c8bc <Objects+508>, B=0x940c6c0 <Objects>) at object/objcollide.cpp:1645 (gdb) info locals ctype = 257 check_collision = 0x86b2eb9 <collide_ship_ship(obj_pair*)> key = 4096 swapped = 0 collision_info = 0xa850f4f4 valid = true new_pair = {a = 0x940c8bc <Objects+508>, b = 0x940c6c0 <Objects>, check_collision = 0x86b2eb9 <collide_ship_ship(obj_pair*)>, next_check_time = 1, next = 0xbfffe790} | ||||
| Tags | No tags attached. | ||||
|
|
zookeeper indicated that Shield_hit_data must not be init'd right but can't figure out why it wouldn't be at that point on the standalone. |
|
|
================================================================= ==6463== ERROR: AddressSanitizer: SEGV on unknown address 0x00000008 (pc 0x082fc5cd sp 0xbfc69950 bp 0xbfc69988 T0) AddressSanitizer can not provide additional info. #0 0x82fc5cc in hud_shield_quadrant_hit(object*, int) /home/chief1983/fs2_open_build/code/hud/hudshield.cpp:573 (discriminator 1) 0000001 0x8a2f1fb in ship_weapon_do_hit_stuff(object*, object*, vec3d*, vec3d*, int, int, vec3d) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:97 0000002 0x8a31255 in ship_weapon_check_collision(object*, object*, float, int*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:403 0000003 0x8a31b9b in collide_ship_weapon(obj_pair*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:479 0000004 0x86bfc87 in obj_collide_pair(object*, object*) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1645 0000005 0x86bd2f2 in obj_find_overlap_colliders(SCP_vector<int>*, SCP_vector<int>*, int, bool) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1265 0000006 0x86bcf1f in obj_sort_and_collide() /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1230 0000007 0x86cf5bf in obj_move_all(float) /home/chief1983/fs2_open_build/code/object/object.cpp:1502 0000008 0x805f170 in game_simulation_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4007 (discriminator 2) 0000009 0x80603e7 in game_frame(bool) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4400 0000010 0x8061251 in game_do_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4815 #11 0x8063f60 in game_do_state(int) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:6495 0000012 0x81c54f6 in gameseq_process_events() /home/chief1983/fs2_open_build/code/gamesequence/gamesequence.cpp:409 0000013 0x806546d in game_main(char*) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7062 0000014 0x80658a2 in main /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7196 0000015 0xb5c0f595 in ?? ??:0 ==6463== ABORTING |
|
|
assert-mantis-2971.patch (559 bytes)
Index: code/hud/hudshield.cpp
===================================================================
--- code/hud/hudshield.cpp (revision 10198)
+++ code/hud/hudshield.cpp (working copy)
@@ -572,6 +572,7 @@
shi->shield_hit_timers[num] = timestamp(SHIELD_HIT_DURATION_SHORT);
} else {
+ Assertion(shi->hull_hit_index < shi->members, "shi->hull_hit_index %d is too high (max %d)", shi->hull_hit_index, shi->members);
shi->shield_hit_timers[shi->hull_hit_index] = timestamp(SHIELD_HIT_DURATION_SHORT);
hud_targetbox_start_flash(TBOX_FLASH_HULL);
}
|
|
|
Have builds with the assert patch (attached) reported this issue yet? |
|
|
I hadn't noticed the patch I guess, zookeeper had been planning on taking a look at this one but he's been engrossed in other projects, so maybe I'll take another look at this one here soon. |
|
|
Ok, didn't hit the Assert with a patched standalone, still got this SEGV on the otherwise current trunk revision. ASAN:SIGSEGV ================================================================= ==10391== ERROR: AddressSanitizer: SEGV on unknown address 0x00000000 (pc 0x08328859 sp 0xbf923f90 bp 0xbf923fd8 T0) AddressSanitizer can not provide additional info. #0 0x8328858 in hud_shield_quadrant_hit(object*, int) /home/chief1983/fs2_open_build/code/hud/hudshield.cpp:573 (discriminator 1) 0000001 0x8a60729 in ship_weapon_do_hit_stuff(object*, object*, vec3d*, vec3d*, int, int, vec3d) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:97 0000002 0x8a6277d in ship_weapon_check_collision(object*, object*, float, int*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:403 0000003 0x8a630b7 in collide_ship_weapon(obj_pair*) /home/chief1983/fs2_open_build/code/object/collideshipweapon.cpp:479 0000004 0x86f06a1 in obj_collide_pair(object*, object*) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1645 0000005 0x86edd3c in obj_find_overlap_colliders(SCP_vector<int>*, SCP_vector<int>*, int, bool) /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1265 0000006 0x86ed969 in obj_sort_and_collide() /home/chief1983/fs2_open_build/code/object/objcollide.cpp:1230 0000007 0x86ff104 in obj_move_all(float) /home/chief1983/fs2_open_build/code/object/object.cpp:1502 0000008 0x805f14f in game_simulation_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4007 (discriminator 2) 0000009 0x80603c6 in game_frame(bool) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4400 0000010 0x806123a in game_do_frame() /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:4816 #11 0x8063f49 in game_do_state(int) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:6496 0000012 0x81c709a in gameseq_process_events() /home/chief1983/fs2_open_build/code/gamesequence/gamesequence.cpp:409 0000013 0x8065456 in game_main(char*) /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7063 0000014 0x806588b in main /home/chief1983/fs2_open_build/code/freespace2/freespace.cpp:7197 0000015 0xb5c85595 in ?? ??:0 ==10391== ABORTING |
|
|
Anything else that we can try to sort this out for 3.7.2? |
|
|
I'll give this another go with an AddressSanitizer instrumented standalone server and client. |
|
|
Any input from that? Need any help testing with an ASan server? I can probably get mine running again. |
|
|
Based on the stack dump, and narrowing down the code paths based on variable values, the culprit appears to be this line: shi->shield_hit_timers[shi->hull_hit_index] = timestamp(SHIELD_HIT_DURATION_SHORT); Now, on standalone servers Player_obj is specifically set to NULL (per multiteamselect.cpp:937). I think what is happening is that shield_info_reset is clearing the vectors but leaving the hull_hit_index set to 0. Trying to access position 0 in the vector causes an access violation. (The wrinkle in that hypothesis is that Player_obj being NULL should only affect the player's shield data, yet the code dump indicated the crash occurred on the target data. But I may not be following the initialization 100% correctly.) I would recommend littering the hud_shield_quadrant_hit function with logging statements to try to narrow this down. However, the shield_info_reset function needs fixing in any case, and based on my understanding of the situation it is the most likely culprit. I am attaching a patch. |
|
|
hudshield.cpp-Mantis-2971.patch (1,321 bytes)
Index: code/hud/hudshield.cpp
===================================================================
--- code/hud/hudshield.cpp (revision 11079)
+++ code/hud/hudshield.cpp (working copy)
@@ -473,25 +473,20 @@
// associated ship
void shield_info_reset(object *objp, shield_hit_info *shi)
{
- shi->shield_hit_status = 0;
- shi->shield_show_bright = 0;
+ int n_quadrants = (objp != NULL) ? objp->n_quadrants : 0;
- if (objp == NULL) {
- shi->members = 0;
- shi->hull_hit_index = 0;
- shi->shield_hit_timers.clear();
- shi->shield_hit_next_flash.clear();
- } else {
- shi->members = objp->n_quadrants + 1;
- shi->hull_hit_index = shi->members - 1;
- shi->shield_hit_timers.resize(shi->members);
- shi->shield_hit_next_flash.resize(shi->members);
+ shi->members = n_quadrants + 1;
+ shi->hull_hit_index = n_quadrants;
+ shi->shield_hit_timers.resize(shi->members);
+ shi->shield_hit_next_flash.resize(shi->members);
- for ( int i = 0; i < shi->members; i++ ) {
- shi->shield_hit_timers[i] = 1;
- shi->shield_hit_next_flash[i] = 1;
- }
+ for ( int i = 0; i < shi->members; i++ ) {
+ shi->shield_hit_timers[i] = timestamp(0);
+ shi->shield_hit_next_flash[i] = timestamp(0);
}
+
+ shi->shield_hit_status = 0;
+ shi->shield_show_bright = 0;
}
// reset the timers and hit flags for the shield gauges
|
|
|
Program received signal SIGSEGV, Segmentation fault. 0x0835aaa1 in hud_shield_quadrant_hit (objp=0x940a73c <Objects+508>, quadrant=1) at hud/hudshield.cpp:568 (gdb) bt #0 0x0835aaa1 in hud_shield_quadrant_hit (objp=0x940a73c <Objects+508>, quadrant=1) at hud/hudshield.cpp:568 0000001 0x08a99498 in ship_weapon_do_hit_stuff (ship_obj=0x940a73c <Objects+508>, weapon_obj=0x940c304 <Objects+7620>, world_hitpos= 0xbfffe3e8, hitpos=0xbfffe3dc, quadrant_num=1, submodel_num=-1, hit_dir=...) at object/collideshipweapon.cpp:97 0000002 0x08a9b4e7 in ship_weapon_check_collision (ship_objp=0x940a73c <Objects+508>, weapon_objp=0x940c304 <Objects+7620>, time_lim it=0, next_hit=0x0) at object/collideshipweapon.cpp:410 0000003 0x08a9be47 in collide_ship_weapon (pair=0xbfffe6c0) at object/collideshipweapon.cpp:486 0000004 0x0872fa17 in obj_collide_pair (A=0x940a73c <Objects+508>, B=0x940c304 <Objects+7620>) at object/objcollide.cpp:1645 0000005 0x0872d0a4 in obj_find_overlap_colliders (overlap_list_out=0xbfffe810, list=0xbfffe850, axis=2, collide=true) at object/objc ollide.cpp:1265 0000006 0x0872ccd3 in obj_sort_and_collide () at object/objcollide.cpp:1230 0000007 0x0873e7f3 in obj_move_all (frametime=0.0330047607) at object/object.cpp:1507 0000008 0x08060b2a in game_simulation_frame () at freespace2/freespace.cpp:4085 0000009 0x08061da1 in game_frame (paused=false) at freespace2/freespace.cpp:4478 0000010 0x08062bf0 in game_do_frame () at freespace2/freespace.cpp:4894 #11 0x080659bb in game_do_state (state=2) at freespace2/freespace.cpp:6577 0000012 0x081e93fb in gameseq_process_events () at gamesequence/gamesequence.cpp:409 0000013 0x08066f4f in game_main (cmdline=0xb6100610 "-standalone") at freespace2/freespace.cpp:7144 0000014 0x0806739c in main (argc=2, argv=0xbfffedb4) at freespace2/freespace.cpp:7278 Was I supposed to run the patch on the standalone only, or also all the clients? For this run I just had it on the standalone. |
|
|
Good question... I would have thought standalone would be sufficient. Try it on all clients. If that doesn't work, then we'll have to go the logging route. |
|
|
I committed the patch because that function needs fixing anyway, but it sounds like it doesn't fix the whole problem. (Though, make sure that you didn't accidentally test with an unpatched build.) |
|
|
I'll set up a dedicated with 3.7.2 final and some 3.7.2 clients after I compile them and give them all a go. |
|
|
Crash still happens, AddressSanitizer caught a seg fault. This one isn't much use, I don't have symbols for some reason. |
|
|
Yes, AddressSanitizer is still appearing to correctly flag a memory corruption issue here in multi |
|
|
hudshield.cpp.patch (424 bytes)
diff --git a/code/hud/hudshield.cpp b/code/hud/hudshield.cpp
index 74bb250..b4b751a 100644
--- a/code/hud/hudshield.cpp
+++ b/code/hud/hudshield.cpp
@@ -126,6 +126,7 @@ void hud_shield_level_init()
hud_frames temp;
hud_shield_hit_reset(Player_obj, 1); // reset for the player
+ hud_shield_hit_reset(NULL); // reset for target
if ( !Hud_shield_inited ) {
for ( i = 0; i < Hud_shield_filenames.size(); i++ ) {
|
|
|
So I think I've got this one figured out (thanks to a lot of help from chief1983 and his server). When model point shields were added (r10135), shi->shield_hit_timers was changed from an array to a vector. The errors are due to trying to access the uninitialized vector. The reason r11099 didn't affect the crash is because the standalone would never call hud_shield_hit_reset() except for the player object (which would never be the object called for hud_shield_quadrant_hit() due to being NULL). I think the correct solution here, rather than making hud_shield_quadrant_hit() work on the standalone, is to simply not call it on the standalone; it's HUD-related code that probably shouldn't have been called in the first place, it simply used to work due to the fixed-size array being used. (That being said, some assertions to avoid similar errors in the future are undoubtedly a good idea.) After a little more testing to make sure I've got it sorted out, I'll make a pull request for the fix. |
|
|
Ah, excellent sleuthing! |
|
|
Pull request: https://github.com/scp-fs2open/fs2open.github.com/pull/47 |
|
|
And merged. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2013-12-03 18:25 | chief1983 | New Issue | |
| 2013-12-03 18:29 | chief1983 | Note Added: 0015500 | |
| 2013-12-05 07:46 | chief1983 | Note Added: 0015507 | |
| 2013-12-05 07:59 | chief1983 | Reproducibility | always => sometimes |
| 2013-12-05 08:12 | Echelon9 | File Added: assert-mantis-2971.patch | |
| 2013-12-26 07:07 | Echelon9 | Note Added: 0015532 | |
| 2013-12-26 15:02 | chief1983 | Note Added: 0015535 | |
| 2014-01-01 00:44 | chief1983 | Note Added: 0015552 | |
| 2014-01-01 00:45 | chief1983 | Note Edited: 0015552 | |
| 2014-04-21 06:55 | Echelon9 | Assigned To | => Echelon9 |
| 2014-04-21 06:55 | Echelon9 | Status | new => assigned |
| 2014-05-13 20:59 | chief1983 | Note Added: 0015745 | |
| 2014-08-20 13:36 | Echelon9 | Note Added: 0016244 | |
| 2014-08-30 12:59 | chief1983 | Note Added: 0016266 | |
| 2014-08-31 15:34 | MageKing17 | Assigned To | Echelon9 => MageKing17 |
| 2014-08-31 15:35 | MageKing17 | Assigned To | MageKing17 => Echelon9 |
| 2014-09-24 06:32 | Goober5000 | Note Added: 0016291 | |
| 2014-09-24 06:32 | Goober5000 | Assigned To | Echelon9 => Goober5000 |
| 2014-09-24 06:34 | Goober5000 | File Added: hudshield.cpp-Mantis-2971.patch | |
| 2014-09-24 06:34 | Goober5000 | Status | assigned => code review |
| 2014-09-28 15:50 | chief1983 | Note Added: 0016312 | |
| 2014-09-28 17:32 | Goober5000 | Note Added: 0016313 | |
| 2014-09-30 03:55 | Goober5000 | Changeset attached | => fs2open trunk r11099 |
| 2014-09-30 03:56 | Goober5000 | Note Added: 0016318 | |
| 2015-04-16 04:25 | Goober5000 | Target Version | 3.7.2 => 3.7.4 |
| 2015-04-23 17:11 | chief1983 | Note Added: 0016656 | |
| 2015-04-25 20:10 | chief1983 | Note Added: 0016671 | |
| 2015-04-27 03:49 | Echelon9 | Note Added: 0016680 | |
| 2015-04-30 01:29 | MageKing17 | File Added: hudshield.cpp.patch | |
| 2015-04-30 17:44 | MageKing17 | Note Added: 0016685 | |
| 2015-04-30 17:44 | MageKing17 | Assigned To | Goober5000 => MageKing17 |
| 2015-04-30 17:44 | MageKing17 | Status | code review => assigned |
| 2015-05-01 02:56 | Goober5000 | Note Added: 0016687 | |
| 2015-05-01 05:22 | MageKing17 | Note Added: 0016688 | |
| 2015-05-01 05:22 | MageKing17 | Status | assigned => code review |
| 2015-05-01 06:33 | MageKing17 | Note Added: 0016689 | |
| 2015-05-01 06:33 | MageKing17 | Status | code review => resolved |
| 2015-05-01 06:33 | MageKing17 | Resolution | open => fixed |
