Depili

At least some summoned units are also leaking into the saved files. My save also contains about 86 Xaurip spears, even when I don't have any in my stash.

Man you a great guy, thanks a lot! But not every one nowing coding. Please tell what exactly i need to do? I downloaded script ruby "savefile_cleaner.rb" i installed ruby for windows, i found file mobileobject.save and now what? I dont now how to make it work (((( http://editrocket.com/articles/ruby_windows.html should help You need to give the script the file to clean as a parameter, so it should be something like "C:\Ruby\bin\ruby.exe savefile_cleaner.rb MobileObjects.save" and it should produce a file "MobileObjects.save.cleaned that you need to replace the original file inside your savegame. I haven't tested anything with the windows ruby, but in theory everything should work also on windows...

Actually, dug around the resources a bit and seems that the summoned creatures all should match "_Summon.prefab" string, so I modified the improved cleaner ( https://gist.github.com/depili/9f1591b18923e4639204 ) and now it should get rid of all figurine summon residue also, but no promises as always.

Oh, just saw your reply after doing additional investigation. Could probably get rid of the summons with similar script, just need to alter the filter (or just plop both in at the same time, will look at it later) I'd appreciate it a lot. If it's any help. It appears that all summons stay in that file. Offcourse, in my case that skeleton summon is by far the most used, since npc Chanter starts with it, and it is a useful spell, but I also noticed residue from summoning figurine. A improved script is at https://gist.github.com/depili/9f1591b18923e4639204 It's much faster and should clean up all chanter summons also. I have done minimal testing with it (cleaned one save, tested that it seems to load properly) so use at your own risk The chanter summons are matched with string "_chanter_summon", so it should also detect possible other chanter summon residue. For figurine summon residue I would need something to match them with.

Oh, just saw your reply after doing additional investigation. Could probably get rid of the summons with similar script, just need to alter the filter (or just plop both in at the same time, will look at it later)

https://gist.github.com/depili/d2ddadebff00f3f8c4df This link includes the unique strings from the MobileObjects.save after cleaning the traps out. There are still some things that seem suspicious, like: 3 %cre_skeleton_chanter_summon(clone)_26 3 %cre_skeleton_chanter_summon(clone)_20 3 %cre_skeleton_chanter_summon(clone)_14 I haven't used the summon skeletons ability more than a few times, so it might be a additional problem, and would explain why some people are experiencing problems with summoned units. As they disappear (or are at least supposed to) after combat this is another thing that shouldn't get saved... Weird thing is that unlike other cloned objects they use the string (clone) instead of (Clone).

You should also make sure you have free space on the disk, I ran into that once which silently prevented new saves from getting created.

You might also try to just move the temp files to another location and see if that helps (%TEMP%/Obsidian) But yeah, tinker with your saves at your own risk.,,

I didn't touch anything other than one .savegame file in %USERPATH%/saved games/ -directory, but I didn't work with the most recent save. I would suggest saving your game in a unusual location first to make identifying the correct file to work with easier, as the location will be visible in the file name after the GUID.

The mystery of long saving times has been now solved at least in my case: http://forums.obsidian.net/topic/73986-chanter-rime-and-frost-traps-get-saved-causing-mobileobjectssave-to-grow-out-of-control/ and the devs are aware of it. That thread includes a trap-scrubbing script, but it requires a unix system and some knowledge to use. And no, I'm not making a windows utility, but there are now enough info for that if someone is interested.

\o/ Wrote a quick and dirty ruby script that is going towards the right direction: https://gist.github.com/depili/20afa3ca0ccabd427d62 but while it successfully strips 40M worth of traps out of the MobileObjects.save I still have something messed up, as while loading the modified save is successful I'm missing much of my equipment and some partymembers. Will do another try with slightly altered splitting if that ends up working. If interested this shows the distribution of the traps in my save: https://gist.githubusercontent.com/depili/7d32a41573ca94f6ef15/raw/24d25d1eb1b9c0a93967d0ebb79c8ecf53d0ef00/gistfile1.txt each dot is a non-trap chunk, each X is a trap. Edit: Actually after minor modification the script seems to now work, made a bad guess at the chunk boundary, even when the real place was quite obvious in the end. Now my modified savegame loads fine with all party members and equipment and saving times are way down. Use at your own risk etc etc (that is a ruby scrip for unix systems, usage: ./trap_cleaner.rb MobileObjects.save You need to extract that file from your save and then repackage the resultin MobileObjects.save.cleaned file into its place ) The whole exercise from finding the bloat to removing it has been quite nice little project for this evening, haven't worked with unknown binary blobs before For others maybe wanting to implement this in their own way this is pretty much what I learned: The "chunks" in the file start four bytes before the string Root, that is the place to split, then you can toss all chunks containing the string "Rime_and_Frost_Trap.prefab" and keep the rest. Since the traps are mingled around with "good" chunks you really can't do this by hand.

Well, game creates a autosave on each zone load....

First experiment was a success, Deleted a around 20Mb chunk from the MobileObjects.save file to see if it causes save/load errors, but everything seems promising, as the game loaded just fine and arena transitions are now way faster! I cut the chunk starting after the terminating null byte of a string previous to the "Root" string starting a trap chunk and ended at a randomly chosen null byte terminating a chunk. So there appears to be no checksum or a global length info for the file anywhere. Next I need to do a cleaner solution by writing some ruby and toss all the traps away, and only the traps. Some additional challenge: There is a chunk with the party data, that includes the chanter and will thus quite easily match the trap-filter, so need to avoid that. Edit: Something unexpected apparently happened also, after loading the edited game, doing a area transition and saving again the MobileObjects.save is now down to 2,5Mb, so something might have triggered some garbage collection as a side-effect. If so I will leave my experimentation at this and just be happy. Edit2: Well, it wasn't such a success, that brute force test apparently also got rid of 4 recruited NPCs, so need to really write that script

Looking at the difference between two MobileObjects.save files from two different saves suggests that there might be some evil lengt/checksum fields in the file, which might make a quick-and-dirty savegame scrubber impossible. Tempted to still have a go at it since I haven't done any tampering of unknown binary blobs in the past Current plan is to just first split the stored objects (Making some assumptions on their boundaries per the start and end of the file) and then toss out all chunks containing the Rime and Frost string, reassemble the file and hope for the best...

Each saved trap seems to contain a serialization of the chanter that cast the spell and array of (probably all) "teams" on in the game, and they take up tons of space. The teams are there probably to set who can trigger the trap (monsters don't trigger traps on the dungeon floor, only the ones you place yourself). One puzzling association are the "player (Clone)(Clone)...." strings, there are few with various amount of the (Clone) tag appended to them, in the worst case the engine is creating new cloned entities of the chanter for the traps, which might also increase the memory usage while playing if so.

Looking into the amount of data tied into the Rime and Frost traps, one trap (everything between two Root strings) is about 26kb, meaning that the 1600ish traps in my 44Mb MobileObjects.save take up about 42Mb of it. (This is all just from looking at one random trap).

I originally counted each trap twice by mistake, see http://forums.obsidian.net/topic/73986-chanter-rime-and-frost-traps-get-saved-causing-mobileobjectssave-to-grow-out-of-control/ for more details. The more detailed analysis has revealed that there are about 1600 frost traps in the save, and another group of about 1700 of very similar objects.

Thanks for looking into this. It might be worth sending a PM to Roby Atadero (the Obsidian Dev. who responded to the other thread) if you don't get a response. I imagine the team is busy and could miss this if it falls too far down the page. Send a PM. Hope the eventual fix also includes scrubbing of existing savegames, as otherwise it won't be of much use for people already suffering from this.

Digging deeper to the strings in MobileObjects.save. Here are all the unique strings sorted by number of occurances in my save: https://gist.github.com/depili/1117c9f30047c4985a14 From this its quite easy to see that there are 1725 objects of the same unknown type, 1614 rime and frost traps, and that Obsidian loves cloning: 437 Player(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone)(Clone) Maybe a bit excessive, don't you think? =) (The string "(Clone)" appears total of 53k times in MobileObjects.save).

I think this deserves it's own post instead of being buried on page 5 of the savegame issue thread. My post there: http://forums.obsidian.net/topic/72764-quick-saving-and-loading-times-increased-greatly-after-20-hours/?p=1615960 So in short the Chanter "Rime and Frost" chant creates traps when it is triggered and thoise traps get saved into mobileobjects.save file causing it to grow out of control. This in turn causes saves to take way longer than they should. Oddly the chant creating fire traps doesn't seem to be affected with this bug. Here are the statistics in a nutshell: $ du -h MobileObjects.save 44M MobileObjects.save $ strings MobileObjects.save | grep Rime_and | wc -l 3232 $ strings MobileObjects.save | grep Root | wc -l 3308 That is 97% of all store objects! Explanation for the commands: First one shows the size of the MobileObjects.save, secong extracts all strings (text fragments) from that binary file, selects all that match the "Rime_And" stringfragment and counts them, giving us the count of 3232 Rime and Frost traps stored, last one does the same for substring Root that seems the be the start of new saved object data, giving the total count of 3308 saved objects. Workaround would be to avoid the rime and frost chant like plague, a fix would be to write a script to strip the saved traps from saves and to prevent new ones from getting saved at all. Edit: Apparently my first grepping miss-counted the amount of Rime and Frost traps, as the string "Rime_and" appears twice per trap, so a better match is: $ strings MobileObjects.save | grep "Rime_and_Frost_Trap.prefab" | wc -l 1614 Which leaves 50% of objects being traps. But the rime and frost trap definitions seem to be some of the longest objects.

Should maybe later take a peek at what is inside the mobile objects file, as it compresses with such good ratio. Either stuff is repeated many many times inside or there are long strings of same bytes. A quick and dirty fix would be to change the compression settings on the savegame zips to "store" where there aren't any compression, so then the cpu bottleneck vanishes, but your savefiles will be 50+Mb Edit: Quick look into the mobileobjects.save shows that most of the file is repeated object headers with keys stored as text and taking up most of the space it needs. The file also seems to store the locations and status of each and every npc encountered, which explains the rapid growth when entering defiance bay with so many npcs around. Oh, and every trap, like the traps created by the Chanter abilities! So that might be the real culprit: Edit2: Yeah, there are 3k Rime and Frost traps on my mobileobjects. It appears the the firetraps don't get saved, so its a bug with Rime and Frost. Next someone just needs to write a script to strip the rime and frost traps from the save... Root ObjectName LevelNamePrefabResourceMobileGlobalGUIDObjectIDLocationxyzRotationPackedLoadManuallyParentComponentPackets TypeString Variables ScriptTagRestoredTeamTagHostileTeamsCapacityNeutralTeams FriendlyTeamsname hideFlagsm_scriptTagDefaultRelationshipGameFactionInjuredReputationChangeMurderedReputationChangeParamPrereqSerializedSourceSourceSerializedm_id m_stackingKeym_slot m_applied m_durationm_durationOverridem_needsDurationCalculatedm_timeActivem_intervalTimerm_intervalCountTargetSerializedm_deleteOnClearm_effect_is_on_main_targetm_generalCounterBundleIdm_scalem_durationScalem_religiousScalem_suspensionCountm_suppressed m_timeAppliedm_numRestCyclesExtraObjectSerializedAbilityTypeAbilityOriginSerializedEquipmentOriginEquipmentOriginSerializedPhraseOriginPhraseOriginSerializedAfflictionOriginAfflictionOriginSerializedAfflictionKeywordm_damageToReapplym_damageToAbsorbm_triggerCountDurationAfterBreakTicksAfterBreakIncreasePerTickFriendlyRadius IsFromAuram_forceStackableOwnerSerializedAppliedSlotParamsOnDamagedCallbackAbilityTrapPrefabSerializedTrapSerialized2EquippablePrefabSerializedEquippableSerialized2AttackPrefabSerializedAttackPrefabSerialized2AfflictionPrefabSerialized!OnDamageCallbackAbilitySerializedDescriptionIsCleanedUpMergedValue IsHostileApplyAffectsStatDmgTypeValue ExtraValueDurationLastsUntilCombatEndsLastsUntilRest MaxRestCyclesIntervalRate OneHitUseChecksReligionVisualEffectAttachRaceTypeKeywordDontHideFromLogIgnoreAbilityDeactivationScaleAppliedFXNames DeleteOnClearEffectIDTriggerTypeTriggerValueTriggerValueAdjustmentTriggerDurationAdjustmentTriggerRadiusAdjustmentTriggerMaxTriggerCountTriggerRemoveEffectAtMax TriggerResetTriggerOnEffectPulseTriggerResetTriggerOnEffectEndm_is_checking_melee_path _ObjectPersistencePacket, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null bComponentPersistencePacket, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullZSystem.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089ZSystem.Object, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089LTeam, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null[System.Boolean, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089YSystem.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089ZSystem.Single, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089XSystem.Guid, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]System.Boolean[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089ÊSystem.Collections.Generic.List`1[[System.Guid, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089NGender, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null[CharacterStats+Race, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null^CharacterStats+Subrace, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null^CharacterStats+Culture, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\CharacterStats+Class, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullVReligion+Deity, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]Religion+PaladinOrder, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullaCharacterStats+Background, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullÄSystem.Collections.Generic.List`1[[Affliction, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089RAffliction, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullÇSystem.Collections.Generic.List`1[[GenericTalent, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089UGenericTalent, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\System.Single[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089ZSystem.UInt32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089eCharacterStats+NoiseLevelType, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null[System.Int32[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089ÆSystem.Collections.Generic.List`1[[StatusEffect, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089TStatusEffect, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullXPrerequisiteData, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=nullÌSystem.Collections.Generic.List`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Rime_and_Frost_Trap(Clone) $ strings MobileObjects.save | grep Rime_and | wc -l 3232 $ strings MobileObjects.save | grep Root | wc -l 3308 That is 97% of all stored objects being rime and frost traps!

I'm also experiencing this, at the end of act2 the load times on area transitions are starting to get unbearable. But I don't even have that mobileobjects.save file with my savegames. The save files are around 5Mb a pop. The archive contains total of 42Mb of unpacked data according to 7zip, but that data takes up only around half of the savegame size when packed, so there might be something else in the save archive also. Currently when saving the game goes unresponsive for a while and any keypresses during that time will be amplified, ie. tapping a arrow key will scroll the whole map to one edge once the saving has been completed and game is responsive again.

I have also run into this problem with the amulet.