[Solved] Profligacy Desync

[Swan]

So I'm updating the old Profligacy ORPG but I seem to have introduced a desync bug somehow. It occurs as soon as the map starts if any players have played something other than Profligacy without restarting Warcraft. I've also had reports of delayed desyncs in other maps after having played Profligacy without restarting. The fact that it can be resolved by restarting Warcraft before and after playing points me towards map data being damaged somehow but I don't know what it is. Old versions of Profligacy did not desync (I don't have access to their source unfortunately). I had thought it was something to do with the way I optimized/protected the map but it happens in this unprotected version as well.
Edit: I forgot to mention that it seems to occur with any other map played before but it is confirmed to happen with Dota or Twilight's Eve ORPG.

 

[Swan]

Does anyone have suggestions or ideas about what might be a good path to pursue in fixing this?

 

[pOke]

What causes desync problem?

Take a look at this post potentially, although it seems the problem would be the other way around.

Also if you use GetLocalPlayer() anywhere consider looking over those triggers or even posting them here. It might be an edge case that occurs only occasionally and the fact that some players have said they were playing X,Y and Z before might just be a coincidence.

Edit: Here is another post, just trying to give you some information that might make you think of where the problem is originating.
[Trigger] - Desync Detect : Somebody Help Me

Edit2: I have no idea if it's the case, but I just read that "Large Dice Rolls" can cause issues.
Edit3: Are you using vanilla WE? JNGP? Just saw something about the old WEU being behind some problems.

 

[Swan]

My worldeditor/jassnewgen is from here. all the getlocal players are for displaying text or saving and haven't changed. Besides it is consistently reproducible by playing another map before Profligacy to cause it or restarting beforehand to never have it. Large dice rolls may have something to do with it, though I haven't touched any of that and older versions of Profligacy did not have the bug. I'm leaning towards the world editor being the source of the bug. What version is recommended?
The fact that it only occurs when the map starts, never later into it, and that it only occurs when other games have been played before hand makes it seem like something weird is going on with warcraft memory to me.

 

[Emm-A-]

does it always happen, no matter what map has been played before? Does it happen, if a normal melee map has been played before?

 

[Swan]

I'll test that.

 

[Swan]

It did occur with melee.

Edit:
Would someone else opening and saving it with their own Jassnewgen pack fix something that mine has bugged if it is my Jassnewgen that is causing this?

 

[pOke]

Well I am at a loss and don't really have much more to offer.
Maybe @Dr Super Good can help as both of the posts I linked are from him.

 

[Swan]

Well thanks a ton for trying. I'll update this thread if I get it fixed but I'd still love help if anyone has any ideas.

Edit: Might have found it, following the advice in this thread there were tons of black squares.
Chronic Desync Problem | The Helper

 

[Swan]

Fixed it, I just had to delete one of the tiles so that I was only using 15 tiles instead of 16 tiles. I don't know how previous versions of Profligacy didn't have this desync but oh well.
Fixed it hopefully. I actually had another desync after this fix though it may have been due to some other map.

 

[Swan]

Still having the desyncs. I had thought it was fixed but it seems to have just been a fluke. Another weird thing that happened was one player desynced while the rest of us received a quest when clicking on a unit. Normally clicking a quest giver automatically gives you the quest, but the wrong unit was providing the quest. I've never seen this bug occur and it hasn't since.

 

[pOke]

Can you show the quest trigger?

 

[Swan]

Sure I'll just upload the map again. Theres a whole quest system not written by me. My personal guess would rather be that its something to do with unit-ids getting screwed up by whatever is causing the desyncs but I dunno. I'll probably upload tomorrow.

 

[Emm-A-]

that quest thing points towards GetLocalPlayer(). You should check all local blocks in this quest system

 

[Swan]

function PlaySoundForPlayer takes sound snd, player p returns nothing
    if(GetLocalPlayer() == p) then
        call StartSound(snd)
    endif
endfunction

function AddSpecialEffectTargetPlayer takes string sfx, unit target, string attach, player p returns effect
    local string s = sfx
    if GetLocalPlayer() != p then
        set s = ""
    endif
    return AddSpecialEffectTarget(s,target,attach)
endfunction

private function UpdateQuestDialog takes nothing returns nothing
    local real trans = 100
    local integer i = 0
    loop
        exitwhen i >= MAXPLAYERS
        if Player(i) == GetLocalPlayer() then
            if dialogDisplay[i] then
                set trans = 0
            endif
        endif
        set i = i + 1
    endloop
    call CinematicFilterGenericBJ(0,BLEND_MODE_BLEND,"QuestBox.blp",0,0,0,100,100,100,100,trans)
endfunction

function UpdateQuestInfo takes Quest q returns nothing
    local integer i = 0
    call QuestSetEnabled(q.questInfo,false)
    loop
        exitwhen i >= MAXPLAYERS
        if Player(i) == GetLocalPlayer() then
            if q.stage[i] == QUEST_STAGE_ACTIVE or q.stage[i] == QUEST_STAGE_COMPLETE  then
                call QuestSetEnabled(q.questInfo,true)
                call QuestSetDescription(q.questInfo,Objectives2String.evaluate(q,i))
            endif
        endif
        set i = i + 1
    endloop
endfunction

private function ShowChestForPlayer takes unit chest, player p returns nothing
    if GetLocalPlayer() != p then
        call SetUnitScale(chest,0,0,0)
    endif
endfunction

function DropChestItem takes integer itemId, string name, real x, real y, rect r returns nothing
    local ItemType iT = GetItemTypeStruct(itemId)
    local ChestData c
    local unit hero
    local unit chest
    local integer i = 0
    local integer count = 0
    call GroupEnumUnitsInRange(G,x,y,LOOTRANGE,Condition(function CreditFilter))
    if r != null then
        call GroupEnumUnitsInRect(G2,r,Condition(function CreditFilter))
        loop
            set hero = FirstOfGroup(G2)
            exitwhen hero == null
            call GroupAddUnit(G,hero)
            call GroupRemoveUnit(G2,hero)
        endloop
    endif
    set count = CountUnitsInGroup(G)
    if count > 0 then
        set c = ChestData.create()
        loop
            set hero = FirstOfGroup(G)
            exitwhen hero == null
            set c.x = x+GetRandomReal(-75,75)
            set c.y = y+GetRandomReal(-75,75)
            set chest = CreateUnit(GetOwningPlayer(hero),CHESTID,c.x,c.y,270)
            set c.chests[GetPlayerId(GetOwningPlayer(hero))] = chest
            call SetUnitX(chest,c.x)
            call SetUnitY(chest,c.y)
            call ShowChestForPlayer(chest,GetOwningPlayer(hero))
            set Chests[GetUnitId(chest)] = c
            set i = 0
            loop
                exitwhen i >= ITEMSLOTS
                if iT.itemType == i then
                    call UnitAddAbility(chest,iT.abilId)
                else
                    call UnitAddAbility(chest,EmptySlotAbil[i])
                endif
                set i = i + 1
            endloop
            call GroupRemoveUnit(G,hero)
        endloop
        set c.ticks = CHESTHP
        set c.rollText = CreateTextTag()
        call SetTextTagText(c.rollText,"-Roll-",0.024)
        call SetTextTagPos(c.rollText,c.x-32,c.y+60,0)
        set i = 0
        call SetTextTagVisibility(c.rollText,false)
        loop
            exitwhen i >= MAXPLAYERS
            if GetUnitTypeId(c.chests[i]) != 0 then
                if GetLocalPlayer() == Player(i) then
                    call SetTextTagVisibility(c.rollText,true)
                endif
            endif
            set i = i + 1
        endloop
        call SetTextTagPermanent(c.rollText,false)
        set c.itemId = itemId
        set c.name = name
        set c.chestCount = count
        call TT_StartEx(function ChestCore,c,1)
    endif
    set chest = null
endfunction

	if GetLocalPlayer() == GetTriggerPlayer() then
		call PreloadGenClear()
		call PreloadGenStart()
		
		// The line below creates the log
		// Right now, this is:
		//      Hero: (hero name)
		//      Level: (hero level)
		//      Code: -load XXXX
		call Preload("\r\n\t\t\t\tHero: " + heroName + "\r\n\t\t\t\t" + "Level: " + I2S(heroLevel) + "\t\t\r\n\t\t\t\t" + "Code: -load " + SaveCode[her.pNum] + "\r\n\n\t\t    ")
		
		// The line below creates the file at the specified location
		// Right now, this is:
		//      "Warcraft III\Profligacy\(hero name) - (hero level)"
		call PreloadGenEnd("Profligacy\\0.21b\\" +  GetPlayerName(GetLocalPlayer()) + "-" + heroName + " - " + I2S(heroLevel) + ".txt")
		endif

Those are all the get local player references. They play sounds, display images and text and change the scale of a chest as a hacky way to make it invisible for players that shouldn't see it (which I'm not sure even works). The last one is for the save to .txt system which can be disabled or removed without effecting the desync.

 

[Emm-A-]

hmm, does this create or destroy any handles?

call QuestSetDescription(q.questInfo,Objectives2String.evaluate(q,i))

 

[pOke]

Can you switch your tags to [/code]?

native QuestSetDescription takes quest whichQuest, string description returns nothing

Appears to be no?

 

[Emm-A-]

sry p0ke I did not mean the native, I thought, maybe Objectives2String.evaluate(q,i) may do?

 

[pOke]

Hmm. The I don't have much experience with .evaluate. Maybe take a read of this from the manual.

JassHelper 0.A.0.0

It talks about not using .evalute with any sync native. I'm not sure, but that might mean that GetLocalPlayer() calls cannot be used with .evaluate?

 

[Swan]

Is turning off a trigger enough to prevent it from desyncing or would it have to be entirely deleted/rewritten? I could just turn off all the triggers and test for desyncs after that.

 

[pOke]

If you were to disable the quest system, as long as you aren't using .evaluate elsewhere should be a way to test if it is a problem.

 

[Emm-A-]

yep, thats how it works.

 

[Swan]

Alright so after a battery of testing, its something to do with the units in the map. Deleting all triggers doesn't fix it, deleting all doodads doesn't fix it, deleting both of those doesn't but any combination of deleted things that involves deleting all units does fix it. Just deleting all units fixes it without deleting anything else but triggers that referenced units.

 

[Swan]

A version in which only units of the factory dungeon and surrounding shops are deleted does not desync, however whenever I try to refine the deletions further than that the desyncs return. Does anyone know anything about unit based desyncs? The map can still desync will all triggers removed.

 

[Tal'Rarity]

A version in which only units of the factory dungeon and surrounding shops are deleted does not desync, however whenever I try to refine the deletions further than that the desyncs return. Does anyone know anything about unit based desyncs? The map can still desync will all triggers removed.

Yeah, it is sadly possible to have an object that desync's games. That's one of the worst ones to encounter if you have lots of units on map... A few idea's would be to make sure it isn't a imported model or texture then delete groups of the map at a time to narrow it down and once found that area that is causing it then it is best just to delete all the objects in that area and replace them. If you know how to create new MPQ's you can also try that since it might be a corrupted placement file as well.

I don't know much about unit based desyncs because they're so rare however there should always be a way to fix any problem, a few other maps I know of that had such situations was a unit with no movement speed/buttons and put into deep water, units too close to the edge of the map, using certain camera actions or a hidden unit that has its pathing walked/interacted on.

 

[Swan]

I feel like I'm smashing my head against a brick wall here. There are hundreds of units in this map and no matter which selections of units I delete it still desyncs unless I delete every unit. Once all the existing units are deleted I can place new ones without causing desyncs. It kind of seems like the entire war3mapUnits.doo is corrupted somehow. Is there any way to manually edit the war3mapunits.doo file? I'd really like to avoid replacing every single unit in the game.

 

[EdgeOfChaos]

First: How exactly are you updating the map? Did you get an unprotected version, deprotect it, or inject code into the compiled script?

Second: Did you widgetize the map for any reason

Third: What exactly have you done? Can you list every change you made since the last time you tested it and it it functioned? (I would recommend keeping a changelog for things like this)

I know there are some maps that WC3 simply hates. Like Ostarrichi, which always desyncs even with all initiation code removed, etc. But if the original map worked, it should be possible to find the desync cause.

 

[Swan]

I got an unprotected version from Spatulon. That version does desync, but it may be because I compiled. It wasn't playable in the state I recieved it. I do have protected but non-desyncing versions of the map as well.
I don't believe I widgetized the map at all, that would only happen in the map optimizer tool right?
I've added a few units, added waygate functionality to a unit previously only used to indicate the presence of doors and added a bunch of abilities to heroes. Plus just moved around a few units to change how dungeons aggro. I have tried deleting every unit I added or modified, it doesn't appear to be them. In fact deleting every unit and adding new ones works, it would just be incredibly tedious to fix the map that way.
Also just to clarify the map still desyncs with all triggers removed. Right now the only way I can guarantee the map to not desync is by removing all units currently on the map.

 

[Dr Super Good]

There were APIs written as part of a competition to iterate through the doo files of WC3 if I recall. Using one you might be able to dump the contents into a human readable form and then look for any inconsistencies.

 

[EdgeOfChaos]

Try removing all imports. Then remove all extended tiles (as in, all past the default amount you can have). Select all units on map, delete, all doodads, delete etc. Then go and remove all objects of a type (like units, abilities, etc) by importing an empty data file. Hopefully one of those can fix it, and then you at least know which category has the problem.

 

[Swan]

I am 100% sure its something to do with units. Restoring terrain to defaults doesn't help, nor does deleting doodads. I'll try out removing abilities though, the map has a system to create abilities with objectmerger so maybe something got broken there.

Edit:
Restoring abilities to default by importing ability data didn't work, not even when deleting all triggers as well. Deleting all imports didn't work. Restoring unit data to default worked, however its probably just because there are no default units on the map and it effectively did the same as deleting all units.

Edit:
Deleted everything but 70 units in the town area of the game. Leaving them there causes desyncs. Copying that selection, deleting it then pasting them down in an empty area doesn't desync. If world editor's copy and paste functionality wasn't so laggy and inexact I'd have my solution here.

Edit:
Well pasting them back into their original locations on the map brings back desyncs. I don't know what to do.

 

[EdgeOfChaos]

So then it's something with those 70 units in town. Perhaps the reason why pasting them somewhere else does not desync is because they are not visible down there (I assume). So the only thing left to do is delete the units, maybe delete 1 unit-type after another, and see which one is causing it. If there's no pattern with unit-type, then you must delete units 1-by-1. It would take a significant amount of time, but would eventually get you the unit(s) that are causing the problem.

My guess is the game complains about an invalid field somewhere. Perhaps a model was set to something weird that the game doesn't like. Or an attachment or something.

When you find what causes it, maybe if you post what the unit's data is, we can find out what's wrong with it.

 

[Swan]

Alright, I'll try an even more tedious round of deleting things.

 

[Emm-A-]

I would try to choose custom ones first before testing the blizzard models.

 

[Swan]

The map still desyncs with all imports deleted, would that matter?

 

[Emm-A-]

yep, sorry I did not read that. I thought, you still had custom models used by some units. You know that you have to save, reopen we in between tests right?

Edit: sry, I was being stupid! Forget it

 

[Dr Super Good]

You know that you have to save, reopen we in between tests right?

Does WE even stream data to WC3 like SC2E does to SC2?

 

[Swan]

I save and then copy the file over to the warcraft directory and start the multiplayer emulation. I don't see why WE would need to be restarted between versions as all the changes appear to be made successfully.

 

[Swan]

Alright so I've done some Warcraft Science recently. I had a hunch that since the desyncs stop happening with no units OR when deleting almost every unit and moving the remaining ones somewhere else, that the desyncs may be caused by some sort of corrupt location data for the units. So I used Lapik's mpq editor to open the map and extract the war3mapunits.doo to check it out. Honestly I couldn't make heads or tails of most of it, even my test maps didn't seem to work. For example a test map with 5 towers at the center of the map in a + formation correctly listed the number of units but only actually had data for the starting location unit and the first tower from the left. I assumed that the either the center of the map was 0,0 for x,y coordinates or that the bottom left would be. So when then I tried overwriting profligacy's war3mapunits.doo with the test map's, expecting to see them in the center or offset from the bottom left equal to the original smaller map. Instead they ended up way out of bounds at the top of the map (still center horizontally). The picture is attached as is the .doo. I don't know exactly what to make of this. Profligacy is an enormous map dimensions wise. Could a tool used to increase the map size limits some how have put it way into negative coordinates to bypass Warcraft 3's limits, thereby causing some weird desync bug with unit location data? The .doo is renamed to a .txt as it wouldn't upload otherwise, its in hexadecimal format. I used the following guide to decipher it.

 

[Dr Super Good]

The middle of the map is usually the centre, but actually the centre is an offset defined in the w3e file.

 

[Swan]

Is the bottom corner of the map often such a huge number?

 

[Dr Super Good]

Is the bottom corner of the map often such a huge number?

Yes for 480*480 maps. 240 * 128 = 30720

 

[Swan]

Ah okay, so each tile is 128 points?

 

[Dr Super Good]

Ah okay, so each tile is 128 points?

Each tile is 128 by 128 Warcraft III units.

 

[Swan]

Could having a map bounds like the one attached cause problems somehow?

 

[Dr Super Good]

As far as I am aware doing stuff near map edge can cause crashes but not desyncs. Bigger bounds gives more area before map edges so if anything is more safe.

 

[Emm-A-]

didnt the editor refuse to set certain map bounds anyway? I assumed that this is inbuild to avoid problems with bad numbers there. I am pretty sure, at least JNGP has this feature ...

 

[Swan]

I just found it odd that the map begins and ends in negative coordinates.

 

[Dr Super Good]

I just found it odd that the map begins and ends in negative coordinates.

The map offset must not be centred.

 

[Swan]

How would I go about fixing this? I've been trying to go through the war3map.w3e but I'm finding it difficult to decipher.