• Check out the results of the Techtree Contest #19!
  • Listen to a special audio message from Bill Roper to the Hive Workshop community (Bill is a former Vice President of Blizzard Entertainment, Producer, Designer, Musician, Voice Actor) 🔗Click here to hear his message!
  • Read Evilhog's interview with Gregory Alper, the original composer of the music for WarCraft: Orcs & Humans 🔗Click here to read the full interview.
  • Create a void inspired texture for Warcraft 3 and enter Hive's 34th Texturing Contest: Void! Click here to enter!
  • The Hive's 22nd Icon Contest: Creep Abilities is now concluded, time to vote for your favourite set of icons! Click here to vote!

Known causes of desync

Introduction
Players should be able to differentiate a crash, a player leaving and a player being desynchronized.

Desync is the fact that one or more player gets disconnected from a online multiplayer game. It's caused by differences in the game state between the game clients.

Below I tried gathering the all the desync causes identifed in previous post, plus my own testing results.

1. Causes that will desync at 100%
a. GetLocalPlayer() used for anything else than UI-display or sound-play, or used by Blizzard internal functions (search GetLocalPlayer here to get them). Here is a FAQ on how to use it properly.​
b. GUI Function "Select unit group <group> for player <player>", alias JASS function SelectGroupForPlayerBJ. You should replace it by a for loop that adds unit 1 by 1 to selection (source: here).​
c. [Fixed in 1.31] GUI Function "Pan Camera As Necessary", alias JASS function SmartCameraPanBJ (fix here).​
d. [Since 1.32] Old widgetizer tools to protect the map (if players don't clean their cache by restarting Warcraft III between 2 different games)​
e. GetCameraTargetPosition natives (X, Y, Z) returns an asynchronous value (each player has its own camera), so like for GetLocalPlayer() it must only be used for UI-display or sound-play.​
f. Calling subfunctions in the arguments of BlzFrameSetText() in a "GetLocalPlayer() ==" condition. String concatenation is fine (source here).​
g. JASS function GameCacheSync if called more than 344 times in a single frame (source here).​
h. JASS function SetSkyModel called with an invalid model path as parameter (source here).​
i. Iterating over Lua tables with pairs (not ipairs), because they are not sorted so there is a risk of iterating out of order (source and fix here).​
j. Editing some Game Constants related to creeps with high values (sources: here and here) will desync on game start:​
--- "Creed Guard Distance"​
--- "Creeps - Guard Return Distance"​
--- "Creeps - Maximum Creep Camp Radius"​

2. Causes of desync with low propability (so desync may happen at random times)
a. Numerous "Player slot comparison"s, for example to check if a player is connected (source: my map. I recommand storing a player group for connected players updated with a trigger on event "player leaves the game").​
b. Numerous "Player controller comparison"s (source: my map).​
c. Weapon upgrades with effect "Attack Dice Bonus" desync when "Base + (Upgrade level -1) * Increment" is superior to 14 (source here, here, and my own tests). I've no idea why 14 is the limit, I tested many values below and above.​
d. GetLocationZ calls if
--- A terrain deformations is ongoing below the unit (because deformations are not synchronized between the clients, and not displayed for players with low quality graphics settings)​
--- Unit is on an animated walkable destructable.​
e. Models and and textures when badly made (source here).​
f. A very very huge amount of memory leaks.​
g. Creating a periodic trigger with a null period (even with Blizzard has set a safety min of 0.0001). It will likely hit the op limit at different numbers of executions/different times for each computer the game was being run on.​
h. Creating Timers with a very short delay (<= 0.01s) on map initialization has approx 25% chances of desync (source here).​
i. Destroying user-created frames (source here).​

3. Uncertain functions
LeP made a research in common.j for all annotations with "async" and found the functions below. So if you suspect one of your triggers of causing desync, you can check for the presence of any of those primitives.​

GetDestructableName​
GetSoundDuration​
GetSoundFileDuration​
GetCameraBoundMinX​
GetCameraBoundMinY​
GetCameraBoundMaxX​
GetCameraBoundMaxY​
GetCameraField​
GetCameraTargetPositionX​
GetCameraTargetPositionY​
GetCameraTargetPositionZ​
GetCameraTargetPositionLoc​
GetCameraEyePositionX​
GetCameraEyePositionY​
GetCameraEyePositionZ​
GetCameraEyePositionLoc​
GetObjectName​
GetLocalizedString​
GetLocalizedHotkey​
GetLocalPlayer​
GetLocationZ​
GetItemName​
GetUnitName​
GetPlayerSlotState
IsMultiboardMinimized
BlzGetLocalUnitZ
BlzGetUnitZ
BlzGetItemDescription
BlzGetItemTooltip
BlzGetItemExtendedTooltip
BlzGetLocalSpecialEffectX
BlzGetLocalSpecialEffectY
BlzGetLocalSpecialEffectZ


4. Unverified rumors, unlikely
a. Checking player slot in a trigger on Map Initialization Event. Fix would be to call it after 0.00s elasped.​
b. GetEffectZ: I don't know this primitive, but read about it somewhere, and it would be based on host's value and synced.​
c. SetSoundVolumeBJ & playing the same map many times in a row (source here).​
d. GetLocationZ: randomly causes desync for player using a different setup (linux, mac, reforged vs classic). Also walkable destructables and stiff terrains will cause desync if some players are Classic and others are Reforged (source here).​
e. Selection primitives (source here), but not sure. Indeed internally some selections use GetLocalPlayer, but normally it is synchronized by packets and primitives return the selection result properly for all clients only then.​
f. Low "Backgrounf FPS Cap" in options + players tabbing out of the game (source here).​
g. Creating frames while the game is tabbed-out by the player (because screen size getters may return 0).​

5. Fake rumors, does not cause desync
a. Wait and PolledWait do not cause desync, they just may cause more network traffic to keep all clients synchronized, and may cause lags due to lack of time accuracy.​



 
Last edited:
Lots of great research has been put down here, very nice.

Thanks.
Unfortunately I am still stuck with random desync issues in my map.

I managed to create a file logger. However I cannot log too often, because "write in file" operation is time consuming, particularly with big text files. This is way I store a log array, and only write them in file on some event.
I managed to list all the JASS trigger in my map (by executing a little script). I am only missing the custom JASS triggers whose name does not start by "gg_trg_". But if there are not too many, I can add them manually.
I made a macro able to register a callback on trigger execution start (for ex a log)

However I still miss a way to only the piece of code to write in file just after desync happens...
 
Well personally i've gone through every section of my map and eliminated triggers upon triggers.
So it's not triggers.
I've probably checked everything there is and i think it has to be something basic.
Like the bigger a map is the chances increase for sure.

Can totally agree with this. Also have gone through every trigger, put debugging statements in all my triggers and all custom spells and confirmed it's not just triggers.
Whatever the cause of the desync is, map size/complexity and number of players (9+ players) definitely seems to increase the chances of desync.
 
Can totally agree with this. Also have gone through every trigger, put debugging statements in all my triggers and all custom spells and confirmed it's not just triggers.
Whatever the cause of the desync is, map size/complexity and number of players (9+ players) definitely seems to increase the chances of desync.

After I asked some questions, bigbull gave me a promising possibility:

Before I was able to
- I made a logger with rolling logs & the possibility to write them in file
- With a script executed on war3map.j, I manage to list all GUI triggers (I still need something to log JASS global & local triggers)
- So I was able to write in file a log at each GUI trigger execution. I wrote only on "Player leaves" event to avoid lags. It didn't work if you were the player impacted with desync so I was stuck...

bigbull gave me the idea to not log anything on battle.net games. But to log everything in Replay (thanks to a snippet that can differentiate live game & replay). I hope that if I get the replays of many players for the same game, I can find what piece of trigger bugs.

If it is not triggers but graphics, I'm doomed^^
 
I've been suffering from desync since patch 1.33. My map didn't change anything before and after 1.33. After the patch, there was a desync. This is an individual desync, where one specific user just exits the game. Just leaving the game seems to be the type of post-reforged desync. If anyone knows of a new desync cause since 1.33, please let me know.
 
c. Too many weapon upgrades on a unit > each attack has a chance to cause desync
how many is too many? 20? 100? anything over default (3)?

6. Random known causes
a. Player slot comparison
So we shouldn't use this condition AT ALL? or just not during map init?

What conditions could we use as an alternative?
 
Last edited:
how many is too many? 20? 100? anything over default (3)?


So we shouldn't use this condition AT ALL? or just not during map init?

What conditions could we use as an alternative?

About weapon upgrade, I don't really know. Ask ThaïCat maybe, he posted he met the issue here.

When I personally used player slot comparison in frequent triggers (periodic, on player selection) I had tons of desync. It was before Reforged. So as a solution, I called it only sometimes, and saved the result as a "player group" for frequent comparisons. I wonder if it is because when the game is tabbed out of by a player, it's like during loading and some synchronizations are not made properly..
 
Condenscending as it may sound, these posts are nearly useless without test maps for replication. I understand it will take lots of time to create one (and impossible if the map maker does not have the hardware to test in LAN/on same PC).
EDIT: I see Ricola provided sourced almost everywhere, that's good. I didn't look but I hope at least some of them have ready to test maps uploaded.

I understand the likes of "custom game constants desyncing" - something very clear in the game we haven't found yet because you'd need to ask each player what their previous map was.
I don't understand the playerslot comparison example. Because desyncs happen usually due the other data you compared it with.

My life-saving protip so far is to track each change of your map using git. Reforged's map folder option is great for this, but not required (save the entire map if thats the only thing you can do).
Changed a unit's stats? Save in git. Added code - commit in git. If a desync happens you will be able to find the bad change that causes it in less than 5-8 tests by using "git bisect"
 
Condenscending as it may sound, these posts are nearly useless without test maps for replication. I understand it will take lots of time to create one (and impossible if the map maker does not have the hardware to test in LAN/on same PC).
EDIT: I see Ricola provided sourced almost everywhere, that's good. I didn't look but I hope at least some of them have ready to test maps uploaded.

I understand the likes of "custom game constants desyncing" - something very clear in the game we haven't found yet because you'd need to ask each player what their previous map was.
I don't understand the playerslot comparison example. Because desyncs happen usually due the other data you compared it with.

My life-saving protip so far is to track each change of your map using git. Reforged's map folder option is great for this, but not required (save the entire map if thats the only thing you can do).
Changed a unit's stats? Save in git. Added code - commit in git. If a desync happens you will be able to find the bad change that causes it in less than 5-8 tests by using "git bisect"
Excellent suggestion, I'll git my map from now on.
Unfortunately the current desync bugging me came with the release of Reforged, not a change in code. So even with that, it would be hard to figure out what parts of the code are guilty.
 
I'm not convinced player slot does desync, but just incase I removed over 100 of these conditions. If the player is absent then I destroy their main building then just boolean check if that building is alive as a substitute of slot comparison.
 
I'm not convinced player slot does desync, but just incase I removed over 100 of these conditions. If the player is absent then I destroy their main building then just boolean check if that building is alive as a substitute of slot comparison.
"GetPlayerSlotState" is among the functions whose code use "GetLocalPlayer()" if I'm not wrong, so it is suspicious.

Also I'll have to document it when it's ready, but I think I figured a way to help debugging desyncs with many things pieced together:
  • A file logger using Preload primitives but with good performances (thanks to rotating log files + writing logs by packet instead of 1 by 1).
  • Exporting the map script with the menus (Terrain Editor > File > Export script..."). Then getting a list of all triggers (for ex with Notepad++ and searching all CreateTrigger() calls.
  • Macros to declare the necessary functions & hooks to log when all triggers see their conditions checked & their action executed. That's the tricky part but I figured how add a log function as first condition, last condition, first action and/or last action of each trigger.
  • Also, in addition hooks on the natives known for using GetLocalPlayer() to log when they are directly called.
 
Does anyone know if 6e applies to any edit of those gameplay constants or just setting them too high?

edit: I checked some open source maps and found that some edits do seem to work. For example this: Legion TD Mega 3.5 (B4) + 3.41 unprotect The unprotected old version has some guard distance edits
 
Last edited:
Does anyone know if 6e applies to any edit of those gameplay constants or just setting them too high?

edit: I checked some open source maps and found that some edits do seem to work. For example this: Legion TD Mega 3.5 (B4) + 3.41 unprotect The unprotected old version has some guard distance edits
Nice to know. Have you pushed your tests with some values ?

Also for those who may be interested, I'm currently investigasting a new cause of random desyncs since Reforged: ugrades. Most upgrades don't seem to cause desync, but I'm trying to identify which ones do. For ex "Rhme (Iron Forged Swords)" level 3 (with custom dice values) causes random desyncs for me in a map with simply footmen attacking towers. I'll update you when I'm done with my investigation.
 
what is Old widgetizer tools? w3 map optimizer 5.0 is this?
Well I don't know the current state-of-the-art of widgetizers. But Reforged introduced a regression creating desyncs if you play a 2nd version of the map without restarting the game. So I guess if the widgetizer doesn't have a version more recent than Reforged it has this desync issue.

Since it's from 2012, does it still work ? Or maybe are you editing for older versions of w3 ?


I think https://w3protect.eu/ works on the latest patches, but haven't tested it rigorously yet
 
Some other desync causes: basing any gameplay logic off of handle IDs, and StartTimerBJ, but both of these are only problems for lua maps, as handle ids aren't synced between clients in LUA, and variables get garbage collected in lua so bj_laststartedtimer gets garbage collected every time StartTimerBJ is called
 
New desync cause
 
I have discovered a new source of desync (only on patch 1.32 and newer):

- SelectGroupForPlayerBJ

To add to this:
The actual reason for this is calling ForGroup (also works with ForForce) inside a Local Player code.
It will desync regardless what values you send to those natives.

Fortunately, only that GUI function has that problem.
 
Last edited:
In my Lua map, I had many desyncs.
I found that when I remove a loop of TriggerSleepAction(0.25) that was triggered inside a function called with ExecuteFunc, one of the desyncs (that was always happening at the very start of the map) was gone.
I tried removing all code from this loop except the TSA function, and the desync was still there.
Switching the TSA to a timer also resolved this issue.

I'm not sure the TSA was the direct cause for the desync, but for sure it was leading to it somehow (maybe something in another context desynced due to being called at a slightly different time on different clients).

I moved to using [Lua] - Precise Wait (GUI-friendly) (and hooking ExecuteFunc to change it to a coroutine) and this solved all my desync problems
 
How did you start that other trigger? Did you use a very low "Time elapsed" in Lua?
What other trigger?
I had a function that I stripped to something like:
Lua:
function aaa()
    while true do
        TriggerSleepAction(0.25)
    end
end

...
    ExecuteFunc("aaa")
...

I had a lot of other code running in the map, but with this executeFunc I had desync, without it, or with switching the TSA with a timer I did not.
Timer looked something like this:
Lua:
function aaa()
    t = CreateTimer()
    TimerStart(t, 0.25, false, function()
       DestroyTimer(t)
       aaa()
    end)
end
I tried reproducing it in an almost empty map and failed, so for sure that wasn't the sole cause for the desync.

  • Code was written from memory now, so not sure it's exact. I've already deleted all these changes from my map
  • I think I used TimerQueue lib so the timer function actually looked much simpler without the create and destroy
 
Last edited:
Multiple sources state that in a lua map creating Warcraft 3 Objects such as locations and hashtables outside of any scope e.g. In the declaration of a global variable will cause a desync. The solution is to initialize those variables in an init function. Can we add this finding to the above list?
 
@KitsuneTailsPrower How convenient, that I've recently posted the exact Blizzard.j code in Lua mode that the game runs before you enter the lobby:
  • bj_volumeGroupsTimer = CreateTimer()
  • bj_queuedExecTimeoutTimer = CreateTimer()
  • bj_suspendDecayFleshGroup = CreateGroup()
  • bj_suspendDecayBoneGroup = CreateGroup()
  • bj_delayedSuspendDecayTimer = CreateTimer()
  • bj_lastCreatedGroup = CreateGroup()
  • bj_lastStartedTimer = CreateTimer()
Citation and a test map needed.

PS: On the other hand we have this, that went unsolved for years now. Lua GUI timers always desync (minimal repro)
 
@KitsuneTailsPrower thanks for the test map and additional information in your other thread. [Lua] - [Solved] desync with transpiled lua code
I'm on it. Your workaround is indeed valid for your map. Where you re-define the DDLib__TempLoc Location created at map root (globals block) from within the DDLib__onInit function (aka map's main()). It seems to work without the redefinition. The AI must be human (or orc? but not undead and not nightelf) for the desync to be possible.

I kinda overreacted, sorry for that. Thankfully, your test map is simple enough to dissect it.
 

UPD: Only desyncs in LAN, but not BNET?​

Now this is weird. But we should wait whether the current maintenance team will come up with something


github repo (map version as of posting attached here too)
The culprit:
Lua:
DDLib__TempLoc = Location(0., 0.) -- this is done in INIT -- ROOT LOCATION()

-- Create garbage
-- This is equivalent to putting globals in root, something the game does
for i = 1, 100 do -- garbage directly affects WHEN the desync happens
    local key
    if i % 2 == 0 then
        key = "garbage_" .. i .. "_luatable"
        _G[key] = {}
    else
        key = "garbage_" .. i .. "_jarray"
        _G[key] = __jarray(0)
    end
end

function DDLib__onInit()
  DDLib__TempLoc = Location(0., 0.) -- THIS LINE ALONE TOGETHER WITH SAME DEFINITION IN GLOBALS AT THE START WILL DESYNC
  -- REMOVE ONE OR THE OTHER AND THE DESYNC SEEMS TO STOP.
  -- GO FIGURE.
  -- YES, PLEASE DO FIGURE IT OUT
end
Lua:
function main()
    SetRandomSeed(0xf0f0) -- Set seed that's kinda reliable in conjunction with human AI

    -- CAMERA AND MAP SETUP OMITTED FOR HIVE POST --
  
    InitBlizzard()
  

    DDLib__onInit() -- called from here
  
  
    InitGlobals()
    InitCustomTriggers()
    RunInitializationTriggers()
end

So here it is. It desynced in my testing only IF the following conditions were met:
  1. Location is created in map root, where globals block would have been
  2. That Location is overwritten in init phase, part of the main() call
    1. it's alright if created only in globals or only during init
  3. There exists enough garbage to throw off the garbage collector and affect its iteration order (assumption). at the very least the created garbage affects the time when it finally desyncs
  4. AI player must be playing Human (orc?)
  5. for increased reliability: SetRandomSeed(0xf0f0) at the top of main()
The desync always seemed to coincide with AI's order to start a new building or when the worker reached the construction site and laid the foundation. +-1s

Insights:
  • GetHandleId numeric IDs do get out of sync. Especially if it was going to desync. However, they could go out of sync and you can keep playing without desyncing. This is no bueno.
  • GetHandleId eventually resyncs
  • If you don't overwrite location in Step 2 aka DDLib__onInit, but create another one -- no desync
  • Creating 10000 trash object slowed down when the desync happened by about 15-30 seconds. 100 objects seem to be the right amount (remember, Lua GC is slow?)
  • GetLocalPlayer, if recycled by GC, will create a new lua userdata next time. The handle ID (handle to internal object) stays the same.
  • I assume the only damn way for the GC to desync the game is ifit is run in incremental mode, so for player A it collects like the first 50% of the garbage and for player B it collects other 50% of the garbage, assuming that it can do the rest of the object during the next cycle.
    • of course, this assumes that the iteration order is not deterministic
In other words: wtf. That is, I still assume the GC works in a way that is generally not safe.
 

Attachments

Last edited:
So here it is. It crashed in my testing only IF the following conditions were met:

Thanks for the repro and analysis. :thumbs_up: Just to be clear, by "crashed" did you mean that it forced a disconnect? (i.e. forced a player to get booted from the game) Or do you mean it literally crashed the game and the application closed?
 

BlzCreateFrame seems to have a chance to desync when ran in the wrong context.
I had a sporadic desync that happened right on the game start (about 1 out 8 players desynced).
I was doing
Code:
BlzCreateFrame("EscMenuEditBoxTemplate", BlzGetOriginFrame(ORIGIN_FRAME_GAME_UI, 0),0,0)
from
OnInit.global.

Moving the BlzCreateFrame to run from some trigger later in the game, solved the issue.

OnInit.global is supposed to run after the main already finished, so I'm not sure if that's the same issue as in the original post I shared above
 

OnInit.global is supposed to run after the main already finished, so I'm not sure if that's the same issue as in the original post I shared above
That's where you are wrong.
OnInit(function()
print "All udg_ variables have been initialized"
end)
-- - OnInit.global or just OnInit is called after InitGlobals and is the standard point to initialize.
I really should attempt to map out the default WE code structure for jassdoc, but I think this happens while inside main(). Even if most of the data stuff is supposed to be synced at this point. I wouldn't touch UI any earlier than game start. Ideally after the player is presented with the complete world and UI view.

Thanks anyway, I'm going to include it in some form
 
That's where you are wrong.
In Total initialization, it says:

OnInit.root(function()
print "This is called immediately"
end)
OnInit.config(function()
print "This is called during the map config process (in game lobby)"
end)
OnInit.main(function()
print "This is called during the loading screen"
end)
OnInit(function()
print "All udg_ variables have been initialized"
end)
OnInit.trig(function()
print "All InitTrig_ functions have been called"
end)
OnInit.map(function()
print "All Map Initialization events have run"
end)
OnInit.final(function()
print "The game has now started"
end)
Is this not the order of the initializers?
seems like OnInit (or OnInit.global) should come after main finished
 
You're both correct, there's just miscommunication happening:

OnInit.main is executed when main gets called BUT before anything in main executes, otherwise main would execute after global/trig/map phases.
OnInit.global or OnInit is executed after InitGlobals is done (which technically runs inside map's function main's thread)

tldr: OnInit.main,global,trig,map are all executed from function main's thread
 
today one player becomes desync, accompanied by the host's chat message stating like ‘warning: game desync detected by player [xxx]’ When this chat message appears, does it narrow down the cause of the desync? What could it be?
 
Back
Top