[Mapping] Explaining Warcraft's lockstep architecture for mapping (avoid desyncs)

[Luashine]

I haven't seen Warcraft 3's concepts explained anywhere in a condensed manner, let me try it here. Who is this tutorial for? The beggin beginners in War3 modding, though the explanation will probably require some prior programming knowledge or a little experience with World Editor.

Lockstep Networking​

This is the hardest but it is the foundation of everything you'll need to battle with (desyncs go brrrrr). Here are some two top articles from search (one, two) but I will try to explain in my own words too.

Non-lockstep​

You are probably used to the server-client model. The single server receives all players' data, processes it and sends back changes that happened to the game world. The games on players' computers don't run the entire simulation but simply move objects around as told by the server.
An Example? All kids love Minecraft; imagine a cow, a planted wheat and the player on a server. Each world tick, the server runs AI for cow, grows wheat by +1 tick and listens to what player has done (movement, attack). If the cow has moved, server tells player the new position of our cow. If the wheat has visibly grown, server tells player to update the growth stage. If the player had attacked the cow, server processes the health damage and returns to player whether the hit was successful or not and if the cow has died. Player's game client blindly follows server's information and if something's not right - ask the server again.

Notice how the player sends her/his actions and the server knows and runs everything, and responds with world changes to the player. Lockstep is different.

Lockstep​

In lockstep every single player is acting like a server, in the sense that everybody knows everything. Every player runs the full world simulation, runs the same AI etc. Because everything is the same all you need to know is other player's actions (input) to simulate the next step. All players advance the world simulation one step at a time, well, in lockstep with each other. Here's another example featuring 3 persons: game master, Rose and Bonnie. Rose and Bonnie cannot talk or see each other, but can hear GM's instructions. The players must follow instructions to play the game.

Rose	Game Master	Bonnie
	Start with word "pipe"
pipe	Add word "apple" at the beginning	pipe
apple pipe	Remove the second "p" in "pipe"	apple pipe
apple pie	Compare your results!	apple pie
I've got apple pie!		Hey, me too!

This is what lockstep means. Without working together at every step, Rose and Bonnie arrived at the same results given equal input. Warcraft 3 multiplayer works this way too. Since they arrived at the same result, their work was synchronized at every step.

Let's look at an example when things go wrong and out of sync.

Rose	Game Master	Bonnie
	Start with "I like"
I like	Continue with word "this"	I like
I like this	The first letter of your name starts the next word	I like this
I like this r	Add letters "ed" to this word	I like this b
I like this red	Finish the sentence with an exclamation mark	I like this bed
I like this red!	Compare your results!	I like this bed!

Uh-oh, Rose likes the color red and Bonnie really likes that bed. They'll get into a fierce argument over who is right here, although they followed the same instructions! That's the textbook webpage example of a desynchronization (desync in short).

Who is right here? Actually both, we trust both players to execute instructions correctly and if something goes wrong, then we don't know who is to blame, it's a 50-50. If there were more players then we could decide which group (red or bed) has more players, then the smallest group must've been wrong. Simple voting, we kick the smaller group to keep the bigger group playing.

Lockstep in practice​

Most of the time players playing your map will execute the same instructions and arrive and exactly equal results. However there are some cases... where this is not the case and you must rely on per-player data. If done right your players will not desync, and one player's data will be transferred to another (keeping everyone in sync). It is important to understand, that the only desync-able causes in wc3 are unintended network traffic and differences in the game world. If a unit dies on Rose's screen, but not on Bonnie's screen, this is obviously a desync.
What is a prime example of a tolerated difference between players? Messages on screen, unit names, translated unit voices etc. How do you think are Koreans playing on NA servers?
Instruction: Show on screen the message "Hello, <your own name>!"
Rose sees: "Hello, Rose!"
Bonnie sees: "Hello, Bonnie!"
And nobody here desyncs, because it's not affecting the game action.

Let's put it to code:

Spoiler: DisplayTextToPlayer explanation

DisplayTextToPlayer, Parameters:

toPlayer	player	target player
x	real	new text box position (default is 0, clamped to: 0.0-1.0)
y	real	new text box position (default is 0, clamped to: 0.0-1.0)
message	string	text (supports color codes)

end of doc

function DisplayHello takes nothing returns nothing
    local player selfPlayer = GetLocalPlayer()
    local string textMessage = "default msg"

    if GetPlayerName(selfPlayer) == "Rose" then
        set textMessage = "Hello, Rose!"
    elseif GetPlayerName(selfPlayer) == "Bonnie" then
        set textMessage = "Hello, Bonnie!"
    else
        set textMessage = "Hello, stranger?!"
    endif

    call DisplayTextToPlayer(selfPlayer, 0, 0, textMessage)
endfunction

function DisplayHello()
    local selfPlayer = GetLocalPlayer()
    local textMessage = "default msg"

    if GetPlayerName(selfPlayer) == "Rose" then
        textMessage = "Hello, Rose!"
    elseif GetPlayerName(selfPlayer) == "Bonnie" then
        textMessage = "Hello, Bonnie!"
    else
        textMessage = "Hello, stranger?!"
    end

    DisplayTextToPlayer(selfPlayer, 0, 0, textMessage)
end

Take a while to understand what's going on here. Each player is running his own world simulation, but following the same code (everybody runs this code). However "GetLocalPlayer" will return for every player a different result. When seen from Rose's computer, the selfPlayer object will represent "Rose". Bonnie will be storing herself in "selfPlayer".

Following that, the if condition will set different text for each player who observes it. Imagine a fork in the path. But at last we join different paths together in a way that doesn't affect the game world, we only print some innocent text. Hello, stranger! I doubt your name is either Rose or Bonnie

Rose's version of events:

1. GetLocalPlayer() returns player_rose_object
2. GetPlayerName() thus returns "Rose"
3. textMessage is set to "Hello, Rose!"
4. This textMessage is displayed.

Bonnie's version of events:

1. GetLocalPlayer() returns player_bonnie_object
2. GetPlayerName() thus returns "Bonnie"
3. textMessage is set to "Hello, Bonnie!"
4. This textMessage is displayed.

Once again note how one single change led to a cascading effect and totally different outcomes. The only step that was different is (1); then (2) and (3) were the results of that difference.

Dealing with async data​

Of course the trouble doesn't end there with GetLocalPlayer... Let's look at another common example, localized names. Roses are red, violets are blue, Rose is playing in Russian, but she ain't you.

Rose's game is the same version as Bonnie's but Rose has the Russian localization and Bonnie plays the director's cut English version.

• All in-game names, default voice lines etc. will be different between Rose and Bonnie
• However as shown above, it's OK to show different text
• It is BAD to rely on names to change the world (guaranteed desync at some point)

One such function is GetUnitName, it will return a different display name based on player's chosen language. Hell, localization is not set in stone and may change between game versions too! User-facing display names are meant to be changed so if you use them for your GUI triggers and code, you're asking for trouble. Programmers always have other, permanent names and IDs for these purposes. Anyway, let's follow along and show a good and a bad example.

Note the following warning in Jassdoc's description:

async
This function is asynchronous. The values it returns are not guaranteed to be the same for each player. If you attempt to use it in an synchronous manner it may cause a desync.

Good:

function SpawnMyUnit takes nothing returns nothing
    local player forRedPlayer = Player(0)
    local unit myUnit = CreateUnit(forRedPlayer, FourCC("hfoo"), -30, 0, 90)
    call DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a " + GetUnitName(myUnit))
endfunction

function SpawnMyUnit()
    local forRedPlayer = Player(0)
    local myUnit = CreateUnit(forRedPlayer, FourCC("hfoo"), -30, 0, 90)
    DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a " + GetUnitName(myUnit))
end

Rose will see: "I spawned a Пехотинец"
Bonnie will see: "I spawned a Footman"

That's only display text, it's alright to differ. Now onto a bad example where you desync Rose's and Bonnie's world state through improper use.

Bad:

function SpawnMyUnit takes nothing returns nothing
    local player forRedPlayer = Player(0)
    local unit myUnit = CreateUnit(forRedPlayer, 'hfoo', -30, 0, 90)
    local string translatedUnitName = GetUnitName(myUnit)
    call DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a " + translatedUnitName)

    if translatedUnitName == "Footman" then
        call DisplayTextToPlayer(forRedPlayer, 0, 0, "Oh I hate this unit, lets kill him: " + translatedUnitName)
        call KillUnit(myUnit)
    endif
endfunction

function SpawnMyUnit()
    local forRedPlayer = Player(0)
    local myUnit = CreateUnit(forRedPlayer, FourCC("hfoo"), -30, 0, 90)
    local translatedUnitName = GetUnitName(myUnit)
    DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a ".. translatedUnitName)

    if translatedUnitName == "Footman" then
        DisplayTextToPlayer(forRedPlayer, 0, 0, "Oh I hate this unit, lets kill him: ".. translatedUnitName)
        KillUnit(myUnit)
    endif
end

Rose will see: "I spawned a Пехотинец" and the unit will live on.
Bonnie will see: "I spawned a Footman" and "Oh I hate this unit, lets kill him: Footman" and see how the unit dies in front of her eyes

Now we've got a serious problem. Rose and Bonnie have two different timelines. Rose thinks the unit is alive, Bonnie believes the unit must be dead. The game has desynced. It will not take long for Warcraft to realize what happened and kick one of them for desyncing.
The above means we cannot rely on per-player data here, we need something more static and if we search a little, we will find it:

GetUnitTypeId: returns the unique type ID number, that's 'hfoo' for footman. This will 100% never change and is the same between all players.
UnitId2String: returns some internal name (presumably will never change too), coincidentally it's "footman". But it's rather buggy, choose the first function.

function SpawnMyUnit takes nothing returns nothing
    local player forRedPlayer = Player(0)
    local unit myUnit = CreateUnit(forRedPlayer, 'hfoo', -30, 0, 90)
    local string translatedUnitName = GetUnitName(myUnit)
    local integer unitType = GetUnitTypeId(myUnit)

    call DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a " + translatedUnitName)

    if unitType == 'hfoo' then
        call DisplayTextToPlayer(forRedPlayer, 0, 0, "Oh I hate this unit, lets kill him: " + translatedUnitName)
        call KillUnit(myUnit)
    endif
endfunction

function SpawnMyUnit()
    local forRedPlayer = Player(0)
    local myUnit = CreateUnit(forRedPlayer, FourCC("hfoo"), -30, 0, 90)
    local translatedUnitName = GetUnitName(myUnit)
    local unitType = GetUnitTypeId(myUnit)

    DisplayTextToPlayer(forRedPlayer, 0, 0, "I spawned a ".. translatedUnitName)

    if unitType == FourCC("hfoo") then
        DisplayTextToPlayer(forRedPlayer, 0, 0, "Oh I hate this unit, lets kill him: ".. translatedUnitName)
        KillUnit(myUnit)
    endif
end

Rose will see: "I spawned a Пехотинец" and "Oh I hate this unit, lets kill him: Пехотинец" and he will die.
Bonnie will see: "I spawned a Footman" and "Oh I hate this unit, lets kill him: Footman" and he will die.

Now we have achieved what we wanted without desyncing and taking different player languages into account. All we had to do is to know and avoid functions that return different results per player.

Conclusions (so far):​

• Know which functions depend on localization, other per-player data (like local camera data is not synced, unit Z height is not synced etc.)
• Avoid changing the world as a result of these functions
• Don't save this data in array or variables, it will simply desync later, when you use them to change the world
• Use GetLocalPlayer() in places where you need to diverge code execution, for per-player features
• Use it with caution

As a final note, remember that desyncs aren't detected instantly, some time may pass between the desync happening and game detecting it (approx. 0-15 seconds).

To be continued​

It may end there or it will continue, depends on my motivation and if there are questions left. This first chapter was dedicated only to desyncing and the lockstep model.

• Handles, their internal counters
• Special effects and how to show them per-player
• Menus (aka Dialog API)
• Personalized multiboards
• Sounds, personalization and localization
• Your questions

We need to build up knowledge, when you encounter a desync you should try to separate it to the root cause and write it down like here: Narrowed Down Desync
To document bugs within the community there's a bug tracker: Issues · inwc3/w3-bug-tracker
Jass functions are not well documented too, when you encounter a function without a description please do some tests and write it down too. I bet you aren't the first one left wondering. I wrote a guide here: How to contribute to jassdoc for dummies

 

[LeP]

Good explanation. But here is a question which does not require an answer: how do we actually know warcraft3 runs in lockstep? In the sense that is there any citeable source. Of course we just know it's lockstep and we even have like strong evidence but i searched a few years back and couldn't find any real source.

 

[Luashine]

how do we actually know warcraft3 runs in lockstep? In the sense that is there any citeable source. Of course we just know it's lockstep and we even have like strong evidence but i searched a few years back and couldn't find any real source.

@MindWorX would you please tell us under oath that the game is not running a lockstep networking model?

Update: A failure to provide an answer or untrue statements will be punished by a penalty to your social score and limiting your internet bandwidth to 2 Mbit/s for the next five years.