[Lua] Units are created with different handles

[LazZ]

I have posted something similar to this before, but this time it has nothing to do with my character selection and save/load system. Creating units on the map results in different unit handles at some point = desync.
Though, this time I disabled everything related to the save/load and character selection... and units are still created with different handles, even when just respawning them.

Here's a video link of what happens:
Twitch

Here is my trigger and script:

• Creep Dies
  • Events
    • Unit - A unit owned by Neutral Hostile Dies
  • Conditions
    • (Custom value of (Triggering unit)) Not equal to -1
  • Actions
    • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
      • If - Conditions
        • (Level of (Triggering unit)) Less than or equal to 5
      • Then - Actions
        • Set VariableSet RandomInteger = (Random integer number between 1 and 100)
        • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
          • If - Conditions
            • RandomInteger Less than or equal to 7
          • Then - Actions
            • Set VariableSet Point = (Position of (Triggering unit))
            • Item - Create Lesser Healing Potion at Point
            • Custom script: RemoveLocation(udg_Point)
          • Else - Actions
      • Else - Actions
        • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
          • If - Conditions
            • (Level of (Triggering unit)) Less than or equal to 10
          • Then - Actions
            • Set VariableSet RandomInteger = (Random integer number between 1 and 100)
            • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
              • If - Conditions
                • RandomInteger Less than or equal to 7
              • Then - Actions
                • Set VariableSet Point = (Position of (Triggering unit))
                • Item - Create Greater Healing Potion at Point
                • Custom script: RemoveLocation(udg_Point)
              • Else - Actions
          • Else - Actions
            • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
              • If - Conditions
                • (Level of (Triggering unit)) Less than or equal to 15
              • Then - Actions
                • Set VariableSet RandomInteger = (Random integer number between 1 and 100)
                • If (All Conditions are True) then do (Then Actions) else do (Else Actions)
                  • If - Conditions
                    • RandomInteger Less than or equal to 7
                  • Then - Actions
                    • Set VariableSet Point = (Position of (Triggering unit))
                    • Item - Create Major Healing Potion at Point
                    • Custom script: RemoveLocation(udg_Point)
                  • Else - Actions
              • Else - Actions
    • Custom script: CreepRespawn()

function CreepRespawn()
    local unitType = GetUnitTypeId(GetTriggerUnit())
    local customValue = GetUnitUserData(GetTriggerUnit())
    local player = GetOwningPlayer(GetTriggerUnit())
    PolledWait(GetRandomInt(20, 30))
    CreateUnitAtLocSaveLast(player, unitType, udg_Creep_Point[customValue], GetRandomInt(0, 359))
    SetUnitUserData(bj_lastCreatedUnit, customValue)
    BJDebugMsg("Creep ID: " .. GetHandleIdBJ(bj_lastCreatedUnit))
end

I'm starting to think the desyncs are caused by Lua not being implemented properly, because this shouldn't happen.

 

[Planetary]

Try putting this custom script at the top of the trigger:

math.randomseed( GetTimeOfDay() )

 

[LazZ]

Try putting this custom script at the top of the trigger:

math.randomseed( GetTimeOfDay() )

Units are still created with a different handle :/

 

[Planetary]

Hmm. I have an experiment for you to try:

Do you have timer utils already installed by sir Bribe? ([Lua] TimerUtils)

Add that if you don't have it already and try this timer instead of the PolledWait:

function CreepRespawn()
    math.randomseed ( GetTimeOfDay() )

    local polledWaitRandom = math.random(2,6) -- reduced for testing
    local unitType = GetUnitTypeId(GetTriggerUnit())
    local customValue = GetUnitUserData(GetTriggerUnit())
    local player = GetOwningPlayer(GetTriggerUnit())
    local facing = math.random(0,359)

    TimerStart(NewTimer(player, unitType, customValue, facing),polledWaitRandom,false,function()
        CreateUnitAtLocSaveLast(player, unitType, udg_Creep_Point[customValue], facing)
        SetUnitUserData(bj_lastCreatedUnit, customValue)
        BJDebugMsg("Creep ID: " .. GetHandleIdBJ(bj_lastCreatedUnit))
        ReleaseTimer()
    end)

end

-- wrote this without testing but it should work

I have a suspicion that the Lua transpiler for GetRandomInt does bad stuff with PRNGs i.e. setting the randomseed on something not reliable. If that's the old PolledWait and not a hooked variant, I suspect that might also cause problems since it's Lua land (not sure there). For the record, I'm not certain that GetTimeOfDay() is the best seed (replaces the os.time() method I read about when learning Lua) but it's been working okay for my multiplayer sessions thus far.

(edit: forgot to add, to be scientific you probably want to temporarily disable the item drops)

 

[LazZ]

Still same issue, though your solution to a respawn system is probably cleaner, so I'll keep that.
I suspect the difference in handles is coming from somewhere else, but I'll have to check tomorrow, because it's getting late. We'll see what happens when I start deleting all triggers except the respawn.

 

[Planetary]

Really buggin' me. Hopefully you come back with something.

I'm wondering if there are series of other units being made elsewhere in the map and there's something else throwing off the handle Ids (I noticed the top left frame in the Twitch video was the one that desync'd -- could be a GetLocalPlayer() issue related to that player at that time).

 

[LazZ]

Deleting every trigger except the new unit respawn still does nothing. Handles are still different. I even tried deleting all imported assets and it still produces different handles...
I think my map got corrupted somewhere down the road. I do remember one time I tried to "calculate shadows and save map" but just stood for hours and I decided to force the editor to shut down. Can I in some way move everything I have into a new map? That's the only solution I can think of.

 

[Planetary]

Deleting every trigger except the new unit respawn still does nothing. Handles are still different. I even tried deleting all imported assets and it still produces different handles...
I think my map got corrupted somewhere down the road. I do remember one time I tried to "calculate shadows and save map" but just stood for hours and I decided to force the editor to shut down. Can I in some way move everything I have into a new map? That's the only solution I can think of.

I'm not sure there's an easy way to mass export/import with a Lua map. Could try an MPQ editor (not sure if this one still works Ladik's MPQ Editor 32bit).

I think there are a few things to try before going scorched earth (ordered in the priority I'd do):
1) have you tried making a copy of that trigger that is simplified; i.e. simply regenerate the unit after a few sec and that's it, remove all of the random generator stuff and what not (could also try CreateUnit instead of relying on a bj_ variable; though I doubt that's the problem)
2) have you tried recalculating the shadows again; could make make a back up and let it run a 2nd time (I think it overwrites the existing file)
3) I was doing some research and apparently you could attempt to manually delete the shadow file after unpacking the map

Definitely recommend backing up maps always and constantly (I personally have 41 sequential copies of my map, kek).

(edit: on point 3 I couldn't find the details, just hive veterans like Dr. Super Good mentioning it)

 

[LazZ]

Tried just setting the respawn time to 5 seconds and eliminate all random generation, handles are still different (and this is even with no other scripts). There are no imports, I don't use any PolledWait or Wait actions and I deleted the shadow file from the map (which generated itself again when I saved it). I think it's doomed

 

[Planetary]

Tried just setting the respawn time to 5 seconds and eliminate all random generation, handles are still different (and this is even with no other scripts). There are no imports, I don't use any PolledWait or Wait actions and I deleted the shadow file from the map (which generated itself again when I saved it). I think it's doomed

I'm now in speculation territory of weird WE things that can happen: have you tried copying the map and resetting gameplay constants and game interface?

 

[LazZ]

Just tried it. Same thing.

 

[Planetary]

oh...

I guess I can only really keep asking random questions, such as if you are using a custom editor, etc., or anything non-script that would corrupt it outside of the shadow calc. Seems like you would have already checked those, though.

However, in 1.32 Blizzard did change how the garbage collector works... I'm wondering if they broke something in the process. I was reading this long thread about it [lua] garbage collection issues

 

[LazZ]

Warcraft III does not use the standard Lua garbage collector. Since allocations may occur locally outside the synchronized game state the standard one, used by 1.31, was prone to causing desync as a garbage collection cycle might finish on one client before any others resulting in all gc method calls running out of sync from other clients causing a deviation of synchronized game state. In 1.31 the solution was to explicitly disable the standard Lua garbage collector and periodically call for a full garbage collection in response to a timer, which is synchronized between all clients, expiring. The idea behind this solution was so effective that it was integrated natively in some form in 1.32. The garbage collection functions are now restricted functions that users cannot call.

In that case the garbage collector seems to be doing everything correctly out of the box (as of 1.32). I think the hooked functions I imported a while back did something to break the PolledWait function/action. Or maybe the map was already broken before hooking the functions, I'm not sure. At this point I think I will just transfer everything to a new map and see what happens.

 

[LazZ]

I've got some fantastic bad news. I spent 3 hours copy/pasting the terrain (and also all object data) from my map into an empty one, and just making a respawn trigger still causes different unit handles.

• Initialization
  • Events
    • Map initialization
  • Conditions
  • Actions
    • Set VariableSet Counter = 1
    • Set VariableSet UnitGroup = (Units in (Playable map area) matching ((Owner of (Matching unit)) Equal to Neutral Hostile))
    • Unit Group - Pick every unit in UnitGroup and do (Actions)
      • Loop - Actions
        • Unit - Set the custom value of (Picked unit) to Counter
        • Set VariableSet RespawnPoint[Counter] = (Position of (Picked unit))
        • Set VariableSet Counter = (Counter + 1)

• Creep Respawn
  • Events
    • Unit - A unit owned by Neutral Hostile Dies
  • Conditions
  • Actions
    • Wait 5.00 seconds
    • Unit - Create 1 (Unit-type of (Triggering unit)) for Neutral Hostile at RespawnPoint[(Custom value of (Triggering unit))] facing Default building facing degrees
    • Unit - Set the custom value of (Last created unit) to (Custom value of (Triggering unit))
    • Custom script: BJDebugMsg(GetUnitTypeId(GetTriggerUnit()) .. " - " .. "Handle: " .. GetHandleIdBJ(bj_lastCreatedUnit))

 

[Uncle]

I'm a little confused. Units always have a different handle unless i'm mistaken, that's what sets them apart from one another and makes them unique. If you're looking for the Unit TYPE Id, which is the static value that say all Footman have for example, then reference GetUnitTypeId().

function Test()
    local u = CreateUnit(Player(0), FourCC("hfoo"), 0, 0, 0) --Creates a Footman
    print(GetHandleId(u)) --Prints the specific unit's handle. This will always unique for any unit.
    print(GetUnitTypeId(u)) --Print the unit-type id. This will always be the same value for units of this Unit-Type (Footman in this case)
end

Also, I see you're using custom value, this is fine and maybe you did this intentionally, but with Lua you can simplify things by plugging the unit into the index of your tables.
For example:

function CreepTable()
    --Create Tables
    CreepX = {}
    CreepY = {}
    CreepFacing = {}
    --Create The "Creep" Unit
    local u = CreateUnit(Player(0), FourCC("hfoo"), 0, 100, 270)
    --Save Unit To Tables
    CreepX[u] = 0 --Now we always have the unit's starting X coordinate
    CreepY[u] = 100 --Now we always have the unit's starting Y coordinate
    CreepFacing[u] = 270 --Now we always the unit's starting facing angle

    --[[So when our creep dies, we can reference these variables without the
    need of a unit indexer, and "respawn" it at the proper spot facing the proper angle.--]]
end

I attached a map with a working creep respawn system. I don't know about desyncs but none of these functions have ever caused any problems for me before.

 

[LazZ]

I appreciate it, but what I'm talking about is my units have different handles per machine (should probably have clarified that lol). I know handles are supposed to be unique for each unit, but everything has to be synced up correctly, otherwise we get a desync, no? This is not unique to this map though. In frustration I created a totally different project just to focus on something else, but you know what? That map also produces different unit handles per machine and desyncs (also written in Lua).

Screenshot:


This is the code that spawns units in the new map:

function SpawnUnits()
    for t = 1, 2 do
        local player = udg_SpawnPlayers[t]
        local spawnLoc = GetUnitLoc(udg_Fortress[t])
        local moveLoc = GetRectCenter(udg_AttackRegions[t])
        for i = 0, 40 do
            if (unitSpawns[player][i] > 0) then
                local unitType = tableSpawnUnitTypes[i]
                for j = 1, unitSpawns[player][i] do
                    local u = CreateUnitAtLoc(Player(player-1), unitType, spawnLoc, 270)
                    IssuePointOrderLoc(u, "attack", moveLoc)
                    SetUnitUserData(u, 1)
                    BJDebugMsg(GetHandleIdBJ(u))
                end
            end
        end
    end
end

I don't see how any of this is gonna produce different handles for each client, so I guess it's a bug with the game.

 

[Uncle]

Ah, I see what you mean. That's extremely weird...

 

[Planetary]

It's confusing that it's sequential up to a certain point then spikes a few thousand positions.

In your original example with Azeroth RPG it did the opposite and the handle size moved backward. It's as if the recycling of handle Ids is bugged.

Very suspicious, the lot of it. There definitely is a cut-off point where the jump occurs in either direction which causes the mistmach.

Did any of this desync crap happen prior to 1.32.3?

 

[LazZ]

Yes, it also happened on 1.32.2, but I'm unsure if it did on any patches prior to that (it most likely did).

 

[Luashine]

@LazZ did you figure it out in the end? I'm facing a desync issue on a Lua map too, but it's very big and I haven't started to narrow it down yet. You had a smaller test map for this I think? If so can you send it to me, I want to have a look too.
The map I'm working on has many timed triggers with intervals down to 1ms and in 80% of games someone desyncs early on, approximately 40-60s after game start. Though if no one desynced at that point (40-60s), we will continue playing fine for the entire 2-3 hours game session. The Jass version plays much better (1-5% desync, if that).

UPDATE: My map desync is not related to timers, there seems to be a problem with Damage Engine (v3.x) that only causes a desync in Lua, but not the original Jass code.

 

[LazZ]

I need to clarify that the entire thread is very misleading, since I now know there’s no problem with handle ids varying across clients when the map is set to Lua mode, so long as the ids still point to the same game objects.

I never solved my desync issues though, so I ended up remaking all scripts in JASS.

@Luashine You will not benefit from the test map since it’s based on wrong assumptions.

 

[Tasyen]

I know of 3 reasons that can go totally wrong in Lua and result into such a mess. But first make sure we talk about the same thing, with HandleId I talk about the number got by calling the function GetHandleId.

When you use pairs to loop a table which uses LuaObjects[function, table, userdata] as keys, because the LuaObject-Keys are async and then the order of execution is not the same anymore.

Creating Warcraft 3 Objects (Like Unit,Group, Loc, Timer, Trigger, ...) before function Main happens, aka create Warcraft 3 Objects in the Lua root code. This is a problem because the Lua root code executes during Lobby and when the map is started (2 Times). Fix: don't create any Warcraft 3 Object in Lua Root Code. Only as a result of something that run from function main.

In Warcraft 3 V1.31.1 The garbage collector runs based on local demand. Fix: stop local demand approach and run it by yourself in a fixed setup - collectgarbage("stop") then every x seconds collectgarbage()

 

[Eikonium]

Adding to Tasyens list, I experienced desyncs in Lua mode, when

• creating a "Player moves mouse"-event for any trigger during map Init, even when the trigger was deactivated
• Using Wc3 Hashtables with GetHandleId and FlushChildHashtable

Besides the mentioned GetHandleId and pairs, tostring(LuaObject) is also async.

If you want to loop via pairs and the table contains Lua Objects, you can use [Lua] SyncedTable to prevent desyncs.