[vJASS] Sync (Game Cache)

[Frotty]

I'll update soon.

Anyway I use this library on makemehost and nothing like that shows. I'm assuming ENT has some API which uses game cache, and this interferes. Not really my fault but if you have a solution then sure.

EDIT: I also use it on my own personal ghost++ bot.

Yes, it seems to only appear on ENT bots (maybe you can confirm this?). It doesn't seem to cause any issues, just wanted to let you know.
If you so desire I could ask the admins on ENT-forum what it's about.

 

[TriggerHappy]

Updated, v1.2.3.

• optimized addString to allow much larger strings without hitting the OP limit.
• isPlayerIdDone is now a wrapper for isPlayerDone.

For anyone curious, the way I solved it was through preloading the values of Char2I in a gamecache. By using the character and alphabet as keys, I can lookup the value through an inline-able function. However since gamecache isn't case sensitive, I add a simple "0" or "1" in front of the key which determines the case.

The longest string I tested was a length of 1000.

Relevant Code

static method bool2I takes boolean b returns integer
    if (b) then
        return 1
    else
        return 0
    endif
endmethod

static method char2I takes string alphabet, string c returns integer // requires preloading cache with data
    return GetStoredInteger(SyncData.Cache[2], alphabet, I2S(bool2I(StringCase(c, true) == c)) + c)
endmethod

private static method preloadChar2I takes nothing returns nothing
    local integer i = 0
    local string c

    loop
        exitwhen i >= ALPHABET_BASE

        set c = I2Char(ALPHABET, i)

        call StoreInteger(Cache[2], ALPHABET, I2S(bool2I(StringCase(c, true) == c)) + c, Char2I(ALPHABET, c))

        set i = i + 1
    endloop
endmethod

Example

scope GameCacheSyncTest initializer Init

    private function SyncComplete takes nothing returns boolean
        local SyncData d = GetSyncedData()
        local string s = d.readString(0)
    
        call BJDebugMsg("Length = " + I2S(StringLength(s))) // Hello
        call BJDebugMsg(s)
        // clean up
        call d.destroy()

        return false
    endfunction

    private function StartTest takes nothing returns nothing
        local SyncData d = SyncData.create(Player(0))
        local string s = ""
        local string ten = "A12345678a"
        local integer len = 400
        local integer i = 0

        loop
            exitwhen i > len

            set s = s + ten

            set i = i + 10
        endloop

        call d.addString(s, len) // must define the (max) length of the string
        call d.addEventListener(Filter(function SyncComplete))

        call DisplayTimedTextToPlayer(User.Local, 0, 0, 15, "Syncing...")

        // begin syncing
        call d.start()
    endfunction

    private function Init takes nothing returns nothing
        call TimerStart(CreateTimer(), 0, false, function StartTest)
    endfunction

endscope

 

[TriggerHappy]

Updated, v1.2.4.

• Updated to support new version of SyncInteger.
• Char2I is now preloaded through a hashtable rather than a gamecache.

I updated demo map on the first post. It contains a little benchmark.

 

[IcemanBo]

Submission: Sync v1.2.4 Date: 1 March 2017 Status: Approved Note: The desync issue seems fixed and all works for me. Triggerhappy made improvements and added also more functionality, it is very good. No reason why it should not be used, approved.

 

[Kazeon]

Looks like calling ReloadGameCachesFromDisk() will bug this system if called after the system's initialization: on sync callback readString will return empty string with length of specified max string length + 1. The solution is to use init module for calling that ReloadGameCachesFromDisk function so it would run before the system's initialization. I think it worth noticing.

 

[Waffle]

hmm.. fails to compile with pjass error :

"Cannot convert player to index" in SyncData.readBuffer

...
// if everything has been synced
    if ( b ) then
        if ( not s__SyncData_localFinished[data] ) then // async
            set s__SyncData_localFinished[data]=true
            // notify everyone that the local player has recieved all of the data
            call SyncInteger(s__SyncData_LocalPlayer , data) //THIS LINE
        endif
   
    endif

...

"Cannot convert filterfunc to code" in SyncData.onInit

    call OnSyncInteger(Filter(function s__SyncData_updateStatus)) // THIS LINE

 

[TriggerHappy]

@Waffle Update [vJASS] - SyncInteger

 

[Waffle]

thx, will try in a moment.

 

[TriggerHappy]

Updated, v1.2.5.

• Updated to support SyncInteger's SelectionSync struct.
• Fixed some errors in the documentation.

This system seems to work much faster on the newest patch. Here are some benchmarks I took using this system:

[1.28.5]

1MB (250,000 ints) : 1044.739 seconds (957 bytes/s)
10KB (2500 ints) : 0.590 seconds (4,327 bytes/s)
40KB (10000 ints) : 26.882 seconds (1,487 bytes/s)

[1.26.0]

1MB (250,000 ints) : ? seconds (? bytes/s)
10KB (2500 ints) : 19.887 seconds (502 bytes/s)
40KB (10000 ints) : 124.035 seconds (322 bytes/s)

 

[Kazeon]

I wonder is the synchronization fully reliable? I just recognized that sometimes it's failed to sync things yet it still fires the "on sync" event which obviously leads to sudden desync.

I'm using it to synchronize player's mouse coordinate. What I can notice when the desync happen is that all sent data became zero. I will try to break it down:

1. This is how I sync things:

set data = SyncData.create(p)
call data.addReal(GetMouseX())
call data.addReal(GetMouseY())
call data.addInt(playerId)
call data.addEventListener(Filter(function OnSync))
call data.start()

2. This is how OnSync function looks like:

    private function OnSync takes nothing returns boolean
 
        local SyncData d = GetSyncedData()
        local real x = d.readReal(0)
        local real y = d.readReal(1)
        local integer id = d.readInt(0)
        local player p = d.from
        local string s
 
        if Locale == p then
            set s = "war3mapImported\\point_target.mdx"
        else
            set s = ""
        endif
        call DestroyEffectTimed(AddSpecialEffect(s, x, y), 5.) // Use TimedHandle from Jass section
        call Fighter[id].move(x, y) // << Desync here. Basically it orders main unit to move to (x, y)
 
        return false
    endfunction

3. I used multiplayer emulator to test my map. I noticed the syncing failure by controlling client-2's character (Fighter[1]) by moving it around. What I noticed when it's desync'ed is that it's not client-2's character who's receiving the order, instead it's client-1's character (Fighter[0]), and it's moving toward the center of the map. It means that

        local real x = d.readReal(0)
        local real y = d.readReal(1)
        local integer id = d.readInt(0)

all returned zero. It's like what you are gonna see in this video:

If you watch carefully, you may notice that when the desync happened, the archer (in client-2's side) is heading toward the correct target point. Meaning that the sent data were correct. While in client-1's side all sent data became zero, that's why in client-1's side Fighter[0] was the one who's receiving the order (sent id became 0) and it's moving toward the center of the map (x & y became 0).

I'm using latest version of this system and patch version 1.27b. If I'm doing wrong in my code please let me know.
I hope you can solve this issue. Thanks in advance!

EDIT:
Damn, I'm sorry before. I've found out that it's all because of a mistake on my end. I forgot to destroy the instance when it's finished syncing. So that hasInt, etc will return true immediately when it uses the same mission key. That's why the sync event got fired before things are actually synced. Sorry again. : (

 

[TriggerHappy]

EDIT:
Damn, I'm sorry before. I've found out that it's all because of a mistake on my end. I forgot to destroy the instance when it's finished syncing. So that hasInt, etc will return true immediately when it uses the same mission key. That's why the sync event got fired before things are actually synced. Sorry again. : (

Also, addEventListener fires even when there is an error, and you can use .lastError to check if there was an error or not.

You may want to check out the onComplete, onError, and onUpdate fields if you only want the run code for a specific case.

 

[Kazeon]

I changed all to "onComplete" and it seems to work fine now. Still desync'ed once but I couldn't reproduce it until now. Still can't confirm if it's caused by other desync in my map or not.

I wonder.. if I don't register anything to addEventListener and onError and there happens to be an error (timeout), would the instance get destroyed/deallocated automatically? I haven't read that part.

 

[TriggerHappy]

I changed all to "onComplete" and it seems to work fine now. Still desync'ed once but I couldn't reproduce it until now. Still can't confirm if it's caused by other desync in my map or not.

I wonder.. if I don't register anything to addEventListener and onError and there happens to be an error (timeout), would the instance get destroyed/deallocated automatically? I haven't read that part.

You should check if the system reported an error.

constant integer SYNC_ERROR_TIMEOUT     = 1
constant integer SYNC_ERROR_PLAYERLEFT  = 2

 

[TriggerHappy]

Update.

v.1.3.0

• Added syncInt, syncReal, syncBool, and syncReal which will send the data instantly rather than waiting for .start(). This removes the overhead of having to loop through all of the data when you want to send your request. The request will be started internally when calling these functions and you shouldn't use these while the request is running.
• Added some escape characters to the Sync alphabet which can improve sync time.
• Updated benchmarking demo.

 

[warpten090]

The game cache only works if u always use 1 computer to host game. What if i use new computer to host game, is this system still work?

 

[Frotty]

The game cache only works if u always use 1 computer to host game. What if i use new computer to host game, is this system still work?

The gamecache is not used between games in this library. In general that is only possible in campaign maps.