• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • It's time for the first HD Modeling Contest of 2024. Join the theme discussion for Hive's HD Modeling Contest #6! Click here to post your idea!

Order-based String Synchronization

Status
Not open for further replies.
Level 19
Joined
Aug 8, 2007
Messages
2,765
Next-gen GUI Codeless Save/Load

NOTE: THIS IS NOT A CONSUMER PRODUCT. DO NOT EXPECT TO USE THIS SYSTEM WITHOUT CONTRIBUTION AND/OR REWORK

SYSTEM IS NOW READY FOR USER IMPLEMENTATION. INSTRUCTIONS ARE IN THE FOLDER. THE SYSTEM IS STILL VERY RAW AND HAS VERY FEW CONFIGURATION INSTRUCTIONS FOR COMMON USER. YES ITS IN GUI BUT REQUIRES JNGP

PLEASE implement this into your map and let me know how this works. The entire community bugtesting this is much more efficient than just me

<latest version attached>

KNOWN BUGS
*None (0.03 fixed all known bugs)


changelog

v0.01 - initial release
v0.02 - improved synhch speed with unary strings, fixed a bug with synchronization, fixed alot of bugs with CodeHandler.
Affected Files - CodeHandler, BinaryStream, ArhowkIO
v0.03 - removed a debug, fixed a bug
Affected Files - CodeHander, PlayerOriginalName

TODO:
1)
4) Fix the wierd NumberStack bug that doesnt allow me to save all 3 stats with maxes above 2^20 (nestharus gogogo)

Order-based StringSynchronization

The background

Since warcraft 3 came out, one of the biggest pursuits of mapmakers was to efficiently transfer data from one computer to another. The first and still the most common method is to encode it through ASCII. Around 2010, the Preload native was discovered to load text data from files but it wasnt synchronized. the SyncStoredInteger could be used, but its rather slow for larger files.

The theory
The theory behind this system is to force a player to a sequence of orders to represent his load code. Using the SelectUnit and ForceUIKey natives, the game can force a player to use abilites on a unit. The unit and the abilities performed represent the code to be synched.

The The implementation
In this demo, a series of spellbooks are used to represent letters a-aA-A0-9. I'm about to fall asleep so not going into more detail

The issues
SelectUnt is buggy as hell. To get it to work, i have to thread sync, select unit, than thread sync again. It also cant be used in mid-gameplay since SelectUnit will corrupt the selections of ppl ingame.


theres also the occasional desync.

The issues with the community
As it stands now, I'm the only one that I'm aware of thats experimented with this system. If it were to be fixed, it would require a hell of alot more testing which would mean that someone else would need to re-do all the experimentation and discoveries that I've done over the past year (I havent exactly been a scientific journal-type person) unless anyone wants to make my AS12/S19 interactive debugger + runtime test emulator for me.


//rant

I've been working on this for the past year or so and its in a state where I would comfortably insert it into my RPG, but thats only because I know i can fix any issues that arise. I'd highly recommend someone else just look at my code and do it their own way. Some suggestions

1) Look into using construction queues instead of abilitys, I didn't but ti was just a theory
2) Most of the code is found in BinaryStream Copy, too lazy to filter everything else out. If you can figure out what everything else does to implement the RPG-style code selection than halla.

If you want a demo or how-to-use than you're not the intended consumer of this script.

note: the attachment says "demo", if i patch it up it mght be a demo but atm its just a conjuration of a bunch of script files that arent even assembled.

Don't anticipate any work by me on this script as i have school + work + robotics + dota 2 to work on. If before Jan. 3rd i get some extra time I might assemble the map into an actual "demo" with readable scripts and shit.

If anyone else is actively looking into this and has questions about the script or needs a bnet test partner than leave a post on this thread.

Heres a little sample (well... all of it) of the synch code

JASS:
library ArhowksNetwork initializer init requires Table, Ascii, GetDummy //everything is little endian
    globals
        private integer A0 = 'A#*0'
        private integer A1 = 'A#*1'
        private integer A2 = 'A#*2'
        private integer A3 = 'A#*3'
        private integer AF = 'A#*/'
        private integer A4 = 'A#*4'
        private integer array ABILS
        
        private string O0 = "A"
        private string O1 = "B"
        private string O2 = "C"
        private string O3 = "D"
        private string OF = "T"
        private string O4 = "E"
        private string array ORDERS
        
        
        private integer B_ALL = 'A%%7'
        private string B_AK = "W"
        private string B_LV = "E"
        private string B_WXYZ = "R"
        private string B_NUM = "Q"
        private string BIG_AK = "T"
        private string BIG_LV = "Y"
        private string BIG_WXYZ = "U"
        
        private string array oCache
        private string array bCache
   //     private integer cacheIndex = 0
        private boolean Syncing = false
        private unit dummyUnit
        private Table timerData
    endglobals
    
    function ClearUnit takes unit u returns nothing
        local location l = GetUnitLoc(u)
        call SetUnitPositionLoc(u, l)
        call RemoveLocation(l)
        set l = null
    endfunction
    
    struct BinaryDecode extends array
        implement Alloc
        integer val
        
        public method isFinished takes nothing returns boolean
            return val == 0
        endmethod
        
        public method next takes nothing returns integer
            local integer one = ModuloInteger(val, 4)
            set val = val / 4
            return one
        endmethod
        
        public static method create takes integer input returns thistype
            local thistype this = thistype.allocate()
            set this.val = input
            return this
        endmethod
    endstruct
    
    struct BinaryStream
    
        integer val
        integer res
        integer bakedRes
        integer count
        static integer DEFAULT_RES = 3
        
        public method add takes integer i returns nothing
            set val = val + (i * R2I(Pow(bakedRes, count)))
            set count = count + 1
        endmethod
        
        public method get takes nothing returns integer
            return val
        endmethod
        
        public method flush takes nothing returns nothing
            set val = 0
            set count = 0
        endmethod
        
        public method destroy takes nothing returns nothing
            call deallocate()
        endmethod
        
        public static method create takes integer res returns thistype
            local thistype this = thistype.allocate()
        
            set val = 0
            set this.res = res
            if res == 2 then
                set bakedRes = 4
            else
                set bakedRes = R2I(Pow(2, res))
            endif
            set count = 0
        
            return this
        endmethod
        
    endstruct
    
    struct UnitData extends array
        implement Alloc
        
        integer typee
        integer value
        
        public method recieve takes integer abilId returns nothing
            if typee == 0 then
               // call FMsg("iqu")
                call IntegerPacket(value).recieve(abilId)
            elseif typee == 1 then
                call IntegerQueuePacket(value).recieve(abilId)
            else
                call StringQueuePacket(value).recieve(abilId)
            endif
        endmethod
        
        public method destroy takes nothing returns nothing
            call deallocate()
        endmethod
        
        public static method create takes integer typ, integer thisv returns thistype
            local thistype this = thistype.allocate()
            set this.typee = typ
            set this.value = thisv
            return this
        endmethod
        
    endstruct
    
    struct IntegerPacket extends array
        implement Alloc
        integer val
        BinaryStream incoming
        UnitData unitData
        boolean fin
        integer value
        player p
        unit u
        
        method getValue takes nothing returns integer
            return value
        endmethod
        
        method destroy takes nothing returns nothing
            if not fin then
                call Network_RemoveDummy(u)
                call unitData.destroy()
                set value = incoming.get()
                call incoming.destroy()
            endif
            call deallocate()
        endmethod
        
        method wait takes boolean b returns boolean
            if b then
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            return fin
        endmethod
        
        method recieve takes integer i returns nothing
            local integer l = 0
            if i == AF then
                call Network_RemoveDummy(u)
                call unitData.destroy()
                set value = incoming.get()
                call incoming.destroy()
                set fin = true 
            else
                set l = 0
                loop
                    if ABILS[l] == i then
                        call incoming.add(l)
                        exitwhen true
                    endif
                    set l = l + 1
                    exitwhen l == 16
                endloop
            endif
        endmethod
        static thistype tem
        static trigger broadc = null
        private static method broad2 takes nothing returns nothing
            local integer i
            local thistype this = tem
            local BinaryDecode stream = BinaryDecode.create(val)
            
            set u= Network_GetDummy(p)
            call UnitAddAbility(u, A0)
            call UnitAddAbility(u, A1)
            call UnitAddAbility(u, A2)
            call UnitAddAbility(u, A3)
            call UnitAddAbility(u, AF)
            call SetUnitUserData(u, unitData)
            
            if GetLocalPlayer() == p then
                call ClearSelection(  )
                call SelectUnit( u, true)
            endif
            
          //  call TriggerSyncStart()
           // call TriggerSyncReady()
            if not Syncing then
                set Syncing = true
                call SyncSelections()
                set Syncing = false
            else
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            
            if GetLocalPlayer() == p then
                if stream.isFinished() then
                    call ForceUIKey(ORDERS[0])
                endif
                loop
                    exitwhen stream.isFinished()
                    call ForceUIKey(ORDERS[stream.next()])
                endloop
                call ForceUIKey(OF)
            endif
            call stream.deallocate()
        endmethod
        
        public method broadcast takes nothing returns nothing
            set tem = this
            call TriggerExecute(broadc)
        endmethod
        
        public static method create takes integer value, player playerFor returns thistype
            local thistype this = thistype.allocate()
            set val = value
            set incoming = BinaryStream.create(0)
            set unitData = UnitData.create(0, this) 
            set p = playerFor
            set u = null
            if broadc == null then
                set broadc = CreateTrigger()
                call TriggerAddAction(broadc, function thistype.broad2)
            endif
            set fin = false
            return this
        endmethod
        
    endstruct
    
    struct IntegerQueuePacket extends array
        implement Alloc
        
        BinaryStream incoming
        UnitData unitData
        Table incomingQueue
        integer incomingCount 
        integer incomingQueuePos
        integer popPos
        boolean recievingCount 
        
        Table Queue
        integer count
        boolean fin
        player p
        unit u
        
        method pop takes nothing returns integer
            set popPos = popPos + 1
            return incomingQueue[popPos-1]
        endmethod
        
        method getCount takes nothing returns integer
            return incomingCount
        endmethod
        
        
        method outOfData takes nothing returns boolean
            return popPos == incomingCount
        endmethod
        
        method add takes integer value returns nothing
            set Queue[count] = value
            set count = count + 1
        endmethod
        
        method destroy takes nothing returns nothing
            call Queue.destroy()
            call incoming.destroy()
            call unitData.destroy()
            set p = null
            call incomingQueue.destroy()
            if not fin then
                call Network_RemoveDummy(u)
            endif
            call deallocate()
        endmethod
            
        method wait takes boolean b returns boolean
            if b then
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            return fin
        endmethod
        
        method recieve takes integer i returns nothing
            local integer l = 0
            if fin then
                return
            endif
      //      call BJDebugMsg("RECIEVED")
     //       call BJDebugMsg("RECIEVED ORDER : " + I2S(i))
            if i == AF then
                if recievingCount then
      //              call BJDebugMsg("RECIEVD CCUNT")
                    set incomingCount = incoming.get()
                    set recievingCount = false
                    if incomingCount == 0 then
                        call Network_RemoveDummy(u)
                        set fin = true
                    endif
                else
                    //if incomingQueuePos = 100 then
                    //    call BJDebugMsg("count : " + I2S(incomingCount))
                    //endif
                    set incomingQueue[incomingQueuePos] = incoming.get() 
                    set incomingQueuePos = incomingQueuePos + 1
                endif
                call incoming.flush()
                if incomingQueuePos == incomingCount then
                    call Network_RemoveDummy(u)
                    set fin = true
                endif
            else
                set l = 0
                loop
                    if ABILS[l] == i then
                        call incoming.add(l)
                        exitwhen true
                    endif
                    set l = l + 1
                    exitwhen l == 16
                endloop
            endif
            call ClearUnit(u)
        endmethod
        
        
        static integer tr_i
        static trigger t = null
        
        private method broadcastint takes integer i returns nothing
            local BinaryDecode stream = BinaryDecode.create(i)
            local integer next
            
            if GetLocalPlayer() == p then
                if stream.isFinished() then
         //           call BJDebugMsg("FINIHSED ORDER : " + ORDERS[0])
                    call ForceUIKey(ORDERS[0])
                else
                    loop
                        exitwhen stream.isFinished()
                        set next = stream.next()
          //          call BJDebugMsg("NEXT ORDER : " + ORDERS[next])
                        call ForceUIKey(ORDERS[next])
                        
                    endloop
                endif
            
                call ForceUIKey( OF )
            endif
            call stream.deallocate()
        endmethod
        
        public method broadcast takes nothing returns nothing
            local integer i
            
            set recievingCount = true
            set u = Network_GetDummy(p)
            call UnitAddAbility(u, B_ALL)
            //call PauseUnit(u, true)
            //call PauseUnit(u, false)
            call SetUnitUserData(u, unitData)
            
                call TriggerSyncStart()
                call TriggerSyncReady()
            if GetLocalPlayer() == p then
                call ForceUICancel()
            endif
            if GetLocalPlayer() == p then
                call ClearSelection()
                call SelectUnit(u, true)
            endif
         //   call BJDebugMsg("1")
            if not Syncing then
                set Syncing = true
                call SyncSelections()
                set Syncing = false
                call TriggerSyncStart()
                call TriggerSyncReady()
            else
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            
            if GetLocalPlayer() == p then
                call ForceUICancel()
            endif
            call UnitRemoveAbility(u, B_ALL)
            call UnitAddAbility(u, A0)
            call UnitAddAbility(u, A1)
            call UnitAddAbility(u, A2)
            call UnitAddAbility(u, A3)
            call UnitAddAbility(u, AF)
            
            call TriggerSyncStart()
            call TriggerSyncReady()
            
            
            
            call broadcastint(count)
            set i = 0
            loop
                exitwhen i == count
     //           call BJDebugMsg("BROADCASTING INT : " + I2S(Queue[i]))
                call broadcastint(Queue[i])
                set i = i + 1
            endloop
      //      call BJDebugMsg("3")
        endmethod
        
        public static method create takes player playerFor returns thistype
            local thistype this = thistype.allocate()
            set incoming = BinaryStream.create(2)
            set p = playerFor
            set u = null
            set Queue = Table.create()
            set popPos = 0
            set incomingCount = 0
            set incomingQueuePos = 0
            
            set incomingQueue = Table.create()
            set count = 0
            set unitData = UnitData.create(1, this) 
            set fin = false
            return this
        endmethod
    endstruct
    
    struct StringQueuePacket extends array
        implement Alloc
        player player
        unit u
        UnitData unitData
        Table stringData
        
        Table bTable
        Table oTable
        string packet 
        integer sDataIndex
        integer sCacheIndex
        integer packetIndex
        integer cacheIndex
        
        public method operator [] takes integer i returns string
            return stringData.string[i]
        endmethod
        
        public method wait takes boolean pause returns boolean
            if pause then
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            if packetIndex >= sCacheIndex and u != null then
                call RemoveUnit(u)
                set u = null
            endif
            return packetIndex >= sCacheIndex
        endmethod
        
        public method recieve takes integer i returns nothing
            local string name = GetObjectName(i)
          //  call BJDebugMsg("GOT")
            if name != "invalid" then
          //      call BJDebugMsg("not invalid")
                if name == "/" then
                 //   call BJDebugMsg("PACKET : " + packet)
                //    call BJDebugMsg(
                    set stringData.string[sDataIndex] = packet
                    set sDataIndex = sDataIndex + 1
                    set packet = ""
                else
         //           call BJDebugMsg("PACKET : " + packet)
                    if name == "space" then
                        set packet = packet + " "
                    else
                        set packet = packet + name
                    endif
                endif
            endif
            set packetIndex = packetIndex + 1
            if packetIndex > sCacheIndex then  
                call Network_RemoveDummy(u)
                set u = null
            endif
        endmethod
        
        public method broadcast takes nothing returns nothing
            local integer i = 0
            set sCacheIndex = cacheIndex
            set packet = ""
            
            set u= Network_GetDummy(player)
            call UnitRemoveAbility(u, A0)
            call UnitRemoveAbility(u, A1)
            call UnitRemoveAbility(u, A2)
            call UnitRemoveAbility(u, A3)
            call UnitRemoveAbility(u, AF)
            call UnitAddAbility(u,B_ALL)
            call SetUnitLifeBJ(u, 1)
            call SetUnitUserData(u, unitData)
            
                call TriggerSyncStart()
                call TriggerSyncReady()
            if GetLocalPlayer() == player then
                call ClearSelection(  )
                call SelectUnit( u, true)
            endif
            if not Syncing then
                set Syncing = true
                call SyncSelections()
                set Syncing = false
                call TriggerSyncStart()
                call TriggerSyncReady()
            else
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            
            call ForceUIKey("Q")
            
         //       call BJDebugMsg("cacheINdex : " + I2S(cacheIndex))
     //   call BJDebugMsg("broadcast broadcast broadcast")
            loop
                exitwhen i == cacheIndex
                    if GetRandomInt(0,10) == 1 then
                        call PrintTextConditional(GetLocalPlayer() == player, bCache[i])
                    endif
                if GetLocalPlayer() == player then
           //         call BJDebugMsg("book cache : " + bCache[i])
                    call ForceUIKey(bTable.string[i])
                    call ForceUIKey(oTable.string[i])
                    call ForceUICancel()
                endif
                
                set i = i + 1
            endloop
            
        endmethod
        public method add takes string orderKey returns nothing
            local integer asc = S2A(orderKey)
            if asc >= 'a' and asc <= 'k' then
                set bTable.string[cacheIndex] = B_AK
                set oTable.string[cacheIndex] = orderKey
            elseif asc >= 'l' and asc <= 'v' then
                set bTable.string[cacheIndex] = B_LV
                set oTable.string[cacheIndex] = orderKey
            elseif asc >= 'w' and asc <= 'z'  then
                set bTable.string[cacheIndex] = B_WXYZ
                set oTable.string[cacheIndex] = orderKey
            elseif asc == '/' then
                set bTable.string[cacheIndex] = B_WXYZ
                set oTable.string[cacheIndex] = "T"
            elseif asc == ' ' then
                set bTable.string[cacheIndex] = B_WXYZ
                set oTable.string[cacheIndex] = "B"
            elseif asc >= '0' and asc <= '9' then
                set oTable.string[cacheIndex] = Ascii2Char(asc + 17)
                set bTable.string[cacheIndex] = B_NUM
            elseif asc >= 'A' and asc <= 'K' then
                set bTable.string[cacheIndex] = BIG_AK
                set oTable.string[cacheIndex] = orderKey
            elseif asc >= 'L' and asc <= 'V' then
                set bTable.string[cacheIndex] = BIG_LV
                set oTable.string[cacheIndex] = orderKey
            elseif asc >= 'W' and asc <= 'Z' then
                set bTable.string[cacheIndex] = BIG_WXYZ
                set oTable.string[cacheIndex] = orderKey
            else
                set oTable.string[cacheIndex] = "A"
                set bTable.string[cacheIndex] = B_WXYZ
            endif
            set cacheIndex = cacheIndex + 1
        endmethod
        
        public method destroy takes nothing returns nothing
            set cacheIndex = 0
            call unitData.destroy()
            call stringData.destroy()
            call oTable.destroy()
            call bTable.destroy()
          //  if packetIndex >= sCacheIndex then  
                call Network_RemoveDummy(u)
                set u = null
            //endif
            call deallocate()
        endmethod
        
        public static method create takes player p returns thistype
            local thistype this = thistype.allocate()
            set player = p
            set unitData = UnitData.create(2, this)
            set stringData = Table.create()
            set oTable = Table.create()
            set bTable = Table.create()
            set cacheIndex = 0
            return this
        endmethod
    endstruct
    
    struct StringPacket extends array
        implement Alloc
        player p

        Table data
        integer length
        boolean hasRecievedLength
        boolean hasRecievedData
        IntegerQueuePacket lengthPacket
        StringQueuePacket dataPacket
        integer dataIndex
        
        public method pop takes nothing returns string
            set dataIndex = dataIndex + 1
            return dataPacket[dataIndex - 1]
        endmethod
        
        public method operator [] takes integer i returns string
            return dataPacket[i]
        endmethod
        
        private method broadcastData takes nothing returns nothing
            local integer i = length
            local integer ib = 0
            local string bc
            local string add 
            local boolean b = GetLocalPlayer() == p
            local integer len
            set dataPacket = StringQueuePacket.create(p)
            loop
                exitwhen ib == dataIndex
                set bc = data.string[ib]
                set i = 0
                set len = lengthPacket.pop()
                loop
                    exitwhen len == 0
                    set len = len - 1
                    
                    if b then
                        set add = SubString(bc, 0, 1)
                        set bc = SubString(bc, 1, StringLength(bc))
                    else
                        set add = ""
                    endif
                    
                    call dataPacket.add(add)
                    
                endloop
                call dataPacket.add("/")
                set ib = ib + 1
            endloop
            //call dataPacket.add("/")
            call lengthPacket.destroy()
            call dataPacket.broadcast()
        endmethod
        
        private method compileData takes nothing returns nothing
            call data.flush()
            set dataIndex = 0
            /*local integer i = length
            local integer da 
            local boolean b = false
            call data.flush()
            set dataIndex = 0
            loop
                set da = dataPacket.pop()
                exitwhen da == ' #^Q' or (b and da == 0)
                set b = da == 0
                if da == ' #^Z' then
                    set dataIndex = dataIndex + 1
                else
                    set data.string[dataIndex] = data.string[dataIndex] + A2S(da)
                endif
            endloop
            set dataIndex = 0
            call dataPacket.destroy()*/
        endmethod
        
        
        public method wait takes boolean pause returns boolean
            if not hasRecievedLength then
                if lengthPacket.wait(false) then   
             //       call BJDebugMsg("length packet")
                    call TriggerSyncStart()
                    call TriggerSyncReady()
                    set hasRecievedLength = true
                    call broadcastData()
                    set dataIndex = 0
                endif
            elseif not hasRecievedData then
             //   call BJDebugMsg("wating data")
                if dataPacket.wait(false) then
                    set hasRecievedData = true
                    call compileData()
                endif
            endif
            
                
            if pause then
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            return hasRecievedData
        endmethod
        integer compDataLength
        
        public method broadcast takes nothing returns nothing
            call lengthPacket.broadcast()
        endmethod
        
        public method add takes string s returns nothing
            call lengthPacket.add(StringLength(s))
            set data.string[dataIndex] = s
            set dataIndex = dataIndex + 1
        endmethod
        
        public method destroy takes nothing returns nothing
            call data.destroy()
            call lengthPacket.destroy()
            call dataPacket.destroy()
            call deallocate()
        endmethod
        
        public static method create takes player pFor returns thistype
            local thistype this = thistype.allocate()
            set compDataLength = 0
            set p = pFor
            set data = Table.create()
            set lengthPacket = IntegerQueuePacket.create(pFor)
            return this
        endmethod
        
    endstruct
    
    private function onAbil takes nothing returns nothing
     //       call BJDebugMsg("RECIEVED ORDER : " + I2S(GetSpellAbilityId()))
    
        if GetUnitTypeId(GetTriggerUnit()) == 'h000' or GetUnitTypeId(GetTriggerUnit()) == Network_GetDummyId() then
        
            call UnitData(GetUnitUserData(GetTriggerUnit())).recieve(GetSpellAbilityId())
        endif
    endfunction
    
    
    
    private function init takes nothing returns nothing
        local trigger t = CreateTrigger()
        call TriggerRegisterAnyUnitEventBJ( t, EVENT_PLAYER_UNIT_SPELL_CAST )
        call TriggerAddAction( t, function onAbil )
        set t = null
        set dummyUnit = Network_GetDummy(Player(0))
        call UnitAddAbility(dummyUnit, B_ALL)
        call Network_RemoveDummy(dummyUnit)
       // call IntegerPacket.create(15,Player(0)).broadcast()
        set ORDERS[0] = O0
        set ORDERS[1] = O1
        set ORDERS[2] = O2
        set ORDERS[3] = O3
        set ABILS[0] = A0
        set ABILS[1] = A1
        set ABILS[2] = A2
        set ABILS[3] = A3
        set timerData = Table.create()
    endfunction
    
endlibrary
//old StringPacket, binary
/*struct StringPacket extends array
        implement Alloc
        player p

        Table data
        integer length
        boolean hasRecievedLength
        boolean hasRecievedData
        IntegerQueuePacket lengthPacket
        IntegerQueuePacket dataPacket
        integer dataIndex
        
        public method pop takes nothing returns string
            set dataIndex = dataIndex + 1
            return data.string[dataIndex - 1]
        endmethod
        
        private method broadcastData takes nothing returns nothing
            local integer i = length
            local integer ib = 0
            local string bc
            local integer add = 0
            local boolean b = GetLocalPlayer() == p
            local integer len
            set dataPacket = IntegerQueuePacket.create(p)
            loop
                exitwhen ib == dataIndex
                set bc = data.string[ib]
                set i = 0
                set len = (lengthPacket.pop() - 1)/ 4 + 1
                loop
                    exitwhen len == 0
                    set len = len - 1
                    
                    if b then
                        set add = S2A(SubString(bc, 0, 4))
                        set bc = SubString(bc, 4, StringLength(bc))
                    else
                        set add = 0
                    endif
                    
                    call dataPacket.add(add)
                    
                endloop
                call dataPacket.add(' #^Z')
                set ib = ib + 1
            endloop
                call dataPacket.add(' #^Q')
            call lengthPacket.destroy()
            call dataPacket.broadcast()
        endmethod
        
        private method compileData takes nothing returns nothing
            local integer i = length
            local integer da 
            local boolean b = false
            call data.flush()
            set dataIndex = 0
            loop
                set da = dataPacket.pop()
                exitwhen da == ' #^Q' or (b and da == 0)
                set b = da == 0
                if da == ' #^Z' then
                    set dataIndex = dataIndex + 1
                else
                    set data.string[dataIndex] = data.string[dataIndex] + A2S(da)
                endif
            endloop
            set dataIndex = 0
            call dataPacket.destroy()
        endmethod
        
        public method destroy takes nothing returns nothing
            set p = null
            call deallocate()
        endmethod
        
        public method wait takes boolean pause returns boolean
            if not hasRecievedLength then
                if lengthPacket.wait(false) then
                    set hasRecievedLength = true
                    call broadcastData()
                    set dataIndex = 0
                endif
            elseif not hasRecievedData then
                if dataPacket.wait(false) then
                    set hasRecievedData = true
                    call compileData()
                endif
            endif
            
                
                
            if pause then
                call TriggerSyncStart()
                call TriggerSyncReady()
            endif
            return hasRecievedData
        endmethod
        integer compDataLength
        public method broadcast takes nothing returns nothing
            call lengthPacket.broadcast()
        endmethod
        
        public method add takes string s returns nothing
            call lengthPacket.add(StringLength(s))
            set data.string[dataIndex] = s
            set dataIndex = dataIndex + 1
        endmethod
        
        public static method create takes player pFor returns thistype
            local thistype this = thistype.allocate()
            set compDataLength = 0
            set p = pFor
            set data = Table.create()
            set lengthPacket = IntegerQueuePacket.create(p)
            
            return this
        endmethod
        
    endstruct*/
 

Attachments

  • Order-based Synch Demo v0.03.w3x
    210.7 KB · Views: 82
Last edited:
Level 19
Joined
Aug 8, 2007
Messages
2,765
Why is ordering units faster than SyncStoredInteger? Latter sounds more elemental and there you can implement your sequential code, too. Especially when working with data packets it's more of an encoding question.

My theory is that when multiple orders are issued at once, than their packets combine. When you use SyncStoredInteger, you have to wait after every packet for some reason. the formulas roughly (iirc)

order - 300 + 20 * # of integers (ms) (constant is higher due to SyncSelection native)
sync stored - 200 + 100 * # of integers (ms)

but yes, testing was done a while ago and this alot faster than SyncStoredInteger untill u piss off the ingame packet warden. (which SyncStoredInteger can do anyway)

e/ the script is 150kb lawl

2e/ ugh i was bored and found a method that would be incredibly helpful for synchronization. ima show up to work on monday with jack shit i just know it
 
Last edited:
Level 19
Joined
Aug 8, 2007
Messages
2,765
stabilized the system a helluvalot more, added an actual test map, refactored the code to make it easier to implement. If the system doesn't desync than it should be fine for public use.
 

Attachments

  • Order-based Synch Demo.w3x
    209.7 KB · Views: 76
Level 31
Joined
Jul 10, 2007
Messages
6,306
Ehh, Arhowk compared this to my Network library. This ran a lot faster. Network packs every 32-bit integer completely so that it sends as little information over the network as it can. It also does bursts to prevent it from running into the warden. Arhowk's beat it majorly, and he doesn't pack his integers because he can't exactly have enough orders to fill out 32 bits.

The major problem of Network is that it sends a lot of extra traffic because it sorta has to do its own ACKs on top of the network ACKs. This is the thread synchronization stuff. If this isn't done, the game desyncs or warden gets pissed off at too much data. With orders, the game synchronizes everything for you. Sure, there are more synchronizations, but I'm sure that 1000 low-level syncs are faster than 1 JASS sync, lol. Well, I'm exaggerating there, but still, the thread syncs really slow things down. It went from near instant to a slow death crawl. Without them, data gets lost.
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
Ehh, Arhowk compared this to my Network library. This ran a lot faster. ... Arhowk's beat it majorly.

Beautiful! :)

Now if you don't want to learn how to do this save/load stuff, my recommendation is to find someone to join your team for your map to handle the save/load dev ^)^, because you are still going to need 100% custom save/load to handle ur custom inventory, and you are going to need some damn complicated item catalogs, hahaha : D. I have item catalogs made that will probably fit your needs perfectly, but I don't want them released yet, so neh neh ;p.

01-05-2012. honestly thats probably the only reason I got into programming

JASS:
            //All
              //Rings
                call catalog.add('I00S', v03G, gA, 1)
                call catalog.add('I016', v03G, gA, 1)
                call catalog.add('I01E', v03G, gA, 1)
                call catalog.add('I00W', v03G, gA, 1)
               // call catalog.add('I02A', v03H, gA, 1)
              //Armors
                call catalog.add('I004', v03G, gA, 1)
                call catalog.add('I00X', v03G, gA, 1)
                call catalog.add('I01F', v03G, gA, 1)
                call catalog.add('I015', v03G, gA, 1)
              //Helms
                call catalog.add('I00J', v03G, gA, 1)
                call catalog.add('I013', v03G, gA, 1)
                call catalog.add('I01C', v03G, gA, 1)
                call catalog.add('I01L', v03G, gA, 1)
              //Boots
                call catalog.add('I00K', v03G, gA, 1)
                call catalog.add('I010', v03G, gA, 1)
                call catalog.add('I019', v03G, gA, 1)
                call catalog.add('I01I', v03G, gA, 1)
              //Gloves
                call catalog.add('I00H', v03G, gA, 1)
                call catalog.add('I011', v03G, gA, 1)
                call catalog.add('I018', v03G, gA, 1)
                call catalog.add('I01K', v03G, gA, 1)
              //Amulets


            //Zealot and Anarchist
              //Swords
                call catalog.add('I002', v03G, gZA, 1)
                call catalog.add('I00Y', v03G, gZA, 1)
                call catalog.add('I017', v03G, gZA, 1)
                call catalog.add('I01G', v03G, gZA, 1)
            //Witch and Runemaster
              //Staves
                call catalog.add('I00O', v03G, gWR, 1)
                call catalog.add('I00V', v03G, gWR, 1)
                call catalog.add('I01D', v03G, gWR, 1)
                call catalog.add('I014', v03G, gWR, 1)
            //Amazon
              //Bows
                call catalog.add('I00G', v03G, gAm, 1)
                call catalog.add('I012', v03G, gAm, 1)
                call catalog.add('I01A', v03G, gAm, 1)
                call catalog.add('I01J', v03G, gAm, 1)
              //Quivers
                call catalog.add('I00F', v03G, gAm, 1)
                call catalog.add('I00Z', v03G, gAm, 1)
                call catalog.add('I018', v03G, gAm, 1)
                call catalog.add('I01H', v03G, gAm, 1)

              //Charms
to bad it doesnt look complex because my rpg never got past level 5 on version 0v03G :D

anyway, back on topic, if this doesn't desync due to the dialogs than its pretty good for public use. I still have to get it an actual API though
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
just ran it with a friend and it worked well. next hour or so i get ill throw a gui interface n shit on it
 
Level 6
Joined
Jul 30, 2013
Messages
282
so ..
automatic save-load for multiplayer..
just became feasible? :)

crying tear of joy atm :')
 
Level 31
Joined
Jul 10, 2007
Messages
6,306
just ran it with a friend and it worked well. next hour or so i get ill throw a gui interface n shit on it

Give me a bit and I'll get to making your system really nice and stuff >.>.



Oh, and other bad news. BitInt currently doesn't work, lmao. Apparently, someone was losing a bit here and there with certain values.



There is still work to be done in codeless save/load ;D. Right now I really wanna focus on Attack Indexer >.<, which means I gotta update Lua stuff to work on any machine, update Bonus (why? dunno, just cuz), clean up Unit Indexer modules, double check DDS, and then finally work on the glorious Attack Indexer, haha.


On the codeless save/load side..


Bitwise package with BitInt, BitManip, etc
BigNum package with BigInt and stuff

Cryptography package with AES and MD5 working with BigInt and BitInt

Network package with thread synchronization and new Network lib stuff

File I/O package with file packing for sending minimal data across the network (similar to that StringPacket stuff or w/e that I did)

Codeless Save/Load stuff that puts everything together, providing options for BitInt or BigInt too ^_^.


So we need stuff done for Attack Indexer and stuff done for Codeless Save/Load.


Anything else major need doin? UnitEvent and Buff stuff (applying things like stun to units).


Ok, slightly off topic now.. but eh

1. Attack Indexer
2. Codeless Save/Load
3. Unit Events/Effects
4. Unit Properties (costs, bounty, etc)

For AES, I'd like AES-128, AES-192, and AES-256.
Would also like 1 bit padding at the start for bounds checking
Version checking packing (similar to what I did before)
Hashes: MD5, SHA-3, Knuth Checksum (all 3 handle variable length inputs)
SHA3-224, SHA3-256. SHA3-384, SHA3-512, SHAKE128, SHAKE256
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
so ..
automatic save-load for multiplayer..
just became feasible? :)

crying tear of joy atm :')

its always been feasible (see korean rpg Zodiac RPG which puts nestharus's system to shame) but this makes it practical.

Give me a bit and I'll get to making your system really nice and stuff >.>.
lol gl
Oh, and other bad news. BitInt currently doesn't work, lmao. Apparently, someone was losing a bit here and there with certain values.
meh i never got bitint to work and you werent willing to help so i just stayed with numberstack/bigint
There is still work to be done in codeless save/load ;D. Right now I really wanna focus on Attack Indexer >.<, which means I gotta update Lua stuff to work on any machine, update Bonus (why? dunno, just cuz), clean up Unit Indexer modules, double check DDS, and then finally work on the glorious Attack Indexer, haha.
your still coding wc3? how silly. if I've already moved on so should you

Bitwise package with BitInt, BitManip, etc
BigNum package with BigInt and stuff

Cryptography package with AES and MD5 working with BigInt and BitInt
or just fix BigInt
Network package with thread synchronization and new Network lib stuff

File I/O package with file packing for sending minimal data across the network (similar to that StringPacket stuff or w/e that I did)

Codeless Save/Load stuff that puts everything together, providing options for BitInt or BigInt too ^_^.
imho the end user doesnt care. he just wants the clicky savey loady
So we need stuff done for Attack Indexer and stuff done for Codeless Save/Load.
sure..

For AES, I'd like AES-128, AES-192, and AES-256.
Would also like 1 bit padding at the start for bounds checking
Version checking packing (similar to what I did before)
Hashes: MD5, SHA-3, Knuth Checksum (all 3 handle variable length inputs)
SHA3-224, SHA3-256. SHA3-384, SHA3-512, SHAKE128, SHAKE256
sure... #creativefreedom
 
Level 31
Joined
Jul 10, 2007
Messages
6,306
How much wc3 dev have I done in the last year? Not much :). Gotta do Unreal Engine 4.6 for the next few months :/. Also did OpenGL with shaders, made a cool renderer, and yea. Did Bullet stuff too. I really wanna do an Attack Indexer tho, cuz it's cool. Wc3 coding, atleast this stuff, is as difficult or more difficult than some really hard c++ stuff. It makes this fun and the other stuff kinda boring.
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
lawl if i had any free time than id make dota 2 RPG snippets. wc3's final blow will be about a year after dota 2 modding tools are released. it was *supposed* to be the sc2 editor but that ended up sucking, d2 editor looking good atm
 
lawl if i had any free time than id make dota 2 RPG snippets. wc3's final blow will be about a year after dota 2 modding tools are released. it was *supposed* to be the sc2 editor but that ended up sucking, d2 editor looking good atm

Cool finally released this, thanks.

Yeah.... that's what everyone before you has been saying with every single new game with the option of modding. Try again next year :thumbs_up:

Still cool, must figure out a way to turn this into GUI! Good job on figuring something out Nes didn't as well howk. :grin:
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
Fixed issues with the dialog system, looking pretty stable right now. I had an issue where the ppbk() function didn't work but i think it resolved itself... haven't had it occur since
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
I'm adding new information, thats the purpose of a post. An edit simply changes the already provided information.

I'm not updating it, its just that its super buggy lol.

v0.03 removes a debug and fixes a slight visual bug with CodeHandler
 
Well isn't all your stuff super buggy? :xxd: (BBX cough cough) :wink:

:grin: Don't worry though I suffer from the same Blizzard harassment. Stuff constantly breaks on me for no reason at all because well you know.... Blizzard doesn't want us improving their nearly free and usually broken/ripped decade old crappy however epic game. :thumbs_up:

Oddly enough for that same reason I had accidently made an epic AI for one of my projects that actually moves strategically and traps you.
 

Deleted member 219079

D

Deleted member 219079

I question whether you want this to be use for masses. I guess not, as this is in the lab.

As of now I'm all good with my code-based save n load - system. But subscribing to see if a nice update comes to this.

Edit: And +rep for the tons of code
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
I question whether you want this to be use for masses. I guess not, as this is in the lab.

I'd like a few people to use it at first to catch any obvious bugs. After these few people get it working comfortably, i'll publish it in the spells section

i care :)

got epic plans with this, thx! :D
:D if you have any suggestions / bugs please don't hesitate to tell me.

also a small perk- anyone helping me here gets my help with implementation & management (not all of it... slackers) whereas the Spells version will just have a txt file + the community.

on a side note, i forgot to remove the "known bugs" section when i uploaded 0.03
 
As long as it requires local files enabled, codeless save/load will never be the go-to for any serious mapper. You just want that Bnet compat. And you can't force players to enable local file support manually. Been there, done that.

Don't get me wrong; I appreciate all the work you guys have put into this. But imho, this is the second step before the first.
 
Level 31
Joined
Jul 10, 2007
Messages
6,306
As long as it requires local files enabled, codeless save/load will never be the go-to for any serious mapper. You just want that Bnet compat. And you can't force players to enable local file support manually. Been there, done that.

Don't get me wrong; I appreciate all the work you guys have put into this. But imho, this is the second step before the first.

Well, there is the automatic bat file generation that they can run + the message telling them to run it. The file can be saved while they are playing, then they can run it and restart wc3 to load =). It's not that big of a deal.
 
As long as it requires local files enabled, codeless save/load will never be the go-to for any serious mapper. You just want that Bnet compat. And you can't force players to enable local file support manually. Been there, done that.

Don't get me wrong; I appreciate all the work you guys have put into this. But imho, this is the second step before the first.

They got the hacked/edited .dll to bypass 8MB working on Bnet and other servers, I saw lots of people using it. Not as much now, but I don't usually look.
 
Well, there is the automatic bat file generation that they can run + the message telling them to run it. The file can be saved while they are playing, then they can run it and restart wc3 to load =). It's not that big of a deal.
Still, that doesn't help the average random player for two reason:
1) Most people probably don't care
2) It takes a certain amount of ... trust ... to execute a file that was magically created by a map without you agreeing to it. The fact that it's basicly abusing a major security hole certainly is not helping.

And yes, you could now argue that saving doesn't require local file support and you usually don't have a code to load in your first game. Still, you'd probably still want to include a code based save/load in addition to the codeless just for compatibility reasons to other OS. Which means you not only need to have two different loading scripts in your map, but also that the codeless one needs to be synced and thus is terribly slow.

On the other hand the one that just requires you to alt+tab out of the game, open the text file and copy the code in will load instantly. Alt tabbing takes less than a second. Copying your code maybe 2 seconds.
Which again, makes you wonder why you actually dealt with the hassle to implement the codeless save/load in the first place, as it isn't even a big improvement to convenience if it takes half a minute or longer for all data to sync.
 
Still, that doesn't help the average random player for two reason:
1) Most people probably don't care
2) It takes a certain amount of ... trust ... to execute a file that was magically created by a map without you agreeing to it. The fact that it's basicly abusing a major security hole certainly is not helping.

And yes, you could now argue that saving doesn't require local file support and you usually don't have a code to load in your first game. Still, you'd probably still want to include a code based save/load in addition to the codeless just for compatibility reasons to other OS. Which means you not only need to have two different loading scripts in your map, but also that the codeless one needs to be synced and thus is terribly slow.

On the other hand the one that just requires you to alt+tab out of the game, open the text file and copy the code in will load instantly. Alt tabbing takes less than a second. Copying your code maybe 2 seconds.
Which again, makes you wonder why you actually dealt with the hassle to implement the codeless save/load in the first place, as it isn't even a big improvement to convenience if it takes half a minute or longer for all data to sync.

The average random players are slowly evolving, most save/load ones already know the .txt file in folder in wc3 directory which is good news.
It's true, most don't care and would rather just ignore what the map tells you to do. However that certain amount of trust you speak of is a bit magical itself considering I have never seen anyone complain about that gigantic security flaw.

That's the problem, the codeless ones opens up a new world of possibilities for wc3 maps. It means you can actually enable trading and a market-type system without having to worry about cheaters. That's one of the possibilities.
Maybe have both, but one can save/load more while the other only saves normally and restricts trading. If you can detect who has it enabled and who doesn't that is.

Sometimes alt+tab will simply fatal/crash or desync or lag and host drops you due to either leaky code,flawed models/sfx's, impatient childish hosts. Depends on the map and pc, not everyone can do this in 2 seconds like you. I do wonder why we've never seen an actual save/load with a trade system. =(
Apparently this is suppose to do it faster then you think and I have seen it in action, it's not that slow. Though it only loaded numbers at the time in his BBX.
 
I do wonder why we've never seen an actual save/load with a trade system. =(
This is because the only way we can save data currently is by storing it in a TXT file. And even if you can overwrite said file easily by just giving it the same name, people could still just go into the folder, make a copy of that file and rename it to effectively archive an earlier version of the save.

As long as saves can not be stored on a web server, you can always cheat any kind of trading system to duplicate items via using older savecodes.

It doesn't matter if your save/load is codeless or not. As long as it writes on your HDD, you can dupe.
 
At least with codeless you can hide and encrypt it. If TriggerHappy's multiplayer extended thing ever finishes then you could store the codes somewhere instead of on someone's pc since everyone joins a host/server.
You already can do that, as almost all games I know are hosted via hostbots anyway. I could imagine f.ex. Makemehost to implement some kind of web folder for mapmakers to store data.
The beauty about this is: it would actually also get rid of the local file support issue, as only the host server needs local file support enabled.

But lets be honest: something like this would have been possible 5 years ago when WC3 was still a thing and SC2 wasn't around to do all the mapping stuff lightyears better. It's unlikely that someone will invest the time and effort to create hostbot software for this.
 
You already can do that, as almost all games I know are hosted via hostbots anyway. I could imagine f.ex. Makemehost to implement some kind of web folder for mapmakers to store data.
The beauty about this is: it would actually also get rid of the local file support issue, as only the host server needs local file support enabled.

But lets be honest: something like this would have been possible 5 years ago when WC3 was still a thing and SC2 wasn't around to do all the mapping stuff lightyears better. It's unlikely that someone will invest the time and effort to create hostbot software for this.

Yeah that'd be cool if someone was actually intelligent enough to do so, but sadly nobody like that exists for the hosting bots.

Honestly...? It's been possible for a long time, but well intelligence was a rare feat in many people. WC3 was a thing..? Still is luckily because you know your still here and well SC2 is pretty well garbage. I'd rather learn C+ then SC2, SC2 isn't able to do anything lightyears better and well it isn't fair to compare anyways though the only thing that it can actually do WC3 can't is full custom UI's, more like slowyears. Though yeah nobody will do this because everyone gives up anyways no matter who they are apparently. If WC3 dies then dota 2 will be the thing to move too.
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
Shame, a couple years ago and I might've been able to get this off the ground. Oh well. Arrivederci.

@Zweb most of your arguments are easily counterable but I have not the time nor energy to do so.
 
Level 21
Joined
Mar 27, 2012
Messages
3,232
Shame, a couple years ago and I might've been able to get this off the ground. Oh well. Arrivederci.

@Zweb most of your arguments are easily counterable but I have not the time nor energy to do so.

If you are capable of doing it, then do it. Warcraft III modding has never really been about profit and there are still people who play, so I don't see what's wrong with trying.
 
Level 19
Joined
Aug 8, 2007
Messages
2,765
If you are capable of doing it, then do it. Warcraft III modding has never really been about profit and there are still people who play, so I don't see what's wrong with trying.

Time.

The lack of testers.
 
Status
Not open for further replies.
Top