HandleIndexing + Recycling.

[MapperMalte]

It's not good for warcraft.

I have already proven, that my system is safer and faster.

 

[Element of Water]

You haven't proven it's safer. And according to Deaod you also haven't proven it's faster.

 

[MapperMalte]

Mine is faster than yours.
Mine is as fast as PUI, sometimes faster.
How I said: PUI recycles each 0.035 seconds.
HandleIndexing recycles unit handles, when a unit decays (or you use the steel flavour).
===> If units don't die 3 or more times in a second, my system is faster.
Otherwise my system is as fast as PUI.

Ok. You don't need a test to know that this is safer.
You have created 100 000 handles.
Your system returns 149 corrupt Indexes!
(100000/8191)²
(Or 670, I am not sure)
Mine returns none corrupt Index.
And if the Max Handle ID isnt as big as 100 000, the user will receive an error message.

 

[Element of Water]

You apparently haven't tested it properly...

and what would you need 100000 handles for?

 

[MapperMalte]

Handles that were ever created.
If your game created that many handles...
I don't have to test it proberly, because it's clear.

Ok. I have made a new (very safe) flavour.

//  ________________________
// | How does this work?    |
// |________________________|

// - If you call NewIndex(handle), an integer for the Index is increased by 1.
//   If you call ReleaseIndex(handle) then, the integer is decreased by 1.
//   If the Integer is 0, the Index is released.
// - Handles you create Indexes for are stored in an array and removed until you release the Index.
// - When there is no unique Index left, it loops through the Handle Array, and the Indexes of destroyed handles are released.
// - When the Handle ID of your Handle goes over MAX_HANDLE_ID, it will create an Index, that may cause Overlay.
//   It will swap to a Modulo Hash
// - The system counts, how many times it has been used, how often any Limits were reached and.. A lot of stuff.
//   If you write -indexstats ingame, the system will give you a lot of information.
//   If you appear bugs and think, the bugs are the fault of HandleIndexing, you should write that.
//   The system will output bugs.

Name: Diamond Flavour.
==> Safer than Steel Flavour.
Name, because Diamond is harder than steel. Harder = Safer.




@Element of Water: Why shouldn't anybody want to have the also possible method thats safer but not slower?

Code:

<HandleIndexing Diamond>

//        **************************************************
//!       *         H A N D L E   I N D E X I N G          *
//        **************************************************

//| ----------------------------------------------- |
//|                 | ############################# |
//| HandleIndexing  | #### [ Diamond Flavour ] #### |
//|                 | ############################# |
//| ----------------------------------------------- |
//
//            Actually, normal HandleIndexing is the better choice.
//            But this is the safest HandleIndexing.
//            Actually, you can't do anything wrong.
//            But: This needs a lot more memory, and a lot more CPU.
//            If you don't call this periodically every 0.01 seconds or wahtever, you won't even
//            notice that.
//            
//            Differences:
//  ________________
// | Differences    |
// |________________|

// - You dont have to be careful when you Flush an Index with this Flavour.
// - Instead of calling Index(handle) to create AND get an Index, you have to
//   call GetIndex(handle) to return the Index of the unit and
//   call NewIndex(handle) to create an Index for the unit.
// - This is much slower than the Normal Handle Indexing.

//  ________________________
// | How does this work?    |
// |________________________|

// - If you call NewIndex(handle), an integer for the Index is increased by 1.
//   If you call ReleaseIndex(handle) then, the integer is decreased by 1.
//   If the Integer is 0, the Index is released.
// - Handles you create Indexes for are stored in an array and removed until you release the Index.
// - When there is no unique Index left, it loops through the Handle Array, and the Indexes of destroyed handles are released.
// - When the Handle ID of your Handle goes over MAX_HANDLE_ID, it will create an Index, that may cause Overlay.
//   It will swap to a Modulo Hash
// - The system counts, how many times it has been used, how often any Limits were reached and.. A lot of stuff.
//   If you write -indexstats ingame, the system will give you a lot of information.
//   If you appear bugs and think, the bugs are the fault of HandleIndexing, you should write that.
//   The system will output bugs.

//  ______________________________
// |           Speed              |
// |______________________________|
// |          18% slower          |
// | (than normal HandleIndexing) |
// |______________________________|

library SystemAffector
    globals
        integer array Affeted
    endglobals
    
    function Affect takes integer index returns nothing
        set Affeted[index] = Affeted[index] + 1
    endfunction
    
    function DeAffect takes integer index returns boolean
        set Affeted[index] = Affeted[index] - 1
        
        if (Affeted[index] <= 0 ) then
            return true
        endif
        return false
        
    endfunction

endlibrary


library HandleIndexingDiamond initializer Init requires SystemAffector
globals
    private constant integer MAX_HANDLE_ID = 300000
    private constant integer tosub = 0x100000
    private integer array StoredIDs[MAX_HANDLE_ID]
    private integer IDs = 0
    private integer gaps = 0
    private integer array Gaps
    
    private integer Usages = 0
    private integer HIT = 0
    private integer HitMAX = 0
    private handle array HandleData
    private trigger trig
endglobals

private function h2i takes handle h returns integer
    return h
    return 0
endfunction

function ReleaseIndex takes integer index returns nothing
    set gaps = gaps + 1
    set Gaps[gaps] = index
endfunction

function FlushIndex takes handle h returns integer
    local integer num = h2i(h)-tosub
    local integer re = 0
    local integer GetExtendedArrayOnce = StoredIDs[num] // INDEX of the handle. Range between 0 and 8190.
    
    if ( num > MAX_HANDLE_ID ) then
        set HitMAX = HitMAX + 1
        debug call BJDebugMsg("HandleIndexing: Increase MAX_HANDLE_ID!")
        set re = ModuloInteger(num-MAX_HANDLE_ID,8191)
        set HandleData[re] = null
        return re
    endif
    
    if ( DeAffect(GetExtendedArrayOnce) ) then

            call ReleaseIndex(GetExtendedArrayOnce)
            set re = GetExtendedArrayOnce
            set StoredIDs[num] = 0
            set HandleData[GetExtendedArrayOnce] = null
            
    endif
    
    return re
endfunction

private function FixIndexIssue takes nothing returns nothing
    local integer i = 0
        loop
            set i = i + 1
            exitwhen i > 8190
            if ( HandleData[i] == null ) then
                call ReleaseIndex(i)
            endif
        endloop
endfunction

function CreateIndex takes integer num returns integer
    //! Recycle Indexes
    if ( gaps > 0 ) then
        set StoredIDs[num] = Gaps[gaps]
        set gaps = gaps - 1
        return Gaps[gaps+1]
    else
    //! Create new Index
        set IDs = IDs + 1
        set StoredIDs[num] = IDs
            
                if IDs > 8190 then
                call BJDebugMsg("Handle Indexing; Created too many Indexes! Don't forget to call ReleaseIndex(handle). Trying to fix Issue.")
                call FixIndexIssue()
                set HIT = HIT + 1
                endif
                
        return IDs
    endif
endfunction

function GetIndex takes handle h returns integer
    local integer hashnum = h2i(h)-tosub
    set Usages = Usages + 1
    
    if ( hashnum > MAX_HANDLE_ID ) then
        set HitMAX = HitMAX + 1
        debug call BJDebugMsg("HandleIndexing: Increase MAX_HANDLE_ID!")
        return ModuloInteger(hashnum-MAX_HANDLE_ID,8191)
    endif
    
    return StoredIDs[hashnum]
endfunction

function NewIndex takes handle h returns integer
    local integer hashnum = h2i(h)-tosub
    local integer Return = 0
    set Usages = Usages + 1
    
    if ( hashnum > MAX_HANDLE_ID ) then
        debug call BJDebugMsg("HandleIndexing: Increase MAX_HANDLE_ID!")
        set Return = ModuloInteger(hashnum-MAX_HANDLE_ID,8191)
        set HandleData[Return] = h
        return Return
    endif
        
    if ( StoredIDs[hashnum] == 0 ) then
        set Return = CreateIndex(hashnum)
    else
        set Return = StoredIDs[hashnum]
    endif
    
    set HandleData[Return] = h
    call Affect(Return)
    return Return
endfunction

function DebugIndexing takes nothing returns nothing
    call BJDebugMsg("=======================")
    call BJDebugMsg(" ---Handle Indexing---")
    call BJDebugMsg("=======================")
    call BJDebugMsg("Recycler: "+I2S(gaps)+" Indexes that can be recycled.")
    call BJDebugMsg("Highest Index was: "+I2S(IDs))
    call BJDebugMsg("Indexes in use: "+I2S(IDs-gaps))
    call BJDebugMsg("HandleIndexing was used: "+I2S(Usages)+ " times")
    call BJDebugMsg("Hit Index Limit: "+I2S(HIT) + " times")
    call BJDebugMsg("Hit MAX_HANDLE_ID Limit: "+I2S(HitMAX) + " times")
    call BJDebugMsg("=======================")
endfunction

private function Init takes nothing returns nothing
    set trig = CreateTrigger()
    call TriggerRegisterPlayerChatEvent( trig, Player(0), "-indexstats", true )
    call TriggerAddAction( trig, function DebugIndexing )
endfunction

endlibrary

 

[Element of Water]

OK, I think neither of us are going to win this argument... let's just end it here.

 

[MapperMalte]

Okay.

 

[Akolyt0r]

max handle id 100.000 or 300.000 is just a bad joke ...REally..
Ive done a footman-wars like map ....(if you know that kind of maps, you know that they involve LOTS of units) ...and after 2 hours testing ...the highest handleid was 15.000 ...

if you have 100.000 (your example) or more handles, you failed ...And you should fix your other code, instead of just bypassing that problems and using a indexing-system which can handle that high handle-ids.

 

[MapperMalte]

On strategy maps...
Actually it's just a bonus feature that it can handle that high IDs.
If you think, your handle IDs are small enough, just decrease MAX_HANDLE_IDs to the value you want.
This is just meant to attach.
You can also make it look like an attachment system:

<Simulate Attaching>

// ===================================================================================================
//                                 --- Handle Struct Attachment ---
// ===================================================================================================
// Handle Struct Attachment based on HandleIndexing(Steel)                                           
// --- IMPORTANT:                                                                                    
//  If you want to use it, you have to call RemoveHandleInt and SetHandleInt for                     
//  the same amount of times!                                                                        
//  So every time, you attach an integer, you have to Remove it afterwards!                          
// Otherwise the System will be destroyed.                                                           
//! textmacro HSA takes NAME, TYPE                                                          
library HSA$NAME$ requires HandleIndexingSteel
globals
    private TYPE array $NAME$Data
endglobals

function SetHandleData$NAME$ takes handle h, $TYPE$ i returns nothing
    set $NAME$Data[NewIndex(h)] = i
endfunction

function GetHandleData$NAME$ takes handle h returns $TYPE$
    return $NAME$Data[GetIndex(h)]
endfunction

function RemoveHandleData$NAME$ takes handle h returns nothing
    set $NAME$Data[FlushIndex(h)] = 0
endfunction

endlibrary
//! endtextmacro

//! runtextmacro HSA("","integer")
// ===================================================================================================
//                              --- END OF Handle Struct Attachment ---                               
// ===================================================================================================

==> The code is inlined, so the macros don't create senseless functions.

Something I want to add for the Diamond Flavour:
It's actually just for noobs or for people that used the system wrong(ly?) for a long time and don't want to look over each Usage of the System again.

 

[Deaod]

a very easy solution would be to not call FlushIndex() at all. Just leave the integers attached.

Simple attaching to handles is already done by CSData. Wrap a textmacro around it, if you really need it that badly (or use HSAS).

cohadar's PUI or Rising_Dusk's UnitIndexingUtilities are the way to go. And im with Vexorian here: Any and all indexing of handles other than units or items is the wrong approach.

Fast attaching is done by: HSAS, CSData, ...
Safe attaching is done by: ABC, ...

For timers theres TimerUtils with it's various flavors.

 

[MapperMalte]

The Indexing is just a feature! You don't have to run textmacros then.
That it's for all handles is also just a feature. This will not inhibit you to use it for units.
If you use this instead of PUI or UnitIndexingUtils, you will not see any difference.

And: You said, for fast attaching: HSAS. I tested this against HSAS.
Result:
HSAS: 100% (0.013 seconds)
HandleIndexing: 93% (0.014 seconds)

==> Is that a huge difference?
Yes. You use 15 spells that use HSAS, have a handle Count of 35000.
===> 75 globals.

And you can easily swap from HSAS to this.


And: HSAS is very safe! But when the Indexes get too high, the system becomes a lot slower.
But this is very safe, too.

 

[MapperMalte]

UPDATE

I finalized HandleIndexing Steel, so w've got 5 Styles yet: Normal, Steel, Precomfort, Diamond and the New One, Gold.

HandleIndexing Gold is by far the fastest HandleIndexing and it has the same comfort as HandleIndexing Steel.

HandleIndexing Gold's NewIndex and FlushIndex functions are slightly slower than those of PUI (NewIndex, because of Recycling), but the Get function is incredibly fast.
When I test 250*250 times PUI against HandleIndexing Gold (1 Time NewIndex, 7 times GetIndex and 1 Time FlushIndex), HandleIndexing is 20% faster.

I also added a comfort function that allows better debugging (prevents double flushing).
Here is the code of HandleIndexing Gold:

//| --------------------------------------------- |
//|                 | ########################### |
//|  HandleIndexing | #### [  Gold Style   ] #### |
//|                 | ########################### |
//| --------------------------------------------- |
// By Mr.Malte

library SystemAffector
    // Just making this prettier, but the functions are inlined.
    globals
        public integer array Affeted
    endglobals

    function Affect takes integer index returns nothing
        set Affeted[index] = Affeted[index] + 1
    endfunction

    function DeAffect takes integer index returns nothing
        set Affeted[index] = Affeted[index] - 1
    endfunction

endlibrary


library HandleIndexingGold initializer Init requires SystemAffector
globals
    private constant integer MAX_HANDLE_ID = 8191
    private constant integer MIN_HANDLE_ID = 0x100000
    private integer array StoredIDs[MAX_HANDLE_ID]
    private integer IDs = 0
    private integer gapcount = 0
    private integer array Gaps
    private trigger trig
endglobals

private function h2i takes handle h returns integer
    return h
    return 0
endfunction

private function CreateIndex takes integer num returns integer
    // Test was started in NewIndex, but is ended here, since we don't want to cause open threads :-)
    //! Recycle Indexes
    if gapcount > 0 then
        set StoredIDs[num] = Gaps[gapcount]
        call Affect(Gaps[gapcount])
        set gapcount = gapcount - 1
        return Gaps[gapcount+1]
    else
    //! Create new Index
        set IDs = IDs + 1
        set StoredIDs[num] = IDs

                debug if IDs > 8190 then
                    debug call BJDebugMsg("Handle Indexing; Created too many Indexes! Don't forget to call ReleaseIndex(handle). Trying to fix Issue. May cause Errors or bugs!")
                    debug call BJDebugMsg("Swapping to non-injective HASH.")
                    debug return ModuloInteger(num-MAX_HANDLE_ID,8191)
                debug endif
                
        call Affect(IDs)
        return IDs
    endif
endfunction

function GetIndex takes handle h returns integer
    debug if h == null then
    debug    call BJDebugMsg("HandleIndexing | New Index: Made Index for null handle.")
    debug endif
    return StoredIDs[h2i(h)-MIN_HANDLE_ID]
endfunction

function NewIndex takes handle h returns integer
    local integer hashnum = h2i(h)-MIN_HANDLE_ID
    local integer Index = StoredIDs[hashnum]
    
        debug if h == null then
        debug    call BJDebugMsg("HandleIndexing | New Index: Made Index for null handle.")
        debug endif
        
    if Index == 0 then
        return CreateIndex(hashnum)
    else
        call Affect(Index)
        return Index
    endif
    return 0
endfunction

function FlushIndex takes handle h returns integer
    local integer num = h2i(h)-MIN_HANDLE_ID
    local integer Index = StoredIDs[num]
    local integer re = 0
    
    call DeAffect(Index)
    
    if SystemAffector_Affeted[Index] <= 0 then
    
            set gapcount = gapcount + 1
            set Gaps[gapcount] = Index
            set re = Index
            // set StoredIDs[num] = 0 // I dont know, if this is necessary...

    endif
    return re
endfunction

public function DebugIndexing takes nothing returns nothing
    call BJDebugMsg("=======================")
    call BJDebugMsg(" ---Handle Indexing---")
    call BJDebugMsg("=======================")
    call BJDebugMsg("Recycler: "+I2S(gapcount)+" Indexes that can be recycled.")
    call BJDebugMsg("Highest Index was: "+I2S(IDs))
    call BJDebugMsg("Indexes in use: "+I2S(IDs-gapcount))
    call BJDebugMsg("=======================")
endfunction

private function Init takes nothing returns nothing
    local trigger t = CreateTrigger()
    set trig = CreateTrigger()
    call TriggerRegisterPlayerChatEvent( trig, Player(0), "-indexstats", true )
    call TriggerAddAction( trig, function DebugIndexing )
    call TriggerRegisterPlayerChatEvent( trig, Player(0), "-Test", true )
endfunction
endlibrary

And the comfort library:

library HelpErrorHI requires HandleIndexingGold
    globals
        private constant boolean DEBUG = false
        System array UnitSystem
    endglobals
    
struct System
    string name
    handle indexed
    integer index
    
    static method GetFrom takes handle h returns System
        local System s = UnitSystem[GetIndex(h)]
        if h == null then
            call BJDebugMsg("HandleIndexing | System.GetFrom used on null handle.")
            return
        endif
        if s == 0 then
            call BJDebugMsg("HandleIndexing | System.GetFrom used on handle without system attached.")
            return
        endif
        if DEBUG then
            call BJDebugMsg("Got "+s.name+" properly.")
        endif
        return s 
    endmethod
    
    method Flush takes nothing returns integer
        call .destroy()
        return FlushIndex(.indexed)
    endmethod
    
    static method create takes handle toIndex, string name returns System
        local System s = System.allocate()
        set s.indexed = toIndex
        set s.index = NewIndex(toIndex)
        set s.name = name
        set UnitSystem[s.index] = s
        return s
    endmethod
endstruct

endlibrary

Edit: I made more tests. This is 30% faster than HSAS (!!!) ( 7 times get )
This is only faster than PUI if you get more than 2 times.