Most efficient way to run a big loop

[Ezekiel12]

Hey guys,

I want to get all abilities a unit has ingame. I dont think there is an easy way for it, so what I do at the moment is to loop through all printable characters of the standard Ascii table (96 chars) and just check with GetUnitAbilityLevel. This is quite an expensive calculation, so I want to do it as fast as possible.

So my question is what is the most efficient way to run through a very big loop? Currently my best method needs ~1m30s..

 

[TriggerHappy]

If the abilities are added manually you can just store which the unit has everytime you call UnitAddAbility.

The only other faster dynamic way I think would be if you stored the abilities in an array, then looped that.

How are you using ASCII to determine abilities? Their raw code?

 

[Ezekiel12]

If the abilities are added manually you can just store which the unit has everytime you call UnitAddAbility.

The only other faster dynamic way I think would be if you stored the abilities in an array, then looped that.

I should've made my description a bit less vague to prevent such answers. There are hidden abilities which are listed nowhere, I also want to find them, so those two proposals wont work.

How are you using ASCII to determine abilities? Their raw code?

Yes. Im looping right through the integer representations of all printable standard Ascii combinations. Im quite sure this can not be done faster, the only question is how to do the big loop (which hits the oplimit a lot of times ofc).

 

[TriggerHappy]

The only way I can think of is TriggerSleepAction(0)

 

[Nestharus]

TriggerEvaluate


It's what all of the resources that have big loops in them use.

 

[WaterKnight]

I should've made my description a bit less vague to prevent such answers. There are hidden abilities which are listed nowhere, I also want to find them, so those two proposals wont work.

Those special abilities, if you can catch them at all, are limited and react under certain conditions. So there is no problem to put them in a list and ask for them additionally, respectively verify their existence through the conditions.

You can as well write yourself a tool that browses the Units\AbilityData.slk and the war3map.w3a of the map to save all possible abilities. GMSI by gexxo should also have such functionality.

Do you plan to use this as a debugging method? Because otherwise it does not seem to make sense to consider the special, hardcoded abilities as abilities.

 

[Ezekiel12]

TriggerEvaluate

Thanks, I will try to wrap everything in triggerconditions are report the results.

Do you plan to use this as a debugging method?

Just testing stuff, yes, so performance does only matter because I dont want to wait 10 minutes .

You can as well write yourself a tool that browses the Units\AbilityData.slk

There are abilities which are not listed in the AbilityData.slk files, thats why I said listed nowhere

 

[WaterKnight]

There are abilities which are not listed in the AbilityData.slk files, thats why I said listed nowhere

What are those abilities? And how do they matter?

 

[Nestharus]

Don't wrap everything in trigger conditions or you're going to get freezes. You want to use a counter that goes until right before the thread is going to crash, then returns out of the triggercondition. Outside, it'll evaluate the trigger until the iterations are done.

so localcounter = internal steps before thread crash
externalcounter = total iterations to be done


so each loop that you have = 2 loops

 

[Ezekiel12]

What are those abilities? And how do they matter?

Helper abilities added to certain unit types or when abilities are used. Im doing it just for fun, so they might not matter at all

@Nestharus: Yes thanks, Im not going to let it crash

 

[defskull]

I want to get all abilities a unit has ingame.

You want to make a check ?

Instead of looping it one-by-one, you can use many different search algorithm such as Binary Search, etc.

 

[Nestharus]

Btw, there is this.. lol

http://www.hiveworkshop.com/forums/submissions-414/snippet-needs-work-get-learned-abilities-203539/

You'd get the same results either with your gigantic loop or by using that since ur checking GetUnitAbilityLevel. If you want to check for actual abilities that the unit can learn, then you'd have to iterate over all abilities within your map and then make the unit attempt to learn that ability and then check GetUnitAbilityLevel > 0.


It only needs work because people want it to use a lib by Bribe, and I haven't gotten around to that yet, otherwise it works perfectly, heh.

 

[Ezekiel12]

Instead of looping it one-by-one, you can use many different search algorithm such as Binary Search, etc.

Hm? Binary Search is useless here. Its like telling someone who has to look in thousand boxes to use binary search..

@Nestharus: I dont see how using Get Learned Abilities would increase performance.

I got two new questions:

1) With using the first TriggerEvaluate I noticed a better performance. Old runtime was ~2:25 min and new runtime is ~1:47 min. But why do code 1 and code 2 run in the same time (~1:47 min each) though code 1 uses more TriggerEvaluate's?
The two main differences are marked with green commented lines

2) Why does code 1 not hit the opLimit? Does TriggerEvaluate also create a new thread?

code 1

function Id2String takes integer i returns string
local string s=""
local integer start=0
local integer stop=3
local integer m

loop
    set m=ModuloInteger(i, 256)
    set s=SubString(udg_maxAlphabet, m-32, m-31)+s
    set start=start+1
    exitwhen start>stop
    set i=(i-m)/256
endloop

return s
endfunction

function countID_LoopD takes nothing returns nothing
set udg_iD=0
loop
        exitwhen udg_iD>94
        if ( GetUnitAbilityLevel( udg_currentlySelected, udg_i) < 1 ) then
        else
            set udg_abilitiesFound = udg_abilitiesFound + 1
            set udg_abilitiesFoundInfo[udg_abilitiesFound] =  GetObjectName(udg_i) + " (" + I2S(udg_i) + ") " + Id2String(udg_i)  
        endif
        
        set udg_i=udg_i+1 
        set udg_iD=udg_iD+1
    endloop
endfunction

function countID_LoopC takes nothing returns nothing
set udg_iC=0
loop
        exitwhen udg_iC>94
        call TriggerEvaluate(udg_trigC)
        
        set udg_i=udg_i+161
        set udg_iC=udg_iC+1
    endloop
endfunction

function countID_LoopB takes nothing returns nothing
set udg_iB=0
loop
        exitwhen udg_iB>94
//===========================================================================
        call TriggerEvaluate(udg_trigB)
//===========================================================================

        set udg_i=udg_i+41216
        set udg_iB=udg_iB+1
    endloop
endfunction

function countID_LoopA takes nothing returns nothing
    set udg_iA=0
    set udg_abilitiesFound = 0
    set udg_i=538976288
    loop
        exitwhen udg_iA>94
//===========================================================================
        call TriggerEvaluate(udg_trigA)
//===========================================================================

        set udg_i=udg_i+10551296
        set udg_iA=udg_iA+1
    endloop
endfunction

function InitTrig_countID takes nothing returns nothing
    set gg_trg_countID = CreateTrigger(  )
    set udg_trigA = CreateTrigger(  )
    set udg_trigB = CreateTrigger(  )
    set udg_trigC = CreateTrigger(  )
    call TriggerRegisterPlayerChatEvent( gg_trg_countID, Player(0), "countID", true )
    call TriggerAddCondition( gg_trg_countID, Condition(function countID_LoopA) )
    call TriggerAddCondition( udg_trigA, Condition(function countID_LoopB) )
    call TriggerAddCondition( udg_trigB, Condition(function countID_LoopC) )
    call TriggerAddCondition( udg_trigC, Condition(function countID_LoopD) )
endfunction

code 2

function Id2String takes integer i returns string
local string s=""
local integer start=0
local integer stop=3
local integer m

loop
    set m=ModuloInteger(i, 256)
    set s=SubString(udg_maxAlphabet, m-32, m-31)+s
    set start=start+1
    exitwhen start>stop
    set i=(i-m)/256
endloop

return s
endfunction

function countID_LoopD takes nothing returns nothing
set udg_iD=0
loop
        exitwhen udg_iD>94
          
        if ( GetUnitAbilityLevel( udg_currentlySelected, udg_i) < 1 ) then
        else
            set udg_abilitiesFound = udg_abilitiesFound + 1
            set udg_abilitiesFoundInfo[udg_abilitiesFound] =  GetObjectName(udg_i) + " (" + I2S(udg_i) + ") " + Id2String(udg_i)  
        endif
        
        set udg_i=udg_i+1 
        set udg_iD=udg_iD+1
    endloop
endfunction

function countID_LoopC takes nothing returns nothing
set udg_iC=0
loop
        exitwhen udg_iC>94
        call TriggerEvaluate(udg_trigC)
        
        set udg_i=udg_i+161
        set udg_iC=udg_iC+1
    endloop
endfunction

function countID_LoopB takes nothing returns nothing
set udg_iB=0
loop
        exitwhen udg_iB>94
//===========================================================================
        call ExecuteFunc("countID_LoopC")
//===========================================================================

        set udg_i=udg_i+41216
        set udg_iB=udg_iB+1
    endloop
endfunction

function countID_LoopA takes nothing returns nothing

    set udg_iA=0
    set udg_abilitiesFound = 0
    set udg_i=538976288
    loop
        exitwhen udg_iA>94
//===========================================================================
        call ExecuteFunc("countID_LoopB")
//===========================================================================
        set udg_i=udg_i+10551296
        set udg_iA=udg_iA+1
    endloop
endfunction

function InitTrig_countID takes nothing returns nothing
    set gg_trg_countID = CreateTrigger(  )
    set udg_trigC = CreateTrigger(  )
    call TriggerRegisterPlayerChatEvent( gg_trg_countID, Player(0), "countID", true )
    call TriggerAddCondition( gg_trg_countID, Condition(function countID_LoopA) )
    call TriggerAddCondition( udg_trigC, Condition(function countID_LoopD) )
endfunction

 

[WaterKnight]

Helper abilities added to certain unit types or when abilities are used. Im doing it just for fun, so they might not matter at all

Give an example please. If there are abilities not covered by the slk, I am interested as well and would like to investigate.

Why does code 1 not hit the opLimit? Does TriggerEvaluate also create a new thread?

Yes.

 

[edo494]

ExecuteFunc gives the running code reset OP limit
TriggerEvaluate gives each condition registered to the trigger its own OP limit reset or gives all conditions one OP limit, dont know, too lazy to test this one
TriggerExecute gives each action registered to the trigger its own OP limit reset or gives all actions one OP limit, dont know, too lazy to test this one
When timer expires it runs function with freshly restarted OP limit as well
ForGroup starts new OP limit per unit evaluated
ForForce starts new OP limit per player evaluated

cant think of anything more that refreshes OP limit

 

[Nestharus]

I want to get all abilities a unit has ingame. I dont think there is an easy way for it, so what I do at the moment is to loop through all printable characters of the standard Ascii table (96 chars) and just check with GetUnitAbilityLevel. This is quite an expensive calculation, so I want to do it as fast as possible.

Ohhhh, I see, so you also want to get abilities that were added via GetUnitAddAbility.

There's an easy way to do this

first, learn ability event, when a unit learns an ability, add it to the list of abilities for the unit

second, hook UnitAddAbility. When an ability is added to a unit, add that ability to the list.

Next, when you iterate over abilities, check to make sure that the ability level of each one is > 0. If it's not, remove the ability from the list. Boom, you have a list of all abilities on a given unit without having to do your huge loops.

GetLearnedAbilities only handles when a unit learns an ability. It does not handle UnitAddAbility. You could use GetLearnedAbilities + do your own list of UnitAddAbility.

 

[TriggerHappy]

Ohhhh, I see, so you also want to get abilities that were added via GetUnitAddAbility.

I don't think that's what he wants..

I would have suggested hooks long ago lol

 

[edo494]

Nes, and what if I give unit ability through object editor, neither of what you said would work

 

[Ezekiel12]

Give an example please. If there are abilities not covered by the slk, I am interested as well and would like to investigate.

For example the ability 'Anei' is a helper ability for shops and is automatically added when the unit has for example Select Hero ('Aneu'). It is that icon to choose the hero that is using the shop. When removing the ability the icon is removed too. Without looping through all combinations I would not have found that ability.

ExecuteFunc gives the running code reset OP limit
TriggerEvaluate gives each condition registered to the trigger its own OP limit reset or gives all conditions one OP limit, dont know, too lazy to test this one
TriggerExecute gives each action registered to the trigger its own OP limit reset or gives all actions one OP limit, dont know, too lazy to test this one
When timer expires it runs function with freshly restarted OP limit as well
ForGroup starts new OP limit per unit evaluated
ForForce starts new OP limit per player evaluated

cant think of anything more that refreshes OP limit

TriggerSleepAction comes to my mind and there are probably more.

@Nestharus: No vJass stuff will help me here as abilities added by the game itself can not be captured, right? See the upper example with 'Anei'..

Question number 1) is still open. Why does adding TriggerEvaluate in the most inner loop add performance while the other ones dont? Could it be because the outer loops are by far not called that often?

 

[edo494]

library Lib
    globals
        private integer i = 0
        private boolean sleep = true
    endglobals
    
    private module m
        private static method a takes nothing returns nothing
            loop
                set i = i + 1
                if i > 17000 and sleep then
                    call TriggerSleepAction(0)
                    set sleep = false
                endif
            endloop
        endmethod
        private static method b takes nothing returns nothing
            call BJDebugMsg(I2S(i))
        endmethod
        private static method onInit takes nothing returns nothing
            call TimerStart(CreateTimer(), 0.00, false, function thistype.a)
            call TimerStart(CreateTimer(), 1.00, false, function thistype.b)
        endmethod
    endmodule
    
    private struct s extends array
        implement m
    endstruct
endlibrary

wont print higher number then without TriggerSleepAction, so TriggerSleepAction does not reset OP limit

edit: I wont read that code until you fix the indention, sorry

 

[TriggerHappy]

wont print higher number then without TriggerSleepAction, so TriggerSleepAction does not reset OP limit

Because your test is invalid. Don't use waits in a timer callback.

library Lib initializer Init

    globals
        private constant integer ITERATIONS = 500000
    endglobals
    
    private function Init takes nothing returns nothing
        local integer i = 0
        local integer x = 0
        loop
            exitwhen i == ITERATIONS
            if (x == 8000) then
                call TriggerSleepAction(0) // not actually 0
                set x = 0
            endif
            set i = i + 1
            set x = x + 1
        endloop
        call BJDebugMsg(I2S(i))
    endfunction

endlibrary

EDIT: Here's an example of using TriggerEvaluate to avoid the OP limit, it can be a bit slow though (game will freeze while looping, if long enough loop).

library Lib initializer Init

    globals
        private constant integer ITERATIONS = 8000
        private constant integer MULTIPLIER = 100
        
        private integer x = 0
        private trigger OpTrig = CreateTrigger()
    endglobals
    
    private function Do takes nothing returns boolean
        local integer i = 0
        loop
            exitwhen i == ITERATIONS
            set x = x + 1
            set i = i + 1
        endloop
        return false
    endfunction
    
    private function Begin takes nothing returns nothing
        local integer i = 0
        loop
            exitwhen i == MULTIPLIER // ITERATIONS * MULTIPLIER operations
            call TriggerEvaluate(OpTrig)
            set i = i + 1
        endloop
        call BJDebugMsg(I2S(i) + " " + I2S(x))
    endfunction
    
    private function Init takes nothing returns nothing
        call TriggerAddCondition(OpTrig, function Do)
        call TimerStart(CreateTimer(), 0, false, function Begin)
    endfunction

endlibrary

 

[edo494]

well, my test was invalid but at least I learned that timer callback is equivalent to triggercondition callback heh

 

[WaterKnight]

For example the ability 'Anei' is a helper ability for shops and is automatically added when the unit has for example Select Hero ('Aneu'). It is that icon to choose the hero that is using the shop. When removing the ability the icon is removed too. Without looping through all combinations I would not have found that ability.

'Anei' is within Units\NeutralAbilityFunc.txt respectively Units\NeutralAbilityStrings.txt