Accurate Benchmarking

[TriggerHappy]

Here's a test map showing how to accurately benchmark things, it may be useful or interesting to some who didn't already know about the StopWatch natives.

The attached map also includes other tests such as UnitAlive vs GetWidgetLife, TriggerExecute vs TriggerEvaluate, and some operator comparisons.

It requires RtC and patch 1.24.

Template

scope BenchmarkTemplate initializer Init

    globals
        private constant integer ITERATIONS = 8190
        private trigger t = CreateTrigger()
    endglobals
    
    private function Actions takes nothing returns nothing
        local integer curLoop
        local integer sw      
        local real array result
        
        call DisableTrigger(t)
        call BJDebugMsg("\n\n") 
        
        // TEST 1
        //===========================================================================
        set curLoop = 0
        set sw = StopWatchCreate()
        loop
            exitwhen curLoop == ITERATIONS
            // YOUR CODE HERE
            set curLoop = curLoop + 1
        endloop
        set result[0] = StopWatchMark(sw)
        call StopWatchDestroy(sw)
        //===========================================================================
        
        call BJDebugMsg("Test #1: " + I2S(curLoop) +" iterations took " + R2S(result[0]))
        call PolledWait(0.1)
        
        // TEST 2
        //===========================================================================
        set curLoop = 0
        set sw = StopWatchCreate()
        loop
            exitwhen curLoop == ITERATIONS
            // YOUR CODE HERE
            set curLoop = curLoop + 1
        endloop
        set result[1] = StopWatchMark(sw)
        call StopWatchDestroy(sw)
        //===========================================================================
        
        call BJDebugMsg("Test #2: " + I2S(curLoop) +" iterations took " + R2S(result[1]))
        
        if (result[0] < result[1]) then
            set result[2] = 100 - (result[0]/result[1] * 100)
            call BJDebugMsg("Test #1 was " + I2S(R2I(result[2])) + "% faster than Test #2")
        else
            set result[2] = 100 - (result[1]/result[0] * 100)
            call BJDebugMsg("Test #1 was " + I2S(R2I(result[2])) + "% slower than Test #2")
        endif
        
        call EnableTrigger(t)
    endfunction

    //===========================================================================
    private function Init takes nothing returns nothing
        call TriggerRegisterPlayerEvent(t, Player(0), EVENT_PLAYER_END_CINEMATIC)
        call TriggerAddAction(t, function Actions)
    endfunction

endscope

 

[Troll-Brain]

Have you "wc3mapoptimized" the test map ?
I mean it seems there is a difference with a script with "long" names and "short" names for variables/functions.
At least that was true in the past with the old fps stress test.
Don't ask me why, though, since jass is supposed to be converted in bytecode when the map is loaded, or something like that, no ?

If you didn't, the difference between arrays and hahstables should be bigger.

 

[TriggerHappy]

Have you "wc3mapoptimized" the test map ?

Not at all.

I mean it seems there is a difference with a script with "long" names and "short" names for variables/functions.

So if a variable has a longer name, it should take more time to process?

 

[Troll-Brain]

I suppose so, at least there was a difference with an fps stress test (more fps lost for the very same code, excepted longer or shorter names).
In fact empiric tests tend to show that jass is interpreted on the fly ...

 

[chobibo]

I mean it seems there is a difference with a script with "long" names and "short" names for variables/functions.

Yeah, I remember reading something about variable with longer identifiers being slower than those with minimum.

Here's the link, 2004

 

[TriggerHappy]

Well, in my hashtable test coincidentally the variable names have the same length.

hashtable hash
integer array data

 

[chobibo]

try using a single identifier.

 

[Troll-Brain]

Yes, the margin between arrays and hashtables could still be bigger if there is only one character length for names, because then this behavior will be at his lowest effect for jass efficiency.

And for a "real" test you should use 2-3 characters, because 1 is clearly not enough for any real map ...

 

[Magtheridon96]

It's not perfectly linear however.

If you were to map variable name length vs time, you find that it gets worse every 4 characters. Slope is constant over all the parts, but the line is not continuous.

 

[Malhorne]

Every 4 characters means that if my var is 4 characters will be faster than a var with 5 ?
If yes let's change every variable name ^^

 

[chobibo]

Just use the optimizer.

 

[Malhorne]

Ah ok ^^'

 

[Magtheridon96]

Every 4 characters means that if my var is 4 characters will be faster than a var with 5 ?
If yes let's change every variable name ^^

Yes.

But don't change your variable names.

Use variable names that express exactly what your variables are for.
Don't even use things like pos. Use FULL variable names for fuck's sake.

 

[Malhorne]

Ok don't kill me mag D:

 

[TriggerHappy]

Yeah readability is better in this case, don't worry about variable lengths the optimizer will take care of that for you.
The speed difference is extremely minor.

Anyway pos is fine in my opinion

 

[Troll-Brain]

It's not perfectly linear however.

If you were to map variable name length vs time, you find that it gets worse every 4 characters. Slope is constant over all the parts, but the line is not continuous.

Ok ok, i suppose you have tested your claim, because i'm not sure but i thought there was a difference beetween let's say 1 and 3 variable name length, last time i tried.
I remember that, because when i was thinking about a jass preprocessor/optimizer, i thought about a keyword for variables/functions which would lead to make this name 1 character length, or at least the lowest length possible, in the resulted jass code.
But meh it's very old so maybe i have never made a such test and just assumed more characters == slower ...

 

[Magtheridon96]

What I meant does not contradict what we have known for the last few years.

There speed difference between a variable with 3 characters and 4 characters is N, while the speed difference between a variable with 4 characters and 5 characters is M where M > N, but for 5 characters vs 6 characters, you have N again.

It happens because of some sort of Hashing mumbo jumbo that uses every 4 characters.