Accurate Benchmarking

Status
Not open for further replies.
Here's a test map showing how to accurately benchmark things, it may be useful or interesting to some who didn't already know about the StopWatch natives.

The attached map also includes other tests such as UnitAlive vs GetWidgetLife, TriggerExecute vs TriggerEvaluate, and some operator comparisons.

It requires RtC and patch 1.24.


JASS:
scope BenchmarkTemplate initializer Init

    globals
        private constant integer ITERATIONS = 8190
        private trigger t = CreateTrigger()
    endglobals
    
    private function Actions takes nothing returns nothing
        local integer curLoop
        local integer sw      
        local real array result
        
        call DisableTrigger(t)
        call BJDebugMsg("\n\n") 
        
        // TEST 1
        //===========================================================================
        set curLoop = 0
        set sw = StopWatchCreate()
        loop
            exitwhen curLoop == ITERATIONS
            // YOUR CODE HERE
            set curLoop = curLoop + 1
        endloop
        set result[0] = StopWatchMark(sw)
        call StopWatchDestroy(sw)
        //===========================================================================
        
        call BJDebugMsg("Test #1: " + I2S(curLoop) +" iterations took " + R2S(result[0]))
        call PolledWait(0.1)
        
        // TEST 2
        //===========================================================================
        set curLoop = 0
        set sw = StopWatchCreate()
        loop
            exitwhen curLoop == ITERATIONS
            // YOUR CODE HERE
            set curLoop = curLoop + 1
        endloop
        set result[1] = StopWatchMark(sw)
        call StopWatchDestroy(sw)
        //===========================================================================
        
        call BJDebugMsg("Test #2: " + I2S(curLoop) +" iterations took " + R2S(result[1]))
        
        if (result[0] < result[1]) then
            set result[2] = 100 - (result[0]/result[1] * 100)
            call BJDebugMsg("Test #1 was " + I2S(R2I(result[2])) + "% faster than Test #2")
        else
            set result[2] = 100 - (result[1]/result[0] * 100)
            call BJDebugMsg("Test #1 was " + I2S(R2I(result[2])) + "% slower than Test #2")
        endif
        
        call EnableTrigger(t)
    endfunction

    //===========================================================================
    private function Init takes nothing returns nothing
        call TriggerRegisterPlayerEvent(t, Player(0), EVENT_PLAYER_END_CINEMATIC)
        call TriggerAddAction(t, function Actions)
    endfunction

endscope


attachment.php
 

Attachments

  • Untitled.png
    Untitled.png
    358.2 KB · Views: 370
  • benchmark.w3x
    50 KB · Views: 222
Level 17
Joined
Apr 27, 2008
Messages
2,455
Have you "wc3mapoptimized" the test map ?
I mean it seems there is a difference with a script with "long" names and "short" names for variables/functions.
At least that was true in the past with the old fps stress test.
Don't ask me why, though, since jass is supposed to be converted in bytecode when the map is loaded, or something like that, no ?

If you didn't, the difference between arrays and hahstables should be bigger.
 
Level 22
Joined
Sep 24, 2005
Messages
4,821
I mean it seems there is a difference with a script with "long" names and "short" names for variables/functions.
Yeah, I remember reading something about variable with longer identifiers being slower than those with minimum.

Here's the link, 2004
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Yes, the margin between arrays and hashtables could still be bigger if there is only one character length for names, because then this behavior will be at his lowest effect for jass efficiency.

And for a "real" test you should use 2-3 characters, because 1 is clearly not enough for any real map ...
 
Every 4 characters means that if my var is 4 characters will be faster than a var with 5 ?
If yes let's change every variable name ^^

Yes.

But don't change your variable names.

Use variable names that express exactly what your variables are for.
Don't even use things like pos. Use FULL variable names for fuck's sake.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
It's not perfectly linear however.

If you were to map variable name length vs time, you find that it gets worse every 4 characters. Slope is constant over all the parts, but the line is not continuous.

Ok ok, i suppose you have tested your claim, because i'm not sure but i thought there was a difference beetween let's say 1 and 3 variable name length, last time i tried.
I remember that, because when i was thinking about a jass preprocessor/optimizer, i thought about a keyword for variables/functions which would lead to make this name 1 character length, or at least the lowest length possible, in the resulted jass code.
But meh it's very old so maybe i have never made a such test and just assumed more characters == slower ...
 
What I meant does not contradict what we have known for the last few years.

There speed difference between a variable with 3 characters and 4 characters is N, while the speed difference between a variable with 4 characters and 5 characters is M where M > N, but for 5 characters vs 6 characters, you have N again.

It happens because of some sort of Hashing mumbo jumbo that uses every 4 characters.
 
Status
Not open for further replies.
Top