1. The long-awaited results for Concept Art Contest #11 have finally been released!
    Dismiss Notice
  2. Join Texturing Contest #30 now in a legendary battle of mythological creatures!
    Dismiss Notice
  3. The Aftermath has been revealed for the 19th Terraining Contest! Be sure to check out the Results and see what came out of it.
    Dismiss Notice
  4. Hivers united and created a bunch of 2v2 melee maps. Vote for the best in our Melee Mapping Contest #4 - Poll!
    Dismiss Notice
  5. Check out the Staff job openings thread.
    Dismiss Notice

Memory Leaks

Discussion in 'Trigger (GUI) Editor Tutorials' started by IcemanBo, Mar 6, 2015.

  1. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    Memory leaks


    << Read this tutorial for a quick lookup for most important actions. >>




    Introduction


    What is a memory leak? top


    Memory leak is if you can't refer to a certain part in the memory.
    The part in the memory is occupied, but the user has no possibility
    to read or to manipulate said part in memory anymore.
    You can say that leaked memory is unuseable and will hold up space for just nothing.​


    Why are memory leaks bad? top

    Single leaks won't affect anything but the dignity of a programer,
    but tons of leaks instead will heavily affect the runtime process and may
    lead to unsmooth execution, and in extreme cases to unuseability of the application.

    Relation to Warcraft III top

    Having memory leaks is a symptom for poor coding, so this of course also applies for our beloved Warcraft III maps.
    In case we don't take care of our code to be leakfree, we might face lag-spikes in our game sessions which just leads to less fun, and in extreme cases to a total freeze.

    Data Types

    Primitive Data Type top

    There exist primitive (or "native") data types. Primitive means it's not a set or combination of other data types, and that you can not break it down anymore:
    Code (vJASS):
    integer
    boolean
    real
    string
    code
    handle

    The
    handle
    type is a bit special. Handles work as pointer for complex data types,
    which means a handle can point to different data types in the end, instead of defining one constant and static data type.
    So a handle is just a complex data type's reference.

    Complex Data Type top

    Complex data types are something like "Effects" or "Groups" and are just different combinations of primitive/ native data types.
    In theory you can break Complex Data Types down to multiple primtive-only data types.
    handle
    type is used to point to Complex Data Types and complex types are also often called "Objects".​

    Enum Type top

    There are some other custom data types called "enums". Enums aren't real objects, they are only some sort of integer wrappers.
    That means, you can neither create an enum dynamicly, nor can you destroy one. Enums are just constantly defined integer sets that Blizzard declared already by default.
    (related Post by Aniki)

    List

    Code (vJASS):
    enums: [
        race,
        alliancetype,
        racepreference,
        gamestate, igamestate, fgamestate,
        playerstate,
        playerscore,
        playergameresult,
        unitstate,
        aidifficulty,
        eventid,
        gameevent,
        playerevent,
        playerunitevent,
        unitevent,
        limitop,
        widgetevent,
        dialogevent,
        unittype,
        gamespeed,
        gamedifficulty,
        gametype,
        mapflag,
        mapvisibility,
        mapsetting,
        mapdensity,
        mapcontrol,
        fogstate,
        playerslotstate,
        volumegroup,
        camerafield,
        playercolor,
        placement,
        startlocprio,
        raritycontrol,
        blendmode,
        texmapflags,
        effecttype,
        version,
        itemtype,
        attacktype,
        damagetype,
        weapontype,
        soundtype,
        pathingtype,
    ]
     



    Relation to Memory Leaks top

    Important to know is basicly only following:
    • Primitive types don't leak.
    • Enum types don't leak.
    • Complex Types/ Objects have potential to leak. Likewise their handle reference.

    For the ones who are interested in the question "why", I tried to phrase an explaination:
    Explaination
    When the JASS VirtuelMachine starts a new thread (starts to run certain functions) it allocates certain part of the RAM that will be used for local variables, object references, and some other minor required data. This allocated part of the RAM is called Stack.
    Then, when the virtuel thread has finished, the allocated memory of the Stack should be freed again as good as possible.
    For memory used for primitive data types, the Virtuel Machine will have no issues to re-use the memory again by it's own, so we don't have to bother with freeing it.
    But the JASS machine lacks a proper garbarge collector for complex data types, so this means we will need to free this occupied memory on our own, with help of certain JASS operations.
    We will learn how to deal with those JASS operations in the following topics.



    Object Leaks



    What is a object leak? top

    An object is basicly just a complex data type.
    Once you create an object you also have to destroy it properly after usage.
    If you don't destroy the object, but also can't refer to it it anymore, this is what is called a (object) leak. (dynamic objects)
    You don't have to destroy objects that you will use over and over again. (static objects)

    Static object: Dynamic object:
    A static object gets created on map initialization.
    It is an object you have access to all over the time.
    A dynamic object gets created at runtime.
    You lose access to it when it falls out of scope.
    So finaly, there is no need to destroy static objects, because you still may use them at some time.
    But "temporary" created objects need to be removed before you lose access to them.


    How to prevent object leaks? top

    So for each object you create you also have to use the appropriate remove function after usage.
    As said, objects can also be used static, so you will use them over and over again. You don't need to destroy these statics.
    But all dynamicly created ones, or objects you don't need anymore need to be removed properly.

    Examples to remove objects in GUI:

    • Destructible - Remove <Destructible> Object

    • Environment - Remove <Weathereffect>

    • Image - Destroy <Image>

    • Floating Text - Destroy <FloatingText>

    • Item - Remove <Item>

    • Leaderboard - Destroy <Leaderboard>

    • Blitz - Destroy <Lightning>

    • Multiboard - Destroy <Multiboard>

    • Unit - Remove <Unit> from the game


    ^These are just examples. There are even some more destroy functions that you can use with GUI.
    Note for SpecialEffects: If you directly destroy the effect after creation, it still will show the death animation properly!

    Sadly the GUI does not provide all remove functions we need to prevent leaks.
    Sometimes we have to use advantage of custom script in here. Don't worry it's always only one line.

    Locations

    • Set location = somewhere
    • ... Usage of location ...
    • Custom script: call RemoveLocation(udg_location)
    Unit Groups

    • Set Group = Pick every unit in (Playable Map Area)
    • ... Usage of Group ...
    • Custom script: call DestroyGroup(udg_Group)
    For groups you can even use an other method.
    If you set "bj_wantDestroyGroup" boolean = true via custom script, it will automatically destroy the next unit group creation, that is done via GUI.

    • Custom script: set bj_wantDestroyGroup = true
    • Unit Group - Pick every unit in (Playable Map Area) and do (Unit - Hide (Picked unit))
    Player Groups


    • Set PlayerGroup = (All enemies of Player 1 (Red))
    • ... Usage of force ...
    • Custom script: call DestroyForce(udg_PlayerGroup)
    Be warned! "AllPlayers" does not leak, it is a static and constant variable. Never destroy it!
    So following does not leak, and can be used just like that:
    • Player Group - Pick every player in (All players) and do (Actions)
      • Loop - Actions


    Technique of removal top

    There always comes up this very same question, so here we go.

    As you can see we use temporary variables to help us to refer to our objects for proper removal.
    Important is that when you use help variables that you remove/destroy the object before you re-assign the said variable.
    If you can't ensure the object removal before an re-assignment of the variable, you will leak.
    You will never be able again to refer the object that the variable has referenced before the re-assignment.

    If you found this confusing, don't worry. With an example it's actually very easy to understand:

    Good example: Bad Example:
    • Set loc = Location_1
    • Unit - Move Unit_1 to loc
    • Custom script: call RemoveLocation(udg_loc)
    • Set loc = Location_2 // <-- re-assignment
    • Unit - Move Unit_2 to loc
    • Custom script: call RemoveLocation(udg_loc)

    • Set loc = Location_1
    • Unit - Move Unit_1 to loc
    • Set loc = Location_2 // <-- re-assignment
    • Unit - Move Unit_2 to loc
    • Custom script: call RemoveLocation(udg_loc)


    In the "Bad example" you lost access to Location_1, because the variable was overwritten.
    Location_1 can't be removed anymore, so it will leak.


    Reference Leaks



    << First to say: Reference Leaks only affect JASS. >>
    << If you use GUI you can skip this chapter. >>​


    What is a reference? top


    If an object gets created it will automatically get an unique ID.
    This ID will be used as further reference to the object. (no matter what type of object)​



    How does a reference leak? top


    When an object gets removed, it's ID-reference is not really in use anymore.
    And in case the IDs are never recycled, an internal ID counter will permanently increase.
    The handle ID would be like for ever in use and take up space for absolutely no reason.
    The issue is, the game does not automatically recycle these ID references, if there are still variables (pointers) pointing to the removed object.
    So for not to leak the ID reference we have to ensure that the object is "freed" of all it's pointers when we remove/destroy it.

    But here the same principle applies as with object leaks. If your object is static and will never be removed from game (like
    player
    ),
    there obviosuly won't occur any reference ID leak, as you will always be able to refer to the very same object at any time.
    There is absolutely no reason to care about reference leaks for those statics.​


    How to prevent reference leaks? top


    Basically you have to
    null
    handle variables that referred to an object.
    Attention, you don't have to null all handle type objects, but only agents.
    The reason for this is that only agents do not handle to reuse their handle ID automatically.

    Example - JASS

    Code (vJASS):

    function foo takes nothing returns nothing
        local unit u = SomeUnit
        // do some actions
        ...
        set u = null
    endfunction



    Technique of removal top


    Sometimes there is some misunderstanding on when exactly the variable has to be nulled.
    For this you have to imagine the variable as pointer again that points on a part in memory.
    When you set the pointer to
    null
    , this will mean that it does not point to said part of memory anymore.
    This also means that you have to destroy the object first, before you null your pointer. Else you won't be ever be able again to destroy the object.

    If you found this confusing, don't worry. With an example it's actually very easy to understand:

    Good example: Bad Example:
    Code (vJASS):
    local group g = CreateGroup()
    call DestroyGroup(g)
    set g = null
    Code (vJASS):
    local group g = CreateGroup()
    set g = null
    call DestroyGroup(g)


    In the bad example the destroy function will fail, because the pointer does not point to the object anymore.


    Extra Note:
    You have to be careful with reference leaks in functions which return a localy created handle.
    You won't null the local variable before you return it's pointed handle, so technically seen, the local pointer
    can't be ever set to null again, as you will lose access to it after the function returns.
    To avoid this issue we can use global handle variables (only ones which extend agent are needed, explained here) to ensure further access to the variable.
    You may use your own global variables, or also ones by blizzard. Just be aware that you overwrite the current value of the variable and it technicaly might affect other aspects in other codes.

    Good example: Bad Example:
    Code (vJASS):
    function someUnitReturn
        bj_lastCreatedUnit = CreateUnit( ... )
        return bj_lastCreatedUnit
        // Hooray! We still have access to the global variable :)
    endfunction
    Code (vJASS):

    function someUnitReturn
        local unit u = CreateUnit( ... )
        return u
        set u = null // <- Will never run :(
    endfunction


    Difference between local and global top

    Quote from @PurgeandFire:​

    Each agent has a reference count associated with it. Whenever any variable (local or global) is assigned to an agent, the agent's reference count is incremented by 1. When that variable gets reassigned (either to a new value or null), the agent's reference count is decremented by 1. Once an agent's reference count becomes 0, its memory can be freed.

    Both local and global variables affect agents' reference counts; however, we tend to worry most about the local case. Globals are often reassigned throughout the map's lifetime, so you don't really need to worry about nulling them. Local variables, on the other hand, only live for as long as their function runs, and if they leave scope without being assigned to null, that agent's reference count will never be able to reach 0 (i.e. its memory can never be freed).


    Other stuff


    Things that always leak top


    There are some leaks that you can't prevent.

    • Terrain deformations in general. Including default abilities like shockwave. (reference leak)

    • A unit will always keep a tiny bit of memory, even when removed properly. 0.04 kb and unpreventable. Link


      • Camera - Pan camera as necessary for <Player to <Loc> over <Real> seconds
      ^Leaks a location. Furthermore this is a black listed function, because it can cause sync error. Use this instead: Link

      • Wait <real> game-time seconds
      ^Leaks a reference. But you can write your own polled wait function if needed.
    I named some important ones, but there are actually more BJs (Blizzard functions) that leak reference(s).
    I won't list them all here, but if you use JNGP you easily can look it up in the functions list if you want to get sure.​


    Miscellaneous top

    • If the black screen freezes after game-end it may be an indicator for huge amount of leaks. Don't worry if it's a only a few seconds - it might also be the normal memory usage your map needs.

    • Lags are not necessarily a proof for leaks. Highly inefficient coding may also result in lags for example.

    • An indicator for major leaks might be noticed in task manager. Just watch the memory usage of WC3 while running the map.
      If it's increasing very fast and
      continuously, you might check your triggers again. (code that runs periodicly or is called very often is most important)

    • As GUI handles certain object initialization by default, (for arrays index [0] and [1]), it does technically leak once if you create for example a unit group for the first time and so overwrite the initialy created group, without having destroyed it. Example creation:
      • Set Group = (Units owned by Player 1(Red))
      ^But this leak fires only once, as you would destroy your self-created groups each time after usage. So this is good to know, but has no real effect.

    • Here's a tool by Ralle that may help to find your leaks: JASS Checker


    Conclusion


    Always remove objects after usage, else you will have an object leak.
    Always null agent type variables, else you will have a reference leak.
    Object leak is the major leak, but the reference leak is still a leak so don't neglect it.

    That's it, thanks for reading! If you have anything to add or did not understand something, feel free to post. :csmile:
     
    Last edited: Dec 31, 2016
  2. LordDz

    LordDz

    Joined:
    May 11, 2007
    Messages:
    4,305
    Resources:
    0
    Resources:
    0
  3. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    "Things That Leak" is okay, but in my opinion the user could be teached some more general information about leaks.
    It is like an extension, but in tutorial section. There will be probably a link in "Things That Leak" that will link here.
    Also I have not really found anything about reference leaks on THW. These explainations might be new for most users.
     
  4. Quilnez

    Quilnez

    Joined:
    Oct 12, 2011
    Messages:
    3,195
    Resources:
    37
    Icons:
    2
    Tools:
    1
    Maps:
    7
    Spells:
    21
    Tutorials:
    2
    JASS:
    4
    Resources:
    37
    I have my own definition of leak, I think this is more general, including "memory leak" defined in other programming languages as well. I'm not an expert so don't trust me.

    Memory leak is where you create something (in this case agent) which is never been automatically destroyed by the system except if the program is ended (halt). Thus if you de-reference it without destroying, you will never be able access it anymore whereas it's still there. It consumes memory, but you have no access and can not remove it anymore except if the program (in this case, the map) is closed. So you can imagine how memory leaks make your map become heavier over times.

    Phew, I think wikipedia (Memory leak) define it better after all LOL. In short, memory leak is allocating memory space without deallocating that space later.

    Other information:
    Someone can indicate whether his/her map is badly leakful if the map needs some "freeze moment" when the game is closed, including restart, return to menu, or exit game. Perhaps you can add this.
     
    Last edited by a moderator: Jul 19, 2015
  5. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,419
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    It was decided through PM's that IcemanBo's guide would prove useful since it gives a deeper explanation of what is going on with leaks. In general, it is better to know the reason behind removing leaks than just knowing the functions to remove them.

    @IcemanBo: One small fix before it is ready for approval (didn't notice in my first skim):
    The handle ID leak only occurs with local variables. There is a minor leak associated with not nulling globals, but it is really insignificant (most likely just things associated with an objects reference count, which is static and probably takes up a few bytes at most). It is optional to remove in the case of globals. In locals, it is mandatory (at least for approval)--for all non-permanent agents.
     
  6. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    Okay, PurgeandFire (the wise) explained to me via PM why not nulling globals leaks a bit different as with locals.
    It is an other type of reference leak, which I did not know. But the point is, that it can be totaly ignored for realistic usage.
    Changed.
     
  7. Dat-C3

    Dat-C3

    Joined:
    Mar 15, 2012
    Messages:
    2,454
    Resources:
    10
    Models:
    1
    Maps:
    5
    Spells:
    3
    Tutorials:
    1
    Resources:
    10
    Doesn't creating new units/hero's in-game make permanent leaks that aren't cleanable? I know they do, but some likely won't believe me. Also you should make a note about shockwave as well war stomp. This is likely only for triggers/code, but technically its all code. Good tutorial. By the way you should always clean as many leaks as possible even when your map/game ends to prevent leaks from carrying over to your next game. Though what you can do to clear your memory is to restart WC3 entirely, close it then start it back up again.

    Edit: It's easy enough to test quickly if anyone needs to find out for themselves. Just clean all the leaks you know how too then continue to make units and remove them about 10,000 times for a noticeable number. Same with other stuff if you don't clean it, at least we get to clean some of the leaks I suppose. =)
     
  8. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    Yes, units will always use a bit of memory. Also added something from Dalvengyr's post.
    There might be some more things that always leak, I did not analyse each single function.
    There also might be some more interesting things to know that I could add to "Miscellaneous".
    I'm positive that we can gather most interesting stuff together with community's participation. :)
     
  9. edo494

    edo494

    Joined:
    Apr 16, 2012
    Messages:
    3,855
    Resources:
    5
    Spells:
    1
    JASS:
    4
    Resources:
    5
    since Jass is event driven, this is not true, because I can lose all references to a unit, but I can still get more, if the unit fires some event.

    Wiki definition is also not really that good, because all memory is deallocated when you end the program, or at worst, when you turn off computer, so technically you never create memory leaks by that definition.
     
  10. Zwiebelchen

    Zwiebelchen

    Joined:
    Sep 17, 2009
    Messages:
    6,784
    Resources:
    11
    Models:
    4
    Maps:
    1
    Spells:
    1
    Tutorials:
    1
    JASS:
    4
    Resources:
    11
    I think this tutorial is straight and to-the-point. The "Things that leak" sticky isn't as descriptive and tutorial-esque as this submission. I vote for approval.
     
  11. Flux

    Flux

    Joined:
    Feb 6, 2014
    Messages:
    2,334
    Resources:
    28
    Maps:
    1
    Spells:
    19
    Tutorials:
    2
    JASS:
    6
    Resources:
    28
    The in-depth details were useful. +rep
     
  12. BPower

    BPower

    Joined:
    Mar 18, 2012
    Messages:
    1,745
    Resources:
    21
    Spells:
    15
    Tutorials:
    1
    JASS:
    5
    Resources:
    21
    I think you should replace "a certain boolean = true"
    with "bj_wantDestroyGroup = true"

    It's a good tutorial
     
  13. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    ^Changed it. Also I came up with some more imporvements.
     
  14. blancfaye7

    blancfaye7

    Joined:
    May 21, 2014
    Messages:
    560
    Resources:
    1
    Maps:
    1
    Resources:
    1
    So even when a unit dies in-game (or should I say, finished decaying), it causes leaks? (I am not asking of how much it would leak, but an answer to that is appreciated as well.)

    BTW, best tutorial about leaks. Ever.
     
    Last edited: Apr 11, 2015
  15. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    Yes, units will permanently occupy a small part of memory, even after removed properply. (0.04kb iirc)
    Ty! :)
     
  16. blancfaye7

    blancfaye7

    Joined:
    May 21, 2014
    Messages:
    560
    Resources:
    1
    Maps:
    1
    Resources:
    1
    Oh my... That is indeed bad for my map :sad:
    Good thing I did not make any dummy units at all.
    But the spawns...

    Ah well... :( Thanks for the info!

    You should also add that this function:
    • Camera - Pan Camera as needed...
    also causes permanent leaks.
     
  17. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,054
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    Ah, right. This function anyway should not be used, and I will note it shortly, thanks.

    But well, 0.04 kb is not much. Don't think too much about it.
     
  18. danger_friendGR

    danger_friendGR

    Joined:
    Apr 30, 2013
    Messages:
    4
    Resources:
    0
    Resources:
    0
    Special

    Ye that's so helpful icemanbo is da best and he is SPECIAL!!!He knows better! :D
     
  19. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,419
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
  20. Flux

    Flux

    Joined:
    Feb 6, 2014
    Messages:
    2,334
    Resources:
    28
    Maps:
    1
    Spells:
    19
    Tutorials:
    2
    JASS:
    6
    Resources:
    28
    Since Unit groups are initialized at Map Initialization as Empty Group, that means the first time using
    • Set Group = (Units within Range of Location)

    will create a leak since ^that action overwrites the Empty group right?
    Code (vJASS):

    //Map Init
    set udg_Group = CreateGroup()

    //When using the GUI Pick Units the first time, the previously pointed unit group that was
    //created at Map Init leaks
    function GetUnitsInRangeOfLocMatching takes real radius, location whichLocation, boolexpr filter returns group
        local group g = CreateGroup()
        call GroupEnumUnitsInRangeOfLoc(g, whichLocation, radius, filter)
        call DestroyBoolExpr(filter)
        return g
    endfunction