• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • It's time for the first HD Modeling Contest of 2024. Join the theme discussion for Hive's HD Modeling Contest #6! Click here to post your idea!

[System] Automatic Memory Leak Destroyer

Status
Not open for further replies.
Level 11
Joined
Nov 4, 2007
Messages
337
Well, this is a system I wrote mainly for GUI users.
It is not very useful for JASS, but it makes coding in GUI so much easier.
And when you make maps, you dan't do everything in JASS.
You shold write the systems in vJass, but not everything.

This is a tool that automatically detects and kills memory leaks caused by these handle types:

Unitgroup, Effect, Location

which are mainly used in GUI.

How it works?
Using native hooking it responses to any function that uses groups, effects or locations (in Jass you don't need locations, don't use the BJ functions for effects and never destroy groups) and registers them. After a few seconds they get destroyed.
I could also make it work with Jass, but that would be senseless, because you don't want your groups in JASS to be destroyed automatically. Actually that would break many systems.

And when you don't want the data to be destroyed, you can simply call
ProtectHandle or ProtectVariable on them.
That will prevent the system catching the data that the variables are filled with.

So this catches only BJ functions!
It will not break JASS Systems!
It will just remove memory leaks the user didn't destroy on his own.

Here is the code:

JASS:
library MemoryLeakHelper initializer Init requires Table
// ==================================
// Give credits to Mr.Malte when used!
//===========================================================================
// Information: 
//==============
//
//  There are things called 'memory Leaks'. When you create a group or use a location
//  without destroying it, you will cause lag that stays in the whole game.
//  If you implement this library into your map it will automatically fix a big part
//  of those memory leaks, so reduce the lag extremely.
//  This should mainfully be used by GUI-users because the system is designed for them.
//
//  Of course no system can work totally automatically.
//  But there is only one thing you have to do in order to prevent bugs:
//  If you make groups or locations that have to be filled for more than CLEAN_UP_INTERVAL seconds
//  you have to save them with the code:
//
//  call ProtectHandle(XXX)
//
//  Where XXX is filled with your variable. Otherwise that variable gets destroyed
//  automatically. You can also fill in XXX with
//
//  GetLastCaughtHandle()
//
//  But if you save the things in a variable, I'd recommend to directly put the 
//  variable into the brackets. Note: GUI variables have the prefix 'udg_' in JASS.
//
//     This gives the 'Do Nothing' function in GUI a sense!
//     When you call DoNothing, all data that were caught by this system
//     will be destroyed in CLEAN_UP_INTERVAL seconds, ignoring how big
//     the number of caught handles is. This will not work, if the system is
//     already cleaning up.
//===========================================================================
// Implementation: 
//===============
//
//  The easiest thing is to directly implement this thing into your map, when you start making
//  it, so you don't have to look over your globals and use ProtectHandle on them.
//  These are the steps you have to do to clear the memory leaks:
//
//  1. Download a tool called 'JassNewGen', and unpack it somewhere. You need that
//     edit to use this tool. The 'JassNewGen' is used very commonly and offers other
//     nice features. You can find it at:
//     [url]http://www.wc3c.net/showthread.php?t=90999[/url]

//  2. Make a new trigger, and convert it to custom text. Insert everything
//     the library contains into that trigger.
//
//  3. Download a system called 'Table' from this link:
//     [url]http://www.wc3c.net/showthread.php?t=101246[/url]
//     Do the same installation stuff for 'Table' as for this system.
//
//  4. Save your map and enjoy :-)
//    
//  Note: Instead of doing 2 and 3 you can also copy and paste the folder 'MemoryLeakHelper' 
//        from the example map.
//===========================================================================
// How bad are memory leaks? 
//==========================
//  If you don't remove memory leaks, they suck memory:
//
//  Location:  0.361 kb
//  Group:     0.62 kb + 0.040 kb for each unit in the group.
//  Effect:    11.631 kb
// 
// Both, locations and groups are used very frequently. So when you don't fix those memory leaks,
// you will experience lag.

//        When you want to see, how useful this is for your map, implement it
//        and write 'call DisplayLeaks()' into a custom script that is fired when
//        they game ends.
//===========================================================================
// Changelog:
//===========
// v1.00 --- first version
// v1.01 --- able to detect special effects, too now.
// v1.02 --- made the system safer and reduced the number of variables to protect greatly.
// v1.03 --- Gave a sense to 'DoNothing'* GUI function and made the Pass Data part
//           more accurate, so the time until data get destroyed are much more explicit
//           now.
// v1.04 --- Added the very important constant MAX_LEAK_INSTANCES
//
// *if you don't want it to be hooked, comment line 350.
//===========================================================================
// FAQ:
// ====
//  1. Why don't you hook functions like GetLocationX or the ForGroup without BJ?
//
//      Answer: Well, in jass you would never destroy groups, rather have one global group
//              and clear/recycle it. But GUI always creates new groups with the functions.
//              So actually, jass groups don't have to be destroyed.
//              And special effects are mostly instantly destroyed and locations are never used.
//              So I don't want to endanger breaking jass systems, I rather make it for GUI, where it is
//              really useful and neccessary
//
//  2. Why should I protect my variables instead of killing my leaks?
//
//      Answer: In GUI, unitgroup effect and location variables are actually just used
//              for destroying stuff. It is rare that you really want to keep the groups.
//              So in fact, it is like one-hundred times less frequent that you want to keep
//              your data instead of destroying it.
//
//  3. I can't use jass. How can this system be useful for me?
//
//      Answer: This system works mainly automatically; You just have to use jass very rarely (and then a simple function).
//              The functions you need are:
//
//                  ProtectVariable(udg_###)
//                  GetLastCaughtHandle()
//
//              where ProtectVariable saves something you want to keep from getting destroyed. Just replace ### with your
//              variable name. NOTE: You don't have to protect the variable, when you want to keep the data inside less than
//              CLEAN_UP_INTERVAL seconds.
//
//              and where GetLastCaughtHandle responses to the handle* that was used lastly.
//              That can be for example the point where you just spawned a unit.
//              
//      * 'handle' means a specialeffect, a location or a unitgroup
//
//  4. If you give functions like 'DelayMMH', why don't you add functions like 'Disable/EnableMMH'?
//
//      Answer: Well, I want to protect the user. You do not need Disable/Enable functions.
//              To me the danger is too big, that you forget to activate it again or do
//              something like that. DelayMMH is totally enough if you don't want this system
//              to affect the code that comes next. Also that prevents, that there are
//              spells like 'Trojan Horses' that get too much access to this and can
//              change it's infrastructure.
//
//===========================================================================
// Functions:
//==========
// ProtectHandle              : Saves a handle from getting destroyed 
// ProtectVariable            : Same.
// DoNothing()                : Destroys all data caught by the system right now in X seconds.
// DelayMMD()                 : Stops the system working until the trigger ends/next wait *
//
// * This is as fast as an automatic memory leak destroyer can get. Why should
// you want to disable the system? Because it offers the possibilty to make things
// more efficient. I don't want to say, this is unefficient, because it is not.
// But this will destroy leaks like 10% slower.
//
//===========================================================================


    globals
        // The system fires when you do something that creates a leak.
        // The data that cause leak are saved in a variable then.
        // And every CLEAN_UP_INTERVAL seconds those data are destroyed.
        // This shouldn't be too high, or too low.
        private constant real CLEAN_UP_INTERVAL = 120.
        // If this is set to true, the system will work more slowly (but you wont notice)
        // and count, how much memory this system was able to save.
        // This value is display by the function DisplayLeaks() then.
        // WARNING: This sucks a lot of performance. I would ONLY use it when you want
        // to test, if this is useful for your map. Later set it to false.
        private constant boolean DISPLAY_SAVED_MEMORY = false
        // The Data are only cleaned up, when that many handles were caught
        private constant integer MIN_LEAK_NUMBER = 1750
        // How often are data passed to the destroyer?
        // Leaks stay for a random time between CLEAN_UP_INTERVAL and CLEAN_UP_INTERVAL+PASS_INTERVAL
        // in the game
        private constant real PASS_INTERVAL = 2.5
        // Memory leaks occur pretty frequently. When a leak is caught it is saved in
        // an array. But the array can't have more than MAX_LEAK_INSTANCES instances, so
        // if more than MAX_LEAK_INSTANCES memory leaks occur during a destroy interval,
        // the system fails.
        private constant integer MAX_LEAK_INSTANCES = 60000
    endglobals
    
    globals
        private HandleTable IndexData
        private HandleTable IsSaved
        
        //! textmacro MemoryLeakVars takes NAME, TYPE
        private integer Caught$NAME$Leaks = 0
        private $TYPE$ array $NAME$LeakData[MAX_LEAK_INSTANCES]
        private integer $NAME$DestroyCount = 0
        private $TYPE$ array $NAME$DestroyData[MAX_LEAK_INSTANCES]
        //! endtextmacro
        //! runtextmacro MemoryLeakVars("Location","location")
        //! runtextmacro MemoryLeakVars("Effect","effect")
        //! runtextmacro MemoryLeakVars("Group","group")
        
        
        private integer DestroyedLeaks = 0
        private integer CaughtLeaks = 0
        private integer DestroyedLeaksUser = 0
        private handle LastCaught
        private timer PassTimer = CreateTimer()
        private timer CleanTimer = CreateTimer()
        private timer DelayTimer = CreateTimer()
        private boolean IsDestroying = false
        private real SavedMemory = 0.
        private real LastCheckedGroupMemoryUsage = 0.
        private boolean DestroyThreadRunning = false
        private boolean Disabled = false
        
        // These values were found out in a big leak test by gekko.
        private constant real LOCATION_MEMORY_USAGE        = 0.361
        private constant real GROUP_MEMORY_USAGE           = 0.62
        private constant real GROUP_UNIT_MEMORY_USAGE      = 0.040
        private constant real EFFECT_MEMORY_USAGE          = 11.631
        private constant real REMOVED_EFFECT_MEMORY_USAGE  = 0.066
    endglobals
    
    // ======================================
    // ============= Basic Code =============
    // ======================================
    
    function GetLastCaughtHandle takes nothing returns handle
        return LastCaught
    endfunction
    
    function ProtectHandle takes handle h returns nothing
        set IsSaved[h] = 1
    endfunction
    
    function ProtectVariable takes handle h returns nothing
        set IsSaved[h] = 1
    endfunction

    private function EnableMMH takes nothing returns nothing
        set Disabled = false
    endfunction

    function DelayMMH takes nothing returns nothing
        set Disabled = true
        call TimerStart(DelayTimer,0.00,false,function EnableMMH)
    endfunction
    
    function DisplayLeaks takes nothing returns nothing
        call ClearTextMessages()
        call BJDebugMsg("======= MemoryLeakHelper =======")
        call BJDebugMsg("Destroyed Leaks: "+I2S(DestroyedLeaks))
        call BJDebugMsg("Destroyed Leaks by user: "+I2S(DestroyedLeaksUser))
        call BJDebugMsg("Percentage System: "+R2S(I2R(DestroyedLeaks)/I2R(DestroyedLeaks+DestroyedLeaksUser)*100.)+"%")
        call BJDebugMsg("Percentage User: "+R2S(I2R(DestroyedLeaksUser)/I2R(DestroyedLeaks+DestroyedLeaksUser)*100.)+"%")
        call BJDebugMsg("Leaks until next destroy: "+I2S(MIN_LEAK_NUMBER-CaughtLeaks))
        call BJDebugMsg(" === In Destroy Queue === ")
        call BJDebugMsg("   Group Leaks: "+I2S(GroupDestroyCount))
        call BJDebugMsg("   Location Leaks: "+I2S(LocationDestroyCount))
        call BJDebugMsg("   Effect Leaks: "+I2S(EffectDestroyCount))
        call BJDebugMsg(" === Not in Destroy Queue yet === ")
        call BJDebugMsg("   Group Leaks: "+I2S(CaughtGroupLeaks))
        call BJDebugMsg("   Location Leaks: "+I2S(CaughtLocationLeaks))
        call BJDebugMsg("   Effect Leaks: "+I2S(CaughtEffectLeaks))
        call BJDebugMsg("Time until next PassSequence: "+I2S(R2I(TimerGetRemaining(PassTimer)+0.5))+" seconds.")
        call BJDebugMsg(" ")
        if DISPLAY_SAVED_MEMORY then
            call BJDebugMsg("All in all the MemoryLeakHelper could release "+R2S(SavedMemory)+" kb of memory.")
        endif
        call BJDebugMsg("================================")
    endfunction
    
    private function GroupGetMemoryUsageEnum takes nothing returns nothing
        set LastCheckedGroupMemoryUsage = LastCheckedGroupMemoryUsage + GROUP_UNIT_MEMORY_USAGE
    endfunction
    
    function GroupGetMemoryUsage takes group g returns real
        set LastCheckedGroupMemoryUsage = 0.
        call ForGroup(g,function GroupGetMemoryUsageEnum)
        return LastCheckedGroupMemoryUsage + GROUP_MEMORY_USAGE
    endfunction
    
    //! textmacro ResponseOnLeak takes NAME, VALUE
    private function Catch$NAME$ takes $VALUE$ l returns nothing
        set LastCaught = l
        
        if Disabled then
            return
        elseif Caught$NAME$Leaks == MAX_LEAK_INSTANCES then
            debug call BJDebugMsg("MemoryLeakHelper: Failed to store leak because of size limitations")
            return
        endif
        
        if IndexData.exists(l) == false then
            //call BJDebugMsg("Caught $NAME$")
            set Caught$NAME$Leaks = Caught$NAME$Leaks + 1
            set $NAME$LeakData[Caught$NAME$Leaks] = l
            set IndexData[l] = Caught$NAME$Leaks
        endif
    endfunction
    
    private function AddTo$NAME$DestroyQueue takes $VALUE$ l returns nothing
        set $NAME$DestroyCount = $NAME$DestroyCount + 1
        set $NAME$DestroyData[$NAME$DestroyCount] = l
        set IndexData[l] = $NAME$DestroyCount*-1 // Put his to negative, so we know that this is used in the DestroyQueue now.
    endfunction
    
    private function Release$NAME$ takes $VALUE$ l returns nothing
        local integer index
        if IsDestroying == false and IndexData.exists(l) then
            set index = IndexData[l]
            // If this is true, the index wasn't put to a destroy queue yet.
            if index > 0 then
                set $NAME$LeakData[index] = $NAME$LeakData[Caught$NAME$Leaks]
                set Caught$NAME$Leaks = Caught$NAME$Leaks - 1
            else
                set index = index * -1
                set $NAME$DestroyData[index] = $NAME$DestroyData[$NAME$DestroyCount]
                set $NAME$DestroyCount = $NAME$DestroyCount - 1
            endif
            call IndexData.flush(l)
            set DestroyedLeaksUser = DestroyedLeaksUser + 1
        endif
    endfunction
    //! endtextmacro
    //! runtextmacro ResponseOnLeak("Location","location")
    //! runtextmacro ResponseOnLeak("Group","group")
    //! runtextmacro ResponseOnLeak("Effect","effect")
    
    private function DestroyMemoryLeaks takes nothing returns nothing
        set IsDestroying = true
        
        //call BJDebugMsg("DESTROYING Memory Leaks")
        //! textmacro DestroyLeaks takes NAME, DESTROYCALL, MEMORYUSAGE
        set DestroyedLeaks = DestroyedLeaks + $NAME$DestroyCount
        loop
            exitwhen $NAME$DestroyCount == 0
            
            if DISPLAY_SAVED_MEMORY then
                set SavedMemory = SavedMemory + $MEMORYUSAGE$
            endif
            
            call $DESTROYCALL$($NAME$DestroyData[$NAME$DestroyCount])
            call IndexData.flush($NAME$DestroyData[$NAME$DestroyCount])
            set $NAME$DestroyCount = $NAME$DestroyCount - 1
        endloop
        //! endtextmacro
        //! runtextmacro DestroyLeaks ("Group","DestroyGroup","GroupGetMemoryUsage(GroupDestroyData[GroupDestroyCount])")
        //! runtextmacro DestroyLeaks ("Location","RemoveLocation","LOCATION_MEMORY_USAGE")
        //! runtextmacro DestroyLeaks ("Effect","DestroyEffect","EFFECT_MEMORY_USAGE")
        
        set IsDestroying = false
        set DestroyThreadRunning = false
        //call StartPassTimer.execute() // Strange. This causes bugs sometimes and the function isn't called
        // This is slower, but safe.
        call ExecuteFunc("StartPassTimer")
    endfunction
    
    function StartDestroyThread takes nothing returns nothing
        if DestroyThreadRunning == false then
            set DestroyThreadRunning = true
            call TimerStart(CleanTimer,CLEAN_UP_INTERVAL,false,function DestroyMemoryLeaks)
            call PauseTimer(PassTimer)
        endif
    endfunction
    
    hook DoNothing StartDestroyThread
    
    // We want that the user doesn't have to protect too many variables, but all the variables that are filled longer
    // than CLEAN_UP_INTERVAL seconds. But what, when the handle thing is put into the destroy stack and the next destroy is
    // in 5 seconds, because the last one was 15 seconds ago? We can simply avoid something like that by using a 2-step-system
    // that goes sure, the handle is only destroyed when it passed the CLEAN_UP_INTERVAL twice.
    // Having two kinds of variables is simply easier and more efficient than having another variable that refers to
    // how many times the handle passed the timer; If it isn't passed/cleared in the Interval then, we can't loop
    // that easily through the data and we'd have to fix gaps later; That would suck a lot of performacne.
    private function PassMemoryLeaks takes nothing returns nothing
        //call BJDebugMsg("PassMemoryLeaks")
        //! textmacro PassLeaks takes NAME
        set CaughtLeaks = CaughtLeaks + Caught$NAME$Leaks
        //call BJDebugMsg("Caught $NAME$s: "+I2S(Caught$NAME$Leaks))
        loop
            exitwhen Caught$NAME$Leaks < 1
            if IsSaved.exists($NAME$LeakData[Caught$NAME$Leaks]) == false and $NAME$LeakData[Caught$NAME$Leaks] != null then
                call AddTo$NAME$DestroyQueue($NAME$LeakData[Caught$NAME$Leaks])
            endif
            set $NAME$LeakData[Caught$NAME$Leaks] = null
            set Caught$NAME$Leaks = Caught$NAME$Leaks - 1
        endloop
        //! endtextmacro
        //! runtextmacro PassLeaks ("Group")
        //! runtextmacro PassLeaks ("Location")
        //! runtextmacro PassLeaks ("Effect")
        
        if CaughtLeaks > MIN_LEAK_NUMBER then
            set CaughtLeaks = 0
            //call BJDebugMsg("Caught Leaks: "+I2S(MIN_LEAK_NUMBER))
            //call BJDebugMsg("Now start Destroy Timer")
            set DestroyThreadRunning = true
            call TimerStart(CleanTimer,CLEAN_UP_INTERVAL,false,function DestroyMemoryLeaks)
            // We have to pause this timer a bit; Otherwise it would break the CLEAN_UP_INTERVAL rule.
            call PauseTimer(PassTimer)
        endif
        
    endfunction
    
    // =================================
    // ============= Usage =============
    // =================================
    
    private function PP takes location source, real dist, real angle returns nothing
        call CatchLocation(source)
    endfunction
    
    private function CU takes integer count, integer unitId, player p, location l, real face returns nothing
        call CatchLocation(l)
    endfunction
    
    private function IPO takes unit k, string order, location l returns nothing
        call CatchLocation(l)
    endfunction
    
    private function SUP takes unit who, location l returns nothing
        call CatchLocation(l)
    endfunction
    
    private function SUF takes unit who, location l, real dur returns nothing
        call CatchLocation(l)
    endfunction
    
    private function GUR takes real radius, location l, boolexpr filter returns nothing
        call CatchLocation(l)
    endfunction
    
    private function CUF takes integer count, integer unitId, player whichPlayer, location loc, location lookAt returns nothing
        call CatchLocation(loc)
        call CatchLocation(lookAt)
    endfunction
    
    hook PolarProjectionBJ PP
    hook CreateNUnitsAtLoc CU
    hook CreateNUnitsAtLocFacingLocBJ CUF
    hook IssuePointOrderLocBJ IPO
    hook SetUnitPositionLoc SUP
    hook SetUnitFacingToFaceLocTimed SUF
    hook GetUnitsInRangeOfLocMatching GUR

    hook RemoveLocation ReleaseLocation
    
    
    private function FG takes group g, code callback returns nothing
        call CatchGroup(g)
    endfunction
    
    hook ForGroupBJ FG // :D This should catch all GUI usages for groups.
    hook GroupPickRandomUnit CatchGroup
    hook CountUnitsInGroup CatchGroup
    
    hook DestroyGroup ReleaseGroup
    
    private function ASETU takes string bla, widget d, string blu returns nothing
        // We can not catch THIS effect, but the effect that was created before.
        // So we can destroy all SpecialEffects excpet one.
        call CatchEffect(GetLastCreatedEffectBJ())
    endfunction
    
    private function ASE takes location where, string modelName returns nothing
        call CatchLocation(where)
        call CatchEffect(GetLastCreatedEffectBJ())
    endfunction

    hook AddSpecialEffectLocBJ ASE
    hook AddSpecialEffectTargetUnitBJ ASETU
    hook DestroyEffect ReleaseEffect
    hook DestroyEffectBJ ReleaseEffect
    
    // When I want to make the timer run the PassMemoryLeaks things, I have to use an .execute command which requires an extra func.
    function StartPassTimer takes nothing returns nothing
        //call BJDebugMsg("Restarting PassTimer")
        call TimerStart(PassTimer,PASS_INTERVAL,true,function PassMemoryLeaks)
    endfunction
    
    private function Init takes nothing returns nothing
        set IndexData = HandleTable.create()
        set IsSaved = HandleTable.create()
        call StartPassTimer()
    endfunction
endlibrary
 
Last edited:
Level 11
Joined
Nov 4, 2007
Messages
337
No.
I often make maps where I use vJass for the systems/spells and GUI for the rest of the script. Believe me, when you make a map, it's a cruel to kill all those memory leaks.
Also it can help many GUI users that experience lag, but don't want to look over their code and fix all the memory leaks.
I know, that would destroy the globals.
But thats a lot less to change, because:

Who would want to store groups ore locations or SpecialEffects in GUI as backrefence?
There is simply no reason to have such variables, EXCEPT
they are just TempVariables to destroy memory leaks.

And this does everything automatically, really, it saves a lot of time.
And almost everybody has got the JassNewGen.
And these people who don't can simply download it in 2 minutes.
 
Level 11
Joined
Nov 4, 2007
Messages
337
Code:
This is stupid. It basically encourages lax coding in GUI AND screws up your map every 20 seconds. Global groups, sfx or locations? Screwed.

That's wrong.
I've tested it: I implemented it into any map in my maps folder and it was able to catch more than 1500 memory leaks in 5 minutes.
And it didn't even lag a bit.
The destroy functions are very very fast.
 
Level 12
Joined
Feb 23, 2007
Messages
1,030
Code:
This is stupid. It basically encourages lax coding in GUI AND screws up your map every 20 seconds. Global groups, sfx or locations? Screwed.

That's wrong.
I've tested it: I implemented it into any map in my maps folder and it was able to catch more than 1500 memory leaks in 5 minutes.
And it didn't even lag a bit.
The destroy functions are very very fast.

1500 leaks? Seriously? You must suck at coding, which is contradictory to your work here. @_@
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,201
Hardly. 1 native call per object and directly after it has been used. Ofcourse the nulls still remain to prevent locals leaking handle indexes but that has nothing to do with this system so I am not counting it.

Generally for good triggeres fixing leaks is of little problem and is automatically built into the code they produce. I can see a use for this with GUI but still, this is nowhere near as efficent as just cleaning the objects up after you are done with them, or recycling objects like groups or locations which can then obviously never leak.
 
Level 11
Joined
Nov 4, 2007
Messages
337
No, you don't have to null handles that really exist on the map (like doodads or units).

Generally for good triggeres fixing leaks is of little problem and is automatically built into the code they produce. I can see a use for this with GUI but still, this is nowhere near as efficent as just cleaning the objects up after you are done with them, or recycling objects like groups or locations which can then obviously never leak.

Sorry, but this is simply not unifficient.
In fact it's is maybe two times slower than destroying the data directly.
But these two times are lilke nothing.
It's really a worse issue to have memory leaks.
Also there are many users that know how to fix memory leaks, but don't know that the location used for PolarProjections causes a memory leak, too.

Would you like to go through all your code and fix that issue?

also, automatically is the thing MY code does.
It clears leaks you forgot.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,201
Actually no, memory leaks do not make lag unless your PCs RAM becomes full as memory has no processor maintence time. Leaks will result in crashes though as WC3 can not exceed 1.5 GB or something in size.

The problem is leaks leak handle indexes as well, and for some reason the handle index system of WC3 sucks. The more handle indexes, the slower handle indexes are to allocate and so the longer it takes to create an object.

You still need to null all locals which represent objects which get removed, eg units and destructables as well if they are able to die (units) or get removed (destructables) otherwise their handle indexes never get recycled.

Simlar behaviour is present with globals but to a lesser degree as they can always be changed or nulled at a later time if the code permits it.
 
Level 11
Joined
Nov 4, 2007
Messages
337
Well, not only that.
I know, if you've got 1,5 GB of memory, you can fill it with about 129 000 special effects; But more realistic would be 90 000, since there are still other things that suck memory.

And now calculate:
You've got a spell that is called 'Bladestrom' and sucks units to the caster of Bladestorm.
So you move each unit in range every 0.01 seconds. The spell lasts 10 seconds.
There are 60 units near. You want to create a special effect on each of those units.

That's one unitgroup filled with 60 units, 120 locations (PolarProjection) and 60 special effects per 0.01 seconds.
Multiplied with 100 you suck

74,42 Megabytes of memory per second!
So all in all the spell sucks

744,2 Megabytes of memory.

This is a real example. I made such a spell a few years ago.
Ok, maybe 60 units is a bit much, but that could happen, and I set the interval
to 0.03, so only sucked 248 Megabytes per spell, but it is still a lot.

That makes you see, how quick your memory gets filled.

Also: Have you every created many locations at ony point? It doesn't lag, except when you go with your screen close to that point. So that makes me think, memory leaks do not only suck memory, they also cause lag.
 
Level 11
Joined
Nov 4, 2007
Messages
337
If you're going to get better at coding you HAVE to learn how to deal with those memory leaks anyway. Using this will only make you lazy.

That's not a point.
It just saves time. Or are you lazy when you want to download systems that are not yours?



And sorry if I wrote 'suck' memory all the time. I'm from Germany and my English is not so good.
Well, this is not useful for you, because you script in jass and not in GUI.
But I use both; and it's really painful to always change from the mouse to the keyboard/use custom script.

I've got an idea: you put this into a map of yours that contains a lot of GUI
and see how many percent of memory this releases.

What if a noob doesn't know what memory leaks are and finds it out then?
Also you can still remove the leaks on your own, this is more like a backup then.
 
Level 11
Joined
Apr 13, 2006
Messages
353
There is really no logical reason to be giving this guy flak. The fact of the matter is that most people on this site are GUI users, and this is probably the best thing to happen to them. As has been stated, a lot of people do not know about all leaks, or do not care to fix them. If you use JASS, or are a good GUI user (if such a thing can exist) then move along now...
 
Putting this thing in your map basically makes extra calls for each location / sfx / group you create. It'd slow already-inefficient GUI spells/systems right down! Imagine you had a system that looped every 0.01 seconds and used polar projection + a group loop and sfx in order to make a knockback. This system would just make it lag like hell!
 
Level 11
Joined
Apr 13, 2006
Messages
353
Imagine you had a system that looped every 0.01 seconds and used polar projection + a group loop and sfx in order to make a knockback. This system would just make it lag like hell!

I'll let the author defend his system, but in your example the game would already be lagging like hell. Who uses 0.01? 0.03 at the minimum. 1/0.03 = 33.3333 fps, more than movies today.
 
Level 21
Joined
Aug 21, 2005
Messages
3,699
Putting this thing in your map basically makes extra calls for each location / sfx / group you create. It'd slow already-inefficient GUI spells/systems right down! Imagine you had a system that looped every 0.01 seconds and used polar projection + a group loop and sfx in order to make a knockback. This system would just make it lag like hell!

I don't think garbage collection is designed to be used in systems, especially ones that should perform. But in any case I don't think this is going to cause a lot of lag for knockbacks
 
I don't think garbage collection is designed to be used in systems, especially ones that should perform. But in any case I don't think this is going to cause a lot of lag for knockbacks
If you put this in a map it affects all GUI triggers which use locations, sfx and groups + a few JASS BJ's. So, even though it's not designed for use in systems, if you use it in your map, it will automatically work in them anyway.
 
Level 11
Joined
Nov 4, 2007
Messages
337
Yes, and it will not destroy anything.
Do you know which dimension you're talking about, Element?
The speed losses are slower than miliseconds.
Maybe knockbacks will slow down for 1 Frame per second per 10 units (when they start each 0.01 seconds.

I can do speed tests if you want.
Really, GUI isn't that slow, it adds just one table usage per location/group/effect.
And Table uses hashtables now and IS really fast.
Maybe even as fast as your own hashes (ModuloInteger,ID,8190).

Ok.
Tests were done.
I moved X units every 0.01 seconds to a random place on the map.

With this system: Started lagging heavily at about 100 units
Other destroyment: Started lagging heavily at bout 115 units.

You see, that's not a big deal.

Also I updated the main post code.
 
I guess it just adds to the lag GUI already has.

Anyway, you could make it more optimized by making the catch thing a textmacro and just invoking it in each hook, rather calling a function, and by using the globals associated with GetLastCreateEffectBJ etc. directly, to avoid a function call.

Then it might be reduced to lagging with 100 running when it usually doesn't lag until it gets to about 110.
 
Level 11
Joined
Nov 4, 2007
Messages
337
No, function calls are incredibly fast.
Also I can't directly insert the catch thing into the code, because it needs the same parameters to be hooked.

No I think I could increase the speed by offering two variants:
one with table and another one which is much faster (like handle ID - 0x1000000)
 
Funtion calls in JASS are not incredibly fast (in terms of computer speed). They're actually quite slow. Why do you think you should avoid avoid BJs? It's slower than using the native.

And yes, you can and should make the catch functions part of the hook functions. You can easily make a textmacro take a variable name (such as "bj_lastCreatedEffect") which it uses in all it's stuff.

The speed gain from Table to H2I - 0x100000 is minimal. It's just pointless and it makes it prone to bugs.
 
Level 11
Joined
Nov 4, 2007
Messages
337
Why do you think you should avoid avoid BJs

Read in the system description under FAQ '1'

And yes, you can and should make the catch functions part of the hook functions. You can easily make a textmacro take a variable name (such as "bj_lastCreatedEffect") which it uses in all it's stuff.
But there are no 'LastCreatedGroup' and LastCreatedLocation stuffs.

The speed gain from Table to h2i is better than saving function calls that can't be saved.
The problem is, that I can't get what a hooked function returns.
So I can only hook functions that use locations, no ones that create them.

Well, Ill keep Table

edit: Wait, saving values in hashtables is 20 times slower than saving ones in arrays.
But the H2I method is only 3 times slower :)
 
Read in the system description under FAQ '1'
Ok, let me reformat that:

Q: Why do you you think you shouldn't use BJs?
A: Because it's lower than user the natives directly.

What I meant was, the reason you don't use BJs is because the extra function calls make them slower than just using the natives. This also applies to all custom functions.

But there are no 'LastCreatedGroup' and LastCreatedLocation stuffs.
For god's sake. USE THE LOCATION PARAMETER DIRECTLY IN THE TEXTMACRO!

edit: Wait, saving values in hashtables is 20 times slower than saving ones in arrays.
But the H2I method is only 3 times slower :)
Where did you get that from? If storing in hashtables was 20 times slower than arrays they'd be useless. It's 2 times slower.
 
Level 11
Joined
Nov 4, 2007
Messages
337
What I meant was, the reason you don't use BJs is because the extra function calls make them slower than just using the natives. This also applies to all custom functions.

But this would affect jass.
And GUI always uses BJs.

USE THE LOCATION PARAMETER DIRECTLY IN THE TEXTMACRO!
That's what I do, don't I?


Where did you get that from? If storing in hashtables was 20 times slower than arrays they'd be useless. It's 2 times slower.

I got that from speed tests.
 
SInce you can't understand me, I've done it for you. Look at my changes to the textmacros/hook functions.
JASS:
library MemoryLeakHelper initializer Init requires Table
// ==================================
// Give credits to Mr.Malte when used!
//===========================================================================
// Information: 
//==============
//
//  There are things called 'memory Leaks'. When you create a group or use a location
//  without destroying it, you will cause lag that stays in the whole game.
//  If you implement this library into your map it will automatically fix a big part
//  of those memory leaks, so reduce the lag extremely.
//  This should mainfully be used by GUI-users because the system is designed for them.
//
//  Of course no system can work totally automatically.
//  But there is only one thing you have to do in order to prevent bugs:
//  If you make groups or locations that are not just meant to be temporary,
//  you have to save them with the code:
//
//  call ProtectHandle(XXX)
//
//  Where XXX is filled with your variable. Otherwise that variable gets destroyed
//  automatically. You can also fill in XXX with
//
//  GetLastCaughtHandle()
//
//  But if you save the things in a variable, I'd recommend to directly put the 
//  variable into the brackets. Note: GUI variables have the prefix 'udg_' in JASS.
//  Note: This can't fix memory leaks caused by special effects.
//
//
//===========================================================================
// Implementation: 
//===============
//
//  The easiest thing is to directly implement this thing into your map, when you start making
//  it, so you don't have to look over your globals and use ProtectHandle on them.
//  These are the steps you have to do to clear the memory leaks:
//
//  1. Download a tool called 'JassNewGen', and unpack it somewhere. You need that
//     edit to use this tool. The 'JassNewGen' is used very commonly and offers other
//     nice features. You can find it at:
//     [url]http://www.wc3c.net/showthread.php?t=90999[/url]

//  2. Make a new trigger, and convert it to custom text. Insert everything
//     the library contains into that trigger.
//
//  3. Download a system called 'Table' from this link:
//     [url]http://www.wc3c.net/showthread.php?t=101246[/url]
//     Do the same installation stuff for 'Table' as for this system.
//
//  4. Save your map and enjoy :-)
//    
//  Note: Instead of doing 2 and 3 you can also copy and paste the folder 'MemoryLeakHelper' 
//        from the example map.
//===========================================================================
// How bad are memory leaks? 
//==========================
//  If you don't remove memory leaks, they suck memory:
//
//  Location:  0.361 kb
//  Group:     0.62 kb + 0.040 kb for each unit in the group.
//  Effect:    13.61 kb
// 
// Both, locations and groups are used very frequently. So when you don't fix those memory leaks,
// you will experience lag.

//        When you want to see, how useful this is for your map, implement it
//        and write 'call DisplayLeaks()' into a custom script that is fired when
//        they game ends.
//===========================================================================
// Changelog:
//===========
// v1.00 --- first version
// v1.01 --- able to detect special effects, too now.
//
//===========================================================================

    globals
        // The system fires when you do something that creates a leak.
        // The data that cause leak are saved in a variable then.
        // And every CLEAN_UP_INTERVAL seconds those data are destroyed.
        // This shouldn't be too high, or too low.
        private constant real CLEAN_UP_INTERVAL = 60.
        // If this is set to true, the system will work more slowly (but you wont notice)
        // and count, how much memory this system was able to save.
        // This value is display by the function DisplayLeaks() then.
        private constant boolean DISPLAY_SAVED_MEMORY = true
    endglobals
    
    globals
        private HandleTable IndexData
        private HandleTable IsSaved
        
        private integer CaughtLocationLeaks = 0
        private location array LocationLeakData
        
        private integer CaughtGroupLeaks = 0
        private group array GroupLeakData
        
        private integer CaughtEffectLeaks = 0
        private effect array EffectLeakData
        
        private integer DestroyedLeaks = 0
        private handle LastCaught
        private timer Cleaner = CreateTimer()
        private boolean IsDestroying = false
        private real SavedMemory = 0.
        private real LastCheckedGroupMemoryUsage = 0.
        
        // These values were found out in a big leak test by gekko.
        private constant real LOCATION_MEMORY_USAGE        = 0.361
        private constant real GROUP_MEMORY_USAGE           = 0.62
        private constant real GROUP_UNIT_MEMORY_USAGE      = 0.040
        private constant real EFFECT_MEMORY_USAGE          = 11.631
        private constant real REMOVED_EFFECT_MEMORY_USAGE  = 0.361
    endglobals
    
    // ======================================
    // ============= Basic Code =============
    // ======================================
    
    function GetLastCaughtHandle takes nothing returns handle
        return LastCaught
    endfunction
    
    function ProtectHandle takes handle h returns nothing
        set IsSaved[h] = 1
    endfunction
    
    function ProtectVariable takes handle h returns nothing
        set IsSaved[h] = 1
    endfunction
    
    function DisplayLeaks takes nothing returns nothing
        call ClearTextMessages()
        call BJDebugMsg("======= MemoryLeakHelper =======")
        call BJDebugMsg("Destroyed Leaks: "+I2S(DestroyedLeaks))
        call BJDebugMsg("Undestroyed Group Leaks: "+I2S(CaughtGroupLeaks))
        call BJDebugMsg("Undestroyed Location Leaks: "+I2S(CaughtLocationLeaks))
        if DISPLAY_SAVED_MEMORY then
            call BJDebugMsg("All in all the MemoryLeakHelper could release "+R2S(SavedMemory)+" kb of memory.")
        endif
        call BJDebugMsg("================================")
    endfunction
    
    private function GroupGetMemoryUsageEnum takes nothing returns nothing
        set LastCheckedGroupMemoryUsage = LastCheckedGroupMemoryUsage + GROUP_UNIT_MEMORY_USAGE
    endfunction
    
    function GroupGetMemoryUsage takes group g returns real
        set LastCheckedGroupMemoryUsage = 0.
        call ForGroup(g,function GroupGetMemoryUsageEnum)
        return LastCheckedGroupMemoryUsage + GROUP_MEMORY_USAGE
    endfunction
    
    //! textmacro MemoryLeakHelper__CatchLeak takes NAME, VAR
        set LastCaught = $VAR$
        if IndexData.exists($VAR$) == false then
            set Caught$NAME$Leaks = Caught$NAME$Leaks + 1
            set $NAME$LeakData[Caught$NAME$Leaks] = $VAR$
            set IndexData[$VAR$] = Caught$NAME$Leaks
        endif
    //! endtextmacro
    
    //! textmacro MemoryLeakHelper__ReleaseFunc takes NAME, VALUE
    private function Release$NAME$ takes $VALUE$ l returns nothing
        if IsDestroying == false then
            set $NAME$LeakData[IndexData[l]] = $NAME$LeakData[Caught$NAME$Leaks]
            set Caught$NAME$Leaks = Caught$NAME$Leaks - 1
            call IndexData.flush(l)
        endif
    endfunction
    //! endtextmacro
    //! runtextmacro MemoryLeakHelper__ReleaseFunc("Location","location")
    //! runtextmacro MemoryLeakHelper__ReleaseFunc("Group","group")
    //! runtextmacro MemoryLeakHelper__ReleaseFunc("Effect","effect")
    
    private function DestroyMemoryLeaks takes nothing returns nothing
        set DestroyedLeaks = DestroyedLeaks + CaughtLocationLeaks + CaughtGroupLeaks + CaughtEffectLeaks
        set IsDestroying = true
        //call BJDebugMsg("Destroy Leaks: "+I2S(DestroyedLeaks))
        loop
            exitwhen CaughtLocationLeaks == 0
            if IsSaved.exists(GroupLeakData[CaughtGroupLeaks]) == false then
                if DISPLAY_SAVED_MEMORY then
                    set SavedMemory = SavedMemory + LOCATION_MEMORY_USAGE
                endif
                
                call IndexData.flush(GroupLeakData[CaughtGroupLeaks])
                call RemoveLocation(LocationLeakData[CaughtLocationLeaks])
            endif
            set CaughtLocationLeaks = CaughtLocationLeaks - 1
        endloop
        
        loop
            exitwhen CaughtGroupLeaks == 0
            if IsSaved.exists(GroupLeakData[CaughtGroupLeaks]) == false then
            
                if DISPLAY_SAVED_MEMORY then
                    set SavedMemory = SavedMemory + GroupGetMemoryUsage(GroupLeakData[CaughtGroupLeaks])
                endif
                
                call IndexData.flush(GroupLeakData[CaughtGroupLeaks])
                call DestroyGroup(GroupLeakData[CaughtGroupLeaks])
            endif
            set CaughtGroupLeaks = CaughtGroupLeaks - 1
        endloop
        
        loop
            exitwhen CaughtEffectLeaks == 0
            if IsSaved.exists(EffectLeakData[CaughtEffectLeaks]) == false then
            
                if DISPLAY_SAVED_MEMORY then
                    set SavedMemory = SavedMemory + EFFECT_MEMORY_USAGE
                endif
                
                call IndexData.flush(EffectLeakData[CaughtEffectLeaks])
                call DestroyEffect(EffectLeakData[CaughtEffectLeaks])
            endif
            set CaughtEffectLeaks = CaughtEffectLeaks - 1
        endloop
        
        set IsDestroying = false
    endfunction
    
    // =================================
    // ============= Usage =============
    // =================================
    
    private function PP takes location source, real dist, real angle returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "source")
    endfunction
    
    private function CU takes integer count, integer unitId, player p, location l, real face returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "l")
    endfunction
    
    private function IPO takes unit k, string order, location l returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "l")
    endfunction
    
    private function SUP takes unit who, location l returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "l")
    endfunction
    
    private function SUF takes unit who, location l, real dur returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "l")
    endfunction
    
    private function GUR takes real radius, location l, boolexpr filter returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "l")
    endfunction
    
    private function CUF takes integer count, integer unitId, player whichPlayer, location loc, location lookAt returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "loc")
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "lookAt")
    endfunction
    
    //hook GetLocationX CatchLocation
    //hook GetLocationY CatchLocation
    hook PolarProjectionBJ PP
    hook CreateNUnitsAtLoc CU
    hook CreateNUnitsAtLocFacingLocBJ CUF
    hook IssuePointOrderLocBJ IPO
    hook SetUnitPositionLoc SUP
    hook SetUnitFacingToFaceLocTimed SUF
    hook GetUnitsInRangeOfLocMatching GUR

    hook RemoveLocation ReleaseLocation
    
    
    private function FG takes group g, code callback returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Group", "g")
    endfunction
    
    hook ForGroupBJ FG // :D This should catch all GUI usages for groups.
    //hook ForGroup FG
    hook GroupPickRandomUnit CatchGroup
    hook CountUnitsInGroup CatchGroup
    
    hook DestroyGroup ReleaseGroup
    
    private function ASETU takes string bla, widget d, string blu returns nothing
        // We can not catch THIS effect, but the effect that was created before.
        // So we can destroy all SpecialEffects excpet one.
        //! runtextmacro MemoryLeakHelper__CatchLeak("Effect", "bj_lastCreatedEffect")
    endfunction
    
    private function ASE takes location where, string modelName returns nothing
        //! runtextmacro MemoryLeakHelper__CatchLeak("Location", "where")
        //! runtextmacro MemoryLeakHelper__CatchLeak("Effect", "bj_lastCreatedEffect")
    endfunction

    hook AddSpecialEffectLocBJ ASE
    hook AddSpecialEffectTargetUnitBJ ASETU
    hook DestroyEffect ReleaseEffect
    
    private function Init takes nothing returns nothing
        set IndexData = HandleTable.create()
        set IsSaved = HandleTable.create()
        call TimerStart(Cleaner,CLEAN_UP_INTERVAL,true,function DestroyMemoryLeaks)
    endfunction
endlibrary
It's just like using textmacros as functions which automatically inline - it's faster.
 
Status
Not open for further replies.
Top