1. Updated Resource Submission Rules: All model & skin resource submissions must now include an in-game screenshot. This is to help speed up the moderation process and to show how the model and/or texture looks like from the in-game camera.
    Dismiss Notice
  2. DID YOU KNOW - That you can unlock new rank icons by posting on the forums or winning contests? Click here to customize your rank or read our User Rank Policy to see a list of ranks that you can unlock. Have you won a contest and still havn't received your rank award? Then please contact the administration.
    Dismiss Notice
  3. The Lich King demands your service! We've reached the 19th edition of the Icon Contest. Come along and make some chilling servants for the one true king.
    Dismiss Notice
  4. The 4th SFX Contest has started. Be sure to participate and have a fun factor in it.
    Dismiss Notice
  5. The poll for the 21st Terraining Contest is LIVE. Be sure to check out the entries and vote for one.
    Dismiss Notice
  6. The results are out! Check them out.
    Dismiss Notice
  7. Don’t forget to sign up for the Hive Cup. There’s a 555 EUR prize pool. Sign up now!
    Dismiss Notice
  8. The Hive Workshop Cup contest results have been announced! See the maps that'll be featured in the Hive Workshop Cup tournament!
    Dismiss Notice
  9. Check out the Staff job openings thread.
    Dismiss Notice
Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

[Lua]Obliterate all GUI leaks with 1 trigger!

Discussion in 'The Lab' started by Dr Super Good, Jun 5, 2019.

  1. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Imagine a world where GUI triggers do not need a mess of temporary variables to hold locations. Or a world where the annoying custom script lines like "RemoveLocation" do not exist. A world where we could leak as much as we want without any performance consequences.

    Well imagine no more GUI users. Thanks to the power of patch 1.31 with Lua that world is real!

    Behold a single Lua trigger solution which will fix all force, group and location leaks. No custom script calls required, it works fully automatically as if by witchcraft!
    Code (Text):
    GC = {}
    GC.Table = {}
    setmetatable(GC.Table, {__mode = "k"})
    GC.Native = {}
    GC.Type = {}
    GC.Type.__index = GC.Type
    GC.Debug = true

    function GC.Type:new(name, remove, constructors)
        local type = {}
        setmetatable(type, self)

        type.name = name
        type.remove = remove
        type.constructors = constructors
        type:register()
        return type
    end

    function GC.Type:printstats()
        if self.stats then
            print("Statistics for " .. self.name .. ":")
            print("Automatically freed: " .. self.stats.autofree)
            print("Explicitly freed: " .. self.stats.explicitfree)
            print("Currently alive: " .. self.stats.alive)
            print("Bad free calls: " .. self.stats.badfree)
            print("Bad constructor calls: " .. self.stats.badconstruct)
            print("Free calls on untracked: " .. self.stats.unknownfree)
        else
            print("Can only print statistics when GC is running in debug mode.")
        end
    end

    function GC.Type:register()
        local debug = GC.Debug

        if not GC.Native[self.remove] then
            GC.Native[self.remove] = _ENV[self.remove]
        end

        local removefunc = GC.Native[self.remove]

        if debug then
            if not GC.FreedTable then
                GC.FreedTable = {}
                setmetatable(GC.FreedTable, {__mode = "k"})
            end

            self.stats = {
                autofree = 0,
                explicitfree = 0,
                alive = 0,
                badfree = 0,
                badconstruct = 0,
                unknownfree = 0
            }
        end

        local collectfunc
        if not debug then
            collectfunc = function(obj)
                removefunc(obj[1])
            end
        else
            collectfunc = function(obj)
                removefunc(obj[1])
                self.stats.autofree = self.stats.autofree + 1
                self.stats.alive = self.stats.alive - 1
            end
        end

        local collectmetatable = {
            __gc = collectfunc
        }

        if not debug then
            _ENV[self.remove] = function(obj)
                local table = GC.Table[obj]
                if table then
                    removefunc(obj)
                    setmetatable(table, nil)
                    GC.Table[obj] = nil
                end
            end
        else
            _ENV[self.remove] = function(obj)
                local table = GC.Table[obj]
                if table then
                    removefunc(obj)
                    setmetatable(table, nil)
                    GC.Table[obj] = nil
                    GC.FreedTable[obj] = true
                    self.stats.explicitfree = self.stats.explicitfree + 1
                    self.stats.alive = self.stats.alive - 1
                else
                    if not obj or GC.FreedTable[obj] then
                        if obj then
                            print("Tried to free an already freed object: " .. obj)
                        else
                            print("Tried to free nil of type: " .. self.name)
                        end
                       
                        self.stats.badfree = self.stats.badfree + 1
                    else
                        self.stats.unknownfree = self.stats.unknownfree + 1
                    end

                end
            end
        end

        local registerfunc
        if not debug then
            registerfunc = function(obj)
                local table = {obj}
                setmetatable(table, collectmetatable)
                GC.Table[obj] = table
            end
        else
            registerfunc = function(obj)
                local table = {obj}
                setmetatable(table, collectmetatable)
                GC.Table[obj] = table
                self.stats.alive = self.stats.alive + 1
            end
        end

        for i, v in ipairs(self.constructors) do
            if not GC.Native[v] then
                GC.Native[v] = _ENV[v]
            end

            local constructorfunc = GC.Native[v]

            if not debug then
                _ENV[v] = function(...)
                    local result = constructorfunc(...)
                    if result then
                       registerfunc(result)
                    end
                    return result
                end
            else
                _ENV[v] = function(...)
                    local result = constructorfunc(...)
                    if result then
                       registerfunc(result)
                    else
                        print("Constructor call returned nil: " .. v)
                        self.stats.badconstruct = self.stats.badconstruct + 1
                    end
                    return result
                end
            end
        end
    end

    GC.Location = GC.Type:new("location", "RemoveLocation",
        {"BlzGetTriggerPlayerMousePosition",
            "CameraSetupGetDestPositionLoc",
            "GetCameraEyePositionLoc",
            "GetCameraTargetPositionLoc",
            "GetOrderPointLoc",
            "GetSpellTargetLoc",
            "GetStartLocationLoc",
            "GetUnitLoc",
            "GetUnitRallyPoint",
            "Location"
        })

    GC.Group = GC.Type:new("group", "DestroyGroup",
        {"CreateGroup"
        })

    GC.Force = GC.Type:new("force", "DestroyForce",
        {"CreateForce"
        })
    Attached is a demo map to show its usage, and allow one to experiment with it. Press Esc to get technical information such as how many leaks were caught.

    Proof that it works is easy. Duplicate the actions of the Leaky Demo Trigger 20 or 40 times and test the map. Thanks to the Lua based garbage collector the Warcraft III application memory usage will not really increase. Now if one disables the Garbage Collector trigger and tests then the Warcraft III application memory usage will increase at the rate of 10s of megabytes per second.

    From a technical point of view it works by replacing all references to constructor and destructor functions for force, group and location within the Lua virtual machine to proxies. These proxies register or deregister an object for automatic garbage collection. Automatic garbage collection is implemented using the properties of a weak keyed map to map an object to a table holding the object which uses the garbage collected metamethod to destroy the object it wraps. When all references to a location are lost, the properties of the weak keyed map allow the wrapping table to be garbage collected which then runs the clean up function.

    It is worth noting that this approach only works if the Lua garbage collector ever runs. Since the object memory itself is not factored in as part of the Lua heap size it is possible that the default Lua garbage collector settings do not run the collector frequently enough to prevent leak related performance issues. To prevent this one might have to fine tune the Lua garbage collector settings to encourage it to run more frequently. This is done in the demo map in the Lua Test trigger. In theory this could be tuned on a per map basis as required.

    WARNING: Garbage collection is run locally for each client. Local object destruction will cause clients to go out of sync. The snippet below can be used to synchronize garbage collection at the cost of disabling incremental collection and potentially causing stutter with complex maps
    Code (Text):
    collectgarbage("stop")

    nsgc = {
    trigger = CreateTrigger(),
    func = collectgarbage}

    nsgc.event = TriggerRegisterTimerEvent(nsgc.trigger, 10, true)
    nsgc.action = TriggerAddAction(nsgc.trigger, nsgc.func)
    One can turn off statistic logging and warning messages by changing commenting or removing the line containing GC.Debug = true. This mode should offer a minor performance improvement and intended for non-development versions of maps. This mode cannot be toggled dynamically, it is effectively a compile time flag.
     

    Attached Files:

    Last edited: Aug 10, 2019
  2. Devalut

    Devalut

    Joined:
    Feb 9, 2009
    Messages:
    795
    Resources:
    2
    Spells:
    2
    Resources:
    2
    Sweet Merciful...
     
  3. pOke

    pOke

    Joined:
    Mar 24, 2013
    Messages:
    1,101
    Resources:
    1
    Maps:
    1
    Resources:
    1
    I don't understand lua or what is happening in the code.

    BUT..the concept sounds nice, and hopefully with the likely new crop of GUI coders coming soon with reforged this will be a nice tool.
     
  4. JFA

    JFA

    Joined:
    Jun 2, 2009
    Messages:
    437
    Resources:
    3
    Maps:
    3
    Resources:
    3
    Sounds amazing. But is someone tested in multiplayer?
     
  5. Wadjet

    Wadjet

    Joined:
    Sep 14, 2009
    Messages:
    243
    Resources:
    6
    Skins:
    1
    Template:
    5
    Resources:
    6
    Is the TypeId2Integer function necessary? Can you not access UnitIds with single quotes in Lua like you do in JASS?

    Code (Text):
    CreateUnit(Player(0), TypeId2Integer("Hamg"), 0, 0, 0)
    vs
    Code (Text):
    CreateUnit(Player(0), 'Hamg', 0, 0, 0
     
  6. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    No Lua does not support that. Read every Lua tutorial every written and you will see that such syntax does not exist in Lua. JASS only supports it because it was specifically made for Warcraft III rather than Lua which is a general programming/scripting language.

    Also that is part of the test script so not required for the system to work.
     
  7. Wadjet

    Wadjet

    Joined:
    Sep 14, 2009
    Messages:
    243
    Resources:
    6
    Skins:
    1
    Template:
    5
    Resources:
    6
    Interesting. So to access any object by it's Id value with Lua, we would need to create a function like the one you wrote?

    Is there any easier way to do it without using shifts and other bit operations?
     
  8. Tasyen

    Tasyen

    Joined:
    Jul 18, 2010
    Messages:
    1,300
    Resources:
    16
    Tools:
    2
    Maps:
    2
    Spells:
    7
    Tutorials:
    4
    JASS:
    1
    Resources:
    16
    GUI Lua code uses FourCC("hfoo") to convert unitIds.
     
  9. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    It is only needed for object types to go from human readable raw code to integer. All object types returned by functions are already integers so can be passed around directly just like with JASS.

    There is also a way to decode directly using built in Lua functions (not shown in the map). Additionally I think with 1.31.1 there are now natives to perform the conversion from string as well.
    Yes you can use the Lua standard library encode and decode mechanisim. This may or may not be faster. There might also be natives to do such a conversion.
    This was added after the PTR, hence why the code above did not use it as it was test code left in from the PTR in the map I developed this system.
     
  10. Yousef

    Yousef

    Joined:
    Jul 19, 2011
    Messages:
    516
    Resources:
    0
    Resources:
    0
    Hey, thanks a lot for your work, but I'm having a problem, when I try to save it gives me a syntax error with a message saying "missing endblock", any idea how to fix this?
    Untitled.jpg
     
    Last edited: Jun 14, 2019
  11. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Try disabling JASSHelper. Apparently there are compatibility issues with it and maps running in Lua mode. For some reason...

    If importing into your own map. Make sure the map is operating in Lua mode. This is an Lua only fix for using the Lua virtual machine. The JASS2 virtual machine does not allow one to perform such a fix.

    One can still use GUI and JASS (JASS2) with Lua thanks to a transpiler on save. Just for some reason it does not work well with vJASS.
     
    Last edited: Jun 14, 2019
  12. Yousef

    Yousef

    Joined:
    Jul 19, 2011
    Messages:
    516
    Resources:
    0
    Resources:
    0
    Well now I got a whole bunch of other errors about the lua code after I disabled vJass.
    Untitled.jpg
    With vJass it works on other maps without missing endblock error.
     
    Last edited: Jun 14, 2019
  13. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Did you set the map to use Lua? One must explicitly specify the virtual machine used and all older maps will default to JASS2.
     
  14. Yousef

    Yousef

    Joined:
    Jul 19, 2011
    Messages:
    516
    Resources:
    0
    Resources:
    0
    I think it's already activated, can you tell me how to activate it in case it's not activated?
     
  15. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Scenario -> Map Options... -> Script Language:

    From the pulldown select Lua.
     
  16. Marshmalo

    Marshmalo

    Joined:
    Jul 1, 2008
    Messages:
    970
    Resources:
    5
    Icons:
    1
    Maps:
    4
    Resources:
    5
    Hi Dr Super good,

    If it works in multiplayer this looks like a great tool! Me and Yousef are working on a popular Bnet map: Lordaeron the Foremath (LTF), however since latest Blizzard patches the game becomes very unstable after 1 hour game play. As this is an epic RTS game sometimes the battles can rage on for up to 2 hours so stability is paramount to the map's enjoyment, we think this Lua trash collector could really help us.

    However, we already use a lot of Jass custom scripts in the game, this is usually for stuff like cleaning up location variables and other simple functions. If I understand correctly we cna remove these if we are using your clean script as they will become redundant?

    Unfortunately we do also use jass custom scripts for other functions, such as making sounds only play to single player, or setting filters for a specific player. Examples below, can you advise on how we can replace these jass scripts with Lua versions? Neither me nor Yousef are trigger experts.

    We should then be able to convert the map script to Lua and try out your script. Annoyingly there are quite a lot of triggers on this map, its huge so it will be no easy task for us!

    Custom Jass script:
    if GetLocalPlayer() == Player(4) then
     
  17. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    You could. However there is also no harm keeping them in as the system will work around them to stop leaks that might otherwise have been missed.

    That said in order for the system to work efficiently one might have to tune how often the garbage collector is run, as I did in the demo map. When the Lua garbage collector runs it will remove all leaks, but the rate at which that happens might not be optimal with the default garbage collector settings. In the demo map I changed the settings, but one could also explicitly run the garbage collector, eg every 15 or 60 seconds.
    The built in JASS2toLua transpiler should handle all that. GUI still compiles to JASS on map save but when the map is in Lua mode it will convert the resulting JASS script to Lua using the built in transpiler.
     
  18. Marshmalo

    Marshmalo

    Joined:
    Jul 1, 2008
    Messages:
    970
    Resources:
    5
    Icons:
    1
    Maps:
    4
    Resources:
    5
    I think once very 60 seconds would be perfect, the map is huge so it really depends how much extra stress the cleanup script causes on the game when it runs.

    If the build in transpiler can do all the converting that would make it so much easier, there's ALOT of triggers. However, the problem I'm finding is that I don't even get the option to convert the script. Its greyed out:

    Lua Disabled.jpg
     
  19. Tasyen

    Tasyen

    Joined:
    Jul 18, 2010
    Messages:
    1,300
    Resources:
    16
    Tools:
    2
    Maps:
    2
    Spells:
    7
    Tutorials:
    4
    JASS:
    1
    Resources:
    16
    if your map contains one line of hand written code you are not allowed to swap language mode. From offical patch-notes.
    • Maps save in the selected language
    • Maps that are written only in standard libraries can switch to Lua
    • Maps that use custom script will have to disable all custom triggers before conversion
    Disable all triggers with custom script then you should be able to swap to Lua.
     
  20. Yousef

    Yousef

    Joined:
    Jul 19, 2011
    Messages:
    516
    Resources:
    0
    Resources:
    0
    Ok so after doing as mentioned above and changing scripting language, then adding the Garbage Collector trigger and saving I got this syntax error. Untitled.jpg

    EDIT: It doesn't seem to have anything to do with the Garbage Collector because I still got the syntax error after removing the garbage collector and saving.
     
    Last edited: Jun 30, 2019