1. Updated Resource Submission Rules: All model & skin resource submissions must now include an in-game screenshot. This is to help speed up the moderation process and to show how the model and/or texture looks like from the in-game camera.
    Dismiss Notice
  2. DID YOU KNOW - That you can unlock new rank icons by posting on the forums or winning contests? Click here to customize your rank or read our User Rank Policy to see a list of ranks that you can unlock. Have you won a contest and still havn't received your rank award? Then please contact the administration.
    Dismiss Notice
  3. The Lich King demands your service! We've reached the 19th edition of the Icon Contest. Come along and make some chilling servants for the one true king.
    Dismiss Notice
  4. The 4th SFX Contest has started. Be sure to participate and have a fun factor in it.
    Dismiss Notice
  5. The poll for the 21st Terraining Contest is LIVE. Be sure to check out the entries and vote for one.
    Dismiss Notice
  6. The results are out! Check them out.
    Dismiss Notice
  7. Don’t forget to sign up for the Hive Cup. There’s a 555 EUR prize pool. Sign up now!
    Dismiss Notice
  8. The Hive Workshop Cup contest results have been announced! See the maps that'll be featured in the Hive Workshop Cup tournament!
    Dismiss Notice
  9. Check out the Staff job openings thread.
    Dismiss Notice
Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

W3Protect - Jass and Lua map optimizer

Discussion in 'Warcraft Editing Tools' started by Frotty, Sep 26, 2019.

  1. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Info

    Yo, this is just a little fun project I worked on a bit.
    After adding some preliminary Lua support I thought to just publish it here for some feedback :)

    [​IMG]

    The name is just for fun as well. There is no "special protection" going on here.
    The main reason for this tool was/is to have an all in one solution for map finalization.
    (If Water and me fix SLK transform at some point)

    Install

    Requires an installation of Java 8 or later (Latest release | AdoptOpenJDK)
    If you have Java installed, just download the .zip file from this post and extract the .jar file from it.

    Usage

    Run the .jar from the downloaded archive with no arguments to start open the graphical interface.
    Use the "Browse" button or drag & drop a map onto the app window to load it.
    If the load was successful, you should see the preview image.
    Otherwise the map is already protected or otherwise unreadable.
    (If you think it's unprotected but not working, please report the map)

    You can use w3protect from the commandline by passing a map path as first and only argument.
    The config will be read from the .wpc file next to the .jar .

    In both cases w3protect will generate a copy of your input map with the suffix "_w3pro" in the same location.

    Options explained

    • "Extreme Compression" - re-compresses every file in the mpq with best settings. (optionally zopfli)
    • "Encrypt Strings" - does no encryption, but it's a cool name. Turns string literal comparisons into hash comparisons.
    • "Lua minify" - uses 'luamin' to minify the map's lua script.

    Feedback

    Please let me know if there are any problems with maps after being processes by w3p, I only did minimal testing.
    You can also suggest cool features to implement.
    Reporting Maps: Please make sure your maps runs and is openable with the world editor.

    0.5.2
    • Initial release
    0.5.3
    • TriggerRegisterVariableEvent is now allowed. Referenced variables will be kept unchanged.
    • Added second inliner pass after local optimizations to get rid of functions which were shortened significantly
    • Added rewrites for Jass "ternary" functions and equality comparisons with constant booleans
    0.5.4
    • Trigger string replacement in plain text files doesn't stop after the first match any more
    • Trigger strings that are not inlined are now kept intact correctly
    • Trigger strings with quotes are now properly escaped in script files
    • Disabled experimental function rewrite
    0.5.5
    • Variables used in TriggerRegisterVariableEvent are now also protected during optimization.
    • Charmap used for obfuscation is now customizable
    • Default charmap now uses all characters
    • Re-enabled experimental rewrites
    • Fixed program hang when output map not writable
    0.5.6
    • Added more files for WTS inlining and fixed objmod inlining
    • Fixed WTS inliner normalization of line breaks and max length
    • Fixed war3map.wct not being removed
    • Fixed uppercase color codes not being displayed
    • Fixed variable protection for TriggerRegisterVariableEvent hooks
    0.5.7
    • Fixed WTS parser bug in underlying lib
    0.5.8
    • Fixed useless ExecuteFunction calls stopping script optimization. They now get removed.
    • Fixed .wav files getting corrupted with extreme compression option
    • Fixed input map getting modified after load
    0.5.9
    • Fixed WTS inliner corruption bug
     

    Attached Files:

    Last edited: Oct 18, 2019 at 8:42 PM
  2. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,178
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    In the status bar i get "Code has errors" in the end, with this map: Ice Escape Template v2.3

    I also tried a different map, from older patch and without new natives, and it succeeded.
     
  3. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks for testing. The .j files are up to date, but the map in question uses `TriggerRegisterVariableEvent` which is not allowed in the underlying wurst optimizer :ugly:
    Should probably fix that somehow.
     
  4. TriggerHappy

    TriggerHappy

    Code Moderator

    Joined:
    Jun 23, 2007
    Messages:
    3,665
    Resources:
    22
    Spells:
    11
    Tutorials:
    2
    JASS:
    9
    Resources:
    22
    Looks good. Maybe I can start recommending this instead of explaining how to get vex's map optimizer and w3x2lni to work for people.

    Is it open source?
     
  5. yxq1122

    yxq1122

    Joined:
    Jun 5, 2018
    Messages:
    107
    Resources:
    0
    Resources:
    0
    Cool tool that might be what any map maker needs! The interface looks great. l like it. Continue this project please!
     
  6. Vicboy

    Vicboy

    Joined:
    Aug 20, 2007
    Messages:
    1,048
    Resources:
    2
    Maps:
    2
    Resources:
    2
    Can it optimize map folders?
     
  7. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    The tool itself isn't (yet), but the underlying libraries are.

    No I haven't worked with "map folders" yet. Feel free to upload an example & specification then I might add it.
     
  8. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    @IcemanBo I uploaded version 0.5.3 which contains a fix for TRVE. The references variable names are now kept intact and constant string usage is accepted.
    Can you verify if everything still works correctly after optimization?

    I added some new rewrites inspired by your mapscript as well.
     
  9. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    725
    Resources:
    12
    Tools:
    1
    Maps:
    5
    Spells:
    6
    Resources:
    12
    Great work! I've experimented with the tool abit using a map of mine built mostly out of GUI.

    Using Inline Trigger Strings
    This causes the Loading Screen Text to disappear and it also removed all quest text
    and possibly other text I missed. Possibly all "TRIGSTR_"?


    Using Obfuscate Script
    I had different results using two actions:
    • Set MapSavePath = (CHAMPIONS_TRIAL\\AP_Data + (String(MAP_VERSION)))
    this generated:
    set ooL="CHAMPIONS_TRIAL\\\\AP_Data"+I2S(OOo)


    • Custom script: set udg_MapSavePath = "CHAMPIONS_TRIAL\\AP_Data" + I2S(udg_MAP_VERSION)
    this generated:
    set ooL="CHAMPIONS_TRIAL\\AP_Data"+I2S(OOo)


    Notice the 2 additional "\\".

    This option also broke old save files from Codeless Save and Load (Multiplayer) - v3.0.0; had no issue saving a new file and load it.


    Using Optimize Code
    This caused units not to spawn and some events went haywire and fired at map start, completely broke the map.


    I also get the following error from time to time when I run the .jar file.
    error.PNG
     
    Last edited: Sep 28, 2019
  10. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Cool, thanks for testing. Can you link or attach your map? Then I can debug the issues.

    Well no, obviously the values should be "inlined". If they are empty, something went wrong. What is your locale?

    This has nothing to do with the tool, it's normal GUI behavior.
    [​IMG] [​IMG]

    If you are writing Jass (custom script) you need to manually escape backslashes, in GUI it's done for you (should only use 1)

    Not sure why tbh, need more info.

    Sounds strange, might be due to my recent changes from Bo's map, I'm still waiting on his response.
     
  11. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    725
    Resources:
    12
    Tools:
    1
    Maps:
    5
    Spells:
    6
    Resources:
    12
    Sure, I'll PM you an unprotected version of the map; be warned though, that the triggers are a mess. If you need any pointers, feel free to ask.

    enUS

    Oh, didn't know that GUI manipulated string variables you set yourself.

    I've got no information regarding this issue to share unfortunately.
     
    Last edited: Sep 28, 2019
  12. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks, I fixed the Trigger Strings issue and an optimization problem, but the save system still seems stuck in "synchronizing" for some reason.
    I uploaded version 0.5.4 to the first post.
     
  13. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    725
    Resources:
    12
    Tools:
    1
    Maps:
    5
    Spells:
    6
    Resources:
    12
    Yeah, I'm experiencing the same issue; but it's worth mentioning that the system works if you leave the Script Options unchecked and just protect the map using the other options.

    I've looked into it a little but can't find the issue.

    A possible feature could be to let the user add a comment that forces the Script Options to ignore certain blocks of code?

    Code (vJASS):

    //W3 protect ignore this
    function HelloWorld takes nothing returns nothing
        call BJDebugMsg("Hello World")
    endfunction
     
     
  14. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,546
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Map folders are basically an unpacked form of the map MPQ archive. I do not see the point of this tool supporting them seeing how one only really needs to finalize MPQ maps as a final step before publishing them.
     
  15. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Version 0.5.5 released. See first post for download & changelog.

    Thanks, it was another case of TRVE usage, which is fixed in 0.5.5
    The map runs fine with optimization & obfuscation now.

    As a last resort, I suppose. Ideally it should be able to handle all inputs :D

    True, but that also means packing them into a map should be easy enough?
     
  16. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    725
    Resources:
    12
    Tools:
    1
    Maps:
    5
    Spells:
    6
    Resources:
    12
    I've tried a couple of different combinations of options when protecting the current version of Champions Trial and everything appears to be working without any issue. :)

    I did find one minor string issue though:
    Code (vJASS):
    call CreateQuestBJ(bj_QUESTTYPE_REQ_DISCOVERED,"Credits 2/2","|cffffcc00Icons|r|nPeeKay, Darkfang, CRAZYRUSSIAN, PrinceYaser|n|n|cffffcc00Models|r|nMythic|n|n|cffffcc00MapPreviewImage|r|nmade by Wei Wang (https:","ReplaceableTextures\\WorldEditUI\\Editor-MultipleUnits.blp")
    call CreateQuestBJ(bj_QUESTTYPE_REQ_DISCOVERED,"Contact","|cffffcc00I've found a bug/issue and/or got feedback/suggestions!|r|nPlease post a comment at Hiveworkshop (a Warcraft 3 forum): https:","ReplaceableTextures\\CommandButtons\\BTNLoadDwarf.blp")

    I'm not sure which option causes it, but the "https:"-link and everything after is removed from the string.

    Here are the strings as they should be:
    Code (vJASS):
    "|cffffcc00Icons|r|nPeeKay, Darkfang, CRAZYRUSSIAN, PrinceYaser|n|n|cffffcc00Models|r|nMythic|n|n|cffffcc00MapPreviewImage|r|nmade by Wei Wang (https://www.artstation.com/weiwangart)|n|n|cffffcc00Thank you for playing!|r"
    "|cffffcc00I've found a bug/issue and/or got feedback/suggestions!|r|nPlease post a comment at Hiveworkshop (a Warcraft 3 forum): https://www.hiveworkshop.com/threads/champions-trial-3-9a.311955/ |n|nPlease check "Known Issues" below and make sure what you're about to submit isn't already known.|n|n|cffffcc00Known Issues|r|n - The "Hell Chain"-effects from Demo'vilesh attached to creeps can sometimes bug out and become invisible (this might be an issue with the current Warcraft 3 patch).|n - The ability hotkeys are currently not "QWERT" as there's no time efficient way for me to make it so. I'll add it in the future, just unknown when."


    Keep up the good work!

    ---

    Update:

    I decided to update Bribe's Damage Engine 5.4.2.3 in my map from v5.1.3.1 and I'm now unable to protect the map using the tool. I get "Code has Errors!".

    I tried to protect the demo map itself and get the same result.
     
  17. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Version 0.5.6 released. See first post for download & changelog.

    Thanks for the reports, I fixed the linebreak issues and nothing should be cut off anymore.

    The general problem is
    TriggerRegisterVariableEvent 
    which uses a string to reference a variable unsafely.
    That's fine if the string is constant - then we just read all the referenced variables. But if it is not constant, i.e. a reference, it could change and have all sorts of values.
    In recent updates Bribe now uses a hook which causes a TriggerRegisterVariableEvent call to be in the map without constant argument for the parameter.
    Code (vJASS):
    function h__TriggerRegisterVariableEvent takes trigger a0, string a1, limitop a2, real a3 returns event
        //hook: DamageEngine___PreSetup
        call sc___prototype6_evaluate(1,a0,a1,a2,a3)
    return TriggerRegisterVariableEvent(a0,a1,a2,a3)
    endfunction


    I added an additional check if the call is in a hook which fixes this issue, but others will still fail.
     
  18. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    725
    Resources:
    12
    Tools:
    1
    Maps:
    5
    Spells:
    6
    Resources:
    12
    Thanks for the explanation and update,

    I tried the new version and didn't get any code errors. Neither did my testing show any weird behaviour.

    The string issue is still present though (but easy enough to work around):
    WC3ScrnShot_100619_165037_01.png
     
    Last edited: Oct 6, 2019
  19. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,428
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Ah I see, it was an issue in the underlying library used for WTS parsing. The part after the colon was treated as comment due to "//".
    It is fixed in 0.5.7 . Let me know if everything works now :p
     
  20. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,178
    Resources:
    22
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    4
    JASS:
    3
    Resources:
    22
    It stucks at "Parsing script.." (waited ~25-30min).
     

    Attached Files: