Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

W3Protect - Jass and Lua map optimizer

Discussion in 'Warcraft Editing Tools' started by Frotty, Sep 26, 2019.

  1. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks for reporting, but the map itself is broken, it doesn't even run in warcraft III.
    The war3map.j contains vjass code, which seems very wrong.

    [​IMG]

    Btw you can see debug info in console if you run from there.
     
    Last edited: Oct 9, 2019
  2. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,537
    Resources:
    23
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    5
    JASS:
    3
    Resources:
    23
    Ups, sorry for this.

    Anyways I tried with again with playable map, and get "Code has errors", or with console the following:

    upload_2019-10-10_23-33-16.png
     
  3. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks. The "Critical Error" from console only means that you don't have the same (i.e. missing) permissions when running from there (for some reason).

    The "Code has Errors" is because Wurst detects potential faults in your code ;)
    The war3map.j contains calls to ExecuteFunc with non existing functions as arguments, e.g.
    call ExecuteFunc("s__PlayerTools_Init__onInit")
    .
    For development that makes sense, but the optimizer should simply remove the call. I will push an update for this.
     
  4. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    841
    Resources:
    13
    Tools:
    1
    Maps:
    6
    Spells:
    6
    Resources:
    13
    Works like a charm :)
     
  5. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    841
    Resources:
    13
    Tools:
    1
    Maps:
    6
    Spells:
    6
    Resources:
    13
    New issue found,

    Using "Extreme Compression" completely obliterates all(?) imported sounds; causing them to become, as UndeadImmortal said, an "ear splitting noise".
     
  6. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks for the report. Actually only .wav files are affected, since they are exempt from recompression due to increasing in size by deflation.
    I will push a fix for this soon.
     
  7. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Version 0.5.8 released. See first post for download & changelog.

    @IcemanBo Okay, useless EF calls get removed. Do all of your maps work now?

    @lolreported I added a "fix" for the .wav problem, but in general .wavs are just problematic. The world editor uses a lossy compression for wavs with around 30% compression ratio, which would turn to 90%+ ratio if we just used lossless zlib. If we don't change the sector size we can just keep the files of course, but we want to increase the sector size to better compress all other files. Recompression is also not really feasible, because the first sector of wavs is not lossy compressed (assumably to retain header info) and with a big sector size there only is one sector per file, thus the lossy compression wouldn't even get into action.
    Depending on how much space the wavs take up in the map a higher sector count could still be okay, so I will add an option to force recompression in the future.
    For now it will just show a message when .wav files have been detected, and keep the sector size and .wav files as is.
    I don't think there is much point in further research due to the generous map file size constraint in battle.net.
     
  8. lolreported

    lolreported

    Joined:
    Aug 16, 2007
    Messages:
    841
    Resources:
    13
    Tools:
    1
    Maps:
    6
    Spells:
    6
    Resources:
    13
    It sounds like a very complicated problem and just ignoring the .wav files altogether is fine, in my opinion. As you said, the map file size is huge nowadays and file sizes aren't that big of a problem.

    I'll report back how it works when I'm done updating my map. :)


    EDIT
    Used the recent version of the tool and the map now crashes when loading, using Extreme Compression or not doesn't matter. Version 0.5.7 works without issue (not using Extreme Compression to avoid the sound glitch).

    crash.PNG
     
    Last edited: Oct 18, 2019
  9. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Thanks for the report, it was yet another bug in the wts inliner :D
    I pushed 0.5.9 with a fix.
     
  10. IcemanBo

    IcemanBo

    Joined:
    Sep 6, 2013
    Messages:
    6,537
    Resources:
    23
    Maps:
    3
    Spells:
    11
    Template:
    1
    Tutorials:
    5
    JASS:
    3
    Resources:
    23
    All works fine for me. People should use this!
     
  11. juvian

    juvian

    Joined:
    Sep 25, 2017
    Messages:
    50
    Resources:
    0
    Resources:
    0
    upload_2019-12-19_21-28-7.png

    been waiting for 30 minutes. Map sent by dm
     
  12. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Did you read the first post?

    The editor crashes when attempting to open the map.
    Maps saved with the reforged editor aren't supported until this ticket is implemented: New reforged W3I format · Issue #48 · inwc3/wc3libs

    You can run from console for some logs.
     
    Last edited: Dec 20, 2019
  13. juvian

    juvian

    Joined:
    Sep 25, 2017
    Messages:
    50
    Resources:
    0
    Resources:
    0
    Running from console gives me this:

    upload_2019-12-20_18-18-23.png



    This is not a reforged map, its a 1.28.5 unprotected korean map. As for editor crashing, if you remove trigger file (wtg) it will work. I am not sure if they use a modified world editor in korea or they have some kind of plugin, but the issue I believe is that it uses in gui some functions which are not defined in either blizzard.j or common.j. In a wc3 game it works because it seems to override blizzard.j with the maps custom blizzard.j inside the scripts folder, but it would seem world editor doesn't do that and behaves differently. Maybe just changing the blizzard.j file inside war3.mpq would make it work but never tried as I never use world editor.

    Anyway map is unprotected and editable, have been using w3x2lni for a while already with no issues but wanted to try out this one. It's a complex map that will probably have a lot of border cases (most hive programs like leak checker won't work with it), so I thought it would be a good example for you to find bugs. But if you deem it too uncommon to bother fixing thats okay too, will keep using w3x2lni. As for the gui, is the map name and author supposed to show the TRIGSTR number? Showing the wts values would be nicer (Eden RPG S2 4.5A Fix6 and scvscvgo)
     
  14. Darklycan51

    Darklycan51

    Joined:
    Jan 12, 2011
    Messages:
    1,396
    Resources:
    3
    Maps:
    3
    Resources:
    3
    "inline strings" makes all strings appear on the same line so it's not just internal, basically useless setting since loading screens and even HUD text gets fucked up.


    But there is a much more broken thing, "optimize code" breaks the code, like actually breaks it, I was optimizing an rpg and the button to talk to a villager started causing the town to go hostile lol
     
  15. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    You should've simply given me all that info up front. I'm not a fan of spending time on maps that are somehow broken or protected on purpose.

    It shows the raw values from the .w3i - whether those are inline values or TRIGSTR. I suppose a conversion could be added.

    What do you mean by "not just internal" ? It looks like the lineending stuff depends on some file encoding shenanigans - it's an issue I will check soon,
    but obviously not widespread as many have used this tool successfully by now.

    This tool is making some more extreme optimizations, which can - even though the maps me and others have tested work - break something in certain edge cases that haven't been found.
    A situation like you describe can be caused by something miniscule in the output code.

    If you want to help, attach/send your map, omit the banter.
     
  16. Darklycan51

    Darklycan51

    Joined:
    Jan 12, 2011
    Messages:
    1,396
    Resources:
    3
    Maps:
    3
    Resources:
    3
    Not my map, it's actually a map that is here on hive and I was fixing a crash issue since the map is popular but the author hasn't updated it, called northrend bound, it's unprotect it you can check yourself, try doing hte extreme optimization on it and then talk to a villager and the village will go hostile.


    Also do the string thing on the map, you'll see the issues on both the loading screen and UI screens.
     
  17. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Okay I pushed 0.5.10 with WTS inlining fixes.

    It's pretty far from that. The existing war3map.j inside the map doesn't even compile on its own, it requires functions defined in a custom "blizzard.j" file.
    The game is apparently okay with that, but of course most tools aren't. I can look into adding such a detection, but it's not really a priority for me.

    The strings are fixed, as I suspected the issue originated from an underlying library.

    I could reproduce the issue, but am not sure what causes it. There are many calls to alliance modifications in the code, it might even be some anti tamper mechanic.
    If you can pinpoint what part of the code causes it I can look into it further.
     
  18. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    @Darklycan51 Alright I actually found the issue. The map works fine with 0.5.11 for me now.

    Cheers.
     
  19. Darklycan51

    Darklycan51

    Joined:
    Jan 12, 2011
    Messages:
    1,396
    Resources:
    3
    Maps:
    3
    Resources:
    3
    Alright thanks!
     
  20. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,479
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    I uploaded version 0.6.0 to the first post.
    Fixed a bug and w3p now uses reforged native files.

    Cheers.