W3Protect - Jass and Lua map optimizer

Level 21
Joined
Jan 1, 2009
Messages
1,554
In the status bar i get "Code has errors" in the end, with this map: Ice Escape Template v2.3

I also tried a different map, from older patch and without new natives, and it succeeded.

Thanks for testing. The .j files are up to date, but the map in question uses `TriggerRegisterVariableEvent` which is not allowed in the underlying wurst optimizer :ugly:
Should probably fix that somehow.
 
Level 19
Joined
Aug 16, 2007
Messages
871
Great work! I've experimented with the tool abit using a map of mine built mostly out of GUI.

Using Inline Trigger Strings
This causes the Loading Screen Text to disappear and it also removed all quest text
and possibly other text I missed. Possibly all "TRIGSTR_"?


Using Obfuscate Script
I had different results using two actions:
  • Set MapSavePath = (CHAMPIONS_TRIAL\\AP_Data + (String(MAP_VERSION)))
this generated: set ooL="CHAMPIONS_TRIAL\\\\AP_Data"+I2S(OOo)

  • Custom script: set udg_MapSavePath = "CHAMPIONS_TRIAL\\AP_Data" + I2S(udg_MAP_VERSION)
this generated: set ooL="CHAMPIONS_TRIAL\\AP_Data"+I2S(OOo)

Notice the 2 additional "\\".

This option also broke old save files from Codeless Save and Load (Multiplayer) - v3.0.0; had no issue saving a new file and load it.


Using Optimize Code
This caused units not to spawn and some events went haywire and fired at map start, completely broke the map.


I also get the following error from time to time when I run the .jar file.
error.PNG
 
Last edited:
Level 21
Joined
Jan 1, 2009
Messages
1,554
Great work! I've experimented with the tool abit using a map of mine built mostly out of GUI.

Cool, thanks for testing. Can you link or attach your map? Then I can debug the issues.

Using Inline Trigger Strings
This causes the Loading Screen Text to disappear and it also removed all quest text
and possibly other text I missed. Possibly all "TRIGSTR_"?

Well no, obviously the values should be "inlined". If they are empty, something went wrong. What is your locale?

Using Obfuscate Script
I had different results using two actions:
  • set.gif
    Set MapSavePath = (CHAMPIONS_TRIAL\\AP_Data + (String(MAP_VERSION)))
this generated:
set ooL="CHAMPIONS_TRIAL\\\\AP_Data"+I2S(OOo)


  • page.gif
    Custom script: set udg_MapSavePath = "CHAMPIONS_TRIAL\\AP_Data" + I2S(udg_MAP_VERSION)
this generated:
set ooL="CHAMPIONS_TRIAL\\AP_Data"+I2S(OOo)


Notice the 2 additional "\\".

This has nothing to do with the tool, it's normal GUI behavior.
Z7R9bZM.jpg
Xesoezf.jpg


If you are writing Jass (custom script) you need to manually escape backslashes, in GUI it's done for you (should only use 1)

This option also broke old save files from Codeless Save and Load (Multiplayer) - v3.0.0; had no issue saving a new file and load it.

Not sure why tbh, need more info.

Using Optimize Code
This caused units not to spawn and some events went haywire and fired at map start, completely broke the map.

Sounds strange, might be due to my recent changes from Bo's map, I'm still waiting on his response.
 
Level 19
Joined
Aug 16, 2007
Messages
871
Cool, thanks for testing. Can you link or attach your map? Then I can debug the issues.
Sure, I'll PM you an unprotected version of the map; be warned though, that the triggers are a mess. If you need any pointers, feel free to ask.

Well no, obviously the values should be "inlined". If they are empty, something went wrong. What is your locale?
enUS

This has nothing to do with the tool, it's normal GUI behavior.
Z7R9bZM.jpg
Xesoezf.jpg


If you are writing Jass (custom script) you need to manually escape backslashes, in GUI it's done for you (should only use 1)
Oh, didn't know that GUI manipulated string variables you set yourself.

Not sure why tbh, need more info.
I've got no information regarding this issue to share unfortunately.
 
Last edited:
Level 21
Joined
Jan 1, 2009
Messages
1,554
Sure, I'll PM you an unprotected version of the map; be warned though, that the triggers are a mess. If you need any pointers, feel free to ask.

Thanks, I fixed the Trigger Strings issue and an optimization problem, but the save system still seems stuck in "synchronizing" for some reason.
I uploaded version 0.5.4 to the first post.
 
Level 19
Joined
Aug 16, 2007
Messages
871
Thanks, I fixed the Trigger Strings issue and an optimization problem, but the save system still seems stuck in "synchronizing" for some reason.
I uploaded version 0.5.4 to the first post.
Yeah, I'm experiencing the same issue; but it's worth mentioning that the system works if you leave the Script Options unchecked and just protect the map using the other options.

I've looked into it a little but can't find the issue.

A possible feature could be to let the user add a comment that forces the Script Options to ignore certain blocks of code?

JASS:
//W3 protect ignore this
function HelloWorld takes nothing returns nothing
    call BJDebugMsg("Hello World")
endfunction
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Version 0.5.5 released. See first post for download & changelog.

Yeah, I'm experiencing the same issue; but it's worth mentioning that the system works if you leave the Script Options unchecked and just protect the map using the other options.

I've looked into it a little but can't find the issue.

Thanks, it was another case of TRVE usage, which is fixed in 0.5.5
The map runs fine with optimization & obfuscation now.

A possible feature could be to let the user add a comment that forces the Script Options to ignore certain blocks of code?

As a last resort, I suppose. Ideally it should be able to handle all inputs :D

Map folders are basically an unpacked form of the map MPQ archive. I do not see the point of this tool supporting them seeing how one only really needs to finalize MPQ maps as a final step before publishing them.

True, but that also means packing them into a map should be easy enough?
 
Level 19
Joined
Aug 16, 2007
Messages
871
I've tried a couple of different combinations of options when protecting the current version of Champions Trial and everything appears to be working without any issue. :)

I did find one minor string issue though:
JASS:
call CreateQuestBJ(bj_QUESTTYPE_REQ_DISCOVERED,"Credits 2/2","|cffffcc00Icons|r|nPeeKay, Darkfang, CRAZYRUSSIAN, PrinceYaser|n|n|cffffcc00Models|r|nMythic|n|n|cffffcc00MapPreviewImage|r|nmade by Wei Wang (https:","ReplaceableTextures\\WorldEditUI\\Editor-MultipleUnits.blp")
call CreateQuestBJ(bj_QUESTTYPE_REQ_DISCOVERED,"Contact","|cffffcc00I've found a bug/issue and/or got feedback/suggestions!|r|nPlease post a comment at Hiveworkshop (a Warcraft 3 forum): https:","ReplaceableTextures\\CommandButtons\\BTNLoadDwarf.blp")
I'm not sure which option causes it, but the "https:"-link and everything after is removed from the string.

Here are the strings as they should be:
JASS:
"|cffffcc00Icons|r|nPeeKay, Darkfang, CRAZYRUSSIAN, PrinceYaser|n|n|cffffcc00Models|r|nMythic|n|n|cffffcc00MapPreviewImage|r|nmade by Wei Wang (https://www.artstation.com/weiwangart)|n|n|cffffcc00Thank you for playing!|r"
"|cffffcc00I've found a bug/issue and/or got feedback/suggestions!|r|nPlease post a comment at Hiveworkshop (a Warcraft 3 forum): https://www.hiveworkshop.com/threads/champions-trial-3-9a.311955/ |n|nPlease check "Known Issues" below and make sure what you're about to submit isn't already known.|n|n|cffffcc00Known Issues|r|n - The "Hell Chain"-effects from Demo'vilesh attached to creeps can sometimes bug out and become invisible (this might be an issue with the current Warcraft 3 patch).|n - The ability hotkeys are currently not "QWERT" as there's no time efficient way for me to make it so. I'll add it in the future, just unknown when."

Keep up the good work!

---

Update:

I decided to update Bribe's Damage Engine 5.4.2.3 in my map from v5.1.3.1 and I'm now unable to protect the map using the tool. I get "Code has Errors!".

I tried to protect the demo map itself and get the same result.
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Version 0.5.6 released. See first post for download & changelog.

I'm not sure which option causes it, but the "https:"-link and everything after is removed from the string.

Thanks for the reports, I fixed the linebreak issues and nothing should be cut off anymore.

I decided to update Bribe's Damage Engine 5.4.2.3 in my map from v5.1.3.1 and I'm now unable to protect the map using the tool. I get "Code has Errors!".

I tried to protect the demo map itself and get the same result.

The general problem is TriggerRegisterVariableEvent which uses a string to reference a variable unsafely.
That's fine if the string is constant - then we just read all the referenced variables. But if it is not constant, i.e. a reference, it could change and have all sorts of values.
In recent updates Bribe now uses a hook which causes a TriggerRegisterVariableEvent call to be in the map without constant argument for the parameter.
JASS:
function h__TriggerRegisterVariableEvent takes trigger a0, string a1, limitop a2, real a3 returns event
    //hook: DamageEngine___PreSetup
    call sc___prototype6_evaluate(1,a0,a1,a2,a3)
return TriggerRegisterVariableEvent(a0,a1,a2,a3)
endfunction

I added an additional check if the call is in a hook which fixes this issue, but others will still fail.
 
Level 19
Joined
Aug 16, 2007
Messages
871
Version 0.5.6 released. See first post for download & changelog.



Thanks for the reports, I fixed the linebreak issues and nothing should be cut off anymore.



The general problem is TriggerRegisterVariableEvent which uses a string to reference a variable unsafely.
That's fine if the string is constant - then we just read all the referenced variables. But if it is not constant, i.e. a reference, it could change and have all sorts of values.
In recent updates Bribe now uses a hook which causes a TriggerRegisterVariableEvent call to be in the map without constant argument for the parameter.
JASS:
function h__TriggerRegisterVariableEvent takes trigger a0, string a1, limitop a2, real a3 returns event
    //hook: DamageEngine___PreSetup
    call sc___prototype6_evaluate(1,a0,a1,a2,a3)
return TriggerRegisterVariableEvent(a0,a1,a2,a3)
endfunction

I added an additional check if the call is in a hook which fixes this issue, but others will still fail.
Thanks for the explanation and update,

I tried the new version and didn't get any code errors. Neither did my testing show any weird behaviour.

The string issue is still present though (but easy enough to work around):
WC3ScrnShot_100619_165037_01.png
 
Last edited:
Level 21
Joined
Jan 1, 2009
Messages
1,554
Thanks for the explanation and update,

I tried the new version and didn't get any code errors. Neither did my testing show any weird behaviour.

The string issue is still present though (but easy enough to work around):

Ah I see, it was an issue in the underlying library used for WTS parsing. The part after the colon was treated as comment due to "//".
It is fixed in 0.5.7 . Let me know if everything works now :p
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Ups, sorry for this.

Anyways I tried with again with playable map, and get "Code has errors", or with console the following:

Thanks. The "Critical Error" from console only means that you don't have the same (i.e. missing) permissions when running from there (for some reason).

The "Code has Errors" is because Wurst detects potential faults in your code ;)
The war3map.j contains calls to ExecuteFunc with non existing functions as arguments, e.g. call ExecuteFunc("s__PlayerTools_Init__onInit").
For development that makes sense, but the optimizer should simply remove the call. I will push an update for this.
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Using "Extreme Compression" completely obliterates all(?) imported sounds; causing them to become, as UndeadImmortal said, an "ear splitting noise".

Thanks for the report. Actually only .wav files are affected, since they are exempt from recompression due to increasing in size by deflation.
I will push a fix for this soon.
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Version 0.5.8 released. See first post for download & changelog.

@IcemanBo Okay, useless EF calls get removed. Do all of your maps work now?

@lolreported I added a "fix" for the .wav problem, but in general .wavs are just problematic. The world editor uses a lossy compression for wavs with around 30% compression ratio, which would turn to 90%+ ratio if we just used lossless zlib. If we don't change the sector size we can just keep the files of course, but we want to increase the sector size to better compress all other files. Recompression is also not really feasible, because the first sector of wavs is not lossy compressed (assumably to retain header info) and with a big sector size there only is one sector per file, thus the lossy compression wouldn't even get into action.
Depending on how much space the wavs take up in the map a higher sector count could still be okay, so I will add an option to force recompression in the future.
For now it will just show a message when .wav files have been detected, and keep the sector size and .wav files as is.
I don't think there is much point in further research due to the generous map file size constraint in battle.net.
 
Level 19
Joined
Aug 16, 2007
Messages
871
@lolreported I added a "fix" for the .wav problem, but in general .wavs are just problematic. The world editor uses a lossy compression for wavs with around 30% compression ratio, which would turn to 90%+ ratio if we just used lossless zlib. If we don't change the sector size we can just keep the files of course, but we want to increase the sector size to better compress all other files. Recompression is also not really feasible, because the first sector of wavs is not lossy compressed (assumably to retain header info) and with a big sector size there only is one sector per file, thus the lossy compression wouldn't even get into action.
Depending on how much space the wavs take up in the map a higher sector count could still be okay, so I will add an option to force recompression in the future.
For now it will just show a message when .wav files have been detected, and keep the sector size and .wav files as is.
I don't think there is much point in further research due to the generous map file size constraint in battle.net.
It sounds like a very complicated problem and just ignoring the .wav files altogether is fine, in my opinion. As you said, the map file size is huge nowadays and file sizes aren't that big of a problem.

I'll report back how it works when I'm done updating my map. :)


EDIT
Used the recent version of the tool and the map now crashes when loading, using Extreme Compression or not doesn't matter. Version 0.5.7 works without issue (not using Extreme Compression to avoid the sound glitch).

crash.PNG
 
Last edited:
Level 21
Joined
Jan 1, 2009
Messages
1,554
EDIT
Used the recent version of the tool and the map now crashes when loading, using Extreme Compression or not doesn't matter. Version 0.5.7 works without issue (not using Extreme Compression to avoid the sound glitch).

Thanks for the report, it was yet another bug in the wts inliner :D
I pushed 0.5.9 with a fix.
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Last edited:
Level 4
Joined
Sep 25, 2017
Messages
56
Did you read the first post?



The editor crashes when attempting to open the map.
Maps saved with the reforged editor aren't supported until this ticket is implemented: New reforged W3I format · Issue #48 · inwc3/wc3libs

You can run from console for some logs.
Running from console gives me this:

upload_2019-12-20_18-18-23.png




This is not a reforged map, its a 1.28.5 unprotected korean map. As for editor crashing, if you remove trigger file (wtg) it will work. I am not sure if they use a modified world editor in korea or they have some kind of plugin, but the issue I believe is that it uses in gui some functions which are not defined in either blizzard.j or common.j. In a wc3 game it works because it seems to override blizzard.j with the maps custom blizzard.j inside the scripts folder, but it would seem world editor doesn't do that and behaves differently. Maybe just changing the blizzard.j file inside war3.mpq would make it work but never tried as I never use world editor.

Anyway map is unprotected and editable, have been using w3x2lni for a while already with no issues but wanted to try out this one. It's a complex map that will probably have a lot of border cases (most hive programs like leak checker won't work with it), so I thought it would be a good example for you to find bugs. But if you deem it too uncommon to bother fixing thats okay too, will keep using w3x2lni. As for the gui, is the map name and author supposed to show the TRIGSTR number? Showing the wts values would be nicer (Eden RPG S2 4.5A Fix6 and scvscvgo)
 
Level 17
Joined
Jan 12, 2011
Messages
1,487
"inline strings" makes all strings appear on the same line so it's not just internal, basically useless setting since loading screens and even HUD text gets fucked up.


But there is a much more broken thing, "optimize code" breaks the code, like actually breaks it, I was optimizing an rpg and the button to talk to a villager started causing the town to go hostile lol
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
so I thought it would be a good example for you to find bugs. But if you deem it too uncommon to bother fixing thats okay too, will keep using w3x2lni.
You should've simply given me all that info up front. I'm not a fan of spending time on maps that are somehow broken or protected on purpose.

As for the gui, is the map name and author supposed to show the TRIGSTR number? Showing the wts values would be nicer (Eden RPG S2 4.5A Fix6 and scvscvgo)
It shows the raw values from the .w3i - whether those are inline values or TRIGSTR. I suppose a conversion could be added.

"inline strings" makes all strings appear on the same line so it's not just internal, basically useless setting since loading screens and even HUD text gets fucked up.

What do you mean by "not just internal" ? It looks like the lineending stuff depends on some file encoding shenanigans - it's an issue I will check soon,
but obviously not widespread as many have used this tool successfully by now.

But there is a much more broken thing, "optimize code" breaks the code, like actually breaks it, I was optimizing an rpg and the button to talk to a villager started causing the town to go hostile lol

This tool is making some more extreme optimizations, which can - even though the maps me and others have tested work - break something in certain edge cases that haven't been found.
A situation like you describe can be caused by something miniscule in the output code.

If you want to help, attach/send your map, omit the banter.
 
Level 17
Joined
Jan 12, 2011
Messages
1,487
You should've simply given me all that info up front. I'm not a fan of spending time on maps that are somehow broken or protected on purpose.


It shows the raw values from the .w3i - whether those are inline values or TRIGSTR. I suppose a conversion could be added.



What do you mean by "not just internal" ? It looks like the lineending stuff depends on some file encoding shenanigans - it's an issue I will check soon,
but obviously not widespread as many have used this tool successfully by now.



This tool is making some more extreme optimizations, which can - even though the maps me and others have tested work - break something in certain edge cases that haven't been found.
A situation like you describe can be caused by something miniscule in the output code.

If you want to help, attach/send your map, omit the banter.

Not my map, it's actually a map that is here on hive and I was fixing a crash issue since the map is popular but the author hasn't updated it, called northrend bound, it's unprotect it you can check yourself, try doing hte extreme optimization on it and then talk to a villager and the village will go hostile.


Also do the string thing on the map, you'll see the issues on both the loading screen and UI screens.
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Okay I pushed 0.5.10 with WTS inlining fixes.

Anyway map is unprotected and editable

It's pretty far from that. The existing war3map.j inside the map doesn't even compile on its own, it requires functions defined in a custom "blizzard.j" file.
The game is apparently okay with that, but of course most tools aren't. I can look into adding such a detection, but it's not really a priority for me.

Also do the string thing on the map, you'll see the issues on both the loading screen and UI screens.

The strings are fixed, as I suspected the issue originated from an underlying library.

try doing hte extreme optimization on it and then talk to a villager and the village will go hostile.

I could reproduce the issue, but am not sure what causes it. There are many calls to alliance modifications in the code, it might even be some anti tamper mechanic.
If you can pinpoint what part of the code causes it I can look into it further.
 
Level 1
Joined
Feb 1, 2020
Messages
1
I'm having problem after protecting my map since my map contains characters other than English. I messed with some settings and found out it was caused by the feature "Inline Trigger Strings". It seems when running with this feature checked, the encoding will mess up the texts in triggers. Other than that, it is working fine :)
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
I'm having problem after protecting my map since my map contains characters other than English. I messed with some settings and found out it was caused by the feature "Inline Trigger Strings". It seems when running with this feature checked, the encoding will mess up the texts in triggers. Other than that, it is working fine :)

If you send me the map I can take a look
 
Level 21
Joined
Jan 1, 2009
Messages
1,554
Hey trying to protect my map it just gets stuck on Parsing Script forever, I gave it 2 hours two times with Extreme Compression and Without.
My map
-v1.31
-45,214 lines of code
-Lua
-25mb loads of Imports, models, Icons, wav, Mp3, textures

Attach/send map. That info alone doesn't really help too much, but thanks for the report.
You can also run from console to see further output. It won#t ever take more than ~5 minutes so no need to wait that long.
 
Level 7
Joined
Apr 1, 2019
Messages
154
-with/without extreme compression
-remove world editor data
-inline trigger strings
-Minify Lua

Edit, if I use "Inline Trigger Strings" it gets stuck on Parsing Script. Everything Else works

Using only "Minify Lua" I can open in the editor but in map lobby when I select it I get the error "There was an error reading the map file"
 
Last edited:
Level 21
Joined
Jan 1, 2009
Messages
1,554
-with/without extreme compression
-remove world editor data
-inline trigger strings
-Minify Lua

Edit, if I use "Inline Trigger Strings" it gets stuck on Parsing Script. Everything Else works

Using only "Minify Lua" I can open in the editor but in map lobby when I select it I get the error "There was an error reading the map file"

I see, the "Inline Trigger Strings" option is probably screwing up the lua code.
If only "minify lua" doesn't load, it must mean that it makes the map code invalid for some reason. Will have to further investigate this.
 
Top