• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • 🏆 Hive's 6th HD Modeling Contest: Mechanical is now open! Design and model a mechanical creature, mechanized animal, a futuristic robotic being, or anything else your imagination can tinker with! 📅 Submissions close on June 30, 2024. Don't miss this opportunity to let your creativity shine! Enter now and show us your mechanical masterpiece! 🔗 Click here to enter!

New Script Optimizer

Status
Not open for further replies.
Level 23
Joined
Jan 1, 2009
Messages
1,610
Froptimizer

Froptimizer is a small tool to optimize warcraft3 script- and mapfiles (.j,.w3x,.w3m)

Features

  • Removing useless stuff like comments, whitespaces and empty lines
  • Compressing variable and function names
  • Replacing some JassVars to have the smallest possible filesize

Pros&Cons @ vex'
Pros:
  • Compressing names generates with a more advanced charmap (a-z, A-Z, _, 0-9)
  • Namecompression with custom natives works
  • name compression with ExecuteFunction and TRVE uses a StringPattern to leave the code functional
Cons:
  • Only optimizes script (yet) and doesn't provide any protection
  • Should be used in combination with vex'

Coming Features:
  • Some sort of BJ/GUI removal
  • Removing unused variables & functions
  • maybe numbershortening

I already have some more feature plans, but would like your guys feedback on this.

Requirements

As I'm doing this in Java, you need to have the latest Java Runtime Enviroment(1.6 or higher) installed on your system.


UI
Screenshot of the current GUI
froptimizer.JPG


Please post requests, comments and/or constructive criticism.

Download in Attachments.
(It's 10 mb because I had to import many dependent libs)

Also there is no real error-handling at the moment. When the progressbar doesnt complete/the labael doesn't say "Finished!" report the bug with the labelmsg and map.
 

Attachments

  • Froptimizer.7z
    9.3 MB · Views: 190
Last edited:
Is Java Version 7 different than JRE? I have the latest JRE on my system. Why not compile it to an Exe so that it doesn't need Java?

+12 rep when this thing is released and shown to be working right.

Advice for ExecuteFunc and TriggerRegisterVariable event:

if you have SCOPE_PRIVATE + "variableName" in vJass it compiles to "libraryName__" + "variableName", which breaks Vexorian's optimizer. If yours treated constant string concatenation better than his then we can ignore this problem. Obviously something like "FunctionName" + I2S(index) is not going to compile intuitively in which case it gets annoying like finding every function that could start with that function name and has an integer-type suffix, and then shortening both the string and then all compatible functions except for, of course, the integers appended to the end of the function name. Obviously then TRVE and EF relevent strings should compile first.

Vexorian currently uses all-capital variable names if there are cases where TRVE and EF are found, but you don't have to be so closed minded. All you have to do is make sure that any TRVE and EF strings do not have any case clashing, problem solved.

Also every call of Condition() should be turned into Filter() because it does the same but is a shorter function name.

Don't worry about things that already exist like compressing model files, but if you have a passion for that then fine.
 
Level 23
Joined
Jan 1, 2009
Messages
1,610
Is Java Version 7 different than JRE? I have the latest JRE on my system. Why not compile it to an Exe so that it doesn't need Java?
JRE 7 is what you need. http://www.oracle.com/technetwork/java/javase/downloads/java-se-jre-7-download-432155.html
Advice for ExecuteFunc and TriggerRegisterVariable event:

if you have SCOPE_PRIVATE + "variableName" in vJass it compiles to "libraryName__" + "variableName", which breaks Vexorian's optimizer. If yours treated constant string concatenation better than his then we can ignore this problem. Obviously something like "FunctionName" + I2S(index) is not going to compile intuitively in which case it gets annoying like finding every function that could start with that function name and has an integer-type suffix, and then shortening both the string and then all compatible functions except for, of course, the integers appended to the end of the function name. Obviously then TRVE and EF relevent strings should compile first.
Well I didnt think about that so much, but if you call EF terribly like that ("FunctionName" + I2S(index)) :/
Also how does:
if you have SCOPE_PRIVATE + "variableName" in vJass it compiles to "libraryName__" + "variableName", which breaks Vexorian's optimizer.
Break exactly?
Vexorian currently uses all-capital variable names if there are cases where TRVE and EF are found, but you don't have to be so closed minded. All you have to do is make sure that any TRVE and EF strings do not have any case clashing, problem solved.
Did I understand that right, that TRVE and EF can't take a string as argument that has case clashing?
Also every call of Condition() should be turned into Filter() because it does the same but is a shorter function name.
okay
Don't worry about things that already exist like compressing model files, but if you have a passion for that then fine.
Shadowdemon wanted to update blplap with a commandline function, so I thought about adding it. It's not yet implemented.
 
Well if you pass to TRVE the variable "variable" it will be treated the same as if you had a variable named "VariAble". Same with ExecuteFunc.

Vexorian's optimizer doesn't recognize concatenation at all. If you have:

JASS:
function HelloWorld takes nothing returns nothing
endfunction
...ExecuteFunc("Hello" + "World")

His optimizer will turn it into:

JASS:
function A takes nothing returns nothing
endfunction
...ExecuteFunc("Hello" + "World")
 
OMG, I love you <3

Tip: To avoid mistakes that Vex made, you could look at the changelog of his optimizer.

Here's another optimization option you could add:
"Nes Mode" xD

It looks for boolean expressions like this: x==0
and changes them to: 0==x
x==null -> null==x

If a function is called only once, inline it.

If a local isn't used, delete it.
Same for globals.

Good luck with this Frotty :3
+3 rep :D
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
You could use some constants defined by the user, in a GUI or in a configuration text file (or just both which is better).
Like them :

SAFE_MODE = true/false

If it's true it doesn't try to shorten up real variables and all functions which could be called with an ExecuteFunc (string concatenation).

For Example :

JASS:
call ExecuteFunc("MyF"+I2S(<integer variable>))
call ExecuteFunc("<string variable>" + "MyF"+I2S(<integer variable>))

function MyF1 ... // won't be shorten up

function ThisIsMyF3 ... // won't be shorten up

function MyF ... // will be shorten up 

function MyF1a // will be shorten up

The first function won't be shorten up because it match the pattern of the first ExecuteFunc call, and there is no safe way to be sure of the variable value part (not a constant).

The second function won't be shorten up because it match the pattern of the second ExecuteFunc call, and there is no safe way to be sure of the variable value part (not a constant).

The two other functions will be shorten up because they don't follow any ExecuteFunc call pattern (even if that are closed to it).

Ofc it's the same for real variable events (TriggerRegisterVariableEvent), but i think you would safely ignore this safety for real locals (why the hell would you link a such event on a local variable ?)

Or if you think it's just to much you can also support some prefix or postfix like "__protected" :

function ThisIsMyFunction__protected // won't be shorten because of the postfix "__protected".

These postfix/suffix could be included in a jass preprocessor with a special keyword, or directly used in jass by the user.

Again you are welcome to support both systems :p

About variables and functions shorten up, don't forget the character "_", even if this one can't be used in the first character, the first character has to be a letter, and the last one can be anything except "_" (letter or number).
Also don't forget you can user uppercases and lowercases.
In fact most of real maps will have at max 2 characters length name for variables and functions, using the full power of this special base.

You should reserve one character length for locals.

Remember that in newest patches the globals have the priority against locals, meaning variable shadowing doesn't work anymore, and if you declare a local variable which has the same name of a global, the global will be used.

Though, you should avoid this conflit, 2 character length seems enough fair.

And again with a jasspreprocessor (or not) and a postfix/prefix you could reserve the available one character for functions/variables defined by the user (speedfreak power, like the word "register" in C, even if of course isn't the same at all in the background)

Something really important, you should make your tool usable without a GUI in command line (this way you can include the optimisation directly in the JNGP or whatever, even if i've only sucessfully executed a .exe file, not a java one or whatever else, but maybe i did it wrong)
Also your tool should be able to work with a text file, not only with a map.

Care also that the user could inlude in his map a custom common.j (and maybe blizzard.j,common.ai) with new custom natives.

Good luck.
 
Level 23
Joined
Jan 1, 2009
Messages
1,610
It looks for boolean expressions like this: x==0
and changes them to: 0==x
x==null -> null==x
Will be easy to add
If a function is called only once, inline it.
planned that
If a local isn't used, delete it.
Same for globals.
Maybe this will be included with unused code removals


You could use some constants defined by the user, in a GUI or in a configuration text file (or just both which is better).
Like them :

SAFE_MODE = true/false

If it's true it doesn't try to shorten up real variables and all functions which could be called with an ExecuteFunc (string concatenation).
There is an .ini file for all options available in the GUI

Or if you think it's just to much you can also support some prefix or postfix like "__protected" :

function ThisIsMyFunction__protected // won't be shorten because of the postfix "__protected".

These postfix/suffix could be included in a jass preprocessor with a special keyword, or directly used in jass by the user.

Again you are welcome to support both systems :p
Possible Idea
About variables and functions shorten up, don't forget the character "_", even if this one can't be used in the first character, the first character has to be a letter, and the last one can be anything except "_" (letter or number).
Also don't forget you can user uppercases and lowercases.
In fact most of real maps will have at max 2 characters length name for variables and functions, using the full power of this special base.
I know, tho i'll just stay with a-z and A-Z. With 3 chars there are > 14000 possible combinations.
Something really important, you should make your tool usable without a GUI in command line (this way you can include the optimisation directly in the JNGP or whatever, even if i've only sucessfully executed a .exe file, not a java one or whatever else, but maybe i did it wrong)
Also your tool should be able to work with a text file, not only with a map.
Commandline-maybe
and y you can open .j files
Care also that the user could inlude in his map a custom common.j (and maybe blizzard.j,common.ai) with new custom natives.
Since rtc is broken there is no reason at all to use custom native files.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Ignoring the numbers and "_" is bad :

Only letters :

1 character = 52¹ = 52 combinations
2 characters = 52² = 2704 combinations
3 characters = 52³ = 140608 combinations

Jass grammar :

1 character = 52¹ = 52 combinations
2 characters = 52*62 = 3224 combinations
3 characters = 52*63*62 = 203112 combinations

I wouldn't say it's negligible, even if i'm agree that even only with letters, you would hardly need more than 3 characters, but in map with really huge scripts you could have to use 3 characters instead of 2, sooner than following the jass grammar.

And it's not that hard to use this special base, now i'm agree that in maps which are not heavily scripted that doesn't matter.

About RTC, it's just a general purpose i was talking, you can include some common.ai natives in a custom common.j, for example UnitAlive.
And nothing in programming world is dead for ever and ever.
But i suppose if you list the natives of common.j and common.ai, it will be enough for now, i'm just trying to make it work in every situations (suggestions)

Oh and btw you shoud'nt include speedfreak senseless optimizations if someone says it without a valid benchmark providen (Nestharus or not)
 
Last edited:
Level 17
Joined
Apr 27, 2008
Messages
2,455
That reminds me by "locals" i meant also function arguments (takes ...)

Also inlining every constant globals is bad, you should inline only constants value (basically not a function call, except for the code type maybe)

constant timer Tim = CreateTimer() // fail it will create a new timer each time Tim is used ...

You could argue that constant handles are useless, but at least they are (i mean it's possible to use them)
 
Level 10
Joined
May 27, 2009
Messages
494
well i suggest something like before compiling the map, check first for the listfile of the imports, if there are any unused imports, delete them. perhaps this maybe pain in the computer since it will look for both in the units and trigger strings but kinda useful especially unused icons

.slk optimization please
 
Let's not put too much workload on the Froptimizer all at once.

Personally I feel the priorities should be:

Compress names/remove comments/whitespace and inline code, real, integer, string and boolean constants.

Then, optimization things, like inlining functions which are only called once, for example.

Afterward, then it's the non-JASS related things.
 
Level 7
Joined
Dec 3, 2006
Messages
339
well i suggest something like before compiling the map, check first for the listfile of the imports, if there are any unused imports, delete them. perhaps this maybe pain in the computer since it will look for both in the units and trigger strings but kinda useful especially unused icons


If your gonna do this. Make checks for files imported with TerrainArt\ . . . (which are tiles). And LoadingScreen.mdx, and FullScreen.blp (for the loading screen ofc). As these aren't used in either object editor or trigger editor. Also remember that trigger strings always have two \ 's for paths.
 
Level 23
Joined
Jan 1, 2009
Messages
1,610
well i suggest something like before compiling the map, check first for the listfile of the imports, if there are any unused imports, delete them. perhaps this maybe pain in the computer since it will look for both in the units and trigger strings but kinda useful especially unused icons

.slk optimization please
if you got unused imported stuff youre doing it wrong.
Use Vex for .slk opt since i have no idea what he is doing with them.

A thing to add in the priorities list
make it a stand alone executable file D:

well first version may be java one o.o
Whats the problem with a .jar?
Let's not put too much workload on the Froptimizer all at once.

Personally I feel the priorities should be:

Compress names/remove comments/whitespace and inline code, real, integer, string and boolean constants.

Then, optimization things, like inlining functions which are only called once, for example.

Afterward, then it's the non-JASS related things.
Yup, I'll focus on Script Optimization right now.

EF and TRVE are kinda nasty, like why would you use concatenation if not with a variable part. Also this ("abc" + variable "cde") would be ass to use with compress names.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,222
Just because a .exe is easier to spread around (doesn't require
users to download JRE on top).
Only for windows machines. Mac machines should come with java and they can not run .exe files. Remember that a .exe is a windows executable only.

Thus he has eithor 2 choices.
1. Use java (like he is) as that is cross platform and expect all windows users to have an appropiate JRE installed.
2. Use C/C++ and create different executables for different opperating systems.

2. Would provide optimum performance due to its native level execution on the appropiate platform. However as a jass script optimizer is hardly computationally challenging nor is it a realtime system, such performance gain is pointless and does not justify the inconvenience of cross platform C/C++ development.
 
Level 16
Joined
Aug 7, 2009
Messages
1,403
I should have simply pressed the subscribe button, but I felt like telling you that I'm excited to see this new optimizer may be a good idea. So hell yea, here it is.

Finally the most annoying bugs of Vexrorian's optimizer are getting fixed. Thought this day will never come.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
I don't know if you're still on it, but maybe fix lame jass things would be good, such as the handle local leak :

JASS:
function Test takes ... returns unit
    local unit u = ...
    // stuff with u
    return u // awesome we will got an handle id leak when the unit will be removed
endfunction

How to fix it :

JASS:
globals
    unit Temp_unit = null
endglobals

function Test takes ... returns unit
    local unit u = ...
    // stuff with u
    set Temp_unit = u
    set u = null
    return Temp_unit
    // no more handle id leak, it could still make a tiny memory one in some
    // cases, but seriously we can't handle that
endfunction

But yes it's in fact quite the opposite of a script optimisation, so it may not fit with a script optimizer.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Because handle id leaks can cause a performance issue later, not because how many memory it uses, but because it slows down the jass virtual machine.
Ofc it will happen only if there is a massive handle id leak.

For me, that's something that should be fixed by some tool, coz it's way too anoying for a manual process, and senseless anyway.
 

BBQ

BBQ

Level 4
Joined
Jun 7, 2011
Messages
97
People, people, people. The bug with IsUnitType() occurs only when UNIT_TYPE_TOWNHALL is used. It's one of the many "superstitions" that have been present in the JASS world since forever.
JASS:
library Test
{
    function B2I(boolean whichBoolean) -> integer
    {
        return whichBoolean;
        return 0;
    }
    
    function onInit()
    {
        unit townHall = CreateUnit(Player(0xF), 'htow', 0., 0., 0.);
        
        BJDebugMsg("Testing for UNIT_TYPE_TOWNHALL:\n"
        + I2S(B2I(IsUnitType(townHall, UNIT_TYPE_TOWNHALL))) + "\n" /* Prints 64. The bug is present ONLY here. */
        + I2S(B2I(IsUnitType(townHall, UNIT_TYPE_TOWNHALL) == true))); /* Prints 1. No bug. */
        
        KillUnit(townHall);
        
        BJDebugMsg("\nTesting for UNIT_TYPE_DEAD:\n"
        + I2S(B2I(IsUnitType(townHall, UNIT_TYPE_DEAD))) + "\n" /* Prints 1. No bug. */
        + I2S(B2I(IsUnitType(townHall, UNIT_TYPE_DEAD) == true))); /* Prints 1. No bug. */
        
        townHall = null;
    }
}
 
I have heard it reported for UNIT_TYPE_HERO as well.

To test it, we can use this (I will do some tests in some hours when I am home)

JASS:
struct Tester extends array
    static method check takes nothing returns boolean
        call SelectUnit(GetTriggerUnit(), false)
        return IsUnitType(GetTriggerUnit(), UNIT_TYPE_HERO)
    endmethod
    static method run takes nothing returns nothing
        call BJDebugMsg("True")
    endmethod
    static method onInit takes nothing returns nothing
        local unit u = CreateUnit(Player(0), 'Hpal', 0, 0, 0)
        local trigger t = CreateTrigger()
        call TriggerAddCondition(t, Filter(function thistype.check))
        call TriggerAddAction(t, function thistype.run)
        call TriggerRegisterUnitEvent(t, u, EVENT_UNIT_SELECTED)
        set t = null
        set u = null
    endmethod
endstruct
 

BBQ

BBQ

Level 4
Joined
Jun 7, 2011
Messages
97
I remember testing all of them, and only UNIT_TYPE_TOWNHALL was buggy. I just tested UNIT_TYPE_HERO (as per your suggestion), and it worked as expected.
 
Ok, I have updated it. Can you test it again?

JASS:
struct Tester extends array
    static method check takes nothing returns boolean
        call SelectUnit(GetTriggerUnit(), false)
        return IsUnitType(GetTriggerUnit(), UNIT_TYPE_TOWNHALL)
    endmethod
    static method run takes nothing returns nothing
        call BJDebugMsg("True")
    endmethod
    static method onInit takes nothing returns nothing
        local unit u = CreateUnit(Player(0), 'htow', 0, 0, 0)
        local trigger t = CreateTrigger()
        call TriggerAddCondition(t, Filter(function thistype.check))
        call TriggerAddAction(t, function thistype.run)
        call TriggerRegisterUnitEvent(t, u, EVENT_UNIT_SELECTED)
        set t = null
        set u = null
    endmethod
endstruct
 
I think at least an option to remove all "==true" would be nice for a future edition, and "== false" would only be replaceable with the "not" keyword if you were somehow able to detect the complete compared value on the left, which could get ugly with things like "if blahandblah == false and blahnotblah == false and blahorblah == false" where you have to decypher the keywords as well, I just think the removal of "==false" is not worth the effort though removing ==true is pretty much find & delete.
 
Status
Not open for further replies.
Top