1. Join other hivers in a friendly concept-art contest. The contestants have to create a genie coming out of its container. We wish you the best of luck!
    Dismiss Notice
  2. The Melee Mapping Contest #4: 2v2 - Results are out! Step by to congratulate the winners!
    Dismiss Notice
  3. We're hosting the 15th Mini-Mapping Contest with YouTuber Abelhawk! The contestants are to create a custom map that uses the hidden content within Warcraft 3 or is inspired by any of the many secrets within the game.
    Dismiss Notice
  4. The 20th iteration of the Terraining Contest is upon us! Join and create exquisite Water Structures for it.
    Dismiss Notice
  5. Check out the Staff job openings thread.
    Dismiss Notice

Jass Hot Code Reload

Discussion in 'Warcraft Editing Tools' started by LeP, Mar 20, 2019.

  1. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0

    Jass Hot Code Reload



    A compiler to allow hot code reload in WarCraft 3. This means you can update
    your map script and see the changes in a running game without restarting it.
    Do note though that this is alpha software. Expect bugs and make backups.
    This works without memhack.

    Compiler usage



    The compiler has two commands: init and update. The first command you execute
    has to be the init command. The init command expects atleast three arguments:
    Path to common.j, path to Blizzard.j and path to your maps war3map.j .
    If your map was compiled by jasshelper you should use the --jasshelper flag
    for the compiler to work correctly.

    Code (Text):

    $ jhcr init common.j Blizzard.j war3map.j --jasshelper
     
    The above usage will create a file jhcr_war3map.j. You should import this
    as your maps new war3map.j.

    Now you can make changes to your map. Once you've done this, get the new
    war3map.j and call jhcr again.

    Code (Text):

    jhcr update war3map.j --preload-path Path\To\CustomMapData --jasshelper
     
    Again, if you've used jasshelper to compile your map, pass the --jasshelper flag.
    The update command will create a file called JHCR.txt in the path you've specified.

    As this tool operates only on the maps script and not on the map itself you
    have to extract and insert the war3map.j yourself. Personally i use zezulas
    mpqeditor via the command line.

    For example this script could be used to create the map tmp.w3x from the orignal
    map test.w3x:
    Code (Text):

    cp test.w3x tmp.w3x
    MPQEditor extract tmp.w3x war3map.j
    jhcr init common.j Blizzard.j war3map.j --jasshelper
    pjass common.j Blizzard.j jhcr_war3map.j
    cp test.w3x tmp.w3x
    MPQEditor add tmp.w3x jhcr_war3map.j war3map.j
     
    Now you can start WarCraft 3 with the --loadfile argument to run tmp.w3x.

    And this can be used to load the updates:
    Code (Text):

    cp test.w3x tmp2.w3x
    MPQEditor extract tmp2.w3x war3map.j
    jhcr update war3map.j --jasshelper --preload-path 'C:\Users\lep\Documents\Warcraft III\CustomMapData\'
     

    Map preparation



    The runtime has to get initialized. Use
    call ExecuteFunc("JHCR_Init_init")
    for example at 0 elapsed game time.

    • init
      • Events
        • Time - Elapsed game time is 0.00 seconds
      • Conditions
      • Actions
        • Custom script: call ExecuteFunc("JHCR_Init_init")


    To load updates to the mapscript use
    call ExecuteFunc("JHCR_Init_parse")
    when appropiate. I use pressing escape.

    • reload
      • Events
        • Player - Player 1 (Red) skips a cinematic sequence
      • Conditions
      • Actions
        • Custom script: call ExecuteFunc("JHCR_Init_parse")
     

    Attached Files:

    Last edited: Jun 9, 2019
  2. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,392
    Resources:
    10
    Models:
    3
    Tools:
    1
    Maps:
    4
    Tutorials:
    1
    Wurst:
    1
    Resources:
    10
    Cool stuff. Will tests again when I get the time :)
     
  3. Chaosy

    Chaosy

    Joined:
    Jun 9, 2011
    Messages:
    10,562
    Resources:
    17
    Maps:
    1
    Spells:
    10
    Tutorials:
    6
    Resources:
    17
    Sounds awesome. How does it work?
     
  4. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    It's a multi step process. init takes the map script and injects the runtime and transforms the map script in such a way that the runtime can work with it. update looks which functions have changed since the last update or init and compiles those to a simple language. You can look at the generated code in human readable form by passing the --asm flag to update. This code is then loaded into the map via preload and interpreted by the runtime.
     
  5. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,270
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Looking at some of the above, I am guessing it exploits preload calls.
     
  6. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    i don't know what that's supposed to mean
     
  7. Chaosy

    Chaosy

    Joined:
    Jun 9, 2011
    Messages:
    10,562
    Resources:
    17
    Maps:
    1
    Spells:
    10
    Tutorials:
    6
    Resources:
    17
    Hm, that does not tell me much.

    My concern would be if it uses some sort of 'clever game mechanics' in order to function, meaning a bugfix from blizzard could render it useless.
    This really seems mind blowing as I cannot imagine how to update I'd imagine that updating the map's script file does not make reload it into memory.
    So since I cannot imagine how to do it, I kinda think there is some questionable method at play.

    My best guess is that the program does some memory editing, but then I do not see the point of using custom scripts to use the system.
    Needless to say I do not understand x)
     
  8. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    You could assume that i know better what i wrote than you))
    You can look at the generated map script and search for memory hack stuff. I even developed this on 1.30.4. Also i never said it updates the maps script file.
    I would advise you to reread my original post, as it really explains how it's done.
    But if you've got other questions i'm happy to answer.
    Unless Blizzard does something stupid like limiting the amount of nested ifs or something this should just continue to work with any coming patch as it really just is plain jass.

    Also please give code arrays Blizzard. That would reduce the output of this by so so much.
     
  9. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,270
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    By the sounds of LeP's explanation what it does is compile JASS into some custom language, a special form of bytecode, which then gets fed into a realtime interpreter. The bytecode is read using preload like save/load codes are and the interpreter changes to using the new code when an update is applied. The underlying JASS and JASS bytecode is not modified.
     
  10. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    517
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    This is pretty awesome. It took me some time to set things up but I am pretty sure that hot reloading is faster than exiting the game, rebuilding the map and starting the game again.

    I am not sure how many xs of overhead it is to run a bytecode vm (Lep) in a slow bytecode vm (Jass) but... hey it works!

    The setup I ended up with (more or less):
    Code (Text):

    // init
    clijasshelper.exe --scriptonly  --debug common.j blizzard.j main.j war3map.j
    jhcr.exe init common.j blizzard.j war3map.j --jasshelper
    clijasshelper.exe common.j blizzard.j jhcr_war3map.j test.w3x

    // start map
    "Warcraft III.exe" -window -loadfile test.w3x

    // should we 'update' after starting the map?

    loop
        // update
        clijasshelper.exe --scriptonly  --debug common.j blizzard.j main.j war3map.j
        jhcr.exe update war3map.j --jasshelper --preload-path "C:\Users\lep\Documents\Warcraft III\CustomMapData\

        // This step can get annoying... maybe an optional jhcr.exe flag can be set
        // to send an Esc key to Warcraft III.exe's window automatically?
        // I guess this would require some Windows shenanigans.
        //
        call ExecuteFunc("JHCR_Init_parse") // e.g by pressing the Esc key

        // make changes to main.j etc., then trigger an update
    endloop

     
     
    Last edited: Mar 21, 2019
  11. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    Speed seems OK to me; in my albeit small tests i reloaded a timer function which was executed 64 times a second. i also designed everything in such a way that it would be rather fast to execute (ofc there is always room to improve).
    No need to directly update after init. in fact that would probably result in empty bytecode.
    just run update when you changed something in your code.

    also i don't plan on adding automatic reload on the warcraft side. you could probably use autohotkey or something after calling jhcr update. but i dont know about that stuff.
     
    Last edited: Mar 21, 2019
  12. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    517
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    It seems that adding a global variable (or modifing its initial value) is not reflected after an "update".
    On the other hand, it seems that new functions can be declared (functions that were not present at "init" time).

    It's kind of hard to tell but I don't think integers come before the other types in the dispatch of the "Set/GetGlobal/Array" instructions. I think integers are the most common type.

    PS: The productivity boost from this tool, scriptwise, is nuts! Most importantly, I think it brings back the fun of scripting!
     
  13. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    Yes, the current limitations are that you can define and use new globals but they're initialized to
    0/0.0/""/null
    . Also you can define up to 100 new functions and use them except in
    ExecuteFunc
    . I plan on fixing both.

    Also the reason i wrote this tool is exactly the fast iteration process when developing; always restarting maps costs so much time. And the first successfull reload was quite magical.
    I don't think changing
    integer
    s id to come first would result in any meaningfull performance boost but i will keep it in mind.
     
  14. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,270
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Most likely due to some optimization issues with how Warcraft III loads data. There is no other reason why a map should take more than 15 seconds to load in 2019.
     
  15. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    517
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    I think I found a few bugs.

    Single quote integers:
    Code (vJASS):

    private function on_reload takes nothing returns nothing
        call writeln("Reload OK")
        // call UnitAddItemById(g_unit, 'bspd') // not ok
        call UnitAddItemById(g_unit, 0x62737064) // ok
    endfunction
     


    Negation:
    Code (vJASS):

    private function on_reload takes nothing returns nothing
        local integer a
        local integer b
        call writeln("Reload OK")

        set a = 0
        set b = -48 // 0 - 48 seems ok

        call writeln("a: " + I2S(a))
        set a = a - b
        call writeln("a: " + I2S(a))
        set a = a - b
        call writeln("a: " + I2S(a))
    endfunction
     


    Button stops working after an update (I have no idea what goes wrong):
    Code (vJASS):

    private function draw_loop takes nothing returns nothing
        local integer x
        local integer y
        local integer w
        local integer h
        local Bd_Mouse m
        local Bd_Mbtn lmb

        set m = Bd_Mouse(0)
        set lmb = m.lmb

        call bd_draw_begin(0)
        call bdw_auto_wid_base(1000)

        set w = 256
        set h = 64

        set x = 0 - 1024
        set y = 768

        if bdw_text_button("click me", x, y, w, h, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, false, lmb) then
            call writeln("A") // change "A" to "B" then trigger an "update"
        endif

        call bd_draw_end()
    endfunction

    private function on_start takes nothing returns nothing
        call TimerStart(CreateTimer(), 1.0/10.0, true, function draw_loop)
    endfunction
     



    PS: Sending the Esc key to Warcraft III's window automatically requires some Windows minutia but doesn't seem that bad:
    Code (C):

    #pragma comment(lib, "kernel32.lib")
    #pragma comment(lib, "user32.lib")

    using u16 = unsigned short;
    using u32 = unsigned int;
    using s32 = int;
    using bool32 = s32;
    // we should probably static assert that sizeof(X) is what we wanted

    using HWND = void*;

    struct MOUSEINPUT {
        s32 dx;
        s32 dy;
        u32 mouseData;
        u32 dwFlags;
        u32 time;
        void* dwExtraInfo;
    };

    struct KEYBDINPUT {
        u16 wVk;
        u16 wScan;
        u32 dwFlags;
        u32 time;
        void* dwExtraInfo;
    };

    struct HARDWAREINPUT {
        u32 uMsg;
        u16 wParamL;
        u16 wParamH;
    };

    struct INPUT {
        u32 type;
        union {
            MOUSEINPUT mi;
            KEYBDINPUT ki;
            HARDWAREINPUT hi;
        };
    };

    #define cast(T, e) (T)(e)
    #define WINAPI __stdcall

    extern "C" {
        void exit(s32);
        s32 printf(char const* fmt, ... );

        HWND WINAPI FindWindowA(char const* class_name, char const* win_name);
        bool32 WINAPI SetForegroundWindow(HWND w);
        u32 WINAPI SendInput(u32, INPUT*, s32 key);
    } // extern "C"

    void
    send_key(s32 k) {
        enum {
            INPUT_KEYBOARD = 1,
            KEYEVENTF_KEYUP = 2,
        };

        INPUT input = {};
        input.type = INPUT_KEYBOARD;
        input.ki.wVk = cast(u16, k);

        // press the key
        input.ki.dwFlags = 0;
        SendInput(1, &input, sizeof(INPUT));

        // release the key
        input.ki.dwFlags = KEYEVENTF_KEYUP;
        SendInput(1, &input, sizeof(INPUT));
    }

    s32
    main(s32 args_len, char const** args) {
        enum { VK_ESCAPE = 0x1B, };

        // char const* class_name = "ConsoleWindowClass";
        // const char* win_name = "C:\\Windows\\system32\\cmd.exe";

        // char const* class_name = "Warcraft III"; // before patch 1.30?
        char const* class_name = "OsWindow"; // after patch 1.30?
        const char* win_name = "Warcraft III";

        if (args_len > 1) { class_name = args[1]; }
        if (args_len > 2) { win_name = args[2]; }

        HWND w = FindWindowA(class_name, win_name);
        if (w == 0) {
            printf("error: could not find Warcraft III window\n");
            exit(1);
        }

        SetForegroundWindow(w);
        send_key(VK_ESCAPE);

        return 0;
    }
     
     

    Attached Files:

  16. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    Really appreciate your feedback. The first two bugs are already fixed. Uploaded to first post.
    For the third, could you upload a map which i can just drop into WorldEditor and save there without jhcr already injected? Or the exact clijasshelper commands i guess.
     
    Last edited: Mar 23, 2019
  17. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    517
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    Yup.
     

    Attached Files:

    • bd.w3x
      File size:
      13.4 MB
      Views:
      7
  18. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0
    Sorry, can't reproduce. Works for me (after adding init and reload trigger).
    So i don't really know how to progress. Try downloading the new version, although from what i can see your code doesn't use the former bugged rawcodes and negate. Try printing a message before calling
    ExecuteFunc("JHCR_Init_parse")
    to make sure it really is called. Have you maybe made other changes than just printing something different?
     
    Last edited: Mar 23, 2019
  19. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    517
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    Hm. I added some debug printing in bdw_text_button. Before an update the value of g_Wid

    stays 1001 (as it should be). After an update the value starts to be incremented by 1 on each draw_loop call.

    Code (vJASS):

    function bdw_text_button takes string text, integer x, integer y, integer w, integer h, integer br, integer bg, integer bb, integer ba, integer tr, integer tg, integer tb, integer ta,  boolean eye_candy, Bd_Mbtn btn returns boolean
        local integer p = bd_p
        local integer wi = next_auto_wid(p)
        local boolean clicked = false
        local integer xx = x
        local integer yy = y
        local integer ww = w
        local integer hh = h
        local string s = ""

        if g_Wi[p] == 0 then
            if btn.down and is_point_in_rect(btn.down_x, btn.down_y, x, y, w, h) then
                set g_Wi[p] = wi
                call player_disable_drag_select(p)
            endif
        endif

        if btn.up then
            set s = "U"
        else
            set s = "D"
        endif
        call writeln("p: " + I2S(p) + ", wi: " + I2S(g_Wi[p]) + ", g_Wi[p]: " + I2S(g_Wi[p]) + ", btn: " + s + ", g_Wid[p]: " + I2S(g_Wid[p]))

        if wi == g_Wi[p] then
            if btn.up then
                set g_Wi[p] = 0
                call player_enable_drag_select(p)
                if is_point_in_rect(btn.up_x, btn.up_y, x, y, w, h) then
                    set clicked = true
                endif
            endif

            if eye_candy then
                set xx = R2I(xx + 0.05*ww)
                set yy = R2I(yy + 0.05*hh)
                set ww = R2I(ww - 0.1*ww)
                set hh = R2I(hh - 0.1*hh)
            endif
        endif

        call bd_rect(xx, yy, ww, hh, 2, br, bg, bb, ba)
        call bd_text_centered(text, xx, yy, ww, hh, tr, tg, tb, ta)

        return clicked
    endfunction

     


    This seems like a bug as well
    Code (vJASS):

    private function on_start takes nothing returns nothing
        local code fn = function draw_loop
        call TimerStart(CreateTimer(), 1.0/24.0, true, fn)
    endfunction

    function JHCR_lep__on_start takes nothing returns nothing
        local integer fn= function lep__draw_loop
        call TimerStart(CreateTimer(), ( 1.0 / 24.0 ), true, JHCR_Wrap_i2code(fn))
    endfunction
     



    Edit:

    Adding the two dummy lines in this function seems to fix the problem =)?
    Code (vJASS):

    function bdw_auto_wid_base takes integer base returns nothing
        local integer no_inline_1
        local integer no_inline_2
        set g_Wid[bd_p] = base
    endfunction
     
     
    Last edited: Mar 23, 2019
  20. LeP

    LeP

    Joined:
    Feb 13, 2008
    Messages:
    436
    Resources:
    0
    Resources:
    0



    Sorry but g_wid

    keeps on printing 1001 after update on my side.
    Something seems very different between our tests.

    What do you consider buggy about this?