1. Choose the best Old Gods themed skin at the Texturing Contest #29 - Poll!
    Dismiss Notice
  2. Melee Mapping Contest #2 - Poll is up! Vote for the best competitive 1v1 map!
    Dismiss Notice
  3. Please help test custom maps and more on the latest PTR!
    Dismiss Notice
  4. Evolution complete! Make Darwin proud and go vote in the Techtree Contest #12 - Poll.
    Dismiss Notice
  5. Icon Contest #17 - Results are out! Step by to congratulate our winners!
    Dismiss Notice
  6. We've created the Staff Job Openings thread. We're currently in need of icon, video production, and social/multimedia positions to be filled. Thank you!
    Dismiss Notice

Accessing memory from the script - it's time of the revolution

Discussion in 'The Lab' started by leandrotp, May 17, 2016.

  1. MyPad

    MyPad

    Spell Reviewer

    Joined:
    May 9, 2014
    Messages:
    1,025
    Resources:
    2
    Models:
    1
    Icons:
    1
    Resources:
    2
    This ability to access memory is amazing! Even overkill at making the life of the coder much more comfortable. Thank you, @leandrotp for working on this library. But, ...

    Is there a way to access Gameplay Constants in-game, using Memory Hack? (without having to create a global and assigning the same value of a certain data field to that global)

    On another note, I found some of the offsets that may prove useful to the coder, and ported some from the Memory Hack thread:

    Code (vJASS):

        function GetUnitAttackAbility takes unit u returns integer
            return Memory[ConvertHandle(u)/4+0x1e8/4]
        endfunction

        function GetUnitBaseDamage takes unit u returns integer
            return Memory[GetUnitAttackAbility(u)/4 + 0xA0/4]
        endfunction

        function GetUnitAttackSpeed takes unit u returns real
            return RMemory[GetUnitAttackAbility(u)/4 + 0x1B0/4]
        endfunction

        function GetUnitAttackSpeedSec takes unit u returns real
            local real r = GetUnitAttackSpeed(u)
            return 1 / r
        endfunction

        function GetUnitAttackBackswing takes unit u returns real
            return RMemory[GetUnitAttackAbility(u)/4 + 0x190/4]
        endfunction

        function GetUnitZ takes unit u returns real
            return RMemory[ConvertHandle(u)/4 + 0x28C/4]
        endfunction

        // I made this type-safe, thanks for the 'AHer' ability field!
        function GetHeroStatGain takes unit u, integer attribute returns real
            if GetUnitAbilityLevel(u, 'AHer') != 0 then
                return RMemory[Memory[ConvertHandle(u)/4 + 0x1F0/4]/4 + 0xCC/4 + 2*attribute]
            endif
            return 0.
        endfunction

        function CountUnitsInGroupEx takes group g returns integer
            return Memory[ConvertHandle(g)/4 + 0x34/4]
        endfunction
       
        function IsUnitEthereal takes unit u returns boolean
            return Memory[ConvertHandle(u)/4 + 0x1C0/4] != 0
        endfunction
     
     
  2. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    Code has been updated, please check the changelong and get the updated versions of all libraries.

    Are you sure? I always test it in 1.28.5 before releasing and it was working just fine. Try the new code now, things changed a bit but it should be working as well, I tested here and it worked.

    I don't remember exactly, that function is from the time me and Draco were messing with memory hack and discovering new stuff. That integer contains a lot of flags, things like invisibility, invulnerability and so on. I don't recall the meaning of each flag, maybe someday I will make functions like IsUnitInvisible that make use of it. Maybe you can find some info at Draco's Memory hack thread too.

    The old code was using fixed memory offsets, and because of that it only worked up to 1.28.1, since I needed to manually update offsets each time a patch was released. As I said I am migrating everything to automatic detection, so the mouse functions are disabled until I write the code to detect the mouse offsets.

    But don't worry, I've already got a method to find these offsets, soon I will be releasing an implementation. It won't be too long, I promise.

    Of course! Any information that is used by the game can also be retrieved with memory access, it's just a matter of finding where it is. For now I am still developing the basis of memory hack (address detection and base API), but in the future things like this shall be implemented and much more! Maybe gameplay constants are stored in a place that will require me to detect another game offset, I don't know, but it shouldn't be much of a problem.

    Also don't forget that you have Draco's thread as a reference, even though his code is a bit messy he made a huge progress in many areas, and you can port many functions from his code to this library if they use offsets that are already being detected.
     
  3. Sir Moriarty

    Sir Moriarty

    Joined:
    Jun 13, 2016
    Messages:
    174
    Resources:
    2
    Spells:
    1
    Tutorials:
    1
    Resources:
    2
    This reminds me, on a slightly tangetial note. I was meaning to ask someone this. Excuse me if you've been asked this a ton of times already.
    How do you find the offsets for each version? Do you use signature scanning for it, or have you been doing it manually in disassembly or something?
    Can you please give a general overview of the procedure you use to update the offsets? Just what tools you use and the general outline of the process, not asking for a detailed tutorial, haha.

    I've just been recently looking into a certain, but very useful tool for WC3 called Lua Engine (I think it's on d3scene), but it's outdated and it uses fixed offsets (a few hundred of them), which have obviously broken with new versions, and I'm looking to update it for the latest version of WC3 since I can use it for my map. You'd help me immensely if you gave me some pointers in which direction I should dig.
     
  4. Aniki

    Aniki

    Joined:
    Nov 7, 2014
    Messages:
    490
    Resources:
    4
    Spells:
    1
    JASS:
    3
    Resources:
    4
    When compiling in debug mode (vJass) it doesn't work (the GameState variable is not initialized), but in release mode (vJass's inliner kicks in) it does work. I tend to always "stay" in debug mode =)...
     
  5. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    Having a ton of experience in this area, I can tell you that there isn't a standard step-by-step way that will work for every case. Actually what introduced me to the programming world many years ago was exactly this matter: I got the source code of a hack for the game Gunz: The Duel and, having never programmed anything in my life, I managed to find the updated offsets and recompile the hack for a newer version of the game client.

    Obviously I didn't figure everything just by myself. Someone on those forums had posted a simple tutorial of finding some offsets in gunz.exe through signature searching. It worked on that particular case, but this method isn't guaranteed to work everywhere, especially in the case of WC3.

    More important than specific byte signatures, is the ability to recognize similar code patterns across different versions. This requires some disassembly and reverse engineering skills, but it's something you'll be acquiring with time. Let's take a look at an example:

    1.png

    This is the disassembly of the native
    GetUnitPointValueByType
    from patch 1.26. As you can see it hashes the rawcode, then calls GetObjectData passing the UnitData pointer as the 1st argument, in the ECX register.

    Now let's take a look at the same native from patch 1.27a:

    2.png

    Looks completely different, doesn't it? Where's the call to GetObjectData, and what is all this new code after IntegerHash? Did Blizzard change this function so drastically?

    Now just take a look at GetObjectData itself, from 1.26:

    3.png

    Can you see the similarities? The code that comes after the IntegerHash call looks very similar to the code of GetObjectData itself. With the difference that we now have
    MOV EAX,[67B1C494]
    in the place of
    MOV EAX,[ECX+24]
    and
    MOV ECX,[67B1C48C]
    instead of
    MOV EDX,[ECX+1C]


    The reason looks simple now: the call to GetObjectData has been inlined! And this is not something unexpected, because Blizzard changed the compiler for the 1.27 patch, migrating to a newer version of Visual Studio, which has a more aggressive optimizer. Many other function calls have been inlined like this.

    So we can deduce, if [67B1C494] is [ECX+24], and [67B1C48C] is [ECX+1C], then the address of pUnitData is certainly 67B1C470. Subtract it from the base of game.dll (at this time it was 66F30000) and the result is 0xBEC470, the offset of the UnitData table for patch 1.27.

    Well I suppose I did nothing but complicate the things for you until now right? As I said this job inevitably requires good disassembly skills, but I can give you some hints:

    First step: Open game.dll from 1.26 and go to the offset you want. See if it's located in code or data sections, and find references to it.

    As an example, I'll use the offset of MouseEnv object (game.dll+AB4F80), which gives access to mouse functions.

    4.png

    As you can see this object has 5 references, and 3 of them are in the same function. So the deal is, if we find this function in patch 1.27, we got our address.

    Second step: now what you need to do is find something in this function that could possibly be used to search for it in the new patch. Strings and constants are always good candidates, WC3 has a lot of debug information in it's executables, like classes and source file names, and sometimes they helps us to find functions across versions.

    6.png

    Well, it found 42 references to the string "CGameUI.cpp". This is not so bad, in the worst case we could look at those functions one by one, and see if we find something more-or-less similar to what we are looking for. But let's try something different, the constant 1AAF:

    7.png

    Wow, we've got it! Only one reference, and it's exactly what we want! Just compare the two functions, and see that they are very similar. Some small changes here and there, like a conditional move (CMOVNZ) instruction in the place of a Jump, but as I said, this is just compiler optimization.

    If you want more confirmation, just look at the function calls: they both are calling Storm.#401 with exactly the same parameters, and after that an object constructor (67279FA0) is called, which is also very similar to the one from 1.26 (6F2FE9F0) This is definitely the function we want.

    So the address of MouseEnv in this version is 67B16350, subtract from base of game.dll and we get 0xBE6350.

    If you cannot locate something noticeable that you can use to search, or if the search yields too many results, you can try looking for the callers and callees of that function. Try repeating the process above with a function that calls or is called by the function you want, and if succeed you will most likely find it too.

    If the address you're looking is located in a Jass native, or a function that is called by a native, then it's bingo! All natives have well-defined addresses, and you can easily locate them through their name:

    5.png

    That is the initializer of the Jass natives table. The address of the native is in the MOV ECX instruction, right below the native name. In this case it is 6F3B2E20 for the native
    GetUnitPointValueByType
    . You can easily find any native this way.

    Well, these are some methods that I use to find offsets myself. Other things include looking at objects' vTables and searching for generic instructions (OllyDbg allows to search for generics like "MOV R32, [R32+24]" which returns any MOV instruction that fits this pattern, independently of the used registers). As I said, every case is specific, and it all depends on your skills.

    I'm pretty sure you'll learn a lot if you really dig into this subject. Reading some assembly lessons is a good starting point (it is how I started myself, learned ASM before any other language). And don't be afraid to ask for any further help, if I don't help you, certainly someone else will.
     
    Last edited: Oct 2, 2017
  6. Sir Moriarty

    Sir Moriarty

    Joined:
    Jun 13, 2016
    Messages:
    174
    Resources:
    2
    Spells:
    1
    Tutorials:
    1
    Resources:
    2
    Man, this is great! More than I'd ever hope for, and it certainly helps me get started. This subject always felt like somewhat arcane to me, even though I have a decent grasp on low-level concepts, and most of what you said made perfect sense to me. Although, I could never quite figure out how to even approach this, but this helped a lot. It's how they say, the hardest thing is to get started. Huge, huge, huge thanks to you!

    This really inspires me to go and dig into this myself.
     
  7. MyPad

    MyPad

    Spell Reviewer

    Joined:
    May 9, 2014
    Messages:
    1,025
    Resources:
    2
    Models:
    1
    Icons:
    1
    Resources:
    2
    Quite a nice approach to something low-level. Reading that also made me want to go about this memory thing myself, too. However, while using the debugging software, should warcraft 3 be running?

    Another thing, how did you get the index of the natives? (Some arbitrary R for a native N)
     
    Last edited: Oct 2, 2017
  8. GhostWolf

    GhostWolf

    Joined:
    Jul 29, 2007
    Messages:
    4,836
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    Very nice explanation.
    Making hacks for Gunz was certainly a fun time. By now I don't remember anything about reverse engineering though :(
     
  9. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    I found the reason, it's because I'm invisibly "returning" a value from function Step1, even though it returns nothing, I'm manipulating R0 to actually get a value passed back to bytecode. But when inliner is disabled, the Array sets become function calls, and every function call resets the value of R0, causing the VM crash when trying to access it. I'm gonna fix it in the next release

    It doesn't need to be running, OllyDbg has the option to open a dll file directly for analysis, so you don't need to run wc3 at all, you can open game.dll directly with OllyDbg and see everything.

    But it's possible to use it with the game running too! You can launch WC3 directly from OllyDbg, or launch it from WE and attach the debugger to it. But keep in mind that the game must ALWAYS be running in window mode if you're going to do that. If you're using NewGen it's better, as it can already launch WC3 in window mode.

    If you forget to run in window mode, and the game gets paused by the debugger, you'll be unable to ALT-TAB out of the game screen, and you'll be forced to restart your computer! So don't make that mistake.

    Also, what do you mean by "index of natives"? I didn't understand.

    Did you make gunz hacks too? Did you release them? I'm AnĂ´nimo from GamerzPlanet (http://www.gamerzplanet.net/members/anĂ´nimo.2818941/).
     
    Last edited: Oct 2, 2017
  10. BritneySpears2016

    BritneySpears2016

    Joined:
    Oct 3, 2017
    Messages:
    26
    Resources:
    0
    Resources:
    0
    It's more convinient to read the code as C. Just take a look at GitHub - DracoL1ch/WC3Memory game126.c
    That's way it's much easier to see constants, look for a strings, debug info, etc. IDA does this stuff, althought it's payed software, the best, with ExtraPass plugin. Converted code is much easier to compare between versions as well, if it's not that drastical jump as from 26 to 27.

    Many constants are still searchable by patterns, offsets, etc. Just in case if you dont like byte-searching (like me) you can take this approach. It's not easier in common sense, it's just more human-like. Leandro the best, of course, but simple man needs simple solutions :)
     
  11. MyPad

    MyPad

    Spell Reviewer

    Joined:
    May 9, 2014
    Messages:
    1,025
    Resources:
    2
    Models:
    1
    Icons:
    1
    Resources:
    2
    Okay, thanks for that! :)

    When you're calling the natives, how did you know which numbers correspond to them? (Like 623 for GetPlayers...)
     
  12. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    Those ids come from the string table. Aniki had dumped the string table (see JASM - Let's dive into bytecode) so you can use those ids when referencing anything from common.j or blizzard.j. For stuff in the main map script, you need to manually obtain the id (via memory reading).
     
  13. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    New version of libraries have been released. Mouse functions are back and working in all versions, ObjectData library can now get AbilityData too (don't forget to use GetObjectName if the ability has not been loaded by the game yet).

    Development of automatic detection engine has been finished. I've rushed it in the last few days since I'm gonna be absent for some time (I'm getting married). So the progress is probably gonna be slower now. But the code has already reached a stable version, so feel free to use it in production environment, I've tested it under all versions up to 1.28.5 and it worked flawlessly.

    However I am unable to test it under Mac. I wrote the detection code for Mac environment by looking at the mac executables from patches 1.26, 1.27a and 1.27b. But I never got the chance to actually run the code, so I don't know if it's working or if I made any mistakes there. I'd be happy if Mac users could test it for me (@Bribe, @PurgeandFire?).
     
  14. ZiBitheWand3r3r

    ZiBitheWand3r3r

    Joined:
    Nov 21, 2012
    Messages:
    853
    Resources:
    14
    Maps:
    6
    Spells:
    8
    Resources:
    14
    hey, thanks for great update,
    Your ObjectData requires this library by Bribe: [Snippet] New Table ?

    ps All the best for you and your family ;)

    @MyPad
    GetUnitBaseDamage, GetUnitAttackSpeed crashes the game when used on unit Fountain od Power (no attacks enabled, attack speed=0.00, base damage=0.00)

    moreover GetUnitAttackSpeed(u) returns wrong values for footman/knight for example
     
    Last edited: Oct 4, 2017
  15. MyPad

    MyPad

    Spell Reviewer

    Joined:
    May 9, 2014
    Messages:
    1,025
    Resources:
    2
    Models:
    1
    Icons:
    1
    Resources:
    2

    Yeah, I observed that GetUnitAttackSpeed function will only return 0 if you perform a division operation, more specifically like this:


    Code (vJASS):

    // Regard as null and void
    function GetUnitAttackSpeed takes unit u returns real
        return RMemory[GetUnitAttackAbility(u)/4 + 0x1B0/4]
    endfunction

    function GetUnitAttackSpeedSec takes unit u returns real
        local real r = GetUnitAttackSpeed(u)
        return 1 / r
    endfunction
     


    My assumption is that the Memory array still treats the value internally as an integer, which will lead to wrong values (if the unit can attack but does not have the attack ability).

    I did not check if the pointer Unit's attack data exists. If it cannot attack, then the pointer to the unit's attack data will be 0.
    Perhaps this will fix it:
    Code (vJASS):

    function GetUnitBaseDamage takes unit u returns integer
        local integer i = GetUnitAttackAbility(u)
        if i == 0x0 then
            return 0
        endif
        return Memory[i/4 + 0xA0/4]
    endfunction
     
    function GetUnitAttackSpeed takes unit u returns real
        local integer i = GetUnitAttackAbility(u)
        if i == 0x0 then
            return 0.
        endif
        return RMemory[i/4 + 0x1B0/4]
    endfunction

    function GetUnitAttackSpeedSec takes unit u returns real
        local real r = GetUnitAttackSpeed(u)
        if r == 0. then
             return 0.
        endif
        return 1/r
    endfunction

    function GetUnitAttackBackswing takes unit u returns real
        local integer i = GetUnitAttackAbility(u)
        if i == 0x0 then
            return 0.
        endif
        return RMemory[i/4 + 0x190/4]
    endfunction
     
     
  16. BritneySpears2016

    BritneySpears2016

    Joined:
    Oct 3, 2017
    Messages:
    26
    Resources:
    0
    Resources:
    0
    There are also a special AS penalty @ 0x1C0, the value is alwways negative or 0. You can see it when affected by poison attack. IDK why it uses separate field but it's worthy to know for non-dota too. Basially u need to sum both fields to get proper value.
     
  17. BritneySpears2016

    BritneySpears2016

    Joined:
    Oct 3, 2017
    Messages:
    26
    Resources:
    0
    Resources:
    0
    Does anyone got new ConvertHandle to work? This opt doesn't work for me as ~20-30 seconds ingame offset with [+412] changes it's value, causing access error instnantly. Can't really understand what causes that. Also, value is different when loading screen enabled and right after that. IDK whats so special about that.
     
  18. leandrotp

    leandrotp

    Joined:
    Jul 30, 2012
    Messages:
    148
    Resources:
    1
    Tutorials:
    1
    Resources:
    1
    Could you please upload a testmap that reproduces this problem? I tested it here and it was working just fine, but I got a similar problem when I tried to optimize GetStringAddress with the same method (because address of StringHandle table changes when new strings are inserted). Maybe the same is happenning with handles, I tried to spam handle creation and it still worked properly, but if you can upload a map where the problem is happening, I'll try to find the cause.
     
  19. BritneySpears2016

    BritneySpears2016

    Joined:
    Oct 3, 2017
    Messages:
    26
    Resources:
    0
    Resources:
    0
    No need, I investigated it myself. 26a, as always
    sub_6F42CB50 gets called, like, on every handle creation? Takes environment
    Code (vJASS):
    RMem(RMem(GameState) + 28)  + 404

    (readmem == non-divided)
    everytime handle counter getting 1k bigger than before, it asks for sub_6F42B060 to re-allocate memory for storage.
    [​IMG]
    In case when we need to increase size ReAlloc returns 0, which leads to a new MemAlloc with known result - table get moved.
    So, there are no way to relate on this opt, unless you're 100% sure you allocated as many handles as you could ever have in map, and it's really hard to say so as counter doesnt seem to decrease.
     
    Last edited: Oct 7, 2017
  20. BritneySpears2016

    BritneySpears2016

    Joined:
    Oct 3, 2017
    Messages:
    26
    Resources:
    0
    Resources:
    0

    Things ppl can play with memhack. Thanks god.