1. Updated Resource Submission Rules: All model & skin resource submissions must now include an in-game screenshot. This is to help speed up the moderation process and to show how the model and/or texture looks like from the in-game camera.
    Dismiss Notice
  2. DID YOU KNOW - That you can unlock new rank icons by posting on the forums or winning contests? Click here to customize your rank or read our User Rank Policy to see a list of ranks that you can unlock. Have you won a contest and still havn't received your rank award? Then please contact the administration.
    Dismiss Notice
  3. The Lich King demands your service! We've reached the 19th edition of the Icon Contest. Come along and make some chilling servants for the one true king.
    Dismiss Notice
  4. The 4th SFX Contest has started. Be sure to participate and have a fun factor in it.
    Dismiss Notice
  5. The poll for the 21st Terraining Contest is LIVE. Be sure to check out the entries and vote for one.
    Dismiss Notice
  6. The results are out! Check them out.
    Dismiss Notice
  7. Don’t forget to sign up for the Hive Cup. There’s a 555 EUR prize pool. Sign up now!
    Dismiss Notice
  8. The Hive Workshop Cup contest results have been announced! See the maps that'll be featured in the Hive Workshop Cup tournament!
    Dismiss Notice
  9. Check out the Staff job openings thread.
    Dismiss Notice
Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

Producer Update: Natives List

Discussion in 'Patch & Reforged Discussion' started by Kam, Dec 7, 2017.

  1. Rui

    Rui

    Joined:
    Jan 7, 2005
    Messages:
    8,497
    Resources:
    8
    Maps:
    8
    Resources:
    8
    So how about a small comment towards my proposal?^^
     
  2. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    That would be just bad design in a lot of ways, and lots of bugs could be created through this. If people are actually writing natives that utilize the actual data structures as saved in the programmed model, then there'll be less bugs and more safety in general. Even the mem hack has a lot of bugs/really hard to offset things due to this, if people making it could use the programmed model it'd be much easier.

    Also do realize that it isn't all about just reading memory, it has to be structured after being read, and memhack does this, so this means that wc3 devs would also have to restructure "randomly read" memory. So it's just easier for them to return things from the program model, rather than hacking around the memory themselves, without any knowledge of what it really represents (since this knowledge was created through countless hours of hacking around the wc3 engine, which they have not put into it).

    Also, this update is going to create lots of bugs, and people who implement natives as "readmemory" won't be able to debug anything, if they implement them properly, they will be able to debug this.
     
  3. Rui

    Rui

    Joined:
    Jan 7, 2005
    Messages:
    8,497
    Resources:
    8
    Maps:
    8
    Resources:
    8
    I didn't specify parameters, but logically one of them would specify what we're extracting^^ Anyhow, as I said, this would come with unveiling object structure. I mean, if the engine as a whole is optimized, then you may as well overload the API with natives, but last I checked the resulting bytecode is quite inefficient.
    I like meddling with low-level stuff :p.
     
  4. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    Low level stuff has no place here, though, and that's my point. Unless you're talking about loading mouse/keyboard stuff directly from the drivers, most of what's needed here is high level stuff that deals with the object structure that's present in wc3 source code.

    This is all presuming that blizzard has access to the source code. If they don't, then hackers will be greatly appreciated.
     
  5. Daffa the Mage

    Daffa the Mage

    Map Moderator

    Joined:
    Jan 30, 2013
    Messages:
    7,731
    Resources:
    28
    Packs:
    1
    Maps:
    8
    Spells:
    17
    Tutorials:
    2
    Resources:
    28
    All BonusMod thing should become accessible publicly in World Editor.
    In short, that's already 10 points to deal with.
     
  6. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    ^Yes, but BonusMod stuff is something that's easy to integrate into a map anyways. Most features I want are things that are impossible to do with wc3 right now.

    Also, we need native benchmarking too.
     
  7. Daffa the Mage

    Daffa the Mage

    Map Moderator

    Joined:
    Jan 30, 2013
    Messages:
    7,731
    Resources:
    28
    Packs:
    1
    Maps:
    8
    Spells:
    17
    Tutorials:
    2
    Resources:
    28
    Yeah, in most case if JNGP or WEX(?) with the LUA stuff available to deal with the 198 abilities of BonusMod. I don't know, but Wurst might be able to handle this field.

    Exposing tooltip modification mid-game would be pretty great. Trigger wise at least, if applicable. Less percentage stuff to write, and more precise tooltips.
     
  8. chobibo

    chobibo

    Joined:
    Sep 24, 2005
    Messages:
    2,700
    Resources:
    0
    Resources:
    0
    What memhack? The return bug?
     
  9. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    I'm currently working on a bonus mod system for wurst that lets you define a whole bonus in a few lines of code, so it is more than possible, it's easy as hell. That's why I am not so hard pressed to ask to be able to do stuff bonus mod does. But if those improvements were to be made, then I would just turn all ability mods into code mods in my bonus system, which would benefit me, too. I just don't need it. As I said, things that cannot be done at the moment like detecting key press/release and mouse coordinates, as well as cooldown control/detection are more up my alley, but I do understand that most people want other sorts of convenience or new features.

    Well, yeah, for example wurst allows you to easily edit tooltips in compiletime, but it can't edit tooltips in runtime, hence why I would really like this feature, but it is kind of included in my "allow us to edit ui elements" request, since tooltips are an UI element.
     
  10. Kyrbi0

    Kyrbi0

    Joined:
    Jul 29, 2008
    Messages:
    7,913
    Resources:
    1
    Models:
    1
    Resources:
    1
    I didn't know you were a coder, HappyTauren. Man of many talents.

    I don't care about any of those things, so I'm perhaps the most impartial between you; I gotta say, HT makes a good argument for adding functionalities that can't currently exist, rather than ones that don't but currently can be made.
     
  11. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    Of course, that's only if implementing those things that can currently exist require too much additional effort to code on blizzard's part. Most of the things that are also already possible (through workarounds) would probably be easy to implement alongside the whole "allow people to access all unit's data" theme. Meaning that beside the fact that bonus systems exist and are pretty powerful and easy to use, if a system to access all object's properties is introduced, then it is going to be pretty good.

    As I said, returning unit's data from its data table/object should be uniform as Get/SetUnitData[String|Real|Int|Bool](unit u, string data, setterdata) and not require 100 functions. All we need is to access the unit's data table, not 10235345 functions for everything specifically. But we already went over that in this thread, I just want to repeat the importance of this because Kam is inquiring about things.
     
  12. Rui

    Rui

    Joined:
    Jan 7, 2005
    Messages:
    8,497
    Resources:
    8
    Maps:
    8
    Resources:
    8
    I was mostly talking about digging through raw data, but I wouldn't mind this either. :grin:
     
  13. moyackx

    moyackx

    Joined:
    Feb 15, 2006
    Messages:
    795
    Resources:
    7
    Maps:
    4
    Spells:
    2
    Tutorials:
    1
    Resources:
    7
    So, is there an estimated date to release a BETA of this implementation? I assume this will be visible in the Warcraft III BETA package
     
  14. Kam

    Kam

    Joined:
    Aug 3, 2004
    Messages:
    2,630
    Resources:
    23
    Models:
    8
    Icons:
    2
    Maps:
    13
    Resources:
    23
    Some of the list will be included in the next patch. =)
     
  15. Daffa the Mage

    Daffa the Mage

    Map Moderator

    Joined:
    Jan 30, 2013
    Messages:
    7,731
    Resources:
    28
    Packs:
    1
    Maps:
    8
    Spells:
    17
    Tutorials:
    2
    Resources:
    28
    Horray for getting additional natives!
    Oh, and maps failing in older patches coming :D
     
  16. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    361
    Resources:
    2
    Spells:
    1
    Tutorials:
    1
    Resources:
    2
    Hey everyone.

    I would like to offer a few separate propositions, each of which would individually facilitate working with save/load systems in WC3 and make them easier to write and use.
    Currently, it is already possible to do most of what I am about to offer - you can look in Wurst's StdLib2 "Network" and "MultifileIO" packages to have a taste of how it can be done, but there are some issues with what can be currently achieved:
    1. There is no nice way of reading/saving files. You can exploit the Preload API, but it is far from being even remotely nice to use:
    1.1 You have to use SetPlayerName to be able to read data from files, which limits you to 16 lines per file. This means that if you want to write more than that, you have to split your data into multiple files (You can see in Wurst's StdLib2 how I accomplish this, for instance),
    1.2 There is no way to check the existence of a file or directory,
    1.3 There is no way to write binary (or even text) data to a file without "dead weight" of the Preload API,
    1.4 There is no way to read contents from a file other than using GetPlayerName or other stateful APIs,
    1.5 You have to enable a specific registry value in order to be able to read files,
    2. There is no nice way of sending data over the network. You can use GameCaches and the Sync natives, but:
    2.1 The SyncStoredString native is broken, requiring you to serialize strings as int-streams,
    2.2 The API is weird and unintuitive, keys are strings (would be more convenient to have them as ints for this use-case),
    2.3 GameCaches are very slow compared to even HashTables, making syncing code rather resource-intensive, especially for large payloads,
    2.4 The API is not synchronous and is very hard to use in JASS or even vJASS. You can write an asynchronous API using callbacks in Wurst, which is nicer, but it is still not very easy to use,
    2.5 You are requried to constantly restart threads if you want to handle large payloads
    3. There is no nice way of working with raw binary data. This issue is actually not specific to just save/load, but also other applications:
    3.1 No bitwise operations. Bitwise ops have to be coded in using lookup tables or MemoryHack,
    3.2 No char-int conversions. To get an int value of a char you have to use lookup tables, and vice-versa

    Hence, I propose several additions, each of which addresses these issues. The simplest to implement here would be #3, so let's start with that:

    Code (vJASS):

    // Each function will be annotated with it's C++ equivalent

    // ~a
    native BitwiseNot takes integer a returns integer
    // a & b
    native BitwiseAnd takes integer a, integer b returns integer
    // a | b
    native BitwiseOr takes integer a, integer b returns integer
    // a ^ b
    native BitwiseXor takes integer a, integer b returns integer

    // a << b
    native BitwiseShl takes integer a, integer b returns integer
    // a >> b
    native BitwiseShr takes integer a, integer b returns integer

    // Returns the byte value of the first character in the string, or -1 if string is empty
    // Return value is guaranteed to be in the bound of [0, 256)
    native Char2Byte takes string a returns integer

    // Converts the byte into a single character corresponding to that character's value
    // The argument should be in the bound of [0, 256), if it is not, the returned string should be null
    native Byte2Char takes integer a returns string

     


    Next is a proposal for a File IO API. This API should work out of the box, without requiring any tweaks to the registry:

    Code (vJASS):


    constant integer FILEMODE_READ = 1
    constant integer FILEMODE_WRITE = 2
    constant integer FILEMODE_READWRITE = 3

    type file extends handle
    type directory extends handle

    native FileExists takes string path returns boolean
    native DirectoryExists takes string path returns boolean

    // Returns an iterator to a directory's contents
    // Returns null if path doesn't exist or isn't a directory
    native GetDirectory takes string path returns directory

    // Callback-style iteration
    native ForDirectory takes directory dir, code callback returns nothing
    native GetEnumPath takes nothing returns string

    // Pops a single path from this directory object, allowing us to iterate it using a while loop
    native PopDirectoryPath takes directory dir returns string

    // Opens a new file, creating it if necessary
    // There should probably be a restriction on which extensions are allowed
    // Good candidates are probably ".txt" and ".dat"
    native OpenFile takes string path, integer mode returns file

    // Opens a new file only for a certain player
    // This is necessary to allow reading/writing locally, because creating a handle locally would desync, so
    // we need to create it in shared code
    // Reading/Writing for other players on this handle will have no effect, unless SyncFile has been called,
    // then reading operations should become available
    native OpenFile takes string path, integer mode, player owner returns file

    // (I'm not sure if this should be added or not) Tries to send the contents of an entire file from one players to the rest
    // Calling this in a local block will read the whole file and start sending it to other players, allowing us to read the file
    // synchronously in shared code without issues
    native SyncFile takes file f, player sender returns nothing

    // If this can be implemented, it'd be really neat
    // Pauses the current thread until the file has finished syncing
    native SyncFileWait takes file f returns nothing

    // Async events for syncing to signal when it's finished
    native TriggerRegisterFileSync takes trigger t returns nothing
    native GetSyncedFile takes nothing returns file

    // Flushes the changes and closes the file
    native CloseFile takes file f returns nothing

    native GetFileSize takes file f returns integer
    native SetFileSeekPos takes file f, integer pos returns nothing
    native GetFileSeekPos takes file f returns integer

    // Checks whether the file has hit EOF
    native IsFileEOF takes file f returns boolean

    // Reads the contents of the file into a string, up to specified amount of characters
    native FileReadString takes file f, integer max returns string

    // Writes the contents of the string into the file as-is, without null-terminators
    native FileWriteString takes file f, string s returns nothing

    // Natives for reading and writing bytes directly, rather than using a string
    // Since WC3 runs only on little-endian platforms, these too should probably be little-endian
    native FileWriteUInt8 takes file f, integer a returns nothing
    native FileWriteUInt16 takes file f, integer a returns nothing
    native FileWriteUInt32 takes file f, integer a returns nothing
    native FileWriteInt8 takes file f, integer a returns nothing
    native FileWriteInt16 takes file f, integer a returns nothing
    native FileWriteInt32 takes file f, integer a returns nothing

    // Natives for reading bytes directly
    native FileReadUInt8 takes file f returns integer
    native FileReadUInt16 takes file f returns integer
    native FileReadUInt32 takes file f returns integer
    native FileReadInt8 takes file f returns integer
    native FileReadInt16 takes file f returns integer
    native FileReadInt32 takes file f returns integer

    // Writing/Reading reals
    native FileWriteReal takes file f, real a returns nothing
    native FileReadReal takes file f returns real

     


    Last is the proposal to address the issue of networking data across players. There are two solutions I'd like to offer:
    The first one is the most straightforward and requires minimal new natives
    But for it to be worth it, the code should be optimized to make most use of the bandwidth available, and the fact that we are sending
    data in a batch (instead of how SyncStored* GameCache natives seem to work, which are very slow, only 1-4 kbps max)
    Code (vJASS):


    // When called locally, will start syncing the contents of this hashtable to other players
    native SyncHashtableParent takes hashtable ht returns nothing
    // Same as SyncHashtable, but only for a child hashtable
    native SyncHashtableChild takes hashtable ht, integer parentKey returns nothing
    // Pauses the thread until the specified hashtable has finished syncing
    native SyncHashtableWait takes hashtable ht returns nothing
    // Events for hashtable syncs
    native TriggerRegisterHashtableSync takes trigger t returns nothing
    native GetSyncedHashtable takes nothing returns hashtable

     


    Another way, which could turn out more efficient, would be to create a new type specifically for syncing
    This type would be something like a byte-buffer with natives for reading, writing and syncing it's contents
    It's internal C++ representation could be something as simple as std::vector<uint8_t>, allowing us to use it
    as a dynamic array type, which has also been long since requested in the community
    Code (vJASS):


    type bytes extends handle

    // self-explanatory
    native CreateBytes takes nothing returns bytes
    native DestroyBytes takes bytes b returns nothing

    native BytesWriteUInt8 takes bytes b, integer pos, integer a returns nothing
    native BytesWriteUInt16 takes bytes b, integer pos, integer a returns nothing
    native BytesWriteUInt32 takes bytes b, integer pos, integer a returns nothing
    native BytesWriteInt8 takes bytes b, integer pos, integer a returns nothing
    native BytesWriteInt16 takes bytes b, integer pos, integer a returns nothing
    native BytesWriteInt32 takes bytes b, integer pos, integer a returns nothing

    native BytesReadUInt8 takes bytes b, integer pos returns integer
    native BytesReadUInt16 takes bytes b, integer pos returns integer
    native BytesReadUInt32 takes bytes b, integer pos returns integer
    native BytesReadInt8 takes bytes b, integer pos returns integer
    native BytesReadInt16 takes bytes b, integer pos returns integer
    native BytesReadInt32 takes bytes b, integer pos returns integer

    native BytesWriteReal takes bytes b, integer pos, real r returns nothing
    native BytesReadReal takes bytes b, integer pos returns nothing

    native BytesWriteString takes bytes b, integer pos, string s returns nothing
    native BytesReadString takes bytes b, integer pos, integer amount returns string

    // Get the amount of data in this buffer
    native GetBytesSize takes bytes b returns integer
    // Reserve capacity for this buffer
    native ReserveBytesCapacity takes bytes b, integer capacity returns nothing
    // Get capacity for this buffer
    native GetBytesCapacity takes bytes b returns bytes

    // Tries to sync the local contents of this bytes object to other players
    native SyncBytes takes bytes b returns nothing

    // Pauses the current thread until syncing has finished
    native SyncBytesWait takes bytes b returns nothing

    // Events
    native TriggerRegisterBytesSync takes trigger t returns nothing
    native GetSyncedBytes takes nothing returns bytes

     
     
  17. Frotty

    Frotty

    Wurst Reviewer

    Joined:
    Jan 1, 2009
    Messages:
    1,437
    Resources:
    11
    Models:
    3
    Tools:
    1
    Maps:
    5
    Tutorials:
    1
    Wurst:
    1
    Resources:
    11
    Since it just happened to me once again, something I would prefer over 99% of the native suggestions in this thread: Crash/Critical error debug information!

    It would be superb if the "critical error" window with the report function after wc3 crashes would also display the last Jass execution point or ideally a stacktrace, so we have some idea what caused it.

    While there are some known causes, like units out of bounds, others like endless execution loops can be very hard to prevent without any idea where/why they happen.

    For pre 1.24 there existed a tool named "war3err" which allowed exactly that.
     
  18. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,616
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    JASS should not even cause the game to crash... Similar errors do not in Galaxy used by StarCraft II.
     
  19. HappyTauren

    HappyTauren

    Joined:
    Nov 3, 2006
    Messages:
    8,413
    Resources:
    87
    Models:
    61
    Icons:
    23
    Packs:
    1
    Tutorials:
    2
    Resources:
    87
    ^Patching JASS to make it impossible to crash the game through it probably requires too much work, and it'd be dumb for blizzard to dump too many resources into a thing like that. Even this patch is dumb from a business perspective. On the other hand, even if jass VM crashes, it is likely pretty easy to retrieve the last function from the call stack, or even the stack trace.
     
  20. Nowow

    Nowow

    Joined:
    Jun 15, 2016
    Messages:
    435
    Resources:
    2
    Tutorials:
    2
    Resources:
    2
    Another big quality of life improvement you might want to consider is a few more AI natives. One of the biggest problems with the AI is the very inflexible use of captain widgets.

    Currently, captain widgets are a "middleman" in charge of orders given to the computer player. The AI player moves, or teleports, the captain widget to a target location and the units assigned to the captain follow. There are two captains: attack and defense, each with it's own problems.

    The units assigned to the defense captain will stay still even when given an order to change location, until one of three cases: aggroed, player town under attack, another unit is added to the defense captain. But the bigger problem is with the attack captain: once a unit has been added to the attack captain, it cannot be removed manually. If it is injured, and the attack group requires units with a higher health percentage, then the unit will be removed, but it cannot be done manually.

    I have two ideas for solutions in mind, both in a way that should not break backwards compatibility and enhance the AI greatly:

    1. Add an
    Empty attack group
    native. A simple native that will allow one to manually remove all units from the attack group. That alone will not change existing scripts and will remove a lot of limitations.

    2. Allow multiple captains. This one is a bit more radical change, but it will make the AI far better and more flexible. The thing with captains is that you can only have one of each, but that is not to say that you can't create more. In fact, you can use the native
    CreateCaptains 
    as many times as you want, and it will create new captains. Also, it doesn't remove the old captains, as units assigned to them cannot be added to the new captains, and will continue following the old captain to the last target it received. The problem here, is that referencing the captains is done by constant integers set in the common.ai file:
    Code (vJASS):

    constant integer ATTACK_CAPTAIN     = 1
        constant integer DEFENSE_CAPTAIN    = 2
        constant integer BOTH_CAPTAINS      = 3
     

    Once you create new captains, these variables will reference the new captains instead of the old ones. If you could allow users to reference older captains (for example allocating another 3 numbers for older captains for every time
    CreateCaptains 
    is called), AI will be much more usable.