[Altered Melee] Map Script-Independent Jass-driven AI Experiment

[MyScorpion42]

Documentation of what functions and objects do and do not work in AI
Documentation of which races are and are not assignable by the game
Documentation of how units and equivalencies are counted by different JASS Functions

9 months ago, I started this project... then 2 months later I stopped... then 2.5 months later I started again... then 2 months later I stopped again...
Now, 2.5 months later... I am not sure if I will pick it up again so there is even less of a reason for me not to share it on here.

Hive Workshop, I present to you...
AI scripting but with custom stats for units but with minimal use of the object editor and map triggers

Looks like you spent 10 minutes in a hex editor converting non-hero units into units

No proper names defined in the object editor.

"You just set them with triggers" I hear you say, but if we go into the trigger editor:

Oh, would you look at that. Nothing there, except... an AI script? And if we go to imports...

99 Kilobytes??? That's right, mofos, I have 2483 lines of JASS in my AI Script.
WHY?!
How, you ask? It starts, as I mentioned, 9 months ago when I realized that common.ai (the library of native functions for the AI engine) used some functions from common.j (the library of native functions for the trigger engine), which prompted me to start experimenting with which functions worked and what could be done with it.

My first idea was to create an interface to allow the AI engine to send commands back to the trigger engine. The latter has the CommandAI function for relaying messages to the former, but it can only send pairs of integer values, and the AI engine does not have a way to send messages back. My solution was to have the AI engine create a unit to serve as an "interface". The trigger engine can hook this unit because it sends an entering unit-signal when spawned. From there, both engines can communicate back and forth, and more importantly send lots more data at a time, as a unit can contain two locations (unit's current location and its rally point), a widget (by rally point), a string (its name), reals (hit points, mana, armor, attack cooldown, acquisition range...), integers (ability levels, custom values, damage values...) and probably more.
Because I am one of those kinds of people, I instantly knew what I wanted to do with it.

However, due to issues with testing and too many things not working (most functions with strings corrupt them, converting anything to a string crashes the game), I started over. Rather than focusing on creating new factions whole cloth, I would first recreate an existing faction and its AI, and then this could be slowly converted into a wholly new faction.
Not how, why?!
Oh right, well, I also realized I had tricked myself into doing Distributed Systems even though I did not enjoy that course, so I opted to try to do as little communication with the map as possible, and not use an interface. This would also make it easier to reuse the script on different maps, because it would require less setup with triggers from the map's side.
However, AI scripts generally NEVER refer to units directly, and one of the things that are totally non-functional in the AI engine are triggers. So the only way to effectively access units' handle is for the AI script itself to create it. This also means the script ends up having to keep track of all the units and buildings explicitly. Because the target locations of build orders cannot be accessed, and a building being constructed can't be accessed, the script also has to recreate building planning and construction. It maintains multiple arrays for these stages of buildings for this purpose. Because I wanted to be able to use the same unit type for multiple "unit types", unit counting was also recreated in the script. Oh yeah, and I recreated the Dungeon Keeper naming system for fun once I moved to testing with Hero Units.
No, why did you do all this instead of just using the object editor???
Oh right, uh:

• Autism
• Desperately wanting to prove to myself that I am not a script kiddie
• Genuine interest in finding out what could and couldn't be done with JASS in an AI script
• Previous attempts to create custom factions had me lose interest due to having to set up the techtree from scratch in the object editor

Okay, so why did you stop
Mostly because right now, the Orc AI ends up building like 5 Voodoo Lounges as if it is not keeping track of how many are already under construction but it's the only building it happens with and I dunno why. I am considering rewriting from scratch again and maybe come up with a way to track it better? But yeah.

Also no idea if this works on Reforged, I am on 1.30b or smthing

 

[khopesh]

The benefit I see in this is that it would save you the manual modification in the object editor and allow you to do everything with code. This is much easier since it would only be a matter of copying and pasting. Once generated, I imagine it would reduce the time compared to the normal method. Even more so now that AI can generate code in seconds.

 

[MyScorpion42]

The benefit I see in this is that it would save you the manual modification in the object editor and allow you to do everything with code.

Yeah, there's a lot of setup with the object editor that I find drains my enthusiasm for a project quickly. This'll help me with my decision paralysis when I have to pick stats for the different units, and then once I've found a balance I am happy with I can start converting it to proper custom object data.

I have discovered the bug that was causing the script to build too many Voodoo Lounges. I was accidentally overwriting the counts of buildings in progress when trying to add the counts of upgrades in progress. Now that I have finally fixed this, my enthusiasm for the project is reinvigorated.

I also wanted to write somewhere more explicitly what native functions do and do not work when run in an AI script. Some of them are mentioned here, but it is not entirely correct.

What is true is that all native functions that get/return strings corrupt the strings (although the corruption is deterministic). Only strings set as variables and arrays directly are uncorrupted. I2S (integer to string) crashes the game.
EDIT:
ExecuteFunc WORKS with MAP SCRIPT functions, just not AI SCRIPT functions. I have attached a map where the ai executes the map initialization actions as proof.

Setting and checking pathing works perfectly fine in the AI Script; it is just confusing because IsTerrainPathable returns false if the pathing IS allowed, but SetTerrainPathable takes false to disable pathing at a position, so when you have used false with SetTerrainPathable, IsTerrainPathable returns true.

"Play Animation for Doodads in Region/Circle" works perfectly fine. Functions that report if a unit is in a group or region or if a player is in a force seem to work fine.

Trigger Actions and Conditions do not work in the ai script. TriggerAddCondition and TriggerAddAction can be called, but actions thus registered won't be run when the trigger is executed, and the trigger will always evaluate to false if it has any conditions.
If a function defined in common.j or common.ai is registered as an action or condition, the game crashes immediately.
The same thing happens for all the "For all/Pick every..." functions, so as described in the sourceforge link, it is most likely due to the shared attribute of a function being passed. However! common.ai does contain one native function that takes another function as input: StartThread.

EDIT 2:
Trigger Events DO work in the AI script.
Triggers, even if they have no actions, are triggered by registered events. While we cannot use actions directly, we CAN check if a event has triggered, with GetTriggerExecCount and GetTriggerEvalCount: These will return a value greater than 0 if the trigger has executed.
ResetTrigger sets these counts back to 0, so we can use them to check if the event has triggered again.
I have included an updated map as proof: a trigger is created and the event of a unit entering the map is registered with it. The execution count is reported before a new unit is created, at the same time, and right after. The execution count is greater than 0 after the unit is created. Then the trigger is reset, and the count is found to be 0 again.

I am very happy to have found this, even if it means that I could have made my code simpler in some ways. It will make keeping track of a lot of things easier.

I don't think there's any way to make the event response/get triggering x-functions to work outside actions. So events in the AI script will mostly be useful for checking the state and actions of known units. Events with unknown units will require the Map script relaying these to the AI script.

EDIT 3: Timers and Timer Dialogs WORK, I was confused because they act strange and seemingly never actually reach 0, but I think that is just Warcraft. A timer can be determined to have run by by checking the execution count of a trigger with a corresponding Timer Elapsed event registered. Just remember that nothing about them works until after map initialization is finished.

Timer Dialogs will make debugging MUCH easier when implementing timers. I actually think difficulties with debugging were another reason I opted not to use them from the start.

Dialogs, Quests and Leaderboards all seem to work, but Multiboards genuinely do not seem to work. Good riddance tbh.

EDIT 4: Ubersplats and Images work. Unit- and Itempools work, and Trackables also work. I have added a map as proof, where a trackable with the appearance of a peasant is in the middle of the map. If it is hovered over, a message is displayed every second, and if it is clicked, another message is also displayed.

 

[khopesh]

There's still a lot we don't know about how it all works. I wonder if you've learned about variables and their size in memory. I mean total Memory Footprint (pointer,handle table allocation,Alignment, etc), especially the force type variables because not much is known besides the native ones and the pointers/handleid 32 SD, 64 HD?

 

[MyScorpion42]

There's still a lot we don't know about how it all works. I wonder if you've learned about variables and their size in memory. I mean total Memory Footprint (pointer,handle table allocation,Alignment, etc), especially the force type variables because not much is known besides the native ones and the pointers/handleid 32 SD, 64 HD?

Can you elaborate some more? I was under the impression from the Memory Leaks thread that we have a pretty good understanding of them.

While I don't have specific benchmarks, I have experienced the AI running out of memory when I failed to clean up unused hashtables or tried to start new threads constantly (even if these did not have loops, which implies that the AI script does not clean up terminated threads).

 

[khopesh]

Can you elaborate some more? I was under the impression from the Memory Leaks thread that we have a pretty good understanding of them.

While I don't have specific benchmarks, I have experienced the AI running out of memory when I failed to clean up unused hashtables or tried to start new threads constantly (even if these did not have loops, which implies that the AI script does not clean up terminated threads).

nvm but is more https://www.hiveworkshop.com/threads/memory-hack-api-description.289823/ The bit weight of the native variables is known, but not that of the game's proprietary variables.

 

[MyScorpion42]

nvm but is more https://www.hiveworkshop.com/threads/memory-hack-api-description.289823/ The bit weight of the native variables is known, but not that of the game's proprietary variables.

I am not sure how I can help. I was surprised to see that memory hack was from 2016.
I am under the impression that there is slow but steady work on reverse engineering WC3, so I think these questions will resolve itself over time. If there is anything specific I can do, let me know though.

I don't think I am at a stage where I feel comfortable playing with the memory hack yet, especially if it only works for 1.27. Check again in 2 months ig?

EDIT:
To Do:

Features and Improvements

• Town Halls are currently replaced with a different unit when upgrading right now. Replace Town Hall Upgrades with changes to the animation, name, health, id etc.
• Buildings are too bunched up. Use a combination of pathing checks and a "increasing area that peon will build in over time" to enable more varied placement. Maybe keep track of and take into account where the center of the base is?
• Change Build system to allow for easy use of different faction styles of construction
• Town Names
• Buildings names after Peon that built them
• Implement cheats for quicker debugging

Debugging

Use Timer dialogs and leaderboards to monitor

• Number of units of each type
• Number of units in each town
• Lengths of queues
• Statuses of units in the queues.

Glitches

• AI does not attack right now.
• Buildings provide food before they are finished
• Worker is never released when building expansion. Peons are invisible forever.
• Expansion Towers are not handled correctly currently
• Count units in town is not implemented
• Peons can return resources to unfinished town halls

Refactoring

• Replace Arrays with Hashtable usage.
• Null all agent pointers to ensure memory can be freed up https:www.hiveworkshop.com/threads/memory-leaks.263410/
• "Heroes" are built instantly
• Rewrite majority of code to utilize triggers and events

 

[MyScorpion42]

Okay, I have sorta made the steps towards restarting after all. Implementing debug features into the existing code frankly does not look very fun to me.

Interesting findings:

• When a building has been ordered, but the worker has not yet reached the construction site (so it technically doesn't exist), the building is added to the count for ALL town counts. So GetTownUnitCount for that unit type will count that unit for ANY town number, regardless if the town exists or will ever exist
• Units being trained are seemingly not counted by any town count.
• The tech percentage on the scorescreen is calculated by going through all research whose race matches the player race. In the attached map, I have removed all but Cannibalize from Undead, so Undead immediately gets 100% research when Cannibalize is researched. So if you want it to be reflected properly for custom races you have to set it manually.

Will have to check if Demon is calculated correctly. Don't know if there is any way to get Other as a player race, but I can try with ConvertRacePreference with an invalid number? Could also check if the neutral player slots register any upgrades

UPDATE: Did some more work on the debug menus, improved the counting and did some testing comparing the map common.ai counting functions (GetUnitCount, GetUnitCountDone, GetTownUnitCount) with the common.j counting functions (GetPlayerUnitCount,GetPlayerTypedUnitCount,GetPlayerStructureCount). I forgot GetPlayerUnitTypeCount from common.ai.

A building in the process of upgrading is generally not counted as a "Done" unit, except for GetPlayerTypedUnitCount, which counts a building in the process of upgrading as a done instance of any of the buildings it has upgraded from. Similarly, units that have morphed are not counted as the original unit by the standard ai functions.
Blizzard had to write functions extending functions in common.ai to account for these equivalencies when counting.
The counting method of GetPlayerTypedUnitCount seems to follow the one the game uses for determining techtree requirements. Does this mean it follows the Dependency Equivalents field as well?
Does counting work for the generic fields like "An Altar" and "A Hero"? Can I write more of these sorts of map equivalency fields?

So next avenues:
Add testing for GetPlayerUnitTypeCount to determine if it has any functional difference from GetPlayerTypedUnitCount.
Original undead.ai is about 23 kb, while my current file is about 47, meaning that I have written 24 kb of Jass. It is getting pretty ugly to look at, so I would like to refactor it down to half if possible.

UPDATE: Testing equivalency was surprisingly quick.

1. Yes, adding a unit type to another's equivalency list does indeed make GetPlayerTypedUnitCount count units of the former type when counting the latter
2. GetPlayerUnitTypeCount does NOT count units from the equivalency list.
3. A equivalency list with no actual unit can actually be defined in general. In the attached map, I have defined an equivalency list under the id XXXX and added the Farm to XXXX's EquivalencyOr, and made peasants require this equivalency. so you can't train peasants until you build a farm. This was done with a custom war3mapMisc.txt, but it can be imported in the map and the map won't overwrite it unless you use the Game Constants window to make changes.
4. GetPlayerTypedUnitCount and GetPlayerUnitTypeCount are NOT able to take an id without an actual unit and just count all units in the equivalency list. Otherwise, in the map, the game should report 4 heroes, but it only reports one.

 

[MyScorpion42]

Mixed to Positive results in my research on races.

It is possible to assign a sixth race to a player, meaning that it is possible to have 6 different race icons on the score screen at once, as shown here on the right. It turns out, however, that this is NOT the elusive Other Race that the JASS library makes reference to. It is the one matching ConvertRace(0). If you don't know what I mean by that, observe this section from common.j: constant race RACE_HUMAN = ConvertRace(1) constant race RACE_ORC = ConvertRace(2) constant race RACE_UNDEAD = ConvertRace(3) constant race RACE_NIGHTELF = ConvertRace(4) constant race RACE_DEMON = ConvertRace(5) constant race RACE_OTHER = ConvertRace(7) The different Player races are defined using the output of the function ConvertRace. You can see the Demon Player race, which was planned but scrapped early, is technically still there, and a "Other" race. We also see that there is a jump from 5 to 7 in the input to ConvertRace, implying another scrapped race. These player races are basically glorified integer wrappers, but it is interesting. However, there is no JASS function for setting a player's race. GetPlayerRace returns this type, but the player race is designated by the game when starting the map. BEFORE loading a map, in the map setup/waiting room, other jass code is used to set up the players and teams, and how customizable they are. The world editor compiles this code from the Scenario.../Player Properties. The function SetPlayerRacePreference sets the Race Preference, which can be Human, Orc, Undead, Nightelf, Demon, or Random. The function SetPlayerRaceSelectable is self-explanatory, and is also used for the waiting room setup. If a player is set to not be able to select their race, the game will try to give them the player race matching their race preference. But what happens if that does not exist? The answer: It gives up. Or rather, it seems that it breaks, but it does not crash Warcraft. It seems that ALL players start with the Null Race. Then, during map loading, the game loops through the players in order and assigns their race, but if it can't, it breaks the loop, meaning all players that have not received a race by that point, REGARDLESS of their preference, end up with the "null" race. This also means that the 4 standard Neutral player always have the null race. So the good news are that we can have our 2 additional races. The bad news are that It is not supported by the standard World Editor, so it requires editing war3map.j directly, and if the map is loaded and saved with the world editor the changes are overwritten. All the players with the null-race have to come last in player number (admittedly minor) Players cannot select between the extra races, they are bound to that player slot.

The attached map lets one test out how the different player races count their tech%. Demon, as expected, count all Research Upgrades with the demon race, and the null race count all upgrades with the None/unknown race. Remember to launch it though the map selection, and not the world editor. It also tests the players races against different iterations of ConvertRace(), and shows that the Null race matches ConvertRace(0), and not 6 (or) 7

I have set the the scorescreen icons for demons and null with an edited war3mapSkin.txt file (the same file that is created when you have changed the Advanced/Game Interface settings). If the map does not have this, and you don't have a modded UI/war3skins.txt, the scorescreen icons show up as green squares indicating missing textures.

Additional fun facts:

• There is some indication that players are intended to be able to have multiple preferences, as among other things the matching integers of the Race Preferences are powers of 2, meaning any combination of them is a unique number and can be represented with a bitflag.
• common.j contains a race preference designated RACE_PREF_USER_SELECTABLE, but this one is invalid and is not used. If a players race is set as "Selectable" in the Player Properties, the compiled code actually sets it to RACE_PREF_RANDOM

UPDATE:
I previously wrote that player races are distinct from unit races, but this is in fact incorrect. I have added a race test map, and it shows
None/unknown = 0
human = 1
orc = 2
undead = 3
night elf = 4
demon = 5
other = 7
creeps = 8
commoner = 9
critters = 10
naga = 11

Once again, we observe a gap between 5 and 7.
Going through game.dll in Ghidra, there is a function that matches these exactly, and it has no match that returns 6.
My best guess is that they left the gap as an indication that the remaining indices were not player races.

Since the unit and player races are not distinct, this means that there technically can exist a game state where a player is of one of the other races, but it is unlikely that this can be achieved outside of editing a save or editing memory. If it was possible, it would hinge on understanding how Warcraft assigns races to players when loading a map, to find out if there are any more legitimate values for Race Preferences.

 

[MyScorpion42]

I accidentally got way too invested in the hero ability assignments, so despite my refactoring my script size has increased to over 50kb.

Otherwise nothing new. Discovered that for some reason Neutral Hostile can be given an AI, but none of the other neutral players can. I have not been able to determine any accessible differences between them: They all have the same player slot state, they all have the same player controller type, none of them are designated as observers.

Also if you haven't seen it, check out my updates on equivalency and the relationship between unit races and player races

EDIT: No progress on the AI, but I have been experimenting with the ids of different things. If you pass an invalid terrain ID when setting a terrain type, the game renders it as a black square. But I have made a map that tries to do this hundreds of times, and it seems at some point to default to rendering all the invalide terrain as the same, valid terrain type. Maybe when it first encounters a valid type, it puts it in a buffer, which is then used for invalid types going forward. I have attached the map (clump); be warned that it is full of memory leaks so lag will probably happen

EDIT: I had not realized how the 16 tile limit worked in relation to changing terrain type. My new theory is that if an invalid id is used, the terrain is set to the first unused map tile index, and the first time a previously unused terrain id is used, it is then slotted into the first unused map tile index, so therefore all the black squares get replaced with the tile that has been put into that slot.

EDIT: Did some quick tests this morning to see what other functions in common.ai work in map script.
UnitInvis, CreepsOnMap, IsTowered, GetBuilding and GetPlayerUnitTypeCount work
UnitAlive, Sleep and surprisingly GetCreepCamp do not work
Whenever I tried to use StartThread, the Map compiler would change the function to an integer and complain, so I don't know if it actually works.