Listen to a special audio message from Bill Roper to the Hive Workshop community (Bill is a former Vice President of Blizzard Entertainment, Producer, Designer, Musician, Voice Actor) 🔗Click here to hear his message!
This thread is a place for Warsmash developers to post notes on their updates and progress in maintaining and improving the engine code of Warsmash.
Current progress history:
Project ideation began 12/02/2018 fall. I had practiced in the spring with the HiveWE source to do unit animations in a rendered War3 style game map preview. I created a web document describing my visions and goals:
Use Hive's "View in 3D" viewer code as a basis for rendering units
Use HiveWE terrain code as a basis for rendering map terrain
Copy game engine unit logic from my previous attempt at making a strategy game where possible
Use HiveWE's ShadowMap.cpp as a basis for parsing .shd files
Each Map File will be an MPQ archive parsed by Dr Super Good’s Java MPQ parser
There will be a terrain data file with the extension “.w3e” that will contain a grid of map terrain information, loaded with a Java transcription of “Terrain.cpp” from HiveWE (a “w3e” parser)
There will be a terrain shadow map file with the extension “shd” that will contain a grid of shadow information. This can be made from either a transcription of “ShadowMap.cpp” from HiveWE, or “ShadowMap.java” from OgerLord’s open source WcDataLibrary
There will be basic Game Object Data information stored in a series of SLK tables, for which I already have Java code to parse, available in my “JWC3” code workspace, using my own SLK parser implementation. These files will be specific to the Game Application, and should not be included in Map Files
There will be Game Object Data Changesets stored in class-specific binary changeset files. These files will be used in Map Files to describe which Game Objects have been modified. For these, I already have transcribed code from PitzerMike’s C++ Game Object Data Changeset parser.
There will be a class of object that is immutable at runtime and baked into the terrain called Doodad that will serve as a 3-dimensional decal. These will be stored in “.doo” files which will specify Doodad type and location. Doodad type will be an “integer” variable referencing an entry in the list of possible Doodads, which will be described by Game Object Data
There will be a class of object that is mutable at runtime, and can be destroyed or removed, called Destructable. These will also be included in the “.doo” file, for simplicity, but will have an integer key referencing a different hashtable of unit data information.
The combined pathing map of the Doodads and Terrain Tiles and decals will be generated by the Map Editor and saved in a “.wpm” file, which will be a binary definition of pathing map information. This can be parsed with a transcription of the “PathingMap.cpp” file from HiveWE
ANTLR will be used to generate Java-based parsers for the JASS syntax, and then user created JASS code will be present in maps. The parser from “wc3libs” for this can probably be used (see “Jass.g4”), in conjunction with the prototype written by Retera used to illustrate that code arrays can be safely added in the Java Virtual Machine with a custom interpreter for the JASS language.
TGA Targa image files will be loaded using OgerLord’s “TgaFile.java” open source code
BLP image files will be loaded using “blp-iio-plugin”, an open source BLP ImageIO parsing plugin for Java
ANTLR will be used to generate Java-based parsers for Frame Definition files to build the User Interface for the Game Application. An open source Java repository named “wc3libs” claims to have an FDF parser, but they seem to only support a small subset of the specification, for parsing constant definitions. This will be improved so that we can parse the entire UI definitions for the Game Application
Rendering models will used a transcription of “mdx-m3-viewer” from JavaScript to Java. I already have a working transcription of this code in “JWC3” however it was written hastily and with bad performance. For this project, we want to try to support as many units in the game at a time as possible, so we will need to transcribe the instanced rendering methodology from “mdx-m3-viewer” as well as the logic.
Methodology must be added for selecting units and issuing them commands. Units can be given commands with a target, and will have one active command at a time. The specific behaviors of the commands will be defined as hard-coded behaviors for the various entries in “Units\AbilityData.slk”, although multiple entries can use the same behavior by way of the “code” column, which will be a hashtable entry.
For this logic, I will borrow an adaptation of the command system used in my own turn-based strategy game, modified so that it utilizes lock step per frame instead of per turn. Any player will be allowed to command any unit controlled by that player. Each behavior will have an Order ID hash value, and multiple abilities using the same behavior will not both be usable by the player -- save for the ‘ANcl’ entry, which will be allowed to specify a different Order ID hash value to use for its network traffic.
In the case of a unit with Order ID collisions, the first available ability will be activated when the unit is commanded by a player. If the ability is on cooldown, or the unit has insufficient mana, then the second ability with that Order ID will be used instead.
Players will be able to chat in multiplayer games. This feature already works in my own strategy game, so the same system can be borrowed
Fall 2019: Experiments shows that a direct port of the JavaScript MDLX save/load code present in mdx-m3-viewer was able to run from within Java and load and save valid MDX model files that still worked in the Warcraft 3 game as expected
January 2020: First functioning gameplay display is created through a lot of hackery and attempts to push something out by Jan 28 to appear relevant. There is no combat and units still pas through each other having no collision system
Mid 2020: Basic pathing systems are implemented using the A* search algorithm as described on wikipedia.
What is above is a summary of progress that has led to where things are at the time of writing.
As more progress gets made, we can update this thread more!
Practiced how it might work to onboard a new developer in a chill call with @Spellbound . We both redownloaded WarsmashModEngine from source and then changed our setups to load Warcraft III content from a Reforged 1.32 install. It was a good time.
We gave ourselves the goal of eventually making Human Build while chilling, probably in a future call. This will require first making Human Repair. In order to make repair, I chatted with Spellbound about the current API and how to configure a new Java class to extend CAbility. The goal is that the class we worked on will eventually be spawned for Abilities with code=Ahrp, although we did not do that configuration yet.
The chilling and chatting already left us brainstorming how we might improve the Warsmash API for implementing new builtin skills in the future. I am kind of hoping at some point the skills will all be implemented in scripts external to the engine, but for the time being I am using basic Java classes and CAbility as a Java interface.
Later in the weekend I posted on the Tools section download of the binary build with a sample config for running with a 1.32 installation.
In accordance with that post, I updated the main branch of the public repo with a few changes to improve 1.32 compatibility when browing Campaign menus. However, the Warsmash engine still is not capable of launching maps created with the 1.32 map editor, so clicking the maps on the Campaign menu does not play them when using a 1.32 installation as a source of Warcraft III game data because Warsmash while crash during map loading.
All in all, I would say this weekend was a bit of a vacation as far as Warsmash development goes (aren't they all).
Last weekend I was trying to implement some JASS natives for the APIs such as Timers and Regions. Back then I wrote a lot of untested code and I was hoping to build out Jass support and then test it "all at once" with systems testing. But I am not at that testing phase. That will probably be for a future weekend. The goal here would be to have a stable building block that can easily support the "config" function in JASS so that map configurations can be better supported.
(I do not get much done during the week. I have a job! So, 'til next week, folks!)
Tonight @Hayate and I had an opportunity to play Warsmash together even though he was on an ArchLinux machine, so we had to do some workarounds to get the downloads from Hive to run on his machine.
After getting the download to run for him, I hosted some games from one of my Windows machines using the warsmash.net server and we played:
a melee map to completion in a small 1v1 battle
WarChasers together and got past the second mini boss with some hackery (and then were stuck at the WayGate, at which point we turned around and my game crashed)
A custom map that I drew in the moment with some models downloaded from Hayate's resources section here on Hive, where we each had a Murloc Fisherman captain and we fought some enemies to obtain some items named "Good Code +12" (this tested playing a map that his machine did not already have)
A second game of this custom map where I updated some stuff and gave us a custom base that could respawn our units and hero and reduced the difficulty
All in all, had some good fun, but a ton of bugs! Definitely a lot of work to do if we want to make this system bulletproof.
I did some traveling and got away from my bug list created for the last update, so in the meantime I had fun with the single player side of things, and the in-game entities and behaviors.
I created a custom map to play on the Warsmash engine where I tried to let myself think of systems that I wanted and then add them to accommodate:
Updated the Immolation ability that I had started work on in September and got this to where it was working, including mana drain when active and related behaviors
Added mana potions to help with using this immolation skill
Added "issued order" event handlers into the game engine, so that I could fire map triggers when units were given orders
Added "unit acquires item" event handlers into the game engine, so that I could fire map triggers when units were given items
Fixed some bugs relating to CTimer and CTimerJass implementations
Fixed some missing natives to issue an order to a unit, so that (for example) a wisp can be told via trigger to build a structure
Implemented a concept of "owner" for an item, so that the "RemoveItem" native will not only destroy the item from the game's 3d world entities but also remove it from any hero who was carrying it (instead of putting the item into a conflicted state that still showed the icon on the hero despite being remove)
Implemented ALT+ENTER hotkey to toggle the game between windowed and fullscreen at runtime
Implemented support for Tinting Value on the Doodads and Destructables and fixed how this field does a lookup per doodad variation
Partially implemented the native for camera pan, so that triggers can do a rudimentary (but instantaneous) assignment of a player's camera.
Also, some time was spent toying with the notion of Root for the Night Elf structures. The ability is partially functional but quickly crashes the game after use, because of invalid writing to the world collision grid that is as of yet unsolved.
There might be a few other changes that I forgot, so for a full log feel free to see the git commit history. Also, some of the above changes might only be on my local workstation and will probably only be pushed to the GitHub later after writing this post.
If you want to see a video of the little custom RPG map that inspired most of these changes, see my YouTube channel.
Updates through the end of 2022: * For the winter holidays of 2022-2023, when I had some downtime I decided to try to get the Heaven's Fall Warcraft Mod to play on the Warsmash engine largely as-is without substantial changes to the mod data that was meant to target Frozen Throne
- The only Blizzard.j change required, if I recall, was that I natively changed the GetPlayerRace and GetPlayerRace natives to return a larger list of races, rather than to use the Nirvana MultiRace Template2 natives how Heaven's Fall historically had been. So the MeleeStartingUnits function no longer had to check a game cache and sync the cache in multiplayer with gamecache sync natives or whatever, and instead went back to the simpler system from Frozen Throne but with more raced added. To accomodate this, inside the Warsmash engine itself I added race manager classes that allow the race list to be entirely pulled from the Warsmash.INI config and not be hardcoded to the game. That is, perhaps, one of the substantial changes I always wished or dreamed would be added to Frozen Throne, although admittedly if Blizzard were to do it today it would be better to make it configurable on the map level rather than on the game engine settings level. The Warsmash engine currently defaults back to WC3 races if no config setting is specified, however, so users don't have to update their Warsmash.INI when upgrading to a new version of Warsmash.
- For this holiday patch cycle, I also got neutral item shops like Goblin Merchant to function as well as player-owned racial shops from Frozen Throne. However, in practice there is still a lot missing, including item stock and stock delay and racial player-owned shop tier requirements. So in practice, as was evident in my video teaser from January, currently if a human player builds an Arcane Vault they can buy an infinite supply of healing potions or mana potions even at tier 1 (Town Hall), and they can buy other items there that mostly all don't function because their corresponding abilities are not implemented.
- While I was working towards possibly playing the Heaven's Fall + Warsmash experience in person with some folks on the holidays, as shown in the January teaser video I also realized that Water Elemental and Feral Spirit based abilities accounted for a larger than normal number of Heaven's Fall abilities and that by adding support for these Object Editor settings, rach of the six races in Heaven's Fall would probably have at least one hero with at least one working ability. That included introducing the blue Timed Life bar and related buff functionality for the summoned units, and was also shown in my January teaser. Updates for early 2023 * At the end of the winter holidays when I finally tried to actually play multiplayer of the holiday update with Heaven's Fall, everything crashed and burned on my last night in vacation when the other folks actually had the chance to play. I had intended to play a multiplayer custom map with preplaced enemies so that it could be a team game. However, I made the mistake of assuming all map systems would be operational in multiplayer. In particular:
- The first major bug happened on multiplayer game joining. Visually as a user, the game lobby systems all broke apart and didn't work. Careful analysis after the fact made me realize that this happened because I was trying to use preplaced player teams with preset slots that were not in order. For example, Blue, Green, and Teal were used as the Allied team and Red, Pink, and Orange were used as the Enemies computer team, or something like this. While that would have been a nice visual, the translation layer between local client "slot index" and the "server slot index" where the server just shows 6 slots -- NOT "slot 1 (red), slot 7 (green), slot 13 (dark red)" as we see in map data but just 1 2 3 in order -- systemically was all broken. With the server confused, it was effectively booting players and overwriting lobby data without notifying the player client to exit the lobby so users could join and not see themselves and the game lobby was just very broken.
- The second major bug was long-standing, unsolved issue that I still do not know the full explanation for. Basically, during the ongoing lockstep simulation of the game, there is a certain circumstance or edge case in the current code when all clients to the server believe that they are all waiting for all other clients. Units run in place, and all game behaviors stop but players can still pan around their cameras and look at the game world. When the simulation reaches this state, even though both clients are still running and the server is still running, the game never recovers and is mysteriously over. About a month later I tried playing with someone in person and realized that there is a fairly easy way to introduce an expected latency in the server code so that the two clients do not have to "work as hard" to keep up. When this expected latency is added, the clients almost never reach the situation in which they are neck and neck waiting for each other, and so they almost never reach the situation where they are hitting this bug case where everything freezes waiting forever described above. So I have a "solution" that reduces the probability of failure, but while that increases the probability of me having fun when I use the solution it also decreases the probability of me learning about the nature of this bad bug and fixing it.
Because of the two bugs listed above, no multiplayer game was actually successfully had during the holidays. But after that when trying to get a new player to test Warsmash with me in early Feb, I used this artificial expected latency trick and had maybe an hour long multiplayer play session with someone who had never played Warsmash nor Warcraft III before, and to be honest we had a pretty good time in my opinion.
Between the winter holiday technical issues and the early Feb successful multiplayer session, I recorded and uploaded the "Warsmash - Night Elf" technical demo video where I built a night elf base and enjoyed a ton of new functionalities that I had been programming leading up to the recording of the video. The upgrades included the Root ability for Night Elves, which I upgraded to support changing armor types and attacks in the expected way. I also added the Bone_Turret system that turns a tower to face its target although that system obviously has issues. I also tried to finish building out the Load/Unload mechanic so that I could have the Entangled Gold Mine actually work, although the way it ticks to supply gold is not in time with how Warcraft III does it, I don't think. But it is functional enough that the Night Elf techtree becomes usable now on Warsmash, which is fun to me. I also put in the moonwell ability with animated water height per the well's mana pool. That was also fun, and shows in the "Warsmash - Night Elf" video. A user on discord once told me that video was "boring" because it was "just a melee game," and it was almost funny to me when I think about the almost indescribable experience of building out this technology after all these years, that someone can appreciate none of it at all so perfectly like that, and yet when they do so it might mean the facade of what I was emulating was actually working.
Updates for March 2023 * Around the beginning of March, there was a day when I imagined to myself that it would be fun to modify Warsmash and add a WASD controlled third person player with a ported WoW model, simply for my own entertainment. Sometimes, it can be good to feel free.
- After about 4 hours of programming on the first night, as I went to sleep I published a video of myself enjoying this new technology toy that was only the product of 4 hours. I played on a basic melee map -- The Two Rivers, if I recall -- where I recorded a video that begins with the voice of Thanos in the background saying, "Reality is often disappointing. Or at least, it was. But now... Reality can be whatever I want." but after he says this, I press the ESC key on my keyboard and switch the camera to the Third Person camera with the WoW model and the video starts playing Jace Hall's, "I'm playing WoW!" song in the background for fun. I titled this video "World of Warsmash," poking fun at how the experience kind of looked like playing World of Warcraft but on a Warcraft III map.
- On the second night of this project, I changed the code so that the player character could pass over or under a bridge based on their Z height that was used for jumping and falling. A bunch of code changes were happening under the hood for this, so although the player character functionality looked roughly like the same one that was on the first "World of Warsmash" video on YouTube, the underlying programming actually changed quite a bit. Originally the 3D model of the WoW character was gliding upon the War3 simulation as a model instance in 3D managed by the User Interface. In this way, it was only as much a part of the game as the Rally Point model or maybe the ghosted building cursor used to place a building. These User Interface-managed models have absolutely no netsync. If you tried to multiplayer with it, each player would have their own WoW style character on their own computer, and they couldn't see the other character from the other computer. This meant movement was really smooth because it was all on the local client. So it was fun for me to run around, but it can be valuable to understand the limitations of this design.
- For this second night, as well as going under and over objects based on Z height, I also put in special code that the player could "ride" on a mechanical unit like a boat or steam tank, in a very hacky way. Notably this used the model data of the mechanical unit and not its unit data, so if the mechanical unit ever passes offscreen, the player falls through it. This was fun to mess around with but would probably need a lot of changes to be used as a system in some kind of production-level game. I recorded a video of these systems, using WoW ported models downloaded from Hive like the Undead Ziggurat I could walk inside of, and also using Reforged models for some of the mechanical units to ride, and uploaded this video titled "World of Warsmash Reforged."
- On the third night it was becoming clear to me how much I was partial to enjoying the World of Warcraft style experience and wanted to push this experiment for as far as it could go for fun. This was a Friday night and a weekend where I had more time. I began trying to load up some old experimental code I wrote for @Hermit to port Alpha/Beta WoW models to Warcraft 3, which he requested from me when trying to find a very old special variant of the Murloc model from WoW. That experimental code allowed me to assets in Warsmash that I otherwise might have had to spend a long time porting. Even with that code available, I was still patching the model parser and losing a lot of time behind the code and having less time playing the experience, so after a time I fired up JNGP and drew myself a map with terrain tiles ported from the old Alpha/Beta WoW archive that Hermit had asked me to look through using this port script, and with doodads that were all automatically converted to MDX when I used the code I had written for Hermit to port all of the MDX models in that Alpha/Beta archive all to the Warcraft 3 format at once in an automated way.
- The resulting map created that third night was a fun little world with Feralas style trees and some elven style decals. I found a model called HighElfFemale_Warrior that had all the right animations for me to support making the character feel more and more WoW ish (notably this was before the Burning Crusade and before its style of High Elf, so the character is basically a Night Elf without purple skin). I recorded a video of myself running around this custom map at 5 AM, with a Text-To-Speech synthesizer explaining that maybe that 5 AM experience being drawn into whatever "World of Warsmash" was might be an experience that showed the misguided element that got me here in the first place, that perhaps I was more partial to liking that style of game than I usually admit to myself when I think of myself often as a Warcraft 3 custom faction type of guy, or that a custom faction engine might be the long term goal of Warsmash after it achieves Frozen Throne feature parity in the future.
- In this strange simulation, I wanted more, and in particular I wanted a UI and an inventory and the ability to combat with enemies, so that I could go on an adventure and collect the loot. And maybe use YouTube as a platform to tell the story of that adventure, like the "Retera's Dungeon" video, which was a fun way to experience Warsmash even if it was in many ways like a choreographed play where I knew in advance what would happen at each step.
- In pursuit of that adventure, time started getting away from me. Actually, it might have already been the fourth night when I published the Feralas style video with the female High Elf running about. I made another video titled "Getting trapped in World of Warsmash's first dungeon" where I drew a large geometric playground in Retera Model Studio, with a jump puzzle to enter, but when I got into the geometric playground I could not get out. It turned out that my "ray test" to intersect objects beneath the player's feat was sometimes optimizing its code by only checking for one intersection, instead of finding the nearest, for any given model. This meant if the downward ray could hit 2 or more parts of the geometric playground, I would possibly just fall through them. I was able to patch this, but I only figured it out after that video was recorded.
- By this time, it seemed YouTube or other systems were successfully drawing user attention to these "World of Warsmash" videos, which was sometimes fun, but it is important to remember what Warsmash is so that accordingly we remember what it isn't. And everything in these videos that was demonstrated was not multiplayer, and was not about to be, because it did not tap into the Warsmash lockstep simulation system, nor did it have its own WoW style client/server which would have needed to be entirely re-written versus the Warsmash server and multiplayer systems. Even by the third day or so when I refactored my short code for the User Interface managed 3D character model, and I moved it to its own file called Player Pawn to maybe step closer to a world where code would support a system where I could have 2 of them in the world, despite this they still did not touch the Warsmash lock step simulation layer at all.
- In the YouTube comments, somebody caught on to what I just described, and said in their opinion the "World of Warsmash" toy project was not worth pursuing because it was not interfacing with the Warcraft-3-style simulation at all. More and more, this helped me to visualize what I wanted.
- To get to a future where I could go on an adventure as the "Player Pawn" with the WASD movement and XYZ instead of XY coordinate system, the two major systems I would need would be either (1) a User Interface in the style of a third person game, and (2) the Player Pawn being able to deal out damage to Units/Widgets of the Warsmash simulation and receive damage from the same
- Pretty quickly I knew that for problem #1 it would probably be easiest to load in some WoW UI assets from the old WoW Alpha/Beta archive I was porting from, even though for problem #2 there were two possible solutions depending on if I never wanted multiplayer or not. If I never wanted multiplayer, I could take my existing Player Pawn code that ran only on the local computer and use it for a visually perfect/smooth experience that will never be networked, allow it to deal damage to Warsmash simulation Unit objects (relatively easy by calling the function to deal damage) and then also allow Player Pawn to be targeted and attacked by those objects by making a new type of Widget (like how units can attack Items or Destructables even though those aren't units), but where this new Widget wraps the Player Pawn. Sometimes the wrapped player would surely move around and act in ways inconsistent with the Warsmash lock step simulation, but if I didn't care it could be ignored. The other possible solution that might look worse but might support multiplayer, I figured, would be instead to reinvent what Player Pawn is as an ability given to a Warsmash unit. That way, in the same way a Warsmash unit with the Move ability can do the Warcraft 3 style walking around or pathfinding around objects, maybe a unit with Player Pawn ability would idle visually until moved with WASD, which would then be their current action.
- So with these two dilemmas, at first I started porting WoW XML ui into the Warsmash FDF system which honestly worked pretty well and made me think WoW UI probably originally started as a fork of the Warcraft 3 FDF system at least partially. Then I started trying to use LuaJ as means for executing the WoW Lua on top of the WoW XML so that the UI would begin to come to life. After a couple of nights I got that working, but because my job is a priority the time requirement for getting XML+Lua slowed down my releasing of Warsmash videos, since I could only work when I managed to get time.
- The WoW UI got to a point that is promising but not entirely satisfactory for me yet, and then I started to want the combat. I decided to try to build the Player Pawn "as an ability" idea in the hopes that maybe this "World of Warsmash" stuff might work in multiplayer in the future, albeit in the confined of a lock step Warsmash map simulation under the existing code. I got this working, but it took a toll on the quality and smoothness of the unit movement now that the player is actually a Unit and is constrained by the Warsmash lockstep simulations 20 turns per second rule (as opposed to a local client character updating smoothly on every frame update, as featured in previous "World of Warsmash" videos). This loss of visual fidelity was a cost but in return it meant that now the WASD and arrow keys were operating not directly, but as "issued orders" in the system of units that would also probably work in multiplayer. The Player Pawn ability stored a special Z value on the unit and I changed the concept of units in Warsmash so that if they have this ability, they abide that value. Since my intention was not to give the unit who had Player Pawn ability any instance of the WC3 Move ability, therefore the player should never have code running that would make them move in the style of wc3 and should be able to still intersect geometry or jump under and over physical objects while moving. This loss of visual fidelity in return for functionality probably makes the character movement much closer to what we would expect on Warcraft 3 Reforged from Blizzard, for indeed I am not sure how to simulate the smooth moving characters feature in my other videos while using the Reforged engine, and it might be impossible.
- Despite the trade off, in some circles online I circulated an unlisted YouTube video of a group of Gnolls attacking the "Player Pawn" that was now a unit, while the player gets help from a WC3 hero because it did not have attack abilities to fight back yet. Surprisingly, for those few people who saw it, not many people commented on the loss of visual smoothness in the motion of the character. This may be due to the case that although the character simulation updates on a slow 20 turns per second clock, the Warsmash engine interpolates the movement visually between its true locations (it already was doing this as a part of the Warcraft 3 emulator, but I tried to make it now do the same in the Z dimension for units with the Player Pawn ability).
- In the video with the gnolls, the unit with the Player Pawn ability dies and is no longer able to move, which is actually all inherited behavior from the Warsmash unit concept and did not need to be manually added, which is cool.
- At the time of writing, this video dodging gnolls was the state of the art. Who knows what the future will hold?
Updates for Summer/Fall of 2023 Hey folks! I don't know if anyone reads these updates. I suppose it has been a while.
The previous updates stop in March 2023. After that time, in April 2023, I received a device in the mail that I originally pre-ordered in 2019 that works as a replacement phone for open source software enthusiasts, running neither Android nor iOS. After I received this hardware -- a handset that gives power to the user to decide what to do -- I started to face a new, big challenge in my life: the issue holding back GNU+Linux from being a valid phone solution wasn't hardware, but actually a matter of software. Giving the user the power to do anything actually incentivizes to do something. The word "phone" and this idea that society wants us to have is having an increasingly profound impact in all of our lives, and often not for the better. Holding in my hand a phone that honors the principles of the Free Software Foundation as closely as possible, and seeing it work in all its ways, then realizing that I wasn't allowed to use it for certain life tasks where I need a phone because of the world trying to compel me to use other stuff via the social influence maybe helped flush me from the Matrix a little and take a step back and look at myself and how even despite all the time spent writing Warsmash, in many ways I am as trapped as anyone into a consumerist future that tries to stop us all from understanding the power of technology and instead push us towards being exploited.
Because of this, a lot of my "Warsmash time" on weekends that would have gone towards advanced progress ended up getting sucked away by these interesting technology toys. Additionally, around April or so of this year I discovered a recording of myself in the year 2022 sitting at a computer and explaining why I wanted to disappear from the Warcraft III modding online scene. To say, "poof," and vanish was very alluring to me at that time. And I laid out plainly in front of the camera my reasoning, then never published the video. The old video made me realize something. I had been doing "emotional damage control" from the standpoint of how I handled the world's response to the my summer 2022 parody being taken out of context as though it were truth -- and all these news outlets that reported that Warsmash was given some shutdown letter from Activision, even though Warsmash was never given any such thing. And in that damage control mode, I forgot my original rationale. I made an apology thread on Hive back then in a bit of a hurry that has a lot of words, but not necessarily a lot of introspective words. Seeing a literal recording of myself from before the events of July 2022, explaining why I would want Bobby Kotick and his henchmen to have no faith in me, gave me a weird moment of clarity. I described in plain English that:
(1) Activision was clear about the fact that from their perspective, the future of Warcraft III is a mobile game called Arclight Rumble
(2) Our formerly insider friend, Kam, had one time off-hand mentioned something to me essentially espousing the idea that if Activision ever fully ruined Reforged, we might need Warsmash to replace that and save the world
And in this unpublished video describing my frustration recorded back in Spring 2022, I was having a moment of this sort of cognitive dissonance because I was facing a lot of uncertainty in my work life away from Hive and Warcraft, and to have this career uncertainty while in the meantime on the side having a former Reforged producer/Blizzard employee telling me that the future of Warcraft III might entirely depend on me, felt deeply and ethically wrong. I used the word "indentured servitude" to explain this idea of the feeling of being owned by Activision in the video, while their people perhaps knowingly leave Reforged to rot and focus instead on Arclight Rumble and assume that people more passionate than themselves will go rewrite and upgrade what Warcraft III is for them, as though being passionate about a game is itself weakness.
For me that self-reflection is profound. It realizes the idea that people online mistakenly thinking Warsmash was taken down was tangentially originally my intention, because I was in this dark place wanting to intentionally destroy my reputation in Activision's eyes so that they would never depend on me, and instead would do their job as the maintainers of Warcraft III technology. I never meant for this to sabotage Warsmash, but I did not want the "official" maintainers of this game to depend on me, because I am a hobbyist and that is not the best future for their paying customers and our collective player groups like Hive, at least not yet. As comical and unlikely as it seems in hindsight to even try to imagine it, if Activision announced tomorrow that they were shutting down Warcraft III and telling everyone to download Warsmash, I would probably get a lot of hate mail because of all the unfinished features. Anyhow, because of how profound the recording of myself was to me at the time, and how much I had wanted after the fact to try to lie to myself and say the misinformation rumors about a take-down request were all some mistake and I never meant for it all to go how it did, I began to wonder if other equally profound self reflection could be discovered in other recordings and files about Warsmash that I had lying around on my computer. Attempting to compile unpublished videos and works into a collective work ended up with me creating a 12 hour video file on my computer over the summer, that is about 30 GB and focuses primarily on the development of Warsmash and ingame footage annotated with features being gradually added to the simulation. But I also included clips of situations such as:
Fall 2019 when I was describing to a friend online how YouTube was trying to get in my head and make me think of Warsmash like a takeover
Summer 2020 when I was getting a lot of online communication asking me to help with modding Reforged, and I found myself resenting the Hive and its people for caring about that instead of Warsmash, and found peace in developing Warsmash and forgetting about them all
Fall of 2021 where I kept thinking to myself that maybe I was going to hit the end of it all, and have this grand opportunity of Warsmash be taken away in my life, and kept wondering if each iteration of this beautiful thing might be a sort of last stand of me pushing the code to the breaking point as far as it would go so that I could shove it into a YouTube video and brag about it by creating a vertical slice of the technology that I actually wanted
Summer 2022 where right before mucking up Warsmash's public image, I recorded myself porting Warsmash to a phone, and felt almost legitimately terrified at how the phone world we live in today made me feel like I had DONE SOMETHING, that might legitimately bring down the hammer of Activision on me, because I had Frozen Throne on my phone and that's a better game than Arclight Rumble, and suddenly it was like Warsmash wasn't just Warcraft III and I had actually made something that this company could not, by bringing Frozen Throne to mobile
Summer 2022 when I had about 30 messages on Discord from Hive users trying to reach out to me to learn more about if there was a take-down, and express their condolences and support, and I ignored them all and went on vacation
Fall of 2022 after Warsmash's public image was ruined and I struggled greatly with it and regretted it, and started to doubt myself and the reason for anything
Early 2023 where I modded Warsmash to look like WoW and then recorded myself in front of the WoW footage saying that "if it was wanted" I proved we could have had a better Warcraft with a better editor, and that "I unequivocally win" and that I did prove it, even though Activision doesn't care and no one cares.
After that, it seemed a little like this world was on pause for me. What was I doing, and why was I doing it? Maybe I thought that creating that 12 hour compilation would exonerate me from the people who still get mad at me when they find out Warsmash was never taken down. Maybe I thought that it wouldn't. Maybe I just wanted to produce a video that portrayed a creative expression of the feeling of making Warsmash, as closely as I could approximate it. And I think that in the end, it was as profound as I had wanted it to be and was what I sought to create. But I did not put it on YouTube, because in all the self reflection there is also this truth that I have been very self interested on this topic. And it is probably not actually healthy for any other person to watch the "12-Hour Chronological Evolution of Warsmash" memoir. History is being made every day. It would be wiser for our people to put the Warsmash code to work, to serve them.
In the fall of 2023 I found myself visiting in-person with some folks and then we managed to get into a 3v3 game of Warsmash against the computer, there as our in-person LAN party. By the end of the game, two of the three human players were sitting at about 7 fps and only the user with the beefiest computer could really see the units or what was happening. Since we were all folks who had played Warcraft 3 twenty years earlier, it certainly felt a lot like a regression.
But the third user, who had recently built a PC and had something so beefy that it was dumping out max FPS no matter what we threw at it, got a good game experience, and has been making epic contributions to Warsmash on GitHub in a pull request that is not merged yet. He sought to solve the obvious problem that custom abilities should have their behaviors be defined outside of the game engine itself, and constructed a JSON based ability definition mechanism and then proceeded to use it to implement many, many hero abilities. I've seen Crypt Lord using Impale on his Warsmash fork, and units with morphing skills like Druids of the Claw who can turn into Bears, and Druids of the Talon who can transition to flying units, and a Demon Hunter who could Metamorphosis, and acolytes converting into Shades at a Sacrificial Pit, most all of the Aura abilities for the heroes, Peasants being able to use Call to Arms, and many other epic additions that were all running in his Warsmash fork on top of the open source system. To help facilitate his work, I also have a branch on the Warsmash repo that changes the SLK parser systems to finally load custom data more like Warcraft III into a single property map, instead of loading editor-like layers like what Warsmash was doing previously.
And so, I think there may be exciting new progress on the horizon for Warsmash, but most recently this has been due to the works of others moreso than myself. So probably now, more than ever, I ought to give my thanks here to Hive and to the contributor described above and to anyone who has helped me on this awesome, epic journey of trying to build an open simulator of Warcraft III.
Tonight I received a new laptop that tries to follow the principles of the Free Software Foundation as much as possible while also being fairly decent hardware, and I opened up this Linux machine, cloned Warsmash via git, downloaded Warcraft III for the assets, linked the two together, and did a single-player play session of Warsmash on a freedom respecting system in my (hopefully) freedom-respecting rewrite of this game.
And to my surprise, it was beautiful. The game sat at a perfect 60 FPS for the entire duration of play. This was a far superior experience to the Windows laptops that I had used at that LAN party mentioned above, and this libre machine doesn't even have a graphics card. After I was finished building my army and having a great time, I came here to post also using the freedom respecting machine.
So, thanks again to the Hive for the insurmountable and indescribable contributions to modding that I was able to copy as the basis for Warsmash, and for all the inspiring people that I have met on my Warcraft III journey. As I played this melee map against no one, smashing creeps inside the simulation that belted out the perfect 60 FPS for who-knows-what reason on the software-enthusiast freedom laptop, I thought about the different philosophies for modding Warcraft III and how it is fascinating to me that I could have this experience and build these units without Activision ever coughing up their source code and without ever sitting around and decompiling the game. Because, on a fundamental level, I realized that when we play and mod Warcraft III we are experiencing a digital simulation made by human hands. And human hands can make that simulation again. Despite the process of writing Warsmash having mostly been a matter of me recording my thoughts and my memories of the Warcraft III experience and not a de-compilation of the original -- despite all those little subtle times where I was "wrong" about something and the way the game behaved didn't match my childhood -- I was nevertheless "right" that this game is an idea. Something in our minds and in our dreams, that we collectively choose to pretend to experience.
And Bobby Kotick will never truly own those minds. Not really. Because their belief that their old legacy code is the only Warcraft III -- the foundational idea that their secret sauce was their unwillingness to share power -- is wrong. The power comes from willing suspension of disbelief, and the choice to imagine that there was ever a strategy game at all. And I can play Warcraft III entirely without them, just by thinking about doing it, because I figured out what it is.
Updates for Winter Holidays (2023-2024) During the winter holiday season, I was in a situation where I wished that I could do a LAN party comp stomp on Warsmash, but this was blocked by the fact that I don't run the Scripts\human.ai style of AI scripts yet. To move in the direction of running them, I used up some time trying to rewrite my JASS execution layer so that it would execute with its own concept of "JASS VM threads" that were pause-able, so that many could run simultaneously and each use wait functions. This would have the side benefit of fixing TriggerSleepAction to be something that Warsmash could actually implement technologically, but primarily be useful for implementing the StartThread and Sleep natives from Scripts\common.ai into my emulator.
The endeavor was a success, although I still did not build off of the pausable JASS execution layer to actually create the rest of the needed common.ai natives. To implement the pausable jass feature, I had in mind that I needed to choose one of two strategies for how to build the code:
Change every step of whatever JASS does -- such as the concept of what a jass code statement AST node in the interpreter are -- so that each execute(...) call on a statement would include some callback to do the thing after it instead of directly returning the result of its computation.
Change the fundamentals of the JASS interpreter so that it has an instruction pointer for what to execute next and its own stack that isn't the Java Virtual Machine's own call stack, but rather a literal system of my own making that tracks JASS stack, our point of execution within it, and the parent stack frame to return to, all with the intention of making this construct something we could freeze in place and come back to -- and instantiate multiple times.
While trying to decide the best course of action for the above two points, I had some design discussions with ChatGPT and Google Bard on the topic. Both of them wanted to push me towards idea #1 as being the superior solution. So, I decided that AI sucks and I wanted to do idea #2, because I am the human and I make my own decisions. Existing within some sea of infinite callbacks seems like it would end up as unreadable sludge when I could simply make some state trackers and then decide when to execute or not to execute them one instruction at a time.
To accomplish this, I would have to change a bunch of code. Previously I did a simple parse of the JASS instructions into an abstract syntax tree (AST) and then kept a simple mapping from function name to the top of the function AST. So, in the Warsmash JASS execution layer that existed from late 2021 until now, executing a function was as simple as functions.get("main").execute(new TriggerExecutionScope()) for the most part.
As an example, let's use a simple JASS file that was in the Warsmash jassparser module and was capable of executing for test purposes without running the game itself:
JASS:
globals
constant string TEXT = "Hello world global value"
endglobals
native BJDebugMsg takes string msg returns nothing
function PrintLocalText takes nothing returns nothing
local string TEXT = "Hello world local value"
call BJDebugMsg(TEXT)
endfunction
function PrintGlobalText takes nothing returns nothing
call BJDebugMsg(TEXT)
endfunction
function main takes nothing returns nothing
call PrintGlobalText()
call PrintLocalText()
endfunction
In order to execute the above, first the ANTLR-generated parser runs on the code file. This offers callbacks from the parser generated by the ANTLR library into Warsmash stuff that responds in ways I wrote, which in turn creates its own Warsmash-specific designs that I created based on the callbacks it receives from the parser. The Warsmash handling for this includes a class called GlobalScope that tracks the value of global variables and a simple map of function name to a "function". The interface definition of a Warsmash function was quite simple:
Using the Java interface system, I can write code in terms of the idea of a callable function who must be given his list of evaluated arguments, a reference to the "global scope" under which he executes, and a third thing called "TriggerExecutionScope." That is what this snippet of Java defines. But it allows me to make the simple but powerful declaration within "global scope" that we have a map of string "function name" mapped to instances of this "jass function interface." And thus, later on, we will be able to define what is the functionality of call PrintGlobalText() for example, based on the idea of:
A lookup to "PrintGlobalText" as a Java string we stored while parsing
A simple step to .call(args, myGlobalScope, myTriggerScope) the interface object returned to us in the lookup.
In hindsight, the class I created for Warsmash that was called "TriggerExecutionScope" might as well have been called "Event Response" because it was a state holder for all the Event Response - Triggering Unit type of variables in the trigger editor. Of course, when executing the simple file above, this TriggerExecutionScope was fed with some nil or empty value since we have no triggers firing.
This simple map of name to "function" stored in the global scope had only two types of functions that it could point to -- native functions, which essentially were written in Warsmash game emulator to do some game behavior that cannot be expressed in JASS, and also "user functions" which are the ones defined in code above. The "user functions," quite literally, were a simple Java class containing only List<JassStatement> statements as their class members. As a result, the implementation for the UserJassFunction.call(...) function in Warsmash was kept decently simple:
Java:
@Override
public JassValue innerCall(final List<JassValue> arguments, final GlobalScope globalScope,
final TriggerExecutionScope triggerScope, final LocalScope localScope) {
for (final JassStatement statement : this.statements) {
final JassValue returnValue = statement.execute(globalScope, localScope, triggerScope);
if (returnValue != null) {
if (!this.returnType.isAssignableFrom(returnValue.visit(JassTypeGettingValueVisitor.getInstance()))) {
if (...)
... // The "..." indicates portions of Warsmash omitted in this post, for brevity. The full source code is available here: https://github.com/Retera/WarsmashModEngine/blob/main/jassparser/src/com/etheller/interpreter/ast/function/UserJassFunction.java
}
else {
throw new JassException(globalScope, "Invalid return type", null);
}
}
return returnValue;
}
}
if (JassType.NOTHING != this.returnType) {
throw new JassException(globalScope, "Invalid return type", null);
}
return null;
}
If we wanted to express this as pseudocode for readers less familiar with Java, this is quite simply:
Code:
for each statement in this function:
execute the statement
if the statement returned with a "return value" command, we return the same (interrupting the loop above)
if loop finished but function's JASS code says "returns bla" where "bla" is not "nothing":
crash with exception "Invalid return type"
else:
return nothing
What this means, effectively, is that the snippet:
JASS:
function PrintLocalText takes nothing returns nothing
local string TEXT = "Hello world local value"
call BJDebugMsg(TEXT)
endfunction
...now can be executed by Warsmash using the above. The List<JassStatement> inside the function named "PrintLocalText" will be constructed by the parser as containing two items:
An instance of new JassLocalDefinitionStatement("TEXT", JassType.STRING, new LiteralJassExpression("Hello world local value"))
An instance of new JassCallStatement("BJDebugMsg", new ReferenceJassExpression("TEXT"))
And it was thus, if you follow me this far, that the statement loop inside of the UserJassFunction named "PrintLocalText" essentially ends up looping 1 and 2 above, first performing the respective behavior encoded in the JassLocalDefinitionStatement.java file and then afterwards performing the behavior encoded in JassCallStatement.java file.
The entire 2021 interpreter for the JASS language that exists inside Warsmash was built in this way. Everything that I described until this point are the details of the 2021-2023 system prior to the changes I am writing about today. If you followed me this far, you might be able to understand the primary pitfall of this structure: the execution stack performing the behavior encoded in the JASS program is literally the Java execution stack. This is good for performance; I would not hardly be surprised if that first iteration were capable of running much faster than Warcraft III's JASS, although I never benchmarked it. In many ways, that might run nearly at the performance of Java itself, although it was still clearly an interpreter and not a compiler and as such was never generating Java class files based on the JASS or anything of that sort (which would have performed much faster at runtime).
If we ask ChatGPT or some novice programmer how we can implement a way for TriggerSleepAction to exist as a native in this system, they confuse the Java thread "wait" functions with our desire to pause a Trigger in the Warcraft III game emulator without pausing any other part of the game. In English, we could describe this situation as follows:
Three units walk into three circles of power at roughly the same time: a footman, a grunt, and an archer. Each circle of power has its own Trigger in the Trigger Editor:
Footman Region Trigger
Events
Unit - A unit enters (Footman Circle of Power Region)
Conditions
Actions
Game - Wait 3.00 seconds
Item - Create a (Crown of Kings) at (Position of (Entering unit))
Grunt Region Trigger
Events
Unit - A unit enters (Grunt Circle of Power Region)
Conditions
Actions
Game - Wait 5.00 seconds
Unit - Kill (Entering unit)
Archer Region Trigger
Events
Unit - A unit enters (Archer Circle of Power Region)
Conditions
Actions
Game - Wait 7.00 seconds
Unit - Remove (Elune's Grace) from (Entering unit)
In this situation, as a user we want one consistent 3D game world where all three units enter all three regions simultaneously, and fire all three triggers simultaneously.
Then, after 3 seconds, a Crown of Kings appears. After another 2 seconds, the Grunt dies. After another 2 seconds, the Archer loses her Elune's Grace ability.
So even if we were to create a separate java.lang.Thread (the built-in Java thread class that talks to the computer operating system) and call some wait function or perhaps the static function java.lang.Thread.sleep(long milliseconds) it would then be necessary for each Trigger to have its own virtual thread. Otherwise it would surely be broken, with a wait for 3 seconds, then a wait for 5 seconds, then a wait for 7 seconds -- totalling 15 seconds waited by the player -- instead of the reality we know that we want where only 7 total seconds are used waiting. However, creating independent "Threads" in the Java sense would be insane. These actually do execute on separate processors in our desktop computers. So, if you have an 8 core desktop computer, you would have the footman's trigger on one core, the grunt's trigger on another, and the archer's trigger on a third. This is absolutely nuts. Warcraft III is a single-threaded game for a reason -- we don't want to wait for other CPU cores to have their own little "brain" of memory tracking its own copy of the game state that they have to piece back together. Maybe a multi-million dollar game like Starcraft 2 does that, but I don't.
Then, on a high level, a user might say that this problem could be solved by creating a data structure like a delta list that stores "the time until the next event," such that we encode a list containing [(3 sec, footman trigger), (2 sec, grunt trigger), (2 sec, archer trigger)] as some list concept that is aware of the difference in time until the next action must be taken. Then, perhaps we could use a sleep/wait function for each time value in order and run entirely on one thread. Such "delta list" structures are well-documented in computer science literature.
But at this point, I bring your attention to another problem: because Warsmash stores game state in variables that exist on the primary game logic thread, and I did not want to build out the complex machinery to copy this game logic off to other CPU cores and back, it is therefore the case that any "sleep" function that we invoke would fundamentally freeze the entire game, as if it were lagging, such that no units would move or attack or do anything else during this time because the JASS handling is embedded into the same game processing that handles the rest of the game so that it can touch game unit state and other things.
So, even if we have a delta list of sorts, it cannot actually "wait." Barring a substantial redesign, it was the case that it must necessarily be kept as something we could periodically "check" or "count down" on the same thread/CPU core that the game variables (i.e. all the units and stuff) were live on.
But how would this be? Imagine we stored our delta list solution described above in a List<Pair<Milliseconds,JassFunction>> as a count down until we execute a JASS function pulled from the world of JASS processing machinery described above. What would this look like? We could imagine something such as:
Java:
// The following is a brainstorm and was never a part of Warsmash [it has not been tested]
class SleepQueue
LinkedList<Pair<Milliseconds, JassFunction>> deltaList = new LinkedList<>();
void update() {
while (!deltaList.isEmpty()) {
Pair<Milliseconds, JassFunction> nextAction = deltaList.getFirst();
nextAction.milliseconds -= Game.ELAPSED_GAME_TIME_MS;
if (nextAction.milliseconds <= 0) {
nextAction.jassFunction.call(...);
deltaList.removeFirst();
} else {
break;
}
}
}
}
And this idea, although reasonably sound from a bird's eye view, is the reason that I waited from 2021 until 2023 to implement TriggerSleepAction: because of the consequences if we actually stop to consider it. Indeed, I oversimplified something; notice how I assumed that the concept of a callable JassFunction would be what we put into this sleep queue. That is incorrect! A function that must sleep must be able to sleep in the middle of running. What if our simple example described above is modified ever so slightly so that the list of Trigger Actions includes behavior both before and after the sleep?
Grunt Region Trigger
Events
Unit - A unit enters (Grunt Circle of Power Region)
Conditions
Actions
Trigger - Disable (This trigger)
Game - Wait 5.00 seconds
Unit - Kill (Entering unit)
This snippet will generate a JASS function similar to the following:
And that... that pokes a hole in the fundamental concept of the entire system described until this point. The basic code for how to execute a function is wrong. It is not as simple, Java-side, as "for every statement in this function, call that function." At each step along the way, the function we call might tell us to stop what we are doing but remember where we are. What, was I going to have functions that stop in the middle generate a callback on-the-fly for how to pickup where they left off? What would the code for that actually look like? It is certainly not the simple UserJassFunction solution pasted earlier in this message.
It turns out, the problem is even more complex than this. Above I was only focused on the logic for how to proceed forward through the statements of a JassFunction. If we trivially spent time making each step check and ask, "Was I put to sleep?" we will simply break on another similar case down the line. For example, maybe my JASS code uses call DoThing(5, "value", WaitAndGetOtherValue()) where WaitAndGetOtherValue is itself a function that needs to sleep for a time. In such case, it becomes necessary for other behaviors such as the simple evaluation of arguments that must be passed to a function call to likewise constantly check themselves for possible interrupts that must freeze them for a time, but not lose track of what they were doing.
And the inability to solve this problem all stems from Java not having a way for me to yield in the middle of the execution of Java code, convert the yielded thing into some program variable, store it somewhere, and then fire off the continuation of that thing in the future.
In a world where I could play a modified build of WarChasers and run around my hero and fire off special events and open locked doors using the Moon Key(s), and even run Timer objects created in JASS with the TimerStart function, it didn't make sense to users why this problem would be difficult. Why can we have literally working Timer variables but not have triggers that freeze mid-execution? Without considering all that I described above, one Warsmash contributor @thatkyle on GitHub -- quite reasonably -- attempted to modify UserJassFunction described above with the intention to change this class to check at each step of the loop if the given JassStatement of the function slept.
This is a quite reasonable attempt based in a fairly solid understanding of what Warsmash is doing! When we think about the Trig_Grunt_Region_Trigger_Actions brainstorm above, it makes sense as a human to say, "When I consider each next step of the JASS function, if that function did not return normally but instead put us into a hung state, then let us save what we were doing."
The problem I suppose, and where my code in Warsmash may have confused this contributor, is that UserJassFunction is not a state holder of the execution of the function but rather of the definition of the function. As such, if three grunts walk into the region and fire this "Grunt Region Trigger" three times quickly -- and each instance of the trigger firing sleeps -- then there is still only one variable of class UserJassFunction for the Trig_Grunt_Region_Trigger_Actions in this design. Not three variables of this class. As such, it turns out that having int currentStatementIndex = 0; as a mutable member variable of the function -- as this contributor tried to do -- will end up stepping on its own toes from the different instances of the same trigger firing. They have only one state holder to remember their current statement index, but in reality the three times the trigger fired each need their own variable state. And, in that state, they each need their own current statement index.
It was in that context, and after seeing this situation, that during my 2023-2024 winter holiday I took it upon myself to try to solve the issue with a different perspective: by being OK with burning what came before to the ground. Namely, when a contributor tries to "fix" a problem like this, sometimes even for the brightest of programmers when in an unfamiliar codebase it may be hard to tell when something is "faulty by design" such as my 2021 JASS execution layer. In reality, there probably wasn't a wholly complete way to solve the simple JASS interpreter and solve this problem that wasn't stupid.
And this is why I set about during the break to use the same JASS parser, but write an almost entirely new execution layer for the pipeline that actually executes the code actions. I based what I created on the manner in which my computer science professors at my university described how actual threads work in low level computer operating systems, minus the fact that I use Java classes rather than memory addresses and therefore I have potentially worse performance because of variable sized contents in certain intentionally simple systems. Although I do not know for certain, this same jump in design could quite possibly be the reason that the mem-hackers say JASS uses some "JASS byte code" system instead of just literally using an interpreter or whatever.
I started from the assumption that in the new system, every action that the system might take should have a JASS call stack updating live that is independent of the Java execution stack used to run the interpreter. These simpler actions would be much simpler; JassStatement that represented a line of text from the parser like local string TEXT = "Hello world local value" should be discarded as a representation. Instead, I would have a new idea that needed a new name, so I called this the JassInstruction. This would represent one step forward in a system that was unaware of any state from Java other than what "JassThread" context it was supposed to run inside of:
Java:
public interface JassInstruction {
void run(JassThread thread);
}
In particular, this concept's run function also does not return a JassValue instance because it is decoupled from running an actual function in the Jass environment. If something returns something to something else, that would remain totally within the new JASS ecosystem whose state was not stored in Java stack variables. This abstract interface that we can call on would not perform the meaning of a section of JASS code and give us the result -- instead, it would be the much simpler abstract content of telling JASS to "take a step forward."
As, as such, taking the next step forward after this one might happen, or it might not. We might throw the JassThread in a SleepQueue, and we might not.
The concept of JassThread, then, was also a second new concept introduced in this rewrite. It is, as far as I can figure, the same concept as what @thatkyle was trying to do in his branch -- a currentStatementIndex of sorts -- but the key intuition is that it would be variable state with a different scope. A different lifetime for how long this memory would exist -- it is the quintessentially missing idea of "one firing of a trigger" in the 3 grunts example. As such, the idea that UserJassFunction was meant to represent -- a constant, unchanging, immutable variable in memory that represents how to do a function would still exist, and it would still be immutable -- but the new addition that @thatkyle sought to add for tracking our position in the execution of a function would exist elsewhere in these temporary JassThread objects.
Java:
public class JassThread {
public JassStackFrame stackFrame;
public GlobalScope globalScope;
public TriggerExecutionScope triggerScope;
public int instructionPtr;
public boolean sleeping = false;
public JassThread parent;
And, therefore, if "Grunt Region Trigger" the concept fires three times, each of those times would track their own currentStatementIndex of sorts using the variable that I happened to have named instructionPtr, and meanwhile "Grunt Region Trigger" the concept would still exist in one singular state holder that wouldn't change.
So, all of this was easy to say, but it is another matter to re-implement all of the interpreter's behaviors for the various components of JASS syntax as immutable classes that implement JassInstruction interface. What would the code look like to execute the equivalent of a UserJassFunction in this new world?
To keep the model of this concept fairly simple and consistent with the world as I learned it to be in my computer science courses, I decided to represent the entire JASS code parsed from the JASS files as one gigantic array of JassInstruction instances in this new system. To understand what that even means, let us return to the previous example with the function called PrintLocalText:
JASS:
function PrintLocalText takes nothing returns nothing
local string TEXT = "Hello world local value"
call BJDebugMsg(TEXT)
endfunction
Previously, upon encountering this text in a file, the 2021 Warsmash parser would spit out something holding those JassStatement classes:
An instance of new JassLocalDefinitionStatement("TEXT", JassType.STRING, new LiteralJassExpression("Hello world local value"))
An instance of new JassCallStatement("BJDebugMsg", new ReferenceJassExpression("TEXT"))
But now, in the world that treated everything as one long list of "instructions," this same text would instead be parsed out as instances of the following Java variables:
new BeginFunctionInstruction(lineNo, sourceFile, name)
new PushLiteralInstruction("TEXT")
new LocalReferenceInstruction(0)
new NewStackFrameInstruction(6 /*the pop instruction below*/, 1)
new NativeInstruction(nativeIdForBJDebugMsg, 1)
PopInstruction.INSTANCE // throw out the return value, because this was a "call" directive
new PushLiteralInstruction(JassType.NOTHING.getNullValue()) // for now every function has this appended at the end, to return null in case they didn't return anything else from user code
new ReturnInstruction()
All functions were converted into a bunch of instructions like this, but unlike the "statements" held within a user function in the old system these instructions were all stored in one place in GlobalScope. Therefore, when we desired to call a function of any kind, instead of a simple map from "function name" to a "user jass function" class instance, instead this new system mapped the function name string to an integer. The integer, in turn, was the index in the "instructions" array where the flattened contents of this function had been added. In this way, for example, the call and loop syntax in any original JASS file were both implemented using the same class of "instruction" -- a BranchInstruction class -- that simply updated the JassThread to set its instructionPtr where it was executing to a new, different value.
After going to the effort to invest this new kind of scaffolding to express the original JASS file as instructions, the last step was to simply update the game loop that happens roughly 20 times per second on Warsmash to call this.globalScope.runThreads(); inside the Warsmash game simulation class.
Java:
/**
* @return true if all threads have terminated
*/
public boolean runThreads() {
boolean anyThreadsAdded = false;
do {
runOneThreadLooop();
anyThreadsAdded = !this.newThreads.isEmpty();
this.threads.addAll(this.newThreads);
this.newThreads.clear();
}
while (anyThreadsAdded);
return this.threads.isEmpty();
}
private void runOneThreadLooop() {
for (int threadIndex = this.threads.size() - 1; threadIndex >= 0; threadIndex--) {
final JassThread thread = this.threads.get(threadIndex);
this.currentThread = thread;
try {
while (!thread.isSleeping()) {
if (thread.instructionPtr == -1) {
this.threads.remove(threadIndex);
this.yieldedCurrentThread = false;
break;
}
else {
this.instructions.get(thread.instructionPtr++).run(thread);
}
}
}
catch (final Exception exc) {
throw new JassException(this, "runThreads() encountered exception", exc);
}
if (this.yieldedCurrentThread) {
this.currentThread.setSleeping(false);
this.newThreads.add(this.threads.remove(threadIndex));
}
}
this.currentThread = null;
}
Although the plumbing for this includes some other boilerplate concepts, the basic idea is encapsulated within runOneThreadLoop() which we could oversimplify with the following minimal conceptual logic:
In other words, to actually step forward, we execute instructions.get(thread.instructionPtr++).run(thread) until the current thread is tagged as sleeping by the system, or is terminated with no more work to do because its top-level main function it was told to execute has returned.
After some testing, this reinvention and the inclusion of this loop to constantly process many, pausable JassThread objects meant that suddenly I could emulate Warcraft III maps on Warsmash that include the TriggerSleepAction call on their map triggers. Suddenly, within the Warsmash emulator, events could fire, then wait, then perform other actions!
So, among other tiny improvements, I believe that this holiday refactor was probably the most notable substantial refactor in Warsmash worthy of posting an update about. Also, to be fair, it's not perfectly done yet. At the moment of writing, this system was created and shown to work in a few test cases but has not been merged to the main branch of Warsmash yet. However, after seeing how it caused map features that were previously dead to be brought to life, I am certain this reinvention of the system will end up in the main branch of Warsmash eventually. Perhaps that will be after some additional branch cleaning -- the first draft of this stuff existed as a second system without deleting the previous system on Warsmash, and there were some cases such as BoolExprs that could evaluate without falling into this system that were instead calling into the old system that instantly returns in Java -- not because they have to, but because I had not refactored them. But it is all solvable, and should work out in the end.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.