Warsmash

[Retera]

Is there any way to download/test your version.

Sure. Reforged is 30 GB, but Warsmash is 1 MB, so that is 30000x smaller, which is entertaining. Obviously this is because it does not include the assets.
I have attached a copy of the 1 MB repo to this post. If you're more interested, I already added a couple of people to the private repo on bitbucket but nobody actually made contributions yet to my knowledge.

I want to make it public and put the repo on GitHub in the future, but I was waiting to do that until it's more of something that players could actually enjoy for something other than making trollish YouTube videos. Namely, I would like to see it be multiplayer capable first, among other things (items, heroes, etc).

There are still huge, drastic issues with this build that make it almost unplayable for certain users.

1. The attached zip file contains code from the git branch titled "blasphemy" which is supposed to be capable of loading from a Reforged Installation. I tried that one time, and it was able to launch. But, on Reforged:
   • The FLAC audio hack is mostly untested because the new dialogue SLKs will prevent audio from working, so you'll be missing most sounds if you use a Reforged install
   • Obviously only SD graphics are supported, I've never claimed this thing to be capable of parsing HD, although the "blasphemy" branch here is using a version of the MDLX parser from the newest builds of Retera Model Studio that other authors besides myself (thanks to @GhostWolf and a couple patches from @twilac and BogdanW3 ) updated to be HD compatible -- although the engine would have no idea how to render the HD models, so don't try that, because I never ported those portions of Ghostwolf's viewer into the rendering pipeline. The reason for including this code from newer RMS builds is to get the SD models from the Reforged Installation to parse, since they are nonsensically in the new MDX1000 format on Reforged
   • The unit shadows are missing when using the Reforged install -- this is probably due to some issue with how UnitSkins.txt is being loaded but I'm not interested in pursuing that for the moment
2. So, my personal preference even with this new branch is to use Patch 1.22 (NOT 1.32, not a typo) as the source of game data. There's no reason to use a patch newer than that for testing; it only makes things more complicated. You can use 1.30 or 1.31 if you would like. They would probably work because I have some basic support for some of those file location names and lookups and the "blasphemy" branch has stuff copied from Retera Model Studio to load CASC (previous idealistic non blasphemy versions were MPQ only). So, in summary, if you configure it the correct way this engine will probably be capable of launching given any Warcraft 3 install from 1.22 to 1.32 that you feed it, with the above noted limitations. Also, for 1.29+ you have to go to WarsmashConstants.java and change MAX_PLAYERS=16 and replace it with MAX_PLAYERS=28.
3. The only records that I have of other users attempting to launch version(s) of this prototype in the past encountered technical issues in order to launch it. @Antkibo who was interested in contributing was originally not able to launch this because his hardware could only use OpenGL 3.X or so, and I use code copied from HiveWE that requires OpenGL 4.X or so. Essentially, I was requiring computers with GPUs from 2017+ or so without really thinking about it, or something like that. I changed around the code in the hopes of getting OpenGL 3.X and did some shader backporting from the HiveWE-based stuff and I got really, really close. I was able to launch the engine on my crappy 2013 lenovo laptop that gets 5 fps on Patch 1.31 because it only has integrated graphics, and I saw ~40 fps on this engine. That is, of course, because of Blizzard's choice to make 1.31 default to DirectX 11 or 12 or whatever. 1.31 running with the command line args to use the old DirectX9 code on the official Blizzard War3 game sees 55 fps or something, much better than my code, on 1.31. BUT I was only able to launch then engine on this OpenGL 3.X environment on the 2013 crap laptop by using local code modifications to remove features that I did not commit (again these OpenGL version numbers are not totally accurate, I'm writing this from memory, this might still have required OpenGL 3.3). So, most likely Antkibo never got the engine running but it's been a while since I checked in with him about it. The hardware problem stems from the number of textures available on the ground tiles. WC3 only allows 16, and likewise these GPUs are only allowing 16 shader texture units. The HiveWE code that I copied wants to assume we have some new hardware magic without the 16 limit and uses a 16th GPU shader texture for blight I think, and then a 17th shader texture for map .shd shadow file data rendering baked into the ground colors or something like that. So, on my laptop, what I did was that I removed the .shd shadow file rendering baked into the ground so that doodads and buildings had no shadows (except on cliffs which were not affected and used less texture units).
4. The only other occurrence that I know of someone trying to get this code to run on their own hardware was someone who grabbed a copy of the code that I posted in a zip file on a discord server in a somewhat trollish way. I posted the code trying to point out to people that they could aspire to do something like this project. Probably 2-3 months later this guy found the link again and downloaded it and actually tried to run it. I do not recall if the bugs he was seeing were things that I fixed since then -- they might just have been hardware differences between his computer and mine. But suffice it to say that the unit portrait and the UI and stuff did not really line up at all the way it does on my computer.

Spoiler: The only screenshot of someone else running this codebase that I have ever seen

Notes:

1. The font that was loaded on his computer is something that I have not seen, and there is no text other than "No Upkeep" which was hardcoded in the sourcecode at the time for the sake of example. So I believe he was launching this with a different language locale of WC3 that I have never tested with that used special characters, and possibly that language locale is not supported yet on Warsmash at all (since I have never tested it). This could be Chinese characters, Cyrillic, etc, I have no idea.
2. The portrait is not behind the portrait UI FDF component and looks gross and offset. This is probably an ongoing issue on Warsmash because I have not seen this on my own machine, so I never fixed it. The portrait X/Y logic is decoupled from the FDF rendering logic in a hastily written way because I don't yet support the FDF frame type "MODEL".
3. The move/hold position/attack/patrol icons are all highlighted at the same time. That was an engine-level issue (my own fault quite directly with the highlighting decision logic) that existed for a week or two and should be fixed in the version included in this post
4. The cliffs are missing in the upper left hand corner of the screen. I have no knowledge of why, so it is possible that they would also be missing for other users, since they were not ever missing on my PC and so I do not anticipate ever fixing a bug related to missing cliffs.

(Thanks to the user 271828 on Discord who posted this screenshot in a public place)

So, this is all super off-topic and if you wanted to pursue discussion on this it should probably be done somewhere else, but I thought I should mention some of that if I was going to post the code, since the probability is like 90% that anybody trying to run it without a guide is going to run into issues.

Spoiler: How to launch the repo

How to launch the repo

1. Download IntelliJ community editor
2. Open the project as a gradle project, not just generically as java
3. Find "core/assets/warsmash.ini" and reconfigure this INI file with the location of your Warcraft 3 install (the one in the repo is literally a copy of my own using absolute paths and won't work on any other machine by default, so consider it only as a template)
4. Run the "DesktopLauncher" class's main method

 

[Ralle]

Retera,

I moved your post to a new thread. I did it for two reasons:

1. Showing off Warsmash really is off topic when we're talking about how the dev team was dismantled.
2. Warsmash is good news, not bad and deserves to have its own thread.

Awesome that you shared the source code. Very cool

 

[Dark_Spyro_003]

Ooooh, shiny! I have coincidentally recently been looking on the Hive again (what, with Blizzconline and all and some WC3 nostalgia) when I saw some of your posts in other threads. I was somewhat interested in seeing Warsmash evolve from watching some of your YouTube videos, but it's really cool of you to share it.
I'm running on Linux currently and it seems to be launching fine - and here's a screenie since you apparently don't have many of them, on the Andorhal map.

Looking good and works fine! I wish you all the luck, maybe you'll build us a better Warcraft III
To be honest, it took me more effort to log into my account again than it took to get this running, so good job! Thank you for all of your hard work.

 

[Ralle]

Wow!

So Retera, how about networking?

 

[Dark_Spyro_003]

Since I've already gotten a question, and anticipate more may come up and since I would be sad if people gave up because of such minor hurdles:

• If it can't find warsmash.ini: You're launching through IntelliJ IDEA as instructed, however the project expects the Working Directory to be set to the same directory (./core/assets) that warsmash.ini is in - e.g. warsmash_02_19_2021/core/assets
• All of the DataSources configured need to exist. If you want to remove one (e.g. the line pointing to Retera's backup), don't forget to change the numbers of the following paths, as well as the DataSources count at the top.
• At the bottom of the file, under [Map], the Map that does not have // in front is the one that is configured to be run on start-up. You'll get a Nullpointer if this file does not exist - which it likely will as it seems to be configured to some custom map in the zip file.
• Warsmash will only load data (that is, both 3D models, maps, the UI, ...) only from configured folders: ensure the WC3 MPQ files are configured, and ideally add the Maps directory as well. Do not delete the paths for "." or for "../../resources", I believe these respectively contain essential resources and the UI for Warsmash.
• If you want to run with Gradle, this project is only set-up to run with JDK 8, so if you're not running from IntelliJ, use an appropriate Java build to launch Gradle.
• Please do not expect this to be identical to a feature-complete Warcraft III - it is a huge undertaking to make an entire videogame after all ! For those with no coding experience, please believe me this is already more than I would realistically expect from a single or small group of developers for such an effort, so it is great work so far!

All in all, this project is still in a development phase, so minor configuration hurdles are to be expected to be launching this. Best of luck to everyone.
Respect and my thanks go out to Retera and the other contributors! They're doing a great job on a promising project.

 

[Retera]

Do not delete the paths for "." or for "../../resources", I believe these respectively contain essential resources and the UI for Warsmash.

Yes. The "." data source is equivalent to Allow Local Files setting for Warcraft 3, and "../../resources" indeed contains certain extra files that are needed for Warsmash that are not needed for Warcraft 3 currently. This mostly consists of FDF files for UI components that are hardcoded on Warcraft 3 as far as I can tell. I'm hopeful that I can allow users to overwrite FDF game UI with imports in custom maps, and if I get that working having more external UI files hopefully makes it easier for developers to change the Warsmash UI without changing the code.

Respect and my thanks go out to Retera and the other contributors! They're doing a great job on a promising project.

Thanks! Rather than other direct contributors, what I have right now is mostly all assembled together into a project by me but where some portions of the code are copy pasted or ported from other projects. It is based on the philosophy that because GhostWolf's Garbage Bin has a WebGL based map viewer as well as the Hive's "View in 3D" preview for single models, and I was able to copy portions of that WebGL code to run in a desktop app in Retera Model Studio, I decided it would be fun to port that viewer to a desktop Java game. This meant I needed to "rewrite" it in Java instead of JavaScript, but if you look at the code in flowtsohg/mdx-m3-viewer written by Ghostwolf, a large portion of my render pipeline is literally structurally a 1:1 replica other than being a different programming language.

Also, I currently parse BLP, MPQ, and CASC using open source Java code from DrSuperGood that is dumped into the project in several places, even though ideally these should probably be included by reference at build time from other online repos instead of included as source. But for getting a quick product going including the source means I can fix things, such as how the JCASC code in this project is backported to Java 8 from the official Java 11 version of JCASC. I also parse TGA in a likewise manner with a library from Oger-Lord I believe it was, and FLAC using code from a project that includes a citation in the copied sourcecode but from an author who published it for general purpose Java and who isn't from Hive and whose name is escaping me at the moment.
So, when I transition the repo back to GitHub to make the code more easily publicly available I plan to include a list of these citations in the readme, but for the most part I have interacted directly with stuff like MDX and W3X and FDF but not with the data storage and image storage formats that mostly rely on existing libraries.

Speaking of that,

So Retera, how about networking?

I'd been mostly of a mind to wait to get testers until the game is multiplayer so that the testers can really enjoy it and feel that it is worth their time, but going more public with the existing prototype is beginning to sound like something that would entertain more people than I might initially expect.

 

[Ralle]

I'd been mostly of a mind to wait to get testers until the game is multiplayer so that the testers can really enjoy it and feel that it is worth their time, but going more public with the existing prototype is beginning to sound like something that would entertain more people than I might initially expect.

lock-step needs to get quite deeply integrated. All player orders are basically funneled into a socket and comes back from the server when its time. All trigger events are scheduled by the server, I think. And you can create a dummy server object that just passes the orders back immediately but it's good to go with the right structure in that regard from the start.

 

[Dark_Spyro_003]

lock-step needs to get quite deeply integrated. All player orders are basically funneled into a socket and comes back from the server when its time. All trigger events are scheduled by the server, I think. And you can create a dummy server object that just passes the orders back immediately but it's good to go with the right structure in that regard from the start.

I'm not familiar with lock-step but I do agree that the best method of development for a game that is anticipated to have multi-player is to have Single-player essentially be Multi-player running against a "local/internal" server, so that it is semantically identical - this means your pipelines for singleplayer and multiplayer are identical and you do not get semantic/functional differences between single and multi-player.

...

Thanks for the clarifications!
Also, I'm just being curious, but why did you opt to back-port dependencies to Java 8 instead of upgrading the project to a newer LTS Java, e.g. 11?

 

[Retera]

Thanks for the clarifications!
Also, I'm just being curious, but why did you opt to back-port dependencies to Java 8 instead of upgrading the project to a newer LTS Java, e.g. 11?

I was too lazy to go find a JRE 11 and was wrongly convinced for a while that there was no JRE 11, only JDK, and so would be problems with release/deployment. Turns out OpenJDK has JREs for newer Java versions though, as someone showed me.

Edit: Also, if I try to only use JRE 6 compatible features, there was a convenient way to convert Java 1.6 LibGDX games to run on web a few years ago but at the time newer versions didn't work. I just don't feel much call to go cause change to my workflow or dependencies or anything, that can be done when I have something that works, my focus is to make it work first.

 

[Elprede]

From the looks of things, your version of the world editor is not included in this share. Could you perhaps also upload that here as well? I have a fair amount of more interest in that project than in the game itself.

 

[Ralle]

I don't think Retera is remaking the world editor anytime soon. You should try out Hive WE.

 

[Lizreu]

Slightly off-topic, but deterministic lockstep is notoriously difficult to get right, especially with regard to floating point calculations. You basically have to coax the OS, the CPU and everything else to use standard IEEE-754 floating point calculations, because modern compilers and CPUs will try their hardest to cut corners and make those sweet FLOPS bigger. That comes with the caveat that between compilers, compiler versions and CPU architectures, floating point calculations aren't entirely deterministic by default. It's still possible, but you have to reconfigure a bunch of options during compiletime and runtime to prevent CPU-specific instructions from being used, from reordering floating point instructions, and setting any special modes that accelerate floating ops.

On top of all that you definitely must design the engine with this kind of architecture in mind from the grounds up, otherwise its going to be very hard to tackle it on later. Entity IDs have to be stable, events have to be processed in strictly the same order on every client, local state must not creep into shared state, etc. There's a good reason why desyncs became so prevalent in some of the patches - deterministic lockstep hard.

And if you ever want to implement reconnection/desync recovery, that is a whole other can of worms that you'll need to lay down foundations for from the very beginning if you want to actually make it work.

 

[Ralle]

Do you think that floating point operations will be easier in Java because Java tries hard to be the same on all platforms? That's probably something we can research in advance.

 

[Retera]

From the looks of things, your version of the world editor is not included in this share. Could you perhaps also upload that here as well? I have a fair amount of more interest in that project than in the game itself.

I don't think Retera is remaking the world editor anytime soon. You should try out Hive WE.

Yeah, Warsmash would be great with an editor later but I'm only doing one thing at a time. It's pretty likely that I'll just use HiveWE.

Do you think that floating point operations will be easier in Java because Java tries hard to be the same on all platforms? That's probably something we can research in advance.

I was worrying about this the other day. I didn't get to the bottom of it yet but I'm hopeful. I made a Java strategy game that was turn based but which used a hacked together turn based sort of lockstep, and I've played it in multiplayer many times and it was fine. So I'm hopeful that Java solves the problem by being consistent across platforms which would be consistent with my empirical observations on that other game -- desyncs basically always came from solvable known game design issues in the application logic.
I used StrictMath instead of Math for some of the unit movement calculations. I'm hopeful that should contribute towards solving the problem as well.

 

[Lizreu]

Do you think that floating point operations will be easier in Java because Java tries hard to be the same on all platforms? That's probably something we can research in advance.

Probably not by default, because it comes at a big performance penalty. Might also be different from JVM to JVM. There's probably knobs to tweak, though. I'd be more worried about Java just dragging the performance down in general.

 

[Retera]

Yesterday I made a first draft at having inventory items, although no ability behaviors yet. Some days this project still leaves me feeling as though I were living in a dreamworld, but when I go to code and try to add something like items based on my experiences of what I know should be there, it just appears in front of me. And it's that same moddability that leaves me feeling like I know chasing this vision will leave me with a game to play that I know I will enjoy modding.

I'm just really doubtful that performance will be much of a concern. Not because it won't be, but because of how easily I can decide not to care. For example in the tiny map on that video, I get 1200 fps when my cinematic illidan was offscreen but only 400 fps when he is onscreen. I'm not sure why, really. But then when I realize that I don't care, it is nice.

 

[Ralle]

I payed so close attention and yet the only detail I sense is not 1-1 is the foot sprint placement. But I am not even sure that I'm right. It looks so real Retera, good job.

 

[Retera]

• Item does not show name/select circle on mouse hover
• Game engine does not say "Select Target" while item is in cursor hand
• Potion does not show charges

and many more xD

Edit: I made a way to swap items between different slots!

 

[Retera]

Retera, can you bless us with a new code snapshot in the near future?

Hey bers! Hopefully so!

Today I did my first test of running the engine with multiple maps loaded in sequence in the same computer process. I know this video shows a lot of buggy junk, but it's also kind of interesting that I managed to record my first test ever loading multiple maps in sequence in the same process, which was fun.

There's certainly a lot for me to do to polish this...

I was hoping maybe this could move in a direction of being able to support custom campaigns maybe, since it will probably be a while before I stop being lazy and code the multiplayer networking layer.

(The game menus do not have a way to launch anything other than campaign missions yet, although I started the program with -loadfile and a custom map)

Edit: Yes, the video ends due to a crash on "Enemies at the Gate". It says a unit was owned by player index 65537, which is ridiculous and should be impossible, so it's probably some binary misunderstanding in one of the map data parsers that I ported from the model viewer or from HiveWE or something. I'm hoping to move to a model where units are spawned by war3map.j and so at that point the problem might just fix itself anyway.

 

[Barade]

Hi, I did not know about this thread although it is started with my quote
I think it should be sticky but now I see that you uplodated Warsmash Mod Engine (Alpha) which is the best way to spread your project! And of course your GitHub repository: Retera/WarsmashModEngine
Great work! I am currently at work but at home I will try the project. From the rreview I can read that it doesnt automatically detect Reforged files. This would be really helpful. Access and easily starting it are really important features to make people use it.

 

[Retera]

Hi Barade!

Yeah I am not very concerned with accessibility yet. I consider it to be a small issue in comparison with functioning features because accessibility can be easily tacked on at the tail end of development. It also ends up being a lot of unfulfilling grunt work for whoever does it, since we have to load the new Dialogue SLKs and skin data with a totally different format for Reforged to even just play classic off of a Reforged install. This weekend I did a code and chill session with a friend who uses 1.32 (exclusively, I think?) and we started off by both cloning Warsmash from source and renaming the warsmashRF.ini to be our warsmash.ini that actually gets used so that our configurations loaded assets from Reforged. It was a fairly short process and got things working, but the load time for the game is higher than when it loads from MPQs, and I did not yet bother to investigate the reason why. Anything from the DDS parser to FLAC parser that I am using might happen to just be slower than their equivalent counterparts for legacy formats; I do not know.

More discussion on these topics here:

Warsmash Mod Engine

Warsmash is a Warcraft III modding project that aspires to eventually make Warcraft III modding easier by providing a replacement component for the game program that can be more easily modified -- while still requiring the Blizzard Warcraft III game and assets to be installed on the computer...

www.hiveworkshop.com

 

[erfaned]

• Item does not show name/select circle on mouse hover
• Game engine does not say "Select Target" while item is in cursor hand
• Potion does not show charges

and many more xD

Edit: I made a way to swap items between different slots!

I like this remake. For how long have you been working on it?

 

[Retera]

According to my records, I first drafted a design document about what I wanted to do in December of 2018. So, I actually started writing code and taking it seriously at some point in 2019. I have a job so it's just a weekend hobby, really. Keep in mind that substantial portions of the code is based on modding libraries and systems available from here on the Hive workshop.

Now that Warsmash is a hosted project, you can read about the history of all that happening in the Warsmash Forum. In particular, "Updates" and "Teaser Videos" sections contain some history.

 

[Retera]

At the top of this thread I mentioned that I wanted to wait until later to try to get players to try Warsmash, because it was not multiplayer capable yet.

This weekend I got the first draft of a GUI based multiplayer feature working, which (once I build a binary release to put here on Hive and also redeploy the warsmash.net server) would mean that players who are less technical could try Warsmash in multiplayer simply by downloading a binary distribution, configuring the Warsmash to locate their Warcraft 3 install, and then running the Warsmash binary and clicking buttons to "host" or "join" a game. Presently the "host" is actually a designation but not a host in the classical technology sense, so the Port Forwarding problem from the old days of Warcraft 3 is not an issue for the common users. The only people who would have that problem would be anyone who wishes to spin up their own full instance of the entire warsmash.net "gaming network" server that allows chat lobbies and custom game hosting. There is no versus supported yet, only basic custom games.

The first iteration of the server has a known resource leak and after a certain number of games are played, then the server will need to be restarted. The problem is much like the concept of Warcraft 3 hostbots -- in essence that the code is spinning up a new hostbot equivalent when each new game is hosted, but is not smart about closing down these instances after all connected clients have exited the game.

But I thought that if I could accept that level of stupidity in the first draft just to get something working, it may be worth it so that interested but non technical users might be able to have some fun with this project.

There are still a huge ton of bugs, though. There isn't a desync detector so after a while, the parallel simulations may simply break apart with no warning and the two players might end up having different experiences.

Edit: I do not live in the European Union and I am not well familiar with the requirements of the GDPR, however I read somewhere that Reforged can never be reverted to play like the old Warcraft 3 game from the 2000s because the old game was a GDPR violation in some way in the European Union. My understanding of that problem is limited. I have not developed any mechanism for policing or even recording chat messages on the Warsmash "gaming network" test server, and all communications to and from these test servers are unencrypted as far as I know. Also, when you join a "custom game" on Warsmash it may transfer to your computer a map file from another user which could potentially be unregulated and posess content that I do not wish to be responsible for. If I may at some point become responsible for such content under any applicable laws, or that such liability on my part might force me to shut down the public "warsmash.net" test server instance, I would encourage users to spin up their own in-house "gaming network" server instances and/or only play with users that they trust for now while this project is in development.

 

[Retera]

I have not developed any mechanism for policing or even recording chat messages on the Warsmash "gaming network" test server, and all communications to and from these test servers are unencrypted as far as I know.

As a note, I have updated the server running on Warsmash.net so that it does now save a log containing user messages for potential review. Per my previous comment about my lack of knowledge on the GDPR, if somebody happens to chat on there and then wants to ask for their chat messages to be deleted, just let me know.

On this thread I previously mentioned the GUI based multiplayer. According to my limited information (the server was not recording much) it seems that very few people were ever using the server. This might have been in part due to a critical bug deep in the infrastructure for UDP communication at the layer below the lock step. For a long time it was a problem that despite my efforts to the contrary, games would randomly hang while playing and the players would lose their progress.

Last night I got tired of that bug that has existed for multiple years and played a game and caused the hang while Wireshark was running and then I sat and did packet analysis which led me to the conclusion that the "Ordered UDP Communication" class responsible for in-game replay requests after dropped packets had a typo that would cause some of the recovered packets to have their contents eliminated when they would actually replay to the application logic. This might mean that multiplayer Warsmash games will be more stable in the future.

I'm pretty excited about it, but my lock-step solution still has no desync detector and there is still a long list of non-networking related game logic bugs to fix.