[Technical Sharing] A more efficient way to manage custom campaigns

[haochuan]

Hi dudes, I finally purchased the War3 reforged recently because the mod community is getting bigger. Being a player myself when I was a kid, it's disappointed to witness what Blizzard has done to the game.

After I started to install many of the old custom campaigns, I found it was complicated to manage (add/remove/update) these campaigns. Therefore I decided to improve the workflow of importing the maps.

This post assuming you understand the basic knowledge and steps involved in importing custom campaigns AND some Javascript. I targeted it more to authors and technical modders. If you don't but wish to learn, please read the following post/resources:
Try to make custom campaigns in Refrorged
Reforged - Guide: Changing campaigns
Understanding vanilla files: Casc Storages - Main page

The eventual objective of the project is to provide non-technical players an automatic yet simple-to-use campaign management tool.
Special credits to:
WARCRAFT III REFORGED GRAPHIC MOD: QUENCHING
Quenching-Warcraft3 Reforged Mod

I was using some of their work as starting point. The author also gave me some tips along the way.

Ok, enough chit-chat

------------------------------------------------------------------------------
Pain points of state-of-the-art methods:

1. When adding new custom maps, player has to extract and modify the ui/campaigninforeforged.txt
2. The player also has to modify the webui/gluemanager.js in order to display the campaign
   • gluemanager.js is a transpiled react.js app, which has been uglified and minified, making it ambiguous to read.
   • There are multiple places to add code (and it's JavaScript!!!), you can easily mess things up.
   • Huge problem when Blizzard sometimes updating the gluemanager.js
3. Additional resources such as background images & videos must be put in
   • webui/campaign/campaignselect/backgrounds
   • webui/webms
   • ...

The current progress of my method will largely improve #2. It uses a "Hook" script to minimize the modification of the glue manager:

1. Get .w3x and resources such as .webm and .png background ready
   • Alternatively install the Quenching mod: Quenching-Warcraft3 Reforged Mod
2. Download the attached file and unzip the content to <game-root>/_retail_/webui
3. Modify the "var campaigns" variable, it's self-explanatory to understand:
   • id - starts from 9, as there are 8 official campaigns
   • background - the .webm background video for mission UI
   • sound - the ambient sound of the mission start UI, as defined in ui/soundinfo/ambiencesounds.slk
   • styles
   • listItemHeaderColor - header color for campaign selection item
     
   • listItemBackground - background image URL for campaign selection item
   • missionHeaderColor - header color for individual missions in the campaign
   • banner - banner URL for victory screen
4. Enjoy

The next step I will make some simple .exe to facilitate the importing.

For those who are curious about the technical details, just reformate the gluemanager.js and search for "Qhook". I only inserted the line at 4 places.

 

[Tommi Gustafsson]

Did you manage to install old custom campaigns to Reforged so that Campaign Data and Campaign Imports actually work?

 

[Zorrot]

it's a good idea to make a jshook, and so far no attribute left behind,i may use this directly, thank you so much

 

[Vibe]

It's nice to see you working on custom campaigns and development tools. The community is becoming great again!

Looking forward to your next step.

 

[haochuan]

Did you manage to install old custom campaigns to Reforged so that Campaign Data and Campaign Imports actually work?

Unfortunately not via .w3n at the moment.
However I can extract the files and make it work by putting linked resources into _retail_/... following the same folder structure

 

[haochuan]

it's a good idea to make a jshook, and so far no attribute left behind,i may use this directly, thank you so much

Thanks.

If my understanding is correct, another hurdle is how to manage the linked resources in .w3n file. I reckon we can only play limited numbers of maps at same time for these reasons:
1. Any linked resources/assets must be extracted from .w3n and copy to _retail_/... folder
2. The game can't read nested "_retail_ like" sub-folder structures
3. Thus, if TWO custom campaign have resource file name clash (which is common for files under /replaceabletextures), the player have to reinstall the "active" campaign.

I'm trying to crack the GameClient in gluemanager.js, hopefully we can leverage on that to decode the .w3n files.

 

[Razorclaw_X]

Reforged - Custom Campaign Data is useable as Local Files

Thanks.

If my understanding is correct, another hurdle is how to manage the linked resources in .w3n file. I reckon we can only play limited numbers of maps at same time for these reasons:
1. Any linked resources/assets must be extracted from .w3n and copy to _retail_/... folder
2. The game can't read nested "_retail_ like" sub-folder structures
3. Thus, if TWO custom campaign have resource file name clash (which is common for files under /replaceabletextures), the player have to reinstall the "active" campaign.

I'm trying to crack the GameClient in gluemanager.js, hopefully we can leverage on that to decode the .w3n files.

The 'easiest' way is to just make multiple copies of the "_retail_" folder, one for each custom campaign. Truthfully only one folder in each of those "_retail_" folders are going to be the same in all of them (the one with the game client).

 

[Retera]

I'm trying to crack the GameClient in gluemanager.js, hopefully we can leverage on that to decode the .w3n files.

From my understanding, in Reforged Beta somebody had found a way to do this prior to the release of the Beta by getting the game client off of a Blizzard CDN server. This was kind of a point of public shame for Blizzard. The hackers accessed the GameClient thing you are looking at and made an escape mechanism out of the webui and it would let them play Reforged GameClient with the Patch 1.31 menu UI. So it actually felt like Warcraft 3 again and had all the old menus, but was able to play with Reforged graphics. Likewise it would then also have supported custom campaigns, I assume.
I can assure you that technology temporarily existed on this Earth. But Blizzard shut it down somehow because escaping to the pre-Reforged menu meant that players did not have to log in to Battle.net in order to play. Suddenly it was like old Warcraft 3 again and anybody could just play it by clicking Single Player > Custom Game.

There was even a set of Reforged HD model files for the HD 3D backgrounds. You can find them in Retera Model Studio if you open the Reforged Beta. One of these HD 3D menu backgrounds (for the RoC mode) is an HD grunt standing in a field. The TFT version is similar but doesnt have the grunt and is a little different.

But I am sure you can imagine, unlocking the ability to do that again would probably cause Blizzard to need to take action against you for financial reasons.

 

[haochuan]

From my understanding, in Reforged Beta somebody had found a way to do this prior to the release of the Beta by getting the game client off of a Blizzard CDN server. This was kind of a point of public shame for Blizzard. The hackers accessed the GameClient thing you are looking at and made an escape mechanism out of the webui and it would let them play Reforged GameClient with the Patch 1.31 menu UI. So it actually felt like Warcraft 3 again and had all the old menus, but was able to play with Reforged graphics. Likewise it would then also have supported custom campaigns, I assume.
I can assure you that technology temporarily existed on this Earth. But Blizzard shut it down somehow because escaping to the pre-Reforged menu meant that players did not have to log in to Battle.net in order to play. Suddenly it was like old Warcraft 3 again and anybody could just play it by clicking Single Player > Custom Game.

There was even a set of Reforged HD model files for the HD 3D backgrounds. You can find them in Retera Model Studio if you open the Reforged Beta. One of these HD 3D menu backgrounds (for the RoC mode) is an HD grunt standing in a field. The TFT version is similar but doesnt have the grunt and is a little different.

But I am sure you can imagine, unlocking the ability to do that again would probably cause Blizzard to need to take action against you for financial reasons.

Agree, I found the old 3d menu background in ui/glues/singleplayer directory but apparently they have removed the HD version

 

[Zorrot]

From my understanding, in Reforged Beta somebody had found a way to do this prior to the release of the Beta by getting the game client off of a Blizzard CDN server. This was kind of a point of public shame for Blizzard. The hackers accessed the GameClient thing you are looking at and made an escape mechanism out of the webui and it would let them play Reforged GameClient with the Patch 1.31 menu UI. So it actually felt like Warcraft 3 again and had all the old menus, but was able to play with Reforged graphics. Likewise it would then also have supported custom campaigns, I assume.
I can assure you that technology temporarily existed on this Earth. But Blizzard shut it down somehow because escaping to the pre-Reforged menu meant that players did not have to log in to Battle.net in order to play. Suddenly it was like old Warcraft 3 again and anybody could just play it by clicking Single Player > Custom Game.

There was even a set of Reforged HD model files for the HD 3D backgrounds. You can find them in Retera Model Studio if you open the Reforged Beta. One of these HD 3D menu backgrounds (for the RoC mode) is an HD grunt standing in a field. The TFT version is similar but doesnt have the grunt and is a little different.

But I am sure you can imagine, unlocking the ability to do that again would probably cause Blizzard to need to take action against you for financial reasons.

For now i think this function is still remain in the 1.32 but need a special websocket to woke up, unfortunatly we can't get this unless we decompile the local war3 server. Use package catcher with that old test version might work but surly the UI haven't been build in 1.32. So just screw this, i'm going to build a w3n compiler by my own.

Reforged - Custom Campaign Data is useable as Local Files



The 'easiest' way is to just make multiple copies of the "_retail_" folder, one for each custom campaign. Truthfully only one folder in each of those "_retail_" folders are going to be the same in all of them (the one with the game client).

There is muti-way to just open a campaign map, combine slk or data files, or just put them together or in retail folder can work, but w3n is still unreadable. I currently use a similar trick, using war3mapconverter to combine campaign data with map data, and import the model and texture files from w3n to maps.

 

[haochuan]

From my observation, war3 core executable DOES read from nested resource files. If you look at CASC viewer, there is a "_hd.w3mod" and a nested "_teen.w3mod", all of which follows the same folder structure.
What's more, the log file at C:\Users\<username>\Documents\Warcraft III\Logs\War3Log.txt logs a line like "...Current Directory: "C:\Program Files\Warcraft III\_retail_"".

I believe:
1. There is a way to set the root directory for game core to load resources.
2. It must be following some directory traversal pattern when reading the resources (like how to override/replace textures and models)

 

[Zorrot]

Thanks.

If my understanding is correct, another hurdle is how to manage the linked resources in .w3n file. I reckon we can only play limited numbers of maps at same time for these reasons:
1. Any linked resources/assets must be extracted from .w3n and copy to _retail_/... folder
2. The game can't read nested "_retail_ like" sub-folder structures
3. Thus, if TWO custom campaign have resource file name clash (which is common for files under /replaceabletextures), the player have to reinstall the "active" campaign.

I'm trying to crack the GameClient in gluemanager.js, hopefully we can leverage on that to decode the .w3n files.

you can use some webpack tracker like wireshark to do this

From my observation, war3 core executable DOES read from nested resource files. If you look at CASC viewer, there is a "_hd.w3mod" and a nested "_teen.w3mod", all of which follows the same folder structure.
What's more, the log file at C:\Users\<username>\Documents\Warcraft III\Logs\War3Log.txt logs a line like "...Current Directory: "C:\Program Files\Warcraft III\_retail_"".

I believe:
1. There is a way to set the root directory for game core to load resources.
2. It must be following some directory traversal pattern when reading the resources (like how to override/replace textures and models)

In case you don't know, the webcore using websocket to control the gameclient, if you find the correct socket, it might changing the working dictionary

 

[Razorclaw_X]

There is muti-way to just open a campaign map, combine slk or data files, or just put them together or in retail folder can work, but w3n is still unreadable. I currently use a similar trick, using war3mapconverter to combine campaign data with map data, and import the model and texture files from w3n to maps.

Adding in all the media assets to every map is where I draw the line. That is just asking to eat disk space for no reason.

w3n is really just another MPQ file with really high priority (the highest being a map itself) and the issue isnt so much that it's unreadable is that you can't make the game client read it.

 

[Zorrot]

Adding in all the media assets to every map is where I draw the line. That is just asking to eat disk space for no reason.

I've just tried and it works, use mpqeditor to add data files into w3x, and leave war3campimported in retail folder, it save space yet worked. That's the only way for now before the absolute solution.

Well i wonder if you like and have spare time, you may use this method to convert some w3n and hand it to players, which is what i'm doing right now. I'm using this method to convert COTF.

TBH I dont know what I could do that anyone couldn't do that had MPQ Editor.

I guess what I'm really asking is I dont know what you're asking to do?


Yeah I came to this conclusion myself, it's only a stopgap solution at best, but it does work from what I can tell (I don't have Reforged, just only used it on the regular current client, but I assume it works just fine there)

Surly I'm aware, it's respectful that you could figure this out, and yes, it's working on reforged very well.

 

[Razorclaw_X]

Maybe I could help you with that (or you could help me convert more old campaign to everyone before I found the absolute solution?)

TBH I dont know what I could do that anyone couldn't do that had MPQ Editor.

I guess what I'm really asking is I dont know what you're asking to do?

I've just tried and it works, use mpqeditor to add data files into w3x, and leave war3campimported in retail folder, it save space yet worked. That's the only way for now before the absolute solution.

Yeah I came to this conclusion myself, it's only a stopgap solution at best, but it does work from what I can tell (I don't have Reforged, just only used it on the regular current client, but I assume it works just fine there)

 

[haochuan]

Adding in all the media assets to every map is where I draw the line. That is just asking to eat disk space for no reason.

w3n is really just another MPQ file with really high priority (the highest being a map itself) and the issue isnt so much that it's unreadable is that you can't make the game client read it.

I'm guessing in that way you need to include all required assets in each and every .w3x files of the campaign, which end up creating tons of duplicated data (since they should be shared across all .w3x in that .w3n container)

I'm able to send message through the browser console atm using the captured GameClient,

Also, I've tried overriding the following section of the code:

              u.GameClient.sendMessage("PlayMission", {
                // filename: a,
                filename: "Campaign/Rise_of_the_Blood_Elves_v3.0.w3n",
                difficulty: this.state.difficulty,
                missionType: r,
                missionIndex: void 0 !== i ? i : -1,
                campaignIndex: this.props.campaign.campaignIndex
              });

Which immediately crashed the game. (Although the "filename" field CAN either be an Array or a String, for which I tried the normal .w3x file and worked)

 

[Razorclaw_X]

Assuming it isn't just a matter of the file type being different (which shouldn't matter because they're both MPQs at the very core) what would happen if you put map data from a .w3x file into the .w3n? (because I'm guessing the game client is looking for map data which a .w3n wouldn't have normally. In theory it wouldn't matter what it was in the .w3n because any .w3x you ran would override that anyway)

 

[haochuan]

Here are more research and test cases for the follow code:

              u.GameClient.sendMessage("PlayMission", {
                filename: a,
                difficulty: this.state.difficulty,
                missionType: r,
                missionIndex: void 0 !== i ? i : -1,
                campaignIndex: this.props.campaign.campaignIndex
              });

filename: string | string[] - Array will handle exceptions and won't crash the game
difficulty: int - Difficulty level

• 0 - Story
• 1 - Normal
• 2 - Hard

missionType: int - Depends on the first string of each Mission setup in ui/campaigninforeforged.txt:

• 0 - CAMP_INTRODUCTION
• 1 - CAMP_CINEMATIC
• 2 - CAMP_CHAPTER_X, CAMP_CHAPTER_X_PART_X, CAMP_SECRET_LEVEL
• 3 - (Will immediately return to the campaign list)
  
• 4 - CAMP_INTERLUDE, CAMP_FINALE
• 5 and above - (Will immediately return to the mission list)

missionIndex: int - Actual mission starts from 0. It can also be -1 to indicate intro and opening
campaignIndex: int - The Campaign index in the webui list, doesn't really matter

 

[haochuan]

Assuming it isn't just a matter of the file type being different (which shouldn't matter because they're both MPQs at the very core) what would happen if you put map data from a .w3x file into the .w3n? (because I'm guessing the game client is looking for map data which a .w3n wouldn't have normally. In theory it wouldn't matter what it was in the .w3n because any .w3x you ran would override that anyway)

So here's what I did for testing:
Before Test:
1. Extract "BloodElves01.w3x" from "Rise of the Blood Elves v3.0.w3n"
2. Copy the .w3x file to "_retail_/campaign"
3. Put all linked assets in "_retail_/", keeping the folder structure required by the campaign

Test cases:
1. Only override the filename to ["Campaign/BloodElves01.w3x"] --> Start mission 1 of Prologue (WORKING)
2. Rename the .w3x to .w3n and override the filename to ["Campaign/BloodElves01.w3n"] --> Start mission 1 of Prologue (NOT WORKING)
2. Rename the .w3n back to .w3x, create a new .w3n and putting content of .w3x in, then override the filename to ["Campaign/BloodElves01.w3x"] --> Start mission 1 of Prologue (NOT WORKING)

 

[Retera]

Is there any chance you could just launch the w3x without extracting it (so that you don't break it) with the exclamation notation? Like "_retail_/Rise of the Blood Elves v3.0.w3n!BloodElves01.w3x" or something? If you open the campaign in the World Editor the title bar will show this notation that they use for loading the map inside of a campaign's archive, it might not be an exclamation mark but it was something similar to this. Maybe a hyphen.

 

[haochuan]

Is there any chance you could just launch the w3x without extracting it (so that you don't break it) with the exclamation notation? Like "_retail_/Rise of the Blood Elves v3.0.w3n!BloodElves01.w3x" or something? If you open the campaign in the World Editor the title bar will show this notation that they use for loading the map inside of a campaign's archive, it might not be an exclamation mark but it was something similar to this. Maybe a hyphen.

yeah, it's a space followed by double hyphen. It doesn't work
Doesn't work with slash "/" either

 

[Zorrot]

Assuming it isn't just a matter of the file type being different (which shouldn't matter because they're both MPQs at the very core) what would happen if you put map data from a .w3x file into the .w3n? (because I'm guessing the game client is looking for map data which a .w3n wouldn't have normally. In theory it wouldn't matter what it was in the .w3n because any .w3x you ran would override that anyway)

well it's a tragedy, just crash (But i may not do this right?) and even if it's somehow running, i guess it might only contain one level.

Is there any chance you could just launch the w3x without extracting it (so that you don't break it) with the exclamation notation? Like "_retail_/Rise of the Blood Elves v3.0.w3n!BloodElves01.w3x" or something? If you open the campaign in the World Editor the title bar will show this notation that they use for loading the map inside of a campaign's archive, it might not be an exclamation mark but it was something similar to this. Maybe a hyphen.

I know there is a classique trick, use .mix instead of a mpq files, the old war3 shall read this as a full file dictionary, i wonder if it's still work in 1.32

yeah, it's a space followed by double hyphen. It doesn't work
Doesn't work with slash "/" either

Yes, cant work

 

[pyf]

I know there is a classique trick, use .mix instead of a mpq files, the old war3 shall read this as a full file dictionary, i wonder if it's still work in 1.32

This is imho very unlikely, because the ability to automatically load mix / mixtape files (which are really dll files afaik) was bound to the Miles Sound System, which was removed with v1.30.0

 

[erfaned]

I got no idea whats going on here .....