Building a C++ source to WASM for W3CE

[GayPornStudio]

Hey, fellow wc3 modders!

Prerequisites:​

• wasi-sdk. This is the cornerstone of the whole process. It bundles a clang preconfigured to build for a proper target and has crt and stdc++ for that. Don't worry if it confuses you, you just need to download it for your OS, unpack somewhere and use compiler or CMake toolchain from there.
• CMake. Technically speaking, it is optional. However, due to existing toolchain file this is the easiest approach to the whole thing.
• Some IDE/text editor.
• Common.j, optionally Blizzard.j and some way to process or access them from C++. I made a simple tool, which I would post later, that processes each of them into a .h, .cpp and .txt file group. .txt file contains list of all functions that would be required later.
• Some way to insert your script into some map. I have an empty-ish folder map and a trivial 30 lines of War3Net program that copies wasm from build output to map, packs it into single file and sends it to my maps folder.
• W3CE. You, technically, don't need it to build a map. Just to run it.

TL;DR:​

An example of how this can be done is attached to the post. It does not include was-sdk or CMake binary, you have to install them manually. However, .j parser and map builder are provided with win64 binaries and their "entire" source, consisting of 5 files including project files. You would also need to pass wasi toolchain path to the CMake. Example of such path can be found in the CMakePresets.json or bellow. The CMake file has 3 main options, that can be passed as -DNAME=VALUE at configuration step or via CMakePresets.json:

• MAP_SOURCE - path to the source map directory. To suport map archives MapBuilder has to be modified. Defaults to source.w3x map in the project directory.
• MAP_DIRECTORY - path to the directory to save wc3 map to. Defaults to $ENV{OneDrive}/Documents/Warcraft III/Maps/Dev.
• MAP_NAME - name of the map file to save map as in the target directory. Defaults to target.w3x

The meat:​

You need to run cmake configuration step (the first one) with --toolchain PATH_TO_WASI_SDK_ROOT/share/cmake/wasi-sdk.cmake.

add_executable(war3map.wasm main.cpp)
target_link_options(war3map.wasm PRIVATE -Wl,--no-entry -mexec-model=reactor)
target_link_libraries(war3map.wasm PRIVATE Natives)

This is a simplified version of my CMakeLists.txt in the source directory. Let's take a closer look at that. My Natives target (which preprocesses CJ and BJ as I have described above and builds cpps to a static library) has this line:
target_link_options(Natives PUBLIC -Wl,--allow-undefined-file=${NATIVE_IMPORTS})
Where NATIVE_IMPORTS is a file that combines all function names from Common.j and Blizzard.j. After that I have a target that packs the map. And also one that combines two .txt files into a single one for -Wl,--allow-undefined-file.

Some clang-specific notes:

• -Wl, means the option after that (without space) is passed to linker. In C++ source file is compiled into an object (main.cpp -> main.obj). Those objects are than linked into something, WASM "executable" in our case (main.obj -> war3map.wasm).
• -Wl,--no-entry allows us to build an executable with no entry point. Usually it is int main() or int main(int,const char*). We don not need our main to be invoked by CRT, so we disable it. Clang explicitly forbids any other signature on any function called main, but that we would workaround at a later point.
• -mexec-model=reactor This changes a bit how the program is treated by CRT.
• -Wl,--allow-undefined-file=SomeFile makes the linker ignore every symbol that is not defined from the provided file. Syntax is one name per line. This is needed for functions that W3CE would import in WASM instance. This is WASM equivalent of natives in JASS. We could allow all undefined symbols if we don't want complex processing with -Wl,--unresolve-symbols=ignore -Wl,--import-undefined. This way one could probably just preprocess .js into a c/c++ header, but unresolved symbols that are not actually imported from WC3 would be only found on opening the map there.

Exporting from WASM with specific name is tricky, but possible. I use the following macro for that:

#if !defined(__INTELLISENSE__)
#define WASM_EXPORT(NAME) __attribute__((export_name(#NAME)))
#else
#define WASM_EXPORT(NAME)
#endif

I use it as a header WasmExport.h. The INTELLISENSE check is only used for MSVS syntax highlighting without more complex configuration. attribute((export_name("main"))) makes clang export the function it is used on to be exported from WASM as main. AFAIK this is the only way one can export a symbol called main that is compiled by clang.

So we use it like this:

WASM_EXPORT(init) void init()
{
}

It declares a function named init with no return or parameters, that is exported as init and defines it as a function with an empty body.

A kind of complete map script example:

#include <WasmExport.h>
#include <Natives/Common.h>
#include <Natives/Blizzard.h>
#include <malloc.h>
#include <string>
 
using namespace std::literals;

WASM_EXPORT(init) void init()
{
}
void InitCustomPlayerSlots()
{
    for (auto i = 0; i < 6; ++i)
    {
        const auto player = Player(i);
        SetPlayerStartLocation(player, i);
        SetPlayerColor(player, ConvertPlayerColor(i));
        SetPlayerRacePreference(player, RACE_PREF_HUMAN);
        SetPlayerRaceSelectable(player, true);
        SetPlayerController(player, MAP_CONTROL_USER);
    }
}
void InitCustomTeams()
{
    for (auto i = 0; i < 6; ++i)
        SetPlayerTeam(Player(i), 0);
}
WASM_EXPORT(config) void config()
{
    SetMapName("TRIGSTR_001");
    SetMapDescription("TRIGSTR_003");
    SetPlayers(6);
    SetTeams(1);
    SetGamePlacement(MAP_PLACEMENT_USE_MAP_SETTINGS);
    DefineStartLocation(0, 512.0, -1216.0);
    InitCustomPlayerSlots();
}
WASM_EXPORT(main) void wc3main()
{
    SetCameraBounds(-3328.0 + GetCameraMargin(CAMERA_MARGIN_LEFT), -3584.0 + GetCameraMargin(CAMERA_MARGIN_BOTTOM), 3328.0 - GetCameraMargin(CAMERA_MARGIN_RIGHT), 3072.0 - GetCameraMargin(CAMERA_MARGIN_TOP), -3328.0 + GetCameraMargin(CAMERA_MARGIN_LEFT), 3072.0 - GetCameraMargin(CAMERA_MARGIN_TOP), 3328.0 - GetCameraMargin(CAMERA_MARGIN_RIGHT), -3584.0 + GetCameraMargin(CAMERA_MARGIN_BOTTOM));
    SetDayNightModels("Environment\\DNC\\DNCLordaeron\\DNCLordaeronTerrain\\DNCLordaeronTerrain.mdl", "Environment\\DNC\\DNCLordaeron\\DNCLordaeronUnit\\DNCLordaeronUnit.mdl");
    NewSoundEnvironment("Default");
    SetMapMusic("Music", true, 0);
    const auto str = "C++ main successful!"s;
    DisplayTextToPlayer(Player(0), 0, 0, str.c_str());
}
WASM_EXPORT(malloc) std::int32_t wc3malloc(std::int32_t size, std::int32_t align)
{
    return reinterpret_cast<std::int32_t>(malloc(size));
}
WASM_EXPORT(free) void wc3free(std::int32_t ptr, std::int32_t size, std::int32_t align)
{
    free(reinterpret_cast<void*>(ptr));
}
WASM_EXPORT(native_callback) void native_callback(std::int32_t id)
{
}

The notable point here is that main, malloc and free use different C++ names to prevent name collisions and "invalid" main signature error.

Also, here is my complete MapBuilder.exe source:

using War3Net.Build;
using War3Net.IO.Mpq;

var OUTPUT_FOLDER_PATH = args[0];
var BASE_MAP_PATH = args[1];
var SOURCE_SCRIPT_PATH = args[2];
var TARGET_SCRIPT_PATH = @$"{BASE_MAP_PATH}\war3map.wasm";
var OUTPUT_MAP_NAME = args.Length > 3 ? args[3] : @"target.w3x";

if (File.Exists(TARGET_SCRIPT_PATH))
    File.Delete(TARGET_SCRIPT_PATH);

File.Copy(SOURCE_SCRIPT_PATH, TARGET_SCRIPT_PATH);

Directory.CreateDirectory(OUTPUT_FOLDER_PATH);

// Load existing map data
var map = Map.Open(BASE_MAP_PATH);
var builder = new MapBuilder(map);
builder.AddFiles(BASE_MAP_PATH, "*", SearchOption.AllDirectories);

// Build w3x file
var archiveCreateOptions = new MpqArchiveCreateOptions
{
    ListFileCreateMode = MpqFileCreateMode.Overwrite,
    AttributesCreateMode = MpqFileCreateMode.Prune,
    BlockSize = 3,
};

builder.Build(Path.Combine(OUTPUT_FOLDER_PATH, OUTPUT_MAP_NAME), archiveCreateOptions);

It can be used from CMake like this:

find_program(
    MAP_BUILDER
    NAMES MapBuilder
    HINTS ${CMAKE_CURRENT_SOURCE_DIR}/../MapBuilder/bin/Debug/net8.0
)
set(MAP_SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/source.w3x")
set(MAP_DIRECTORY "${WC3_MAPS_DIR}/Dev")
set(MAP_NAME "target.w3x")
set(MAP_RESULT "${MAP_DIRECTORY}/${MAP_NAME}")
file(GLOB_RECURSE MAP_FILES ${MAP_SOURCE}/**.*)
add_custom_command(
    OUTPUT "${MAP_RESULT}"
    DEPENDS war3map.wasm ${MAP_FILES}
    COMMAND ${MAP_BUILDER} ${MAP_DIRECTORY} ${MAP_SOURCE} $<TARGET_FILE:war3map.wasm> ${MAP_NAME}
)
add_custom_target(map ALL DEPENDS "${MAP_RESULT}")

This would automatically rebuild it with all targets build if any file inside map folder changes or war3map.wasm has been rebuilt.

This approach works with static libraries very well. This is how I build natives and some other stuff. The only thing is you have to add linker attribute for exports like this:

target_link_options(stdwc3 PUBLIC -Wl,--export=native_callback)

Also, for C++ std one need to disable exceptions (AFAIK they are not supported by WASI stdc++). It can be done for the entire project in CMake like this:

add_compile_options(-fno-exceptions -fno-rtti)

Just to be safe it also disables rtti. I didn't test it yet, so unless you need it (e. g. for dynamic_cast), you are probably better of without it.

From TriggerTrap's suggestion from discord: one has to refcount everything extending agent. That means manually calling two extra "natives":

void __handle_refcnt_inc(std::int32_t agentHandle)
void __handle_refcnt_dec(std::int32_t agentHandle)

The process can be automated if you implement some wrapper for WC3 API and use it rather than natives directly. Improper use or lack thereof would lead to either random destructions with now invalid (pointing to another object) ids or leaks. I plan to implement and publish such a wrapper library, but the basic idea is like this:

extern "C"
{
    extern void __handle_refcnt_inc(std::int32_t agentHandle);
    extern void __handle_refcnt_dec(std::int32_t agentHandle);
}
namespace wc3
{
    Agent::Agent(std::int32_t id)
        : Handle(id)
    {
        __handle_refcnt_inc(id);
    }
    Agent::~Agent()
    {
        __handle_refcnt_dec(id);
    }
}

This two natives also have to be accounted for exports either with the .txt files or -Wl,--export=__handle_refcnt_inc -Wl,--export=__handle_refcnt_dec linker flags.

Some final notes:​

• In theory this should work on any OS.
• I have only tested it on Windows 11, using MSVS 17.9 preview as IDE, but it doesn't require any specific IDE and should work just as fine on ubuntu with vi or whatever. At least up to the point of starting w3ce.
• AFAIK the only compiler that can be used for this kind of thing is clang. At least I am unaware of any other able to compile C++ to WASM. In fact, emcc is a clang warper.
• I have ditched emcc in my attempts due to a lot of strange things that it bundles are browser-related and hard to exclude.
• One can use all of this for pure C script with basically no modifications.
• I have attached my natives archive to the post. One can produce Natives static library target for the CMake above or just add it to the war3map.wasm target.
• Technically W3CE uses wasm3 interpreter currently doesn't support WASI. But it also doesn't check for missing imports, which allows to use WASI except for stuff like std::chrono, std::fstream, etc.
• At the moment W3CE doesn't invoke void _initialize() method, generated by clang. This leads to memory (like globals) not being initialized. That can be worked around like this:

extern "C"
{
    extern void _initialize();
}

WASM_EXPORT(init) void init()
{
    static bool t = []()
    {
        // We want to ensure this gets called before anything else, but not twice
        _initialize();
        return true;
    }();
}