Patchwork - MapConverter

[Insanity_AI]

Introduction​

Patchwork - MapConverter is a Node.js console application used to convert between warcraft III's map binaries and their JSON representation. This allows for easier way of distributing map's source code through git and potentially being able to resolve merge conflicts for stuff that is typically edited through the World Editor, like terrain or Object data.

Additionally, patchwork can extract trigger files into multiple files replicating the category tree from trigger editor into your filesystem and also compile this same directory tree into a valid triggers/custom scripts files.

Requirements​

To install and use this tool, you will require the following:

• Node.js v17.1.0 or higher
• npm package manager installed (bundled with Node.js installation)
• node added to system PATH variable (also an option in Node.js installation)

After node.js installation just simply run this command in your terminal: npm i -g patchwork-mapconverter

How to use​

Running npx patchwork-mapconverter will display the options and commands this tool has to offer:

To use this tool, you must provide it a triggerdata.txt file, which you can obtain by opening wc3 archive in CascView. It can be found on _retail_/UI path.
To provide this file you've extracted to the tool, you must active the following option npx patchwork-mapconverter -td <filePathToTriggerData.txt>

Note: This tool works for maps saved in latest World Editor, I can't guarantee they will work for map files created/saved using older World Editor versions.

I believe most of these descriptions offer good enough explanation of what specific options do. However, I will point out a few particularly useful features:

Spoiler: Smart imports

Smart imports option works differently depending on command:

war2json​

When this command is chosen along with this option, smart imports will copy all files that are defined inside war3map.imp (world editor's imports registry file) and paste them into the <output>/<importsFolderName> path, effectively separating the import files from the other map files.

json2war​

When this command is chosen along with this option, smart imports will copy all the files from <input>/<importsFolderName> into <output> but retaining the relative path that was inside the <importsFolderName>, furthermore the option also generates its own war3map.imp file into the output path, populating it with all those copied files within <importsFolderName>, eliminating the need of doing imports by using the imports manager in World Editor.

Spoiler: Compose triggers

Compose triggers options also works differently depending on command:

war2json​

In this mode, the tool will take war3map.wtg and war3map.wct files (GUI triggers, and custom scripts) and export them into <output>/<sourceFolder> while retaining the relative path that was found in the trigger editor. Trigger Library and Category elements are converted into folders, while everything else turns into files. The kind of file depends on the configured extension. (Note: changing the file extension for those sub-options will not cause the tool to change the format these triggers are exported as, they will always be JSON)

Furthermore, for every category the tool will generate a .ini file which specifies a few category options and most importantly, the order of variables/custom scripts/triggers inside said category, because one cannot rely on alphabetical ordering of files (which is done for folders) when it comes to custom scripts due to map loading process.

json2war​

In this mode, the tool will take all files and folders found inside <input>/<sourceFolder> and convert them into categories, triggers, variables, scripts, etc.. retaining the relative paths and trigger editor element ordering stored in .ini files, if they're provided. If the ordering is not provided, the order of these elements is at the discretion of whatever the Node.js runtime decides (safe to assume that it will be random)

Spoiler: Ignore

With this option, you can specify a file with Regex rules of which files should the tool ignore, this file works for both war2json and json2war, if you want the ignore file to work for only 1 of the commands, consider giving the tool a filepath that doesn't exist, or having 2 separate files for each command. (Note: this doesn't ignore tool-exported files, such as the imports file or triggers file, in case of smart imports or compose triggers options enabled.)

Spoiler: TriggerData

Using this option you will have to supply a triggerdata.txt file which can be acquired from CascView opening the game, or a modified triggerdata.txt like GrapesOfWath's Enhanced GUI triggers. The tool and the World editor uses this file in order to read the triggers file.

Note: This tool cannot process map in file mode, maps must be saved in folder mode in order for this tool to process the map's files.

A nice suggestion​

In case of using VSCode, my suggestion is to have a .vscode/launch.json with following content:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "map2json",
            "program": "patchwork-mapconverter",
            "request": "launch",
            "runtimeExecutable": "npx",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "type": "node",
            "args": [
                "-d",
                "-ct",
                "-si",
                "-td",
                "./.vscode/triggerdata.txt",
                "war2json",
                "./build.w3m",
                "./map"
            ]
        },
        {
            "type": "node",
            "request": "launch",
            "runtimeExecutable": "npx",
            "name": "json2map",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "program": "patchwork-mapconverter",
            "args": [
                "-d",
                "-ct",
                "-si",
                "-td",
                "./.vscode/triggerdata.txt",
                "json2war",
                "./map",
                "./build.w3m"
            ]
        }
    ]
}

Doing so, allows you to have a quick way of executing this tool's commands whenever you have something that needs to be tested, or simply for making a map for general public.

map2json task should executed after every World Editor change, like modifying the terrain or defining new units in object editor, so that their JSON counterparts can get these updates.
json2map task should be executed after every code or import change, before testing a map or saving the map to a file for public release.

json2map task, with the forementioned configuration in tasks.json file will generate a build.w3m folder, which can be opened in World editor to test/edit the map.

Another advice is to have this folder added to your .gitignore file, since the idea of this tool is to have only textual representation of these files in your repository.
Furthermore, in patchwork.ignore file, you could add these files:

war3map.wtg
war3map.wct
war3map.j
war3map.lua

So that when executing map2json task, it doesn't touch your source code. And there's no need to have the compiled .j/.lua files in your repo.

If you have any issues you can comment on this thread, or if you find any bugs you can raise an issue on GitHub.

Happy coding/map making!

EDIT: Hello, a few issues have arisen by various users trying this out, so a few things to watch out for.

[Bug] Some object data is not getting translated, primarily preplaced objects stuff, units, their item drops, waypoints, gold mines, etc... so if you have those you shouldn't rely on patchwork to keep that data for you, in that case you should add

.doo

to the patchwork.ignore file.

[Bug] Having an empty patchwork.ignore file will make the conversion process fail.

 

[WaterKnight]

I have converted a .w3a to json to see what the format looks like. This operation should not require a TriggerData.txt. The output file looks like

{
    "original": {},
    "custom": {
        "A000:Amls": [
            {
                "id": "amcs",
                "type": "int",
                "level": 1,
                "column": 0,
                "value": 25
            }
        ]
    }
}

Does "column" stand for the data pointer (DataA, DataB, ...)? I am not sure about using <newId>:<baseId> as the key. You would have to parse these ids again if you wanted to know them and semicolons might also be used in an id. The advantage here is that this structure enforces that there can be no double entries, yet "A000:AHre" would also be a unique key, even though then there are two entries with the same new id. Imho it would make more sense and be more stable to write it like:

{
    "original": {},
    "custom": {
        "A000": {
            "baseId": "Amls",
            "modifications: [
                {
                    "id": "amcs",
                    "type": "int",
                    "level": 1,
                    "column": 0,
                    "value": 25
                }
            ]
        }
    }
}

Original objects can use the id of the object they are modifying without specifiying a base id. And version 3 of the Warcraft format can have different sets of modifications per object: war3map(skin).w3* Modifications

The other reason why I am writing though is to note that to use the json effectively, it would be good to have standardized schemas. There is JSON Schema. This would allow for broad support of tools to consume and write the instances. There are also tools that can work with the schemas like parser generators.

The schema for object modification files could look similar to this:

{
  "$schema" : "https://json-schema.org/draft/2020-12/schema",
  "$defs" : {
    "ObjectsChunk" : {
      "type" : "object",
      "properties" : {
        "objects" : {
          "type" : "array",
          "items" : {
            "type" : "object",
            "properties" : {
              "baseId" : {
                "type" : "string"
              },
              "mods" : {
                "type" : "array",
                "items" : {
                  "type" : "object",
                  "properties" : {
                    "dataPointer" : {
                      "type" : "integer"
                    },
                    "endToken" : {
                      "type" : "string"
                    },
                    "id" : {
                      "type" : "string"
                    },
                    "level" : {
                      "type" : "integer"
                    },
                    "value" : {
                      "anyOf" : [ {
                        "type" : "number"
                      }, {
                        "type" : "string"
                      } ]
                    },
                    "valueType" : {
                      "type" : "string",
                      "enum" : [ "INT", "REAL", "UNREAL", "STRING" ]
                    }
                  }
                }
              },
              "newId" : {
                "type" : "string"
              },
              "unknown" : {
                "type" : "array",
                "items" : {
                  "type" : "integer"
                }
              }
            }
          }
        }
      }
    }
  },
  "type" : "object",
  "properties" : {
    "customObjectsChunk" : {
      "$ref" : "#/$defs/ObjectsChunk"
    },
    "extended" : {
      "type" : "integer"
    },
    "format" : {
      "type" : "string",
      "enum" : [ "OBJ_0x1", "OBJ_0x2", "OBJ_0x3" ]
    },
    "originalObjectsChunk" : {
      "$ref" : "#/$defs/ObjectsChunk"
    }
  }
}

 

[Insanity_AI]

I have converted a .w3a to json to see what the format looks like. This operation should not require a TriggerData.txt.

I am planning on finishing a rewrite of this tool, so the next version won't have this issue. Another reason why I'm doing a rewrite is because the original project that I cloned from does not translate all information; like unit droptables, waygates, a lot of .w3i information, and some other things I find commented in the code.

About the object id in JSON, I am perfectly fine with changing it to fit the JSON schema you provided. And if you have schemas for other files, I'd be happy to implement those designs as well.

 

[WaterKnight]

I am planning on finishing a rewrite of this tool, so the next version won't have this issue. Another reason why I'm doing a rewrite is because the original project that I cloned from does not translate all information; like unit droptables, waygates, a lot of .w3i information, and some other things I find commented in the code.

Yes, saw it on GitHub shortly after.

About the object id in JSON, I am perfectly fine with changing it to fit the JSON schema you provided. And if you have schemas for other files, I'd be happy to implement those designs as well.

That was only an example, it's not finished yet. It was generated from other code I am currently working on in exploration efforts and I will be discussing the schemas with Luashine and others before putting it in a separate repository. A question is also how faithful and permissive it should be. For instance, in the above case, the value type could be derived from the value and/or meta slk, so it's not necessarily necessary to include in the json, unless you want to be able to express any native object modification file. Maybe it could be optional. And there are still some fields we do not know or if all bits in a bit set are used.