war3map.w3e file

[eejin]

I am currently working a lot with GhostWolf on map rendering and loading logistics and think that a MediaWiki as Ralle mentioned would be very nice. We've got some information that's just waiting to be noted down

 

[Dr Super Good]

and loading logistics

Loading a map is logistically challenging? I can understand the logic is quite arcane but the amount of data to be loaded is small by modern terms...

 

[eejin]

Loading a map is logistically challenging? I can understand the logic is quite arcane but the amount of data to be loaded is small by modern terms...

Well it might not have been the right wording, but there are some intricacies in how to load the map correctly. Things like the hierarchy (map -> tileset -> game mpqs), slk files and their information, etc. Some of these might be known already, but I don't think they have been centralized. (Please send me a link to it if they have been written down somewhere).

 

[eejin]

Until an official wiki is established I have started to write down some things on my GitHub page. Right now I think it is the most detailed and correct description about the war3map.w3e file there is.

 

[Dr Super Good]

Until an official wiki is established I have started to write down some things on my GitHub page. Right now I think it is the most detailed and correct description about the war3map.w3e file there is.

Ground height and water height logic are wrong...

They are only 14 bits each, upper 2 bits unused for both. This is why height for both ground and water has the strange 0x2000 constant since that represents approximately half its valid range of 0 to 0x3FFF.

The magic number "-89.6" is actually defined by the base tileset in the game MPQ files. The slk file lists these in normalised (SC2 style) units. So it is actually the tileset used for testing has a water height offset of -0.7 which translates to -89.6 (-0.7 * 128). Be aware that although most tilesets use -0.7, others do have different values for this so one must first parse the tileset data files before one can compute actual water height for a node. For that reason it is technically not part of the w3e format but rather part of the WC3 terrain rendering process.

 

[eejin]

They are only 14 bits each, upper 2 bits unused for both.

Have you ever found those bits to be any different that 0? And does the water_height then not have a boundary flag?

I also already did write a little passage about -89.6 at the water_height bit.

Any other thought on it?

 

[Dr Super Good]

Have you ever found those bits to be any different that 0?

No, but setting them to 1 makes no difference either. I extensively tested this when creating my Java w3e APIs. I am guessing in the WC3 source code they are implemented as bit fields which is why it results in 2 unused bits due to alignment requirements for 14 bits (2 bytes). Instead of defining a field for flags, they might have made each flag a 1 bit member which is why it manages to pack with the water height field but does not with the ground height due to the order of declaration. In my Java APIs I assign these unused bits special flag values in case people wanted to encode hidden data with them lol.

 

[GhostWolf]

Interesting. Any other information not present (or not wrong) in any specs that you know of?
I already assumed the boundary flag is wrong because it made no sense for it to be there.

I can indeed see water heights of -0.6, -0.7, -0.75, and -1.5.
Just another SLK to load

 

[Dr Super Good]

I already assumed the boundary flag is wrong because it made no sense for it to be there.

Boundary flag should be the black mask that makes up part of the border. It can also be painted in WorldEdit and is often used in RPGs to separate areas.

There seems to be an off by 1 bug in WorldEdit where some of the map edges (I forget exactly which) do not have the boundary flag set for the nodes directly touching the edge despite being out of the map bounds. All subsequent nodes not touching the edge do have the flag correctly set. This results in a slight visual bug with the terrain and objects placed directly on the map edge that is effected.

It is valid to define more than 16 tile types and 16 cliff types. It is pointless to do so however as the terrain mesh can only use the first 16 tile types and 16 cliff types. Also it seems meshes and textures for only the first 2 cliff types are loaded, despite being able to address 16 of them. I recall testing that it is definitely 16 cliff types a node can use as values above 2 did effect the nearby terrain graphics, just not in a useful way.

Cliff types always load their associated tile type and are always loaded before the tile types listed in the tile type list. This is why they are always first in WorldEdit. One can skip declaring cliff tile types from the list to save up to 8 bytes although this is not recommended. If not using a custom tile set, both cliff type and tile type arrays are completely ignored but should still be filled in to match the standard tile set.

 

[GhostWolf]

@Dr Super Good
After thinking further about the tile points - I don't think you are right.
It would make more sense to me that the ground/water heights are indeed 16 bit integers, but that the game logic simply caps them to 16384.

I tried messing with cliffs, and you can get amusing very buggy behavior in WE, but so far didn't see anything that can be used.

@Cokemonkey11
PMs seem ineffective, I still got no response from Frotty. In fact, the only way to get responses from a couple of you is to intentionally write inflammatory messages.
Jass DB is not the main focus here, but rather a part of a whole.
In the end I am simply trying to push anyone here to actually do something.
It's easy to open this thread and complain, and then disappear without any word or replying to anything.
Not saying that's you, but that's this thread.
Prove me wrong.

 

[Dr Super Good]

It would make more sense to me that the ground/water heights are indeed 16 bit integers, but that the game logic simply caps them to 16384.

C/C++ offers functionality called bit fields. Judging by how things such as tile type and cliff type are only 4 bits it would make sense that the type being used is using them.

Why 14 bits and not the logical choice of 16? I am guessing Blizzard decided saving 1 byte per node was a worth while optimization then. The irony would be if the file format is not translated to internal structures or padded with 1 byte by Warcraft III since if anything it would degrade performance due to unaligned memory reads. This is likely a case of pointless optimization, especially when viewed today.

If you are referring to how Warcraft III treats them internally, then if they are bit fields it would treat them as a 16 bit type but with automatic masking applied. The value is not clamped and is subject to under/overflow behaviour as one would expect of a 14bit integer. This can be seen in WorldEdit by editing the mechanics of the ground deform tool and trying to raise or lower ground too much, it will wrap around on under/overflow.

My evidence for ground being 14 bit is that I tried making a 15 and 16 bit sloped map and it suffered from overflow style wrap around at the expected frequency. At first I wondered why this happened and confirmed the mechanics both in game and in editor are consistent. Only when reducing it to 14 bits did overflow stop and the result was a map that was entirely and continuously sloped. To rule out bad programming one can force such values to a w3e file using a hex editor thanks to the simple file format.

If you are referring to how the terrain meshes are created/positioned by the game, they likely use 32 bit floating points, or at least a much larger type than a 14 bit bitfield as the game has no problem correctly producing max cliff level + max ground height.

I tried messing with cliffs, and you can get amusing very buggy behavior in WE, but so far didn't see anything that can be used.

Exactly my conclusion. Although I guess someone may want square grass/dirt cliffs for some reason?

 

[Kakerate]

I am currently working a lot with GhostWolf on map rendering and loading logistics and think that a MediaWiki as Ralle mentioned would be very nice. We've got some information that's just waiting to be noted down

Possibly unrelated, but do you have a fix for the generated script of where units are. the local unit variable isn't nulled, could you make a fix for that?

 

[eejin]

I have significantly updated most of my formats and added new ones which you can find here. In particular (thanks to GhostWolf) that the maximum terrain height is defined in "UI/MiscData.txt".

// Global height limits
// 0 = low ground, each cliff level = 128
MaxHeight=1536
MinHeight=-128

 

[sethmachine]

eejin, thank you again for your effort in this work. It is already a valuable resource.

As a request, is there any reason the World Editor you are created is only Windows based? Can you make it OS independent, e.g. compile for macOS? I'll put this as an issue. I looked through your code and I don't see anything you're doing that ties it to Windows--after all, Warcraft III does run on macOS (but Blizzard hasn't made a new world editor). For example, perhaps switching to Java? I would be interested in helping this effort if you think it's feasible (I have experience in Python and Java, but not C++ or .NET).