ObjectMergerFixer: fixing some known problems with ObjectMerger

[EdwardElric]

about
i created this tool to fix some known errors of "ObjectMerger" (part of GRIMEX), hence the name "ObjectMergerFixer".

ObjectMerger is one of the features from JNGP (= Jass New Gen Pack). Beside other tools, it can be found in "...\JNGP\grimext\ObjectMerger.exe".​

reason behind decision to write another tool instead of fixing ObjectMerger & recompile:
the latest available source code of ObjectMerger misses some features of current ObjectMerger.exe, like LUA support.
unluckily the latest source code of ObjectMerger got lost.
quote from PitzerMike (author of ObjectMerger):

Unfortunately it's not the most up-to-date version of the code.
The latest parts of the grim extensions are missing (mainly the code for PatchGenerator.exe which is used to generate the UMSWE patch).
If anyone has downloaded the code from the old repository, please post it here.
You will know it's the latest verson if it has a sub-folder called PatchGenerator.

source:
Source code of all my work - Wc3C.net



what it does
fixes following issues of latest ObjectMerger.exe:
- unit field (uaen) values

should be: 0, 1, 2 or 3
ObjectMerger.exe generates: 48, 49, 50 or 51
info:
0 = nothing
1 = only attack 1
2 = only attack 2
3 = both
note:
as far as i know there is NO workarround for this issue!
link:
[Solved] - LUA: Ascii typecasting on ObjectMerger​

- unit field (utco) values

should be: -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 or 12
ObjectMerger generates: 12589, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 12337, 12593, 12849
note:
there is a workarround which works without this tool by using LUA via JASS externalblock instead of JASS external
link:
[lua] bug in object generation script​

how to setup
- unzip: ObjectMergerFixer.zip

note:
- i recommend freeware "7-Zip"​

- move: ObjectMergerFixer.exe

to: ...\JNGP\grimext\ObjectMergerFixer.exe
note:
- JNGP = name of your JGNP (= Jass New Gen Pack) folder.
- "ObjectMergerFixer.exe" needs "sfmpq.dll" which is in this folder.​

- open: ...\JNGP\jasshelper.conf

look for: [externaltools]
paste: "ObjectMergerFixer","grimext\ObjectMergerFixer.exe"
note:
- paste can be anywhere, but before the next []-block begins.​

how to use
important! before you use this tool:
this is a first version without much feedback so far, i strongly recommend you make a backup of your map before using this tool. theoretically, this tool could corrupt your map by damaging your war3map.w3u. improvements or experienced C++ programers are cordially invited to make the tool as safe as possible.
now if you saved a backup there is nothing which keeps you from using it like:
after all your calls to ObjectMerger from WE trigger JASS code, add: //! external ObjectMergerFixer.
when you save your map it gets called automatically from "JassHelper".
you have to restart WE & reload your map to see changes, this also has to be done only once identical to calls to ObjectMerger.



plans
- add fix for umlaute (ä, ö, ü)
- maybe create a thread in the tools section or even it could be added to the next JNGP version



requirements
- OS (= Operation System) Windows (as far as i know you need it for the WINAPI functions used in the code...)



source code
ver 4.1.1: C++ code by SourceSeeker - 275 lines - codepad
ver 4.1.0: C++ code by SourceSeeker - 279 lines - codepad
ver 4.0.1: C++ code by SourceSeeker - 197 lines - codepad
ver 4.0.0: C++ code - 378 lines - codepad
attached "ObjectMergerFixer.exe" was compiled with dated freeware "DEV C++".
"DEV C++" must be configured to use 32 bit compiler because of the sfmpq.dll from PitzerMike, otherwise following seems to cause troubles:
hSFMpq = LoadLibrary("SFMpq.dll");



version history

Hidden

version 4.1.1 [2016_11_05]
- fix: in case of wrong (uaen) fields, the fixed bytes didn't get written back to MPQ if there were no wrong (utco) fields.

problem was:
function getFixedW3U(...) returned an empty vector instead of the input vector. now it returns input vector if no fixes were applied, which makes use of this function more apparent & also gets rid of an if-block checking for empty vector.​

version 4.1.0 [2016_10_30]
- changed function "getFixedW3U_uaen" to "getFixedW3U":

the function now takes 4 parameters instead of 2 to act more generically.

new api:
vector<char> getFixedW3U( vector<char> v_input, const char * c_wrongBytes, const char * c_rightBytes ) {...}​

improved reading of hex values!

instead of reading at fixed index, everything after uaen.... or utco.... is read until the next null byte appears.
the read content is checked for a digit in range from -9 to 99 (range of utco is -1 to 12).​

- new feature for (utco):

fixes ObjectMerger error with WE (= World Editor) unit field 'utco' (art - team color).​

unreleased version 4.0.2 [2016_10_24]
- fix: in case "war3map.w3u" has nothing to be fixed, don't try to write back an empty W3U (resulted in false "MpqAddFileFromBufferEx(...)" & showed an error message box).

version 4.0.1 [2016_10_23]
- renamend function "function_getOutputVector" to "getFixedW3U_uaen".
- cloned project to have a private & public version for max & min comments.
- cleaned up code, added link to version with min comments.

version 4.0.0 [2016_10_22]
first public version:
- fixes ObjectMerger error with WE (= World Editor) unit field 'uaen' (combat - enabled attacks).

ObjectMerger generates wrong values. this derives from a wrong variable type. 'uaen' needs integer type, but ObjectMerger generates string type.
example:
put following into a WE trigger as JASS code, save map & compare field 'uaen' of created units:

//! external ObjectMerger w3u hpea 1d1A uaen 0 unam "Arbeiter-custom-test1"
//! external ObjectMerger w3u hpea 1d1B uaen 0 unam "Arbeiter-custom-test2"
//! external ObjectMerger w3u hpea 1d1C uaen 0 unam "Arbeiter-custom-test3-uaen"
//! external ObjectMerger w3u hpea 1d1D uaen 1 unam "Arbeiter-custom-test4"
//! external ObjectMerger w3u hpea 1d1E uaen 2 unam "Arbeiter-custom-test5"

instead of 0, 1 & 2 it has 48, 49 & 50! these are simply the ASCII values of the characters 0, 1 & 2.​

 

[Aniki]

Is the fix "generic" or specific for the (uaen) field? If its specific there could be more problematic fields, I think the unit field "Art - Team Color (utco)" has the same problem.
Well anyway, I guess it helps either way.

PS:

improvements or experienced C++ programers are cordially invited to make the tool as safe as possible

I couldn't write C++ to save my life... ;P

 

[GhostWolf]

I would agree that it will make more sense to change function_getOutputVector (which by the way is a horrid name) to be more of a generic patching function.

Either way, good job, it gets the job done, albeit you should probably clean up your code.

 

[EdwardElric]

Is the fix "generic" or specific for the (uaen) field?

at the moment the fix is just specific for the field (uaen), because that was the first error i faced from ObjectMerger i couldn't workarround. if you know more such errors like the stated field (utco) you could tell here & i will see if i can fix them.
edit:
searched for utco if there exist any workarrounds & found this thread:
[lua] bug in object generation script
WaterKnight uses \0 instead of 0.
made a test:

note:
be careful when testing this with a W3M map. as soon as you have a unit with the field (utco) customized, the map automatically gets saved as W3X. since i had a W3M & missed the WE popup, i reopened the old map which had the old data obviously! had a little headache before i got it.​

(utco) range = -1 to 12
with //! external ObjectMerger w3u opeo 1d2A utco 0 unam "Peon-custom-test0":

-1 --> 12589
0 --> 48
1 --> 49
2 --> 50
3 --> 51
4 --> 52
5 --> 53
6 --> 54
7 --> 55
8 --> 56
9 --> 57
10 --> 12337
11 --> 12593
12 --> 12849​

with //! external ObjectMerger w3u opeo 1d2A utco \0 unam "Peon-custom-test0" or //! external ObjectMerger w3u opeo 1d2A utco "\0" unam "Peon-custom-test0":

\-1 --> 3222876
\0 --> 12380
\1 --> 12636
\2 --> 12892
\3 --> 13148
\4 --> 13404
\5 --> 13660
\6 --> 13916
\7 --> 14172
\8 --> 14428
\9 --> 14684
\10 --> 3158364
\11 --> 3223900
\12 --> 3289436​

=> so, the mentioned workarround doesn't seem to work... well, i didn't test it with LUA, but does that make a difference?
edit2:
made another test with LUA like WaterKnight showed & what should i say? it really works! there must be a different interpretation of data between LUA (externalblock) calls & direct (external) calls to ObjectMerger.exe. so, the workarround indeed works! anyhow, for guys who don't know this workarround & for getting away from workarrounds it's better to implement a fix for (utco). will have a look!

I would agree that it will make more sense to change
function_getOutputVector
(which by the way is a horrid name) to be more of a generic patching function.

- added "version history" to post #1 & ReadMe.txt
- changed function name "function_getOutputVector" to "getFixedW3U_uaen"
- built new version 4.0.1 & uploaded to post #1
i plan to add 1 function for each wrong unit field id, i guess otherwise, to be able to have a more generic function including all similiar fields, i have to make big code changes. moreover i have no clue how to approach this. only learning reference i had/have is source code of ObjectMerger.
edit:
oh wait! with "generic" i understood saving W3U contents to a C++ map or mmap & refer to their members like ObjectMerger does (this is the part i didn't want to learn yet from ObjectMerger source code in "common/obj.cpp" as it seems quite complicated). but if i just adjust the function to take 4 parameters instead of 2 i guess it already could be used in a quite "general" way. i think of something like:
vector<char> getFixedW3U( char * c_buf, int size, char * c_wrongBytes, char * c_rightBytes ) {



edit:
faced an error: in case "war3map.w3u" has nothing to be fixed, code tries to write back an empty W3U (result in false "MpqAddFileFromBufferEx(...)" & shows an error message box).
already fixed it in 4.0.2, but still want to add fix for (utco) before releasing new version...

 

[EdwardElric]

new version 4.1.0 is out!
code fixes & new feature: fix for (utco)! thx Aniki for the hint!
see changelog in post #1 for details.

 

[GhostWolf]

The patching function you ended up making is indeed what I meant.
Good job!

 

[WaterKnight]

If you want to generalize the uaen problem, rather than checking for the fields, you should probably look at the variable type specified in the meta slk.

 

[EdwardElric]

...you should probably look at the variable type specified in the meta slk.

looking into "...\Warcraft III\War3Patch.mpq" into "Units\UnitMetaData.slk", variable type of...
...uaen is "attackBits"
...utco is "teamColor"
searching this SLK for "attackBits" or "teamColor" just mentioned 2 field IDs are matches... so what else to look for? or should i also look into other SLKs?
edit:
looking for other variable types like "attributeType" also returns only 1 match for field ID upra (found on hero units). tried to generate such an object with object merger to see whether is causes problems, but it works... ok, have to admit, that i called the preprocessor code like //! external ObjectMerger w3u Hpal Ad1A upra "INT" unam "Paladin-custom-test1". so type "attributeType" seems to work with strings anyway (range: AGI, INT, STR).
edit2:
ah, now after our chat i understand what you mean. so, just for the core problem, instead of using code to search for field IDs with wrong variable types, it would be rather apparent to load the "Units\UnitMetaData.slk", translate field IDs with variable types & search for wrong variable types. ok, that would be really more direct, but as the code works for now & im not aware of other fields beside (uaen) & (utco), i will keep it the way it is for now.



edit:
version 4.1.1 out!
fixed a bug, details in version history of post #1.