Compressing object data, removing unnecessary fields

[Barade]

Hey, could it be possible that if you create an ability based on a hero ability like channel and change some fields but then decide to make it to a unit ability that the fields which only concern a hero ability stay changed?
If so, is there a tool which removes all changes which affect fields which are not important for the unit ability anymore.
The same goes for unit -> item abilities.

It seems that I have a lot of string entries from such fields which are hidden since the abilities are unit abilities now. I'd like to reset/remove them.

 

[Xonok]

I avoid this problem by generating abilities with a special script that can read in object data from text files.

Example

raw4=ABlk
baseid=AEbl
type=abilities
aher=0
alev=1
amcs1=0
anam=Blink(Spirit)
aub11=Teleports the Spirit a short distance.

It's basically a wrapper on top of objectmerger.
Since I have this I can just delete whatever abilities I want to refresh and they'll be back after the next save. Have to close and open the map, but that's not a problem for me.

 

[Zwiebelchen]

I think MapOptimizer removes all the junk data anyway.


But yes, using LUA and object data to generate abilities is the Endlösung of object editing.

 

[Barade]

I've added this function to my own object editor. You can simply select "compress data" and it removes all changes to hidden fields.

 

[WaterKnight]

Well, I do not know if you can generally say that a field is obsolete. E.g. there are 3 different unit types: normal units, buildings and heroes. Only buildings show the "Can be upgraded to" field, yet there are a couple of maps that use this feature successfully and still revert it to a normal unit to gain the traits of both, may not be so obvious what can be useful in the end.

 

[Dr Super Good]

Well, I do not know if you can generally say that a field is obsolete. E.g. there are 3 different unit types: normal units, buildings and heroes. Only buildings show the "Can be upgraded to" field, yet there are a couple of maps that use this feature successfully and still revert it to a normal unit to gain the traits of both, may not be so obvious what can be useful in the end.

In which case the field "can be upgraded to" should be available to all units. Not so much an unused field, but rather an incorrectly hidden field.

 

[WaterKnight]

Yeah, but can you say for sure that a specific ability field ABCD that is only shown for EFGH is never used by IJKL? Still, you would not want to have them all displayed.

 

[Barade]

Maybe I could use an exclude option or do you have a list of such fields? Or if the meta data is incorrect I could overwrite it since I do already provide custom meta data files for water and weather effects

 

[Barade]

I've fixed some stuff in the latest version of wc3lib and used the compression for my modification's map and it reduced a great amount of strings since I did not rely on auto generated Object Data. I am wondering now if I should go further and check if the objects are used at all in the map script or better in multiple map scripts since the object data is shared between many maps.

I guess wc3optimizer does the same already but only with one single map script. Besides my goal is not to optimize the map for playing only but to optimize it to reduce the strings in the war3map.wts file further to make the translation process easier.

If anyone is interested in using the current version of my object editor:
- I ported wc3lib to Qt5 and removed dependencies to KDE4 and KF5, so it should be easier to compile it on Windows.
- Unfortunately I still get undefined references for the Boost Libraries on Windows: Using Boost 32 bit libraries on Windows with MinGW32 leads to undefined references
- Linux users should not have any problem compiling it. I will update the RPM soon and upload it here: wc3lib

What do you think of a better search and redo/undo functionality for the Object Editor?

 

[Xonok]

I remember specifically testing at one point whether the optimizer gets rid of duplicate strings. It doesn't.

 

[Barade]

Are you talking about unused object editor strings or just about the same strings used multiple times? The optimizer should replace the STRING ID references by the strings itself and remove war3map.wts or reduce it in the end.

edit:
Updated the RPM for wc3lib

 

[Xonok]

Duplicates. When you have several identical strings it will still have a separate entry in the string table for each one of them.

 

[Barade]

Well I thought the optimizer removes the string table and places the entries directly but I am not sure.

 

[Xonok]

That would actually increase map size in many cases, so I don't see why it should do that.

 

[GhostWolf]

I would assume it does it in a smart way (pretty simple to check if something will take more bytes inlined or not).

 

[Barade]

Why should it increase the map size if nearly the complete war3map.wts file can be removed afterwards? Having a big war3map.wts file + references to STRING_<xxx> should be bigger.

 

[Xonok]

The strings themselves are represented pretty much the same way in w3a as they are in string tables, with the exception that they are named.
In cases where a string exists in multiple places the string table would be more efficient... except that it isn't, because the optimizer doesn't fix this issue.
I've tested that the string table doesn't detect duplicates at all and neither do optimizers.
I'd like to see an optimizer that deduplicates things within the string table. That would give my use case way more benefit than moving strings into the object data files.

 

[Barade]

I've discovered already two fields of object data which have been compressed by my wc3lib object data compressor but have actually been changed and used:

• AItx - DataA1
• Arsg - UnitID

I've checked "AbilityMetaData.slk" in the archive "War3Patch.mpq" and there are no entries for 'AItx' or 'Arsg'.

So I will fix the bug in wc3lib to compress fields which have no entry in "AbilityMetaData.slk" at all. I am just wondering what the field IDs are then.

Btw. I'd like to add a validate/warning function which checks tooltips for not existing object/field IDs or warns if the IDs are not from the current object or anything related. This could help to find wrong entries much faster.

 

[DracoL1ch]

Im using custom IDs in text docs to provide traslate-ability, for instance