Standalone object editor - practical experiences?

[Zwiebelchen]

I recently stumbled upon this:
http://www.hiveworkshop.com/forums/tools-560/war3-object-editor-0-7-1-a-239577/

Has anyone here already tried that or uses it regulary? Seems like the perfect tool to make creating items a lot faster, since it basicly gets rid of the actual editor and its slowness.

Does it have bugs or issues that could mess up the map or is it save to use?

 

[Adiktuz]

IDK, the actual object editor seems to work fast for me...

 

[WaterKnight]

No, cannot start it. But I have an own object editor for my project, which is basically a simple data grid and you can implement custom fields/add own object types. Another set of tools convert the files and build wc3 native objects and other stuff to import. One major advantage is that you can have multiple windows parallel. Same goes for coding. Export it in separate files/import it in eclipse and you gain multitasking.

What I would like to possess however, is an additional lightweight program that lists all the icons, models etc. similar to the World Editor for fast previewing and copying the paths.

 

[Zwiebelchen]

Bummer. So do you have any plans to merge your seperate tools into one WaterKnight?

 

[WaterKnight]

At least the builder is realized programmatically, meaning I constantly change the code to achieve the effects I want. For example the uberTooltips of abilities in my project are assembled from several fields. Nor does this allow a proper co-existence with the World Editor, the native objects are all done in slk and unless you start the tools as an external of JassHelper, the map would be locked, and ofc the WE does not reload the data etc. The reason why I can neglect this is that I rarely need to access the WE as most relevant tasks are exported with exception of the starting placements like terrain, placed units.

It's not so neat and isolated that I would make it a public tool, it follows my own conventions.

 

[Barade]

Are there any features you would require from an independent object editor? I myself thought about importing/exporting single objects with dependencies (hero with all abilities etc.) and merging it to another map.

At the moment I am working on my own object editor in my project wc3lib. I have two layers. One basic format layer:
http://wc3lib.org/api/html/classwc3lib_1_1map_1_1_custom_objects.html

where you can read/modify and write object data (useful for scripting and specific tools)
and a GUI layer which emulates the object editor from wc3:
http://wc3lib.org/api/html/classwc3lib_1_1editor_1_1_object_editor.html

one advantage probably will be that it can be used on Linux but some suggestions would be nice which features are required by users.

 

[Zwiebelchen]

Basicly one of the main requirements for a standalone object editor is that you can still browse and preview textures and models for the icon/model fields, including imported resources.

If we could have an external tool that allows that, object editing would be a breeze (and removes the stupidly long saving process of war3 maps).

 

[Deaod]

Such an editor would require:

• being able to read MPQs (im glad theres already a library for this: StormLib)
• replicating WC3s MPQ loading algorithm, with priorities and all that stuff
• parsing a lot of different file formats, most of which are documented (the file formats themselves arent all that hard to parse)
• correlating all the information within those files (a task i find very hard)
• displaying models and their animations (which i have no experience in, so i cant judge the difficulty)

And then you actually have to slap a working GUI onto this.

 

[Zwiebelchen]

Such an editor would require:

• being able to read MPQs (im glad theres already a library for this: StormLib)
• replicating WC3s MPQ loading algorithm, with priorities and all that stuff
• parsing a lot of different file formats, most of which are documented (the file formats themselves arent all that hard to parse)
• correlating all the information within those files (a task i find very hard)
• displaying models and their animations (which i have no experience in, so i cant judge the difficulty)

And then you actually have to slap a working GUI onto this.

I don't see this being a problem. There's been plenty of tools that do this already. WC3 viewer, magos model editor, blp lab, etc.

 

[Barade]

The problem is to use them all together, I think. Except for displaying models the rest is not that hard but takes time to get done.

I can already display BLP files, parse custom units and SLKs.
There's also an MPQ slave and priority support but for working all together it still needs more improvements.
One major feature will be Linux support but I was actually asking for features which the usual object editor does not offer already. Performance might be one. Luckily I've designed all modules as independent applications which only need a set of sources (MPQ archives).

 

[Deaod]

I can already display BLP files, parse custom units and SLKs.

As i said, parsing all that stuff is not all that hard. The hard part comes when youre trying to figure out what value goes where, and how all the different formats interact.

For example:
war3map.wts contains all custom strings. This file of course has its own text-based file format.
war3map.w3u contains modifications to default units and new units. String entries in this file that are text refer to strings in the WTS file.
However, default values for the default units are not in the w3u, but in some SLKs, and a couple other files that look like INI files (and these INI files contain comma separated values for certain properties that are INDEXED).
There is also a metadata SLK that contains information about the individual properties of unit objects, like how to sort those properties in the editor, or the string thats used to describe the property in the editor (which of course is in some entirely different file thats never referred to anywhere).

And this is only the fucking units. There are a bunch of other objects like abilities, or doodads, or upgrades which i dont have the faintest idea of how to parse all the data for. Its a mess and i seriously question the sanity of the guys who came up with this.

So yeah, im really sorry, but replicating the object editor is not all that easy, and i dont think displaying models or textures is the hard part.

 

[WaterKnight]

It's the pretty much the same for other object types. Only that units use multiple slks and their meta slk is shared with items.

Each object editor type has (a) base slk(s), a meta slk, the modification formats only differ in the aspect that abilities, buffs, doodads and upgrades possess an additional "variation/level" attribute per modification.

There is a bunch of base text (profile) files. They are outsourced because they are language-dependent. It does not matter where the objects are placed, so just traverse through all of them/write to a single one if you want to edit it.

Afaik the variation comma separation can be replaced by multiple "field=", else you should put tooltips in quotes to be able to have commas within them.

It's not that hard either, you just have to know a few special things (like how some meta slk fields translate) but that can be debugged.

 

[Barade]

The main problem is that I do not have that much time and there are not many people who still work on any tools especially most wouldn't use C++ or have any idea of the code I write at all.
As WaterKnight already posted the relations between all meta data etc. is not that hard to be figured out.
It's just a lot of work to put everything together and make it flexible and maintainable. In the end I write a lot of parsers and many many unit tests for all formats and maybe finally there will be another object editor. My intention is to make the interface as abstract as possible that one could use a SLK parser for any file or a txt parser or a custom units parser etc. The GUI itself puts all sources together in a very simple way.
I did the same with a trigger editor module which still is not finished but it is really just the time you need to create a usable, abstract API.

Unfortunately, most tools do use their own hand written messy code and nobody can reuse it (except for StormLib).

 

[Dr Super Good]

or have any idea of the code I write at all

Well your code is quite strange... Such as why are you defining fixed width types that standard C already does (unit32_t as an example)? I know in the very old days you needed to do this so that code could be ported between C compilers where int and types were not standardized and so had to be changed on a compiler basis but that is why they eventually introduced standard int library which comes with all int types and even compiler helper types (allowing better optimization).

And speaking of portability... For some very silly reason you are using bit fields (example at tilepoint.cpp). This is not portable as the exact behaviour of bit fields is not defined and is compiler specific. Most modern compilers no longer support bit fields (ignores them and allocates base type) without using special compiler flags. This is because it is impossible to define the atomicity of bit fields in a multi threaded environment as most processors are incapable of manipulating memory in units less than a single word size. So instead of introducing strange and difficult to detect compiler specific bugs when two threads try and manipulate bitfields, they have decided that ignoring bit fields completely except for the actual evaluation value (which is simple bit masking) was the best solution.

The byte order of any member of a struct is also compiler dependant so cannot be trusted. Additional padding may also be introduced on certain platforms.

 

[Barade]

wow you mentioned two things which you think are strange ...

First of all, types with fixed size have been introduced in C++11. When I started with the project this was not supported by the compilers I used. Boost libs did provide such types and I used them but I still used typedefs for each module separately to name the types like in the specification (for example the MDX/MDL file says "long" so I created a typedef named long). Finally, I could replace the typedefs again by the C++ builtin types but it really is a minor thing.

The thing about bit fields I did not know that way. I just knew that they might not be that portable. Anyways it still could be fixed and its not a really major problem.
Even if you would bring up concerns about the design of the lib, everything is open source and you could rather participate instead of bringing two minor points up here only to show people what you do know about programming languages.

Comparing wc3lib to any other tools (except for StormLib but which has really not the cleanest code I've ever seen ...) for wc3 it is one which finally runs natively on Unix systems at all. So please use your time to help improving it instead of wasting your time to write something like that.

 

[Dr Super Good]

First of all, types with fixed size have been introduced in C++11.

Actually it was introduced in C99... I do admit some extensions were added in 11 but I am not sure if those have anything to do with the standard int sizes.

Comparing wc3lib to any other tools (except for StormLib but which has really not the cleanest code I've ever seen ...) for wc3 it is one which finally runs natively on Unix systems at all. So please use your time to help improving it instead of wasting your time to write something like that.

I am slowly working on my own interface for w3e in c++. Other formats may come later.

Currently I am still messing around with bit field wrapper classes. The idea is to emulate a bit field with such a class and use those types for the interface so that internally it is not possible to set a unsupported value.

 

[Deaod]

There are a bunch of other objects like abilities, or doodads, or upgrades which i dont have the faintest idea of how to parse all the data for.

I contradicted myself here. What i meant was that fitting all the different data-pieces together is something i have no really good idea of how to do yet. I shouldnt have used "parse" in a different meaning than before.

It's the pretty much the same for other object types. Only that units use multiple slks and their meta slk is shared with items.

Each object editor type has (a) base slk(s), a meta slk, the modification formats only differ in the aspect that abilities, buffs, doodads and upgrades possess an additional "variation/level" attribute per modification.

There is a bunch of base text (profile) files. They are outsourced because they are language-dependent. It does not matter where the objects are placed, so just traverse through all of them/write to a single one if you want to edit it.

Afaik the variation comma separation can be replaced by multiple "field=", else you should put tooltips in quotes to be able to have commas within them.

It's not that hard either, you just have to know a few special things (like how some meta slk fields translate) but that can be debugged.

Have you actually tried putting it all together for units? I have (successfully), and i still dont think its a trivial task.

If this was just about merging the contents of a bunch of WTS and W3U files, then it really wouldnt be all that hard. Unfortunately its not that easy, so you have to deal with a bunch of hardcoded stuff and piece everything together from what little sense you can make of it.

Let me repeat: Parsing all those different formats is easily done. Whats hard is fitting the different files together so you can actually get the default values for custom units or abilities.

 

[WaterKnight]

Let me repeat: Parsing all those different formats is easily done. Whats hard is fitting the different files together so you can actually get the default values for custom units or abilities.

Mapfile > War3Patch.mpq > War3x.mpq > War3.mpq

objMods > slks, txts

Get the files of topmost priority, slks and txts grant the base data, read the objMod files to get the custom objects with their parent ids, using the meta slks as a translator, the objMods can overshadow the base data.

What I have done before and use in my current project is an objMod extractor (because I did not want to manually transfer everything to my own external format), an object builder that converts my definitions to slk and stuff that cannot be done via slk (data of > level 4) to objMods and profile data to txt. So no real merging with exising data but of course I have some slk lib and wc3binaryFile in my signature and parsing the txt should not be a hindrance.

I do not see why units are so special except that their slk data is distributed.

I see no merit to work with objMods though. The problem is that the editor does not reload a map's slks/txts. If you want to concentrate on a specific map, deal in straight slk/txt and edit the local files for the editor (convert the objMods everytime you save) or work with that standalone object editor.

 

[Barade]

Does anybody know how the model and icon lists are created? Is there any slk/txt file which lists them or is it created dynamically from the standard unit/abilities etc. lists?

 

[WaterKnight]

The model/icon selection in the editor? It iterates over the object definitions (slk/profile).

 

[Barade]

Okay thanks. Do you know what the columns "changeFlags", "section" and "sort" are for in a meta data slk file?
As far as I know the values are sorted alphabetically.

 

[WaterKnight]

section is used in SkinMetaData.slk and MiscMetaData.slk because war3mapSkin.txt, war3mapMisc.txt and war3mapExtra.txt are divided into sections. Ex:

[Misc]
ReviveBaseFactor=0.0
ReviveLevelFactor=0.0
ReviveMaxFactor=0.0
DamageBonusNormal=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
DamageBonusPierce=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
DamageBonusMagic=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
DamageBonusSiege=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
DamageBonusHero=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
MinUnitSpeed=1.0
MaxUnitSpeed=522.0
DamageBonusChaos=0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
SummonedKillFactor=0.0
GrantHeroXPFormulaC=0.0
IntManaBonus=0.0
StrHitPointBonus=0.0
ResearchRefundRate=0.0
StrRegenBonus=0.0
UpkeepGoldTax=0.00
UpkeepLumberTax=0.00
GrantHeroXP=0
EtherealDamageBonus=0.00,0.00,0.00,0.00,0.00,0.00,0.00
MissDamageReduction=0.0
DamageBonusSpells=1.00,1.00,1.00,1.00,1.00,1.00,1.00,1.00
IntRegenBonus=0.0
ReviveMaxTimeFactor=20.0
ReviveTimeFactor=20.0
MaxHeroLevel=20
AgiDefenseBonus=0.2
PickupItemRange=200.0
DropItemRange=150.0
GrantNormalXPFormulaB=0.0
GrantNormalXPFormulaC=0.0
GrantHeroXPFormulaA=0.0
HeroExpRange=0.0
NeedHeroXPFormulaB=0.0
NeedHeroXP=200,500,850,1250,1700,2200,2750,3350,4000,4700,5450,6250,7100,8000,8950,9950
UpkeepUsage=10000
DayLength=60.0
DefenseArmor=0.04
StructureDecayTime=100.0
BoneDecayTime=100.0
TradingIncLarge=5
TradingIncSmall=200
ReviveRefundRate=0.0
UpgradeRefundRate=0.0
TrainRefundRate=0.0
ConstructionRefundRate=0.0
PawnItemRate=0.5
StrAttackBonus=0.0
GrantNormalXP=0
HeroFactorXP=0
Dawn=0.0
Dusk=0.0
AgiAttackSpeedBonus=0.0

[TWN1]
DependencyOr=h000,o000,u000,e000,n01S
Name=Town hall level 1

[TWN2]
DependencyOr=h005,o002,u002,e002,n01T
Name=Town hall level 2

[TWN3]
DependencyOr=h007,o004,u004,e004,n01U
Name=Town hall level 3

[TWN4]
DependencyOr=h009,o006,u006,e006,n01V
Name=Town hall level 4

[HERO]
Name=A hero

[TALT]
Name=An altar

I do not know about the other two. Probably just object editor-only information, not necessary to convert to slk. I have not used them in my object builder.

 

[Barade]

Thanks a lot!

 

[Zwiebelchen]

So I guess, you are still working on this Barade?

That's awesome then!

If the scope is the problem, limiting it to items would already be a huge step forward. I don't think lots of people will use this to edit units, because, well, when do you ever have hundreds of different unit types so that a standalone tool would be handy?

Currently, I'm using a LUA script to generate items via a single LUA function.
The script generates tooltips and sets the visual item model depending on the item type I define. It also generates the color coding for the item name, sets vendor prices, etc..
When I'm done, all I need is replacing the LUA script lines by a normal JASS function call (via find & replace) to set up the item in the internal map registry.

It's pretty awesome and creating items was never so fast for me. But there's a downside, again, which is icons. I still need to get the icon paths, attachment model paths, etc. manually. So even if I can generate an item with just one line of text, I still have lots of manual labor to do there, browsing menus, checking the preview window, copy & pasting paths, etc...

I'm writing this as a motivation for you. Even if Newgen 2.x fixed LUA; we still need this!

 

[Barade]

At the moment I am very busy with learning for exams but the project is still under development. For my own mod The Power of Fire I also started using LUA scripts and the ObjectMerger which really is great tool.

Maybe I could add scripting abilities to the stand alone Object Editor at some point. It really would be great to have a stand alone Object Editor with the ability to run scripts.

What's about the linked already existing and working Object Editor? I thought it does already offer the basic editing.

Btw. I've starting writing a WeatherMetaData.slk file which allows you to edit weather effects as well.

 

[Barade]

The model/icon selection in the editor? It iterates over the object definitions (slk/profile).

Hey another question about these combo boxes which show the default icons and models.
There is for example the category Extra for icons and models.
Do you know how this category is created by the iteration?

 

[WaterKnight]

They are special as ithey do not occur on any standard objects. They are defined in UI\WorldEditData.txt as I could find out with a simple mpq search.