• Listen to a special audio message from Bill Roper to the Hive Workshop community (Bill is a former Vice President of Blizzard Entertainment, Producer, Designer, Musician, Voice Actor) 🔗Click here to hear his message!
  • Read Evilhog's interview with Gregory Alper, the original composer of the music for WarCraft: Orcs & Humans 🔗Click here to read the full interview.

wc3lib

Status
Not open for further replies.
Level 27
Joined
Feb 2, 2006
Messages
1,578
Hello there,
I have been working on a project called "wc3lib" for some years now which aims to support various formats of Warcraft III.
It's written in C/C++. The website can be found on: https://wc3lib.org
Unfortunately, I never had any other contributors because there isn't that much Warcraft III modders anymore and many of them don't know C/C++.
Anyway, I just want to point it out to you if you're interested in contribution.
This is an old article about MDLX view implementation using OGRE.
MPQ reading is already partly supported (I still have a bug with BZip compression). Reading BLP and MDX files does also work and there are some GUIs for a World Editor emulation already, as well as interfaces for reading map data (such as cameras, pathmap etc.).
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Seems interesting for developping wc3 tools, like an other jass preprocessor or script optimizer.

I'm guessing you have tested it on linux, have you also tested it on windows, mac maybe ?

Also, you don't say how you would like some help.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,258
I could have sworn someone already made a fully functional MPQ system that supports every MPQ type from Diablo II to StarCraft II (and probably Diablo III soon if not already).

If you want authentic behaviour, they also made a header file that allows you to interact with the WarCraft III libraries to open MPQs in the exact same way as the game does.

Are you sure that the tileset constants have to be coupled into the program data?

Are all those dynamic structures really needed for storing data? I am sure simple arrays would be faster and suffice.

When writing terrain mesh data, you incorrectly limit the player to 16 tile types and 16 cliff types. There is no limit as long as the number is elements is stored correctly. Obviously only the first 16 can practicly be used due to the nibble sized index but WarCraft III will still load more than 16 (is a valid mesh file). Any such restriction has to be applied at a higher level of program architecture as it is not truely an invalid number.

Map vertices are stored in a structure more complicated than needed which makes iterating through all elements needlessly complicated. I imagine the best solution would be to have an array stored in the natural read/write order and then declare mapping functions that depends on X and Y size. Altering map size would force a rebuild of the array during which some translation algorithms could be used (potentially taking advantage of bulk copy functions).
 
Last edited:
Level 27
Joined
Feb 2, 2006
Messages
1,578
I could have sworn someone already made a fully functional MPQ system that supports every MPQ type from Diablo II to StarCraft II (and probably Diablo III soon if not already).

If you want authentic behaviour, they also made a header file that allows you to interact with the WarCraft III libraries to open MPQs in the exact same way as the game does.
Are you talking about StormLib or SMPQ? I am aware of those tools but I want to replace them with my library (StormLib has a very bad code design and no real abstraction). Of course this means more work for me but as I wrote, reading of most files does already work. Furthermore, this library should only support Warcraft III formats for the moment, so I am happy with that.

Are you sure that the tileset constants have to be coupled into the program data?

Are all those dynamic structures really needed for storing data? I am sure simple arrays would be faster and suffice.
I wanted to have an abstract interface for all kinds of tools, so each tool doesn't have to load and write the data in its own way. I know it seems to be a very huge library of classes for each data type but imagine minor tools which only change some parts of an MDX files for example.
Of course it is not as efficient as modifying files directly but I guess in most cases you have to rewrite the file anyway (if you add some structures or their size is being changed).

When writing terrain mesh data, you incorrectly limit the player to 16 tile types and 16 cliff types. There is no limit as long as the number is elements is stored correctly. Obviously only the first 16 can practicly be used due to the nibble sized index but WarCraft III will still load more than 16 (is a valid mesh file). Any such restriction has to be applied at a higher level of program architecture as it is not truely an invalid number.
Yes, you are right. I didn't consider that. It's a minor fix. I just have used my environment and tilepoint classes for fixing some terrain yet.

Map vertices are stored in a structure more complicated than needed which makes iterating through all elements needlessly complicated. I imagine the best solution would be to have an array stored in the natural read/write order and then declare mapping functions that depends on X and Y size. Altering map size would force a rebuild of the array during which some translation algorithms could be used (potentially taking advantage of bulk copy functions).
Hm, most of my other structures are implemented like this (I use vectors if the number is stored). For terrain I thought it would be nice to have a map but maybe I will change this because I've already changed many structures to get a better performance on reading and writing. By now, I prefer this way and mapping the structures if they have to be searched faster (as you suggested, I guess).

@Troll-Brain:
I would like some help in way of contribution (as Dr Super Good already did by writing his post, thank you). If some of you are familiar with C/C++ you can just post reviews or send patches. As you wrote, I only tested it on my Gentoo Linux system, so testing on other operation systems is always appreciated!

Please consider that this was more or less my little hobby project where I improved my C/C++ knowledge, so the design of this library shouldn't be the best but still better than some random C-like source code which is really hard to maintain.
It would be great if I could improve this library with your help so we can maybe have our own World Editor one day with better terraining and pathmapping support etc.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Well, if i finally beat my laziness i will use this tool to make a jass parser, then a jass optimizer and maybe an other fugly jass preprocessor.

I will focalize on map script features only, but with being able to access map data, such as constants and object editor data, anything that can interact with the map script.
 
Level 27
Joined
Feb 2, 2006
Messages
1,578
Well, if i finally beat my laziness i will use this tool to make a jass parser, then a jass optimizer and maybe an other fugly jass preprocessor.

I will focalize on map script features only, but with being able to access map data, such as constants and object editor data, anything that can interact with the map script.
I don't think this tool will help you creating a JASS parser. To access a map script you could already use StormLib (I prefer smpq on systems with KDE if you're going to create a more abstract tool).
Some years ago, I wrote a vJass parsing tool called vjassdoc without any real parsing tools which is quite a bad idea. Later I wanted to add some better parser to wc3lib using Boost Spirit but it seemed to be too much work for me at that moment.
The best would be something like clang provides for C/C++ so your IDE could analyze your code when writing but I really don't know enough about that topic.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,258
so the design of this library shouldn't be the best but still better than some random C-like source code which is really hard to maintain.
Do not be afraid to just use C when appropiate. C++ is not always the best for everything.

You are currently coupling the reading/writing of files to the class for each data type. I would imagine that some factory class would be best suited for that as one does not need access to that functionality if you are creating a file via other means (like a new one). You could also then allow files to be sourced from other locations such as from a converter that could try and generate a file from other data.

An example could be the mesh data that can be sourced from a WC3 mesh file, generated from a height map or try and convert StarCraft II mesh data.

As for parsing script... What about YACC? I know very little about parsing but that apparently can be used to make compilers.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
I just need a tool to extract and handle files from wc3 maps, ofc i have to use a lexical analyser and parser.
I may use QUEX as a Lexical analysiser, and lemon for the parser.

What i need above all is will to do it.

StomrLib doesn't let to easily extract and handle data from wc3 object editor files (at least i mean if they are not .slk), right ?

Now, maybe it's not in your objectives neither.
 
Level 27
Joined
Feb 2, 2006
Messages
1,578
Do not be afraid to just use C when appropiate. C++ is not always the best for everything.
I think when you are at the point you want to use OOP for better abstraction and reusing code, you don't need to use clean C anymore. The funny thing about the StormLib is that it is written in C++ actually but I wouldn't call it real C++. Just take a look on its search API. Looking at its source code it seems that StormLib only stores some results you were already searching for instead of storing all file hash entries in a hash table after loading the archive. Besides you get a SFILE_FIND_DATA pointer as result which stores some file information instead of a proper file class instance which is used by the library itself.

You are currently coupling the reading/writing of files to the class for each data type. I would imagine that some factory class would be best suited for that as one does not need access to that functionality if you are creating a file via other means (like a new one). You could also then allow files to be sourced from other locations such as from a converter that could try and generate a file from other data.

An example could be the mesh data that can be sourced from a WC3 mesh file, generated from a height map or try and convert StarCraft II mesh data.
That's why I used std::istream and std::eek:stream. Usually you can use something like Boost iostreams for alternative sources like buffers, for instance.

As for parsing script... What about YACC? I know very little about parsing but that apparently can be used to make compilers.
Bison and Flex have some disadvantages compared to Boost Spirit but the whole parsing stuff is not that necessary for me at the moment. At Wc3C.net I've read about the further development of the JassHelper (which I really require for my mod). Let's see where it goes and maybe someone will write a new cross-platform JassHelper one day which shows more than one syntax error .
 
Level 27
Joined
Feb 2, 2006
Messages
1,578
I just need a tool to extract and handle files from wc3 maps, ofc i have to use a lexical analyser and parser.
I may use QUEX as a Lexical analysiser, and lemon for the parser.

What i need above all is will to do it.

StomrLib doesn't let to easily extract and handle data from wc3 object editor files (at least i mean if they are not .slk), right ?

Now, maybe it's not in your objectives neither.
When I was working on an object editor emulation I came to the point where I needed to read SLK files.
StormLib can extract them of course but that's all, so if you find some API which handles SLK files (I've only searched around the KOffice/Calligra project yet) let me know please.

wc3lib currently supports object editor data partly. Just checkout map::CustomUnits and map::CustomObjects but I really need to fix some bugs on reading modification values!
 
Level 27
Joined
Feb 2, 2006
Messages
1,578
Hello there!
As a result of some improvements you can convert MDX into MDL files now by using a simple command line tool:
wc3converter --oformat=mdl --overwrite <dir path> <mdx files - require .mdx extensions>

MDL reading doesn't work yet :(
During the next week I am going to improve MDLX displaying support using OGRE hopefully.

Maybe someone could try to compile the project on Windows or Mac which would help me to fix some basic bugs because I only work on my Linux system.
Besides I would like to create an installer or something similiar for those systems but I don't really know where to start yet.
 
Status
Not open for further replies.
Top