Streamlining Tool Development

[Retera]

So I was writing a PM that got entirely too long about an aspiration I had, and I was wondering if others were interested in this.

I had been asked what I was trying to do, what I was trying to ask this guy for, and it led me to a memory about a project idea that I started pursuing but did not finish.

JWC3
A set of Java libraries for editing all relevant Warcraft filetypes, as well as dropping them into program GUIs.

This would in theory include:

• JFlex MDL Parser
• Pure-Java MDX Parser
• A set of format-independent MDLX classes representing all components within WC3 models
• Pure-Java MPQ Parser
• Pure-Java BLP Parser to convenient classes such as BufferedImage
• A JASS syntax highlighter and IDE that could be dropped into any application
• A Trigger Editor with advanced functionalities, a lot like the SC2 trigger editor but using a system such as vJASS for support of class definitions
• An Object Editor interface, almost exactly like the one in WC3 but with advanced potential for customizeability
• All open source, so those who come after us can add features
• W3A, W3U, W3T, W3H, etc parsers, so that you could read and write object data with ease
• So-called 'Profile' parsers, things that could real (maybe write) the game's general format of .txt files outside of the unit data SLKs

Here's what was going to be in my overflow-edly-long PM:

Sometimes I'm not altogether sure exactly what I want to do, either. Earlier this year I had the idea of trying to gather together Java Warcraft libraries (not all my own code) to be able to build a new, better Object Editor and Trigger/JASS Editor program that could save as runnable EXE game mods, so that I and other people would not have such a hard time making melee-type mods.

Here was a picture of a running mockup of one small part of it all (Note that this is, in fact, a JTree and a JTable and NOT the world editor. It is interactive and actually loads all the game units in the tree on the left, and displays their unit data on the right):

Object Editor of JWorldEdit

However, without the borrowed code from ODE, this system was only using its Java-based MPQ loading stuff to load SLKs, and then Java-based BLP parsing to make some BufferedImages out of that stuff.
Both of those, however, involved wrappers for native C code, which are quite a bother. I had it all put together for my model editing software, the Matrix Eater, to have it able to load textures out of the MPQs.

I had also had in mind to embed the 3d modeling aspects of my other software right into the 'Art - Model File' option for units, so that in the same workflow as choosing a unit's model you could also make modifications to it.

I recently acquired a native MDX parser to go with the MDL parser I wrote in high school, and wrote a translator between the classes used by the two. The MDX parser, or my use of it, however, has proven to me to be notoriously bad at things, and the hacky OpenGL code I wrote a few years ago does not draw models well (relative to their true in-game DirectX appearance):

MatrixEater in Action (also a Java program)

The MatrixEater's interface, however, leaves a lot to be desired. With no true animation editing, and selection by vertex instead of VertexGroup, it would almost be more ideal to re-code the whole thing (which might prove not-too-difficult given the MDL/MDX java parsers that my repo has now) and have its interface take after 3DSMax 5 instead of after Oinkerwinkle's Vertex Editor.

In my recent conversations with other Warcraft content developers, my other aspirations like nesting helper abilities inside of main abilities via a customizable Java re-write of the World Editor's Object Editor were conceptually shoveled into JNGP injects for the time being, where I would not have to rewrite a vJASS syntax highlighter, or the trigger editor, or most of the rest of that stuff.

In truth, what I'm striving for is a faster way to make our ideas into game content as Warcraft content developers. When I was in middle school I thought it was cool that I knew how to make a disabled icon by adding a border texture, and then reducing Saturation and Lightness in photoshop. Thought it was cool to make a missile projectile with a dummy unit, dummy model with 1 attachment point, and a mountain of JASS.

They're not. Both of those things are complete wastes of time. A content developer should be able to pick a picture file, perhaps even just a relevant portion of the screen where a unit is currently being rendered, and have that instantly become a unit's icon including a disabled border. Whether an autocast or passive border is applied should be decided by the editor when it saves, based on whether an ability is actually an autocast or passive ability (With the potential to change the templates, of course, for those who prefer their own art style).

Missiles should not be thought of in terms of goofy dummy units, but with a system like I mentioned before:

Retera's Little JNGP xemissile Addon

So to be honest, I could say that I don't know what I'm doing, but there's one thing for certain: the closer we get to all aspects of Warcraft modding being accessible through clear and logical Java classes, the sooner somebody could easily just write one streamlined editor for all of Warcraft 3. And parsing the .W3A format is one more step in that direction. (Hence I was PM'ing this guy about how to do that in the easiest way)

Description of what I was envisioning that I mentioned to someone else a while ago:

Description of mod editor vision

My vision would be something where you could change things like unit codes more easily, import/export groups of units conveniently, and drop units in custom "folders" while editing, although they would all end up back in something the original editor and game could use and understand. Having to use a separate program to create enabled and disabled icons (even with the Button Manager I started using right around the start of HF v2 instead of GIMP or PSD) troubles me. Importing each one and setting the "ReplaceableTextures\CommandButtons" path, or even setting it for a large group, seems like a colossal waste of time.

So, my vision for this program would be that you could click the "Interface Icon" option on a unit, then choose not just an internal blp but also any image file on your computer's hard drive, and have the borders/disabled/importing all done for you internally by the program. If this is executed correctly, I'd also like to see something where a unit's model wasn't set to a path, but to an actual model. Thus, since in the "HFv2" map I was developing I went back to using "war3mapImported" paths to speed up time, there should be some code that understands if I move that model I want to change the unit's reference path as well as the position of the actual model both in the mod and in the Import Manager. Doing things like setting all three of those separately is simply a waste of my time.

For what it does, the world editor (and JNGP) are particularly useful tools, so I'm not entirely sure that some of the more fringe features I'd like to see for modding would be as useful for true mapping.

Another feature I was visualizing would be the embedding of code into abilities. It doesn't seem that unreasonable to think that an ability could have an "On initialization" and "On use" JASS sections in the program I'm visualizing (or maybe "On Use" could just be replaced with a general code section that you could setup triggers in). I also would like to implement model editing directly into this new data editor.

My most anticipated objection: why Java? For me, it's because I already have half of these libraries there, and relevant knowledge to use them. However, if a good argument could be made of a language where all of these tools could be amassed efficiently and used for developing new stuff, I could see there being an argument out there that that might also be likewise worth doing.

I just don't want to be told I'm not a good programmer if I don't use pointers. I do not want to live out my days thinking about how the computer was built in the 1970s, but rather thinking directly of what type of Warcraft game content I wish to develop.

 

[Zwiebelchen]

I'm all supporting your idea to generate a new framework for modding and I love Matrix Eater when it comes to combining model assets into one...

But here's the thing: why extend GUI?
The reason why Galaxy has an extended and powerful GUI language and why Galaxy itself is so cumbersome to use is because blizzard hates modders.
But the Hive doesn't hate modders.
We love scripting languages and we encourage everyone to learn them. And extended GUI is counterproductive to that goal. :/

 

[PurgeandFire]

I have definitely given this thought myself. Many people have made individual tools to parse files, and the specs have been well-fleshed out/reverse-engineered, but we still don't have a stand-alone editor.

There are many things that could be quality-of-life improvements. e.g. an improved import manager, faster object editor, etc. When I envisioned this "new editor", I always thought of it as starting off as an MPQ editor. Over time, you could add support for each file type and have a designated editor/toolkit for it. e.g. pathing (wpm) or shadows (shd) in particular. It is a large undertaking for sure, but it is a man's right to dream. huehue

 

[Ezekiel12]

the closer we get to all aspects of Warcraft modding being accessible through clear and logical Java classes, the sooner somebody could easily just write one streamlined editor for all of Warcraft 3.

I think this is the most important sentence in your post. JNGP can only do so much and is very messy as it combines a clusterfuck of different systems.

If you have the energy, start a Java github repo and search for people who can help.

 

[moyackx]

Just one question: WHY JAVA??????

just to make it OS independent?

 

[Daffa]

Gotta get my World Editor into Mac for sure.

 

[oger-lord]

Maybe you are interested in the following links:

My Java MDX and blp lib: https://github.com/OgerLord/WcDataLibrary/
Crigges Java MPQ lib: https://github.com/Crigges/JMPQ-v3
(still in development, atm only support reading of files)

I also have some unfinished classes for slk/w3e/doo/w3d that only supports the operations I need.
Also you could try to contact Gexxo (creator of castle fight), his java tool GMSI http://www.eeve.org/board/viewtopic.php?f=23&t=1102 should be able to access most map files.

Edit:
Also Wurst is able to acces wc3 object files: https://github.com/peq/WurstScript/...wurstscript/src/de/peeeq/wurstio/objectreader

 

[edo494]

Extending GUI is the worst thing you could ever try to do. It makes mods require to have all the possible GUI extensions out there downloaded and ready to be used when reviewing maps, because otherwise you cant load them in editor.

I like Moyackx's question

 

[Retera]

These are some good questions. I think I have a soft spot for GUI because I learned it many years ago before learning JASS, and it always felt like an easy go-to way to do simple things until I needed JASS.
However, I definitely see where the argument is coming from that GUI is very limited in a lot of dumb and silly ways. Perhaps if a cool editor tool does get made, it should have better scripting syntax highlighting, debugging, and such, rather than any time being spent on having GUI in it.

@oger-lord
I've made a forked copy of your MDX library from several months back that has a function to convert into the MatrixEater's MDL classes, which gives essentially a model editor using your very API, but I have discovered various bugs in the final MDXs (whether due to the MDL/MDX class translation or to bugs in the MDX library I'm not 100% certain). Perhaps we should combine code on this. [Edit: This needs a github repo! We'll see if I can get it set up sometime soon, currently on my way to sleep.]

@moyackx
I am biased because Java was the first language I learned after JASS, and all windowsy things I've made have given me stupid amounts of trouble. Java, although its auto-updater is the dumbest thing on the planet, works on almost anything except when things like the native C libraries stand in the way. My windows 10 laptop won't run Magos's tool even, mostly because of some arcane compatibility issues and the fact that my core Windows API libraries are borked and broken. I called Microsoft tech support and they said I should uninstall the operating system and choose a different one. Java doesn't really have those problems for me -- if software I made was purely in Java, bugs in my OS-dependent proprietary APIs wouldn't suddenly bust all of the code.
With that said, I guess it's mostly a question of what can be done in a reasonable amount of time. Do we have some kind of windows APIs for Warcraft modding, means to assemble them seamlessly, and a good Windows coding IDE?

 

[Nestharus]

I'm all for this. I need Java MPQ stuff too. I use Java with ANTLR4 for compilers. I also think we need a way to support plugin editors. GUI editor for UI design like winforms or Qt anyone?

 

[Zwiebelchen]

I have absolutely no objections against Java. The auto-updater can be dealth with. Java is pretty amazing in it's cross-platform compatibility. And Mac users had been screwed over with Newgen long enough.


Now here's the only problem though: recreating the world editor is a daunting task. It will probably take you months if not years to replicate every feature - and that excluding all the useless features like the AI editor or the object manager (not object editor).

Especially the terraining tools will be complicated as hell to implement, as you might be able to paint directly into a copy of the w3e file, but you constantly have to update your visuals on screen depending on that temporary copied file. And those should look similar to how WC3 actually handles the displayal ingame.


On the bright side...this could allow for some entirely new terraining tools, like actually painting the water height map directly without the cumbersome cliff tools.

 

[oger-lord]

I think there are two steps:
1. Create Java libraries for the different files
2. Create Tools like Editor/ModelEditor/whatever

But there is another question we should decide:
-How do we want to visualize models and stuff?
I use the JMonkey Engine because I know the code and I can display a model without textures using only a few lines of code.
On the other hand the concept for animations seems to be different.
Crigges suggested that I should have a look at libgdx, any other ideas?

 

[Ezekiel12]

I think there are two steps:
1. Create Java libraries for the different files
2. Create Tools like Editor/ModelEditor/whatever

Yes.

-How do we want to visualize models and stuff?

I don't know about JMonkey but libgdx is certainly top notch with frequent updates. Sad thing is that the API is changed quite frequently.

Then LWJGL (the base of libgdx) and maybe JavaFX comes to mind. But no experience with those. Just heard that JavaFX is not all fine and dandy though it's supposed to be the Swing successor.

 

[Dr Super Good]

Just one question: WHY JAVA??????

just to make it OS independent?

Because Java is one of the best multi-platform languages you can get. Unlike C/C++ where you need to re-compile for every platform (not user friendly) with Java the same program will work on all platforms.

 

[edo494]

on the other hand, with Java, you have to install the environment on every machine you want to run it, so it is no more multi-platform than C/C++

 

[Ezekiel12]

The user Barade is the owner of http://wc3lib.org/. I don't know from which sources he got his code but he is still developing his toolset (which is in C++).

I will write on his profile so he can maybe join the discussion.

 

[Barade]

Hey this sounds cool and useful but how far are you? wc3lib basically has the same goals but is written in C++ (still intended to be cross plattform).

I've already started a custom object editor which works for Reign of Chaos files for many field types: http://wc3lib.org/screenshots/objecteditor.png

I've also started a trigger editor which partly works for Reign of Chaos files too: http://wc3lib.org/screenshots/triggereditor.png

I would not underestimate the amount of work you have to do and tests you have to write. One benefit of wc3lib is that it uses Qt/KDE. Actually with Qt5 I want to reduce/remove much of the KDE stuff some day. It fully integrates into your Desktop Environment but as you can see from my Avatar I am Linux fan ...

It is a lot of work to do to write a GUI like the Object Editor or the Trigger Editor from the scratch! Especially if you want to integrate the model format one day. In wc3lib at least the icons are shown.

I don't think that it really matters if one uses Java or C++ or anything else as long as it can be used on Windows and Linux. What matters is how well it is documentated and how far its state is.
For example I wrote my own MPQ reading implementation which looking back maybe was a waste of time.

If you are a skilled programmer and serious about this I would suggest to join my project but that is what everyone with his own project would say.

From my experience I can say it is better first to create one single tool which uses an abstract interface and then continue with other tools.

 

[Dr Super Good]

on the other hand, with Java, you have to install the environment on every machine you want to run it, so it is no more multi-platform than C/C++

Linux and Mac should come with it pre-installed. Only Windows needs to install it and frankly who does not have it installed?!

The main difference is that Java programs should pretty much always work if they work now. Even With Java 12 or 29 in the distant future it should still work if it works now. With C/C++ you almost always will encounter some portability problem down the line requiring a re-build and possibly some maintenance work.

If this was to build a game or other real-time application I would be the first to say Java is a bad choice. But this is an editor and Java is perfect for that sort of program. One distributable for all platforms is just what people want from a usability perspective.

I am currently working on more Java compliant I/O methods. Specifically Blizzard files should be viewed as objects and not as files to be parsed. Since they have a magic number for file type you can construct an ObjectInput stream that acts just like a standard Searilization based ObjectInput stream that Java uses. From a programmer perspective this should be a lot more user friendly since you basically apply what you do with Java but now for Blizzard defined objects and it works.

For people doing MPQ support you need to use the FileSystems API. This is so that browsing a MPQ is no different than a normal file system and does not even need the using programs to link the classes. The service provider interface of the FileSystems API will handle integration in an abstract way so that all MPQ support needs is you to bundle the JAR file for it to work.

 

[edo494]

Linux also comes with GCC pre-installed on pretty much every distro now, so you can compile your standard-compilant C++ on any linux you can dream of too(at very least cc is present, so C can be compiled, but with cc you can compile gcc and then you can compile c++ too).

Im just trying to make obvious that the argument of "java can run on much more than C/C++" is plainly wrong

 

[Dr Super Good]

Im just trying to make obvious that the argument of "java can run on much more than C/C++" is plainly wrong

That is not being argued. What is being argued is that creating and running Java is a lot more easy than C/C++ for a third party program.

Nothing stops people installing MSVC Community on Windows (it is free after all) and building a C++ program (if the program supports it which many do not and require hours of setup time instead). However this time consuming and requires some programming knowledge to do so. GCC has Windows builds but does not work well as it is aimed for Linux and again is another dependency needed. Even on Linux I find myself having to install GCC and other such programs because the version that comes bundled is often out of date or not bundled at all which requires me use complex command line arguments and administrator commands to even install the thing to install something else.

If you are a computer illiterate browsing pre-made builds even choosing the right one for you is difficult. Java offers you a "1 build for all" solution to the problem which is what people need.

People arguing that C/C++ is better than Java are doing so for the wrong reasons. With real-time performance, optimization and OS compatibility I agree C/C++ > Jave. For usability, reliability and ease of development I disagree as Java > C/C++.

 

[edo494]

Opening program, loading files into it, hitting f5 is not a lot of work, but ok.

Reliability? How many times has been Java patched because of critical vulnerabilities? And how many C++?

Ease of development is 100% subjective, as is this debate off topic

 

[Ezekiel12]

The optimum for this whole project would be a work environment similar to https://dolphin-emu.org/?nocr=true , with a designated subforum, chat and news on progress. Make everything as functional as possible with low coupling and extensive documentation, have a prior brainstorm and design phase how things should be implemented, write some coding guidelines and do the ground work first (file format specifications, reading/writing object and .slk files, etc), only release milestone versions, only use things that still work/are used in 10+ years.
End of 101 braindump..

 

[Dr Super Good]

Reliability? How many times has been Java patched because of critical vulnerabilities?

A lot which is why it is so safe.

And how many C++?

At least once and that was for providing developers with bounded string operations to prevent buffer overflows. Even then people still use the unsafe unbounded operations as the revision has not been fully adopted.

Next to that security falls on the programmer rather than the language. Where as Java tries to force safe practices, C/C++ allows programmers to readily do the most stupid and unsafe practices ever. In C/C++ nothing stops you from accessing any memory anywhere with a pointer. Of course usually this generates a segmentation fault/security exception but when it does not you could be changing or reading anything.

 

[Dr Super Good]

The optimum for this whole project would be a work environment similar to https://dolphin-emu.org/?nocr=true , with a designated subforum, chat and news on progress. Make everything as functional as possible with low coupling and extensive documentation, have a prior brainstorm and design phase how things should be implemented, write some coding guidelines and do the ground work first (file format specifications, reading/writing object and .slk files, etc), only release milestone versions, only use things that still work/are used in 10+ years.
End of 101 braindump..

I fully agree. However the problems is that who will take charge? With all the Java implementations I have seen there have always been problems because of a lack of planning or knowledge by the programmer.

For example a Java MPQ library should come as a FileSystem object. Manipulating it should be like all other FileSystem objects and natively supported by the nio libraries. However until now people (even myself due to ignorance at the time) produce libraries that require explicit coupling to work. Sure they work but they do so in a way completely at a tangent to how they should. Why re-invent a file system API when one universally accepted and natively compatible one already exists?

Additionally why have specific I/O methods such as a read constructor/method or write method for a specific game object and expect the programmer to use them correctly? Since the files all have magic numbers why not create an ObjectInput/ObjectOutput stream. This way using such objects is as simple as standard Java serialization with the only difference being a different type of stream is used.

// read object
ObjectInput in = new BlizzardInputStream(src);
Object o = in.readObject();
in.close();

// process object
if( !o instanceof TerrainMesh ) return;
TerrainMesh mesh = (TerrainMesh)o;

// do stuff to object
mesh.Resize(-32, 32, -32, 32);

 

[Ezekiel12]

That is exactly the kind of "widsom" (i.e. experience) this projects needs, and a lot more of it in other places ofc. The project does not have to be perfect though. Nobody gets everything right and we for sure don't have to. But such basic architecture decisions should be made by a knowledgeable person, as changing things later there is very troublesome.

However the problems is that who will take charge?

Hardest question as usual. I for myself am willing to proofread and make suggestions but for sure not take the lead (not that I could anyways).
Patreon for the lead guy =)?

 

[Retera]

Reliability? How many times has been Java patched because of critical vulnerabilities? And how many C++?

This is a really weird argument to me. Like I said before, I was taught Java as my first language after JASS and thus am biased towards it. When I think about the differences between C++ and Java, I'm struck by how they are almost identical in general use cases. You make classes, variables are all assigned a type, etc. I like to think that C++ is the more traditional language where the programmers are very similar to myself except they just were taught to use more * symbols and memorized names such as "push_back", "cstdlib" and the like, instead of "add" or pressing the CTRL+SHIFT+O button to automatically import all needed libraries.

But, these more traditional fellows benefit from the fact that the wiring experts who put the computers together think they're the smarter ones, and tailor the core of the computer design towards them.

Going back to that argument up there, the complaint would appear to be that the language possesses 'insecurity' which is ultimately silly because I think what those patches were for was issues such as Applets in browsers, which nobody uses anymore anyway. If I run a program on any operating system that was written in C, C++, Java, Python, or anything really, security falls to the user account system of the operating system, if any security exists at all, to stop the program from causing trouble. No language syntactical patch is going to change that.

If you're on Windows, any of those languages could still produce a file such as 'C:\Program' which will still force a reinstall of the operating system from my experience due to the number of drivers and administrative programs that assume the shortcut 'C:\PROGRA~1' will go to the 'Program Files' folder, and nowhere else. Seriously. When I accidentally used bad syntax in a .bat file and did this to myself, a popup window came onscreen telling me that I should rename this file named 'C:\Program', and by that point Windows was already so broken that I could not successfully press OK on the popup.

So returning to the question of what language to use, I'm really just left wishing that the divide between C++ and Java did not exist. For the most part, at the core of what the programmer is doing, the guy writing C++ and the guy writing Java code are writing about the same exact thing, with a few small keyword and concept differences, and I almost wish there was a way to just express the classes of Warcraft objects in a language agnostic way then write a script to add the pointer asterisks for C++ or the Java specific stuff like 'final' instead of 'const' or 'import' instead of '#include'.

It's a difficult but deceptively important issue, and the more time we spend thinking about it the less time we spend developing content.

Edit: I am on a broken terrible computer right now where things like downloading and running the EXE of Git for windows magically doesn't work, so after some disgruntling amount of difficulty trying to make a github repo for the JWC3 Java project I started half a year or so ago I am attaching a zip to this post. I definitely feel like that's a sin against programming, but it'll give other people a chance to see some of the code I mentioned. I just really need to make some time to reinstall my OS for the third time this summer because it's Windows. Maybe it shouldn't be Windows next time.

Another Edit: Overcame computer and made a github repo:
https://github.com/Retera/JWC3

But I'm not sure it should be anything more than a temporary thing. I could really see a merge with OgreLord's repo being a good idea, I just have to make more spare time for looking at that.

 

[Dr Super Good]

So returning to the question of what language to use, I'm really just left wishing that the divide between C++ and Java did not exist. For the most part, at the core of what the programmer is doing, the guy writing C++ and the guy writing Java code are writing about the same exact thing, with a few small keyword and concept differences, and I almost wish there was a way to just express the classes of Warcraft objects in a language agnostic way then write a script to add the pointer asterisks for C++ or the Java specific stuff like 'final' instead of 'const' or 'import' instead of '#include'

Things just are not that simple. Java is like a smart guy in the class while C++ is like a calculator. Both will give you correct answers but the way they do it is completely different.

Java is very easy to use. Creating an object is done with the new statement. Objects with no references are automatically garbage collected. Due to the object handle abstraction memory fragmentation is no concern as garbage collection automatically defragments. Everything is bound checked implicitly and produces nice traceable exceptions when something goes badly wrong. You also have a nice toolbox of standard libraries available to you which will work everywhere.

C++ is not very easy to use. Sure one can still "new" an object but that resorts to malloc style allocation one can also place them on the stack as a local variable. All object life cycles need to be manually managed unless you implement various recycling systems in the form of pointers. Memory fragmentation can become a problem because you use direct references to virtual memory addresses and not object handles. Nothing is bounds checked unless you explicitly make it so and if something does go badly wrong chances are you will see some OS level process crash message. Of course you do have a basic toolbox of standard libraries but any advanced functionality will demand OS specific coupling.

The result is that Java will execute slightly slower and get the job done in a more easy to use or reliable way. C/C+ on the other hand will execute slightly faster and get the job done in a slightly less easy to use or reliable way.

 

[edo494]

Well I was just curious why Java, I dont mind it being in java, if it works its fine by my standards.

I was arguing with DSG for what he said, not necessarily with you to the point you would have to defend security

Just one addition, you dont have to use pointers at all in C++ thanks to smart pointers.

Java had some deep problems with GC before tho(you fire thread with null function callback or something, it does not get collected at all, and you leak like shit).

WIthout GC, you can however use nice RAII model, because you have deterministic object lifetime scope, and you always know when the object is not going to live anymore(rather, be functional or not UB).

Yes C++ has really really lacking Standard library, Again, I wouldnt say C++ or Java is more or less reliable than the other, it really boils down to the coder, not the language

 

[Dr Super Good]

Again, I wouldnt say C++ or Java is more or less reliable than the other, it really boils down to the coder, not the language

It actually boils down to how easy it is to fix an error in a program of the language rather than programming in the language itself.

In Java you would get nice "null pointer exception" or "index out of bound" exception with a stack trace where it happened. This alone is often enough to find the cause of an error since you can see where the invalid value is coming from and work to fix it.

C/C++ on the other hand you get either strange behaviour or you get a process crash which is completely meaningless by itself. Obviously you could run in debug build to find the cause, however this does not help diagnose the problem for a user since they are using a release build. Instead you end up with cryptic dumps such as produced by WC3 which can help track down the cause but might not (after all a lot of structure is lost by optimization).

I have two examples of this to support my case. First is a Java utility that I had to use as part of my degree program written by the University. It suffered from a Serialization error (class not Serializable). Using the source code I could easily find the troublesome class and fix it. The second example is an open source transport simulator game I was playing where multiplayer sessions were spontaneously crashing with a stack overflow error. All attempts to re-create this error in a debugger failed as it appeared to be sporadic and get worse as the game progressed. Eventually I found the cause (after about 15-30 minutes of brute force simulation) to be cyclic traverse of objects trying to find if a car could pull out into intersections which formed a loop (turning circle). Where as in Java this would have been spotted almost instantly by giving you a stack trace of the exception, in C/C++ more time was needed because of the varying builds involved and the lack of detailed information about the error (next to "stack overflow").

•Pure-Java BLP Parser to convenient classes such as BufferedImage

This must be implemented as part of the Java imaging framework via various image processor provider classes. This way you can treat blp files like they were jpeg or other standard file formats with the same API. Care has to be taken for WoW blp files which can use real-time compatible compression which are supported by graphics APIs (converting them to a standard image buffer will lose this property).

 

[edo494]

maybe its just you dont know how to debug C++ code properly.

 

[Barade]

I think these discussions of programming languages are quite useless.

There are other qualities which are more important for this. If you start this from scratch and you find enough people or invest very much time it might work but you could have joined my project instead where much of the stuff is already working instead of starting everything from the scratch.

 

[moyackx]

Agreed. The important thing here is to get good enough tools for speed up the WC3 mapping. let's try to not reinventing the wheel, and support already running projects.

Despite the love we could have for certain OS (I love linux too with all my heart), we must be clear that the key platforms of WC3 are Windows and Mac. So, C++ or java could do the job that we need.

@Barade: I think it would be great that you release a WC3lib DLL for this two platforms, this in order to develop interfaces for WC3 map development. This IMO, will boost your project in the community. I will use this in JNGP at once.

 

[Dr Super Good]

I think it would be great that you release a WC3lib DLL for this two platforms

Mac does not use DLLs. It has their own library format. Same applies to Linux with yet another different library format. One could even argue if they should be provided as libraries or rather linked directly into the application.

 

[Napoqe]

Respect you. However, I think I have to redo the Warcraft 3, ask a permission to Blizzard, and verify up allowed the enterprise. Or do a project from the 0 with ideas of World Editor, but without violating copyright rights. A game even with new options and fewer limitations that the Warcraft 3, and I think the success that do, is by cause of the map editor that created the Dota think, etc...

Really unique editor powerful that I know, that is easy (for me) to make a map different of default (PG). Not needing to know programming, design...

Remembering up make a project from the 0. Of to sell, if not violate copyright rights.

 

[Chaosy]

Because Java is one of the best multi-platform languages you can get. Unlike C/C++ where you need to re-compile for every platform (not user friendly) with Java the same program will work on all platforms.

Is that really relevant when it comes to wc3? I mean technically it only supports computers and Windows specifically?
I might be wrong here, don't be too mad if so.

OT: Sounds pretty good to me. I'd love an improved GUI. But I wont get my hopes up before I see it.

 

[Dr Super Good]

Is that really relevant when it comes to wc3?

Yes.

I mean technically it only supports computers and Windows specifically?

Except it does not only support such operating systems. There is old (probably not working on new systems) Mac version of WC3. Additionally nothing stops people running WC3 on both Mac and Linux systems with x86 processors via simulated Windows environments like Wine (which do so quite well as the DirectX8 API is well simulated by now).

Let us not forget the whole host of different Windows systems that can run WC3. Should it target Windows 10 only? Or should it be compatible with Windows NT? Or maybe people might want a 64 bit build (since why not). Again different builds for the same Windows platform.

Java has the advantage of abstracting platform away. Hence 1 build could be rolled out to all systems. This is one of the big reasons people use Java so much.

It is amazing how badly documented the w3e format is, specifically when it comes to "detail". WC3C describes this as some unexplored member responsible for ground tile variants. However from my rough experimentation only the first 5 bits (32 different values) are for ground variation. The upper 3 bits seem to be some kind of flag system for cliff variation. In a single row of cliffs no more than 1 bit out of the 3 is set with no bits being set also being an acceptable value.

 

[Ardenian]

I hope the new tool you would like to create features an import of .obj or .3ds or even .blend, too.

Then finally all modellers using Blender ( a free and non-register modelling tool) would be able to import their stuff to Warcraft without having to ask kind people on Hive to convert them with Gmax.

 

[Dr Super Good]

Here is an example of how my new w3e file interface is shaping up.

		System.out.println("reading terrain");
		TerrainMesh terrain = null;
		try (ObjectInput reader = new BlizzardInputStream(new BufferedInputStream(
						Files.newInputStream(FileSystems.getDefault().getPath("G:\\WC3 maps\\HeighTestWork\\war3map.w3e"))))) {
			Object o = reader.readObject();
			if( !(o instanceof TerrainMesh) ) System.out.println("not a terrain mesh file");
			terrain = (TerrainMesh)o;
		} catch (IOException e) {
			System.out.println("something bad happened reading the file");
			e.printStackTrace();
		} catch (ClassNotFoundException e) {
			System.out.println("something bad happened resolving the class");
			e.printStackTrace();
		}
		System.out.println("performing transfrom");
		terrain.simpleTransform( V -> {
			V.setGroundHeight(0);
			V.setWaterHeight(2000);
			V.setFlags(TerrainMesh.TerrainVertex.FLAG_VERTEX_WATER);
			V.clearFlags(TerrainMesh.TerrainVertex.FLAG_VERTEX_BOARDER);
			V.setGroundDetail(0);
			V.setGroundType(0);
			V.setCliffLevel(2);
			V.setCliffType(0x0F);
			V.setCliffDetail(0);
		});
		terrain.simpleTransform( V -> {
			V.setGroundHeight(512);
			V.setWaterHeight(0);
		}, new TerrainMesh.VertexPosition(15, 15), new TerrainMesh.VertexPosition(17, 17));
		System.out.println("writing terrain");
		try (ObjectOutput writer = new BlizzardOutputStream(new BufferedOutputStream(
						Files.newOutputStream(FileSystems.getDefault().getPath("G:\\WC3 maps\\HeighTestWork\\war3map2.w3e"))))) {
			writer.writeObject(terrain);
		} catch (IOException e) {
			System.out.println("something bad happened writing the file");
			e.printStackTrace();
		}
		System.out.println("done");

Both reading and writing have been tested and appear working. As shown above I am using the new functional support of JAVA 8 allowing me to apply operations to entire areas of a TerrainMesh in a single call. These operations could be cached to be applied to multiple areas. If it is faster I am not sure but it certainly is quite neat.

Mind you most of the time was spent creating the stream support for this.

 

[WaterKnight]

Um, why is this in Town Hall? Totally missed it. http://www.hiveworkshop.com/forums/warcraft-editing-tools-277/about-custom-editor-concepts-227711/

I am also working on wrappers for the formats and tools and think it would be better to provide stackable solutions rather than a new full integrated editor because the wc3 specification is so vast and we never agree on one thing. I want you to consider that functionality could be user script-controlled for flexibility, so sandboxing/code embedding is an issue. Most file specs are unproblematic but quite tedious to cover all the different fields, partially have different versions and you often can/do not want to edit them standalone, e.g. the object data is a combination of slks, txt profiles and mods.

 

[Dr Super Good]

Most file specs are unproblematic but quite tedious to cover all the different fields, partially have different versions and you often can/do not want to edit them standalone, e.g. the object data is a combination of slks, txt profiles and mods.

You create a higher level object which links the functionality of several low level objects. For example you would want a terrain management class which deals with the shadow map, the pathing map, the terrain mesh and the various pre-placed object files. In such a class you have resize methods which then resize all 3 map layers and translates the position of all objects.

 

[Napoqe]

Compensates try improve Warcraft 3 even? I think the revolutionary game, however, several problems. As the chart, place to play, limitations, the advantages of scripts to do some functions, even I like triggers and the editor of objects.

If you and / or Blizzard improve the game and the editor, with graphics, etc ... Depending of the situation of the editor of the custom modes of Dota 2, I come back to play and try to create and develop maps in Warcraft 3.

 

[oger-lord]

Here is an example of how my new w3e file interface is shaping up.

Nice, atm my W3E is a mess and I was about the remake it.
Did you find out which information are in the texture details byte?
I think the first 3 bits are something for cliffs and the last 5 bits are the texture variation.
But I dont know how the cliff informations are working.

Here is my war3map.shd with the possibility to convert the shadow map to a BufferedImage and vice versa.
https://github.com/OgerLord/WcDataLibrary/blob/master/src/de/wc3data/map/ShadowMap.java
Maybe I should add a function to fill a rectangle and methods for merging two ShadowMaps.

 

[WaterKnight]

Yes, that's what I meant by stacked solution. To emphasize on the point I wanted to make this morning: In vJass there is textmacro. While it is often seen as language-levering/patching flaws, there are reasons and use cases not to integrate specific mechanisms in a language/tool or have project-bound automatisms. That's why it makes sense to offer the libraries not exclusive to the tool developers but also for the end users. Or I have already published some stuff that is script-parametrized, something difficult and dangerous like c/c++ would be nonsense at this point, at least offer the possibility of an easy binding/embedded scripting.

 

[Ezekiel12]

Here is my war3map.shd with the possibility to convert the shadow map to a BufferedImage and vice versa.

One thing that counts for everyone who contributes something wc3-hardcoded-related:
You have extensive knowledge of the file type or whatever you are coding. Write the significant part of it down with the code. This makes it so much easier for other people to understand, improve and maintain things.

In your case, comment the code with how a shadowmap file is built, every relevant bit you know about it and also what is yet unknown.

That way everything is in one place and we get a much better meter to measure progress than having some info here and there. Also if you or someone else leaves it's no big deal because the info is in there. Happened far too often that people with really large knowledge left the scene. If everything that e.g. Vexorian or PipeDream encountered would be written down we would know a lot more.

 

[Dr Super Good]

Did you find out which information are in the texture details byte?

Lower 5 bits are ground tile variation. Upper 3 bits are related to cliff variation. I am not sure how cliff variation functions currently but it looks almost flag like (no more than 1 out of 3 bits set with no bits also being valid).

I think the first 3 bits are something for cliffs and the last 5 bits are the texture variation.

Someone got a bit confused . Actually it all depends how one interprets the bytes but generally bit 0 is considered to be the least significant of a byte while bit 7 is the most significant. This is "LSB 0" numbering used for Little Endian machines and some times for Big Endian. The reason it is used is because if you expand from one to two bytes the meaning of the fist byte (as far as bit positions) does not change. Obviously with big endian machines the inverse is the case (however WC3 uses Little Endian).

But I dont know how the cliff informations are working.

Flags by the looks of it. Need more experimentation.

Here is my war3map.shd with the possibility to convert the shadow map to a BufferedImage and vice versa.

If it is a buffered image it should be part of the image service system of Java. If not you should rely on another class to convert it to an image (a sort of adapter). Also if I recall the shadow map uses bytes and not booleans. As such treating them as only boolean looses content.

One thing that counts for everyone who contributes something wc3-hardcoded-related:
You have extensive knowledge of the file type or whatever you are coding. Write the significant part of it down with the code. This makes it so much easier for other people to understand, improve and maintain things.

In your case, comment the code with how a shadowmap file is built, every relevant bit you know about it and also what is yet unknown.

That way everything is in one place and we get a much better meter to measure progress than having some info here and there. Also if you or someone else leaves it's no big deal because the info is in there. Happened far too often that people with really large knowledge left the scene. If everything that e.g. Vexorian or PipeDream encountered would be written down we would know a lot more.

Actually the file formats need a separate dedicated document to specify them. A standards document.

Commenting the standard inside the code is bad practice since it bloats the code file with internal format descriptions. The formats are standard so need a standard and the code can then reference the standard.

Anyway I leave you with this crazy terrain generated by this...

		terrain.spatialTransform( (V, P) -> {
			V.setFlags(TerrainMesh.TerrainVertex.FLAG_VERTEX_WATER);
			double phase = Math.PI * 2 * Math.sqrt(Math.pow(P.x - 240, 2) + Math.pow(P.y - 240, 2)) / 24;
			V.setGroundHeight((short) (Math.cos(phase) * 2048));
			V.setWaterHeight((short) (Math.cos(phase + Math.PI) * 2048 + 358.4));
		});

 

[Barade]

Is this still being developed? The screenshots look quite promising.

 

[Retera]

I've certainly been thinking about it. I downloaded the latest copy of your wc3libs sourcecode again yesterday and was asking myself about whether to restart from there and find myself a decent windows C++ compiler (my experience with C++ was using g++ on course projects in lab machines in linux) or to just pull some needed information and build even better Java libraries.

Classes have been keeping me annoyingly busy, but I do sort of have to care about them.

 

[Barade]

At the moment I do not have much time. Java is certainly better for portability (easier) and probably easier to understand for most people but at least I have state where some parts work (an initial object editor, an initial trigger and MPQ editor etc.).

I tried to create a windows build for wc3lib but failed. It didn't work with CMake on Windows (which it should actually since CMake is supposed to be cross-platform) and on Linux I am missing some libraries which I would have to cross compile with the MinGw.