MDX/M3 Optimizer

[Moderator]

21:28, 13th Nov 2013
PurgeandFire: Approved. A great step-up from mdx squisher with its extra options. Useful for any sc2 modder as well.

 

[Statharas]

Baneling compressed, 72.6kb saved

 

[PurgeandFire]

Haven't tested it but it sounds good.

Does this delete unused bones (ones that aren't connected with vertices)? edit: nvm that may not be the best idea.

And I haven't checked if it does this already, but does this support batch conversions (e.g. input a folder)? If so, it would be a nice addition and make it far more preferable to Guesst's mdx squisher.

 

[GhostWolf]

You can input as many files as you want.
I thought about adding directories as valid inputs, but it would require adding some filesystem library. I'll see if it isn't too annoying.

/Edit
Directories are now valid inputs.
It will search for MDX/M3 files recursively in any directory you specify (meaning it will look also in inner directories). You can grab a directory into it like you can do with files.

Also added a Linux executable.

 

[GhostWolf]

Rewrote it all in C++, so it should be much faster (and the executables are smaller!).

Also removed the chunk table from being printed, since there is nothing to print for M3 files (their size never changes). Should I put it for MDX files anyway?

/Edit
Added a Mac executable, thanks to Magtheridon96 for compiling it.

 

[PurgeandFire]

I tried running it on a mac for a directory, and it gave "Oops, Deadeye is not a valid mdx file".

When I ran it directly on the mdx file, it worked. What method do you use to detect if it is a valid mdx file? I assume you would read the MDX header, but since it works for the direct input and not the directory input I'm guessing it is just a mistake in the directory-related code.

Code I ran:

exec "$HOME/Downloads/copt/copt_mac" -p 16 -t 0.001 -l true "$HOME/Downloads/UArcherLPFK"

Here is the model in case it is relevant (although it likely isn't):
http://www.hiveworkshop.com/forums/models-530/deadeye-113615/

 

[GhostWolf]

Files without mdx/m3 extensions are blatantly ignored, and for files with either, the first 4 bytes are checked for the magic identifiers (MDLX / MD34).

I can't test it on Mac because I don't have one. Magtheridon96 did say he made a few changes to make it compile there, though I don't think it's anything that should make the code fail.
Can't really fix this without being able to compile and run it.
Does dropping a directory on the executable result in the same error?

As a side note, specifying -l makes it true, the "true" after it becomes a file input that is ignored because it doesn't have mdx/m3 as an extension.

 

[PurgeandFire]

Do you have the source on a public repo e.g. github/launchpad?

edit: Off the top of my head, perhaps it is an endianness issue. OS X is big endian. Maybe Mag made the conversion for direct input reading function but not for the directory reading function? It might explain why the same mdx file is accepted as direct input but not accepted when read from a directory.

 

[GhostWolf]

Macs are big endian?
The code only handles small endian, so even if it does work, it will probably mess the models.

Here's the source.

The older code printed the "Oops" message also when it failed to open the file, so that might be the real reason you are getting it.

 

[PurgeandFire]

Yeah, could be. Perhaps you should remove copt_mac from it until it is updated. I'll run some quick tests on my Windows comp and then it should be approvable.

If you're still interested in having a mac-friendly version, I'd do something like this:

#ifdef __BIG_ENDIAN__

    #ifdef __APPLE__ && __MACH__

        #include <CoreFoundation/CFByteOrder.h>

        #define swapInt16(x) CFSwapInt16HostToLittle(x)
        #define swapInt32(x) CFSwapInt32HostToLittle(x)

    #else
        #error unsupported operating system
    #endif

#else
	
    #define swapInt16(x) (x)
    #define swapInt32(x) (x)

#endif

And write functions to swap the struct members after you've read them from the file, or however you want to do it really. Although, it obviously will be difficult to implement without having a Mac yourself. Pester Mag about it.

 

[PurgeandFire]

Sorry for the double post, but the directory input isn't working for me on Windows either. It just shows that all the mdx files are invalid.

cd "C:/Program Files/Warcraft III/Tools"

copt_win -p 16 -t 0.001 "%USERPROFILE%\Downloads\Models\Plants"

Same thing happens when I drag and drop the folder onto the exe. I tried the tool on these:
http://www.wc3c.net/showthread.php?t=82781
(not for any particular reason, just as a quick test).

 

[GhostWolf]

Oh, my bad!
I kept testing with the "." path, so in fact I wasn't testing directories at all.
Seems like I completely forgot to concatenate them to the path.

The models you linked also don't have a sequence chunk, which caused a crash.
I didn't know models without sequences are even valid, fixed the code (it ignores the keyframe optimization if there are no sequences).

Thanks!

I'll compile for Linux soon, and we'll see about Mac...

Added the Linux binary.

 

[PurgeandFire]

Perfect. Works like a charm now.

One last issue, I tested it on this pack:
http://www.wc3c.net/showthread.php?t=87180
And it crashed/did nothing.

 

[GhostWolf]

I was doing a case-sensitive check for the extension against "m3" and "mdx", changed it to case-insensitive.

/Edit
Added support for big-endian machines (as few of them as there are).
Thanks to PurgeandFire for compiling the new Mac executable.

 

[Deleted member 219079]

It cut ~260 kilobytes from my map, which had 50 custom models. Pretty nice tool, I will include you in the credits

 

[Ice Snake]

Hi GhostWolf, I created a GUI version for your software, here it is:

https://www.dropbox.com/s/jossftpc6ieiopc/copt_GUI.rar

Image

Just "Browse..." and select MDX, MDL file and then "Process"

It is simple with only 7-8 lines of code but it (may) make(s) life easier..

 

[GhostWolf]

You should probably post the source.

I don't like GUIs for these sort of things, since grabbing files/folders into the executable is faster than opening a select file menu, but if people want to use it, go for it.

 

[Ice Snake]

You should probably post the source.

I don't like GUIs for these sort of things, since grabbing files/folders into the executable is faster than opening a select file menu, but if people want to use it, go for it.

Okay, I will improve it and post the source code, please wait

Okay, I reuploaded it includes source code, this update support Drag and Drop file to the GUI: https://www.dropbox.com/s/jossftpc6ieiopc/copt_GUI.rar

 

[Warcraft_3]

Okay, I will improve it and post the source code, please wait

Okay, I reuploaded it includes source code, this update support Drag and Drop file to the GUI: https://www.dropbox.com/s/jossftpc6ieiopc/copt_GUI.rar

Thanks alot its a very good tool

does this tool also work for blp ?

 

[jonbon29]

Great and very use tool.

 

[Kakerate]

Hmm..Perhaps the models i tried to optimize have already been optimized, because I used this and Mdx Squisher and it reduced the size of my map by 1194 bytes

 

[GhostWolf]

does this tool also work for blp ?

Does it say BLP anywhere in the description or title?

Hmm..Perhaps the models i tried to optimize have already been optimized, because I used this and Mdx Squisher and it reduced the size of my map by 1194 bytes

A big part of this optimizer is essentially the same thing as Mdx Squisher (but this tool does it a little better).
You can also experiment with the keyframe threshold and interpolations if you'd like, where the latter can cause big size reductions (at the price of quality of course) for some models.
It's all in the description.

 

[Ironside]

Is there a way to make it output the statistics about how much things got decreased like in the screenshot? Because when I tried it, it merely does its thing without the output.

 

[GhostWolf]

I removed the output because there was nothing to actually write about M3 models (at first they also had keyframes removed, but it turned out Starcraft 2 doesn't allow that).
Either way, though, the statistics were only for keyframes and changing interpolations, not for the floating point optimization, since it doesn't actually change the file size.
You can just look at the difference between the input file and the output file, but if you want to see how big it will be in an actual map (MPQ), the only way is to put it in one.

 

[Kakerate]

You can also experiment with the keyframe threshold and interpolations if you'd like, where the latter can cause big size reductions (at the price of quality of course) for some models.

Yeah I was going to try to do that, however I don't know how to change the settings. Tried running it through cmd or whatever you instructed in your desc. but I don't know how to. I typed start + the path and name of executable and it just closes immediately. I had to use someone's GUI they made of this. So how do you run it without using default settings?

 

[GhostWolf]

Run the optimizer with the console to get instructions:

copt_win

For example, to force linear interpolations:

copt_win -l some_mdx_file.mdx

You can have multiple inputs, where every input can be an MDX file, or a folder (in which case all MDX files in it, and recursively in folders in it, will be taken as inputs).

E.g.

copt_win some_mdx_file.mdx folder_with_lots_of_mdx_files

 

[GhostWolf]

You could always create a batch file, if it's such a nuisance.
E.g.:

copt_win -l %1

Then grab files/folders into it as usual.

 

[RanchBurner]

Some .m3 files I try to import into Blender won't, due to a number of "unknown sections". A few others don't seem to import correctly and look mangled. I was hoping this tool would fix all that, it didn't.

 

[Asssssvi]

Hay dude! I tried to change the value of the "Floating point precision" option to " 32 " to disable it.
Using the " bat " file, but I don't get anything

copt_win -p 32

How can I make a working bat file with the parameters I need so that I can also drag and drop them as in "EXE"?

P.S. if you add the path to the file, everything works.

copt_win -p 32 "\%USERPROFILE%\Desktop\MDXM3 Optimizer\Sorcerer.mdx

Edit: Hell, I just had to delete the file name. Everything works!

copt_win -p 32 "\%USERPROFILE%\Desktop\MDXM3 Optimizer

Thank you for the program, it helped me a lot.

Edit: Provided that the file to be optimized is located in the folder where the program and the "bat" file are installed.

 

[Technopig]

So if i was to use this and input the directory of my current maps War3mapimported folder - would it optimise all the MDX files in that folder and all its sub folders?? Also just so i understand :

-Will this make my total map size smaller afterward?
-Does this effect the actual visual appearance of any of the models in the map when playing?
-All im trying to do is sort of optimise the imported MDX, Audio and BLP files in the map to reduce total size - i have used BLP Lab for the BLPs and compressed the audio files in another programme successfully, is this programme the right thing to use for the MDX files or am i looking at this wrong?

thanks in advance

 

[PerPlumeShake]

i can't open it