• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!
  • It's time for the first HD Modeling Contest of 2024. Join the theme discussion for Hive's HD Modeling Contest #6! Click here to post your idea!

Optimizer problems.

Status
Not open for further replies.
Level 23
Joined
Apr 16, 2012
Messages
4,041
this tool was never made to protect maps, it is even said inside the readme. It is only a byproduct of removing several files from your map, which editor despretly needs. Maybe you can try checking the "Remove as much WE-only data as possible", it should at least decrease the size a little bit. Maybe try running the optimizer without widgetizer. Always keep backup, if your map goes bananas(aka, corrupted) or if it gets protected(it does on pretty much any option selection) you will be map-less
 
Level 8
Joined
Dec 11, 2012
Messages
412
this tool was never made to protect maps, it is even said inside the readme. It is only a byproduct of removing several files from your map, which editor despretly needs. Maybe you can try checking the "Remove as much WE-only data as possible", it should at least decrease the size a little bit. Maybe try running the optimizer without widgetizer. Always keep backup, if your map goes bananas(aka, corrupted) or if it gets protected(it does on pretty much any option selection) you will be map-less

Well, no, it doesn't work at all...

When I disable the protection thing, and New Buffer Size, ZLib Deflate and Maintain (Attributes) and start optimizing, my map only gts 0.22% smalled, which i like 70 Kilobytes, and its totally useless.

When I read what Vexorian or whatever said, map has chances to get 20-80% smaller and to me it doesn't get sh!t.

I do now know... What do you mean by "run without widgetizer"?
 
Level 8
Joined
Dec 11, 2012
Messages
412
ah, ok you should never turn off ZLib Deflate, because that does compression on each file(some files are compressed more than others, for instance Jass script can go as low as 20% of original size)

Should I jass off some gui triggers if I understood well?

So, I should use usual options and ignore "New buffer size" and "maintain" options? Still makes it bigger.

EDIT: ... and how do I check if it runs on widgetizer so I can cut that down???
 
Level 23
Joined
Apr 16, 2012
Messages
4,041
widgetizer is another program as far as I know. And no you dont need to convert GUI to Jass, the ZLib Deflate is complicated algorithm that compresses files regardless of what they are, it will compress text files(map scripts), as well as models(.mdx), but it depends on how the file looks and also how the algorithm works
 
Level 8
Joined
Dec 11, 2012
Messages
412
Look.... I've tried everything... it only gets bigger...

I know this sounds retarded, but it could be only solution...

If you're so sure it can be compressed, I will send it to you, you try to do it by yourself and send it back (unprotected)...

If its not problem, let me know... I really need this.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
Can you give me the map before and after optimization (the one that ends up bigger than what you start with). I will run it through a MPQ editor and see what is actually being made larger. Knowing where compression is failing is key to try and make it smaller. It could be that an error results in a single file becoming huge undoing all the compression gains in other files. The solution in this case would be to simply use the un-compressed version of that file with the compressed versions of all other files.
 
Level 8
Joined
Dec 11, 2012
Messages
412
Can you give me the map before and after optimization (the one that ends up bigger than what you start with). I will run it through a MPQ editor and see what is actually being made larger. Knowing where compression is failing is key to try and make it smaller. It could be that an error results in a single file becoming huge undoing all the compression gains in other files. The solution in this case would be to simply use the un-compressed version of that file with the compressed versions of all other files.

Okay. I'll edit the post when I get links.

Nonoptimized - http://www.mediafire.com/download/awagcr42dai8nzr/Raid+Finder+-+Siege+of+Wyrmrest+Temple+-+DS.w3x

Optimized (protected) - http://www.mediafire.com/download/ka7lni819ccwemt/Raid+Finder+-+Siege+of+Wyrmrest+Temple+-+DSR.w3x

The J thing file - http://www.mediafire.com/download/ac31j4orp72aao1/Raid+Finder+-+Siege+of+Wyrmrest+Temple+-+DSR.w3x.j
 
Last edited:

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
The problem is related to all the imported .wav files you are using. They are being compressed less efficiently after optimization. I am guessing this is because they are using the purpose built lossy audio compression MPQ archives support for wav files when added with World Edit where as the optimizer just blanket compresses them with ZLib Deflate that is not designed for audio compression.

You have two options.
1. Convert all .wav into .mp3 since .mp3 are already compressed separately to the MPQ compression so will not change size during optimization.
2. After optimization import all sounds using an external MPQ editor that allows you to specify them to be compressed with "Huffman tree compression combined with ADPCM". MPQEDIT does not allow this as far as I can tell (except maybe in command line).

1 May be the easiest.
 
Level 8
Joined
Dec 11, 2012
Messages
412
The problem is related to all the imported .wav files you are using. They are being compressed less efficiently after optimization. I am guessing this is because they are using the purpose built lossy audio compression MPQ archives support for wav files when added with World Edit where as the optimizer just blanket compresses them with ZLib Deflate that is not designed for audio compression.

You have two options.
1. Convert all .wav into .mp3 since .mp3 are already compressed separately to the MPQ compression so will not change size during optimization.
2. After optimization import all sounds using an external MPQ editor that allows you to specify them to be compressed with "Huffman tree compression combined with ADPCM". MPQEDIT does not allow this as far as I can tell (except maybe in command line).

1 May be the easiest.

... 1 I got no idea how to compress sounds and convert them into .mp3...

... but, if i export the sounds and delete them, 'then reimport, will it do the trick?

... or you can explain me this method of converting them into mp3 and optimize idk how, you tell...
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
... but, if i export the sounds and delete them, 'then reimport, will it do the trick?
No since they will be in exactly the same state before you do this as after and the optimizer will still mangle them because software is dumb (well it is only as smart as the people who program it sadly and it seems Vexorian forgot .wav files need special handling).


... or you can explain me this method of converting them into mp3 and optimize idk how, you tell...
You could try Audacity. Not sure it can produce WC3 compatible MP3s as personally I have never tried it but a glance at the feature list shows ability to import and export (with a plugin downloaded separately) in the appropriate formats. People usually avoid putting sounds into maps because they take up too much file space.
 
Level 8
Joined
Dec 11, 2012
Messages
412
I've used online Audio Format... it compresses sound and converts it, plus... size is 80% lower and sound smooth...

... will see it if works and will say...

EDIT: It actually helped but didn't save much space... anything else I can do?
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
EDIT: It actually helped but didn't save much space... anything else I can do?
So they are still larger after the map is put through the optimizer than before? This would make no sense as MP3s cannot be compressed anyway.

Or are you asking "How can I reduce my map file size?"? The only answer is to delete imports, especially sound and images as they have the highest data density and so compress the worst.
 
Level 8
Joined
Dec 11, 2012
Messages
412
So they are still larger after the map is put through the optimizer than before? This would make no sense as MP3s cannot be compressed anyway.

Or are you asking "How can I reduce my map file size?"? The only answer is to delete imports, especially sound and images as they have the highest data density and so compress the worst.

No, no no... The map is getting smaller, but only 7%... I'm not satisfied with it.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
No, no no... The map is getting smaller, but only 7%... I'm not satisfied with it.
That is because most of the data is already in a highly compressed form. It cannot be made much smaller, even with slightly better compression algorithms.

JPEG based BLPs (not palliated or native formatted blps) cannot be compressed much due to JPEG blocks already being compressed as specified by the JPEG format description. Since images are a huge source of data density this means they will always take a lot of space to store. Although wav sound files can be compressed (even if Vexorian's Optimizer breaks this) they still contain a lot of data density and in mp3 form (most compact) they will still be sizable due to the data density of audio. Models on the other hand are often easily compressed as they have quite low data density with a lot of repeating sequences, however since World Edit already compresses them the amount saved using a better compression (that WC3 still supports, even better to exist but WC3 does not support) is minimal and will save at most a kilobyte here or there.

The end result is that considering what your map contains, a 7% saving is pretty massive since most of the content in your map is already efficiently stored. Remove all the imports and you would see that Vexorian Optimizer is reducing it by 40% odd, the amount it advertises.

You only have two viable solutions.
1. Remove content. Less content obviously requires less space. Are all those sounds really necessary? Will people notice a skin or not? Cut back anything that has poor game play benefit for space trade off (audio taunts seem the most likely victim sadly).
2. Downsize what content you do have. Reduce data density (lower resolution textures for blps or lower sample rate/bits per sample for audio) of imports. You can get a hell of a lot of very small (low quality) imports into a map, but if they are worth it is another question.

If you really want a lot of imported content I might suggest mapping for StarCraft II as they have a much more lenient data limit on maps and no data limit on number of imports via mods (mods have a data limit but these can be public from other people's accounts and there seems no limit to mod number).
 
Level 8
Joined
Dec 11, 2012
Messages
412
Giving up things isn't option anymore. I've gave up every model that is useless and some that I was using. Giving up on hero models would suck up 6 MB.... I could swallow that, but it won't be WoW map anymore. Giving up on 10 models max won't work either.

All of the sounds are really neccesary because I can't script a boss without sound files, wtf.

For now, I'm using 25 3D models and all of them are compressed, all of the sounds are compressed... I got nothing left to do... but to make this Singleplayer and all of my effort for nothing.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
All of the sounds are really neccesary because I can't script a boss without sound files, wtf.
I have never heard of this problem in WC3 before. In what way are they required for the functioning of the script? Does the thread crash without them? As far as I was aware sounds were not deterministic so did not alter trigger operation at all next to some handle creations.
For now, I'm using 25 3D models and all of them are compressed, all of the sounds are compressed... I got nothing left to do... but to make this Singleplayer and all of my effort for nothing.
Or remove content. Those are your only options. You should never have designed the map to depend on so many resources in the first place as the 8 MB map file size limit is well known in every mapping community and has been around for years. Be lucky they raised it a couple of years ago as before then it was only 4 MB (half the current limit).
 
Level 8
Joined
Dec 11, 2012
Messages
412
Mabye not... but, this is only a map...

Giving up content is like.... wtf, how do you imagine Dragon Soul raid without WoW models?

I mean, how do other people succeed with over 100 models and terrains, and sounds? I don0t get it.

I can count up a few projects like that if you want .
 
Level 29
Joined
Jul 29, 2007
Messages
5,174
Hint: content takes memory.

Convert WAVs to MP3s and consider reducing the bitrate.
Convert TGAs to BLPs.
Check that all the BLPs use the fake JPEG compression Blizzard used in Warcraft 3.
Run a MDX compressor on your models (like this, you'd probably want to enable the force-linear option).

But, at the end of the day, your content is too big. Warcraft 3 wasn't designed for models on the size range of 1-3MB, or textures on the size range of 700KB, or to import a whole other soundtrack.
Other people imported hundreds of models because all of them together take less than your 3 biggest models, not to mention all the other ridiculously large (relatively speaking) files.
 
Level 8
Joined
Dec 11, 2012
Messages
412
Hint: content takes memory.

Convert WAVs to MP3s and consider reducing the bitrate.
Convert TGAs to BLPs.
Check that all the BLPs use the fake JPEG compression Blizzard used in Warcraft 3.
Run a MDX compressor on your models (like this, you'd probably want to enable the force-linear option).

But, at the end of the day, your content is too big. Warcraft 3 wasn't designed for models on the size range of 1-3MB, or textures on the size range of 700KB, or to import a whole other soundtrack.
Other people imported hundreds of models because all of them together take less than your 3 biggest models, not to mention all the other ridiculously large (relatively speaking) files.

Have you even checked my new version? You probably did the 90MB one, not sure 'tho...

How many times I must say "HOW DO OTHER PEOPLE MANAGE TO DO IT"... It wasn't designed for it, yet they succeed at 300 models and 7mb.

Quit talking like I dont know what I imported. Only 10% of my models have large size (which is like 2-3 models)....

I don't wanna get in discussions now... if you can't tell me any sort of resolve for my problem, link any program that can help me, why bother commenting... right now, I need help getting this down.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
I mean, how do other people succeed with over 100 models and terrains, and sounds? I don0t get it.
WC3 can have at most 16 terrain tiles. Not 100s.

Models are actually very small and compress well. You can easily get hundreds if they use internal textures. It is custom models that require custom textures that are the problem as the custom textures often compress very badly.

There is an argument from time to time about textures. Some say using JPEG compressed textures is best since that is designed for size optimization however if you use a low colour depth pelleted texture it may appear larger but after compression can actually end up smaller and look better than JPEG. It depends on how much high frequency information is in the texture since JPEG compresses by removing such information while pelleting does not (you would need to apply a blur filter manually to remove it).

Wow models also have a lot of geometry. In WC3 they are often re-mastered to use less geometry and have fewer animations to make the models smaller. A process that cannot be automated well.

As far as I am aware few maps ever use custom sounds. Those that do use very few and they are highly compressed (you can hear the distortion) and are very short. No maps use custom music except if they have a lot of space to blow (the maker just wants to waste space for some reason).

How many times I must say "HOW DO OTHER PEOPLE MANAGE TO DO IT"... It wasn't designed for it, yet they succeed at 300 models and 7mb.

Quit talking like I dont know what I imported. Only 10% of my models have large size (which is like 2-3 models)....
The models themselves usually are not the problem. It is any custom textures they use and sound as that often contains far more information. Also direct WoW port models are excessively large as they were never designed with space constraints in mind which is why people re-master them for WC3 by reducing their geometry, making them share textures which are already in game and cutting out pointless animations.

don't wanna get in discussions now... if you can't tell me any sort of resolve for my problem, link any program that can help me, why bother commenting... right now, I need help getting this down.
We are telling you the solution. You need to either cut content or substitute content which is more space efficient. Using models with in game textures instead of custom textures saves a ton of space. Using no WoW imported models and only using WC3 optimized models saves a ton of space. Converting all sounds to highly compressed MP3 or preferably using as few sounds as possible saves a ton of space.

Compression is limited by the complexity of the input data. As such doing something simple like removing noise from textures and sounds can potentially save a lot of space (JPEG and MP3 do this to some extent automatically).
 
Level 8
Joined
Dec 11, 2012
Messages
412
I would really appreciate simple english answer.

Cut content in 1 way?

To let you know instantly, deleting anymore models from map is not a solution...

What you're saying "using models with ingame textures instead of custom textures"... I don't know what you mean.

I have used audioformat.com ... I converted all .wav to .mp3 and compressed them as low as possible the site could go.

I would please if you explain it 1 more time on "Simple English" what do I need to do and how, 'cuz I'm sorta retard in english and understanding things.
 
Level 23
Joined
Oct 12, 2008
Messages
1,783
This goes for the other thread on the same subject and by the same user.

Can we all agree, that this is why you should understand something before you spout.
Garodar has made laughing stock of himself by basically dissing anyone who helps while crying on the floor like a child expecting work to be done for him.

I have no qualms with people being new.
I have no qualms with people needing help.
But do we really want to encourage further bad behaviour by giving in to him?

Solutions both concise and elaborate have been given, they have been ignored.
Excuses such as "I dont understand English" make no sense when in the same post, the people giving solutions are told to figuratively fuck-off.
What he ultimately wants is for someone to do the work for him.
And until he gets that, both the excuses and crying will continue.
 
Last edited:
Level 8
Joined
Dec 11, 2012
Messages
412
... yea ignoring all options I accepted at the end... I've used all that compresses and was asking if there are any other programs...

Oh, so its my bad I do not know english well as you people do and what you call "figuratively fuck-off" is wrong at the end. I've been told to use mdlvis, I did, blp lab, I did, mdx squisher, I did... everything in my power I had was used... other than that, I was searching for a solution and asked one final "is there anything else I can do"...

What I ultimately want is a solution, not someone to do it. By being sarcasting and cruel, you'll make no changes here, because I see internet as a online network full of disgracing and rude people such as you when it comes to this... You can post comment after comment about the same thing to other people, you can let the whole forum know I asked for optimizer and make a joke of me, but I don't care. I'm here for 1 purpose and its helping other people's project grow and make something of my old work. If I continued, I could've gone in agrument with you and start insulting you, but that's not my thing.

Look clearly, and you'll see nothing but your own point of view of how things should be, as I see it by myself... but if you see me clearly searching for something specific, then either help with that, or fuck-off as you said it perfectly. I did accept help you guys gave me at the ending in both topics but this one is merely for optimizer, not recruiter. Recheck posts and reconsider if your comment was neccessary at all. That's my 5 cents.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
You need to set data targets (how much data to spend on certain types of file). You then measure how much data each type is currently using (measure the compressed size, uncompressed is meaningless in this context). Then you work on reducing the target file types to the appropriate sizes. Here are some tips.

1. Use WC3 optimized models. All models from Hive Workshop model section that are approved should fall into this category. Avoid direct ports from WoW as they are not WC3 optimized.
2. Down sample textures (all blp files). Reducing their resolution by a half in each dimension will result in approximately a 75% reduction in file size of the blp. If the blp files are JPEG compressed you can also try raising the compression factor. Warcraft III imposes a blp restriction that they must be some power of two in dimensions or they will fail to load.
3. Down sample all speech files. 8 KHz sampling rate should be enough for speech and since you are digitally down sampling you do not need bandwidth for anti-aliasing filter. Sure it will sound like a telephone but the space saving from 44KHz is huge.
4. If none of the above are sufficient and you really cannot cut content, then you should repeat them again, higher compressions, lower resolutions and lower sample rate. Yes the content will look bad but that is your only choice next to cutting content. How else do you think old game developers got so much onto a PS1 disc?
 
Level 8
Joined
Dec 11, 2012
Messages
412
Hold on? Use hiveworkshop models? They will be in Warcraft 3 style?

Yea, I was wondering how I downspeech files to 8 KHz.

But, I don't want a bad content... I want something to look as blizzlike as possible. As far as I can tell, it doesn't so much either and cutting it out even more will ruin it completly. I dunno.
 
Level 29
Joined
Jul 29, 2007
Messages
5,174
Let's do simple math here. The limit for Battle.net Warcraft 3 maps is 8MBs last I checked.
You have 5 single models that take 8MB, and then many, many more big files.
No matter what you do, or how you compress this data, it will never reach 8MB.

I will repeat - Warcraft 3 wasn't designed for models on the size range of megabytes.
This is exactly the reason why this site frowns upon WoW rips. They are big, clunky, and basically useless for a Warcraft 3 map.

If you really want to use these models, you will have to edit them.
If you want to use the textures, you will have to reduce their resolutions.

There is no magic "compress this 27MB of binary data to 8MB" button, sorry. There is a limit to what lossless compression can get you, and that limit is too high for your content.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
Yea, I was wondering how I downspeech files to 8 KHz.
Easiest way is Fourier Transform it into frequency domain, cut out all frequencies past 8KHz (make the nyquist frequency 8KHz) and then transform back into time domain with inverse Fourier transform. The result is that of a perfect filter so there is no need for anti-aliasing filter bandwidth.

But, I don't want a bad content... I want something to look as blizzlike as possible.
Well WoW models are not an option then as they look far too modern for WC3. Also there is a huge difference between the 800 MB Blizzard had for content and the 8 MB you have.
 
Level 8
Joined
Dec 11, 2012
Messages
412
Well, I have found the biggest source of space...

Those are following:
BlackFang Rogue (1.28 MB)
Dragon Blue (2.18 MB)
Morchok (1.14MB)
T10Shaman (1.13)
Warlord Zon'ozz (1,19MB)
Deathwing (1.35MB)
DragonRed (2MB)
Riashi (1.06MB)
Tauren Warrior (1.14MB)
Wyrmrest Temple (3.31MB)

Those models have 15 MB in them. Cutting them out would be able to drag it up to 9MB, plus, optimizer would save up to 2 MB, which would save 7MB.

One thing 'tho, 7 of them could prove useless if I want to suck up some space, but...

I'd have to sacrifice hero models, that would save up to 6 MB, then I'd have to delete Red and Blue Dragon (4MB together) and that's 10 MB, Deathwing proves useless and I could use Warcraft 3 version of Deathwing if its low filesize, which would help and replace Blue and Red Drake with some of the lower filesized drakes, or find a way to down their mdx completly.

Wyrmrest Temple in the other hand is a neccessary model and if there was a certain way to compress it as a model (mdlvis keeps crashing it), just the size, I can save the map.

But, without programs for mdx compressing but mdlvis and mdx squisher, I have no idea what to do.
 

Dr Super Good

Spell Reviewer
Level 63
Joined
Jan 18, 2005
Messages
27,180
Wyrmrest Temple in the other hand is a neccessary model and if there was a certain way to compress it as a model (mdlvis keeps crashing it), just the size, I can save the map.
You need to delete geometry (and/or animations) from it. Unless you have experience with modelling this can be very difficult to do and in any case will take quite a few hours of time.

I would consider sacrificing the models just to get the map released, you can work on including them in updates at a later time. If your map is popular it should be easy to find a few people to help optimize the models (after all, people like to see their name in the credits section).
 
Level 23
Joined
Oct 12, 2008
Messages
1,783
What I ultimately want is a solution, not someone to do it

Funny.
Solutions have been given by multiple users, you have ignored them all with inane bullshit excuses.
Its also interesting how in the first thread you begged for someone to do it for you and now you claim you dont want someone to do it?

You can post comment after comment about the same thing to other people, you can let the whole forum know I asked for optimizer and make a joke of me, but I don't care. I'm here for 1 purpose and its helping other people's project grow and make something of my old work. If I continued, I could've gone in agrument with you and start insulting you, but that's not my thing.

I am not here to insult you.
I am here along with everyone else to tell you there is no magic bullet when when it comes to map optimizing. Something that you have blatantly ignored and still continue to ignore.

The fact of the matter is that you do not understand how map optimizing works, as such you have unrealistic expectations of what can be done. You ignore what people have said and are continuing to say, with your "but X map did the same" claims.
You have no idea how X map did what, unless you actually did open the map.

Combine all of that and you can see how appear to be crybaby even though that may not be your intention.

then either help with that, or fuck-off as you said it perfectly. I did accept help you guys gave me at the ending in both topics but this one is merely for optimizer, not recruiter. Recheck posts and reconsider if your comment was neccessary at all. That's my 5 cents.

If that were true, this thread wouldn't exist and you wouldnt be here being a nuisance.

DragonRed (2MB)

To give you some perspective, I could import 2000 simple sfx models using the same filesize that 1 model uses.
This map has dozens of custom skins, icons and effects and is about the size of the red dragon model alone.

Like everyone else has said, your model usage is ridiculous.
You want your magic bullet? remove them. (Which is again what every other user has been saying so far)

Ignore it if you wish.
 
Last edited:
Status
Not open for further replies.
Top