1. Updated Resource Submission Rules: All model & skin resource submissions must now include an in-game screenshot. This is to help speed up the moderation process and to show how the model and/or texture looks like from the in-game camera.
    Dismiss Notice
  2. DID YOU KNOW - That you can unlock new rank icons by posting on the forums or winning contests? Click here to customize your rank or read our User Rank Policy to see a list of ranks that you can unlock. Have you won a contest and still havn't received your rank award? Then please contact the administration.
    Dismiss Notice
  3. We have recently started the 16th edition of the Mini Mapping Contest. The theme is mini RPG. Do check it out and have fun.
    Dismiss Notice
  4. Dismiss Notice
  5. The Highway to Hell has been laid open. Come along and participate in the 5th Special Effect Contest.
    Dismiss Notice
  6. Check out the Staff job openings thread.
    Dismiss Notice
Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

BLP1 and Mac Computers

Discussion in 'Warcraft Editing Tools' started by PurgeandFire, Jun 2, 2013.

  1. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,426
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    BLP1 and Mac Computers:

    Table of Contents


    No one has formally written what crashes and what doesn't.
    I am willing to test a variety of things, so if you have any requests you can
    make a request here. The current established rule is the 7-mipmap rule:
    This seems to work so far. However, for developers who want to support
    mac players, some extra information may be needed. I'll provide some based
    on whatever tests I have been running.


    Testing:
    • I converted an icon using Button Manager (by Spec).
      Spec's Button Manager is on a default setting of automatic quality definition.
      To find the mipmap count, I used BLP Lab. This is how it is displayed:
      BLPLabMipmap.png
      As you can see, it has Mipmap count: 7 (5 fake). From playing with the
      settings, it seems that BLP Lab and Button Manager will maintain this count
      of 7, and supplement some "fake" mipmaps which preserve the size of the last mipmap.
      You can get how many mipmaps were used when saved by doing 7 - 5 = 2.
      The odd thing is that this worked for one icon, but not for another. Both were
      png files of 256x256 size. The cartman icon did not work. The other did.
      EDIT: I realized the cartman one was under JPEG compression. The other one
      was under paletted.
      .
    • Upon testing, the cartman icon seemed to only cause the crash when I first saw it.
      I assigned it to the holy light ability. When I learned the skill (the research icon was unchanged)
      it crashed (which was immediately before the cartman icon would be displayed).
      .
    • When I saved the cartman icon with a mipmap count of 7 (using BLP Lab),
      it worked on my mac without any crashes or issues.
      .
    • I downloaded a sorceress skin and set its mipmap count to 1. I imported it
      into a test map (one that works on mac) and I tested it on Windows. No crashes. (albeit the texture was messed up)
      When I transferred the map to my mac, it crashed on the loading screen.
      .
    • Using that sorceress skin (default mm count of 9), I changed its count
      to 7 with the paletted option. It worked just fine on my mac. With JPEG compression
      at 100% quality, and mipmap count of 7, it crashed. It worked, however, with a mipmap
      count of 9.
      .
    • Using that sorceress skin, I changed its mm count to 6 with the paletted
      option checked, as well as its sub-options. It worked without any crashes.
      Same for count of 5, with paletted. And 4. It worked for 1 as well without
      any crashes, despite the model being horribly disfigured even more than on
      my Windows computer. Tested the same for another skin, which was for Lord
      Garithos. It has a default mipmap count of 12, and I used 4 with paletted, and it
      worked without any crashes!
      .
    • Next the Lord Garithos skin (512x256); it is under JPEG compression. I compressed it
      with a mipmap count of 11, 10, and 9. They all worked
      without any crashes. A mipmap count of 8 crashed.
      .
    • Next a fel guard skin (hq 512x512); it is also under JPEG compression. With the
      settings it had as I downloaded it, it had 12 mipmaps (2 fake), therefore 10 real.
      It worked without any crashes. I dropped it to 9, and shizam it crashed.
      .
    • I tried that fel guard skin with 5 mipmaps and paletted under 128 colors. Worked.
      Same for 16.
      .
    • Using the fel guard skin, I adjusted the quality to 50%, 25%, and 1%. They all worked
      without crashing. Both with and without the JPEG sub-options enabled.
      .
    • Sure enough, the cartman icon worked once I changed it to paletted.


    Conclusions:
    • First, this crash does not apply only to icons. It applies to both regular textures
      and, most likely, any BLP for that matter. (confirmed only for skins and icons)
      .
    • The "fake" mipmaps that are generated by BLP lab/Button manager do not
      seem to fix the crash in most cases. On default settings, button manager can
      potentially cause the map to crash on a mac.

      On default settings, button manager will use the paletted option. It will not
      cause a crash on a Mac.

      Button Manager may choose settings that crash on a Mac.
      .
    • The magic number isn't necessarily 7. Skins will work at other mip map values even under JPEG compression.
      .
    • You can fix BLP1 files that were already converted by increasing its mip
      map count to the appropriate value.


    New Rule:

    After a lot of testing, I found a rule that seems to be consistent with my results.
    For testing, I used BLP Lab, BLPaletter, ButtonManager, and three skins:
    http://www.hiveworkshop.com/forums/skins-552/sorceress-blp-230570/?prev=search=sorceress&d=list&r=20
    http://www.hiveworkshop.com/forums/...al-blp-125990/?prev=search=knight&d=list&r=20
    http://www.hiveworkshop.com/forums/...ht-blp-152087/?prev=search=knight&d=list&r=20

    I'll first explain how mip maps seem to work. The default count of mip maps
    will depend on the dimensions of the BLP. That is why I chose the skins above;
    the first is 256x256, the second is 512x256, and the third is 512x512.
    Now, when you open up a BLP in BLP Lab, the mipmap count will be displayed:
    mipmap.png
    As you can see, there are 9 for this 256x256 texture. Each mip map has its
    own dimensions. BLP Lab organizes it in descending order. The first (#0) has
    dimensions of 256x256, the second (#1) is 128x128, the third (#2) is 64x64,
    the fourth is 32x32, fifth 16x16, sixth 8x8, seventh 4x4, eighth 2x2, ninth 1x1.

    When you decrease the mip map count, a number of "fake" mip maps will be created.
    They will point to the same offset and size as the previous mip map, and have the same dimensions:
    mipmap2.png
    As you can see, mipmap #7 and #8 have the same values as #6. Those are the
    two "fake" mip maps.

    ---

    Well, that explains mip maps a little bit. Now for the new rule. It seems that
    with JPEG compression, there must be a mip map with a dimension of 1,
    either for height or width. This seemed to be consistent with my results.
    At first I thought you had to reach 1x1 or have the max mip map count for that size,
    but the knight/garithos skin worked with 9 mip maps -> lowest dimension is 2x1.
    The quality control and the sub-options for JPEG compression have no effect
    on whether the texture crashes.

    However, paletted textures work with any mip map count. They will not crash,
    and the options (colors/encoding) will not cause the map to crash.

    So how did Weep come to his conclusion? He tested it for icons, and he found
    out what worked. :) Thank goodness too, because this would've taken
    even longer otherwise. The reason why the magic number "7" works for icons
    is that icons are 64x64, which means the mip map count is 7. This follows my
    rule that the final mip map must contain a dimension of 1.

    For regular (a x a) dimension textures, you must have the max mip map
    count if you use JPEG compression. To calculate this, 2n-1 = dimension.
    For example, 2n-1 = 64, ... ln(64)/ln(2) + 1 = n = 7

    For irregular ( a x b) dimension textures, you would use the same equation, but
    you would plug in the smaller dimension for the equation. For example, for garithos:
    2n-1 = 256, ... ln(256)/ln(2) + 1 = 9. :)

    Tools:
    • BLP Lab - Will allow you to set the mip map count of a BLP.
    • ButtonManager - Can work if the settings are adjusted properly.
      If you use JPEG compression, it will work so long as you adjust the settings as Weep described (see the quote above).
      For good measure, always open up the icon in BLP lab and check the mipmap count.
    • Wc3Borderizer Online - Appears to produce the appropriate icons with 7 mm count.
    • BLPaletter - Confirmed that this works. Use the paletted option. Note that this does not
      work on BLP's and PNG files; those formats are not supported by the tool.

    ------

    Thanks to Weep for the preliminary information, Shadow Daemon for BLP Lab
    and ButtonManager, Cohadar for requesting whether his tool works (I ended up testing this afterward),
    PitzerMike for BLPaletter.
     
    Last edited: Aug 9, 2013
  2. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,426
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    Sorry for the double post, just posting that I have finished documenting this and have formed a new rule.
     
  3. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,804
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Sounds like the problem is related to the jpeg block decoding on lower mip-maps. If "fake" mipmaps are used then it will allocate nonsense offsets and sizes. If the game tries to decode these properly, something bad will happen.

    On Windows, it must try and decode them and fail (high robustness).
    On Mac, it must try and decode them and crash (low robustness).

    In either case, fake mipmaps are an error as you are generating invalid mipmap data and it is a reasonable expectation for the program to not behave well in response to this. I doubt any Blizzard textures have "fake" mipmaps.

    Palleted and uncompressed will obviously not suffer this problem as long as a reasonable offset and size is chosen since it will read the file stream as a sequence of pixels (thus maybe the garbage graphics?).

    Does BLP require mipmaps or full mipmaps? If there is a field that tells the game how many mipmaps to use then it could be possible to eliminate them altogether as often the GPU will generate them when loaded if absent.

    Alternatively, specifying the fake mipmaps to target valid mipmaps of garbage might also work. If each mipmap has a header and a series of JPEG blocks then the series of JPEG blocks could be chosen that is compressed the smallest.

    Ultimately it may be best to generate full mipmaps. What few people know is there is no logical constraint on what visual data a mipmap uses, as long as it is the expected format. This means mipmap sequences of different colours could be used to make the surface change colour in-game once it falls below a certain scale. Turning on Anisotropic filtering should make it appear to change colour gradually as scale changes.
     
  4. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,426
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    Seems like a reasonable explanation.

    Mipmap count is determined by dimensions, so I assume the game uses that to determine it. I could be wrong though. Although, if it were as easy as changing a field, I think the compressors would have done that instead of generating fake mipmaps.

    edit: There is no field in the blp format to show the number of mip-maps.

    That would be cool. :)

    Overall, people choose to lower mipmaps because it offers great compression (30% for me once, going 10 to 1 mm, probably looked buggy ingame though). People also probably choose JPEG because it appears to offer better compression. It is true that it will usually give a lower file size than paletted, but paletted compresses better in the map archive and will often lead to lower overall size. It is good to know this, since my initial impression was that no one would fix their BLP's since it could sacrifice a chunk of file size. With the paletted option, they may not have to. :)
     
    Last edited: Oct 9, 2013
  5. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,804
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    That is because mipmaps increase texture size by approximately 33%

    But instead they are sacrificing colour depth since jpeg is 8 bbp RGB where as palleted is limited to 256 colours maximum.
     
  6. Footman16

    Footman16

    Joined:
    Jul 14, 2012
    Messages:
    1,619
    Resources:
    7
    Packs:
    1
    Maps:
    6
    Resources:
    7
    I was talking to ralle about some icons from this site crashing and how it might be due to compressing icons but now that this has been discovered maybe ralle should make a rule about mip maps etc
    It would allow mac users to use all icons from this site. Someone should show this to ralle.
     
  7. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,804
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Or he could code a system that automatically corrects all such icons. Until he has time, moderators could do it manually and explain to people what they did wrong.
     
  8. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,426
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    Sorry for the necro-bump. I just noticed that there is one particular file that kind of evades this rule:
    war3mapMap.blp

    Oddly, it has 1 mip map and is JPEG compressed and doesn't crash. I'll probably have to do some tests on it. For now, I assume that this crash only occurs when it repeats offsets. In war3mapMap.blp, it doesn't repeat offsets (it literally has 1 mip map, even though *technically* it should have 8 as a 256*256 image). And it doesn't have an alpha channel. But I tried importing a BLP without an alpha channel and with improper mipmap count and it still crashed.

    Interestingly enough, there have been reports on macrumors of mini-map bugging (it blackening out). Not sure if these two things are linked, but maybe. I haven't had that issue yet, so I wouldn't know.
     
  9. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    25,804
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    I think with recent discoveries this topic disserves a necro, especially since you advertise it so diligently PurgeandFire.

    BLP files have a field which I call "hasMipmaps" which is used to determine mipmap availability. Mipmaps are only used/loaded if this field is set, in which case it will try to load a full set of mipmap images (down to the 1*1 pixel level). If not set then only the full sized image (mipmap 0) will be loaded.

    The mipmap data chunk table plays no part in deciding if a mipmap exists or not, instead it is assumed to always be correct for the number of mipmaps needed. Number of mipmaps needed is determined by the hasMipmaps field and the height and width field. Entries in the table for mipmap levels that are not needed are ignored.

    If the mipmap image chunk data is invalid then any kind of nonsense can occur. For example if an indexed BLP file has the chunk length specified shorter than what is required for a mipmap image chunk then the result is the mipmap pixel data is sourced from out of buffer bounds memory in an unsafe way (memory garbage, possibly even a fatal error). Mipmaps are an all or nothing affair, you cannot only have some mipmap levels and it is pointless providing extra.

    Mipmaps are only needed when the texture is set to use mipmaps. In the Windows version of WC3 it appears that all UI elements such as minimap and command card buttons undergo bilinear filtering but do not use mipmaps. Hence why you can get away with 1 mipmap level for them.

    However when mipmaps are needed, such as by units and other ingame 3D models, then it is important to use mipmaps. Warcraft III will not automatically generate mipmaps for such images, instead it will load the mipmaps as blank textures (default of transparent black (0,0,0,0) pixels) which is very impractical. Models using a no-mipmap textures will only appear correctly where the full sized mipmap (level 0) is used, with areas using higher mipmap levels appearing either transparent (if applicable) or black.

    When the JPEG decoder encounters an error the game should load the mipmap level as a blank texture. If the JPEG image is the wrong dimensions it should automatically be subsampled through truncation/expansion towards the lower right. My guess is the MAC crash is a result of a non-robust JPEG image decoder which crashes the game on error (trying to decode a JPEG image with just the header and no component data) rather than failing and letting the game use a blank texture.

    That would be the result of the minimap texture being marked to use mipmaps (something that really does not make sense). In Windows none of the buttons and minimap UI elements use mipmaps so they can in theory be created without them. This would save ~30% file space or allow 30% more icons to be packed into a map.

    With the recent MAC patch this may need some retesting. Has anyone confirmed this crash still even exists?
     
    Last edited: Jul 22, 2016
  10. PurgeandFire

    PurgeandFire

    Code Moderator

    Joined:
    Nov 11, 2006
    Messages:
    7,426
    Resources:
    18
    Icons:
    1
    Spells:
    4
    Tutorials:
    9
    JASS:
    4
    Resources:
    18
    Cool info, definitely plausible--especially considering the BLP2 header also has a flag that determines whether mipmaps are used. Thanks for the tests. :)

    It still exists, yeah.