Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

Java MDX lib

Discussion in 'Warcraft Editing Tools' started by oger-lord, Sep 9, 2014.

  1. GhostWolf

    GhostWolf

    Joined:
    Jul 29, 2007
    Messages:
    4,952
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    Please, enough of this DSG says something wrong, everyone corrects him, loop back...
    Especially when it has nothing to do with the topic.
     
  2. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    26,131
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    No I was comparing the buffer typecast to struct approach to IO which is not portable due to poorly defined struct packing as well as not supporting some file protocols very well.

    Most processors lack operators to atomically modify bits of a byte. As such bitfields can produce strange bugs in multi threaded environments which are hard to diagnose. For that reason many compilers like VC do not pack bitfields. Sure if one knew what they were doing with bitfields it is safe, but maybe the next guy to maintain the source will not be aware they are bitfields.

    I was saying that the standard migration can cause incompatibility with existing code. For example in the Simutrans open source project, the VisualC++ compiler of VS2015 was incompatible with the source code. The compiler has to be run in a special compiler mode to disable "sized destructors" functionality for the source to compile which makes it no longer standard compliant.
     
  3. edo494

    edo494

    Joined:
    Apr 16, 2012
    Messages:
    3,846
    Resources:
    5
    Spells:
    1
    JASS:
    4
    Resources:
    5
    if you work on code and dont notice members are bitfields you are bad programmer, thats like not noticing return value of function, it is in the syntax itself
     
  4. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    Instead of arguing about code.... Let's write some!
    My model editing software is still hitting bugs in cameras, so I figured I would upload my entire Eclipse workspace for anyone interested to take a look.

    A good example is the Fetch -> Unit command, and then if you go to Neutral and click on Ogre Warrior as shown in the picture below.
    Once you open it, even after GhostWolf's suggestion, this library is still splitting the Ogre's camera into 2!

    So if we could find a way to fix this that would be great!

    Edit: As shown in the pictures below, the 'Edit/delete model components' is a good way to see a summary of what the Matrix Eater thinks is in the model file, and we can see it is broken for the Ogre Warrior Portrait!
    Also, to run the program, go to the JWC3-MatrixEater project and locate com.matrixeater.src.MainFrame for the main method if you don't already have it available.
     

    Attached Files:

  5. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    26,131
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    The MDX code is bugged. It is encountering a camera part which it has no logic to interpret. Not only is this causing data loss (parts of the camera are not parsed), it also causes the camera chunk to become corrupted as it incorrectly decrements the chunk size remaining (it thinks there are more cameras to parse).

    I am looking into a fix suggestion now.

    EDIT:
    More correctly, it can encounter the following when parsing cameras.
    Code (Text):

    KCTR: float[3] translation
    KCRL: uint32 rotation
    KTTR: float[3] targetTranslation
     
    Currently it only has logic to parse KCTR and KTTR. If it encounters the KCRL part of a camera (which it does) it breaks.

    EDIT 2:
    Looks like the code you are using is different from the GitHub above. I am guessing you changed it?

    The problem is in the CameraChunk.Camera.getSize method since you did not update the size to reflect the new part size.

    It should be...
    Code (Text):

            public int getSize() {
                int a = 0;
                a += 4;
                a += 80;
                a += 12;
                a += 4;
                a += 4;
                a += 4;
                a += 12;
                if (cameraPositionTranslation != null) {
                    a += cameraPositionTranslation.getSize();
                }
                if (cameraTargetTranslation != null) {
                    a += cameraTargetTranslation.getSize();
                }
                if (cameraRotation != null) {
                    a += cameraRotation.getSize();
                }

                return a;
            }
     
    EDIT 3:
    Still not fixed, I am guessing there is something else wrong with size metering. Sigh I wish people would write proper code.
    EDIT 4:
    Solved... Turns out the above does fix the problem. The reason it was still not fixed for me was I broke some other part by mistake.
     
    Last edited: Dec 15, 2015
  6. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    Hmmm...
    Well, thanks! That fix made SOMETHING better, I haven't tested the models in game to see if they're still bugged, but the Ogre Warrior Portrait now shows as having only one camera.

    As for the differences between my code and the github, mine forked from it several months ago and has added functionality to translate into MDL in RAM, to another set of classes for representing the same data that I wrote in high school (and which also have their own bugs).

    Anyway, this is some cool progress! Thanks!
     
  7. Koward

    Koward

    Joined:
    Aug 24, 2015
    Messages:
    19
    Resources:
    0
    Resources:
    0
    Excuse me, I have a hard time trying to follow the thread.
    Who maintains the most recent code here ? Retera ? Dr SuperGood ?
    Is there a GitHub somewhere ? I would like to interface my beginning project of a java M2 (WoW) library (https://github.com/Koward/jM2lib) with an MDX one to promote easy conversion.
     
  8. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    I am just some guy who is using the library written by oger-lord. I think what you are looking for is his GitHub:
    https://github.com/OgerLord/WcDataLibrary

    But, in a previous post, I noted that I have copied this code into a project of my own, and been using it. This was mostly because I added a lot into it, because I wrote an MDL library similar to his MDX library.
    My own version was a copy from several months ago, and is not the official version. It is in the com.hiveworkshop.wc3.mdx package, which is different than the de.wc3data.mdx package used on the official version.

    Perhaps I should offer to contribute the MDL to to oger-lord's repo, but I was convinced it was more of a hack than a solution. If the official MDX API wants to adopt MDL support, it would seem to me best to work it into the same classes, into the same API, than to have two separate APIs with a hack conversion between them.

    tl;dr
    Use the one on the OgerLord's GitHub, not my stuff.

    Or use my stuff if you want hacky MDL support, but knowing full well that all kinds of stuff could be broken, out of date, or unsupported.
     
  9. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    26,131
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    OgerLord's is buggy however. It will not parse CameraChunks correctly as we discussed above. It lacks the capability to parse rotation parts.
     
  10. Koward

    Koward

    Joined:
    Aug 24, 2015
    Messages:
    19
    Resources:
    0
    Resources:
    0
    Okay. Then I should probably make my own mash-up with your WC3Utils as a basis, OgerLord classes for MDX parsing, fixing Cameras, and modifying WCUtils as using a NIO2 FSP if it's not already done (I don't remember).
     
  11. oger-lord

    oger-lord

    Joined:
    Jul 24, 2006
    Messages:
    156
    Resources:
    2
    Maps:
    1
    Tutorials:
    1
    Resources:
    2
  12. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    26,131
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    Under cameras, near "{CameraTargetTranslation}". There is also a "{CameraRotation}" with identifier "KCRL" which is missing and the reason why it breaks cameras.

    The structure of this can be found here. It is a track like the others but instead of an array of 3 floats it is a single uint32.
     
  13. oger-lord

    oger-lord

    Joined:
    Jul 24, 2006
    Messages:
    156
    Resources:
    2
    Maps:
    1
    Tutorials:
    1
    Resources:
    2
    Thanks, added the line to the specification and updated the code.
     
  14. Dr Super Good

    Dr Super Good

    Spell Reviewer

    Joined:
    Jan 18, 2005
    Messages:
    26,131
    Resources:
    3
    Maps:
    1
    Spells:
    2
    Resources:
    3
    When loading cameras, the for loop to detect the parts can be made more efficient by breaking if it fails ("else break;") since there is no reason to loop another 2 or so times if it already failed a loop.
     
  15. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    I'm dropping in with another bug report for the Java MDX Lib that I encountered while using it in my Matrix Eater software: it appears that when parsing FieldOfView, FarClip, and NearClip in Cameras, at least in some circumstances, I was receiving astronomically large and inaccurate values for these fields.

    It may be that a simple read/write test produces the same file that was originally read in this case, but accessing and reading the "camera.fieldOfView", "camera.farClippingPlane", and "camera.nearClippingPane" gave me the incorrect and very large values (maybe the float was parsed in the wrong order from the binary?).

    Edit: It looks like your library treats this value as an int whereas my MDL library is treating it as floating-point. Perhaps it was meant to be read as a float from the binary?

    Another Edit: Switching them to be parsed and saved as floats seems to have fixed the issue in the Matrix Eater codebase.
     
  16. GhostWolf

    GhostWolf

    Joined:
    Jul 29, 2007
    Messages:
    4,952
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    I added all of these fixes in some post on this thread some time ago...just follow the spec.
     
  17. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    I was going to say that just meant my own repo had gotten out of date with the GitHub repo, but maybe the GitHub one hasn't been updated yet? Looks like it's still using ints for those.
     
  18. oger-lord

    oger-lord

    Joined:
    Jul 24, 2006
    Messages:
    156
    Resources:
    2
    Maps:
    1
    Tutorials:
    1
    Resources:
    2
    Added the changes to the git repository.
     
  19. dtnmang

    dtnmang

    Joined:
    Apr 20, 2010
    Messages:
    561
    Resources:
    12
    Models:
    5
    Maps:
    2
    Ported Models:
    1
    Tutorials:
    4
    Resources:
    12
    Does this mean we can get a modern "upgrade" to the Model Editor which has been around for nearly 10 years, and sometimes doesn't work properly?
     
  20. Retera

    Retera

    Tool Reviewer

    Joined:
    Apr 19, 2008
    Messages:
    1,049
    Resources:
    27
    Models:
    18
    Tools:
    2
    Maps:
    6
    Reforged HD Models:
    1
    Resources:
    27
    I think the challenge is one of whether somebody would sit down and write it. I have attached my own "Model Editor", the "Matrix Eater", in one of its newer un-released builds, if you want to take a look.

    But the problem is that even my program is something I started in high school that has had new features injected in over several years by just me (one guy). Without either a really, really, dedicated person or a team working together, it would be hard to totally remake all the features of any one of the model editors out there. I have always faced the issue that the Magos Model Editor, the Matrix Eater, and MdlVis all have differing features and to make a complicated model I often need all three of them... which is not so ideal, but hard to correct.

    Sidenote: talking about modern software, check out the File > Fetch menu in my attached software. It's a feature I'm particularly proud of; a simple yet elegant way to open model files while thinking in a way that is parallel to the world editor.
     

    Attached Files: