Hive Viewer fun possibility for enhancement

[Retera]

The previewer inside Retera Model Studio is based on the Hive Viewer but has some enhancement to try to preview metals and stuff in PBR, and I changed the interpretation of the normal map to be more accurately reflecting how I think the game is supposed to be. After putting time into tinkering with it, I discovered that although I had not been focused on it, HiveWE developers independently made the same realization that I did, to basically realize that the normal maps have the Y coordinate magically flipped or something dumb like that. It's been a little while and I'm writing this without reviewing the technical details, but as an end user there is a visual difference between Hive Viewer HD preview and the ingame/ReteraModelStudio preview at this point:

Here is a 4k screenshot of Blizzard World Editor, Retera Model Studio, and Hive Viewer side by side. If you look closely, basically what I was interested in discussing here is the Normal Map stuff and how the Hive Viewer one has the highlights and such from the normal map in a different place. For me when zoomed out this image makes the model look almost the same in each view, but when I zoom in on the 4K it seems like the details are popping out more when the normal map is loaded this other way that I was doing in recent builds of Retera Model Studio.

I think the recent update to support MDX 1100 on the Hive Viewer by @ThompZon and @twilac are somewhat inspiring. I posted here on the Hive on another thread where people might not have seen it, maybe two weeks ago, where I was saying that we might not have anybody else around interested in doing the technical work to make Hive Viewer load the MDX1100. It's really cool and almost embarrassing to me for me to be proven wrong. So, I thought that while we are on that topic of other people out-performing me, I would bring attention to this normal maps issue like what we see in the images above.

It's important to note that while I was getting my computer science degree, I did not take the elective computer graphics courses. So, in a manner of thinking, I am no better than anyone else on Hive at this. That's the real truth and if I think of it any other way it's just me indulging in wanting to feel good about myself. But so basically in the images above, the far left image (Reforged WE) is a closed source proprietary program, and there is no guarantee that we could ever match how it looks. But the image in the center (Retera Model Studio) is fully open source OpenGL program, so it might be a relatively easy task to make Hive Viewer look exactly like it. But do you want to? There is some manner of subjectivity with regards to what is "right" or "wrong" in how we preview the asset data. But I believed that the normal maps popping out and handling the flipped Y coordinate thing correctly does look more "right" to me than the original Hive Viewer HD shader that I started from in Retera Model Studio.

With that in mind, there is another bug in the way that Ghostwolf's HD shader works that might obscure the issue if you try to fix it. Basically, when done correctly the calculations take into account the user camera and also the light source. There's a conversion typo in Ghostwolf's code if I recall that skips a coordinate transform so that the light source is in OpenGL camera space instead of world space, and effectively moves with the viewer's eye as a result. If you spin the Hive Viewer preview of an HD asset, the lights and darks spin with you. What I found when playing with a copy of that code in Retera Model Studio was that taking a first step to be "more correct" makes things look worse than Ghostwolf at first, and it was necessary to fix other issues to make it get better. Like a pair of bugs that together obscure the pathway to the working code, such that the bugs produce a visualization "better" than how the code would be with either one of them individually fixed but not both fixed. In other words, fixing either bug, in isolation, makes the output look worse. Maybe there's a way to solve these issues more efficiently and with less visual experimentation than what I did last time I played with it, but I do not know. I think this is why Ghostwolf wrote in one of his commits that he wasn't going to do more unless Blizzard gave him their shader code to copy from.

Spoiler: Spinning the Camera on Hive Viewer

Spoiler: Spinning the Camera on Retera Model Studio

So, anyway, I do not know if it is interesting or engaging for @ThompZon or @twilac to do the volunteer work to make the Hive Viewer look more like the Retera Model Studio linked above, or even if it is desirable for the visualization to look this different way, but I wanted to bring attention to the idea that my "improvements" to Ghostwolf that I am using in my version of my own tool are also open source and could be copied from. In addition, the Retera Model Studio preview might be hard to copy from because of the nature of its spaghetti code, but I created a more 1:1 port of the Ghostwolf viewer into Java within the "warsmash" repo on my github and that one might be much simpler and much quicker to copy from, and I tried to put most of the same enhancements from the Retera Model Studio changes into that repo in what was otherwise a Java port of the Ghostwolf Hive Viewer:

On the far right in this group, I show this same test model from Hive using the "Warsmash Model Editor" (this name is partly a joke from trying to skin the app to look like Magos War3ModelEditor - in reality it's an unfinished desktop app that does little more than preview models using the exact same code that the Warsmash repo uses for Gameplay). You may notice that the "Warsmash" preview is much darker. I did this because I was trying to account for having multiple lights in the scene, and it is intentional. But again, everything that I have done was all done with experimentation and from reading this website: LearnOpenGL - Theory
According to that website, there is a fancy equation for multiple light sources in a PBR setting. And this is probably why ultimately nothing I have done on Warsmash matches Reforged 1:1 for visuals.

For example, above we can kind of see that when I combine the multiple lights on Warsmash and add up their colors, the resulting image is much too bright. But some logic similar to this is necessary for an accurate portrait preview, and Retera Model Studio does not process model lights and instead has one hardcoded light, so it is missing some of the lights and darks of different colors that we see on Reforged portraits when they use many light emitters.

So, maybe the takeaway from this is that the Retera Model Studio preview is a "good approximation" that can be achieved while not processing light emitters, and that spending more time than this ends up giving exceedingly diminishing returns unless you are an expert in the field.

But, I thought I would share all of these thoughts because I noticed the Hive had a notice about recent viewer updates, and I thought dumping my current knowledge could help if people are actively editing the code.

 

[Ralle]

Very nice informative thread Retera, really.

@twilac and @ThompZon. I hope you read this and I hope you consider looking into this

 

[Archian]

Thank you for sharing your insights and observations regarding the previewer inside Retera Model Studio and the Hive Viewer. It's cool to hear about the enhancements you've made in Retera Model Studio, particularly regarding the preview of metals and PBR, as well as the changes you've made to the interpretation of normal maps.

It's impressive to see how the HiveWE developers independently made similar realizations regarding the normal maps' Y coordinate flip. It's a testament to the collaborative nature of the community and the shared pursuit of improving the tools and experiences related to modding. It's also encouraging to witness the recent update to support MDX 1100 on the Hive Viewer, proving that there are individuals dedicated to advancing these technologies.

Your contributions and experimentation with Retera Model Studio are valuable. Your efforts to make the normal maps appear more accurate and visually pleasing reflect your commitment to improving the tools at your disposal. The bugs you mentioned in @GhostWolf's HD shader and the complexities involved in achieving the desired visual output highlight the challenges of the task.

You raise an interesting point about whether it's desirable for the Hive Viewer to look exactly like Retera Model Studio, considering the subjectivity involved in previewing asset data. It's essential to balance different perspectives and preferences. However, by sharing your improvements to Ghostwolf's code and the open-source nature of your work, you provide an opportunity for others, such as @ThompZon and @twilac to explore and potentially incorporate these enhancements.

Thank you for taking the time to share your thoughts and knowledge. It's valuable for those actively working on the code and for the broader community interested in modding. Continued collaboration and sharing of ideas will undoubtedly contribute to the ongoing improvement and development of these tools.

 

[twilac]

I've looked a bit at the rendering in the viewer now and noticed that DDS files with ATI2 (aka 3Dc/DXN/BC5) compression is loaded into GL as Luminance-Alpha and not RG. The result of this is that the viewer disregards the green channel, and uses the red channel for both U and V, for most normal textures.

This is my not so elegant solution for the fragment shader since I did not figure out if it was possible to load the texture as RG

vec3 decodeNormal() {
  vec4 temp = texture2D(u_normalsMap, v_uv);
  vec2 xy;
  if((temp.w-temp.z) != 1.0){
    xy = temp.xw * 2.0 - 1.0;
  } else {
    xy = temp.xy * 2.0 - 1.0;
  }
  return normalize(vec3(xy, sqrt(1.0 - dot(xy, xy))));
}

Spoiler: Links

Shaders in Thompzon repo

My RMS vert shader
My RMS frag shader

Wiki ATI2
Microsoft DDS