3D Model Viewer

[GhostWolf]

Is it supposed to look like this? Not sure if it is working or not (IE 11). It is reporting no errors so I assume it is.

Yes.

GW, this is so cool, but I just want to clarify that at least my use case is not limited by the current implementation. I know you are a hacker and want to make it better, I just want to say that .

I know, but this actually interests me, since I never wrote multi threaded code (and while this isn't real multi threading in the sense that data isn't shared and there is no need for locking, in reality the code is still pretty much designed around locking, since passing buffers deletes them on the passer).

As far as viewing a single or a couple of models, version 1 (remember that time when there were two viewers, one for MDX and one for M3?) with zero optimizations was also good enough.

 

[GhostWolf]

Added animations.
Removed all sorts of allocations, but it still spikes (less, but it's still annoying).
I can't really tell why, so far.
Also it lags horribly on Firefox (it runs on 1 worker, since Firefox doesn't supply the number of CPU cores, so that's a big impact by itself).

/Edit
It looks to me like the browsers leak heap memory every time a buffer is moved between threads (and since this demo makes thousands of such moves per second, there are lots of GC hiccups), I opened an issue for Chrome to see what's going on.

 

[Dr Super Good]

It looks to me like the browsers leak heap memory every time a buffer is moved between threads (and since this demo makes thousands of such moves per second, there are lots of GC hiccups), I opened an issue for Chrome to see what's going on.

If this is in the same demo I ran earlier then this is not a problem in Internet Explorer 11. Not only does it seem to be running on multiple threads (4/8 execution units appear to be partly utilized), but the process virtual memory size remains constant to within ~1MB while running.

Considering how Internet Explorer used to be quite rubbish, this is vaguely surprising.

 

[GhostWolf]

If this is in the same demo I ran earlier then this is not a problem in Internet Explorer 11. Not only does it seem to be running on multiple threads (4/8 execution units appear to be partly utilized), but the process virtual memory size remains constant to within ~1MB while running.

Considering how Internet Explorer used to be quite rubbish, this is vaguely surprising.

Just checked with IE, and it doesn't actually run the animations, plus it runs for me with 1 worker (you can see the number in the console, the core information, if it exists, is in navigator.hardwareConcurrency), so I am not sure what you're talking about.

/Edit
Added basic support for textures.
The new code passes the bone buffers "by reference", which is why, counter-intuitively, it is way slower. Like I said, I believe this is related to browser bugs, still waiting for someone on Chrome to reply.

 

[Deolrin]

I have Chrome. Not sure how I can help. I'll note that the "basic support for textures" made it just give me a black screen.
Error messages: http://prntscr.com/7v8170
I have a quad core processor, so 2 workers is legit. Whatever a "worker" is.

 

[GhostWolf]

Ops, forgot to change the paths to the hive format. Should work now.
You can't help, browser developers might.

A worker is an encapsulation of an OS thread, used for multithreading in JavaScript (so all of the animation calculations don't affect the main thread, which is the thread that controls the UI and that the user interacts with, hence making it less laggy).
Basically, the work of the viewer is split so that it can run things concurrently on multiple cores of your CPU.

 

[Dr Super Good]

Just checked with IE, and it doesn't actually run the animations, plus it runs for me with 1 worker (you can see the number in the console, the core information, if it exists, is in navigator.hardwareConcurrency), so I am not sure what you're talking about.

It must have been changing execution unit constantly for some reason (so it was running on 4 cores 1/4 of the time.
The textures load, however everything is not animated and there are no errors shown.

 

[Deolrin]

I see. Well, it works now, even quite smoothly for the most part, but as you said there are spikes of low FPS every now and again.

 

[GhostWolf]

Added batch visibility (geoset animation alpha and layer alpha).
Added the ability to set the sequence.
Added the ability to get the list of sequences.

Because I am mostly interested in re-adding features, I am adding allocations all over the place, hopefully I'll clean them at some point.

I updated my Chrome and it actually runs far, far better.
It runs at a terrible speed on Firefox.
IE seems to run the animations at about 0.1FPS, you can see the models updating their frames one by one.

All tests were done with the same 40 peasants, militia, footmen, and knights.

Note that whatever the animation speed is, the rendering and UI in the main thread will probably work smoothly in whatever browser (kind of the point of web workers!).
Also note that both Firefox and IE don't supply the number of CPU cores, so there's always one worker. When I force the number of workers to 4 (for my CPU), they become quite faster.

/Edit
Changed the number of workers to be either the number of CPU cores, if available, or 2 otherwise.

 

[GhostWolf]

So after talking with Chrome developers, it turns out the problem isn't what I thought it was.
Chrome does leak, but not with the message posting, and their leak is tiny compared to what was going on with the viewer.

The actual issue is that every message object is being cloned implicitly, those are thousands of new objects per second!

Therefore, I changed the code so that instead of every instance sending/receiving update messages, the viewer itself sends one update message to a worker with the data of all instances, and receives one message in return with the data of all instances.
This removed most of the allocations. The code is messy and unoptimized, so it explicitly allocates objects, removing those allocations should make it even better.

This made the code run much better on all browsers. In fact, IE joined the >1FPS group!

Since the changes made it a little harder to work with multiple workers, it now uses a single worker.
Once I change it to use 2+ again, it should run much faster on all browsers.

/Edit
After discussing porting the M3 to Three.js with emnh (sorry, I don't know if you are from the hive, and if so who you are!), he suggested a really neat idea - caching whole sequences in textures.
This means that instead of re-calculating the skeleton every frame, it is pre-calculated on load, and from then on there are no more calculations required.
This trades CPU computations for GPU memory, I am not sure how scalable it is. For reference, the human knight walk animation takes a 512KB texture, but a very big chunk of that texture is empty (POT requirement). In practice, if all sequences will be in the same texture, less memory will be wasted.
This also opens other memory vs CPU considerations, and it's an interesting topic to investigate. For example, caching geoset visibility for every frame will reduce CPU computations by quite a bit, and it's actually not really terrible memory wise (probably a few kilobytes for all sequences).
I can see possible issues with models with really long animations, such as buildings.

 

[Ralle]

I love how this kind of seems like your blog. Keep it up, and good work!

Are you using Three.js now?

 

[GhostWolf]

Not going to use Three.js, but the person mentioned is.
After thinking more about caching sequences, I am not sure how practical it is.
Node calculations still need to be done for attachments, particle emitters, etc., and caching all of the nodes is not ideal.

 

[Dr Super Good]

I think that at the end of the day JavaScript just is not designed for this kind of graphics (currently at least). It has no problems making a previewer but when you get to performance it starts to struggle.

 

[Ralle]

Let's redo it all in WebAssembly .

 

[GhostWolf]

It's not really related to the rendering though (that can still be optimized quite a bit, which I might do).

Most of the computations are split between node updates (that is, matrix multiplication and generating matrices from keyframe data - location, rotation, scale, and the origin), and getting keyframe data.

Caching sequences avoids most of the above, however I don't know how to make it work without caching all of the nodes (or at least all of the attachment/emitter nodes), which can possibly become a memory issue.

 

[Kyrbi0]

So after talking with Chrome developers, ...

You know actual developers at Google for Chrome?

... Who are you?

 

[GhostWolf]

Well, by Chrome I mean Chromium (but the responses are still usually from Google people), and by talking I mean discussing in their issue tracker.

Lot's of experiments going on with optimizations.
Right now I am getting 600 units at 35-40FPS on Chrome, but this can be improved a lot.
In fact, once I switch to instanced rendering, rendering (aka the main thread, aka what shows the FPS) will probably always be 60FPS, also for thousands of models.
The real issue will then be solely updating node hierarchies. Caching animations would have solved that, but MDX files tend to have 6 minute long animations (buildings, decay, decay bone), these kind of ruin any hope of caching without taking a whole lot of memory.

 

[Ralle]

Good job!

You know actual developers at Google for Chrome?

... Who are you?

It's called open source

 

[GhostWolf]

I wonder if I should make a RAM-cached version.
This will add considerable RAM usage, but make run-time blazing fast.

For example, the following human units take 313MB. The sad thing is that without stupidly long animations (<10 seconds), they add up to just 65MB.

Units

Peasant
Militia
Footman
GyroCopter
Rifleman
WaterElemental
Priest
MortarTeam
Sorceress
Farm
AltarOfKings
WarWagon
HumanLumberMill
GryphonRider
HumanBarracks
GryphonAviary
Blacksmith
ArcaneSanctum
HeroArchMage
Knight
Workshop
HumanTower
TownHall
EX_HeroMountainKing
HeroPaladin

 

[Dr Super Good]

Does much happen during the long animations? If not then you could cache results at a lower frequency (compromise quality for memory).

Anyway a little accuracy bug with the current viewer with this model. Ignore the fact that the author specifies the wrong path in the post as that just affects usability and not model display.

In your viewer it appears fine. In fact it appears too fine since the model is actually broken in World Edit. The texture "BlackhandRaider.blp" provided with the model loads completely transparent in the editor and in game.

Below is what it looks like in your model viewer and the ancient "War3ModelEditor.exe" program.

This is how it should look if it were accurately reproduced.

I ruled out the possibility of me importing it incorrectly by substituting it for another custom texture (since it is a WolfRider wrap, any for that model would do) and that texture did appear. Hence something is wrong with the accuracy of the viewer compared with the editor.

Since this happens with a completely unrelated model editor as well, I suspect it might be some inaccurate behaviour with how the blp is being unmarshaled rather than with the actual render code.

 

[GhostWolf]

Hardly anything at all happens during long animations, however caching only parts is pretty troublesome with how OpenGL works.
It's really annoying that I can't figure any practical scalable way to optimize node updates.

The BLP texture you mentioned is JPG compressed, so if it works, it works, there are no different ways to do it.
When stuff doesn't show up like that in the editor, 99% of the time it's because the editor can't find the textures.
I honestly don't remember how importing works in the WE, and I don't have WC3 installed, so I can't say anything further.

 

[Dr Super Good]

When stuff doesn't show up like that in the editor, 99% of the time it's because the editor can't find the textures.

If the editor cannot find a texture the model will not load at all ("Could not load file") and a square box dummy will be used instead. Hence the model I show was correctly detecting the texture, just it was being displayed as completely transparent in game. Likewise if I replace it with a working wolf rider texture it then works in game as expected with that texture.

I am guessing that WC3 does not like the blp so is not loading it properly. Something about it is not compatible with WC3 yet is with third party model editors. Perhaps something to do with the mipmaps?

 

[GhostWolf]

I guess I remembered wrong then, didn't touch WE in years. Either way, my BLP decoder doesn't read mipmaps, and I have no idea what their rules are as far as WC3 is concerned.

I opened the texture in BLP Lab, and it has more mipmaps than required, so that the last 3 were the same. I don't know if that's valid or not, but you can try the attached BLP file which doesn't have them.

 

[Dr Super Good]

Still does not work. Are all the mipmaps the correct sizes?

 

[GhostWolf]

They seem correct. I have no idea what WC3 accepts or doesn't, so I can't help here.

 

[Dr Super Good]

Are all the mipmaps in it correct? I mean the same result could be obtained if all mipmap levels (except full scale) were incorrectly fully transparent. Your model viewer would work then because you only load the full scale image ("my BLP decoder doesn't read mipmaps"). This is also the sort of action other model viewers might make (mipmaps are usually generated automatically if not available anyway so "why bother") but WC3 does not make (Mac version crashes if not all mipmap levels are available).

 

[GhostWolf]

I only read mipmaps for compressed (DDS) textures, since you can't automatically generate mipmaps for them. Otherwise, this is easier

gl.generateMipmap(gl.TEXTURE_2D);

The texture looks fine from what I can see in BLP Lab.
I removed the "BLP comment", whatever that is, but the Hive won't let me upload since "You've already uploaded bla bla bla".
The attached file is the same texture generated in BLP Lap - saved as PNG, saved as BLP.

 

[Dr Super Good]

The attached file is the same texture generated in BLP Lap - saved as PNG, saved as BLP.

Works fine in game (visible).

I wonder what part of the original blp is breaking in WC3 though. Your model viewer is not the only one which can process such broken blp files as if they worked. This means that there is some common assumption about the blp file format which is incorrect.

 

[GhostWolf]

Just did some testing with batching and stuff for M3 models. I don't have time to implement all of the stuff I wanted to test, but the profiling is showing some nice results.
Basically, if I do end up implementing everything, I believe that many thousands of instances could be rendered with no effort for the browser at all.
It's at the level where it really depends on the GPU and drivers, and not on the browser and Javascript.
For MDX it's a little grimmer - the same rendering optimizations can be made, but without animation caching, the node updates will hog the browser.

 

[Dr Super Good]

A lot of what you are saying about animations reminds me of old games. They often had optimizations such as "synchronize animations" which caused all instances of a model to use the same animation state. For example early RTS games would have all walking units of a certain model walking synchronously (same applied to standing) so that the same animation data could be applied to all of them.

 

[InfernalTater]

I'm really looking forward to this! Good work so far and good luck on what's to come! I didn't want to read the entire thread to find out, but is it already confirmed that this will be used by THW when it is completed, or will it be a standalone tool?

 

[Ralle]

It is the model viewer you see when you click "View in 3D".

 

[InfernalTater]

It is the model viewer you see when you click "View in 3D".

The viewer he described did much more than the current model viewer. Were features removed?

 

[Dr Super Good]

GhostWolf is currently messing around with rendering performance. It is not a viewer per say and neither is it a tool. Something might be made of it eventually, or possibly it will only be an experience for him.

 

[GhostWolf]

The viewer he described did much more than the current model viewer. Were features removed?

Nothing was removed, however there is no graphical user interface for most of the stuff.
Blame me for lack of design skills...

And yeah, calling this a viewer probably doesn't do it justice anymore, it evolved to something closer to an engine, with the viewer part being solely the client, but whatever

 

[Ralle]

Aah right. GW is making a 3D library to display wc3 and sc2 models. The model viewer you get when clicking "View in 3D" is built using this library, but the library can do much more.

 

[GhostWolf]

Here's a small demo that implements everything I mentioned.

 

[InfernalTater]

Here's a small demo that implements everything I mentioned.

It's a baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling BANELING BANELING!

2116 SC2 models, no lag whatsoever, and in fact barely any CPU and RAM usage whatsoever. Nice.

Edit: I forgot, I have an 8-core 4 GHz processor, so "barely any CPU Usage", percentage-wise, doesn't mean much. Now that I think about the percentage of CPU that it was utilizing, (~2-4%), this is indeed CPU-intensive compared to a typical CPU, where it would probably use closer to 20-40%. It didn't use much RAM at all, though.

 

[Dr Super Good]

Following errors...

SCRIPT1014: Invalid character
File: shaders.js, Line: 29, Column: 28
SCRIPT5009: 'SHADERS' is undefined
File: viewer.js, Line: 169, Column: 5

First targets...

SHADERS["vsbonetexture"] = `

So second is obviously related to that.

 

[GhostWolf]

It's a baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling baneling BANELING BANELING!

2116 SC2 models, no lag whatsoever, and in fact barely any CPU and RAM usage whatsoever. Nice.

2025, but close enough

Following errors...


First targets...

SHADERS["vsbonetexture"] = `

So second is obviously related to that.

I was going to write "and before DSG mentions this, it doesn't work in IE", but I didn't.
I was too lazy to recompile the shaders, so I messed with ECMAScript 6 template strings, which aren't supported in IE.

/Edit
Recompiled the shaders, works in IE.

 

[Ralle]

Super laggy on my MacBook Air in Chrome and Firefox. Buuuut, it's a 4 year old ULV laptop with Intel 3000 HD graphics...

 

[Dr Super Good]

Confirming it works smoothly on IE11.

Are you using the full SC2 sharders (quite complex on ultra) or customized simplified ones?

 

[GhostWolf]

Super laggy on my MacBook Air in Chrome and Firefox. Buuuut, it's a 4 year old ULV laptop with Intel 3000 HD graphics...

It almost entirely depends on the GPU, hardly anything is going on in the JS side (well, except for IE of course, because in IE the TypedArray.set call to update matrices is taking 40% of execution time for me, while it takes 0% on Firefox/Chrome, go figure).
You can run a profile and see what slows it down.

Confirming it works smoothly on IE11.

Are you using the full SC2 sharders (quite complex on ultra) or customized simplified ones?

It uses whatever I ended up implementing back when I wrote them, which is obviously nothing similar to what SC2 does.
For starters, there are no lights, no shadows, and I use forward rendering and not deferred rendering (which is pretty much a must for games with lots of local lights such as SC2), no special UV mappings, no flipbooks, etc.
So yes, they are very much simplified - I just hacked some parts off old Blizzard shaders that someone gave me (they were apparently in the game's MPQs).

I actually thought about implementing deferred rendering, but WebGL didn't support multiple render targets back then, and deferred rendering without MRTs is a joke.

 

[Dr Super Good]

So yes, they are very much simplified - I just hacked some parts off old Blizzard shaders that someone gave me (they were apparently in the game's MPQs).

They still are in the game. A lot of modern PC games come with their shader source code in some form or another. This is so that the drivers can build and optimize them for the specific hardware. Pre-build shaders are more general so cannot take advantage of specific GPU optimizations.

SC2 shaders are specifically open source because I believe they were developed as some research project. The algorithms used are well documented with even examples showing their application in the WoL campaign interlude sections.

 

[Kyrbi0]

Here's a small demo that implements everything I mentioned.

I was just wondering why I stayed subscribed to this thread...

Maaan, that is impressive (viewing on updated (?) Google Chrome, with a reasonably "meh" computer).

 

[GhostWolf]

Pretty much gave up on ever optimizing MDX models.
Caching the animations quickly moves up to hundreds of megabytes of RAM (not even talking about storing in actual VRAM like the M3 demo), and I can't think of a single way to optimize the node calculations. The only possibility left was to skip calculations when nothing changed, but checking if anything changed turned out to have the same cost as calculating without checking.
Bleh.

Edit: I forgot, I have an 8-core 4 GHz processor, so "barely any CPU Usage", percentage-wise, doesn't mean much. Now that I think about the percentage of CPU that it was utilizing, (~2-4%), this is indeed CPU-intensive compared to a typical CPU, where it would probably use closer to 20-40%. It didn't use much RAM at all, though.

It's running on one core, and hardly anything is actually happening on the CPU, so it isn't related.

 

[GhostWolf]

Made a new tech demo that reads the doodad data from a map, and creates most of them.
Getting all of them is too bothersome without also making a SLK parser, and I don't care enough about it right now.

I had to change quite a bit of the code (and it's a complete mess!) in order for this to not run at about 10FPS.
In particular, the geoset rendering code is now extremely efficient, and in fact, rendering the scene is now so fast that it takes practically nothing compared to updating all of the nodes.

The only drawback is that the main optimization (instanced rendering) does not allow for per-instance texture overriding, but I am fine with that.
The solution to this would be to make a shallow copy of a model, which will hold only overridden things and reference the actual model, and new instances of this shallow model could be created.
This is a lot closer to how games would handle the situation, which is fine in my opinion.

You should probably not try to insert random units with the leftover GUI from the other demo, everything is super buggy, just grab the scene with your mouse to rotate...

 

[Ralle]

Wow, that is super cool! I don't see any menu but wow. I can SO imagine us having a map viewer

 

[PurgeandFire]

Cool stuff!

 

[WhiteFang]

That's very cool