REQUEST: Performance Improver

[Zwiebelchen]

We all know that the amount of units and destructables on the map have a drastical impact on the game FPS.

On my 480x256 sized map, after the need of placing almost 1000 invisible plattforms (thanks to the alpha tile screwing up when under the effect of fog of war), I experienced an FPS drop to like 30 fps now. If I remove all plattforms, the FPS goes back to the normal 55-60 fps.

I did some tests and noticed, that only the amount of actually shown destructables matters - regardless of wether they are within the camera view or not. The same should apply to units.

So I had this idea to use the ShowDestructable() or ShowUnit() native to increase the performance of my map and simply hide all units and destructables, that are currently not viewed by the camera locally.


My question is: Has someone experience with this? Or does a system like that already exist? If not, I wonder why nobody thought about this yet.
Maybe someone is interested to code a system like that?

Just something to consider: getting a unit's or location's Z globally will probably desync on hidden destructables, as hidden destructables are not walkable, so there should be a function to get those values safe.

 

[Troll-Brain]

Unless your unit has no interact at all with gameplay, hide it locally is very like to cause some desyncs, for example if the unit is targeted, unit collision and such.

Now your result is quite strange, IIRC the number of units in a map doesn't matter that much on fps, only how many which are on camera view, because of playing animations.
Make sure you have enough ram available.

Also since animations are not synced, GetLocationZ can already give a different value for walkable destructables (like if some player have not his camera view on it)

There are several threads about the use of GetLocationZ.

 

[Zwiebelchen]

Unless your unit has no interact at all with gameplay, hide it locally is very like to cause some desyncs, for example if the unit is targeted, unit collision and such.

Well, the thread was more pointed towards the hiding of destructables, so I guess that is okay.

Now your result is quite strange, IIRC the number of units in a map doesn't matter that much on fps, only how many which are on camera view, because of playing animations.
Make sure you have enough ram available.

I did some tests just some minutes ago. Removed all invis platforms such as walkable doodads on the map and had a global FPS increase from 30 fps to like 55 fps. I got 8 gigabytes of RAM. It seems that the engine of Warcraft III is so poorly written, that walkable destructables eat performance like shit, even if not even close to the camera.
(And I deactivated all triggers and scripts in the map, so it has nothing to do with the script; it's purely the presence of those destructables that is causing this fps drop)

As I was using the Show command to hide the destructables instead of removing them, I had the same increase in performance, so hiding them is enough.

Also since animations are not synced, GetLocationZ can already give a different value for walkable destructables (like if some player have not his camera view on it)

So I guess it doesn't even matter, hm? In this case, making a system like that would be even more easy.

There are several threads about the use of GetLocationZ.

Well, for the point of this system, I don't really care about it. I don't use GetLocationZ globally in my map, so it doesn't really matter.

A system like that would be very useful, as the FPS drop on slower machines with lots of walkables is insane.



What I'm thinking about now, is how to set up a system like that. Enumeration based on the camera target is certainly not the way to go, as I could imagine it being extremely performance costly. Maybe splitting the map into squares and storing the placed destructables into arrays and then showing only those inside the same square and all squares around might be a solution.

 

[Troll-Brain]

It would be really map specific, but i would go for squares and a periodic timer to hide what it's not already hidden and can be hidden according to the current camera position.
Now to be smooth the period has to be low.

EDIT :

Oh and btw it seems you didn't get what i meant.
I basically said that you shouldn't have a fps drop if in your view you don't have visible destructables, in other words it's not the total number of visible destructables which does matter but the number of visible destructables which are in the current camera view.
So a such system would be pointless, because no performance improvements.
Now it's only a guess, i just remember think that's the case with units and that actually makes sense because of playing animations.

 

[Zwiebelchen]

It would be really map specific, but i would go for squares and a periodic timer to hide what it's not already hidden and can be hidden according to the current camera position.
Now to be smooth the period has to be low.

Well I once wrote this system that changes the animation of destructables based on the camera position, to turn treetops partially invisible.

It works like this: all treetop destructables are stored in an array, the system goes through a set amount of entries every period and decides for the destructables to play the specific animation depending on the offset to the camera. It works fine for this system, as I doesn't need to be instant.

For destructables to show, however, it would be different. It would be fine if they don't disappear instantly, but for the showing routine, it has to be instant, so a hybrid of this and the enumeration approach could be the best solution.

Enumerize those within the screen (considering the enum functions use a boolean tree anyway, it should be faster than anything you can code manually), hide those outside the screen with a refresher only going through them in max step intervals.

Oh and btw it seems you didn't get what i meant.
I basically said that you shouldn't have a fps drop if in your view you don't have visible destructables, in other words it's not the total number of visible destructables which does matter but the number of visible destructables which are in the current camera view.
Now it's only a guess, i just remember that's the case with unit and that actually makes sense because of playing animations.

I did understand you well, don't worry. But it's not the case for destructables, for some reason.

Even when turning the camera to a region without any destructables at all, the FPS remains on 30. If I hide all destructables on the map, it goes up to 60 immediately. It also doesn't make a difference wether the destructables are hidden within the black mask or not.

 

[Troll-Brain]

For destructables to show, however, it would be different. It would be fine if they don't disappear instantly, but for the showing routine, it has to be instant, so a hybrid of this and the enumeration approach could be the best solution.

I don't get it, if the destructable is not anymore on the camera view why should we delay the hide ?
Only destructables which are not on the actual camera view should be hidden.

Enumerize those within the screen (considering the enum functions use a boolean tree anyway, it should be faster than anything you can code manually), hide those outside the screen with a refresher only going through them in max step intervals.

Well, since one thread will be open for each destructable enumeration, hashtable/arrays/whatever reads inside a loop would win.

Ok for the rest of your comments.

 

[Zwiebelchen]

I don't get it, if the destructable is not anymore on the camera view why should we delay the hide ?
Only destructables which are not on the actual camera view should be hidden.

That was not what I meant. The problem is, that we can enumerate the destructables below the camera, but we can not enumerate those outside the camera (except if we loop through all destructables on the map every time, which would be stupid for obvious reasons).

So, performance wise, the best would be to store all destructables in an array, and loop through them with a periodic timer, like 50 of them with every tick and check for them to be close to the camera or not. if not, hide them, if it is, show.
This would be the fastest and performance-wise best approach, as the performance required would even out over time. However it would not be instant, as the amount of time it takes is random depending on the segment of the array the destructable is stored in.
It's okay to use this method for hiding (as this doesn't have to be instant), but definitely not for showing the destructables.

So for showing the destructables, we need a different approach.

Well, since one thread will be open for each destructable enumeration, hashtable/arrays/whatever reads inside a loop would win.

Ok for the rest of your comments.

I agree on that one. I still need to think about what would be the fastest way in this case. Certainly dividing the map into brackets is fast, but I would like a more elegant approach.

 

[Troll-Brain]

If you use an hashtable you can enum destructables which are not in the camera view and the ones which are on it (with a margin error), according to the current camera view.
You just need simple maths and roundings.

You could even use an array in fact, but you would need more code.

 

[Zwiebelchen]

If you use an hashtable you can enum destructables which are not in the camera view and the ones which are on it (with a margin error), according to the current camera view.
You just need simple maths and roundings.

You could even use an array in fact.

Hmm, I'm not sure how you could do that. Sure you could store X as parent and Y as childkeys, but then how do you loop through them? Unlike arrays, you can't tell what the next key in the table is.

 

[Magtheridon96]

I made a system just like this for SotP and I was going to post it publicly, but it needed a lot of work :/
It's pretty much broken now.

 

[Troll-Brain]

Hmm, I'm not sure how you could do that. Sure you could store X as parent and Y as childkeys, but then how do you loop through them? Unlike arrays, you can't tell what the next key in the table is.

I have a vague idea about it, i will try something this evening (in few hours) if i got the motivation.

But basically you store destructables according to the camera position in a dynamic array (its size is increased each time a new destructable is added) through an hashtable, not really their coordinates (even if it's directly related)

 

[Zwiebelchen]

I have a vague idea about it, i will try something this evening (in few hours) if i got the motivation.

But basically you store destructables according to the camera position in a dynamic array (its size is increased each time a new destructable is added) through an hashtable, not really their coordinates (even if it's directly related)

Seems very complicated. I'm not that good in creating databases. I would rather have someone more experienced than me do it, so if you or Magtheridon want to give it a try, I would appreciate it.

I'm pretty sure other people (especially RPG makers) would love a system like that, as walkable destructables are important for those kind of maps and a number of 1000 destructables is so damn easy to be reached.

 

[PurgeandFire]

Doing it based on camera is probably not the best route since they are local and would need to be synced constantly. (which may eat a lot of performance)

Tbh, I think the easiest way would be just to see how many things are in sight radius. In maps like Gaia, you can easily have control over what is in LoS and what isn't since you can easily retrieve whether a unit is in X range of something. The only problem I can think of with that method is when the entire map is revealed, but that only applies for single player w/ iseedeadpeople so you can just disable it for single player. For areas that are naturally visible, you can just not hide those dests/units.

What I would do is just do a constant check if any of those walkable destructables/units are within the sight radius of any player unit. If they aren't, then just hide them. Those invisible destructables will be invisible anyway (as will the units) when fog of war shows up so it should work well, and it is not that heavy on performance to show/hide units/destructables. This method imo is the cleanest.

It won't work for all maps, ofc. For something like an altered melee or w/e with more than 50 units, it will be performance-heavy to constantly enumerate all nearby units and destructables, but for an rpg where there are only about 12 at max, it should be pretty easy. You can even make the range large enough so that you don't have to enumerate as frequently.

 

[Magtheridon96]

We need to approach this matter using something Nestharus would most likely use, because efficiency is crucial here.

We need to use some sort of obscure tree.
TO WIKIPEDIA WE GO.

 

[Troll-Brain]

@PurgeandFire111 :

Actually we were talking about hiding the destructable in a local block, we are perfectly aware that the cameras are local, no need to sync them.
Frankly i don't believe that much in a such system for multiplayers, only for single player, mostly because of sync issues.

But i was not enough clear, i guess a code sample will be clearer, i'm going to try it.

 

[Zwiebelchen]

@PurgeandFire111 :

Actually we were talking about hiding the destructable in a local block, we are perfectly aware that the cameras are local, no need to sync them.
Frankly i don't believe that much in a such system for multiplayers, only for single player, mostly because of sync issues.

But i was not enough clear, i guess a code sample will be clearer, i'm going to try it.

I don't think that simply hiding a destructable would create any problem even in multiplayer, as long as you avoid getting a location Z on a hidden destructable and do something globally with it. Also, its not that hard to fix, as you could always just create a GetLocationZEx function, that simply shows all destructables within a small radius around the position temporarily to avoid async Z getting.
After all, we just hide the destructable. It's still there and can still be enumerated, even when hidden. I don't know what happens if someone targets of tries to kill the destructable, though. A registry for destructable IDs could be useful here, so that it only applies to "non combatant" destructables.

What I would do is just do a constant check if any of those walkable destructables/units are within the sight radius of any player unit. If they aren't, then just hide them. Those invisible destructables will be invisible anyway (as will the units) when fog of war shows up so it should work well, and it is not that heavy on performance to show/hide units/destructables. This method imo is the cleanest.

Wether you take a unit and check its position or the current camera target doesn't matter. Both positions are always accessable. The problem remains the same, you still need a fast way to enumerate through the area and hide all other destructables outside.

 

[Troll-Brain]

I don't think that simply hiding a destructable would create any problem even in multiplayer, as long as you avoid getting a location Z on a hidden destructable and do something globally with it. Also, its not that hard to fix, as you could always just create a GetLocationZEx function, that simply shows all destructables within a small radius around the position temporarily to avoid async Z getting.

No, i'm not talking about GetLocationZ, since it's already local with walkable destructables without fancy hiding stuff.
Correct me if i'm wrong but an hidden destructable has 0 collision while a visible one have some, like units, no ?
Then it's obvious that there are possible desyncs.

 

[Magtheridon96]

I have a solution.

Solution

You need to enumerate through every single platform on map initialization.
Create one unit at each position and pause that unit so that you don't cause lag.

Whenever a unit goes in (Sight Radius + Platform Radius + 256) range of a player unit, add the corresponding platform to a linked list of seen platforms.

Periodically check if the distance between the player unit and each dummy unit in the linked list is greater than (Sight Radius + Platform Radius + 256).
If this is true, hide the unit and remove him from the list.

This is how it's done.
Simple as that.

Any questions?

 

[Troll-Brain]

You're talking about the unit within range event ?

 

[Magtheridon96]

Yeah, that.
It's easier than managing a ton of camera data

 

[Zwiebelchen]

I have a solution.

Solution

You need to enumerate through every single platform on map initialization.
Create one unit at each position and pause that unit so that you don't cause lag.

Whenever a unit goes in (Sight Radius + Platform Radius + 256) range of a player unit, add the corresponding platform to a linked list of seen platforms.

Periodically check if the distance between the player unit and each dummy unit in the linked list is greater than (Sight Radius + Platform Radius + 256).
If this is true, hide the unit and remove him from the list.

This is how it's done.
Simple as that.

Any questions?

I seriously doubt that this would be faster than simply enumerating through the destructables below the cam position by moving a rect to the camera every time - especially as the UnitComesWithinRange event is also just periodic enumeration in disguise (hence why it is not instant).

 

[Troll-Brain]

Yeah, that.
It's easier than managing a ton of camera data

This is not instant like i've previously shown in another thread.
I don't believe at all on this way, not to mention the overhead : units.

 

[Magtheridon96]

It doesn't need to be instant.

edit

I seriously doubt that this would be faster than simply enumerating through the destructables below the cam position by moving a rect to the camera every time - especially as the UnitComesWithinRange event is also just periodic enumeration in disguise (hence why it is not instant).

Yeah, that could be better.
There's really no way to know without doing benchmarks.
Enum functions in warcraft III are pretty slow, as they open up new 'threads' for each widget.

 

[Troll-Brain]

Sure but this event seems too much delayed for that purpose.

But even if the delay is enough, that solution would be lame if players have units on all the map.

 

[Zwiebelchen]

There's really no way to know without doing benchmarks.

I remember someone who wanted to create a missile system and used UnitComesWithinRange to detect missile collision. After some testing, he proved that UnitComesWithinRange is nothing else than a 0.1 second timer with a periodic enumeration.

I think it won't make sense to avoid destructable enumeration by placing units on the destructables (that create an even larger performance drop than the destructables) to enum the units just to access the destructables.

I think the best way would still be some kind of tree logic and looping here.

Usually I wouldn't care about the speed, but for this purpose, speed is crucial, as the system's only purpose is to improve the performance of the map.

 

[Magtheridon96]

No, I wasn't talking about that, yes, I know that the unit in range event is a 0.125* second periodic enumeration

that create an even larger performance drop than the destructables

That's why I said you would pause the units.
Having 1000 paused units on the map is the same as having a few units on the map.
At least, that's what my memory tells me. (Nestharus' tests)

I'm going to test it again.

edit
Damn it Nes. Didn't work.
I even tried SetUnitTimeScale.

edit
Wait, you can simply hide the units

 

[Troll-Brain]

One more con : 'Aloc' units shoudn't trigger this event.

Oh and an hidden unit shouldn't be considered i think, i mean the event would be "disabled" if you use an hidden unit for this event.

 

[Zwiebelchen]

Even if it should work ... it would be damn uncool to have thousands of hidden units just to access thousands of platforms. :/

I'd rather have a solution based on arrays, if you ask me.

 

[Troll-Brain]

Me also, units are the most interactive thing ever.
I'm just arguing about if it would work or not.

 

[Magtheridon96]

It really depends on how many player-owned units you have on the map.

 

[Zwiebelchen]

I mean, using a table approach is not THAT hard, after all.

As far as I see, we need a three-dimensional array.

First value holds X values divided through the square size, second Y values divided through square size and the third holds all the destructables for those two values.

Then all we need to do is get the cam X, the cam Y, divide through the square size, get the destructables with the matching X and Y and loop through the list inside.
Then, all 8 adjacent squares will be checked.


For hiding, a periodic recycler could be used, which loops through all existing destructables; always n at a time and hides them again in case they are out of sight.

 

[Magtheridon96]

3-D array you say?

Here:

static TableArray array threeDimensionalArray

 

[Bribe]

Thread moved.

I am not sure what this was doing in the JASS Resource Submission forum, but clearly that was not the place to post this.

 

[Zwiebelchen]

Sure thing. Just wanted to get the attention of the usual villians in the jass forums the request forum usually misses.

I'm gonna write a system with the solution I mentioned above during the next days. Let's see how it turns out.

 

[Magtheridon96]

If you allow the system to do the same for units, it would be candy dandy :3
(And I can't believe I just used that phrase)

 

[Zwiebelchen]

If you allow the system to do the same for units, it would be candy dandy :3
(And I can't believe I just used that phrase)

I don't think that this would work well. Units are way to interactive to be sync-safe in multiplayer when being hidden locally.

 

[Magtheridon96]

I guess you're right.
It would need a totally different system and it would only be reliable in single player maps.

 

[Zwiebelchen]

I guess you're right.
It would need a totally different system and it would only be reliable in single player maps.

Even if you use it in single player; who knows how it would affect combat outside of the players view or computer AI? After all, the unit doesn't really exist for your machine then. All in all, I think it's not even worth trying.

 

[Troll-Brain]

IIRC, if an unit is not visible (not on the actual camera field and/or simply hidden), there is no fps drop.
So a such system would be 200 % useless, 100 % because it's not necessary and 100 % because it adds more overhead for a worse result.

Now, it should be tested again, just for the sake of knowledge.