• Listen to a special audio message from Bill Roper to the Hive Workshop community (Bill is a former Vice President of Blizzard Entertainment, Producer, Designer, Musician, Voice Actor) 🔗Click here to hear his message!
  • Read Evilhog's interview with Gregory Alper, the original composer of the music for WarCraft: Orcs & Humans 🔗Click here to read the full interview.

Memory leaks - what's the deal

Status
Not open for further replies.
Level 5
Joined
Jan 6, 2010
Messages
116
Two things that's on my mind.

1) Why aren't most leaks cleaned out automatically? For example, I trigger a point for a special effect without assigning it to a variable. There is no way I can retrieve the information of that point after the trigger finishes, so why is the game storing it in memory? Why doesn't it automatically erase every triggered group and location and everything at the end of each trigger? It's so obvious I almost can't believe the engine is written that way. But since it's not, wouldn't this be a fairly easy mod to do for someone skilled?

2) If I have a leak (let say, a position entry) every 5 seconds throughout the game, does it matter? I mean, RAM is not really a problem any longer these days. Bandwith is irrelevant I think, because the information is already sent from server to receiver even if you destroy the leak afterwards.

Appreciate your thoughts.
 
Two things that's on my mind.

1) Why aren't most leaks cleaned out automatically? For example, I trigger a point for a special effect without assigning it to a variable. There is no way I can retrieve the information of that point after the trigger finishes, so why is the game storing it in memory? Why doesn't it automatically erase every triggered group and location and everything at the end of each trigger? It's so obvious I almost can't believe the engine is written that way. But since it's not, wouldn't this be a fairly easy mod to do for someone skilled?

That is how most languages work. It comes down to this question: do you implement a garbage collector, or no?

Blizzard was lazy, so they did not implement a garbage collector. As such, you have to make sure you don't lose reference to anything that needs to be destroyed.

It isn't that simple to implement though. You could hack Wc3 to do that, but that would require users to use the hacked wc3. Programmatically, maybe you could do it to some extent, but there are a lot of parameters (i.e. if someone stores it in a hashtable), and as the level of abstraction becomes higher, it may be annoying to keep track of. It might be doable, but it hasn't been done yet. Why not? Because cleaning leaks is actually pretty easy (and, IMO, it is a good thing to learn).

2) If I have a leak (let say, a position entry) every 5 seconds throughout the game, does it matter? I mean, RAM is not really a problem any longer these days. Bandwith is irrelevant I think, because the information is already sent from server to receiver even if you destroy the leak afterwards.

Appreciate your thoughts.

Every 5 seconds would be insignificant. You can get away with leaks here and there. It only becomes a problem with periodic triggers. An object created every 0.03 seconds can indeed cause performance degradation after a while, especially if it is a visual object (i.e. effects). That is why we require scripts to be leakless. We don't differentiate based on whether it is periodic or not—that would be more work for moderators, and it would be just strange (seeing an issue, but doing nothing about it). It is just a good habit in general to keep clean, leakless code, especially since you do not know how your code will be used.
 
I think of Warcraft III JASS as some kind of "practical exercises for beginner programmers". I think there's a lot of people that learned programming through WC3 mapping.

And as such, I'm very happy that there isn't garbage collection in WC3. It tought me to manage memory manually and write clean and efficient code.


It just bothers me to see how much professional software and even computer games nowadays are coded without clean and leakless coding style. I've played games that got literally unplayable after like 60 minutes into the game. Just play a flash game here and there and you will KNOW what I mean.

I've worked with measurement softwares and tools that literally stopped working after 30 minutes of being connected with USB.


Did Warcraft III need garbage collection when it was released? Hell no. The game itself was written around the multiplayer, but not the actual funmap scene. I guess nobody expected the high quality mods we got nowadays. Those were just additions to increase long-term appeal of the game and I bet nobody at Blizzard ever thought the game would still be played after almost 15 years due to the mapping scene.
 
Level 5
Joined
Jan 6, 2010
Messages
116
Sounds extremely useful, Ralle. I'll definitely use your tool at a later point. Cool thesis you've got yourself btw.

Thanks for your thoughts, everyone. It was an interesting read. Though I still wonder if a leaky map causes problems concerning bandwidth and that kind of lag, or if the problem is limited to clogging up local access memory.
 
Sounds extremely useful, Ralle. I'll definitely use your tool at a later point. Cool thesis you've got yourself btw.

Thanks for your thoughts, everyone. It was an interesting read. Though I still wonder if a leaky map causes problems concerning bandwidth and that kind of lag, or if the problem is limited to clogging up local access memory.

It is local access memory, which can often degrade FPS (frames-per-second) performance after some time (usually with visual leaks, such as effects). AFAIK, it won't cause actual lag ("latency" lag). "Lag" colloquially refers to both frame-rate and latency lag, but technically it is only supposed to refer to the latter.
 

Ralle

Owner
Level 79
Joined
Oct 6, 2004
Messages
10,182
One important point is that Warcraft is synced between people. If somebody starts to slow down due to lag, the whole game must slow down with him. So a single weak link (slow computer etc) will slow down the whole game. Memory leaks will make this worse. In the worst case you have the game split I think.
 
Level 12
Joined
Feb 22, 2010
Messages
1,115
Slow computer only affects its owner, trust me I am playing with onboard graphic card.
 
Level 23
Joined
Apr 16, 2012
Messages
4,041
It shouldnt cause any bandwidth problems because wc3 doesnt pass local variables and stuff over the network, it merely passes orders and few events(player quitting game, disconnecting, desyncing etc), because that is all the other client needs to run perfect simulation of the game on his end as well
 
Level 5
Joined
Jan 6, 2010
Messages
116
Slow computer only affects its owner, trust me I am playing with onboard graphic card.

Depends if you talk about graphics or memory failure. Regarding memoery, every computer has to keep up with the information going on, as Ralle said. I just experienced this on my grand-scale map, which unfortuntely still has some heavy leaks that must be plugged; After approximately 45 minutes of play the weakest computer started to struggle, and all other players constantly got the wait-sign. None of the other computers had any problems, so it was seemingly directly tied to the amount of RAM you have available. Twice the amount of RAM can probably run a leaky map twice as long.
 
Level 23
Joined
Apr 16, 2012
Messages
4,041
from my tests, this is not really right, because I was generating endless leaks and units(locations and units to be precise) and after about 1 Gig of ram it just said I need to insert disk to CD/DVD drive and it crashed. Maybe I should try leaks without units
 
Level 12
Joined
Feb 22, 2010
Messages
1,115
Depends if you talk about graphics or memory failure. Regarding memoery, every computer has to keep up with the information going on, as Ralle said. I just experienced this on my grand-scale map, which unfortuntely still has some heavy leaks that must be plugged; After approximately 45 minutes of play the weakest computer started to struggle, and all other players constantly got the wait-sign. None of the other computers had any problems, so it was seemingly directly tied to the amount of RAM you have available. Twice the amount of RAM can probably run a leaky map twice as long.

Yeah you might be right, about it is only for graphical things.But are live in 2014 and nobody cares about even 100mb memory leak.

Also another thing with memory leaks, the more memory leak you have the more time you will spend when you press "ok" button at score screen.Probably game is clearing all memory about map.
 
Level 17
Joined
Apr 27, 2008
Messages
2,455
Also the jass vm should be less and less efficient as the ram used gets bigger, especially if there are handle id leaks.
Anyway, most people ask to "how to remove leaks", they are taking the wrong way it should be "how to prevent leaks" and learn what's a leak and how to not make one.
The "things that leak" thread is a pity, every dude is asking the same question again and again.
 
Level 12
Joined
Feb 22, 2010
Messages
1,115
Because it is impossible to understand leaks if you don't know jass coding or any programming language.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,255
1) Why aren't most leaks cleaned out automatically? For example, I trigger a point for a special effect without assigning it to a variable. There is no way I can retrieve the information of that point after the trigger finishes, so why is the game storing it in memory? Why doesn't it automatically erase every triggered group and location and everything at the end of each trigger? It's so obvious I almost can't believe the engine is written that way. But since it's not, wouldn't this be a fairly easy mod to do for someone skilled?
That is a very big question most of us have asked for ages. The only practical reason we came up with was that WC3 is horribly programmed as is evidence by many problems from OOS errors due to terrain deformations to crashes when units go out of bounds. This likely has routes with the origins of WC3 as a MMORPG and inexperience of Blizzard's dev team to RTS games of such flexibility.

What is even more astounding is that WC3 JASS uses an indirection table type known as handles in any case. This makes garbage collection considerably easier to implement and something one would expect in such a system. Speed is not even an excuse as the automatic garbage collection would likely be as fast or faster than manual object deallocation (what JASS uses) due to the JASS interpreter overhead. In C++/C garbage collection is often avoided as it introduces problems and overhead, especially with concurrency and real-time systems, that can be avoided by manual destruction. In the end manual destruction should be more efficient as the program then is implicitly told when an object is finalized (part of active job) as opposed to having to waste resources looking for objects that require finalization (separate dedicated job). On the other hand garbage collection can eliminate problems related to memory fragmentation if implemented using an indirection table with little extra overhead (the memory moves and indirection table would still be needed to achieve the same effect with manual deallocation).

Do not fear though as Blizzard has improved massively over the last 10 years. SC2 (the spiritual successor to WC3) has a very effective garbage collection system that will remove points, unit groups, player groups and strings automatically based on reference counters. Only by doing very stupid scripts can SC2 scripts leak.

2) If I have a leak (let say, a position entry) every 5 seconds throughout the game, does it matter? I mean, RAM is not really a problem any longer these days. Bandwith is irrelevant I think, because the information is already sent from server to receiver even if you destroy the leak afterwards.
Memory is not the critical factor, it is memory locality (for caches) and finite engine resources that cause the problem. Enough objects will cause the engine to crash due to some internal limit being exceeded.

Memory bandwidth has only to do with locality. If you mean network bandwidth then you have the wrong idea how WC3 works.

WC3 uses fully synchronous clients with a relay server. The server itself does not run the game session (light weight) where as the clients run the sessions in parallel (heavy weight). This means that hour long sessions of WC3 may only generate a few MB of traffic, probably a lot less than using facebook or even Hiveworkshop for that time.

This is the opposite approach of heavy weight server and light weight clients (as used by Diablo III and WoW) streams game state to the clients that is computed by the server. This approach is required when the state of the game is too complex to mirror with all clients (like a MMORPG) and where latency is critical (FPS games). The latency reduction comes from synchronous servers having at best a Round Trip Time delay (command to server, sever synchronizes command for a time and broadcasts command to all clients) where as light weight steamed state clients can achieve much less (command to server, sever executes command and the results can be streamed at a later time in the future, aka no return delay to execute the command).
 
Level 5
Joined
Jan 6, 2010
Messages
116
Thanks for your thorough and insightful post.
Memory bandwidth has only to do with locality. If you mean network bandwidth then you have the wrong idea how WC3 works.
Rather than having the wrong idea, I think my problem is having no idea. But from your explanation it seems I jumped to the right conclusion (that network bandwidth is irrelevant) but from a wrong assumption.

It seems everything shifts more and more toward heavyweight server/lightweight client these days. I guess that is owing to the substantial increase of network bandwidth we've seen over the last decade.
 

Dr Super Good

Spell Reviewer
Level 64
Joined
Jan 18, 2005
Messages
27,255
It seems everything shifts more and more toward heavyweight server/lightweight client these days. I guess that is owing to the substantial increase of network bandwidth we've seen over the last decade.
It is entirely dependant of the type of game being made.

RTS games use light weight servers and heavy weight clients since you cannot stream all the results. Even on fast connections streaming the results of hundreds of units is a challenge for both server and client.

FPS and RPGs use heavy weight servers with light weight clients since the number of streamed elements is small and you need the heavy weight server anyway to prevent cheating. Compare Diablo III on PS3/XBox360 with PC where the consoles have heavy weight clients (well one of the clients runs the heavy weight server, the rest are light weight) and are plagued by cheating (you can just generate your items by modifying saves) yet the PC version has a heavy weight server and light weight clients and has no cheating at all (you cannot instruct the server to give you items outside of game rules).

The main problem is that few people want RTS games anymore.
 
Status
Not open for further replies.
Top