Experimental WC3 loading performance booster

[Dr Super Good]

Yes, just noticed the pastebin entry expired... I have ziped it and attached it to the post this time (posts do not allow jar files to be attached directly).

The version is slightly improved in that it should be more CPU efficient (requests 1 byte per page instead of iterating through all bytes).

 

[Chaosy]

I just downloaded it, when i start it nothing happens so i guess it works.

tho am i needed to restart the comuter first.

also am i needed to run this everytime i start the computer?

 

[Sliver18]

Dr Super Good, this is a really nice idea for a wc3 supporting program. I was really surprised seeing this - ten years after the game entered the world... Nice that good coders are still out there

And because most people not seem to know WHY they should USE THIS PROGRAM, here's one reason:

When testing a map in the world editor, i change a little bit here, i change a little bit there, and then run the map. 12 seconds loading time are normal on my machine. But if i do this every 5 minutes because i wanna see my changes behave correctly in the game, 12 seconds waiting again and again is terribly annoying. So why not assist the game with an additional chache

I definitely will test your program tonight full of pleasure

 

[Bribe]

Hey, Sliver18, you're totally right. I wish my netbook could support more than 2GB of RAM, otherwise I would have bought more.

 

[Sliver18]

And there is my pleasure...

Starting my huge testmap without WC3BOOSTER took 24,8 seconds (dont panic, i had several memory-intensive tasks running parallel)

WC3 Closed. WC3 restarted.

Starting my huge testmap three times with WC3BOOSTER took 20,1 to 18,5 seconds, thats an improvement of 17 - 26 percent (!!!).

Dr Super Good, thank you very much for this nice tool!

I wanted to check how much space it required in memory, so i wanted to check this in my task manager. I just couldnt find it - how does it work? Does it prioritizes the memory pages and then ends itself after the job is done? Or will it continue running?

Just for the case that the second thought is true (continue running) - a little improvement would be to show a tray icon while the program is active - otherwise unexperienced computer users really don't notice that the program runs and think it doesnt do anything at all.

EDIT: For our Doctor's test statistics:
OS: Windows 7 Build 7601 (64 bit),
CPU: i3, 2.4 GHz x 2
RAM: 4 GB
Java: 1.7.0-b147 (64 bit)

 

[Bribe]

Really nice that it's basically a free performance increase as well. Usually you pay for this sort of stuff.

 

[Sliver18]

Usually you pay for this sort of stuff.

Exactly.

 

[Dr Super Good]

It abuses the File Cache Windows systems keep. Unless specifically ordered, all files are loaded in pages (similar to virtual memory). When data from a file is no longer needed (all references to the file or certain pages of the file are destroyed) the pages are set to be recycled. However the File Cache keeps a table of what pages are still in memory even if they have been freed. The page table also allows for different priorities of recycling so it might take a considerable while, with enough memory, for the system to free the pages although it will still allow them to be freed incase you need to use the memory. If a page of a file is still in the page table, even if the file has not been used for a while, then a soft fault is generated which is resolved nearly instantly compared to a hard page fault which is what usually happens when the game has to load data from disc.

I do admit the implementation is rather poor. Ultimately you would want access to the File Cache and/or Paging system directly so that the pages can be force loaded in without ever going through a process and the priorities set high enough that they will be guaranteed to not be recycled for a considerable time if memory allows. However I cannot find documentation for such libraries.

Also not all data needs to be loaded. The same performance boost you experience using this tool could probably be achieved by loading only 33% of the data if you filter out all the data that is never used (such as the low quality models, obsolete files that were replaced by patches and things like that). This was done as a proof of concept to show that one can get performance better than a SSD with a mechanical hard drive by using the insane quantities of memory modern systems have.

 

[Sliver18]

Thanks for the explanation!

I do admit the implementation is rather poor. Ultimately you would want access to the File Cache and/or Paging system directly so that the pages can be force loaded in without ever going through a process and the priorities set high enough that they will be guaranteed to not be recycled for a considerable time if memory allows.

Couldnt you then set a "shared reference" so that there always is a reference? Like this, you could prevent windows from garbaging the page, couldn't you?

Or is it possible to refresh this priority in a periodic time so that the moment when the page should get deleted is never reached...?

Also not all data needs to be loaded. The same performance boost you experience using this tool could probably be achieved by loading only 33% of the data if you filter out all the data that is never used (such as the low quality models, obsolete files that were replaced by patches and things like that).

...but cause the program cannot know what will be used and what not, theres no other possibility to load all, understand. But i think my performance results are still enough!

 

[shinji]

Does this always need to be run every time I open my computer?

 

[Sliver18]

Does this always need to be run every time I open my computer?

Yes, because your memory gets totally freed after u restart your pc.

 

[shinji]

There's a minor bug. It's minor.

When I run this program, it triggers some programs to run.

 

[Dr Super Good]

Couldnt you then set a "shared reference" so that there always is a reference? Like this, you could prevent windows from garbaging the page, couldn't you?

Except in times of low memory one wants the pages to get garbage eventually. For example if you were to go play SC2 after WC3 then there is no reason for WC3 to stay in memory.

In any case you can make them application level pages which are almost never paged-out until memory is depleted.

Or is it possible to refresh this priority in a periodic time so that the moment when the page should get deleted is never reached...?

All you need to do is get an application to touch the pages periodically to achieve this. However it is not recommended as the file data should remain in memory long enough unless your system starts running out of free memory (in which case you want the files to be freed as hard page faults of processes kill performance).

...but cause the program cannot know what will be used and what not, theres no other possibility to load all, understand. But i think my performance results are still enough!

Some data is guaranteed to never be used. An example is WC3 files in the MPQ which have been patched. The files are loaded only from the patch MPQ so loading them into memory for any other MPQ is a waste of memory. Additionally some such as low quality art can also be ignored as no-one now days should not be maxing WC3.

When I run this program, it triggers some programs to run.

That is very helpful...

It should start only the JVM (needed to run JAVA byte code and only if not running already) which will then create a new process for it. Upon completion both the process and the JVM should get shutdown (unless other processes are using the JVM). No other processes will start as I have not put in any code to start other processes.

If other processes or programs do start then it is highly likely you have been infected by a virus. If you do not believe me I can even send you the source code.

 

[shinji]

@Dr super good

I see. In going to format my computer soon. It's getting sloooow. Maybe because my brother downloaded too much junk.

EDIT: And yeah, another bug.

What I run this program, some flash games in the Internet lags.

 

[Dr Super Good]

What I run this program, some flash games in the Internet lags.

This must be a problem related to your computer setup rather than the program. The program literally takes the game data and places it into memory and then allows it to join the file cache. Once complete it should not degrade performance of other processes at all as the memory used can be freed at any time by the Kernel. The only time it might impact performance is when the process is still loading data into memory or you ran it without enough free memory to fit all the data files in.

 

[shinji]

This must be a problem related to your computer setup rather than the program. The program literally takes the game data and places it into memory and then allows it to join the file cache. Once complete it should not degrade performance of other processes at all as the memory used can be freed at any time by the Kernel. The only time it might impact performance is when the process is still loading data into memory or you ran it without enough free memory to fit all the data files in.

Looks like I really need to format it. Thank you.

 

[Zwiebelchen]

Additionally some such as low quality art can also be ignored as no-one now days should not be maxing WC3.

Wow, you might not know how damn wrong you are here.

In Gaias Retaliation ORPG, the map stops to load at some point for some unknown reason when the Texture Level is not on highest setting.
The amount of reports about this bug was so overwhelming that we had to include a warning into the loading screen itself that tells the player to maximize their graphics settings.

So from this experience, all I can tell you is that there seem to be hundreds, if not thousands of players out there not having everything set to max.

 

[Dr Super Good]

However loading such models when a player is playing on high is a waste of memory. Equally well loading high content when a player is playing on low and low equivalents exist is also a waste of memory. You could make the program fetch the visual settings from the registry directly and adjust its load pattern accordingly.

 

[Bribe]

I can max WC3 on my ASUS 1005px netbook and get decent framerate.

Unfortunately, WC3 doesn't look graphically exciting even then. The detail in the graphics were never the part I enjoyed about WC3, it was always the concept of the graphics. Part of what made StarCraft/WarCraft II so great, because your imagination filled in the "missing" pixels.

 

[Zwiebelchen]

I can max WC3 on my ASUS 1005px netbook and get decent framerate.

Unfortunately, WC3 doesn't look graphically exciting even then. The detail in the graphics were never the part I enjoyed about WC3, it was always the concept of the graphics. Part of what made StarCraft/WarCraft II so great, because your imagination filled in the "missing" pixels.

Totally agree here. I love the art style of WC3 a lot even if the graphics engine is terrible and outdated. It's the same with a lot of old games. Sometimes, leaving room for the player to use imagination is better than showing off everything like a movie.
That's why some classic rpgs even older than WC3 still get my attention nowadays. Especially Infinity Engine games. I actually spend years on designing fanart and thinking about how a movie version of those games would look like.
When playing modern games, you will never experience this again. You can not 'dive in' as much as you did back in the days of 2D and early 3D.