Data Synchronization Model

[Nestharus]

Currently, codeless save/load isn’t possible because it takes too long to synchronize the data between all of the players. It was previously thought that using the gamecache was the reason, which led to a new approach using orders instead, but it could be that the generous use of TriggerSyncReady is the reason.

There are a two things to consider.
1. Will synchronizing 1 player at a time be faster than synchronizing everybody at once?
2. Will minimizing the use of TriggerSyncReady make the thing fast enough to be usable?

The original model would have a function call a sync method and then a wait method. The sync method would tell the player to send a value to a table. The wait method would do a TriggerSyncReady loop and poll the table. This meant that each player would send a lot of TriggerSyncReady.

The new model uses repeating timers to poll the table instead of a TriggerSyncReady loop. Are the timers asynchronous or synchronous is the first question. If they are synchronous, this would generate as much traffic as a TriggerSleepAction call. I personally believe that they are asynchronous because they can’t call synchronous natives. I also think that evaluated triggers are asynchronous.

It’s probably safe to call the function directly from the timer. However, it may be the case that some timers will run again. If we reuse the timers instead of destroying them, this shouldn’t cause any problems.

Someone needs to compare the old model to the new model in performance. They also need to test consideration #1.

I don’t have time to write code anymore : P. All I can do is provide designs and some direction : D. For one, I have way too many resources that need updates : |. For two, I now have a job ^_^.

This is really the first piece of the puzzle that needs to be fixed. I heard that BitInt isn’t working correctly 100% of the time when the grouping isn’t 1.

Also need to bench the transfer rate for x players to determine how long it’ll take to send x bytes. Should test 2-12 players and provide a table. Anyone wanna volunteer? : )

 

[Zwiebelchen]

So this is what you were talking about back then.

And yes, this seems like a possible solution.

A different approach would completely remove the dependency from TriggerSyncReady or TriggerSleepAction, allowing synchronization to run silently in the background, not halting or interrupting the game like Network does.

My biggest gripe with network isn't how slow it is for many players, it is that it basicly polls the game until the sync is done.

For simple tasks, like just synchronizing a single value that is not needed just-in-time, having synchronization running in the background on a timer instead of waits would be a huge benefit. Now this won't help codeless save/load, for obvious reasons (because you usually want to load when the game starts, not 5 minutes in), but it will help in almost all other cases.

If I were you, I'd give up on codeless save/load for multiplayer maps. From my experience with network on semi-large player numbers (5-6 players), no optimization will ever allow it to deal with the enormous amount of data of a regular save BigInt at a reasonable amount of time (I'd say 30 seconds is the magic number here... if it takes longer than that it will be more convenient to just manually copy & paste the savecode via chat; alt+tabbing takes, what, 3 seconds?).

Network works great for syncing a single integer at 4 players or less. It's barely noticable. It's just 5 and 6 player sessions in which it seriously impacts the game (up to 5 minutes of syncing 6 players). If you could shorten this delay on bigger player sessions (or just have it run in the background, not obstructing the game), that would already be huge.


Also, wouldn't it be possible to use ForceUIKey and abilities to sync data? In that case all delay we would have would be the regular ability latency to sync a boolean. Then all we would have to do would be preventing players from interferring by randomly pressing buttons on the keyboard or selecting the wrong units.
This could be done by just using camera bounds (after all, you can't select or deselect units if your screen is covered completely by the black mask). Not sure how to avoid players randomly smashing buttons to break the sync sequence, though.
But maybe we don't even have to. What about implementing a simple checksum to find out if the player interferred and then just re-send the package if he did? After all, this is exactly the same what we do with code-based saves. In that case, a player smashing buttons would just make the syncing slower, not break it.

Let's just say we apply a worst-case 150ms bnet delay to our abilities (west-coast and europe players in one session). Syncing a 32-bit integer with just a single binary detector ability would only take 5 seconds in that case. And the best part: every player can sync in parallel!
And using abilities wouldn't even require to go binary. We can have up to 12 abilities on a single unit, so a base-10 system could be great here.

Last but not least: we can easily cover up the process (and make it non-ugly) by just using a black ability icon on the command card.


Here's what I would do in pseudo-code:

- Run a simple periodic timer (at 0.01 seconds?)
- on timer callback:
--> loop through player ID
--> if not polled[player ID] then
----> ForceUIKey (corresponding to the hotkeys of 10 black instant cast abilities)
----> set polled[player ID] = true
--> endif

- Have a spellcast event with those 10 abilities registered
- on fire:
--> set polled[player ID] = false
--> push the ability number into a base-10 BigInt
- If all data is processed:
--> check against checksum
----> success: run the load trigger from the BigInt
----> failure: start over from the beginning

Basicly this:
http://www.hiveworkshop.com/forums/lab-715/order-based-string-synchronization-260813/
... just without all the clunky SyncSelection stuff. Just have a defined maximum timeout between two ability uses and if the player deselects the unit in the process (latency between two orders exceeds maximum treshold), just start from the beginning.
All the select stuff in this script makes it much slower than it could be without this stuff.

 

[Ruke]

For two, I now have a job ^_^.

Nice!, good luck

 

[Nestharus]

I wouldn't say that codeless save/load isn't possible. We really won't know until we try benching the stuff. The question is, how much do people want it? If nobody cares, I guess it'll never be made even when we have a few possible solutions. Nobody wants attack indexing either : ). People used to want it.

Oh well, just see if anyone picks this up I guess.

 

[Zwiebelchen]

It's not that "we" don't want it, just that the people who will be able to use it are maybe a handful.