Dismiss Notice
60,000 passwords have been reset on July 8, 2019. If you cannot login, read this.

Reforged [DESYNC] - 2 Possible causes found

Discussion in 'Patch & Reforged Discussion' started by Blarto, Mar 9, 2020.

  1. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    (UPDATE) they fixed Walkable destructables as of recent patch, but issues relating to GetLocationZ will remain.

    GetLocationZ can return different values if Reforged user and Classic user are on same game and the following conditions apply:

    1) you use platforms or destructables that are tagged "walkable" and use reforged assets (this means they have different local Z values as the models themselves will be different)
    2) GetLocationZ sometimes returns different values on DEFAULT terrain for some reason (RIP)
    +
    3) you use a system (e.g Missile) which relies on some sort of GetZ that has GetLocationZ embedded in it, and/or use GetLocationZ to set global wc3 values such as unit height

    =
    Lets assume we use Missile jass library
    Shoot 1 projectile across a platform or faulty terrain (again WTF), and reforged missile will have different Fly Height than Classic Missile

    =
    bye game



    I can't be bothered to post screenshots and the map that you can replicate this with, at the moment. Just take my word for it.

    I'm trying to figure out if there can be any viable replacement for GetLocationZ instead. And no, the BlzGetUnitZ are not viable as they are documented to be Local as well.

    Most likely the only solution will be to synchronise (using TriggerHappy's sync library) player 1's GetLocationZ results across a predefined grid and use some sort of algorithm to "GetSmoothZ" or something.


    Edit:
    I found another cause related to force bridges (the transparent ones). Not sure if it applies to regular bridges. I don't know the mechanics of it yet. Shooting a projectile (Missile library, locust/xefx units) over to stuff located on it from water (all GetLocationZ calls removed) desyncs Reforged x Classic games.
     
    Last edited: Apr 28, 2020
  2. DracoL1ch

    DracoL1ch

    Joined:
    Dec 12, 2010
    Messages:
    2,037
    Resources:
    2
    Tutorials:
    2
    Resources:
    2
    if GetZ is used to do something NOT visual-only then it's flawed system and it must be reworked. Z arent synced for any version you can think of.
     
  3. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    Yes. Standard procedure was to remove all shockwave and warstomp based spells to make GetLocationZ safe, since everyone pretty much had the same terrain data after doing that.

    Reforged changed that.

    My theory is that Reforged renders "Steep Terrain" differently. Steep Terrain is when you remove the smoothing factor in World Editor to make some really steep cliffs (useful to make high quality terrain).

    A 256x256 map has 1,000,000 32x32 cells, which is approximately 32MB of Z data. Probably the best way I can think of right now to create a new GetZ is to just scan the map terrain file with a tool and generate jass/lua that saves the XYZ coordinates to a Hashtable for fast retrieval. Unfortunately that's about 100,000-250,000 lines of jass code on top of your script.
     
    Last edited: Mar 9, 2020
  4. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    GetLocationZ was still unsafe between Windows/Mac players. It even warns you in common.j:
    If you need synchronized data about the terrain height, then scan the map using one player and synchronize the data using `BlzSendSyncData`. It might take a while but it'll work. The approach you're suggesting will also work.

    Also, I don't know where you got that 1,000,000 figure from. A 256x256 map has 257x257 height points. That's 66k points. You also don't -really- need full 32 bits of precision most likely, and can represent the height in your map using 16 bits. That's 128 kilobytes. Easily embeddable in Lua maps as a string which can be also easily decoded at init time.
     
  5. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    The point was that it was safe before because terrain was rendered equally (most libraries relying on xefx + missile will have used GetLocationZ as there is no other way to make smooth projectiles otherwise). Because of equal terrain rendering, async GetLocationZ became effectively synced by default (as long as no shockwave or terrain deformation stuff happened). Yeah you might have lost some mac users, but the probability of a mac user in your game was nonexistent. Now with Reforged, every other person has desync potential.

    32 is the lowest "grid" size the world editor lets you see with "G" hotkey.

    256x256 map has 32000x32000 cooridnates, 32000x32000 / 32 = 1000x1000 = 1,000,000 = 1 million integers recording the value of "Z" each point using 32bit each = 32mb (ish)

    Some of the coordinates are trash due to boundaries.. reality is like maybe 28000x28000, and much of your map terrain may not be pathable in any way (so don't need to account for those places).

    You need a fast GetZ function that returns proper value even if the XY coordinate does not align on one of those height points. To use 256x256 height points you'd need some sort of algorithm to smooth out/approximate the Z based on the 4 height points it is caught in between (hence my mention of "GetSmoothZ" function in first post). I don't think this is good as GetZ is used in Missile logic (i.e GetZ calls every 0.03215 seconds per missile). Its better to save 1,000,000 coordinates, and have the "smooth out" function just be R2I(x), R2I(y), and it will inevitabely fall on one of the 1,000,000 coordinates with acceptable accuracy.

    Edit: come to think of it, you could probably adopt a hybrid approach, and extrapolate the 1,000,000 points at run time (at map init or slightly after map init) from the 65,550 or so points you get from 256x256 grid... 65k is still a ton of data to sync though. can probably cut this value by half or more since as you mentioned you only need 16 bits (pack 2-3 zs in 1 integer).

    Syncing 1 million values is not viable IMO. Sync capability of wc3 have limit of about 5000 integers per minute unless they updated their net code greatly.
     
    Last edited: Mar 9, 2020
  6. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    The heightmap in WC3 works on the tile grid, though, not on the 32x32 grid. All the in-between values can be interpolated linearly. Recording them is basically redundant data which you can get out of your 256x256 results anyway.

    If you want a fast approach, then you should just use Lua and stop thinking in the JASS way. This is trivial to do in Lua with performance that will not be a concern.

    If you want to do this in JASS, hashtable performance will degrade with that many values in your hashmap.

    Relying on undocumented, unsupported behaviours where the official API explicitly states that such usage is not supported, is a great way of shooting yourself in the foot. Which is exactly what's happening here.
     
  7. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    Not all maps are convertable to lua. Some maps are 10 years old + with no map maker to update them (+ the dumbass map makers protected them so yeah bye map).

    Also, how do you convert 100,000 lines of jass code to Lua without paying $30,000k+ for a developer to code & test all that shit anyway.


    So official libraries by vexorian and hive's own Missile library that were tested and confirmed to work for 10+ years are a great way to shoot yourself in the foot. Not going to debate on whether they were correct to rely on GetLocationZ or not. It is what it is.

    This isn't a thread to argue about jass vs lua or how to code your stuff. This failure is a regression issue introduced by Reforged x Classic compatibility.

    Easiest way to counter this regression is just save your map to Classic-only with new editor (and "accept" new EULA as a consequence). That is, if you actually have access to the map. Also, by doing this you break compatibility with Netease and Gameranger, which combined are more popular than bnet.2.0 wc3. So yeah. Have to maintain separate versions. Balls. Easy to just keep to classic Jass and try to solve this issue with that tool I guess.

    Edit: if you don't have access to the map you can try your luck by replacing GetLocationZ with 0.0 in the map's script with an MPQ editor. It will mess some logic up but atleas't you won't desync anymore. That's all I got for now.
     
    Last edited: Mar 9, 2020
  8. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    I don't understand what is the exact issue you're jabbing at here.

    Is the issue that this breaks old maps? If yes, then I agree. It's a bummer. But it's what happens when you rely on age-old bugs. Tons of other maps (especially those using typecasting hacks) also broke. Mapmakers fault for relying on unspecified behaviours.

    Is the issue that the old trick doesn't work anymore in new maps? Then I disagree, because there are ways around it.

    If you want to force classic-graphics mode on an old map, it should be sufficient to just replace the war3map.w3i file inside the map with a new one which has the correct flag for SD-only mode set.

    You were going on about how to solve this issue, so I assumed you weren't talking about old maps.

    This is disingenuous nonsense. All you need to do is to set your map into Lua mode. WorldEdit ships with a JASS->Lua transpiler, and that is in fact how it builds GUI triggers in Lua. It just transpiles the JASS code into Lua. There might be a few corner case bugs you will need to fix, but it definitely will not cost anyone $30,000.
     
  9. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    Just stop. 5 star system on hive using GetLocationZ -> [vJASS] - Missile all maps using it are now at risk. I don't care who's at fault and neither should you or anyone else reading this. It is what it is.

    Have you tried this? do you know what happens? I'll tell you: Your computer crashes when trying to open map in wc3 1.32 reforged (100% CPU usage nothing works).

    This issue applies to old and new maps. I provided some solutions but I'll make a more permanent one soon. Probably. (I.e a fast GetSyncedLocationZ)

    Good to know. But even if it costs $1000 worth of your time, its still not a workload you should be doing, in my opinion. Your time is better spent on improving the map itself, and once you reach 50k lines, you're in a "damned if you do, damned if you don't" situation. For reference, $1000 is 3-14 days of software contracting depending on qualifications.
     
    Last edited: Mar 9, 2020
  10. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    Wow, I didn't know mappers contracted other people to develop their WC3 maps for thousands of dollars. Super relevant points you're making. /s

    I don't understand the point of this thread. Is this a bug report? A demand to Blizzard to maintain backwards compatibility (they won't)? If not, why is this in the patch forum? If you're trying to come up with a solution, this should probably go in the lab.

    You offered some solutions. I pointed out the errors in your estimates as well as the potential pitfalls you're going to face if you stick to JASS. And the fact that relying on specifically unsupported behaviour was dumb in the first place, and I don't understand how that library could've been approved by any code moderator worth their salt. You're getting oddly defensive about this.
     
  11. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    "Is this a bug report?" no.
    "A demand to Blizzard to maintain backwards compatibility (they won't)" no.
    "if not, why is this in the patch forum?" because its an issue introduced in wc3 reforged.
    "this should probably go in the lab." no code = no lab.
    "You offered some solutions" yes I did.
    "I pointed out the errors in your estimates as well as the potential pitfalls" yes you did.
    "And the fact that relying on specifically unsupported behaviour was dumb in the first place" nobody cares. Maps are desyncing and people lose interest in custom gaming. We all suffer.
    "and I don't understand how that library could've been approved by any code moderator worth their salt." Let us know if you have a better Missile system, lua or jass or vjass, that doesn't rely on GetLocationZ to animate its flight path
    "Super relevant points you're making." I'm saying you're wasting 1000 dollars worth of time to convert old jass to lua for little to no benefit, instead of just improving your map's features (user's mostly don't care about performance or what language map is written at)
    "You're getting oddly defensive about this." get lost, drama monger.
     
  12. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    ? The lab is a perfect place for technical discussions like this.

    upload_2020-3-10_4-31-14.png

    With the new natives it should be possible to stop using units for missiles. specialeffects can have local positions that will not desync, thus allowing you to use GetLocationZ in a way that doesn't need to be synchronized. I'm not going to do this for you, though, because I have better things to do.

    I thought people developed maps during their free time, which, naturally, they don't get paid for. If the choice is between working on your maps features, or making your map work at all, I think I know which one I'll take.
    Also, hot take, but sometimes people develop for their own pleasure, not for the pleasure of other people.

    Oh no, you poor snowflake. Are you mad because you can't handle a little forum confrontation?
     

    Attached Files:

  13. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    This was a discussion. Now you're hijacking the thread with irrelevant nonsense. But your contributions are appreciated and noted. Now go find the next shiny object to mull over and get lost.
     
  14. mori

    mori

    Joined:
    Jun 13, 2016
    Messages:
    446
    Resources:
    2
    Tools:
    1
    Tutorials:
    1
    Resources:
    2
    Good job continuing to derail your own thread further, though.
     
  15. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    So I'll address some the proposed "solutions" in more detail before signing off:

    This works but you'll have to rewrite all of the missile libraries + xefx. And nobody is going to do this for you, which I estimate will take around 2-4 weeks worth of work to develop and iron out all the bugs (replace xefx +create new missile library). Performance wise it will be worse on moving the missile (instead of SetUnitX, you'll need to setFX_X + record X to get GetFX_X, since SFX are local), but better on creating the missile (CreateUnit vs CreateSFX). Overall effort is not worth it. Probably worth it for new maps though. There is supposingly a new ParticleAPI too but I'm not sure its exposed in 1.32 (it is on Netease, DZ api etc).

    Worth repeating that this trick crashes your computer. Do not do it. I tested multiple variants of this trick.

    Oh if only you knew more about this. Talking about this is against new EULA though... Probably.

    No it isn't. This thread is to let people know (that is, for their benefit) what possible reason could happening if their map desyncs. That is, Missile system with GetLocationZ to regulate its height. Nobody cares about the Lab section. This section is more visible, which serves this purpose just fine.

    This is a typical troll behaviour to attempt to enrage someone and have them start flaming or lose their shit or something. Notice how this guy never offered any applicable solutions or advice aside from "git gud at wc3 editor" or "remake your map" or "sucks to be you".
     
  16. Drake53

    Drake53

    Joined:
    Jan 1, 2018
    Messages:
    447
    Resources:
    0
    Resources:
    0
    2 weeks to write a missile library? rofl

    Source?

    This map supposedly uses this 'trick' as well, haven't seen anyone complain about it.

    I like the lab...

    :peasant-popcorn:
     
  17. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    Don't underestimate how long it takes you to write some system piece of code that must be optimal and work under all circumstances (where 1 small glitch can break an entire map playthrough and force a restart).

    Yea its roughly 3 day for dev 2 weeks for testing (1-hr/day live games) + 3 more days fixing bugs over these 2 weeks

    Source?

    I actually written one and it took about half as long (i aborted due deciding original xefx is better for regression and performance reasons). You have to rewrite xefx to use effect handle, as well as missile library to use new xefx, and handle all regressions from. We had a few testing sessions on many use cases to iron out some bugs.

    Performance is worse because SetX/Y for unit inlines with vexopt to 1 line call whereas SetX for BlzSetSfxX will not inline because you need another line to record the x/y (BlzSetSfx is local so there is no synced GetX/Y). Treating units as a datastructure also has some other implications, such as you can attach multiple effects to 1 unit to make missile trails (a common use case of attaching ribbons). If you relied on SetSfxX/Y you'd have to update each and every one of them, whereas with old xefx you just moved the unit. This means there are less stress on SetX/Y with unit-based xefx than moving effects.

    Of course if you use lua it doesn't matter as lua engine performs better than the jassVM due to lua runtime being "closer" to the OS and consequentially closer to the processor/memory stack.

    Edit: you can achieve optimal performance by removing dependency on xefx all-together (effectively in lining effect native in missiles), but you'll introduce regression issues for already existing code that assumes Missile structs have xefx. In the aforementioned library I written, I decided against it because i'd have to update around 100+ missiles + basically lose support for wc 1.26-1.28 (no BlzSetSFX natives there).

    Good. If I make a jass/lua library to have a SyncedGetZ, you'll find it there.

    I don't think so. It uses new native API (as evidenced from screenshots + UI difference), so they probably just saved their map in 1.32.
    This doesn't work if the map you're trying to play is not yours and "protected". Copying .w3i directly to mpq file (and/or deleting old .w3i) either corrupts the map or make it crash your computer.

    If anyone knows a way to have this trick work I would be most interested, and so are many other map makers.
     
    Last edited: Mar 10, 2020
  18. Drake53

    Drake53

    Joined:
    Jan 1, 2018
    Messages:
    447
    Resources:
    0
    Resources:
    0
    I know coding takes time and it's easy to underestimate, but when you say 2 weeks I'm assuming that's 8hrs per day. "2 weeks testing" and "fixing bugs over these 2 weeks" doesn't sound like you're working full-time.

    Not to sound rude, but I really don't care how jass performs. It's obsolete and only used by people who update old maps and/or refuse to learn the newer/better languages.
    When I asked for source I meant benchmarks (using os.clock), not speculation why you think one system may or may not perform better than another.

    Looking forward to it ^^

    Ah, I did not read that we were talking about using the SD-only feature in the context of old maps. It's true the map I linked was saved in 1.32 editor.

    You could check out this. With some minor adjustments it can work the other way and 'upgrade' the .w3i to 1.32. I cannot test it myself though, since I uninstalled reforged.
     
  19. Blarto

    Blarto

    Joined:
    Jan 17, 2010
    Messages:
    150
    Resources:
    1
    Spells:
    1
    Resources:
    1
    You should know nobody cares about Jass enough to make benchmarks for the new natives. I can't be bothered to do them myself since I won't benefit from it in any capacity. I edited the message to say "Blz native" specialised Missile will be faster as it removes dependency on xefx alltogether. But that only applies to new maps with no regression risk, which by that point might as well use lua. So probably no point even discussing this further as new missiles using BlzSetFX will blow SetUnitX/Y out of the water.

    There is still a sizeable portion of the wc3 community that still stuck on Jass both on battle2.net and elsewhere (small thanks for Blizzard botching 1.30, 1.31, and 1.32 patches). So the "only used by people who update old maps" is not a small only.
     
  20. Drake53

    Drake53

    Joined:
    Jan 1, 2018
    Messages:
    447
    Resources:
    0
    Resources:
    0
    That's exactly what I said. FYI, os.clock is a lua function.

    Also, when I said "update old maps", I was mostly talking about people who play on 1.32, so yes that is a small only. But you're right, jass is only used by people who update old maps and/or refuse to learn the newer/better languages and/or refuse to play on 1.31 or higher. Hope I didn't forget anyone this time.

    I'm probably biased due to lua addition, but I don't get the hate, what was wrong with 1.31 again?