Yes these desyncs and crashes affected replays as well as live games. Actually they were really common in replays which helped a lot with testing.
The reason why this fix was so difficult is because it's a memory corruption writing into random space. With the way your computer set up the game's memory it caused regular desyncs and crashes in replays. For a lot of other people it rarely did anything. So for Brutus's idea, this specific bug would be caught by luck in a simulation CI.
Here's how I see potential improvements compared to my experience:
- Desync tracking would automatically and seriously tell devs of deysncs. I only noticed desyncs from constantly checking forums and chats, and eventually I had to go through a specific replay search to even have any data about desyncs, and even then there wasn't a really big response. Automation would've been a more serious alert and it could've even caught it much earlier on.
- The reason this specific patch wasn't caught early was because there was 0 testing in prod. It's a tricky bug, so fair enough that some testing in dev wouldn't catch it, but since we released it without any feature using it, it took all the way up until the painting feature for its effects to show, which made the change that introduced the bug disconnected from the updates exhibiting the bug.
- With AI we can do much better reviews. This bug was a very human error that is easily picked up by AI or I guess some linter. I don't think we do very complex patches most of the time, so static analysis should work quite easily. In this sense I think we could actually speed up engine patches because of their more reliable code, but of course it's best if we just have the backup testing in prod with the two points above.