Next server update: 12.02.2021 - Preparation log

The next server update has been scheduled for the 13.02.2021 betwen 8.30 - 10.30 GMT (=UTC).

What is planned so far:

  • Migrate ratings and rating changes to new leaderboard tables
  • Upgrading NixOS to release channel 20.09 (including update of all userspace packages and a reboot)
  • Reconfigure Docker to no longer use native ZFS volumes but transparent ext4/xfs volumes (managed by ZFS). Reason: there are performance and management issues with the native ZFS storage support in Docker
  • Set up ZFS autosnapshot and tune it
  • Update all Docker base images to the latest bugfix release for FAF services (e.g. MySQL, MongoDB, RabbitMQ, Prometheus, ...)
  • Update lobby server to next version containing several bugfixes

What could come if we have enough time to make it happen:

  • Move from MySQL to MariaDB
  • Implement ZFS tuning as researched by Mozilla/Let's Encrypt (* requires MariaDB)

What will probably not come:

  • Update the replay server in order to use Zstandard compression for new replays

"Nerds have a really complicated relationship with change: Change is awesome when WE'RE the ones doing it. As soon as change is coming from outside of us it becomes untrustworthy and it threatens what we think of is the familiar."
ā€“ Benno Rice

Shame on me, just fixed the date. It will be on Friday.

"Nerds have a really complicated relationship with change: Change is awesome when WE'RE the ones doing it. As soon as change is coming from outside of us it becomes untrustworthy and it threatens what we think of is the familiar."
ā€“ Benno Rice

Small recap of todays update:

  • The whole update was done in about 1h 15min, so a lot faster than I expected (mostly thanks to @p4block amazing Linux skills)
  • We didn't receive larger complaints so far, so we didn't break anything. major. There seems to be a smaller issue with the lobby server sending queue pop times we'll need to checkup.
  • Unfortunately the MariaDB update is postponed for now due to additional work required

Side note:
Due to the ZFS reconfiguration we were now able to setup auto-snapshots of the whole system. So there are now dedicated internal backups not just of the db but of everything. We still need to improve external backups (right now only the MySQL database is subject to backup)

"Nerds have a really complicated relationship with change: Change is awesome when WE'RE the ones doing it. As soon as change is coming from outside of us it becomes untrustworthy and it threatens what we think of is the familiar."
ā€“ Benno Rice

I definitely Did not Break the Contribution Page https://faforever.com/contribution šŸ˜‰

"The needs of the many outweigh the needs of the few" - Spock