Okay, for posterity sake, you were right.
The two fixes helped me:
-Debloating AMD driver with "Radeon Software Slimmer". In particular, removing the Recording module. It does not surprise me, as AMD Recording never worked properly on my machine and crashed games half of the times I've tried to use it.
-Setting FPS-cap back from 100fps to 75 fps. While game can maintain ~100fps, in many cases it also causes 100% utilization of one of the cores. And when it does hit 100% utilization, every micro-stutter hits x10 harder. Monitoring per-core utilization is a good way to ensure optimal FPS-cap.
After that, the micro-stutter from memory garbage collection is indeed not that noticeable, subjectively.
While its still far from being a butter-smooth experience, its very much good enough to be enjoyable

Most important tidbit: It seems to be in sync with heap stats in Console -> enter "showstats" -> go into Heap. This stutter is everpresent, from the beginning of the game and even during pause.
Is it fixable or is it a quirk of the engine?

Win10, Intel i5-4590, AMD rx 560, latest drivers.
Recorded frametimes graph: