Running now the same project in Reaper with the lowest buffer size of 48 samples, 2.3/2.5ms!
Zooming in and out, selecting items, resizing time-selection like crazy, with no crackles whatsoever.
The key to my success, you ask
Installed older 2012.dec NVIDIA drivers - sometimes newer isn't better...
Case closed, I hope.