In Appearance->Peaks/Waveforms->Custom colors, disable both "Take color" checkboxes, and disable "Automatically color any recording pass that add new takes to existing media items". Make sure that "Track color" checkboxes are enabled. "Item color" checkboxes should be enabled if you want to force an override to the track color manually.