Gradual Freeze / Output Caching

[geraintluff]

Feature: the option to "gradually freeze" tracks during the course of normal playback.

-----------------------------

Problem: I use a sampled orchestra synth on a 3-year-old machine, and it sometimes struggles. I end up freezing / rendering-and-muting sections of the orchestra depending what I'm working on, but I'm blocked/idle during each freeze/render.

However, it's possible to mute a track, and un-mute it later - even in the middle of playback. With the right settings enabled, this stops all processing on the track (or at least stops sending MIDI notes to synths).

Solution: If you monitored the output of a track, you could identify sections of output with sufficient silence at either end. You could cache this result, and essentially mute/unmute the track during the starting/ending silence, replacing the output with the cached audio.

All cached sections would be removed when the user does anything that might invalidate the cached result. There should be an option for whether these cached sections are used for proper freeze/render or not.

Caveats/Extensions: This is not a proper freeze, because we have not unloaded the effects (so it reduces CPU but not memory use). The user may or may not want to upgrade this to a proper freeze if our cached regions span the entire project length.

-----------------------------

Possible details:

Start caching - tracks are eligible if:

1. The user has enabled caching for this track
2. Track is not record-armed (or receiving from record-armed track, etc.)
3. Track has not been "touched" recently (basically any interaction/edits/solo)
4. Track's output has been silent and MIDI-free for some interval (e.g. 3s) or since the beginning of project playback, whichever is closer

It might only attempt to cache one/N tracks at a time, and it should try different ones each time. It might bias towards caching tracks with higher CPU use first.

Stop caching

Tracks that are currently attempting to cache would continue saving their output (audio + MIDI) until playback stops or user changes the playback position.

When playback stops/changes, tracks take the longest section of their saved output that ends in a suitable amount of silence (e.g. 3s), and save this as a cached region.

Using cached sections

When entering playback of a cached section, if the output is currently silent and MIDI free, processing of the track is halted (in the same way as when muting), and the output is replaced with the cached output. If the output is not silent/MIDI-free, the cached section is discarded.

As playback leaves the cached region (during the silent crossover section), or when the user modifies the track, processing of the track is re-enabled and normal output is resumed.

[PitchSlap]

I've had a very similar idea for a soft-freeze feature forever. It's hard to articulate, but the potential for increased performance is enormous.

-CPU is the bottleneck for RT audio. Caching to RAM or HD (especially with SSD) is a no-brainer trade-off.

-Reaper could use the performance window history to identify bottlenecks/drop-outs/CPU-overloads in the timeline and intelligently pre-render/cache tracks for seamless playback. When playback is stopped, use some idle CPU cycles!

-Tracks with high CPU utilization that haven't had any changes for a while are the best candidates for 'soft-freeze'/caching.

-It's basically an advanced, non-linear version of anticipative FX processing, and the devs already know the performance gains that provides.


And here's the kicker, I started with Cool Edit Pro in the 90's with a Pentium MMX 233Mhz, and no matter what I threw at it playback would always be seamless! It did this by pre-rendering before playback, and the playback delay was always more desirable than instant playback that craps out.

20 years later, I'm still waiting for a DAW to take that to the next logical extension...

[semiquaver]

I think a background freeze which simply reverts upon editing would be a great feature.

[geraintluff]

Originally Posted by semiquaver

I think a background freeze which simply reverts upon editing would be a great feature.

Yes - unfortunately, actual background freezing is problematic.

We can't use a plugin to simultaneously calculate a frozen version of a track, and at the same time continue to use it for normal playback.

One option is to duplicate the effects - but not only does this double CPU/memory use for that track, but some of my plugins pop things up at the front when they're loading (which would be disruptive), and anything that uses external hardware or resources (which might not be obvious) could not be duplicated at all.

This proposal is a way I think we could work around these problems, but there might be others.

[foxAsteria]

Not sure I understand this request fully, but it sounds like you want to automate track mutes during their silent sections? This could possibly be done now with ReaGate + MIDItoReaControlPath. ReaGate would detect the silence and send a MIDI note, which MIDItoReaControlPath would translate to a usable MIDI control message for toggling the track mute action. Might work, but I guess the challenge would be to automatically select the correct track for muting without disrupting workflow.

[geraintluff]

Originally Posted by foxAsteria

Not sure I understand this request fully, but it sounds like you want to automate track mutes during their silent sections?

Not really - I want to save sections of output so that it doesn't have to calculate it next time. The silence is to detect points where we can seamlessly switch between the saved output and "real" output.

I guess it's not quite like "Freeze" and more like "Render to stem (and mute originals)", but done opportunistically for any "complete" section of the track (defined as starting and ending in silence).

Does that make any more sense?