Basically this is the same bug as referenced here:
And here's the issue on the deprecated issue tracker:
I decided to open a new thread since this bug changed slightly with Reaper 5, and because I think I have better understanding of conditions under which it is triggered now.
1) Turn Anticipative FX ON and make sure Render-Ahead is set to at least 200ms (it will also show up with shorter lengths but will be harder to reproduce)
2) Open the attached project
3) Start playback
4) Stop playback close to the loop's edge, while monitoring memory use
- I tested this on Windows, don't know whether this bug applies to Mac OS X
[Edit: cfillion confirmed this on OS X with render-ahead set to > 2000ms]
- This bug requires a project with at least 2 tracks, and a send from Track2 to track1. The send can be enabled or disabled, it doesn't matter.
- This bug requires an FX on Track2. The FX can be bypassed or disabled, it doesn't matter.
- This bug only happens in loop mode and when the loop is sufficiently short. With longer loops (I tried about 15mins), it will start the same but after a while will free the RAM.
- This bug only happens when the selection ends at least 16 min or so into the project time.
- With shorter buffer lengths set in the audio interface properties, memory leaks faster. About 3x faster with 64 samples than with 1024 samples.
- The more FX you have on that track, the easier it becomes to trigger the bug. With enough FX, it doesn't matter where you stop the playback.
- As soon as you move the playback cursor, leaked memory will be instantly freed