This is maybe the 5th time I've brought this up since buying Reaper and it appears to still be the same way.
1. Open an existing song for overdubs.
2. Record, overdub, and later decide that it's just not working, put it off until later.
3. Close the song, choose not to save it.
Reaper has no concept of the orphaned audio files. What it really should do is actively track the files recorded or bounced in the current launch and if you don't actually save the song, it should prompt you to clean up those files, delete them, when closing.
Most professional workstations I've used seem to do that if you close a song in that state, like below...