Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Compatibility

Reply
 
Thread Tools Display Modes
Old 03-25-2019, 08:18 PM   #1
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default Low Latency, & how Reaper allocates threads & cores.

I'm currently researching what to build for my next Windows PC. As always, I'm looking for the smallest HW buffer / lowest latency I can get, for my live-performance rig.
I found a thread on Gearslutz about fast cores vs lots of cores, and it helped to clarify something I read a few years ago about what some folks call "the real time audio thread."

I recently had a nightmare situation with Reaper, concerning buffer settings and latency. While I just "solved" that problem, I didn't understand it at all. Now I think I'm starting to, but the answer raises yet new questions.

I think this is an incredibly important topic for discussion, so I'm posting my Gearslutz post here, as well. There are some Reaper-specific issues that could take us some place good.

That whole thread is here:
https://www.gearslutz.com/board/musi...ore-count.html

My specific post, which explains what I just found out about routing in Reaper and how it can affect low-latency performance, is reposted next:
Cableaddict is offline   Reply With Quote
Old 03-25-2019, 08:21 PM   #2
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

I use Reaper for my band's live-performance mixer. Obviously, I constantly strive for the lowest HW buffer possible. It's a pretty heavy setup, but I've never gotten my total cpu usage past maybe 40%, by the time I hit that crackling buffer setting.

I am aware of what some users call the "real time audio thread.” I'm still a little hazy on this concept, but it definitely means that for small HW buffers, faster cores are more important than lots of cores, specifically because your final audio output (outputs?) must go through one path, hence one buffer. (or maybe one buffer per path? I dunno.)

Here's what I wanted to report:
——————————————

About a year ago, I started having horrible latency issues, with my standard live Reaper session. My Lynx / Aurora combo was normally allowing me to run a 256 buffer at 88.2K, giving a true i-o latency of 5.8ms. All of a sudden, I had to run a 1,024 buffer! It's been horrible. I tried newer drivers. I tried older drivers. I even tried different video drivers. Nothing helped.

Well, 2 days ago, I did an exhaustive test to try to find the problem, and thankfully I DID find a solution. - I just didn't quite understand it.
Reading this thread today (researching my next Windows PC build) made a little lightbulb go off.
-------------------------------------

## SO HERE'S WHAT HAPPENED:

I always run all of my tracks through a stereo subgroup, and then send the output of that subgroup to the 2-mix master track.
I do this so I can quickly see if I'm overloading the 2-mix, pre any plugins.
I call this track "PRE-MIX."

Normally, there are no plugins on my PRE-MIX track. It's just there for that visual. Well, for complicated reasons that don't matter here, I needed to change this, putting all of my mastering plugins on the PRE-MIX track instead. (About 5 plugins total) That was the beginning of my nightmare, though for some reason I never connected that change with my crackling issues. Maybe it happened slowly, I dunno.

So 2 nights ago, while removing 1 plugin at a time (hoping to find a corrupt or problematic one) I removed one from the PRE-MIX track, and yep, the crackling went away. It came back soon, but after I removed ALL plugins from that track, the problem went away permanently, and I was able to go back to a 128 buffer. Whew....

There's more to the story, but let's start there: Evidently, in Reaper, instantiating plugins on a track that is itself combining a large number of sources must somehow overload a core. Or a stream. or a "I don't know what," but there it is. AND FOR SOME REASON, those same plugins don't cause any trouble when they are on the "Master" track, with exactly the same number of tracks feeding it.

Maybe the mastering track is only seeing the output of the "PRE-MiX” track, and thus somehow has less work to do? Does that even make sense?
Wait, no, that's not it, because if I remove the PRE-MIX track entirely, and route everything directly to the MASTER track, the crackling problem does NOT go away!

Could Reaper be reserving one core just for the MASTER track ?

Or maybe it has something to do with how resources are allocated to the HW buffer? (HW buffers?)
------------------

BUT WAIT, THERE'S MORE !

(Here's where it gets REALLY weird)

When I had those plugins on the "PRE-MIX" track, and I removed the first one, and the crackling went away for a little while, that plugin I removed HAD BEEN OFFLINE ! And further testing shows that I can put ALL of the mastering plugins offline, AND THEY STILL CAUSE THE MASSIVE CRACKLING, if instantiated on the PRE-MIX track !

If a plugin is offline, it does not pull cpu cycles. So I mean, WTF ???????

Last edited by Cableaddict; 03-26-2019 at 01:55 AM.
Cableaddict is offline   Reply With Quote
Old 03-25-2019, 10:54 PM   #3
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

Also see the "Live" subforum here. I added the Gearslutz thread you mentioned to the introductional thread. Thanks ! Maybe you can contribute even more over there....
-Michael
mschnell is online now   Reply With Quote
Old 03-25-2019, 11:00 PM   #4
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

Do you really use 88.2 K samples /sek ?????
This of course is extremely demanding for the hardware.
Especially for "Live" I would not use more then 48 K.

Why do you think that is necessary ?
-Michael
mschnell is online now   Reply With Quote
Old 03-25-2019, 11:13 PM   #5
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

Quote:
Originally Posted by Cableaddict View Post
Could Reaper be reserving one core just for the MASTER track ?
Of course only the devs can provide a definitve answer to this.

I understand that

- Reaper assigns one OS thread to any thread, plus one OS thread for the GUI and internal proceedings.
- Usually plugins don't open up their own thread but run the thread Reaper provides for that track. So all plugins in a track use the same CPU. (OTOH, Kontakt can do internal threading and provides additional threads combined for to all instances it is used in any tracks).
- routing audio from, one OS thread to another uses up a lot hardware resources, so adding more tracks to take advantage of more CPUs might be a bad idea
- If more tracks are "active" at the same time then there are cores, the OS assigns CPUs to tracks due to it's own ideas of optimizing.
- Cores ("CPUs") have their own 1st level cashes, Intel "Hardware Threads" on each core share a 1st level cash and other Resources. The OS sees these hardware threads as "CPU"s. I don't know if the OS in intelliget enough to decently tell Cores from hardware threads to do perfect optimizing.

-Michael

Last edited by mschnell; 03-25-2019 at 11:18 PM.
mschnell is online now   Reply With Quote
Old 03-26-2019, 02:52 PM   #6
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

Nice response! (thx)

But I must admit, I don't quite understand all of it, or how this relates to real-world session setups.

I'm hoping others can join in this important discussion, maybe with some actual real-world examples & thoughts. (Maybe even post the results of various testing....)
Cableaddict is offline   Reply With Quote
Old 03-28-2019, 02:43 PM   #7
poetnprophet
Human being with feelings
 
poetnprophet's Avatar
 
Join Date: Jan 2018
Posts: 851
Default

hey Cable, I don't know how much i can help but I am interested in this. I track at 96k sample rate 64mb buffer so anything I can do to control CPU usage is a must. I also know from experience that plugins on the master track are usually worse than on a "pre-mix" track like you have.

So, it's quite surprising that you had the opposite experience with the master track plugins. And also very weird to hear about offline plugs causing audio issues.
__________________
https://www.kdubbproductions.com/
https://www.youtube.com/channel/UCpC...2dGA3qUWBKrXQQ
i7 8700k,4.9Ghz,Win10,Reaper 5,Motu 828es,MJE Hulk 990,GAP Pre73/EQ81
poetnprophet is offline   Reply With Quote
Old 03-28-2019, 02:46 PM   #8
poetnprophet
Human being with feelings
 
poetnprophet's Avatar
 
Join Date: Jan 2018
Posts: 851
Default

Quote:
Originally Posted by mschnell View Post
Do you really use 88.2 K samples /sek ?????
This of course is extremely demanding for the hardware.
Especially for "Live" I would not use more then 48 K.

Why do you think that is necessary ?
-Michael
higher sample rate means a lower latency
but, the tradeoff is CpU usage

so if your aim is low latency, one needs to run a higher sample rate.
__________________
https://www.kdubbproductions.com/
https://www.youtube.com/channel/UCpC...2dGA3qUWBKrXQQ
i7 8700k,4.9Ghz,Win10,Reaper 5,Motu 828es,MJE Hulk 990,GAP Pre73/EQ81
poetnprophet is offline   Reply With Quote
Old 03-28-2019, 07:10 PM   #9
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

Quote:
Originally Posted by poetnprophet View Post
higher sample rate means a lower latency
but, the tradeoff is CpU usage

so if your aim is low latency, one needs to run a higher sample rate.
It also means less propagation delay, with any plugins / VSTi's that have it, so the gains are typically even more than you might think.

(Also, some stuff just sounds better, like amp sims.)
Cableaddict is offline   Reply With Quote
Old 03-28-2019, 07:14 PM   #10
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

Quote:
Originally Posted by poetnprophet View Post
hey Cable, I don't know how much i can help but I am interested in this. I track at 96k sample rate 64mb buffer so anything I can do to control CPU usage is a must. I also know from experience that plugins on the master track are usually worse than on a "pre-mix" track like you have.

So, it's quite surprising that you had the opposite experience with the master track plugins. And also very weird to hear about offline plugs causing audio issues.
It truly is a mystery. Hence this thread.

This is phenomenally important stuff, yet there is very little about it on the internet. At least these days, we know about the priority of fast cores vs more cores, and (to some extent) how the "real-time thread" (or more likely, threads) limits HW buffer choices.

- But how to allocated plugins across various tracks is still completely new territory.
And yeah, why offline plugins make a difference is mind boggling.

I sure hope someone from Cockos can weigh-in here.
Cableaddict is offline   Reply With Quote
Old 03-28-2019, 10:53 PM   #11
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

Quote:
Originally Posted by poetnprophet View Post
higher sample rate means a lower latency
Not really.

Of course higher sample rate means a lower latency when keeping the block size and count settings. But the block size and count settings are just arbitrary done to allow the CPU to keep up with the rendering task.

Hence it's a lot better to reduce the block size/count to get lower latency without blowing up CPU load. Of course there are other arguments for a high sample rate,. but usually not in "Live" situations.

-Michael
mschnell is online now   Reply With Quote
Old 03-28-2019, 10:55 PM   #12
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

Quote:
Originally Posted by Cableaddict View Post
It also means less propagation delay, with any plugins / VSTi's that have it
AFAIK, this is not true. But of course we can't predict the internal working of any 3rd party plugins.

Quote:
Originally Posted by Cableaddict View Post
(Also, some stuff just sounds better, like amp sims.)
This might be true, as a higher sample rate reduces audible artifacts certain non-linear algorithms introduce, and either might remain in the signal or are "filtered out" by the VS, both reducing sound quality.

But I decently doubt that this really makes a difference in a "Live" situation with a noisy room etc.

-Michael
mschnell is online now   Reply With Quote
Old 03-29-2019, 01:15 AM   #13
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

Quote:
Originally Posted by mschnell View Post
AFAIK, this is not true. But of course we can't predict the internal working of any 3rd party plugins.

AFAIK, it's ALWAYS true. Almost by definition.

If it's not true, then a whole lotta' smart people, who create plugins for a living, have also been wrong over the years. (I've been a beta tester for countless of these companies since Protools first came out.)

I guess it's possible, but not likely. I'll keep an open mind, though.
-------------------------------------

On the sonic thing, again it's something even the guys who write the code agree upon. Of course, with really good oversampling it no longer holds, but then that eats up cpu power. (The way SoundToys does it.) I suspect, though it's a guess, that VST3 plugins might mitigate this a lot, since they can run fully 64 bit. But again, I don't really know.

Last edited by Cableaddict; 03-30-2019 at 10:58 AM.
Cableaddict is offline   Reply With Quote
Old 03-29-2019, 05:19 AM   #14
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 7,493
Default

E.g. with ReaVerb you can set the latency ("ZE" checked -> No latency, else latency depends on the FFT Window size.)
The lower the latency the more CPU demand.

Many plugins supposedly internally are optimized regarding the amount of latency introduced and the CPU overhead required. Supposedly the developers did the optimization with some 48 KSample/sec in mind. Of course latency in many cases will decease with higher sample rate (if they don't auto-adjust other parameters), but the CPU demand will explode.

-Michael
mschnell is online now   Reply With Quote
Old 03-29-2019, 06:18 AM   #15
Stella645
Human being with feelings
 
Join Date: Sep 2008
Location: UK
Posts: 1,184
Default

I think the mystery is less about computer spec and core loading and more about how Reaper is trying to help out with record monitoring latency.

If you attempt to record monitor through that pre-master channel I believe Reaper will put it into a low latency mode and the combination of that and the plugs latency can cause the crackling.

But if you left the plugs on that channel and routed the record monitored channels direct to stereo out (with no plugs)you should get no crackling AND stereo buss process on the mix playback....and your recording tracks are still in low latency mode.

Great for recording an overdub or two when you're well into the mix...but probably not so applicable for your live use.
Stella645 is offline   Reply With Quote
Old 03-29-2019, 09:29 AM   #16
poetnprophet
Human being with feelings
 
poetnprophet's Avatar
 
Join Date: Jan 2018
Posts: 851
Default

Quote:
Originally Posted by Stella645 View Post
I think the mystery is less about computer spec and core loading and more about how Reaper is trying to help out with record monitoring latency.

If you attempt to record monitor through that pre-master channel I believe Reaper will put it into a low latency mode and the combination of that and the plugs latency can cause the crackling.

But if you left the plugs on that channel and routed the record monitored channels direct to stereo out (with no plugs)you should get no crackling AND stereo buss process on the mix playback....and your recording tracks are still in low latency mode.

Great for recording an overdub or two when you're well into the mix...but probably not so applicable for your live use.
What do you mean "record monitor through pre-master" exactly?
__________________
https://www.kdubbproductions.com/
https://www.youtube.com/channel/UCpC...2dGA3qUWBKrXQQ
i7 8700k,4.9Ghz,Win10,Reaper 5,Motu 828es,MJE Hulk 990,GAP Pre73/EQ81
poetnprophet is offline   Reply With Quote
Old 03-29-2019, 10:12 AM   #17
Stella645
Human being with feelings
 
Join Date: Sep 2008
Location: UK
Posts: 1,184
Default

Sorry....confusing on my part to call it record monitor.

I'm referring to routing of any record armed track into the pre-master....or improving performance and latency for record monitoring (could be midi triggered vsti or audio monitored via FX) by routing those tracks direct to the stereo out while stereo bus process plugs are loaded and working on the pre master for existing mix elements.
Stella645 is offline   Reply With Quote
Old 03-29-2019, 10:16 AM   #18
toleolu
Human being with feelings
 
Join Date: Apr 2014
Location: Sunshine and palm trees.
Posts: 798
Default

Nice to finally see someone draw attention to process rather than specs and tech speak.
toleolu is offline   Reply With Quote
Old 03-29-2019, 10:32 AM   #19
poetnprophet
Human being with feelings
 
poetnprophet's Avatar
 
Join Date: Jan 2018
Posts: 851
Default

Quote:
Originally Posted by Stella645 View Post
Sorry....confusing on my part to call it record monitor.

I'm referring to routing of any record armed track into the pre-master....or improving performance and latency for record monitoring (could be midi triggered vsti or audio monitored via FX) by routing those tracks direct to the stereo out while stereo bus process plugs are loaded and working on the pre master for existing mix elements.
so in essence, direct monitoring basically but through Reaper?
__________________
https://www.kdubbproductions.com/
https://www.youtube.com/channel/UCpC...2dGA3qUWBKrXQQ
i7 8700k,4.9Ghz,Win10,Reaper 5,Motu 828es,MJE Hulk 990,GAP Pre73/EQ81
poetnprophet is offline   Reply With Quote
Old 03-29-2019, 12:55 PM   #20
Stella645
Human being with feelings
 
Join Date: Sep 2008
Location: UK
Posts: 1,184
Default

Well perhaps direct is also not a great way to describe it as it can absolutely include FX on the record track but direct in the sense that routing this way the record path can run independently of the playback path and so you can have high latency in the mix but still low latency record monitoring.

This is true if you need to monitor throuhg an amp sim....but also true for midi live play of a vsti.
Stella645 is offline   Reply With Quote
Old 03-30-2019, 10:58 AM   #21
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

Quote:
Originally Posted by mschnell View Post
Many plugins supposedly internally are optimized regarding the amount of latency introduced and the CPU overhead required. Supposedly the developers did the optimization with some 48 KSample/sec in mind.

Of course latency in many cases will decease with higher sample rate (if they don't auto-adjust other parameters), but the CPU demand will explode.

-Michael
Absolutely. No argument there.
Cableaddict is offline   Reply With Quote
Old 03-31-2019, 10:57 AM   #22
Cableaddict
Human being with feelings
 
Join Date: Apr 2008
Posts: 1,883
Default

So....

No one can explain what happened with my session, or how Reaper allocates track resources?

Please don't tell me this is going to be another mystery, like buffer settings?

Anyone?
Cableaddict is offline   Reply With Quote
Old 04-03-2019, 01:29 AM   #23
azslow3
Human being with feelings
 
Join Date: Nov 2017
Location: Heidelberg, Germany
Posts: 679
Default

There is no mystery in computers...

The primary source of low-latency problems is computer hardware + drivers + audio interface. And general problem there is no low latency optimized computer components, manufacturers simply do not care since much less then 1% of all customers ever need that.

Audio interfaces manufacturers care more, in hardware and in drivers, but they are not always successful. "Lowest possible" settings are close for bright range of interfaces on paper, but really usable settings differ. F.e. you can get 3.3ms under 48kHz/48 samples buffer with RME (or 96kHz/96 samples buffer) and it can be usable without glitches. On the same system with most other interfaces you need to reduce samples down to 32 to get comparable latency. And not all systems can be able to work under "stress".

So the system is also critical parameter. There are optimized computers with proper settings in BIOS and Windows which can be loaded over 90% total CPU and still work without glitches. But unaware consumer computer without any tweaks starts to produce audio drops way early, even at 20-30% load. From where that comes:
(1) the frequency and the buffer size strictly define the "time slot" in which one buffer processing should happened. F.e. 88.2kHz/256 means 3ms. Was processing chain not ready within 3ms? Bum. Audio drop. Even if that happens once per 300 cycles, you have continuous glitches every second. That is audio interface independent. Can be "swap" (must music software designers do not care about memory locks, streamed from disks samples of VSTi, not optimal distributed along cores load, etc.)
(2) audio hardware and drivers define "tolerance" for any system delay when there are about to transport audio (buffer->driver processing->USB/PCIe/etc). That is sometimes called "safety" time. They normally make it the buffer settings dependent, so you have more "relaxed" environment when working with large buffers. Sometimes that parameter is exposed explicitly, independent (!) for the buffer size. Note that that tolerance is way smaller than the buffer time. That is why all latency utilities immediately report "not suitable for real-time audio" in case they detect system delay over 1ms. In practice that is particular hardware/driver dependent. The limit easily can be 0.3ms or less for particular interface/settings, there is no such charts nor tests. Can be "system locking" hardware/software (resource locks, power saving CPU/other components settings, etc.). Unlike with the first case, the result can be not just audio glitch but also severe troubles with the driver up to completely stuck till reboot interface.

Back to the original question. REAPER allocates threads per track, it is not splitting FX chains (that was confirmed by several user tests). What happens in addition is topology dependent. So all interconnected tracks which are in real-time mode participate in total one buffer processing. Note that since REAPER allows "on the fly" changes, not "currently working" FXes still can influence the topology (only developers know how, but logically there must be something). In that respect also note that "bypassed"/"off" internally, "bypassed"/"off" in the DAW, on "muted" tracks, etc. all can produce different results on topology.

How to monitor. First of all using REAPER Performance meter. Do not forget right click and enable all (RT) options there. F.e. watch "RT longest block". It shows how long it was taking to process one buffer/the buffer time (as in (1)). Also start system monitor, it can show "something else" is eating resources. And do not forget system latency monitoring, but note they are influencing (2) and so can produce "extra" audio glitches.

Also note that REAPER has many CPU related preferences, from limiting cores and locking threads to cores to using several cores for live FX processing. Not all options are "good" to enable, but you can compare how they influence your system and project.


Finally, PDC (plug-in delay compensation) is not CPU/performance related. PDC for the whole FX chain is rounded (up) the to buffer size, that is why you can observer "extra live delay" when increasing the buffer (visible in the FX chain window). So it can be wise to group plug-ins with delays into one chain, so there will be rounded sum of delays vs sum of rounded delays.
azslow3 is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 07:16 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.