Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Bug Reports

Reply
 
Thread Tools Display Modes
Old 07-10-2019, 04:40 PM   #41
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Anything above 40,000 in reaper setting slows it down..from what I tried today...Let me do more checking tomorrow...


Meanwhile...

https://forum.cockos.com/showthread.php?t=222807


Can you pls do what's asked on that forum post ? Cos that will help me immensely..Coz then I can make all inactive takes offline...that will reduce the number of files a lot..Coz I have 3 or 4 takes in a lot of items... If I can make all the inactive takes offline...that will bring down the number of files used down considerably..
svijayrathinam is offline   Reply With Quote
Old 07-10-2019, 05:23 PM   #42
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

5.980rc4 will be along shortly and it won't automatically bring everything online after rendering new takes/etc.
Justin is offline   Reply With Quote
Old 07-10-2019, 06:15 PM   #43
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Is all of your media .wav files or is there a mix? maybe I can make you some experimental plugins to reduce the open file count.

Also in the next dev cycle I’ll add an option to fully load peaks files into RAM, reducing the open file count by some decent amount (at the expense of a lot more RAM use and slower media onlining)
Justin is offline   Reply With Quote
Old 07-10-2019, 10:51 PM   #44
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Quote:
Originally Posted by Justin View Post
Is all of your media .wav files or is there a mix? maybe I can make you some experimental plugins to reduce the open file count.

Also in the next dev cycle I’ll add an option to fully load peaks files into RAM, reducing the open file count by some decent amount (at the expense of a lot more RAM use and slower media onlining)
Thank You so much.. My files are a mix of Wav and FLAC
svijayrathinam is offline   Reply With Quote
Old 07-18-2019, 07:36 AM   #45
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Hello Justin,

After making the inactive takes offline with x-ray M script..when i switch between project tabs..The offline files come back online. Can you pls look into this ?
svijayrathinam is offline   Reply With Quote
Old 07-22-2019, 03:13 PM   #46
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Can this be fixed pls ?
svijayrathinam is offline   Reply With Quote
Old 07-22-2019, 03:14 PM   #47
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Quote:
Originally Posted by Justin View Post
Is all of your media .wav files or is there a mix? maybe I can make you some experimental plugins to reduce the open file count.

Also in the next dev cycle I’ll add an option to fully load peaks files into RAM, reducing the open file count by some decent amount (at the expense of a lot more RAM use and slower media onlining)
Is this coming in the next pre release ?
svijayrathinam is offline   Reply With Quote
Old 07-24-2019, 08:35 AM   #48
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Two things coming in the next dev build:

1) An option to fully-load peak files under a certain size into RAM. You should set this to something big (like 100000KB) if you want to make sure all peak files are loaded. You should leave the memory-mapped option at 0 to prevent it from being used.

2) A ReaScript API which can be used to tell REAPER to trim its open files. Keeping extra handles open is necessary for performance, but you can use a script similar to this one in order to have it prevent these open files from accumulating:

Code:
do_log = true -- annoying but maybe interesting
aggressive_if_stopped = false -- not recommended but maybe useful for some things?

function run()
  local f = 1
  msg=""
  if aggressive_if_stopped then
    if 0==(reaper.GetAllProjectPlayStates()&1) then
      f = f + 2
      msg=" (aggressively)"
    end
  end
  local cnt = reaper.reduce_open_files(f)
  if cnt > 0 and do_log then
    reaper.ShowConsoleMsg(string.format("closed %d files%s\n",cnt,msg))
  end
  reaper.defer(run)
end

run()
(you can set aggressive_if_stopped to true if you want it to basically try to close all file handles that are not in use, but this will probably make peaks building incredibly slow and also could cause playback interruptions... so be extra-careful with that! having said that, be extra careful with this script in general...)
Justin is offline   Reply With Quote
Old 07-24-2019, 04:03 PM   #49
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Quote:
Originally Posted by Justin View Post
Two things coming in the next dev build:

1) An option to fully-load peak files under a certain size into RAM. You should set this to something big (like 100000KB) if you want to make sure all peak files are loaded. You should leave the memory-mapped option at 0 to prevent it from being used.

2) A ReaScript API which can be used to tell REAPER to trim its open files. Keeping extra handles open is necessary for performance, but you can use a script similar to this one in order to have it prevent these open files from accumulating:

Code:
do_log = true -- annoying but maybe interesting
aggressive_if_stopped = false -- not recommended but maybe useful for some things?

function run()
  local f = 1
  msg=""
  if aggressive_if_stopped then
    if 0==(reaper.GetAllProjectPlayStates()&1) then
      f = f + 2
      msg=" (aggressively)"
    end
  end
  local cnt = reaper.reduce_open_files(f)
  if cnt > 0 and do_log then
    reaper.ShowConsoleMsg(string.format("closed %d files%s\n",cnt,msg))
  end
  reaper.defer(run)
end

run()
(you can set aggressive_if_stopped to true if you want it to basically try to close all file handles that are not in use, but this will probably make peaks building incredibly slow and also could cause playback interruptions... so be extra-careful with that! having said that, be extra careful with this script in general...)
Thank you very much Justin. But where do I run this script ? Is it in the actions list ?
svijayrathinam is offline   Reply With Quote
Old 07-24-2019, 11:17 PM   #50
EvilDragon
Human being with feelings
 
EvilDragon's Avatar
 
Join Date: Jun 2009
Location: Croatia
Posts: 24,790
Default

You first need to store it in a .lua file, then load it into the action list...
EvilDragon is offline   Reply With Quote
Old 01-04-2020, 09:19 AM   #51
Klangfarben
Human being with feelings
 
Join Date: Jul 2016
Location: Los Angeles, CA
Posts: 1,701
Default

So what was the end result of this? I still have to use OSX for some things and not having max file access is pretty concerning to me. There are MANY situations you can easily hit this limit. And if any sort of solution ends up slowing down the project that is a no go.

I'm talking about a couple different scenarios here. The first is television scoring work. A lot of people are now putting the whole show into one project, especially due to the speed the material must be delivered. It's becoming more and more common. The second is any kind of recording for film music on the scoring stage where again, you have a massive file count - and also you can not have ANY down time/slow downs etc. due to the incredibly high cost.

If this happened to a pro tools operator on a scoring stage or to a mixer on a dub stage using Pro Tools, Avid would have to have a fix pretty much the same day or they would go out of business. I used to do a lot of Pro Tools operating and editing so I have people asking me if Reaper is an option for this kind of work. Right now, I would have to tell them no unless there is some kind of proper solution for this as there is a massive amount of audio files - multiple takes of 48-64 tracks wide for orchestra, lots of soloists for ethnic instruments, percussion, multiple passes of choir, hundreds of prelay tracks from the composers studio. Each cue is going to have a project generally, but you could hit Reaper's limit not even 30 minutes into a session easily.

And that's not even getting into stuff like sampling sessions the big companies are doing. I've never run into an issue like this in Pro Tools, Cubase or Logic and I've dealt with projects that have just a massive amount of audio files. In fact, I haven't seen an error related to an open files limit for over 15 years, even in huge sampling sessions. It was a bigger issue back in OS9 obviously but I'm sort of stunned to hear this is an issue in Reaper, regardless of the number of users doing this kind of work.

Sort of heavily goes against the Reaper mantra of "audio production without limits" IMHO. If this is still indeed problematic, it really needs to be sorted properly and fairly invisible to the user (not just with an ini change, super user command or a lua script) so Reaper isn't the odd bird OSX DAW here.
Klangfarben is offline   Reply With Quote
Old 01-04-2020, 09:33 AM   #52
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Quote:
Originally Posted by Klangfarben View Post
So what was the end result of this? I still have to use OSX for some things and not having max file access is pretty concerning to me. There are MANY situations you can easily hit this limit. And if any sort of solution ends up slowing down the project that is a no go.

I'm talking about a couple different scenarios here. The first is television scoring work. A lot of people are now putting the whole show into one project, especially due to the speed the material must be delivered. It's becoming more and more common. The second is any kind of recording for film music on the scoring stage where again, you have a massive file count - and also you can not have ANY down time/slow downs etc. due to the incredibly high cost.

If this happened to a pro tools operator on a scoring stage or to a mixer on a dub stage using Pro Tools, Avid would have to have a fix pretty much the same day or they would go out of business. I used to do a lot of Pro Tools operating and editing so I have people asking me if Reaper is an option for this kind of work. Right now, I would have to tell them no unless there is some kind of proper solution for this as there is a massive amount of audio files - multiple takes of 48-64 tracks wide for orchestra, lots of soloists for ethnic instruments, percussion, multiple passes of choir, hundreds of prelay tracks from the composers studio. Each cue is going to have a project generally, but you could hit Reaper's limit not even 30 minutes into a session easily.

And that's not even getting into stuff like sampling sessions the big companies are doing. I've never run into an issue like this in Pro Tools, Cubase or Logic and I've dealt with projects that have just a massive amount of audio files. In fact, I haven't seen an error related to an open files limit for over 15 years, even in huge sampling sessions. It was a bigger issue back in OS9 obviously but I'm sort of stunned to hear this is an issue in Reaper, regardless of the number of users doing this kind of work.

Sort of heavily goes against the Reaper mantra of "audio production without limits" IMHO. If this is still indeed problematic, it really needs to be sorted properly and fairly invisible to the user (not just with an ini change, super user command or a lua script) so Reaper isn't the odd bird OSX DAW here.


This issue is solved. Pls follow instructions from earlier posts. Now I am running huge reaper projects and it’s very light on CPU. This implementation made reaper 6 more stable than ever. I tried running a very big reaper project and converted the same project to PT via AATranslator and ran it on PT .. I can confirm that reaper beats PT hands down in terms of performance in my Mac Pro
svijayrathinam is offline   Reply With Quote
Old 01-05-2020, 12:27 AM   #53
Thonex
Human being with feelings
 
Join Date: May 2018
Location: Los Angeles
Posts: 1,719
Default

Quote:
Originally Posted by svijayrathinam View Post
This issue is solved. Pls follow instructions from earlier posts. Now I am running huge reaper projects and it’s very light on CPU. This implementation made reaper 6 more stable than ever. I tried running a very big reaper project and converted the same project to PT via AATranslator and ran it on PT .. I can confirm that reaper beats PT hands down in terms of performance in my Mac Pro
Hi svijayrathinam,

I would have to disagree. I don't think this issue is resolved in a professional way. Case and point:

I am currently recording at an orchestral scoring stage and will be for almost 2 weeks. Yesterday in the middle of a session I got a big nasty dialog window (Mac OS -- the computer belongs to the studio) saying that the Drive was currently not available (or similar -- I forget the exact wording). This happened right as I hit record on the second day of intense recording. We (in the control room) all freaked out. What happened?!?! First we thought that the drive failed... but we could still play back... just not record. Then we though that there was a corrupt project with a messed up directory path. We did a back up on the spot because we weren't sure what the problem was.

Then, something told me to resave the project and remove a BUNCH of audio files that we have already recorded from the project (remove the items). BINGO... now we could record again.

We had reached the 7K-ish limit of audio files mentioned in this thread. It it totally bogus that there isnt a more graceful way to handle this. Other developers that need to access > 7K files find workarounds. At the very least, Reaper should have some warning about the approaching Max Files Limit.

It was rather embarrassing. We minimized the impact because the session players agreed to take their break early.... but this was totally a freak-out moment.

So... yeah... not so resolved from my perspective. Although now I know.

BTW... I was only recording 14 mics at a time... yes... albeit for days on the same project.... but there were reasons for this. Now we adapted and are moving along.

Cheers,

Andrew K
__________________
Cheers... Andrew K
Reaper v6.80+dev0621 - June 21 2023 • Catalina • Mac Mini 2020 6 core i7 • 64GB RAM • OS: Catalina • 4K monitor • RME RayDAT card with Sync Card and extended Light Pipe.

Last edited by Thonex; 01-05-2020 at 01:32 AM.
Thonex is offline   Reply With Quote
Old 01-05-2020, 04:20 AM   #54
svijayrathinam
Human being with feelings
 
Join Date: May 2017
Posts: 981
Default

Hi Andrew,

Did you do in the terminal what was said in earlier posts ? Is it still not working?

Or are you saying reaper should be able to do this without a user having to do these things in terminal ?

Pls clarify. I don’t quite understand what exactly you mean.

I have done what is suggested in earlier posts. And I can confirm I am working on a project with 40,000 audio files without any problem.
svijayrathinam is offline   Reply With Quote
Old 01-06-2020, 12:21 AM   #55
Thonex
Human being with feelings
 
Join Date: May 2018
Location: Los Angeles
Posts: 1,719
Default

Quote:
Originally Posted by svijayrathinam View Post
Hi Andrew,

Did you do in the terminal what was said in earlier posts ? Is it still not working?

Or are you saying reaper should be able to do this without a user having to do these things in terminal ?

Pls clarify. I don’t quite understand what exactly you mean.

I have done what is suggested in earlier posts. And I can confirm I am working on a project with 40,000 audio files without any problem.
Hi svijayrathinam,

I performed the Terminal Max Files command on my computer over a year ago and my system is fine. However, you can't expect a major recording facility to allow you to take their main protools rig computer and start changing Terminal commands and messing with their system. Their admin would never allow this. They would say... "bring your own computer!".

This really should be addressed in a much more graceful manner by Cockos. If they can't find a workaround with stack pointers or something, then at least provide some kind of warning before the Max File limit is reached. Now I'm finding myself (on a regular basis) checking the number of audio items in a project to make sure I'm safe. This is rather silly.

Cheers,

Andrew K
__________________
Cheers... Andrew K
Reaper v6.80+dev0621 - June 21 2023 • Catalina • Mac Mini 2020 6 core i7 • 64GB RAM • OS: Catalina • 4K monitor • RME RayDAT card with Sync Card and extended Light Pipe.
Thonex is offline   Reply With Quote
Old 01-16-2020, 12:17 PM   #56
Thonex
Human being with feelings
 
Join Date: May 2018
Location: Los Angeles
Posts: 1,719
Default

Bumping this.

Quote:
Originally Posted by Thonex View Post
Hi svijayrathinam,

I performed the Terminal Max Files command on my computer over a year ago and my system is fine. However, you can't expect a major recording facility to allow you to take their main protools rig computer and start changing Terminal commands and messing with their system. Their admin would never allow this. They would say... "bring your own computer!".

This really should be addressed in a much more graceful manner by Cockos. If they can't find a workaround with stack pointers or something, then at least provide some kind of warning before the Max File limit is reached. Now I'm finding myself (on a regular basis) checking the number of audio items in a project to make sure I'm safe. This is rather silly.

Cheers,

Andrew K
__________________
Cheers... Andrew K
Reaper v6.80+dev0621 - June 21 2023 • Catalina • Mac Mini 2020 6 core i7 • 64GB RAM • OS: Catalina • 4K monitor • RME RayDAT card with Sync Card and extended Light Pipe.
Thonex is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:54 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.