Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Bug Reports

Reply
 
Thread Tools Display Modes
Old 09-12-2020, 03:26 PM   #1
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default ReaperConfigZip has 2GB limit? (Done)

In this thread:
https://forum.cockos.com/showthread.php?t=225228

we came to the conclusion, that Reaper could have a problem with ReaperConfigZip-files larger than 2GB, which lead to many files were lost, as they weren't accessible anymore in one such case.

And as I've seen several posts in the last few months, were people have hundreds of MB huge statechunks due some fx needing them, this could become a problem, when somebody has track/projecttemplates or FXChains containing such huge amount of data.

In addition to that, Reaper didn't warn that the ReaperConfigZip-file would become too large to be valid, so maybe, this is something that should be implemented at least, so the users can ask the community for workarounds without losing data in that case.

I have no idea, how to solve the >2GB-limit. Multiple files maybe? Don't know, if that is practical at all...

Last edited by Meo-Ada Mespotine; 05-27-2023 at 02:03 AM.
Meo-Ada Mespotine is offline   Reply With Quote
Old 09-13-2020, 01:28 AM   #2
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,066
Default

That would be a monster config, I guess.
TonE is online now   Reply With Quote
Old 09-13-2020, 02:16 AM   #3
akademie
Human being with feelings
 
Join Date: Mar 2007
Posts: 4,772
Default

Tested here in Windows10 x64 and REAPER v6.13.

I did put few big files (RARs and ZIPs) into ProjectTemplate folder, then exported config (all items checked).
It took a while and finished sucessfully by Reaper telling that it was done and config contains 2055 files. (approx 2,6GB).

Then the file size is really 2,6GB.
But renaming to ZIP and trying to open in WinRAR v5.01 (64bit) gives error warning on opening the file:
"test2_over2gig.ReaperConfigZip.zip: Unexpected end of archive"
and content displayed in WinRAR is missing lot of folders of course.

Trying to open the same archive in Windows Explorer by double-clicking ends with error message:
"Compressed folder (ZIP method) test2_over2gig.ReaperConfigZip.zip is invalid"
(note that the message is my translation from Czech version of Windows, so not exact as in EN/US version).

So, REAPER definitely does something wrong or in a strange way at least.

Please, fix or display error message when creating such big invalid archives.
akademie is online now   Reply With Quote
Old 09-13-2020, 04:57 AM   #4
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,066
Default

Why is your config zip so big?
TonE is online now   Reply With Quote
Old 09-13-2020, 05:44 AM   #5
akademie
Human being with feelings
 
Join Date: Mar 2007
Posts: 4,772
Default

Quote:
Originally Posted by TonE View Post
Why is your config zip so big?
This is not my CFG, I made such for testing purpose only.

And, please, read the original thread as linked in original post.
akademie is online now   Reply With Quote
Old 09-14-2020, 09:40 AM   #6
Icchan
Human being with feelings
 
Icchan's Avatar
 
Join Date: Dec 2011
Location: Finland
Posts: 793
Default

Yeah, obvious edge case that devs didn't consider.

(I mean who would put that much data into the condig folder, right? )

I hope it'll get fixed fast.
Icchan is offline   Reply With Quote
Old 09-14-2020, 10:18 AM   #7
akademie
Human being with feelings
 
Join Date: Mar 2007
Posts: 4,772
Default

Quote:
Originally Posted by Icchan View Post
Yeah, obvious edge case that devs didn't consider.

(I mean who would put that much data into the condig folder, right? )

I hope it'll get fixed fast.
There are users who create their FX chains with VSTi plugins that store big amount of data chunks with its states.

It is actual topic here in the forum for the user of Arturia Lab or what...

So FXChains or Project Templates (RPP) can be quite big because of that, also JSFX plugins (samplers store their sample memory in project file), so again, you make a lot of templates and you have it.
akademie is online now   Reply With Quote
Old 09-14-2020, 04:45 PM   #8
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

IIRC, some orchestral libraries also have huge chunks of data stored in their FXChains.

Some ProjectTemplates with them and voila: above the 2GB-limit.
Meo-Ada Mespotine is offline   Reply With Quote
Old 09-14-2020, 05:09 PM   #9
Icchan
Human being with feelings
 
Icchan's Avatar
 
Join Date: Dec 2011
Location: Finland
Posts: 793
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
IIRC, some orchestral libraries also have huge chunks of data stored in their FXChains.

Some ProjectTemplates with them and voila: above the 2GB-limit.
If saving an FX state takes hundreds of MB, then there's a real issue with said library and devs should try and work with the library developer on how to fix that issue.

That's not normal or a good practice
Icchan is offline   Reply With Quote
Old 09-14-2020, 05:22 PM   #10
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

Or the library just needs so much. You don't know in the end unless you reverse-engineer the binary data stored to see what's going on.

And to be honest: even if that's not the case and all loaded plugins "only" create 50 MB in total:

20 Project Templates
20 Track Templates
the rest FXChains and some themes.

It's actually possible to reach 2GB if you're working with many music-projects in different contexts needing a heck lot of templates and FXChains.
And if this export-feature is included in Reaper for the general user, it MUST work or at least warn me of the problem, no matter how messed up my plugins are.
Otherwise I assume it's an actual backup-solution for my config, which it's not in these cases.

In 5 years, this 2GB-problem will not be edgecasy anymore but a problem for more users, even if this doesn't include you or me.

And "Good practice" often denies the practical reality of "colorful" software out in this world
Meo-Ada Mespotine is offline   Reply With Quote
Old 09-21-2020, 02:45 PM   #11
akademie
Human being with feelings
 
Join Date: Mar 2007
Posts: 4,772
Default

bump..

Any word from devs, please?
akademie is online now   Reply With Quote
Old 09-22-2020, 07:07 AM   #12
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 16,124
Default

Fixing this
Justin is offline   Reply With Quote
Old 09-22-2020, 07:42 AM   #13
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

Awesome
Meo-Ada Mespotine is offline   Reply With Quote
Old 09-22-2020, 09:59 AM   #14
akademie
Human being with feelings
 
Join Date: Mar 2007
Posts: 4,772
Default

yes, thanks Justin.
akademie is online now   Reply With Quote
Old 02-03-2021, 08:53 AM   #15
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Justin View Post
Fixing this
Any news on this Justin, I just exported my Config and get the same problem. Granted my Config is massive due to Orchestral libraries and custom actions. Any type fix would be amazing as i am panicking now because i put so much work into this and don't want to loose it.
jadedsean is offline   Reply With Quote
Old 02-03-2021, 12:48 PM   #16
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 16,124
Default

This was fixed in 6.15. If you have an issue with it, would be interested to know the details, but in the interim you can also backup your config manually, you can open the resource path in REAPER (options menu, show resource path) and backup the entirety of that folder.
Justin is offline   Reply With Quote
Old 02-03-2021, 03:04 PM   #17
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Justin View Post
This was fixed in 6.15. If you have an issue with it, would be interested to know the details, but in the interim you can also backup your config manually, you can open the resource path in REAPER (options menu, show resource path) and backup the entirety of that folder.
Thanks for the response Justin, so yeah i made export config and when i tried to import said config i get this message, "Error opening ReaperConfigZip (missing information). I believe i made the mistake of saving Project templates as these are massive Orchestral templates, when i had a look how large the config was i nearly fell off my chair, 8gb all in all. Maybe its possible that this config is way to big. Is there anyway of retrieving it?
jadedsean is offline   Reply With Quote
Old 02-03-2021, 04:10 PM   #18
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 16,124
Default

Did you save the configuration using a newer REAPER or older?
Justin is offline   Reply With Quote
Old 02-03-2021, 04:16 PM   #19
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Justin View Post
Did you save the configuration using a newer REAPER or older?
Newer, i am on the latest version.
jadedsean is offline   Reply With Quote
Old 02-04-2021, 06:05 AM   #20
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

How big is your Resourcefolder and how big is your Resourcefolder/UserPlugins?

And how big is the ReaperConfigZip-file?

You can also try the following:

Rename the ReaperConfigZip-file by adding .zip at the end of the filename and then you can open it as regular zip file.
Can you extract all the files that way or do you get an error?

And the standard question is:
Which operating system do you use?
Which exact Reaper versions for exporting and importing the configuration?
And where did you export it to?
For instance USB-Sticks might have problems with certain filesizes, when the filesystem used is something like FAT32 or something.
Meo-Ada Mespotine is offline   Reply With Quote
Old 02-04-2021, 01:37 PM   #21
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
How big is your Resourcefolder and how big is your Resourcefolder/UserPlugins?

And how big is the ReaperConfigZip-file?

You can also try the following:

Rename the ReaperConfigZip-file by adding .zip at the end of the filename and then you can open it as regular zip file.
Can you extract all the files that way or do you get an error?

And the standard question is:
Which operating system do you use?
Which exact Reaper versions for exporting and importing the configuration?
And where did you export it to?
For instance USB-Sticks might have problems with certain filesizes, when the filesystem used is something like FAT32 or something.

Hi thanks for the response, so its a portable install,and this folder is massive due to the fact of many big orchestral templates. The whole folder is 45gb. I know, i know that is crazy i just forgot to whittle it down.So user plugin size is 20.8MB.

I tried to rename the folder but still no luck. The current name is, .Reaper Configzip. When i tried to remove the first full stop the file becomes unstable.

As for operating system- Windows 10-Reaper 6.21- I exported my configuration into my config folder i Reaper, and did not use a USB stick i used my ssd drive. Hope that is enough info?
jadedsean is offline   Reply With Quote
Old 02-04-2021, 01:45 PM   #22
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

Ok, rename the ReaperConfigZip file simply to "myconfig.zip" and try to open it.
Then extract all its files, dunno, to a subfolder on the desktop or something.
When it's done, check, how big the size of the folder with the extracted filed is. It should be roughly around the same size as the resourcefolder. If not, please post, how big it is.

Oh, and can you post screenshots of the error-messages of the import-dialog?
Maybe they give us a hint.
Meo-Ada Mespotine is offline   Reply With Quote
Old 02-04-2021, 03:15 PM   #23
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
Ok, rename the ReaperConfigZip file simply to "myconfig.zip" and try to open it.
Then extract all its files, dunno, to a subfolder on the desktop or something.
When it's done, check, how big the size of the folder with the extracted filed is. It should be roughly around the same size as the resourcefolder. If not, please post, how big it is.

Oh, and can you post screenshots of the error-messages of the import-dialog?
Maybe they give us a hint.

So i managed to do everything you said, check out the attached pics.
Attached Images
File Type: png Capture.PNG1.PNG (47.1 KB, 372 views)
jadedsean is offline   Reply With Quote
Old 02-04-2021, 03:17 PM   #24
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
Ok, rename the ReaperConfigZip file simply to "myconfig.zip" and try to open it.
Then extract all its files, dunno, to a subfolder on the desktop or something.
When it's done, check, how big the size of the folder with the extracted filed is. It should be roughly around the same size as the resourcefolder. If not, please post, how big it is.

Oh, and can you post screenshots of the error-messages of the import-dialog?
Maybe they give us a hint.
Sorry missed the second one.
Attached Images
File Type: png Capture.PNG (14.0 KB, 374 views)
jadedsean is offline   Reply With Quote
Old 02-04-2021, 04:50 PM   #25
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

Ah, ok. Really looks like it's only exporting parts of the config.
One last question: how big is the ReaperConfigZip-file itself?

@Justin
Could it be helpful to have a list of the available files in the resourcefolder with their filesizes?
Maybe this gives you a hint for further tests(I don't think jadedsean sending you the entire resourcefolder would be feasible...)
Meo-Ada Mespotine is offline   Reply With Quote
Old 02-04-2021, 05:12 PM   #26
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
Ah, ok. Really looks like it's only exporting parts of the config.
One last question: how big is the ReaperConfigZip-file itself?

@Justin
Could it be helpful to have a list of the available files in the resourcefolder with their filesizes?
Maybe this gives you a hint for further tests(I don't think jadedsean sending you the entire resourcefolder would be feasible...)
It’s just over 8gbs
jadedsean is offline   Reply With Quote
Old 02-04-2021, 05:12 PM   #27
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
Ah, ok. Really looks like it's only exporting parts of the config.
One last question: how big is the ReaperConfigZip-file itself?

@Justin
Could it be helpful to have a list of the available files in the resourcefolder with their filesizes?
Maybe this gives you a hint for further tests(I don't think jadedsean sending you the entire resourcefolder would be feasible...)
Do you think it’s retrievable?
jadedsean is offline   Reply With Quote
Old 02-04-2021, 07:50 PM   #28
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 16,124
Default

When you extracted the .zip, what does that folder look like? What files and directories are in it?
Justin is offline   Reply With Quote
Old 02-05-2021, 06:21 AM   #29
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Justin View Post
When you extracted the .zip, what does that folder look like? What files and directories are in it?
Here is what is within the folder.
Attached Images
File Type: png t1.PNG (35.0 KB, 376 views)
File Type: png t.PNG (35.7 KB, 376 views)
jadedsean is offline   Reply With Quote
Old 02-05-2021, 06:24 AM   #30
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,762
Default

Can you check, which folder contains the files it should have and which folders do not contain their files?

I assume, that the ProjectTemplate-folder is the first one, not containing all files compared to the original folder, so all subsequent folders should miss some or all of their files, but I might be wrong about that.

Edit:
I've added a small batchfile and put it into the attachements.
Extract it to your resourcefolder and to the folder, in which you've extracted the the files from the ReaperConfigZip.

Now doubleclick the getfiles.bat in the resource-folder and run the getfiles.bat in the extracted folder.
It will create two files in C:\
C:\listoffiles.txt
C:\listoffiles2.txt

who contain all the filenames and their sizes of your resource-folder and the extracted folder. Send them to support@cockos.com.

If you accidentally run one of the getfiles.bat twice in the same folder, just delete C:\listoffiles.txt and C:\listoffiles2.txt and try again. But as long as these two files have a different size in bytes, you should be fine.

(Hope it's not too tricky. It was the easiest way to retrieve all filenames and their sizes I could think of right now...)
Attached Files
File Type: zip getfiles.zip (175 Bytes, 348 views)

Last edited by Meo-Ada Mespotine; 02-05-2021 at 06:42 AM.
Meo-Ada Mespotine is offline   Reply With Quote
Old 02-05-2021, 07:20 AM   #31
jadedsean
Human being with feelings
 
Join Date: Jan 2016
Posts: 44
Default

Quote:
Originally Posted by Meo-Ada Mespotine View Post
Can you check, which folder contains the files it should have and which folders do not contain their files?

I assume, that the ProjectTemplate-folder is the first one, not containing all files compared to the original folder, so all subsequent folders should miss some or all of their files, but I might be wrong about that.

Edit:
I've added a small batchfile and put it into the attachements.
Extract it to your resourcefolder and to the folder, in which you've extracted the the files from the ReaperConfigZip.

Now doubleclick the getfiles.bat in the resource-folder and run the getfiles.bat in the extracted folder.
It will create two files in C:\
C:\listoffiles.txt
C:\listoffiles2.txt

who contain all the filenames and their sizes of your resource-folder and the extracted folder. Send them to support@cockos.com.

If you accidentally run one of the getfiles.bat twice in the same folder, just delete C:\listoffiles.txt and C:\listoffiles2.txt and try again. But as long as these two files have a different size in bytes, you should be fine.

(Hope it's not too tricky. It was the easiest way to retrieve all filenames and their sizes I could think of right now...)
Please forgive ignorance here but, when you say resource folder do you mean this directory?
C:\Users\Sean\AppData\Roaming\REAPER.
I only ask as this version of Reaper is a portable install and not my main Reaper on my C drive.
jadedsean is offline   Reply With Quote
Old 02-05-2021, 08:00 AM   #32
vitalker
Human being with feelings
 
vitalker's Avatar
 
Join Date: Dec 2012
Posts: 13,636
Default

Quote:
Originally Posted by jadedsean View Post
Please forgive ignorance here but, when you say resource folder do you mean this directory?
C:\Users\Sean\AppData\Roaming\REAPER.
I only ask as this version of Reaper is a portable install and not my main Reaper on my C drive.
You always get the right folder when using Options -> Show REAPER resource path.
vitalker is offline   Reply With Quote
Old 05-25-2023, 02:08 PM   #33
BartR
Human being with feelings
 
BartR's Avatar
 
Join Date: Oct 2014
Location: Somewhere in this universe or so.
Posts: 1,823
Default

Quote:
Originally Posted by Justin View Post
This was fixed in 6.15. If you have an issue with it, would be interested to know the details, but in the interim you can also backup your config manually, you can open the resource path in REAPER (options menu, show resource path) and backup the entirety of that folder.
Hello Justin, it seems that on 6.79 i got this kind of issue back again :/
Please here the topic:

https://forum.cockos.com/showthread....58#post2679858

thank you if you'd like/can take a look for it.
__________________
Reaper: always the most up-to-date.
O.S.: Windows 11 Pro
ReaPack (with bilingual Tutorials): https://bit.ly/ReaPack_Repository
BartR is offline   Reply With Quote
Old 05-26-2023, 07:26 AM   #34
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 16,124
Default

Quote:
Originally Posted by BartR View Post
Hello Justin, it seems that on 6.79 i got this kind of issue back again :/
Please here the topic:

https://forum.cockos.com/showthread....58#post2679858

thank you if you'd like/can take a look for it.

Thanks, fixing! This was broken when we updated our zlib to 1.2.12 in 6.57.
Justin is offline   Reply With Quote
Old 05-26-2023, 09:41 AM   #35
BartR
Human being with feelings
 
BartR's Avatar
 
Join Date: Oct 2014
Location: Somewhere in this universe or so.
Posts: 1,823
Default

Quote:
Originally Posted by Justin View Post
Thanks, fixing! This was broken when we updated our zlib to 1.2.12 in 6.57.
Thank you Justin, I appreciate
__________________
Reaper: always the most up-to-date.
O.S.: Windows 11 Pro
ReaPack (with bilingual Tutorials): https://bit.ly/ReaPack_Repository
BartR is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:08 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.