Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Bug Reports

Reply
 
Thread Tools Display Modes
Old 09-15-2022, 06:50 AM   #1
LDEVO
Human being with feelings
 
Join Date: Jun 2019
Location: London, England
Posts: 18
Default Media Explorer database size limitations and search efficiency issues (Reaper 6.66)

**Originally Posted In Wrong Forum
Hi there,

Repro Steps In Short:
- Have a Database with a large number of files (600,000 approx)
- Search in that database in media explorer
- Observe Slowness

Having used Reaper Media Explorer for years I have recently done a large update on my sound libraries and have most of them stored in one Database as I like having the option of searching everything all at once if needed. This is in addition to more granulated libraries.

This large database has approximately 614,000 (7.7TB) files in it - I appreciate this is a large amount and it is roughly 50% increase to the size of what the database was before - since this increase I've found media explorer to be quite slow and sometimes unresponsive specifically when searching (copying to session is not a problem). I am wondering whether this is a simple fix for this or whether its just down to the number of files in that Database.

For reference my setup details are as follows:
Windows 10 Enterprise (21H1)
Reaper Version 6.66
CPU - 32 Core Threadripper
128GB RAM
SFX Library is on a Network drive with Up/Down of roughly 1GB/s either way
Database is stored locally.
Reaper runs on a 1TB SSD that has about 200GB spare space
LDEVO is offline   Reply With Quote
Old 09-15-2022, 04:56 PM   #2
zaibuyidao
Human being with feelings
 
Join Date: Jan 2020
Location: Amoy
Posts: 140
Default

Can confirm. I created a large database with an 8T HDD (Local drive) and had the same problem. In comparison, Soundminer's database search speed is very fast.

The problem is that the larger the database, the slower the search speed. I am considering splitting a database into multiple ones in the hope that I can improve the search speed, but this is not the most efficient solution to the problem.
__________________
Zaibuyidao Github | ReaPack

Last edited by zaibuyidao; 09-16-2022 at 12:42 AM.
zaibuyidao is offline   Reply With Quote
Old 09-16-2022, 12:19 PM   #3
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 14,871
Default

Hmm can you send a link to a zip file of the very large database to support at cockos dot com? (obvs we don’t want the media just the reaperFileList file)
Justin is online now   Reply With Quote
Old 09-21-2022, 01:28 AM   #4
LDEVO
Human being with feelings
 
Join Date: Jun 2019
Location: London, England
Posts: 18
Default

Hey Justin - Sure thing, ill get this to you today. Thanks!
LDEVO is offline   Reply With Quote
Old 09-27-2022, 10:30 AM   #5
LDEVO
Human being with feelings
 
Join Date: Jun 2019
Location: London, England
Posts: 18
Default

Quote:
Originally Posted by Justin View Post
Hmm can you send a link to a zip file of the very large database to support at cockos dot com? (obvs we don’t want the media just the reaperFileList file)
Hi Justin, Can confirm I sent this over on the 21st - let me know if you have any issues with the link! Thanks.
LDEVO is offline   Reply With Quote
Old 10-17-2022, 06:44 AM   #6
hscottaudio
Human being with feelings
 
Join Date: Jul 2016
Posts: 97
Default

I'm also seeing this in v6.68 Windows 10, several seconds of unresponsive behaviour from Reaper (reaper locks up entirely) whilst the search takes place in a database of around 600,000 files.

I also out of interest, created 1,000,000 dummy wav files (without metadata though) and created a database with them.

This did not result in the slow searches I have been seeing with the real large databases, so possibly pointing in the direction of the amount of metadata being a contributing factor.
hscottaudio is offline   Reply With Quote
Old 11-09-2022, 11:32 AM   #7
Michael AD
Human being with feelings
 
Michael AD's Avatar
 
Join Date: Jul 2017
Location: Hackensack
Posts: 337
Default

I tested the Media Explorer database with a DB containing about 600,000 files. My search results were virtually instantaneous.

I know there was not much metadata in my DB, so perhaps as hscottaudio indicated, it may be the metadata slowing the search results for other users.

I am on Win11, 32GB RAM, AMD Ryzen 9 5900X, with the DB on an M.2 SSD.

BTW, I am really loving the way the Media Explorer has been developing - nice job devs!
Michael AD is offline   Reply With Quote
Old 11-09-2022, 03:41 PM   #8
hscottaudio
Human being with feelings
 
Join Date: Jul 2016
Posts: 97
Default

Issue still occurs in v6.69 Windows 64bit
hscottaudio is offline   Reply With Quote
Old 11-17-2022, 04:30 AM   #9
hscottaudio
Human being with feelings
 
Join Date: Jul 2016
Posts: 97
Default

Issue still occurs in v6.70 Windows 64bit
hscottaudio is offline   Reply With Quote
Old 11-17-2022, 08:52 AM   #10
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 13,998
Default

Looking into possible improvements here!
schwa is offline   Reply With Quote
Old 11-17-2022, 12:14 PM   #11
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 13,998
Default

The latest development builds should improve search performance on very large databases by quite a bit.
schwa is offline   Reply With Quote
Old 11-21-2022, 05:13 AM   #12
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: Berlin
Posts: 11,668
Default

RC1 of new release is in the pre-release section. Neat .
__________________
(Video)Using Latch Preview - Faderport16 for CSI v1.1 - Faderport8 for CSI 1.1
Website
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-29-2022, 02:40 AM   #13
hscottaudio
Human being with feelings
 
Join Date: Jul 2016
Posts: 97
Default

Awesome, yeah much faster in v6.71 for me now, thanks!!
hscottaudio is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 09:14 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.