mp3s over the network (media explorer) freezing Reaper

[ferropop]

I have a NAS full of audio.

When previewing an mp3 (especially long ones) in Media Explorer, Reaper completely freezes for a long time and eventually recovers.

The slower the internet connection, the longer it takes to recover.


What I think is happening is, Reaper is buffering the entire mp3 which is different from playing WAV or even FLAC which begins playing immediately while buffering in the background.

Any chance this can be looked at? I've had to convert mp3s to FLAC (lol) just to navigate these types of folders.

[ferropop]

1. I click on mp3s. they take multiple seconds to load over the network, it seems like it's loading the entire mp3 before starting playback

2. those same mp3s but converted to flac - Instant preview

Even on reasonable network conditions, having to wait for a 100+MB mp3 to Completely load before starting the preview, is not ideal. Especially when FLAC (larger files!) loads instantly.

Thank you!

[Justin]

It builds an index of the mp3 file to support sample-accurate seeking. It should be possible to have it use the Xing/LAME header to avoid some of that (for short files, anyway), but for now live with it.

also FYI for larger files (more than 12MB), it creates a .reapindex file so it only has to build the index once.

[ferropop]

Originally Posted by Justin

It builds an index of the mp3 file to support sample-accurate seeking. It should be possible to have it use the Xing/LAME header to avoid some of that (for short files, anyway), but for now live with it.

also FYI for larger files (more than 12MB), it creates a .reapindex file so it only has to build the index once.

Thanks Justin, just happy to put it on the radar and please if it's something that can get tweaked at some point that's fantastic

[ferropop]

Originally Posted by Justin

It builds an index of the mp3 file to support sample-accurate seeking. It should be possible to have it use the Xing/LAME header to avoid some of that (for short files, anyway), but for now live with it.

also FYI for larger files (more than 12MB), it creates a .reapindex file so it only has to build the index once.

Justin is it possible to generate .reapindex files for an entire folder recursively? if this is a one-off problem that can be mitigated in a bulk way (.reapindex generation) that would be great. unsure what triggers their creation, do we just drag files into Arrange View or is it Media Explorer itself that creates the index files?

thanks again.

[Justin]

Originally Posted by ferropop

Justin is it possible to generate .reapindex files for an entire folder recursively? if this is a one-off problem that can be mitigated in a bulk way (.reapindex generation) that would be great. unsure what triggers their creation, do we just drag files into Arrange View or is it Media Explorer itself that creates the index files?

thanks again.

Not easily -- side note, are your files CBR or VBR? If CBR we can probably optimize them to not require an indexing.

[ferropop]

Originally Posted by Justin

Not easily -- side note, are your files CBR or VBR? If CBR we can probably optimize them to not require an indexing.

almost always CBR yeah, and for one-offs maybe this is impetus to convert them to CBR

[mschnell]

Originally Posted by ferropop

having to wait for a 100+MB mp3 to Completely load before starting the preview, is not ideal.

100 MByte through a 1 GBit Network is abut one second transfer time. Seems not critical.

(I did have huge latency with my NAS when using software that opens some 1000 small files - due to mechanical disk rather than SSD, but this is not the case here.)

[ferropop]

Originally Posted by mschnell

100 MByte through a 1 GBit Network is abut one second transfer time. Seems not critical.

(I did have huge latency with my NAS when using software that opens some 1000 small files - due to mechanical disk rather than SSD, but this is not the case here.)

mschnell not for nothing, but I show above that it's critical given that the entire DAW freezes for a variable number of seconds dependent on file size and connection speed. If I'm at a coffee shop, loading a 25MB file over a 2.5Mbps connection takes 100 seconds, during which Reaper is frozen. FLAC and WAV preview instantaneously, for comparison. This is a totally legitimate thread.

also Justin chimed in and seems willing to optimize for CBR (which will fix this!) so I'm not sure why the pushback?

[mschnell]

No pushback at all. Just confirming that the network is not the problem but something else .

[Justin]

could also make you a .reapindex generator for the command line, let me know what OS you want it for (or if you want to compile it yourself)

[ferropop]

Originally Posted by Justin

could also make you a .reapindex generator for the command line, let me know what OS you want it for (or if you want to compile it yourself)

Oh man thank you so much, for Windows would be amazing. Do they get saved alongside the mp3s or in the peaks folder?

[Justin]

Originally Posted by ferropop

Oh man thank you so much, for Windows would be amazing. Do they get saved alongside the mp3s or in the peaks folder?

https://landoleet.org/unreleased_plu..._reapindex.exe

it will go alongside the media unless you use the -subdir option (which will put them in peaks/)

it will not create .reapindex for CBR files (the next builds will not need them), unless you use the -forceforcbr option (which would be good for 7.35 etc).

as a side note, 7.35 and earlier ignore .reapindex for files that are smaller than 12MB, so this won't help you for those small files. in the next build, there will be an .ini file setting you can set to reduce that threshold to use .reapindex for smaller files.

[ferropop]

Originally Posted by Justin

https://landoleet.org/unreleased_plu..._reapindex.exe

it will go alongside the media unless you use the -subdir option (which will put them in peaks/)

it will not create .reapindex for CBR files (the next builds will not need them), unless you use the -forceforcbr option (which would be good for 7.35 etc).

as a side note, 7.35 and earlier ignore .reapindex for files that are smaller than 12MB, so this won't help you for those small files. in the next build, there will be an .ini file setting you can set to reduce that threshold to use .reapindex for smaller files.

Really really appreciate this Justin, it'll make a tremendous difference!

[ferropop]

Here's some powershell for anyone wanting to run this on their entire sample drive. Just place both this and the exe in the root:

Get-ChildItem -Recurse -Filter *.mp3 | ForEach-Object {
    & "build_reapindex.exe" -subdir $_.FullName
}

[Justin]

Originally Posted by ferropop

Here's some powershell for anyone wanting to run this on their entire sample drive. Just place both this and the exe in the root:

Get-ChildItem -Recurse -Filter *.mp3 | ForEach-Object {
    & "build_reapindex.exe" -subdir $_.FullName
}

you can also do build_reapindex -subdir C:\path\to\mp3s

[ferropop]

MPEG layer 3, @ 44100 Hz 2 channels
Length: 1:23:49.356
Encoding: CBR (no header)
Channel Mode: Joint Stereo
Bitrate (average): 320 kbps
192531 frames in file [indexed]
Metadata:
    Date: 2013
ID3 tags:
    TDAT: 3103
    TIME: 2023
    TYER: 2013
XMP tags:
    dc/format: audio/mpeg
Other file sections:
    ID3:PRIV:XMP

Hey there Justin. This mp3 (hundreds of MB) froze up Reaper for a few minutes. Is this file not affected by your recent work on this?

This mp3 was touched by the above util you wrote btw, just so you know what was in place.

Thank you so much this is so so helpful if it can work!

[Justin]

Originally Posted by ferropop

MPEG layer 3, @ 44100 Hz 2 channels
Length: 1:23:49.356
Encoding: CBR (no header)
Channel Mode: Joint Stereo
Bitrate (average): 320 kbps
192531 frames in file [indexed]
Metadata:
    Date: 2013
ID3 tags:
    TDAT: 3103
    TIME: 2023
    TYER: 2013
XMP tags:
    dc/format: audio/mpeg
Other file sections:
    ID3:PRIV:XMP

Hey there Justin. This mp3 (hundreds of MB) froze up Reaper for a few minutes. Is this file not affected by your recent work on this?

This mp3 was touched by the above util you wrote btw, just so you know what was in place.

Thank you so much this is so so helpful if it can work!

hmm lacks a proper LAME header, wonder why? what was it created with?


can you compare the .reapindex file generated by the util vs the one available after REAPER generates it?


How is REAPER configured for peak-paths? Maybe the util-generated one is in peaks/ and REAPER isn't configured to look in peaks/?

[ferropop]

Sorry this took a sec!

It's still having to load the entire mp3 over the network for me, despite /peaks existing for the target mp3s, so I'll answer the above questions:

Originally Posted by Justin

hmm lacks a proper LAME header, wonder why? what was it created with?

-it's files pulled from Archive.org, not sure if they have some internal encoder standard because it's happening on all files from there

Originally Posted by Justin

can you compare the .reapindex file generated by the util vs the one available after REAPER generates it?

They differ by one character!

Originally Posted by Justin

How is REAPER configured for peak-paths? Maybe the util-generated one is in peaks/ and REAPER isn't configured to look in peaks/?

Always here to help squash!

[Justin]

Originally Posted by ferropop

Sorry this took a sec!

It's still having to load the entire mp3 over the network for me, despite /peaks existing for the target mp3s, so I'll answer the above questions:


-it's files pulled from Archive.org, not sure if they have some internal encoder standard because it's happening on all files from there



They differ by one character!







Always here to help squash!

Hmm link to the file?

This one looks very similar but not the same, maybe yours is truncated slightly?

https://archive.org/details/KmartFebruary1990

anyway post a link to your actual file and I'll take a look, maybe it's corrupt and reaper vs build_reapindex do something slightly different.

[ferropop]

Thanks so much.

mp3 and reapindex here :

https://www.dropbox.com/scl/fo/ucu0m...a55m92bxg&dl=0

[Justin]

Originally Posted by ferropop

Thanks so much.

mp3 and reapindex here :

https://www.dropbox.com/scl/fo/ucu0m...a55m92bxg&dl=0

Confirmed that's the exact same file.


Re: your post in https://forum.cockos.com/showthread....66#post2857966, what version of REAPER was that output from?


What does it show if you view the properties now with 7.48? And what OS/arch?

[ferropop]

Originally Posted by Justin

Confirmed that's the exact same file.


Re: your post in https://forum.cockos.com/showthread....66#post2857966, what version of REAPER was that output from?


What does it show if you view the properties now with 7.48? And what OS/arch?

I imagine it was 0403, I install the pres daily.


Not sure if it's the same file, but same collection for sure. On 7.48 today :

MPEG layer 3, @ 44100 Hz 2 channels
Length: 1:24:42.515
Encoding: CBR (no header)
Channel Mode: Joint Stereo
Bitrate (average): 320 kbps
194566 frames in file [indexed]
Metadata:
Date: 2013
ID3 tags:
TDAT: 3003
TIME: 1325
TYER: 2013
XMP tags:
dc/format: audio/mpeg
Other file sections:
ID3:PRIV:XMP


Win 10/11 x64 Windows

[Justin]

Originally Posted by ferropop

I imagine it was 0403, I install the pres daily.


Not sure if it's the same file, but same collection for sure. On 7.48 today :

MPEG layer 3, @ 44100 Hz 2 channels
Length: 1:24:42.515
Encoding: CBR (no header)
Channel Mode: Joint Stereo
Bitrate (average): 320 kbps
194566 frames in file [indexed]
Metadata:
Date: 2013
ID3 tags:
TDAT: 3003
TIME: 1325
TYER: 2013
XMP tags:
dc/format: audio/mpeg
Other file sections:
ID3:PRIV:XMP


Win 10/11 x64 Windows

I compared the files, they are byte-for-byte the same.


As far as this goes, apparently there was a bug in 0403, however there are no changes to the mp3 decoder's code since then. So maybe it was a different file or it failed reading some of the file over the network. Can you reproduce any issue (reapindex getting regenerated for that file after being made by the command line tool, etc?) now?