Spleeter/Demucs for Reaper (Extract Drums Bass Vocals etc.. from songs) UPDATED

[MusoBob]

EDIT:Updated error in spleeter 2 stems
They should be all updated now, so let me know how they go.
Right click Save Link\Target As
ReaTrak demucs audio item to 4 stems.lua
ReaTrak spleeter audio item to 2 stems.lua
ReaTrak spleeter audio item to 4 stems.lua
ReaTrak spleeter audio item to 5 stems.lua

[MusoBob]

Originally Posted by serr

...Tip #3:
The result fidelity here is so lo-fi crude that this probably isn't worth the time IMHO. But I haven't experimented with my own "training" yet.

Originally Posted by Fabian

Yes, Spleeter seems to cut off everything above 11 kHz. I used the original file to fill in that, as I am most interesting in removing the vocals from some old recordings, this works. But if you have stuff up there that you want to keep...

See https://github.com/deezer/spleeter/w...-output-files-

The released models were trained on spectrograms up to 11kHz. The three base models (spleeter:2stems, spleeter:4stems and spleeter:5stems) discard frequencies above 11kHz in the separated files.

However, there are several ways of performing separation up to 16kHz or even 22kHz:

We released alternate config files for which the F parameter was set to 1536 instead of 1024. The same pretrained model is used (trained with spectrograms up to 11kHz), but spectrogram estimation at separation time is then done until 16kHz. This may result in unexpected artefacts, but so far, it is the easiest and best performing way of doing separation above 11kHz. The config files are packaged under the name spleeter:2stems-16kHz, spleeter:4stems-16kHz and spleeter:5stems-16kHz. You can use them straightforwardly using the -c option:

spleeter separate -i my_audio_file.mp3 -c spleeter:4stems-16kHz -o /output/path

You can modify some parameters in the config json file. There are actually two parameters that will affect the maximum frequency of the separated files:
the mask_extension option can be set to zeros (default) or to average (see the wiki). The latter extends the mask values to the average value of the masks under 11kHz. This is a very basic way of modelling the content above 11kHz so be wary that it generates some interferences in the high frequencies.
the F option define the frequency dimension in frequency bin of input/output spectrogram (see the wiki). The released models were trained with F=1024, but it is actually possible to change it to any value smaller than 2048 which is a multiple of 64 for separation. So you can set it to 2048 to perform separation until 22kHz. Just be aware that models were not trained above 11kHz so spectrogram estimation may lead to unexpected results in this frequency range (while usually much better than basic mask extension) and that doubling the value of F will results in double RAM usage (and then may results in Out Of Memory errors). F=1536 (16kHz) is a good compromise between frequency bandwidth and memory usage and was chosen for the alternate config files presented above.

[Fabian]

Ah! You mean like... RTFM? Sorry
OK, will do.

Adding that 16 kHz switch is of course easy. I'll try it. Thanks.

BTW, I tweaked your script to make it handle all selected items. Maybe something to consider?

Thanks.

[MusoBob]

Originally Posted by Fabian

..
BTW, I tweaked your script to make it handle all selected items. Maybe something to consider?
Thanks.

You will have to give an example of what you are doing.

[WarrenG]

Originally Posted by MusoBob

You will have to give an example of what you are doing.

Please

I (actually you guys) got the 16k working, but stems collectively aren't an exact match to original, closer but missing a lot of the air in upper range. Individual stems Drums for example are missing the upper register I.e. snare cymbals. How do you train it for that? Without accidently bringing in the hi end of other instruments?
Hmmm

Thanks
W

[MusoBob]

I haven't tried 16kHz or even 22kHz with Spleeter yet but the Demucs gave better drums.

[Fabian]

Originally Posted by MusoBob

You will have to give an example of what you are doing.

I just read the number of selected items and run a loop around the code to handle each selected item, instead of just the first one. Like this:

  if os_type == "Win32" or os_type == "Win64" then
  
    local num_items = reaper.CountSelectedMediaItems(0)
    
    if num_items == 0 then
      reaper.MB( "Select at least one audio item", "Select Audio Item", 0 )
      return
    end 
    
    for i = 0, num_items-1 do
      end_script = 0
      sel_item = reaper.GetSelectedMediaItem( 0, i)

      if sel_item then 
      -- and the rest with an appropriate "end" added for the for-loop.

This recomputes the anaconda path and the command string on each iteration, but I didn't bother about that. I also did not bother to add the generated stems to the project, it would take some more string fiddling to get them properly named so I just skipped it.

[MusoBob]

I mean are you doing a lot of full length songs or a lot of short clips ?
as it inserts each media item on a new track at the start of the project.

[Fabian]

Originally Posted by MusoBob

I mean are you doing a lot of full length songs or a lot of short clips ?
as it inserts each media item on a new track at the start of the project.

I "spleetered" a whole LP, 12 full length songs from the early 1980's ripped from vinyl. This will now be re-mixed and re-mastered. Possibly with new guitar overlays, and new vocals.

And as I wrote, I "did not bother to add the generated stems to the project", meaning I commented out those lines. The spleeter_output folder holds all I need and want. I have now created new projects, one for each of the 12 songs and started to re-mix.

[Nikibugsound]

Originally Posted by chip mcdonald

My Reaper resource path is set to AppData/Roaming, I had to put it in Program Files/REAPER (x64)/InstallData/Scripts.

Aha, this has got it working for me too. On running the scripts I still get a Windows console box that says "processing please wait... The system cannot find the path specified." but it all goes ahead nonetheless and does its thing.

My knowledge of Reaper scripts goes no further than being able to install and run them, and maybe doing small edits if clear instructions are given, so I've no idea what the console message means.

Apparently all good anyway, great stuff. I came across Spleeter via Acon Acoustica and then Virtual DJ, so I'm well chuffed to find that I can run it directly inside my regular DAW. Cheers all.

Edit - After reading my own post it occured to me that maybe I just needed to wait longer with the original set up. So I removed spleeter from InstallData/Scripts and it still works. Still get the "cannot find" message but it all works great.

I'm still gobsmacked at how well this spleeter software works. I mean, sure it's not perfect, but for free code (and even realtime processing in Virtual DJ) it's pretty damn amazing imho, especially when I think back to starting out with a couple of hardware synths and samplers and Cubase 2 on an Atari ST in the early 90s.

I reckon with a bit of careful pre- and post-tweaking of material, I'll get some quite usable results from this.

[Nikibugsound]

I'd like to make a video showing this off, and put it on YouTube. Is it ok to link to this thread for people?

[MusoBob]

Originally Posted by Nikibugsound

I'd like to make a video showing this off, and put it on YouTube. Is it ok to link to this thread for people?

Yes please, it will save me doing it.

[Nikibugsound]

Originally Posted by MusoBob

Yes please, it will save me doing it.

Cool : ) Cheers.

[WarrenG]

Originally Posted by Nikibugsound

Cool : ) Cheers.

If your so inclined a video showing how to train Spleeter for better results would be very helpful.

Warren

[V1b3zman]

I followed the instructions but all I get when I run the script is empty tracks. Any idea what I did wrong

[MusoBob]

Sounds like characters in the file path, try an easy path like C:\Temp\song.wav

[V1b3zman]

thank for your reply copied song renamed "song" and put in C:\TEMP1 samething. exept now action list says the the action already exist and if I'd like to replace. is there a way to do a full uninstall of both anaconda and script to start completely from scratch?

[serr]

I'm not sure if this might be the issue...

Spleeter can only use Python v2. I remember seeing a couple different tutorials that told you to look at what version of Python you had installed (v2 or v3) and then proceed to install Anaconda to match the Python version and so on.

This is a mistake! If you have Python v3, you need to install v2 and follow from there.

And then you can be disappointed by the results when you run your first test!

I tried the extended 16k "high quality" training initially. That's what I was describing as crude.

[kulich]

My recording computer has no internet access.

I tried the online version at ezstems for fun to see what I'd get. Then decided to follow a walkthrough to install Spleeter on my office computer .... apparently installed but I had no joy testing (FWIT the context spleeter -h did not return the help menu).

Wandered onto a page that hosts a gui and successfully installed and tested Spleeter that way - https://ezstems.com/installing-spleeter-windows/.

Was thinking I could use this to isolate a gtr solo I did years ago on someone else's track. It works to a point - all acoustic gtr and the electric solo are out front so if I have the time I could rebuild the song.

Just my 2 cents.

P

[serr]

Yeah, the SOP for command line apps is to let an installer script connect to various online libraries to download and collect all the required packages. Doing that manually... You'll just have to learn what all the requirements are, download everything and prepare an installer or install procedure for yourself.

At the end of the day the results are definitely karaoke territory. You will not get professional results. You might get lucky for the right kind of sound element to isolate sometimes though. This is still an interesting advance even if it's not quite there yet!

[MusoBob]

I just tried it again on mine and it's working fine.
You need to make sure you install the version from the links in the first post and follow the guide.
Have a read through this thread there are more solutions.
The scripts are for the Anaconda version.
If you are using another version you will need to modify the scripts.

[V1b3zman]

okay, so I installed the latest Anaconda2 install on a new computer. with a fresh install of reaper. renamed spleeter master and placed in scripts folder. then changed the directory with the command promt and installed conda. ran the script and I get a new error "can't find Anaconda. any Ideas?
thanks

[MusoBob]

Originally Posted by MusoBob

..You need to make sure you install the version from the links in the first post and follow the guide...

Originally Posted by V1b3zman

okay, so I installed the latest Anaconda2 install on a new computer....

This is the version

https://repo.anaconda.com/archive/Anaconda3-2019.10-Windows-x86_64.exe

[V1b3zman]

okay we'll try again
thanks

[V1b3zman]

update unistalled all anaconda started from scratch again used the anaconda link in this thread clicked yes when asked to update installalled. BTW ran everything as administrator. fired up reaper ran the action result again is 5 empty tracks this is so frustrating any one have any idea whats happening?

[MusoBob]

Do you get anaconda prompt with the trace ?
This is real time screen cap, watch it all until the tracks are in:

[V1b3zman]

yes that screen comes up while the empty tracks are created then closes and nothing else happens

[MusoBob]

But what is on the screen ?

[6xes]

theres now a GUI of spleeter(standalone app)... works but has a tendancy to bluescreen intermittently..
it really drills into the CPU... but does a great job.. very handy for stems!!

[6xes]

https://makenweb.com/SpleeterGUI

Note: you must have the AVX instruction set on your cpu for spleeter to work

[fitzy1]

just thought I would let you know of a small bug... DONT have a space in front of the dot eg: 'name .wav' or 'name .mp3' ...these fail .If your file name has a space then the dot, it wont work. seems ok with () and - in the title, hope that makes sense, haven't tried other symbols yet like # = + [] these might be ok but remove them if spleeter fails. dave

edit: I can confirm AVX instruction's must be on pc

[fixerdave]

hello, I have logged this on git hub.. dave

[Anshul999]

I just tried the demucs script, as I'm already using SpleeterGUI but I wanted to try demucs as well. With some effort and small tweaks, I made the demucs script work as intended, but it does not insert the tracks into Reaper. It does the processing properly, and it will create the stem tracks in the appropriate folder with the source file, but it is unable to insert the media into reaper. I have to navigate to that folder, and manually insert the audio files (the audio files themselves are fine).

What could be the reason for this? I'm a script noob but I went through it anyway, and I tried modifying this line:

source_name = GetFilename(filename)
folder_name = string.gsub(filename, '[.]%w%w%w$','')
reaper.InsertMedia( filepath .. "demucs_extra//" .. folder_name .. "//bass.wav", 1 )
reaper.InsertMedia( filepath .. "demucs_extra//" .. folder_name .. "//drums.wav", 1 )
reaper.InsertMedia( filepath .. "demucs_extra//" .. folder_name .. "//other.wav", 1 )
reaper.InsertMedia( filepath .. "demucs_extra//" .. folder_name .. "//vocals.wav", 1 )

However, nothing that I did made any difference. The process works fine, and it creates the stems, but it just inserts 4 empty tracks. Otherwise, this script is very cool and definitely going to help a lot alongside SpleeterGUI (on a side note, I have requested the SpleeterGUI dev to consider adding demucs to his application for easy processing).

Thanks!

[MusoBob]

Did you try it with a short path and filename like C:\Temp\Song.wav as it maybe a character in the path or filename.

[Anshul999]

Originally Posted by MusoBob

Did you try it with a short path and filename like C:\Temp\Song.wav as it maybe a character in the path or filename.

Yep. I tried that too, with different path names and very simple file names. Still didn't work. I'm trying to understand what's happening, but the command line window simply closes after having processed the files, so I don't know exactly where the error is.
Also, the folder names for demucs is different since I am using the 'demucs extra' model, and it names the folder 'demucs_extra'. Hence I changed the name within the script too. But it doesn't seem to work with the default one either

[Anshul999]

Okay I just realized that the problem is that I was using FLAC files. It works fine with WAV or MP3 files. Is there any code modification I may do to use it with FLAC? I regularly use FLAC songs and convert them to Spleeter stems (and now demucs stems too), and I wouldn't want to re-render each file again to make it work properly.


And also, the tracks are added as individual tracks after the original track. Could you recommend any custom action that can automatically put these in a folder? [Edited: solved this using custom actions]
Thanks!

[MusoBob]

This will create child track, just add it in

    if sel_item then
    
      item_track = reaper.GetMediaItemTrack( sel_item )
      
      reaper.SetTrackSelected( item_track, 1 )
      
      sel_track =  reaper.GetSelectedTrack( 0, 0 )
      
      reaper.SetMediaTrackInfo_Value(sel_track,"I_FOLDERDEPTH",1)
    
      reaper.Main_OnCommand( 41173, 0 )

This will do 3 or 4 character extensions, just replace the current code

       os.execute(COMMAND)
       --os.execute([[cmd.exe /C " C:\ProgramData\Anaconda3\Scripts\activate.bat & echo processing please wait... & cd ]] .. script_path2 ..[[ & spleeter separate -i "]] .. filename .. [[" -p spleeter:2stems -o "]] .. filepath .. [[spleeter_output"]])                              
       source_name = GetFilename(filename)
       if string.find(source_name, '[.]%w%w%w$') then
         folder_name = string.gsub(source_name, '[.]%w%w%w$','')
       else
         folder_name = string.gsub(source_name, '[.]%w%w%w%w$','')
       end

[Anshul999]

Originally Posted by MusoBob

This will create child track, just add it in

    if sel_item then
    
      item_track = reaper.GetMediaItemTrack( sel_item )
      
      reaper.SetTrackSelected( item_track, 1 )
      
      sel_track =  reaper.GetSelectedTrack( 0, 0 )
      
      reaper.SetMediaTrackInfo_Value(sel_track,"I_FOLDERDEPTH",1)
    
      reaper.Main_OnCommand( 41173, 0 )

This will do 3 or 4 character extensions, just replace the current code

       os.execute(COMMAND)
       --os.execute([[cmd.exe /C " C:\ProgramData\Anaconda3\Scripts\activate.bat & echo processing please wait... & cd ]] .. script_path2 ..[[ & spleeter separate -i "]] .. filename .. [[" -p spleeter:2stems -o "]] .. filepath .. [[spleeter_output"]])                              
       source_name = GetFilename(filename)
       if string.find(source_name, '[.]%w%w%w$') then
         folder_name = string.gsub(source_name, '[.]%w%w%w$','')
       else
         folder_name = string.gsub(source_name, '[.]%w%w%w%w$','')
       end

Thank you so much! Everything works perfectly now. I made these modifications to the script (excluding child track, as I added a custom action within reaper that does that instead), along with a few other tweaks and I hope you @MusoBob don't mind me sharing these as my MOD versions of the scripts.
The main difference is:

1) These scripts work with the latest version of anaconda3, python 3.8, and also the latest versions of Demucs and Spleeter.
2) The Demucs script uses the 'demucs extra' training model as I found it offers slightly better results than the normal one (but this download will be larger FYI)
3) The Spleeter script now uses the 16kHz version of the model, so it offers better results than the older 11kHz model. I only use 4 stems, so I only edited that, but same could be applied to any of them.

Note: I had to remove certain flags from both spleeter and demucs scripts as they were causing errors, so you should cross-check if you get any errors yourself. It was important for me to delete the 'System32\libiomp5md.dll' file on Windows 10 to make them work, so maybe necessary for other Windows users too.
As noted, now this modified script also works with FLAC files.

A huge thanks to MusoBob for these amazing scripts (including other Reatrak scripts). Totally changed my workflow.

[MusoBob]

Thanks ! the only difference I think is

[[" -p spleeter:2stems -o "]]

[[" -p spleeter:4stems -o "]]

[[" -p spleeter:5stems -o "]]

[Anshul999]

So I just installed this script on a new machine, and it's not working. I guess something is off somewhere.

I'm not able to see any error messages because the script window immediately closes after running.

I tried multiple things, and I can run demucs fine through command prompt, but I don't know what's wrong and where the script is showing an error


So how can I make the script window pause and show the error messages, so I can debug it?