Separating audio by dynamics - Tell me why this will not work!

[serr]

This idea is almost certainly based on misinterpretation of a visual display…

I have audio that has music at a lower level and an voice-over on top of it. (Think radio DJ talking over music.) You can look at the waveform display and see a clear distinction between them. Let’s say the music peaks at -20db and you have clear bursts of voice-over that peak at -0.1db.

Very obvious visually. You’ve seen this before a thousand times.

So I have it in my head that there should be a way to set a threshold point just above the music and truncate everything below that point to leave only the DJ bursts.

Yes, yes, it’s called a gate…

That’s not what I’m after though!
Yes, you can gate it and leave you with the SECTIONS of audio containing the voice bursts.

What I want to do is magically erase all the audio data under that threshold point. Leave just the data that is making up the tops of those waveforms for the vocal bursts.

Visually this seems intuitive.

Now this just has to be a flawed concept right?
There’s some “meat” mixed in with the lower level stuff that can’t be separated out like that? Just the data for the tops of the waveforms would be incomplete somehow? Or doesn't really exist because I'm misinterpreting a visual display?
I’m being stupid considering such a thing right? Or… am I on to a digital concept to exploit?

If it worked, I’d then take the truncated voice bursts and try to use them to cancel the voice out of the music (as much as possible) and leave me with just the original music that was unceremoniously blabbered over.

OK, so make my day with a digital way to do that truncate move or tell my why I’m out to lunch please!

[Fergler]

Actually if you zoom in, you will see it is visually unintuitive. Both audio sources combine to form a single waveform. The only way to separate them is to have one of the sources already separate and subtract if from the both.

I will say what you want -is- possible provided:

You can find the exact same song in the same format as the one in your recording - i.e. if it's a 320kbs MP3 rendered down to 128kbs broadcast, try to get that exact signal chain going.

Secondly, if there's ducking or master compression going on, you will have artifacts no matter what.

[Bri1]

http://www.stillwellaudio.com/plugins/spectro/

[Fergler]

Spectro is not nearly accurate enough to remove the human voice from music, and it works only in rectangles.

Plus, you will be removing those frequencies from the music as well.

[Xenakios]

Separating different audio level ranges into different audio streams is a DSP process I've been thinking about exploring. However, I don't expect anything highly useful to come out of it. (For example it wouldn't work to separate voice from a music background as explained by Fergler above.)

[Bri1]

Originally Posted by Fergler

Spectro is not nearly accurate enough to remove the human voice from music, and it works only in rectangles.

Plus, you will be removing those frequencies from the music as well.

Multitracking recordings solves this problem-more people could adopt using them.

@

Originally Posted by serr

What I want to do is magically erase all the audio data under that threshold point. Leave just the data that is making up the tops of those waveforms for the vocal bursts.

Originally Posted by Bri1

http://www.stillwellaudio.com/plugins/spectro/

[serr]

Thanks for the sanity check Fergler!

Bri1, yeah looking at it in spectral view (I have iZotopeRX) makes it very clear that there is no such delineation in the digital data to exploit in that way.


Yep...
I should put on more coffee.

Thanks.

[serr]

Originally Posted by Bri1

Multitracking recordings solves this problem-more people could adopt using them.

@

That multitrack was recorded over in 1967 I'm afraid.

We're in restoration territory today!

[Bri1]

I should put on more coffee.

I would hint at less coffee-it's a powerfull drug you know.
Makes people -erratic? and stresses the internal immune system. :/

[Bri1]

Originally Posted by serr

That multitrack was recorded over in 1967 I'm afraid.

We're in restoration territory today!

Yes-but today people can multichannel record to their hearts content-it will make future lives much easier for all this type of rea-processings or further manipulations..

[serr]

We live in a golden age of audio right now. No question there!

Mix a live concert with your DAW system and bring home a full multitrack as an aside. Then deliver a full lossless HD 5.1 surround mix to the consumer in FLAC files.

The spectral editor can be quite the magic tool for separating audio elements actually. You can get away with things that weren't remotely possible any other way before.


Wait... Is coffee bad again?
Wasn't it just curing Alzheimers last week?

[Dr Bob]

Originally Posted by serr

... Wasn't it just curing Alzheimers last week?

Uhhh ... I forget ...

dB

[serr]

I can't quite let this go...

OK so drawing an imaginary line across the screen to separate audio at a dynamic threshold point might be a misconception based off misinterpreting the waveform displays.

But... the display is calculated from the audio data. Can't you even derive audio from making that drawing on pencil and paper? (You can. And you can even hear the earliest "recorded" ie drawn audio on youtube.)

So is there a way to make this digital truncation so I can at least listen to the results and prove to myself it's a useless idea?

[TryingToMakeMusic]

Your idea makes me cringe, nothing personal. The "loud" part isn't just loud, it's a waveform oscillating back and forth over the center of the waveform. It spends substantial time near the center, along with the quiet part you want to remove.

But this is the same cringe I did when Melda added this feature to his multi-band plugins. He added an option in his multi-band to let you separate by dynamics instead of by frequency. I haven't heard it ever. Maybe it could work for you.

[DVDdoug]

I can't quite let this go...

Multiple sounds exist at the same time, but there is only one sound wave/waveform.

If someone is talking while your listening to music or watching TV, the sound waves blend together in the air and there is no way to separate or isolate them. There is only one sound wave in the air and it's made up of the combined frequencies and amplitudes of both sources. If one sound is louder than the other it doesn't make it any easier.

The same thing happens when sounds are mixed digitally or electronically.

As Fergler says, you can subtract one from the other but you need at least one of the sounds alone so you subtract it. And usually if you have one of the sounds isolated, you probably have the other one isolated too so there's no need to subtract.

In order for subtraction to work, the isolated sound has to be identical to the mixed version of the sound... If you make two recordings of yourself saying "Hello" and you subtract them, it will sound exactly the same as adding (mixing) the two recordings.

But if you make an exact copy of the digital file, subtracting it from itself will give you dead silence. Or if you mix that "hello" with some music, you can perfectly subtract-out the "hello". (As long as the levels aren't changed during mixing, etc.)

[serr]

I get all that of course. But I still wonder if there isn't some digital approach to exploit. It was impossible to separate multiple broadband sources before spectral editing came about too. Someone exploited being able to work non-linearly with a visual element.

Since the data can be displayed to identify different elements visually, can't this be played with too?

I have no misconception of finding a single button push that does some task like this from start to finish and produces a final magical end result. But maybe a useful tool?

[serr]

Still can't let this thought go...

I tried this "spectral dynamics" plugin from Melda. Turns out the 'spectral' part in the name is more a marketing term and it's just a (really cheap and bad) watered down multiband comp with some broadband NR features thrown in (that work poorly). Oh well on that.


Anyone have any ideas for programming something with a JS plugin?
By all means continue to tell me how silly this is! But a formula or something to play with in the meantime?

[TryingToMakeMusic]

Track 1: your source
Track 2: your source through a hard clipper.
Track 3: the sum of Track 1 with the phase-inversion of Track 2.

Now Track 2 has just the quiet part and Track 3 has just the loud part.

[serr]

Originally Posted by TryingToMakeMusic

Track 1: your source
Track 2: your source through a hard clipper.
Track 3: the sum of Track 1 with the phase-inversion of Track 2.

Now Track 2 has just the quiet part and Track 3 has just the loud part.

The problem is I'm after the low level stuff, not the other way around.

I can peak limit the source. That gives me the low level stuff punctuated with now distorted bits of voice-over (but now level managed so they at least don't pummel the listener anymore).

But that puts them further away from being able to null with any perceived phase tricks.

I want to be able to draw an imaginary line across the display and truncate data below it. However that might translate to actual math.

Thank you for that suggestion though, TryingToMakeMusic.
Approaching it backwards might lead somewhere.

Double backwards because I'm looking to null with the result.

[TryingToMakeMusic]

Originally Posted by serr

... I still wonder if there isn't some digital approach to exploit. It was impossible to separate multiple broadband sources before spectral editing came about too. Someone exploited being able to work non-linearly with a visual element.

Can you please refer me to an example of "spectral editing" so I know exactly what you mean by that? Do you mean this: http://www.stillwellaudio.com/plugins/spectro/?

What you want to do might be possible but it's going to be really hard, like changing the pitch without changing the playback rate, people been working on that for a long time and the results are still iffy.

[audiocan]

Try ISSE : An Interactive Source Separation Editor.

http://isse.sourceforge.net/

[serr]

Originally Posted by audiocan

Try ISSE : An Interactive Source Separation Editor.

http://isse.sourceforge.net/

That looks promising.

I'm a little perplexed that he gets to the first draft stage in the video and then just stops though! I'd like to watch him follow through and see the steps with the tools where he takes it from the first draft to an actual successful separation. He stops every example before he has any of them actually working beyond a crude first draft.

Even if this one turns out not to work, this is exactly what I was asking to do!

[reapercurious]

Originally Posted by serr

Still can't let this thought go...

I tried this "spectral dynamics" plugin from Melda. Turns out the 'spectral' part in the name is more a marketing term and it's just a (really cheap and bad) watered down multiband comp with some broadband NR features thrown in (that work poorly). Oh well on that.


Anyone have any ideas for programming something with a JS plugin?
By all means continue to tell me how silly this is! But a formula or something to play with in the meantime?

melda is an amazing developer. their freeware kit has some plugins that i'd rather not do without. in their defense there are loads of different multiband plugins in the market to choose from, however, melda deserves a thriving business.

if you want to separate audio based on dynamics, try playing around with ReaFir in subtract mode. that does exactly that. use the mouse to grab the line, hold control and drag it down, then up with your track. at least you can use ReaFir as a detector to trigger your gate. Use routing. create a custom FX chain. If it can be done, then ReaJS can do it after you figure out what it is exactly that you are trying to do, in a practical example created by you using ReaFir and ReaGate to start the ball rolling.

I do not think your idea can work in its current form. what you do need is to create FX chains using ReaGate and ReaFir to start with, and once you get close to what you want, post an example. take your time so folks can grasp what it is that you say you want, using audio examples, and perhaps an fx chain to look at.

[serr]

The thing is the audio I'm working with is full program. It's not simply one track of a multitrack recording where once it's in a mix you don't hear the artifacts.

I've got to say that for all the love I have for Reaper and the stock Rea plugins, that I find ReaFIR is shockingly crude and useless! Yeah, you can get away with some crude moves on a single track in a mix...

I use Izotope RX for those tasks and it works very well with the right finesse and workflow tricks. ReaFIR is a big flat blade screwdriver. When you need a star point jewelers driver...
I feel like I must be missing something with this one. I can only get it to act like a very coarse version of what Izotope has had available for many years now.

I AM asking for seemingly impossible tricks! It seems reasonable with what I've seen and been able to pull off with iZotope's spectral editor that someone could find a crafty way to combine spectral identification with photoshop like features for boundary selection. That ISSE looks to be doing just that (in spite of the poor demo video).

I'll report back on that in a bit.


Melda plugins look well worth investigating in general. I happened to be on a specific quest here and that particular one just happened to have a name that suggested it might do something it was never actually intended to do. Not a fail. Just not the tool it appeared to be at first glance.