[JSFX] Live Loudness Normalizer

[StarBrilliant]

GitHub

https://github.com/m13253/sb-jsfx-plugins

Bug Reports and Pull Requests are welcome. Updates will be released on GitHub first.

The GitHub version is always the latest (beta) version, which may contain bugs: Be sure to turn down your headphone before you run the latest version.

Introduction

This plugin is used for live audio streaming. A typical scenario is a live program with music and speech, but the loudness is changing from time to time so it is hard to predict or adjust the volume. Other loudness normalizer may not be able to handle realtime situation. Therefore, I wrote this plugin.

(Note: if you want to normalize non-realtime materials such as prerecorded albums, please use REAPER's built-in normalization.)

The algorithm measures the current Momentary Loudness, and compare it with the Target Loudness. When the difference between these values are high, the algorithm tries to adjust the gain at a higher rate. When the difference gradually approaches to zero, the rate of change also decreases to maintain audio quality and dynamic range. Additionally, special care is taken to handle the quiet sections and the silence between songs.

Parameters

1. Target Loudness (LUFS), default is -23 LUFS.
   The higher this value, the louder output will be.
   Reference values you can try:
   • YouTube: -14 LUFS
   • Spotify: -14 LUFS
   • Apple Music: -16 LUFS
   • TV & Radio: -23 LUFS
   • Pop music: -14 LUFS or lower
   • Speech: -16 LUFS
   • Classical music: -23 LUFS or lower
   • Background music: Foreground speech minus 20 LU
2. Decay Half-life (sec), default is 3 seconds.
   The gain adjustment follows an exponential decay curve: The farther difference from target, the faster gain adjustment will be. The longer decay half-life, the slower gain adjustment will be.
   Reference values you can try:
   • Music: 3 - 10 seconds
   • Speech: 3 seconds
   If you are curious, the gain adjustment formula is:
   

   diff_lu = target_lufs - current_lufs - gain_db
   
   d gain_db   diff_lu * ln(2)
   --------- = ---------------
       dt       half_life_sec

   when diff_lu > 0.
   
3. Upper Hard Limit (LU), default is +13 LU.
   Values in "LU" are added to Target Loudness (e.g. -23 LUFS + +13 LU = -10 LUFS).
   This is a hard limit to prevent sudden shock. Do not try to rely regularly on this, or the audio quality will be damaged.
   If really need a peak limiter, you can try NP1136 Peak Limiter, pre-installed with Reaper.
   Reference values you can try:
   • Pop music: +13 LU
   • Speech: +15 LU
   • Classical music: +22 LU
4. Lower Inflection Level (LU), default is -6 LU.
   Value in "LU" are added to Target Loudness (e.g. -23 LUFS + -6 LU = -29 LUFS).
   Reference values you can try:
   • Pop music: -6 LU
   • Speech: -9 LU
   • Classical music: -13 LU
   • Disable this feature: 0 LU
   In order to handle transitions from quiet sections and loud sections carefully, the gain adjustment curve is slightly different: When the current loudness is below the lower inflection level, the farther different from target, the slower (instead of faster) gain adjustment will become.
   If you are curious, the gain adjustment formula is:
   

                                                 2
   d gain_db   diff_lu * ln(2)         inflect_lu
   --------- = --------------- * ----------------------
       dt       half_life_sec           2             2
                                 diff_lu  + inflect_lu

   when diff_lu < 0 and inflect_lu < 0.
   
5. Bottom Gate Level (LUFS), default -48 LUFS.
   When the measured Momentary Loudness is below this level, either between two songs, or when no one is speaking, the algorithm treats the signal as silence.
   If your material has background noise, set this value higher than the loudness of that noise, or the noise will be amplified to the target loudness. A more ideal solution is to prepend a noise remover plugin.
   

License

This plugin is released under MIT license, which is a very permissive open-source license.
If you are willing to include my plugin in other projects, I recommend you to contact me before doing that. This will allow me to provide further updates more easily.

Update

I submitted a new version on 2022-09-29, which includes some new features you requested.
But be careful! Turn down your headphone before you use the new version!
I can't guarantee whether I introduced any new bugs or not.

[sonicowl]

Wow! This is wonderful. Thnx.

[Venn]

I use an Aphex Compellor for our live stream audio but this looks neat. I will play around with it.

[vitalker]

Originally Posted by Venn

I use an Aphex Compellor for our live stream audio but this looks neat. I will play around with it.

This one is much better. At least it's software, free and JSFX light on resources.

[lachinhan]

Thank you so much for this.
I would like to attach the preset for your Plugin
Thanks

[Madeonfan]

Thank you for jsfx.Its a good utility plugin for my live situations.Jsfx processing like soundradix powair's leveller unit.Am I right?

[mucknog]

Great plugin, thanks for the post.

One thing I don't quite understand.
The gain seems to be able to raise above the hard limit setting.
Shouldn't it stop there?

Another small thing. I feel, that the gain setting should not reset on playback start.

Cheers & thanks again

[rstockm]

Due to some recent REAPER changes (?), this plugin doesn't seem to work anymore (REAPER 6.67 MacOS M1/ARM vanilla install)...

[vitalker]

Originally Posted by rstockm

Due to some recent REAPER changes (?), this plugin doesn't seem to work anymore (REAPER 6.67 vanilla install)...

6.68, works as expected.

[StarBrilliant]

Originally Posted by mucknog

The gain seems to be able to raise above the hard limit setting.
Shouldn't it stop there?

The hard limit is a limit to the resulting signal, not the gain value.
If you set the hard limit to a small value, you will be able to notice it's limiting the output signal.

If you need one setting to limit gain, I can also add a gain limit setting.

Originally Posted by mucknog

Another small thing. I feel, that the gain setting should not reset on playback start.

It's not resetting on my side. (I even feel strange about it since sometimes I do need to reset it.)

[StarBrilliant]

Originally Posted by rstockm

Due to some recent REAPER changes (?), this plugin doesn't seem to work anymore (REAPER 6.67 MacOS M1/ARM vanilla install)...

I can't reproduce using the latest 6.68.
Would you please try to update to 6.68 and see if it fixes?

[rstockm]

Originally Posted by StarBrilliant

I can't reproduce using the latest 6.68.
Would you please try to update to 6.68 and see if it fixes?

Oh my. There seems to be some sort of Routing-foo going on on some of my projects. Your Plugin is just working fine, sorry for the distraction /o\

[nvk]

This is pretty amazing! Can see a lot of uses for this tool. I actually think it could be useful for non-live applications as well, like if you are trying to keep a music track at a consistent volume. Some kind of look-ahead setting would be great for that. And as others have mentioned, a way to limit the amount of gain applied to a range of some sort would be useful too.

[StarBrilliant]

Originally Posted by nvk

This is pretty amazing! Can see a lot of uses for this tool. I actually think it could be useful for non-live applications as well, like if you are trying to keep a music track at a consistent volume. Some kind of look-ahead setting would be great for that. And as others have mentioned, a way to limit the amount of gain applied to a range of some sort would be useful too.

If it's a VST plugin, then the plugin can tell REAPER how much latency it will introduce, and let REAPER compensate for that latency, effectively implementing the look-ahead feature.
But I don't find any way for a JSFX plugin to tell REAPER its latency. Therefore, the look-ahead can't be compensated. (There might be such a way. But I can't find it in the official documentation.)

For non-live situation, you can try MAutoVolume from MeldaProduction, dpMeter from TBProAudio, Youlean Loudness Meter from Youlean, Ozone from iZotope. Some of them are free, some of them cost a castle. REAPER itself also provides normalization when you right click a clip and choose “properties”, or when you render your project.

[StarBrilliant]

Originally Posted by mucknog

The gain seems to be able to raise above the hard limit setting.
Shouldn't it stop there?

Another small thing. I feel, that the gain setting should not reset on playback start.

Originally Posted by StarBrilliant

If you need one setting to limit gain, I can also add a gain limit setting.

Both issues fixed with today's update. Can you please test it?
But be careful: Turn down your headphone before you test. I can't guarantee I didn't introduce any new bugs.

[nvk]

Originally Posted by StarBrilliant

If it's a VST plugin, then the plugin can tell REAPER how much latency it will introduce, and let REAPER compensate for that latency, effectively implementing the look-ahead feature.
But I don't find any way for a JSFX plugin to tell REAPER its latency. Therefore, the look-ahead can't be compensated. (There might be such a way. But I can't find it in the official documentation.)

For non-live situation, you can try MAutoVolume from MeldaProduction, dpMeter from TBProAudio, Youlean Loudness Meter from Youlean, Ozone from iZotope. Some of them are free, some of them cost a castle. REAPER itself also provides normalization when you right click a clip and choose “properties”, or when you render your project.

You can use pdc_delay to add latency in Reaper jsfx: https://www.reaper.fm/sdk/js/vars.php (at the bottom of this page)

So I could be wrong, but I think with those plug-ins you can only measure loudness and compensate for the entire track (aside from MAutoVolume which is close to what I'm looking for except it doesn't make clear how it's actually measuring the volume). I'm looking for something that does what your plug-in does, and targets LUFS in 'real-time' but not necessarily live (so it can have as much latency as necessary)

[ashcat_lt]

Originally Posted by nvk

You can use pdc_delay to add latency in Reaper jsfx: https://www.reaper.fm/sdk/js/vars.php (at the bottom of this page)

Yes but of course that by itself doesn’t get you the lookahead behavior. It’ll just be a negative delay. In fact, it’s exactly how the JS Time Adjustment plugin does negative delay times: Tell Reaper you’re introducing latency, but then don’t.

In order to actually get lookahead, you need to implement a delay buffer where you store the current input sample then analyze it and apply the calculated gain change to some sample in the buffer’s past and use that for output.

[en5ca]

Could this be modified to respond to Short Term Loudness, instead of momentary?

[StarBrilliant]

Originally Posted by ashcat_lt

Yes but of course that by itself doesn’t get you the lookahead behavior. It’ll just be a negative delay. In fact, it’s exactly how the JS Time Adjustment plugin does negative delay times: Tell Reaper you’re introducing latency, but then don’t.

In order to actually get lookahead, you need to implement a delay buffer where you store the current input sample then analyze it and apply the calculated gain change to some sample in the buffer’s past and use that for output.

You are 100% correct. Now I got the idea. But it seems I need to do some large surgery to the code.
I guess I need to find a couple of days with no work to do to implement the lookahead buffer.

I would appreciate very much if anyone can collaborate with me -- The GitHub repo accepts Pull Requests if anyone wants to submit an improvement to my code.

[StarBrilliant]

Originally Posted by en5ca

Could this be modified to respond to Short Term Loudness, instead of momentary?

Yes, find this line in the code:

loudness_buffer_len = max(floor(srate * 0.4 + 0.5), 1);

(Was called buffer_length before 2022-10-02.) Change the number 0.4 to other length.

But the reason I didn't do that, is because I feel it better to have a shorter loudness measurement duration, but a very long decay time; rather than a longer loudness measurement duration.

You can do some experiment and decide what suits you better: Do you prefer a longer measurement duration, or a decay time of 10 seconds?

I can add more feature if a longer measurement duration really makes sense -- but currently, I vote for keeping the measurement duration but allow the user to change decay time.

[JamesX]

[QUOTE=StarBrilliant;2600639]Yes, find this line in the code:

buffer_length = max(floor(srate * 0.4 + 0.5), 1);

Change the number 0.4 to other length.


Would 0.4 = 400ms and therefore 1.0 = 1sec?

What might be the minimum and maximum usable values?

Thanks,

JX

[en5ca]

Originally Posted by StarBrilliant

Yes, find this line in the code:

buffer_length = max(floor(srate * 0.4 + 0.5), 1);

Change the number 0.4 to other length.

But the reason I didn't do that, is because I feel it better to have a shorter loudness measurement duration, but a very long decay time; rather than a longer loudness measurement duration.

You can do some experiment and decide what suits you better: Do you prefer a longer measurement duration, or a decay time of 10 seconds?

I can add more feature if a longer measurement duration really makes sense -- but currently, I vote for keeping the measurement duration but allow the user to change decay time.

Thanks! My main interest is situation, where stream is monitored by a somewhat skilled sound engineer, and level balancing between different sources wouldnt have to rely on Live Loudness Normalizer. Usually engineer 'calibrates' himself against most reliable source, eg stream local host voice, and balances everything against this while maintaining a fixed acoustic monitoring level. In this scenario Live Loudness Normalizer would sit just before final peak limiter, and keep level in check over longer period of time, where for example engineer ears get tired, and has gradually pushed the overall level a couple of dBs higher than it was initially during line check.

[StarBrilliant]

Originally Posted by en5ca

Thanks! My main interest is situation, where stream is monitored by a somewhat skilled sound engineer, and level balancing between different sources wouldnt have to rely on Live Loudness Normalizer. Usually engineer 'calibrates' himself against most reliable source, eg stream local host voice, and balances everything against this while maintaining a fixed acoustic monitoring level. In this scenario Live Loudness Normalizer would sit just before final peak limiter, and keep level in check over longer period of time, where for example engineer ears get tired, and has gradually pushed the overall level a couple of dBs higher than it was initially during line check.

Now I understand your scenario. Can you experiment with these two setups?
1. Modify the measurement length from 400ms to 10s through source code, keep decay time.
2. Modify the decay time to somewhere between 10s and 30s, keep measurement length.

My personal preference is the second setup. But if you feel the first setup sounds better, I can also implement a measurement length slider. (I just don't want to increase buffer length unless proven necessary.)

Also, since the “decay half-life” slider can be dragged at realtime, we can make use of it.
My favorite approach is to first drag left to around 1s–3s to calibrate to a reference level; then drag all the way to the right (for example, 30s) to lock at the level.
Then, let's say, if your broadcast drifts 4dB above your reference level, the plugin will spend 30 seconds to bring down 2dB, then another 30 seconds to bring down 1dB, then another 30 seconds to bring down 0.5dB, etc. The change in gain is so slow that the audience won't notice. (Human can't tell less than 1dB difference.)

But, if we increase the measurement duration, the moment to start adjusting will be later. The remaining time to reach the target will be slightly shorter, thus the change in gain will be slightly faster, which the audience might notice.

[StarBrilliant]

Originally Posted by JamesX

Would 0.4 = 400ms and therefore 1.0 = 1sec?

What might be the minimum and maximum usable values?

Thanks,

JX

Yes.
0.4 = 400ms.
1.0 = 1sec.

There is no limit.
But JSFX does have a limit on how much RAM a plugin can use. Therefore you can't increase to hours long.
I also recommend you don't go below 50ms, otherwise the loudness measurement isn't accurate.

[JamesX]

Originally Posted by StarBrilliant

Yes.
0.4 = 400ms.
1.0 = 1sec.

There is no limit.
But JSFX does have a limit on how much RAM a plugin can use. Therefore you can't increase to hours long.
I also recommend you don't go below 50ms, otherwise the loudness measurement isn't accurate.

Thank you StarBrilliant!

[StarBrilliant]

Originally Posted by nvk

You can use pdc_delay to add latency in Reaper jsfx: https://www.reaper.fm/sdk/js/vars.php (at the bottom of this page)

So I could be wrong, but I think with those plug-ins you can only measure loudness and compensate for the entire track (aside from MAutoVolume which is close to what I'm looking for except it doesn't make clear how it's actually measuring the volume). I'm looking for something that does what your plug-in does, and targets LUFS in 'real-time' but not necessarily live (so it can have as much latency as necessary)

Originally Posted by ashcat_lt

Yes but of course that by itself doesn’t get you the lookahead behavior. It’ll just be a negative delay. In fact, it’s exactly how the JS Time Adjustment plugin does negative delay times: Tell Reaper you’re introducing latency, but then don’t.

In order to actually get lookahead, you need to implement a delay buffer where you store the current input sample then analyze it and apply the calculated gain change to some sample in the buffer’s past and use that for output.

Look-ahead is implemented today!
Please test the latest version and give me feedback.

It is a simple look-ahead: Measure loudness with audio ahead, and apply the gain to audio behind. Therefore, setting it too long may cause the gain to react too late.
I recommend keeping it between 0.4s and decay half-life. My experiments show 3s a good choice for pop songs.

Again, I can't guarantee bug-free with beta version: Please take out your headphones when you hit the play button.

[en5ca]

Originally Posted by StarBrilliant

Now I understand your scenario. Can you experiment with these two setups?
1. Modify the measurement length from 400ms to 10s through source code, keep decay time.
2. Modify the decay time to somewhere between 10s and 30s, keep measurement length.

My personal preference is the second setup. But if you feel the first setup sounds better, I can also implement a measurement length slider. (I just don't want to increase buffer length unless proven necessary.)

Also, since the “decay half-life” slider can be dragged at realtime, we can make use of it.
My favorite approach is to first drag left to around 1s–3s to calibrate to a reference level; then drag all the way to the right (for example, 30s) to lock at the level.
Then, let's say, if your broadcast drifts 4dB above your reference level, the plugin will spend 30 seconds to bring down 2dB, then another 30 seconds to bring down 1dB, then another 30 seconds to bring down 0.5dB, etc. The change in gain is so slow that the audience won't notice. (Human can't tell less than 1dB difference.)

But, if we increase the measurement duration, the moment to start adjusting will be later. The remaining time to reach the target will be slightly shorter, thus the change in gain will be slightly faster, which the audience might notice.

After experimenting, ended up with results (dialogue/voice):

Integration time 3-4 seconds.
Relatively fast decay time, 1-3 seconds.
Output Limit 3 LU over target.
Rest are current defaults.

ReaLimit after Live Loudness Normalizer.

With longer integration time loud bursts 'pass through' Live Loudness Normalizer and hit the brickwall limiter. If this continues a bit longer, then LLN limit starts to grab in, and getting back to 'normal' levels happens more transparently. Also, I think, at least with voice, longer integration time and shorter decay seems to work nicer with inflection function. But this indeed relies on levels already being somewhat balanced and under control by human.

About human perception, that actually isn't my main 'concern'. Final goal is to get the live stream to get 'validated' for broadcaster online portal straight from the live stream, without me having to open stream recording in post-production, and process it manually afterwards to fit within the LRA recommendation for specific content for specific platform etc.

Awesome plugin, by the way. Thank you, very much!

[StarBrilliant]

Originally Posted by en5ca

After experimenting, ended up with results (dialogue/voice):

Integration time 3-4 seconds.
Relatively fast decay time, 1-3 seconds.
Output Limit 3 LU over target.
Rest are current defaults.

ReaLimit after Live Loudness Normalizer.

Thanks for your experiment results! They are valuable!

My original version of Output Limit was a bad quality one, so I only use it as a protective fuse. —But recently I improved the Output Limit feature and they should sound nicer. I'm impressed that you find some creative use of this feature.
However, I still can't guarantee the quality of Output Limit, so if you feel the Output Limit is causing audio quality problems, please let me know and we can reconsider ways to improve it.

I realized you want to keep the dynamic of loud bursts —which is different from my situation where I totally want to avoid them. This makes perfect sense with longer integration time.
I will need to find a lazy afternoon to add a new slider to control integration time.
But it won't be easy: I need to redesign the memory layout, which is a pain with JSFX's flat memory model. Also the user won't be able to change integration time dynamically because buffer resizing loses buffered data, similar to the look-ahead feature I just added today.

Before I find that perfect afternoon to write code, we will have to cope with the patch I wrote in post #20.

[mucknog]

Originally Posted by StarBrilliant

Both issues fixed with today's update. Can you please test it?
But be careful: Turn down your headphone before you test. I can't guarantee I didn't introduce any new bugs.

Sorry for the late response had a project which demanded all my attention.

All I have to say is - PERFECT !!!11one
That gain limiter is so essential for my usage.
Keep gain during silence <- YES
Reset Gain <- must have

Thank you so much - this is fucking awesome

[TonE]

Thanks for sharing.

[JonLinnarson]

Originally Posted by nvk

This is pretty amazing! Can see a lot of uses for this tool. I actually think it could be useful for non-live applications as well, like if you are trying to keep a music track at a consistent volume. Some kind of look-ahead setting would be great for that. And as others have mentioned, a way to limit the amount of gain applied to a range of some sort would be useful too.

This is ofc up to StarBrilliant to decide, but my five cents is that it could be wise to keep focus on the original intent for the plug-in, instead of trying to cram as much versatility into it and maybe having to make compromises to get everything to work nicely.

Having two different versions of the plug-in, one for live and one for non-live use, could maybe be an alternative solution? JS:Loudness Normalizer & JS:Live Loudness Normalizer... sounds like a great bundle

I'm not a dev though, so maybe it could be quite easy have all of the functionality for a live and non-live mode in the same plug-in?

Eitherway, I really like the plug-in so far. Thanks a lot StarBrilliant!