Old 10-11-2019, 01:23 AM   #1
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default STFT Effect Template

Based on this post (an energy-preserving stereo-to-mono sum), I thought I'd adapt it into a re-usable template for STFT-based effects.

I'd like this to be collaborative, so I'm open to questions, code-reviews, example effects. The code here is freely available to use/modify/distribute however you like, and if you suggest a modification to the template (not your own effect) it will be taken under the same conditions.

The STFT: Short-Time Fourier Transform

This is a name for the technique where we:
  • split the audio up into (overlapping) segments
  • separate those segments into individual frequencies (FFT)
  • mess about with those frequencies!
  • recombine back into a result (IFFT and then blend the results back together)
Here's a rough diagram of the flow.

Because we have to wait until we have a whole FFT block before we can do our processing, this adds latency. However, it's a very flexible technique which powers ReaFIR, some phase-shifters, Atlantis Reverb, etc.

Why a template?

While the STFT itself can be quite long, the actual interesting bit is the third step above, where we change the values.

Providing a template lets people just experiment with just this part, without having to write the rest of the STFT.

Writing the interesting parts

The key bits are at the top, surrounded by "////////////////////////", and there are two sections: the main block (called for every frequency, for every segment), and the setup block.

In the main block, the values for each frequency are complex numbers, so we have separate real/imaginary parts for both left and right. We loop through each FFT bin and process them. For example, the default implementation here is a low-pass, where every bin above (fft_size/10) is set to 0:

Code:
    //////////////////////// Main STFT block
    fft_bin > fft_size*0.1 ? (
      left_real = 0;
      left_imag = 0;
      right_real = 0;
      right_imag = 0;
    );
    ////////////////////////
The N-th FFT bin is equivalent to the frequency: srate*N/fft_size. Correspondingly, the frequency F is equivalent to the FFT bin: fft_size*F/srate

So, the example above implements a low-pass, where every frequency above srate/10 (e.g. 4800Hz for a 48K sample-rate) is set to 0.

The "setup block" is called when playback starts, or when the FFT size changes. By default, it does nothing, but if you have some other state which should be reset, this is the place to do it.

The code

There are some choices when designing an STFT (e.g. window function, overlap amount). The default window is Hann, and the default overlap is 6, set by "overlap_factor" (which I think for a Hann window can be any even number >= 4).

FFT size is controllable using a slider, in the range 256-32768 samples.

Here's a Stash link, and here's the code.

Code:
desc:STFT-Based Effect Template

slider1:fft_size_index=2<0,7,1{256,512,1024,2048,4096,8192,16384,32768>FFT size

@init

function process_stft_segment(fft_buffer, fft_size) local(fft_bin, left_real, left_imag, right_real, right_imag) (
  fft_bin = 0; // FFT bin number
  loop(fft_size/2,
    fft_bin2 = fft_bin ? (fft_size - fft_bin) : 0;

    // Unfold complex spectrum into two real spectra
    left_real = fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];
    left_imag = fft_buffer[2*fft_bin + 1] - fft_buffer[2*fft_bin2 + 1];
    right_real = fft_buffer[2*fft_bin + 1] + fft_buffer[2*fft_bin2 + 1];
    right_imag = -fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];

    //////////////////////// Main STFT block
    // The interesting bit - as a placeholder, let's implement a lowpass at samplerate/10
    fft_bin > fft_size*0.1 ? (
      left_real = 0;
      left_imag = 0;
      right_real = 0;
      right_imag = 0;
    );
    ////////////////////////

    // Re-fold back into complex spectrum
    fft_buffer[2*fft_bin] = (left_real - right_imag)*0.5;
    fft_buffer[2*fft_bin + 1] = (left_imag + right_real)*0.5;
    fft_buffer[2*fft_bin2] = (left_real + right_imag)*0.5;
    fft_buffer[2*fft_bin2 + 1] = (-left_imag + right_real)*0.5;

    fft_bin += 1;
  );
);

function setup_stft_state(fft_size, first_time) (
  //////////////////////// Setup block
  // This is called when playback starts, or when the FFT size is changed
  0;
  ////////////////////////
);

MAX_FFT_SIZE = 32768;

freemem = 0;
freemem = (fft_buffer = freemem) + MAX_FFT_SIZE*2;
freemem = (window_buffer = freemem) + MAX_FFT_SIZE;

buffer_length = srate;
buffer_index = 0;
freemem = (input_buffer = freemem) + buffer_length*2;
freemem = (output_buffer = freemem) + buffer_length*2;

function window(r) local(s, s2, gaussian_width, x) (
  // When squared, the Hann window adds up perfectly for overlap >= 4, so it's suitable for perfect reconstruction
  (0.5 - 0.5*cos(r*2*$pi))/sqrt(0.375);
);

fft_size = 0;

@block

overlap_factor = 6;
fft_size = 256<<fft_size_index;
fft_interval = fft_size/overlap_factor;
fft_scaling_factor = 1/overlap_factor/fft_size;

fft_size != prev_fft_size ? (
  setup_stft_state(fft_size, prev_fft_size == 0);
  prev_fft_size = fft_size;
  // Fill window buffer
  i = 0;
  loop(fft_size,
    r = (i + 0.5)/fft_size;
    window_buffer[i] = window(r);
    i += 1;
  );
);

pdc_delay = fft_size;
pdc_bot_ch = 0;
pdc_top_ch = 2;

@sample

input_buffer[buffer_index*2] = spl0;
input_buffer[buffer_index*2 + 1] = spl1;

fft_counter += 1;
fft_counter >= fft_interval ? (
  fft_counter = 0;

  // Copy input to buffer
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    i2 < 0 ? i2 += buffer_length;

    fft_buffer[2*i] = input_buffer[2*i2]*window_buffer[i];
    fft_buffer[2*i + 1] = input_buffer[2*i2 + 1]*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );

  // Process buffer
  fft(fft_buffer, fft_size);
  fft_permute(fft_buffer, fft_size);

  process_stft_segment(fft_buffer, fft_size);

  fft_ipermute(fft_buffer, fft_size);
  ifft(fft_buffer, fft_size);

  // Add to output
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    (i2 < 0) ? i2 += buffer_length;

    output_buffer[2*i2] += fft_buffer[2*i]*fft_scaling_factor*window_buffer[i];
    output_buffer[2*i2 + 1] += fft_buffer[2*i + 1]*fft_scaling_factor*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );
);

output_index = buffer_index - fft_size;
output_index < 0 ? output_index += buffer_length;
spl0 = output_buffer[output_index*2];
spl1 = output_buffer[output_index*2 + 1];
output_buffer[output_index*2] = 0; // clear the sample we just read
output_buffer[output_index*2 + 1] = 0;

buffer_index = (buffer_index + 1)%buffer_length;
I hope you make something fun.

Geraint

Last edited by geraintluff; 10-13-2019 at 04:45 AM.
geraintluff is offline   Reply With Quote
Old 10-11-2019, 01:27 AM   #2
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default

Here's an STFT-based phase-rotator. The only differences are the new slider, and the block in process_stft_segment():

Code:
desc:STFT-Based Phase Rotator

slider1:fft_size_index=2<0,7,1{256,512,1024,2048,4096,8192,16384,32768>FFT size
slider2:phase_rotation=0<-180,180>phase difference

@init

function process_stft_segment(fft_buffer, fft_size) local(fft_bin, left_real, left_imag, right_real, right_imag) (
  fft_bin = 0; // FFT bin number
  loop(fft_size/2,
    fft_bin2 = fft_bin ? (fft_size - fft_bin) : 0;

    // Unfold complex spectrum into two real spectra
    left_real = fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];
    left_imag = fft_buffer[2*fft_bin + 1] - fft_buffer[2*fft_bin2 + 1];
    right_real = fft_buffer[2*fft_bin + 1] + fft_buffer[2*fft_bin2 + 1];
    right_imag = -fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];

    //////////////////////// Main STFT block
    rotate_real = cos(phase_rotation*$pi/180/2); // Yes, this could be moved outside the loop
    rotate_imag = sin(phase_rotation*$pi/180/2);
    // rotate left phase
    tmp_real = left_real;
    tmp_imag = left_imag;
    left_real = tmp_real*rotate_real - tmp_imag*rotate_imag;
    left_imag = tmp_imag*rotate_real + tmp_real*rotate_imag;
    // rotate right phase the other way
    tmp_real = right_real;
    tmp_imag = right_imag;
    right_real = tmp_real*rotate_real + tmp_imag*rotate_imag;
    right_imag = tmp_imag*rotate_real - tmp_real*rotate_imag;
    ////////////////////////

    // Re-fold back into complex spectrum
    fft_buffer[2*fft_bin] = (left_real - right_imag)*0.5;
    fft_buffer[2*fft_bin + 1] = (left_imag + right_real)*0.5;
    fft_buffer[2*fft_bin2] = (left_real + right_imag)*0.5;
    fft_buffer[2*fft_bin2 + 1] = (-left_imag + right_real)*0.5;

    fft_bin += 1;
  );
);

function setup_stft_state(fft_size, first_time) (
  //////////////////////// Setup block
  // This is called when playback starts, or when the FFT size is changed
  0;
  ////////////////////////
);

MAX_FFT_SIZE = 32768;

freemem = 0;
freemem = (fft_buffer = freemem) + MAX_FFT_SIZE*2;
freemem = (window_buffer = freemem) + MAX_FFT_SIZE;

buffer_length = srate;
buffer_index = 0;
freemem = (input_buffer = freemem) + buffer_length*2;
freemem = (output_buffer = freemem) + buffer_length*2;

function window(r) local(s, s2, gaussian_width, x) (
  // When squared, the Hann window adds up perfectly for overlap >= 4, so it's suitable for perfect reconstruction
  (0.5 - 0.5*cos(r*2*$pi))/sqrt(0.375);
);

fft_size = 0;

@block

overlap_factor = 6;
fft_size = 256<<fft_size_index;
fft_interval = fft_size/overlap_factor;
fft_scaling_factor = 1/overlap_factor/fft_size;

fft_size != prev_fft_size ? (
  setup_stft_state(fft_size, prev_fft_size == 0);
  prev_fft_size = fft_size;
  // Fill window buffer
  i = 0;
  loop(fft_size,
    r = (i + 0.5)/fft_size;
    window_buffer[i] = window(r);
    i += 1;
  );
);

pdc_delay = fft_size;
pdc_bot_ch = 0;
pdc_top_ch = 2;

@sample

input_buffer[buffer_index*2] = spl0;
input_buffer[buffer_index*2 + 1] = spl1;

fft_counter += 1;
fft_counter >= fft_interval ? (
  fft_counter = 0;

  // Copy input to buffer
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    i2 < 0 ? i2 += buffer_length;

    fft_buffer[2*i] = input_buffer[2*i2]*window_buffer[i];
    fft_buffer[2*i + 1] = input_buffer[2*i2 + 1]*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );

  // Process buffer
  fft(fft_buffer, fft_size);
  fft_permute(fft_buffer, fft_size);

  process_stft_segment(fft_buffer, fft_size);

  fft_ipermute(fft_buffer, fft_size);
  ifft(fft_buffer, fft_size);

  // Add to output
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    (i2 < 0) ? i2 += buffer_length;

    output_buffer[2*i2] += fft_buffer[2*i]*fft_scaling_factor*window_buffer[i];
    output_buffer[2*i2 + 1] += fft_buffer[2*i + 1]*fft_scaling_factor*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );
);

output_index = buffer_index - fft_size;
output_index < 0 ? output_index += buffer_length;
spl0 = output_buffer[output_index*2];
spl1 = output_buffer[output_index*2 + 1];
output_buffer[output_index*2] = 0; // clear the sample we just read
output_buffer[output_index*2 + 1] = 0;

buffer_index = (buffer_index + 1)%buffer_length;

Last edited by geraintluff; 10-11-2019 at 01:50 PM.
geraintluff is offline   Reply With Quote
Old 10-11-2019, 01:47 AM   #3
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default

Here's an STFT-based reverb (which is roughly how Atlantis Reverb works).

The differences are:
  • We've reserved some memory for a "reverb_energy" array
  • The processing block
  • The setup block (resets the reverb energy to 0)
  • Sliders at the top

In the processing block, we
  • calculate our input energy
  • add the reverb energy to the left/right channels with random phase - (damping reduces the reverb at high frequencies)
  • add our input energy to the reverb energy
  • decay this reverb energy


This isn't perfect (we'll get slight pre-echoes, all frequencies decay at the same rate, etc.), but I think it demonstrates the principle.

Code:
desc:STFT-Based Reverb

slider1:fft_size_index=5<0,7,1{256,512,1024,2048,4096,8192,16384,32768>FFT size
slider2:decay_blocks=10<1,100>reverb length (blocks)
slider3:reverb_wet_amount=0.5<0,1>wet
slider4:damping_bin_ratio=0.1<0,1>damping freq

@init

function process_stft_segment(fft_buffer, fft_size) local(fft_bin, left_real, left_imag, right_real, right_imag) (
  fft_bin = 0; // FFT bin number
  loop(fft_size/2,
    fft_bin2 = fft_bin ? (fft_size - fft_bin) : 0;

    // Unfold complex spectrum into two real spectra
    left_real = fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];
    left_imag = fft_buffer[2*fft_bin + 1] - fft_buffer[2*fft_bin2 + 1];
    right_real = fft_buffer[2*fft_bin + 1] + fft_buffer[2*fft_bin2 + 1];
    right_imag = -fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];

    //////////////////////// Main STFT block
    left_energy = left_real*left_real + left_imag*left_imag;
    right_energy = right_real*right_real + right_imag*right*imag;
    stereo_energy = (left_energy + right_energy)*0.5;

    // Amplitude from energy    
    reverb_amp = reverb_wet_amount*sqrt(reverb_energy[fft_bin]);
    fft_bin/fft_size > damping_bin_ratio ? (
      // Reduce amplitude by 1/f
      reverb_amp *= damping_bin_ratio*fft_size/fft_bin;
    );
    // Add random-phase reverb energy to left and right
    random_phase = rand()*2*$pi;
    left_real += reverb_amp*cos(random_phase);
    left_imag += reverb_amp*sin(random_phase);
    random_phase = rand()*2*$pi;
    right_real += reverb_amp*cos(random_phase);
    right_imag += reverb_amp*sin(random_phase);
    
    // Add current energy to reverb
    reverb_energy[fft_bin] += stereo_energy/overlap_factor;
    // Decay reverb
    reverb_decay_factor = 1/decay_blocks;
    reverb_energy[fft_bin] *= 1 - reverb_decay_factor;
        
    ////////////////////////

    // Re-fold back into complex spectrum
    fft_buffer[2*fft_bin] = (left_real - right_imag)*0.5;
    fft_buffer[2*fft_bin + 1] = (left_imag + right_real)*0.5;
    fft_buffer[2*fft_bin2] = (left_real + right_imag)*0.5;
    fft_buffer[2*fft_bin2 + 1] = (-left_imag + right_real)*0.5;

    fft_bin += 1;
  );
);

function setup_stft_state(fft_size, first_time) local(i) (
  //////////////////////// Setup block
  i = 0;
  loop(fft_size, 
    reverb_energy[i] = 0;
    i += 1;
  );
  ////////////////////////
);

MAX_FFT_SIZE = 32768;

freemem = 0;
freemem = (fft_buffer = freemem) + MAX_FFT_SIZE*2;
freemem = (window_buffer = freemem) + MAX_FFT_SIZE;

// New addition!
freemem = (reverb_energy = freemem) + MAX_FFT_SIZE;

buffer_length = srate;
buffer_index = 0;
freemem = (input_buffer = freemem) + buffer_length*2;
freemem = (output_buffer = freemem) + buffer_length*2;

function window(r) local(s, s2, gaussian_width, x) (
  // When squared, the Hann window adds up perfectly for overlap >= 4, so it's suitable for perfect reconstruction
  (0.5 - 0.5*cos(r*2*$pi))/sqrt(0.375);
);

fft_size = 0;

@block

overlap_factor = 6;
fft_size = 256<<fft_size_index;
fft_interval = fft_size/overlap_factor;
fft_scaling_factor = 1/overlap_factor/fft_size;

fft_size != prev_fft_size ? (
  setup_stft_state(fft_size, prev_fft_size == 0);
  prev_fft_size = fft_size;
  // Fill window buffer
  i = 0;
  loop(fft_size,
    r = (i + 0.5)/fft_size;
    window_buffer[i] = window(r);
    i += 1;
  );
);

pdc_delay = fft_size;
pdc_bot_ch = 0;
pdc_top_ch = 2;

@sample

input_buffer[buffer_index*2] = spl0;
input_buffer[buffer_index*2 + 1] = spl1;

fft_counter += 1;
fft_counter >= fft_interval ? (
  fft_counter = 0;

  // Copy input to buffer
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    i2 < 0 ? i2 += buffer_length;

    fft_buffer[2*i] = input_buffer[2*i2]*window_buffer[i];
    fft_buffer[2*i + 1] = input_buffer[2*i2 + 1]*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );

  // Process buffer
  fft(fft_buffer, fft_size);
  fft_permute(fft_buffer, fft_size);

  process_stft_segment(fft_buffer, fft_size);

  fft_ipermute(fft_buffer, fft_size);
  ifft(fft_buffer, fft_size);

  // Add to output
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    (i2 < 0) ? i2 += buffer_length;

    output_buffer[2*i2] += fft_buffer[2*i]*fft_scaling_factor*window_buffer[i];
    output_buffer[2*i2 + 1] += fft_buffer[2*i + 1]*fft_scaling_factor*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );
);

output_index = buffer_index - fft_size;
output_index < 0 ? output_index += buffer_length;
spl0 = output_buffer[output_index*2];
spl1 = output_buffer[output_index*2 + 1];
output_buffer[output_index*2] = 0; // clear the sample we just read
output_buffer[output_index*2 + 1] = 0;

buffer_index = (buffer_index + 1)%buffer_length;
EDIT - bugfix missing sqrt()

Last edited by geraintluff; 10-11-2019 at 01:49 PM.
geraintluff is offline   Reply With Quote
Old 10-11-2019, 03:09 AM   #4
IXix
Human being with feelings
 
Join Date: Jan 2007
Location: mcr:uk
Posts: 3,387
Default

Now that is pretty cool.
IXix is offline   Reply With Quote
Old 10-11-2019, 01:48 PM   #5
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default

Here's a simplistic STFT-based noise-reduction plugin. It's pretty close to the "subtraction" mode of ReaFIR.

The differences from the template are:
  • We've reserved some memory for "left noise" and "right noise" arrays
  • The processing block
  • The setup block (resets the noise energy to 0, but not when we first load)
  • Sliders at the top
  • @serialize at the bottom to store our noise arrays

In the processing block, we:
  • calculate our input energy
  • in "learning" mode, we use noise_left/noise_right to track the maximum noise level we've seen. In "reset", we set this back to 0
  • we subtract the noise energy from our input energy, to estimate the true signal energy
  • we multiply the inputs so their energy-level matches our "true signal energy" estimate

Code:
desc:STFT-Based Noise Reduction

slider1:fft_size_index=5<0,7,1{256,512,1024,2048,4096,8192,16384,32768>FFT size
slider2:mode=0<0,2,1{clear,learning,reduction}>mode
slider3:reduction_factor=1<0,2>aggressiveness

@init

function process_stft_segment(fft_buffer, fft_size) local(fft_bin, left_real, left_imag, right_real, right_imag) (
  fft_bin = 0; // FFT bin number
  loop(fft_size/2,
    fft_bin2 = fft_bin ? (fft_size - fft_bin) : 0;

    // Unfold complex spectrum into two real spectra
    left_real = fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];
    left_imag = fft_buffer[2*fft_bin + 1] - fft_buffer[2*fft_bin2 + 1];
    right_real = fft_buffer[2*fft_bin + 1] + fft_buffer[2*fft_bin2 + 1];
    right_imag = -fft_buffer[2*fft_bin] + fft_buffer[2*fft_bin2];

    //////////////////////// Main STFT block
    left_energy = left_real*left_real + left_imag*left_imag;
    right_energy = right_real*right_real + right_imag*right*imag;
    mode == 0 ? (
      // Clear the noise energy
      noise_left[fft_bin] = 0;
      noise_right[fft_bin] = 0;
    ) : mode == 1 ? (
      // We're learning - keep track of the maximum level we've seen
      noise_left[fft_bin] = max(noise_left[fft_bin], left_energy);
      noise_right[fft_bin] = max(noise_right[fft_bin], right_energy);
    );
    // Guess what the energy _would_ be, without the noise - but not less than 0
    left_energy_subtracted = max(0, left_energy - reduction_factor*noise_left[fft_bin]);
    right_energy_subtracted = max(0, right_energy - reduction_factor*noise_right[fft_bin]);
  
    // Multiply by the amplitude ratio between the two
    left_real *= sqrt(left_energy_subtracted/left_energy);
    left_imag *= sqrt(left_energy_subtracted/left_energy);
    right_real *= sqrt(right_energy_subtracted/right_energy);
    right_imag *= sqrt(right_energy_subtracted/right_energy);
    ////////////////////////

    // Re-fold back into complex spectrum
    fft_buffer[2*fft_bin] = (left_real - right_imag)*0.5;
    fft_buffer[2*fft_bin + 1] = (left_imag + right_real)*0.5;
    fft_buffer[2*fft_bin2] = (left_real + right_imag)*0.5;
    fft_buffer[2*fft_bin2 + 1] = (-left_imag + right_real)*0.5;

    fft_bin += 1;
  );
);

function setup_stft_state(fft_size, first_time) local(i) (
debug.setup_count += 1;
  // We don't want to clear the noise buffer when we first load :)
  !first_time ? (
  debug.setup_count_executed += 1;
    i = 0;
    loop(fft_size, 
      noise_left[i] = 0;
      noise_right[i] = 0;
      i += 1;
    );
  );
);

MAX_FFT_SIZE = 32768;

freemem = 0;
freemem = (fft_buffer = freemem) + MAX_FFT_SIZE*2;
freemem = (window_buffer = freemem) + MAX_FFT_SIZE;

// New addition!
freemem = (noise_left = freemem) + MAX_FFT_SIZE;
freemem = (noise_right = freemem) + MAX_FFT_SIZE;

buffer_length = srate;
buffer_index = 0;
freemem = (input_buffer = freemem) + buffer_length*2;
freemem = (output_buffer = freemem) + buffer_length*2;

function window(r) local(s, s2, gaussian_width, x) (
  // When squared, the Hann window adds up perfectly for overlap >= 4, so it's suitable for perfect reconstruction
  (0.5 - 0.5*cos(r*2*$pi))/sqrt(0.375);
);

fft_size = 0;

@block

overlap_factor = 6;
fft_size = 256<<fft_size_index;
fft_interval = fft_size/overlap_factor;
fft_scaling_factor = 1/overlap_factor/fft_size;

fft_size != prev_fft_size ? (
  setup_stft_state(fft_size, prev_fft_size == 0);
  prev_fft_size = fft_size;
  // Fill window buffer
  i = 0;
  loop(fft_size,
    r = (i + 0.5)/fft_size;
    window_buffer[i] = window(r);
    i += 1;
  );
);

pdc_delay = fft_size;
pdc_bot_ch = 0;
pdc_top_ch = 2;

@sample

input_buffer[buffer_index*2] = spl0;
input_buffer[buffer_index*2 + 1] = spl1;

fft_counter += 1;
fft_counter >= fft_interval ? (
  fft_counter = 0;

  // Copy input to buffer
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    i2 < 0 ? i2 += buffer_length;

    fft_buffer[2*i] = input_buffer[2*i2]*window_buffer[i];
    fft_buffer[2*i + 1] = input_buffer[2*i2 + 1]*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );

  // Process buffer
  fft(fft_buffer, fft_size);
  fft_permute(fft_buffer, fft_size);

  process_stft_segment(fft_buffer, fft_size);

  fft_ipermute(fft_buffer, fft_size);
  ifft(fft_buffer, fft_size);

  // Add to output
  bi = buffer_index - fft_size + 1;
  i = 0;
  loop(fft_size,
    i2 = bi + i;
    (i2 < 0) ? i2 += buffer_length;

    output_buffer[2*i2] += fft_buffer[2*i]*fft_scaling_factor*window_buffer[i];
    output_buffer[2*i2 + 1] += fft_buffer[2*i + 1]*fft_scaling_factor*window_buffer[i];

    i += 1;
    bi >= buffer_length ? i = 0;
  );
);

output_index = buffer_index - fft_size;
output_index < 0 ? output_index += buffer_length;
spl0 = output_buffer[output_index*2];
spl1 = output_buffer[output_index*2 + 1];
output_buffer[output_index*2] = 0; // clear the sample we just read
output_buffer[output_index*2 + 1] = 0;

buffer_index = (buffer_index + 1)%buffer_length;

@serialize

// Always make sure presets are versioned
plugin_version = 0;
file_var(0, plugin_version);

// Store the noise profile
file_mem(0, noise_left, fft_size);
file_mem(0, noise_right, fft_size);
geraintluff is offline   Reply With Quote
Old 10-13-2019, 03:18 AM   #6
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,005
Default

Cool, thanks for sharing!
__________________
Martinic Kee Bass - Scanner Vibrato - Elka Panther - Tale's JSFX Pack
Tale is online now   Reply With Quote
Old 10-13-2019, 03:52 AM   #7
sai'ke
Human being with feelings
 
sai'ke's Avatar
 
Join Date: Aug 2009
Location: NL
Posts: 672
Default

Thanks man!
__________________
[Tracker Plugin: Thread|Github|Reapack] | [Routing Plugin: Thread|Reapack] | [Filther: Thread|Github|Reapack] | [More JSFX: Thread|Reapack]
sai'ke is offline   Reply With Quote
Old 11-11-2019, 05:45 PM   #8
beingmf
Human being with feelings
 
beingmf's Avatar
 
Join Date: Jul 2007
Location: Jazz City
Posts: 3,948
Default

Killer plugins, thanks a lot, Geraint!

I was quite floored by the reverb already, but what the hell is that NR? It's close to perfect (admittedly I just tried it on a guitar amp track: no obvious artifacts, noise gone completely :-o).

Bravo!!!
__________________
Windows 7x64 SP2 | NVidia Quadro2000, 10.18.13.6191 | Marian Seraph AD2, 4.0.7.1
Experience the Blunzeworscht!
beingmf is online now   Reply With Quote
Old 11-11-2019, 05:52 PM   #9
mespotine
Human being with feelings
 
mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig, Germany
Posts: 1,689
Default

What does energy-preserving mean in your context?
Less energy consumption for electronic devices/batteries or something else?
__________________
Ultraschall-API - a Lua-functions-library4Reaper: https://forum.cockos.com/showthread....98#post2067798
Reaper Internals - Developerdocs4Reaper: https://forum.cockos.com/showthread.php?t=207635
mespotine is offline   Reply With Quote
Old 11-12-2019, 05:08 AM   #10
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default

Quote:
Originally Posted by mespotine View Post
What does energy-preserving mean in your context?
Less energy consumption for electronic devices/batteries or something else?
The original request (here) was from Colox, who prefers to EQ in mono, but a plain mono sum messed with the balance of the mix.

So, I wrote something which did a plain mono sum, but then hacked the spectrum about (dynamic EQ, effectively) so that the energy matched the original stereo signal. It's not perfect, but it works remarkably OK.

Quote:
Originally Posted by beingmf View Post
I was quite floored by the reverb already, but what the hell is that NR? It's close to perfect
It's pretty basic, and similar to the "subtraction" mode of the built-in ReaFIR - which you should try out if you haven't already because the UI is prettier.

All these examples are intended as illustrations to show that the STFT code is common between all of them, and that the "interesting" parts of each effect are fairly small.

Geraint

Last edited by geraintluff; 11-12-2019 at 05:13 AM.
geraintluff is offline   Reply With Quote
Old 11-12-2019, 11:32 AM   #11
beingmf
Human being with feelings
 
beingmf's Avatar
 
Join Date: Jul 2007
Location: Jazz City
Posts: 3,948
Default

Quote:
Originally Posted by geraintluff View Post
It's pretty basic, and similar to the "subtraction" mode of the built-in ReaFIR - which you should try out if you haven't already because the UI is prettier.

All these examples are intended as illustrations to show that the STFT code is common between all of them, and that the "interesting" parts of each effect are fairly small.

Geraint
You sure? To be honest, I haven't re-tested as my results with ReaFir on this very track weren't too overwhelming. Will do though.
__________________
Windows 7x64 SP2 | NVidia Quadro2000, 10.18.13.6191 | Marian Seraph AD2, 4.0.7.1
Experience the Blunzeworscht!
beingmf is online now   Reply With Quote
Old 11-12-2019, 02:39 PM   #12
geraintluff
Human being with feelings
 
geraintluff's Avatar
 
Join Date: Nov 2009
Location: mostly inside my own head
Posts: 323
Default

Quote:
Originally Posted by beingmf View Post
You sure?
I don't know what goes on inside ReaFIR, so not 100% - but my one just does straightforward spectral subtraction, which is what ReaFIR also sounds like to me.

It's possible that ReaFIR is using less overlap between FFT blocks (e.g. 50%, instead of 83%), which would take less CPU but give a slightly less smooth result. ¯\_(ツ)_/¯

Geraint
geraintluff is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 02:15 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.