COCKOS
CONFEDERATED FORUMS
Cockos : REAPER : NINJAM : Forums
Forum Home : Register : FAQ : Members List : Search :

Go Back   Cockos Incorporated Forums > Other Software Discussion > WDL users forum

Reply
 
Thread Tools Display Modes
Old 06-20-2015, 04:37 PM   #1
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default Audio processing in processDoubleReplacing works relative to host playhead?

I have a plug-in that simply reads audio into a buffer and currently also does a Hamming window on it, then outputs it.

What I noticed in my current implementation is that my plug-in seems to only work when the input file starts are aligned in the host exactly at the sizes of the buffers. In this case 1024 samples (or 2048 or 3072 and so on).

See picture in attachments. It's like my plug-in works _relative_ to the host sample position and not based on how many samples of input audio it got. The 3rd track is simply the first one moved forward and the 4th is the plug-in applied to that one. The 4th track cuts track 2 exactly at the amount of samples that's the difference between tracks 1 and 3.

Any ideas? I'm pretty much just reading nFrames in.
I was thinking of whether I need to check for silence or something and not read anything to the buffers if the input is just silence.

Is nFrames == 0 when there's no audio?
Are the in** and out** buffers empty or what if there's no audio input?
Attached Images
File Type: jpg Capture (5).jpg (25.5 KB, 188 views)

Last edited by mviljamaa; 06-21-2015 at 10:02 AM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 05:09 AM   #2
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
I'm pretty much just reading nFrames in.
I was thinking of whether I need to check for silence or something and not read anything to the buffers if the input is just silence.

Is nFrames == 0 when there's no audio?
Are the in** and out** buffers empty or what if there's no audio input?
No. When there is "no audio" your plug-in gets fed zeros in its input buffers. If there really is nothing for your plug-in to do (e.g. when it is bypassed), then your ProcessDoubleReplacing() is simply not called at all.

Quote:
Originally Posted by mviljamaa View Post
Any ideas?
Well, it is hard to say without diving really deep into your code, but I think what is missing is that your internal input and output buffers should be asynchronous FIFO queues, which I don't think they are right now. Consider this typical scenario, where the host happens to feed you blocks of 63 samples, but you want to process blocks of 64 samples:

1. The host sends 63 samples, so add them to the input queue.
2. You need 64 samples, so you can't process them yet.
3. The output queue is still empty, so output 63 initial zeros.
4. The host sends 63 samples, so add them to the input queue.
5. Process 64 samples (63 from step 1 + 1 from step 4), remove these from the input queue, and add 64 processed samples to the output queue.
6. The input queue now holds 63+63-64=62 samples, which is too little to process another block (if it would be enough, then you would have to repeat step 5).
7. There are more than 63 samples in the output queue, so thee is no need for initial zeros anymore.
8. Output 63 samples, and remove these from the output queue.

Notice how after step 8 there will be 1 sample left in the output queue, which we will output the next ProcessDoubleReplacing() call. I don't see how your current code would handle such leftovers.

I guess I could hack together an example that does all this if you think that would help...

Last edited by Tale; 06-25-2015 at 04:31 PM. Reason: Fixed typos
Tale is offline   Reply With Quote
Old 06-21-2015, 06:39 AM   #3
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
No. When there is "no audio" your plug-in either gets feeded zeros in its input buffers. If there really is nothing for your plug-in to do (e.g. when it is bypassed), then your ProcessDoubleReplacing() is simply not called at all.


Well, it is hard to say without diving really deep into your code, but I think what is missing is that your internal input and output buffers should be asynchronous FIFO queues, which I don't think they are right now. Consider this typical scenario, where the host happens to feed you blocks of 63 samples, but you want to process blocks of 64 samples:

1. The host sends 63 samples, so add them to the input queue.
2. You need 64 samples, so you can't process them yet.
3. The output queue is still empty, so output 63 initial zeros.
4. The host sends 63 samples, so add them to the input queue.
5. Process 64 samples (63 from step 1 + 1 from step 4), remove these from the input queue, and add 64 processed samples to the output queue.
6. The input queue now holds 63+63-64=62 samples, which is too little to process another block (if it would be enough, then you would have to repeat step 5).
7. There are more than 63 samples in the output queue, so thee is no need for initial zeros anymore.
8. Output 63 samples, and remove these from the output queue.

Notice how after step 8 there will be 1 sample left in the output queue, which we will output the next ProcessDoubleReplacing() call. I don't see how your current code would handle such leftovers.

I guess I could hack together an example that does all this if you think that would help...
That's what my code is supposed to do. May have to rewrite though, if it's really the code that's wrong (I've tested it on arrays of 1.0s though and it seemed to work there exactly as needed). Could simplify it though.

Is the nFrames == 0 checking trusthworthy? I assume that it should be zero, when the playhead isn't passing over that sinewave file.

Last edited by mviljamaa; 06-21-2015 at 10:03 AM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 07:46 AM   #4
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 7,713
Default

Quote:
Originally Posted by mviljamaa View Post
(I was reasoning that it was because the nFrames aligned exactly as the buffer size divided by some even number, so that there's never buffer overflow).
Never assume anything like that. Depending on the operating system, audio hardware and the host application, the number can be pretty much anything. It's not guaranteed to be an even number or a power-of-2 number. It may also change between calls to the plugin. You need to be prepared to handle all that.
__________________
For info on SWS Reaper extension plugin (including Xenakios' previous extension/actions) :
http://www.sws-extension.org/
https://github.com/Jeff0S/sws
--
Xenakios blog (about HourGlass, Paul(X)Stretch and λ) :
http://xenakios.wordpress.com/
Xenakios is offline   Reply With Quote
Old 06-21-2015, 08:01 AM   #5
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

I'm sorry, but I just don't know how to better explain this... Do you know what a FIFO queue is? If not, then look it up on Wikipedia or something, and then please re-read my explanation (here as well as in the other topic).

If you do know, then I would suggest implementing them instead of using simple buffers, e.g.:

Quote:
Originally Posted by mviljamaa View Post
Here, first output the remaining samples from the outputBuffer (before we can replace it).
You can't do that, because your output queue could hold more than nFrames samples. And you should not replace it, but rather add samples to it, which is why you need a queue.

Quote:
Originally Posted by mviljamaa View Post
Is the nFrames == 0 checking trusthworthy? I assume that it should be zero, when the playhead isn't passing over that sinewave file.
This is not relevant, but yeah, I do think your code should be able to handle nFrames == 0. However, there are probably not many hosts that will actually call your ProcessDoubleReplacing() with nFrames == 0, because then they could just as well not call it at all.

Last edited by Tale; 06-25-2015 at 04:33 PM. Reason: Fixed typos
Tale is offline   Reply With Quote
Old 06-21-2015, 10:02 AM   #6
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Ok, here's a significantly simplified code that does the same as the original one (lol?) and I still get the same problem.
Now it should be pretty obvious to spot, where the error is?

http://pasteguru.com/14516

Could it be related to the in** and out** pointers? I'm assuming that they always start at 0 (i.e. they're of the size [channel][nFrames]) when the processDoubleReplacing is called, but might it be that they don't, i.e. they're accumulated between processDoubleReplacing calls? So at some processDoubleCall the incoming nFrames may start from in[channel][s], s > 0, rather than in[channel][0].

Last edited by mviljamaa; 06-21-2015 at 10:22 AM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 10:45 AM   #7
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
Ok, here's a significantly simplified code that does the same as the original one (lol?) and I still get the same problem.
Now it should be pretty obvious to spot, where the error is?

http://pasteguru.com/14516
Sure. Implement FIFO queues (i.e. a buffer with head and tail) instead of plain buffers...

Quote:
Originally Posted by mviljamaa View Post
Could it be related to the in** and out** pointers? I'm assuming that they always start at 0 (i.e. they're of the size [channel][nFrames]) when the processDoubleReplacing is called, but might it be that they don't, i.e. they're accumulated between processDoubleReplacing calls? So at some processDoubleCall the incoming nFrames may start from in[channel][s], s > 0, rather than in[channel][0].
You assume correctly, they always start at 0.
Tale is offline   Reply With Quote
Old 06-21-2015, 11:42 AM   #8
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
Sure. Implement FIFO queues (i.e. a buffer with head and tail) instead of plain buffers...
So why is this? I thought my buffers were operating in circular fashion (like ring buffers) and should work? I can't see where the problem is.

I think my code does exactly this, although it doesn't add/remove, but rather rewrites:


1. The host sends 63 samples, so add them to the input queue.
2. You need 64 samples, so you can't process them yet.
3. The output queue is still empty, so output 63 initial zeros.
4. The host sends 63 samples, so add them to the input queue.
5. Process 64 samples (63 from step 1 + 1 from step 4), remove these from the input queue, and add 64 processed samples to the output queue.
6. The input queue now holds 63+63-64=62 samples, which is too little to process another block (if it would be enough, then you would have to repeat step 5).
7. There are more than 63 samples in the output queue, so thee is no need for initial zeros anymore.
8. Output 63 samples, and remove these from the output queue.

It doesn't?

Last edited by mviljamaa; 06-21-2015 at 12:04 PM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 12:40 PM   #9
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Well, I don't think your code does that, because a FIFO (first in first out) queue has a head (where you first started adding samples) and a tail (where you will add the next block of samples). If you "remove" samples from a FIFO queue, then you will have to either a) move your head pointer, or b) actually move elements within the queue. Your code seems to do neither.

Quote:
Originally Posted by mviljamaa View Post
1. The host sends 63 samples, so add them to the input queue.
...
4. The host sends 63 samples, so add them to the input queue.
Here I mean add samples to the tail (end) of the queue.

Quote:
Originally Posted by mviljamaa View Post
5. Process 64 samples (63 from step 1 + 1 from step 4), remove these from the input queue
...
8. Output 63 samples, and remove these from the output queue.
Here i mean remove a number of samples (but not all!) from the head (beginning) of the queue. This would require memmove() or similar to move elements within the queue (although a smart implementation might just update the queue head pointer, like with a circular buffer).

The reason you should use a queue, and you can't just use a normal circular buffer, is that the host might send you way more samples than your internal buffer can fit. Also, by the time that you are ready to output a large number of samples, the host might send you a very small number of samples, which means you can only output a small number of samples (and so you can't output all samples in the queue). Hence the input and output "buffers" should be totaly asynchronous (i.e. either can grow or shrink independently of the other, and of the also variable host buffer size).

Anyway, I would suggest replacing your internal input and output buffers with some sort of queue, e.g. WDL_Queue (#include "WDL/queue.h"), or perhaps std::queue. If you have trouble with the implementation, then I guess I could code you an example.

Last edited by Tale; 06-25-2015 at 04:35 PM. Reason: Fixed typos, meh.
Tale is offline   Reply With Quote
Old 06-21-2015, 02:38 PM   #10
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Ok, thanks. I'm switching to queues, using std::queue now and got them to work. The problem now seems to be that using queues Reaper is displaying 6.2% CPU use for my plug-in, using merely one channel. Any idea if this is "normal"? It's like 6x the amount I've seen for other plug-ins.

My queues are of type std::vector<std::queue<double*>> where the outer vector holds the channels and the queue the audio for any particular channel.

Last edited by mviljamaa; 06-21-2015 at 02:45 PM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 02:52 PM   #11
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 7,713
Default

Quote:
Originally Posted by mviljamaa View Post

My queues are of type std::vector<std::queue<double*>> where the outer vector holds the channels and the queue the audio for any particular channel.
Based on my own experiments, std::queue isn't a good choice for audio DSP work, it indeed is relatively slow for shuffling around data like doubles. I've myself used the WDL_FastQueue. It however is relatively painful to use because it works with bytes as the data elements. So you have to remember to deal with that...

However, I wonder...Why is the innermost type in your data structure a double*? Plain direct double should be enough! How are you inserting the double*'s into the queue? Surely not by doing something like "new double"? Because that would be crazy slow(*) to do! And not recommended to be done inside the audio processing threads, anyway.

(*) At least in the worst case scenarios.

Also remember to test your plugin's CPU load characteristics only with optimized release builds. Debug builds are never going to be fast and they are not intended to be that. They are meant to debug problems in your code.
__________________
For info on SWS Reaper extension plugin (including Xenakios' previous extension/actions) :
http://www.sws-extension.org/
https://github.com/Jeff0S/sws
--
Xenakios blog (about HourGlass, Paul(X)Stretch and λ) :
http://xenakios.wordpress.com/
Xenakios is offline   Reply With Quote
Old 06-21-2015, 03:05 PM   #12
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

The reason for using *double was that the in** and out** pointers and advancing them would work with that otherwise what I got was errors about trying to cast double to double*. I'm reading from double * in1 = inputs[0] and writing to double * out1 = outputs[0].

Nvm, I'll switch to WDL_FastQueue then. I get what WDL_FastQueue::Add() does but what about the others like WDL_FastQueue::Advance()? How to do basic usage?

Last edited by mviljamaa; 06-21-2015 at 03:13 PM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 03:14 PM   #13
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 7,713
Default

Quote:
Originally Posted by mviljamaa View Post
The reason for using *double was that the in** and out** pointers and advancing them would work with that otherwise what I got was errors about trying to cast double to double*.

Nvm, I'll switch to WDL_FastQueue then. I get what WDL_FastQueue::Add() does but what about the others like WDL_FastQueue::Advance()?
Hmm, well without seeing your code that put the double*'s into the std::queue I can't determine if what you were attempting was correct to begin with. It might have seemed to work just by luck.

The WDL_FastQueue works so that you put stuff into it with Add. With Available you can check how much stuff the queue has. When you have enough, you can copy elements to a destination buffer with GetToBuf. Then you need to use Advance to advance the queue the same amount you copied into the destination. (GetToBuf by itself won't advance the queue.)
__________________
For info on SWS Reaper extension plugin (including Xenakios' previous extension/actions) :
http://www.sws-extension.org/
https://github.com/Jeff0S/sws
--
Xenakios blog (about HourGlass, Paul(X)Stretch and λ) :
http://xenakios.wordpress.com/
Xenakios is offline   Reply With Quote
Old 06-21-2015, 03:59 PM   #14
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Xenakios View Post
Hmm, well without seeing your code that put the double*'s into the std::queue I can't determine if what you were attempting was correct to begin with. It might have seemed to work just by luck.

The WDL_FastQueue works so that you put stuff into it with Add. With Available you can check how much stuff the queue has. When you have enough, you can copy elements to a destination buffer with GetToBuf. Then you need to use Advance to advance the queue the same amount you copied into the destination. (GetToBuf by itself won't advance the queue.)
So what about SetFromBuf()?

Last edited by mviljamaa; 06-21-2015 at 04:18 PM.
mviljamaa is offline   Reply With Quote
Old 06-21-2015, 04:43 PM   #15
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

So after switching to queues the original problem still persists with the following code, which produces exactly the same that the original code in post #1:

Code:
outputBuffer has been initialized with FFT_BUFFER_SIZE of 0.0s.


void IPlugSideChain::ProcessDoubleReplacing(double** inputs, double** outputs, int nFrames) {
// Mutex is already locked for us.

        double* in1 = inputs[0];
        double* out1 = outputs[0];


        // nFrames to processingBuffer
	processingBuffer.Add(reinterpret_cast<char*>(in1),nFrames*sizeof(double));

        // output nFrames from outputBuffer
	outputBuffer.GetToBuf(0, reinterpret_cast<char*>(out1), nFrames*sizeof(double));
	outputBuffer.Advance(nFrames*sizeof(double));

	if (processingBuffer.Available() > sizeof(double)*FFT_BUFFER_SIZE) {

                // Read out FFT sized chunk from processingBuffer
		double data[FFT_BUFFER_SIZE];
		processingBuffer.GetToBuf(0, reinterpret_cast<char*>(data), FFT_BUFFER_SIZE*sizeof(double));
                processingBuffer.Advance(FFT_BUFFER_SIZE*sizeof(double));

		hamming(data, FFT_BUFFER_SIZE);

                // Add processed data to outputBuffer
	        outputBuffer.Add(reinterpret_cast<char*>(data), FFT_BUFFER_SIZE*sizeof(double));
		
	}
}
I don't get what I'm doing wrong.

Also reinterpret_cast seems not necessary? Why?

Last edited by mviljamaa; 06-21-2015 at 09:30 PM.
mviljamaa is offline   Reply With Quote
Old 06-22-2015, 01:33 AM   #16
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Well, how about this? Not tested, but I think it should work.

Code:
void IPlugSideChain::ProcessDoubleReplacing(double** inputs, double** outputs, int nFrames) {
// Mutex is already locked for us.

        double* in1 = inputs[0];
        double* out1 = outputs[0];


        // nFrames to processingBuffer
	processingBuffer.Add(in1,nFrames*sizeof(double));

	while (processingBuffer.Available() >= sizeof(double)*FFT_BUFFER_SIZE) {

                // Read out FFT sized chunk from processingBuffer
		double data[FFT_BUFFER_SIZE];
		processingBuffer.GetToBuf(0, data, FFT_BUFFER_SIZE*sizeof(double));
                processingBuffer.Advance(FFT_BUFFER_SIZE*sizeof(double));

		hamming(data, FFT_BUFFER_SIZE);

                // Add processed data to outputBuffer
	        outputBuffer.Add(data, FFT_BUFFER_SIZE*sizeof(double));
		
	}
	// Not necessary if you have pre-added zeros to outputBuffer:
	// if (outputBuffer.Available() < nFrames*sizeof(double)) {
		// int n = nFrames*sizeof(double) - outputBuffer.Available();
		// memset(out1, 0, n);
		// n /= sizeof(double);
		// nFrames -= n;
		// out1 += n;
	// }
        // output nFrames from outputBuffer
	outputBuffer.GetToBuf(0, out1, nFrames*sizeof(double));
	outputBuffer.Advance(nFrames*sizeof(double));
}
Quote:
Originally Posted by mviljamaa View Post
Also reinterpret_cast seems not necessary? Why?
I believe any pointer type can be automagically casted to void*. Note that reinterpret_cast is required when casting from void* to another pointer type.

Last edited by Tale; 06-22-2015 at 09:39 AM. Reason: Corrected faulty output zeros code
Tale is offline   Reply With Quote
Old 06-22-2015, 11:24 AM   #17
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

The above code produces the same as the one I posted. So the problem persists. Seems like we're having the same problem of thinking about the buffering?

Last edited by mviljamaa; 06-22-2015 at 11:29 AM.
mviljamaa is offline   Reply With Quote
Old 06-22-2015, 01:36 PM   #18
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

I'm sorry, I don't have time right now to check for myself, but I would temporarily disable the hamming() call and/or change the value of FFT_BUFFER_SIZE, just to see what happens. E.g. with the hamming() call disabled the output should exactly match the input, except it should be delayed by FFT_BUFFER_SIZE. If not, then there is probably something wrong with the rebuffering code. Changing the FFT_BUFFER_SIZE might offer a clue to what is wrong.
Tale is offline   Reply With Quote
Old 06-22-2015, 01:39 PM   #19
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
I'm sorry, I don't have time right now to check for myself, but I would temporarily disable the hamming() call and/or change the value of FFT_BUFFER_SIZE, just to see what happens. E.g. with the hamming() call disabled the output should exactly match the input, except it should be delayed by FFT_BUFFER_SIZE. If not, then there is probably something wrong with the rebuffering code. Changing the FFT_BUFFER_SIZE might offer a clue to what is wrong.
The output does match exactly when the hamming is disabled, but the hamming should be correct (it only works through the length of the array that's passed to it, unless it would get a wrong address to start from?), so rather I believe that applying it simply displays an error in the buffer pointers. I'll check the buffer size.

Might have to do a debugger run on arrays of 1.0s again though with these current implementations.
mviljamaa is offline   Reply With Quote
Old 06-22-2015, 01:53 PM   #20
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
The output does match exactly when the hamming is disabled, but the hamming should be correct (it only works through the length of the array that's passed to it, unless it would get a wrong address to start from?), so rather I believe that applying it simply displays an error in the buffer pointers. I'll check the buffer size.
Maybe instead of applying the Hamming window you could simply set the first sample of data[] to zero. That way you the output should still match the input, but then you can see where each buffer starts.

Last edited by Tale; 06-25-2015 at 04:37 PM. Reason: Fixed typos, !!@##!
Tale is offline   Reply With Quote
Old 06-22-2015, 02:31 PM   #21
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
May instead of applying the Hamming window you could simply set the first sample of data[] to zero. That way you the output should still match the input, but then you can see where each buffer starts.
I set data[0] = 0.0. Comment out the Hamming. I'm using the code you posted.

I can see data[0] appearing at exactly the intervals of 1024 and integer multiples of it, even if I shift the input file (sine wave 1000Hz). It's just that the data[0] doesn't align with the sine wave (the input audio), which it should, but rather follows those sample intervals, i.e. is relative to the host.

This corresponds to why the hamming is seen not aligning with the sine wave. But where's the problem?

It seems like reading audio in is done before it's really input into the plug-in, because what's seen here corresponds to that some nFrames must have been passed in before the audio file starts? Right? Otherwise the processingBuffer wouldn't fill up and we wouldn't see data and data[0] at those intervals, because (in shifted.jpg) the data array shouldn't have existed at those intervals yet.

The reason there isn't initial 1024 zeros in these pictures is because I've set PLUG_LATENCY 1024, so it makes the input and output align. But the initial zeros are outputted correctly (seen if I set PLUG_LATENCY 0).
Attached Images
File Type: jpg aligned.jpg (24.5 KB, 88 views)
File Type: jpg shifted.jpg (24.5 KB, 99 views)

Last edited by mviljamaa; 06-22-2015 at 02:57 PM.
mviljamaa is offline   Reply With Quote
Old 06-23-2015, 01:06 AM   #22
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

You could try one more thing, just to see if the whole rebuffering seems to function as it should: Change your audio buffer size. This should not have any effect on the position of the data[0] = 0.0 samples.

Are you resetting your queues in Reset() (i.e. clear input, reset output to FFT_BUFFER_SIZE zeros)? I ask because if you don't, then your plug-in will start processing right after it has been added to the track. This is not a problem per se, but then it will be very hard to predict exactly where the data[0] = 0.0 samples will go. Note that if you do Reset() then the queues will be reset at playback start, but still not necessarily at the beginning of the audio item.
Tale is offline   Reply With Quote
Old 06-23-2015, 02:19 PM   #23
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
You could try one more thing, just to see if the whole rebuffering seems to function as it should: Change your audio buffer size. This should not have any effect on the position of the data[0] = 0.0 samples.
Changing Reapers buffer from 1024 -> 512 shifts the data[0]=0.0 point in the recording to the left by 512.

Quote:
Originally Posted by Tale View Post
Are you resetting your queues in Reset() (i.e. clear input, reset output to FFT_BUFFER_SIZE zeros)?
No, how to do it? I mean, what input, what output?

Last edited by mviljamaa; 06-23-2015 at 02:38 PM.
mviljamaa is offline   Reply With Quote
Old 06-24-2015, 02:01 AM   #24
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
Changing Reapers buffer from 1024 -> 512 shifts the data[0]=0.0 point in the recording to the left by 512.
I have just tried myself, and changing REAPER's audio buffer size doesn't change the position of data[0], so your/mine/our code seems to be working fine. It does depend on clearing queues in Reset(), because else the posisiton of the first data[0] is semi-random. This is because immediately after loading your plug-in the host starts processing it, and most hosts will just keep on processing continuously, whether you start/stop playback or not. This is why you might want to clear queues on playback, but you don't need to. Note that probably not all hosts will call Reset() on playback. But I guess the exact position of data[0] samples isn't important anyway, as long as they are FFT_BUFFER_SIZE samples apart.

Quote:
Originally Posted by mviljamaa View Post
No, how to do it? I mean, what input, what output?
Code:
void MyPlug::Reset()
{
	TRACE; IMutexLock lock(this);

	// Clear input queue.
	processingBuffer.Clear();
	// Clear output queue.
	outputBuffer.Clear();
	// Pre-add silence to output queue.
	outputBuffer.Add(NULL, sizeof(double)*FFT_BUFFER_SIZE);
}

Last edited by Tale; 06-25-2015 at 04:39 PM. Reason: Fixed yet another typo... *sigh*
Tale is offline   Reply With Quote
Old 06-24-2015, 11:31 AM   #25
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
I have just tried myself, and changing REAPER's audio buffer size doesn't change the position of data[0], so your/mine/our code seems to be working fine. It does depend on clearing queues in Reset(), because else the posisiton of the first data[0] is semi-random. This is because immediately after loading your plug-in the host starts processing it, and most hosts will just keeps on processing continuously, whether you start/stop playback or not. This is why you might want to clear queues on playback, but you don't need to. Note that probably not all hosts will call Reset() on playback. But I guess the exact position of data[0] samples isn't important anyway, as long as they are FFT_BUFFER_SIZE samples apart.


Code:
void MyPlug::Reset()
{
	TRACE; IMutexLock lock(this);

	// Clear input queue.
	processingBuffer.Clear();
	// Clear output queue.
	outputBuffer.Clear();
	// Pre-add silence to output queue.
	outputBuffer.Add(NULL, sizeof(double)*FFT_BUFFER_SIZE);
}
This has no effect. It does not solve the original problem.
mviljamaa is offline   Reply With Quote
Old 06-24-2015, 12:03 PM   #26
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
This has no effect. It does not solve the original problem.
Weird, I could have sworn I just read it did seem to correct the original problem as well... Anyway, clearing the queues does solve the "problem" at my end, so I don't think I can help you any further with this.
Tale is offline   Reply With Quote
Old 06-24-2015, 03:44 PM   #27
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
Weird, I could have sworn I just read it did seem to correct the original problem as well... Anyway, clearing the queues does solve the "problem" at my end, so I don't think I can help you any further with this.
So it does apply the hamming relative to the audio input file and not the host samples?
Or, equivalently, shifting the sine wave also shifts the data[0]=0.0 point?

I tested the code with "static", i.e. not moving any file around. And there it seems to work correctly.

This problem is so odd. Like something really depends on the host samples and not the incoming audio, but why it's so?

Last edited by mviljamaa; 06-24-2015 at 03:51 PM.
mviljamaa is offline   Reply With Quote
Old 06-24-2015, 11:23 PM   #28
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
So it does apply the hamming relative to the audio input file and not the host samples?
Or, equivalently, shifting the sine wave also shifts the data[0]=0.0 point?
Well, only if you also shift the playback cursor, so it starts exactly at the beginning of the item containing the sine wave.

Quote:
Originally Posted by mviljamaa View Post
I tested the code with "static", i.e. not moving any file around. And there it seems to work correctly.
Good.

Quote:
Originally Posted by mviljamaa View Post
This problem is so odd. Like something really depends on the host samples and not the incoming audio, but why it's so?
I don't know how to better explain than I already have:
Quote:
Originally Posted by Tale
This is because immediately after loading your plug-in the host starts processing it, and most hosts will just keeps on processing continuously, whether you start/stop playback or not.
But why should this be a problem anyway? If you want to apply some sort of FFT processing, then exact alignment really doesn't matter.
Tale is offline   Reply With Quote
Old 06-25-2015, 09:03 AM   #29
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
Well, only if you also shift the playback cursor, so it starts exactly at the beginning of the item containing the sine wave.
???

What causes this? Like what's the reasoning behind having it work that way and not based on now many nFrames got in.

Quote:
But why should this be a problem anyway? If you want to apply some sort of FFT processing, then exact alignment really doesn't matter.
It's odd why this happens and it may cause problems at same later time. Is it a feature of the VST SDK then? If I'd want the Hamming process to align precisely, then how'd I do it. I wouldn't do it?
mviljamaa is offline   Reply With Quote
Old 06-25-2015, 04:27 PM   #30
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Alright then, I will give it one more try.

Quote:
Originally Posted by mviljamaa View Post
???

What causes this? Like what's the reasoning behind having it work that way and not based on now many nFrames got in.
In most hosts, after you add your plug-in to a track, the host immediately starts calling your ProcessDoubleReplacing(), and will keep on doing so until you remove your plug-in again. If the track is empty, or if you are currently not playing back anything, then the input will be all zeros (silence), but your plug-in will nevertheless be processing it.

When you start playback, or when you render/freeze a track, then your plug-in will most likely already have processed a couple of hundreds or more sample blocks. Most host will likely start playback at the beginnng of an audio buffer, but because the host doesn't know that your plug-in is rebuffering, let alone what your internal buffer size is, it is unlikely that it begins playback exactly at the beginnng of your block size.

However, if you make sure you start playback at the beginning of your audio item, and you reset your queues when playback starts, then playback will start at the beginnng of your block size. "Problem" solved!

Quote:
Originally Posted by mviljamaa View Post
It's odd why this happens and it may cause problems at same later time. Is it a feature of the VST SDK then? If I'd want the Hamming process to align precisely, then how'd I do it. I wouldn't do it?
I guess you would have to track the input waveform somehow, and e.g. reset your queues at the first zero-crossing or whatever. But this can be troublesome when your audio input isn't a pure sine wave...

Anyhow, have I already mentioned that exact alignment doesn't matter for FFT and alike?
Tale is offline   Reply With Quote
Old 06-25-2015, 10:02 PM   #31
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post

Anyhow, have I already mentioned that exact alignment doesn't matter for FFT and alike?
So by this buffering I'll be able to do:

audio input -> 50% overlapping Hamming -> FFT -> processing -> iFFT -> combine windows -> output audio processed. And there's no loss of data. I.e. the output audio matches exactly the input audio processed exactly as I programmed it (and not relative to some host stuff).

The problem still doesn't make sense though. If I wanted to make that plug-in that only does hamming, then it wouldn't make sense to tell the user "oh but you have to check that your playhead is in the correct position". Does it make sense that the libraries haven't implemented that kind of "sensing" of the beginning of the audio? Might try to make it though, because I'm kind of afraid that the combining of the overlapped windows just might screw up. Not in the program, but in the output (i.e. it outputs wrong audio).

Last edited by mviljamaa; 06-25-2015 at 10:09 PM.
mviljamaa is offline   Reply With Quote
Old 06-25-2015, 11:04 PM   #32
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
The problem still doesn't make sense though.
I think herein lies the real problem... Lets hope someone at KVR can explain this better than me.
Tale is offline   Reply With Quote
Old 06-26-2015, 09:18 PM   #33
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
I guess you would have to track the input waveform somehow, and e.g. reset your queues at the first zero-crossing or whatever. But this can be troublesome when your audio input isn't a pure sine wave...
Even this doesn't work.

I copied isBufferSilence from vsttestsuite.h and am using it to test for input silence, then calling Reset() immediately when the input is not silent.

However, this doesn't produce the desired results. I'm still seeing the original problem. So is it even about the Reset()? Actually, by erasing the commands from Reset() you can replicate what you thought Reset() does. The alignment when the playhead has been placed at the start of the file is not caused by clearing buffers in Reset().

I'm able to recognize the audio start now. But not sure how to get the processing to sync now, or i.e. start from that point.

Last edited by mviljamaa; 06-27-2015 at 12:34 AM.
mviljamaa is offline   Reply With Quote
Old 07-16-2015, 07:38 AM   #34
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Tale,

So did you know why is it that the plug-in works right, when the playhead is placed in the beginning of the audio file?
mviljamaa is offline   Reply With Quote
Old 07-16-2015, 12:51 PM   #35
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
So did you know why is it that the plug-in works right, when the playhead is placed in the beginning of the audio file?
Because it is likely that the host starts playback at the beginning of an audio buffer... ?
Tale is offline   Reply With Quote
Old 07-16-2015, 01:11 PM   #36
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

Quote:
Originally Posted by Tale View Post
Because it is likely that the host starts playback at the beginning of an audio buffer... ?
What does that sentence even mean? What audio buffer?
mviljamaa is offline   Reply With Quote
Old 07-16-2015, 11:11 PM   #37
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
What audio buffer?
Host (sound card) audio buffer.
Tale is offline   Reply With Quote
Old 07-17-2015, 01:39 AM   #38
mviljamaa
Human being with feelings
 
Join Date: Jun 2015
Posts: 348
Default

I still don't understand how to solve the problem though.

I can calculate the required delay in order to make the processed signal align, but it still applied processing to the audio buffer (sound card) lengths, or "relative the host playhead", rather than relative to the audio start.

If there was something that I could do in the beginning of the audio file in order to inform the processing that "this is the audio start". But I can't seem to find anything.
mviljamaa is offline   Reply With Quote
Old 07-17-2015, 06:42 AM   #39
Tale
Human being with feelings
 
Tale's Avatar
 
Join Date: Jul 2008
Location: The Netherlands
Posts: 2,940
Default

Quote:
Originally Posted by mviljamaa View Post
If there was something that I could do in the beginning of the audio file in order to inform the processing that "this is the audio start". But I can't seem to find anything.
You seem to assume that a track contains only a single audio file. However, it could easily contain two or more audio files, optionally with gaps between them, or the audio could be realtime e.g. from a virtual instrument or an audio input. Moreover, the host could be looping (part of) the track, possibly at a length that is not an integer multiple of your (or the host's) buffer size. So you see, there really is no "beginning" in realtime audio processing.

That being said, if the host happens to be REAPER, then perhaps you could query its API for info about the items on a track:

http://wiki.cockos.com/wiki/index.php/REAPER_API

Anyway, if you really need to process a single audio file exactly from beginning to end, then why are you trying to do so in realtime in the first place? Why not simply read the file, process it, and write the output to another file?
Tale is offline   Reply With Quote
Old 07-17-2015, 07:55 AM   #40
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 7,713
Default

Quote:
Originally Posted by Tale View Post

Anyway, if you really need to process a single audio file exactly from beginning to end, then why are you trying to do so in realtime in the first place? Why not simply read the file, process it, and write the output to another file?
Which could easily be achieved with the extension API in Reaper and theoretically with Celemony's ARA in some other hosts. (I've tried doing ARA plugins myself but it's quite difficult.)

Anyway, I suspect mviljamaa's processing is somehow broken if it assumes exact locations of the audio to be played through the processing. (Various oddities and glitches have been present in pretty much every realtime plugin that uses FFT I've myself tried, though. I think it's just the nature of the thing in that case...Extraordinary efforts may be required to get rid of all glitches, and it probably isn't worth it.)
__________________
For info on SWS Reaper extension plugin (including Xenakios' previous extension/actions) :
http://www.sws-extension.org/
https://github.com/Jeff0S/sws
--
Xenakios blog (about HourGlass, Paul(X)Stretch and λ) :
http://xenakios.wordpress.com/
Xenakios is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:42 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.