ReaRack3 Modular Synth Development (was 'Using Audio as a Control Signal')

[Time Waster]

Over on this thread https://forum.cockos.com/showthread.php?t=181406 there has been a bit of discussion about using audio channels for communicating control messages. AFAIK there is no standard for this. The first question that arises is should the control signal value be 0 to 1 or -1 to 1?

The advantages of using 0 to 1 is that 0 is the natural 'off' state for an audio signal and, although fairly trivial, the coding is is a bit easier and more intuitive.

The advantages of using -1 to 1 are higher resolution and you can use control signal output directly as sound signal if desired.

Initially I was leaning towards using -1 to 1, but I'm now thinking that 0 to 1 may be the best choice.

I would be interested in hearing opinions as to which method is best and whether there are any precedents that could be learned from?

[mschnell]

AFAIK; for analog synth there is a standard defining 0..1V so this is consistent to 1.0000 = 0dbV = 1 V, 0 = -inf dBV = 0 V.

With OSC, mostly parameters are encoded 0.0 .. 1.0.

it might be worth considering that in many cases control signals result in exponential effects, coded in dB or volt / octave.

-Michael

[cyrano]

In hardware, I've only seen 0...1 VDC and 0...10 VDC control voltages. 1V for synths and VCA's. 10 V for lighting. There are still some old solutions out there that use 1...2.3 VDC for VCA.

Mind you, that's DC. No negative values. Most audio interfaces can't output DC. The only exception that I know of, is RME's headphone outputs that can really output DC. I've used it with a FF400 and a FF800.

I suspect inside a DAW, you could use negative values. But I can't immediately see an obvious advantage...

[Time Waster]

The idea is that we would be using the audio routing system for sending control signals within the DAW, in other words, just sending a varying stream of numbers. We would not be converting the the signal to an analog voltage for use with hardware, so that's not really a consideration.

The driver for this is that there is talk about implementing a 'wire' view for audio routing, and for FX, this could be used to emulate a 'patch lead' view for connecting up modules.

What I'm thinking of doing is to provide an option in ReaRack for using audio channels for communicating modulation control signals between modules. My idea is that this could be used either exclusively, or in conjunction with MIDI as the communication protocol. Hence I would like to get some consensus on a standard for using audio in this manner before I start on it.

So far it seems it is more intuitive to use 0 to 1, so I think that's probably the best choice, but I will leave it open for a while for further discussion before I commit to it.

[junh1024]

There is a SMPTE LTC reader thingy in your JSFX. If not, see old version here: http://www.1014.org/shiz/code/smpteltcreader

[Time Waster]

DC offset is not the issue. The issue is, do we want to use the full available range of values (-1 to 1) or do we just want to use the positive values?

Another issue is that within the DAW we can use values far in excess of the -1 to 1 range because reaper uses 64 bit for internal processing, whereas the -1 to 1 range is 24 bit, if I understand it correctly. I propose that the comms signal would be hard limited to whatever range we choose, to avoid unexpected behavior.

[mschnell]

Originally Posted by Time Waster

The driver for this is that there is talk about implementing a 'wire' view for audio routing,

In fact in the pre-release forum Cockos showed a "proof of concept" implementation of such a "wire view" of the audio routing routing between tracks in Reaper. This provoked excitement of a lot of Reaper users.

Of course it would be viable to extend this alternate view to the routing within an FX chain and hence usable for ReaPack.

Of course it would be viable to extend this alternate view to the Midi routing and hence usable for ReaPack "as it is".

Both is discussed in detail in the pre-release forum and IMHO it's not unlikely that this is going to happen on the long run.

But as Midi routing supposedly will not be ignored, I don't see that this should drive a move of ReaRack's functionality.

-Michael

[mschnell]

Originally Posted by Time Waster

Another issue is that within the DAW we can use values far in excess of the -1 to 1 range because reaper uses 64 bit for internal processing, whereas the -1 to 1 range is 24 bit, if I understand it correctly.

Nope. Reaper uses 64 Bit Floating Point format. So any predefined range makes use of the full resolution of (approximately) 52 bits. OK a +/- range additionally makes use of the sign bit. But 52 bits will be enough even for the most demanding usage case. And because of this, hard limiting to any predefined range does not make sense, as resolution is only reduced for those overshooting values. The signal receiver should take care of this, if appropriate.

Smaller bit count only comes into play when rendering. But here you again can chose between multiple options.

This said, in certain applications it could be advantageous to "render" the control signals as audio. e.g. you could do a Flak file combining audio and control channels.

-Michael

[mpl]

I have some in progress projects with 0 items count, where Reaper only contain a tracks route between each other CV voltages for reaktor instances and vcvrack bridge.

Huge step further in designing tracks I'd say.

[cyrano]

Originally Posted by Time Waster

The idea is that we would be using the audio routing system for sending control signals within the DAW, in other words, just sending a varying stream of numbers. We would not be converting the the signal to an analog voltage for use with hardware, so that's not really a consideration.

Oops. Sorry. Just disregard the remarks about hardware.

Could an option to send CV to hardware outputs be interesting for some?

[Time Waster]

Originally Posted by cyrano

Could an option to send CV to hardware outputs be interesting for some?

I think it would be, but there are already MIDI to CV converters for that (hardware) AFAIK. Not sure if you can do that with software? Would need to look at audio outlet specs vs CV specs, but I suspect they would not be compatible

[cyrano]

MIDI hasn't got the resolution of a "real" DC voltage, I presume?

That leads to zipping noise, when using it with a VCA, because of stepped DC, not continuous changing.

[Time Waster]

I don't know much about MIDI to CV converters, I'm assuming you can get converters which can accept high res 14 bit MIDI (16384 steps), so it won't be noticeable.

[cyrano]

Most MIDI to CV converters are 7-bit, I think. None can control more than one channel simultaneously. There's always a tiny delay between channels.

CV has true 20 bit resolution and is instantaneous.

Not important, unless you want fi 32 channels to be able to fade in or out at the same time. With 7 bit MIDI, zippering noises are unavoidable. Or modulation on more than one channel. Or...

[mschnell]

Originally Posted by mpl

I have some in progress projects with 0 items count, where Reaper only contain a tracks route between each other CV voltages for reaktor instances and vcvrack bridge.

Huge step further in designing tracks I'd say.

What are you trying to point out here ?

My live playing setup does not feature any items, but some 30 tracks and a lot of audio and Midi routing between them. The FX chains of many tracks hold VSs, VSTis, and JSFX Midi and Audio effects.

I'm perfectly happy with routing the audio streams as Audio, and the control streams as Midi

-Michael

[mschnell]

Originally Posted by cyrano

That leads to zipping noise, when using it with a VCA, because of stepped DC, not continuous changing.

Absolutely correct, especially for standard 7 bit Midi. That is why I did the "Midi Volume" and "1Midi Fader X" JSFX plugins that do their best to suppress zipper noise.

But ReaPack works with 14 Midi, anyway, so the Zipper noise is 42 dB less intense.

-Michael

[ashcat_lt]

One of the coolest things about audio as CV is that you can render (to floating point) or record them and then have "automation items" that actually work like other items and can be shared across multiple parameters across multiple tracks. Yes, you can do that via midi too I suppose, but many of the ways that we might want to manipulate the data are a bit more awkward in MIDI.

One of the least cool things about it is that there are so few plugins that support it. We can use PM from audio signals, and often that's good enough, but there are a number of issues with PM that make it less than ideal in many situations.

But then you CAN route audioCV to sidechain inputs to get at least some control of plugins that use that kind of thing.

[mpl]

Originally Posted by mschnell

What are you trying to point out here ?

I meant it is another level of thinking (closer to what happens in modular) when you are out of classic project structure during producing track (not performing live).

[Time Waster]

Originally Posted by ashcat_lt

One of the coolest things about audio as CV is that you can render (to floating point) or record them and then have "automation items" that actually work like other items and can be shared across multiple parameters across multiple tracks. Yes, you can do that via midi too I suppose, but many of the ways that we might want to manipulate the data are a bit more awkward in MIDI.

One of the least cool things about it is that there are so few plugins that support it. We can use PM from audio signals, and often that's good enough, but there are a number of issues with PM that make it less than ideal in many situations.

But then you CAN route audioCV to sidechain inputs to get at least some control of plugins that use that kind of thing.

ReaRack has an Audio <> MIDI module which can convert MIDI CC to an audio signal and vice versa, so most of that is already possible.

[ashcat_lt]

Good Point!

If we had a pencil tool for audio, that would be cool. What we can do is drop in a simple DC generator plugin and draw automation for that, then render or record to audio if/when necessary. Bump audio buffers down to 1 sample first for best results.

[Time Waster]

You can create automation item for a MIDI CC, then use the Audio <> MIDI module to record that as an audio signal.

[Time Waster]

Originally Posted by cyrano

Most MIDI to CV converters are 7-bit, I think. None can control more than one channel simultaneously. There's always a tiny delay between channels.

CV has true 20 bit resolution and is instantaneous.

Not important, unless you want fi 32 channels to be able to fade in or out at the same time. With 7 bit MIDI, zippering noises are unavoidable. Or modulation on more than one channel. Or...

After a bit of searching, it looks like the hardware modular guys are only interested in using MIDI to connect to a MIDI keyboard, so they are mainly interested in note information, pitch bend and expression. CCs are only used for program changes. However, there are some Arduino projects for MIDI to CV, so maybe someone with Arduino skills could hack something to convert hi res CCs to CV.

[mschnell]

Not cheap but seemingly existing:
-> http://www.kentonuk.com/products/ite.../modsolo.shtml
-Michael

[cyrano]

Originally Posted by mschnell

Absolutely correct, especially for standard 7 bit Midi. That is why I did the "Midi Volume" and "1Midi Fader X" JSFX plugins that do their best to suppress zipper noise.

But ReaPack works with 14 Midi, anyway, so the Zipper noise is 42 dB less intense.

-Michael

I'd like to avoid it. It tends to cumulate and, worse, go up in frequency, leading to all sorts of (often inaudible) noises. Can be easily smoothed with just a capacitor on the CV line, but that also slows the voltage rise/fall a bit.

Using a CV as a voltage avoids ALL of that.

Besides, as others have pointed out, a voltage can be recorded, not only inside the DAW, but also in every other (digital) recorder on earth. It also can easily be "mixed". If coming from several channels, you can easily add, subtract or average the voltages in hardware.

[cyrano]

Originally Posted by Time Waster

After a bit of searching, it looks like the hardware modular guys are only interested in using MIDI to connect to a MIDI keyboard, so they are mainly interested in note information, pitch bend and expression. CCs are only used for program changes. However, there are some Arduino projects for MIDI to CV, so maybe someone with Arduino skills could hack something to convert hi res CCs to CV.

I have no idea what synth users do. It's a very diverse crowd.

Have you seen the prices?

Most people already have a 24 bit audio interface. Technically a lot better than the dacs in a CV generator. And some people have spare channels...

Most interfaces can be easily modded to provide DC CV. Some support it out of the box.

I'm trying to drive 17 motorised faders (and a number of buttons and encoders) from REAPER. MIDI is slow for such a thing. I constantly run into speed/latency problems. Moving multiple faders reliably from/to MIDI seems next to impossible, because it's all sequential. And 10 ms is a lot, if you multiply it by 17.

[preferred.nomenclature]

How exciting! I vote -1 to 1. Some people (hi) will want to record these signals as audio and slice and rearrange them - hell I’d want to throw them in a virtual sampler - point being that these will sometimes be displayed as waveforms, where they’ll look janky and hard to read if they only go from 0 to 1.

[mschnell]

Originally Posted by cyrano

I'm trying to drive 17 motorised faders (and a number of buttons and encoders) from REAPER. MIDI is slow for such a thing.

I do something like this myself via Midi all the time.

Using midi via USB there are no speed problems at all.

But I feature many dropout problems when using OSC via WLAN .

-Michael

[ashcat_lt]

@p.n - We've got the rectified peaks view. That's global of course, so would affect your actual audio too. I'm not sure I could/want to get used to that. I don't really think 0-1 looks all that wonky myself, but there is a bit of wasted space.

In the end it's not much of a difference, but I'm kind of a lazy coder. Basically either the plugin needs to know what to expect or the user needs to know what the plugin expects. Either way, the plugin should protect itself from "invalid" or "unexpected" inputs.

BUT if we ever do intend to use PM to follow this audio CV, we have to remember that the audio detector in there is fullwave rectified. So BTW, are most audio detectors about anywhere you find them. Like, any "sidechain input" that you might hope to press into service. This is a pretty strong argument against bipolar CV. Yes, you could convert to midi first, but that's extra plugins, extra processing...

I suppose it's worth mentioning here that PM has other issues. It IS a conventient way to use audio CV on plugs that aren't coded for such a thing, but it's not meant for this kind of use. It's rectified like I said, and stairstepped at audio buffer blocks, but it's also hard gated at 10^(-3) at best, and it's curvy by default. I wasn't able to figure out what the curve actually is or undo it perfectly with the built in curve graph thing. I submitted a feature request a while ago to have a CV mode in there. Like just a checkbox that disables all of that (well, we can't get around stair steps) and maps audio to parameters directly.

[cyrano]

Originally Posted by mschnell

I do something like this myself via Midi all the time.

Using midi via USB there are no speed problems at all.

I connect one local computer over a network to another remote computer. Doesn't matter if it is wireless, as it is resilient.

All audio I/O hardware is connected to the remote headless computer.

The control surface is connected to the local computer. That's the "control station". It controls RME's TotalMix and REAPER. But TotalMix can also be controlled from an ipad an REAPER even from anything with a decent browser. There's no need for REAPER on the control station. And there's a full screen sharing session if needed.

The control station might also be running the lights, via DMX, over another network, using the same control surface. But that's for later.

That's where MIDI problems start. Sending MIDI over network is worse than OSC, in my limited testing.

But I feature many dropout problems when using OSC via WLAN .

You can do OSC over wireless without problems. But you really need a dedicated network and a router that understands the AV portion of QOS.

I'm not counting on using this live on wireless. That would be stupid. But at home, while testing, I use it over wireless. And yes, sometimes that will result in dropouts because I'm too far from the router, or something else on the network is pulling hard.

But, hey, I just figured that it would be interesting to be able to put CV in an audio channel. At least, if it's not much work on a coding level.

[ashcat_lt]

I don't know how this fits into the conversation we've been having, but we were talking about standards...

Most of the generators that I've been creating to this point are only really putting out one or two CV values. Basically, either mono or stereo. I have been coding them to send those values on channels 5/6. Everybody uses 3/4 for audio sidechains, and I feel like I'd ought to just leave that space by default. I do use 3/4 as sidechain inputs, but that's not the same as CV input to control parameters. Haven't gotten that far yet. Haven't really needed it...

But I did leave that space. I think it's just a little bit safer, and I do it even when the only output is the CV. I'd rather not leave the possibility that I could drop it on a track before muting it and having a sudden jump to a new DC level sent to my speakers, and I definitely don't want to do that to somebody else.

I'm not by any means trying to dictate a standard to the community. I'm saying mostly just that these are things to consider, and this is what I've been doing.

[mschnell]

Other than most other network protocols, OSC uses UDP and not TCP, and with this missing (damaged) packets are not automatically re-transferred. The OSC user need to implement re-transfer in the OSC layer itself, which I did not (yet) do.

-Michael

[Time Waster]

Originally Posted by ashcat_lt

I don't know how this fits into the conversation we've been having, but we were talking about standards...

Most of the generators that I've been creating to this point are only really putting out one or two CV values. Basically, either mono or stereo. I have been coding them to send those values on channels 5/6. Everybody uses 3/4 for audio sidechains, and I feel like I'd ought to just leave that space by default. I do use 3/4 as sidechain inputs, but that's not the same as CV input to control parameters. Haven't gotten that far yet. Haven't really needed it...

But I did leave that space. I think it's just a little bit safer, and I do it even when the only output is the CV. I'd rather not leave the possibility that I could drop it on a track before muting it and having a sudden jump to a new DC level sent to my speakers, and I definitely don't want to do that to somebody else.

I'm not by any means trying to dictate a standard to the community. I'm saying mostly just that these are things to consider, and this is what I've been doing.

I think that could be an agreed standard, like in the case of MIDI CCs being allocated to certain parameters, but I would not like to limit that in the code as there may be cases where someone might want to use to use channels 1 to 4 as CV.

We should perhaps refer to this as CS (control signal) rather than CV, on account of it having nothing to do with voltage.

[Time Waster]

I have uploaded some modified ReaRack modules as ReaRack3: https://stash.reaper.fm/v/34071/ReaR...dularSynth.zip

There is now an LFO in addition to the previous Envelope Multi-Tool and Amplifier modules. I've configured these so that input and output sliders allow selection of either MIDI CCs or audio channels for control signal input or output.

Currently I'm just working on the configuration, so there are probably bugs in these modules. The multi-tool only allows output of audio signals at the moment, I haven't started working on audio input for that module yet. Any comments on the work so far would be appreciated, before I get too far into it.

Thinking more about the standard for audio control, I'm thinking that inputs should perhaps always use either positive values only, or the absolute value of the audio control signal. Absolute value might be better because it improves the ability to add an option to use a sample window to obtain the envelope an audio signal to use as the control. What do you reckon?

[mschnell]

What exactly do you mean by "absolute values" ?

If this means that -1 and +1 are considered the same values, this seems rather weired to me.

-Michael

[Time Waster]

Yes that is correct. Usually if you are using an audio signal (-1 to 1) as a control signal, say for sidechaining, you will use the envelope of the signal, in which case you rectify it (absolute value) and find the maximum (peak) value in the sample range to use as the control value. The minimum size of the sample window depends on the frequency of the signal, which needs to cover at least two peaks. If you rectify the signal rather than discarding the values below zero, you can halve the size of the sample window.

In our case, using the audio routing system to transmit control information, we are transmitting or receiving a positive value only signal, so we can use the value at every sample rather than an average or peak value over a range of samples, giving us sample accurate control. However, it might be nice to have the option to use the envelope of a high frequency audio signal. Taking the absolute value of an input as a default will not affect the dedicated control signals, which are all positive values and it ensures that any audio signal received as a control signal will be within the 0 to 1 range. Received values above 1 should be hard clipped at 1.

That is my theory.

[ashcat_lt]

I guess it depends on just how modular you really want to get. For actually wiggling sliders, 0-1 is (I think obviously) the most sensible way to do it. But do you really want to build in rectification and averaging or peak finding in the plugin itself? You could just hard clip at 0 and 1 and leave all the other conditioning to other modules. Want it rectified? Use a rectifier module. Want to extract the highest peak from a window? Another module. Want an average over time? You guessed it, another module.

[Ivannn Bennnettt]

Thank you much again!
I wonder does jfx support tab bar? It'd be great to put CC route or other stuff there.

[Time Waster]

Originally Posted by ashcat_lt

I guess it depends on just how modular you really want to get. For actually wiggling sliders, 0-1 is (I think obviously) the most sensible way to do it. But do you really want to build in rectification and averaging or peak finding in the plugin itself? You could just hard clip at 0 and 1 and leave all the other conditioning to other modules. Want it rectified? Use a rectifier module. Want to extract the highest peak from a window? Another module. Want an average over time? You guessed it, another module.

I agree that just discarding the out of range values is probably the most straight forward approach and I'm happy to go with extra modules. ReaRack already has an Audio<>MIDI module, so I'll just update that to include additional audio processing options.

[Time Waster]

Originally Posted by Ivannn Bennnettt

Thank you much again!
I wonder does jfx support tab bar? It'd be great to put CC route or other stuff there.

JSFX does not include tabs out of the box. It is possible to code this in the graphics section, however, it is quite a bit of work. @Geraintluff has done a lot of work on a graphical UI, but I haven't gotten into it yet.

[Time Waster]

I know I said earlier that I would allow any audio channel to be used for control signals, but I'm now thinking that I should disallow channels 1 and 2. If do you want to use audio from 1 and/or 2 for control, you could use the Audio<>MIDI module to process and route it to a control channel (3 to 64).

Currently, ReaRack modules that output or process full audio (sound content) do so on channels 1 and 2. I had never thought about what happens if the track is set to use channels not including 1 or 2 and I'm not sure if I need to worry about that? Anyway, because of this it does make sense to limit control signal audio to channels higher than 2.