Control Surface Integration (CSI) Technical Feature Discussion

[MixMonkey]

Originally Posted by Geoff Waddington

New build is up.

CSI Exp.zip

Another try at stepped value auto calculation.

Seems to be working normally , but this:

RotaryPushD3 FXParam 5 [ 0.50 ]

...still needs the value twice.

[Geoff Waddington]

Originally Posted by MT4U

Rotaries still setting a 0, but only on "some" VSTs.

All my VST3 are working perfectly.

Ok, thanks for testing.

[Geoff Waddington]

I am done with these crappy VST vendors.

I use UAD plugins exclusively for a reason.

The DSP sonics are excellent.

Just as important, the VST implementations are spot on.

In my testing I've never had a problem with step size calculations, and I find it very convenient.

So, here's the proposal.

As requested, there will be an option to disable auto step size generation.

However, even if it is enabled, the auto step size generation will only be allowed for certain vendors.

Since CSI stability is of utmost importance, as of this moment, the allowed vendors club consists of one member, UA.

Awaiting discussion/pushback...

[MT4U]

Originally Posted by Geoff Waddington

I am done with these crappy VST vendors.

I use UAD plugins exclusively for a reason.

The DSP sonics are excellent.

Just as important, the VST implementations are spot on.

In my testing I've never had a problem with step size calculations, and I find it very convenient.

So, here's the proposal.

As requested, there will be an option to disable auto step size generation.

However, even if it is enabled, the auto step size generation will only be allowed for certain vendors.

Since CSI stability is of utmost importance, as of this moment, the allowed vendors club consists of one member, UA.

Awaiting discussion/pushback...

What about some "manual" invocation of the step size generation?

[Geoff Waddington]

Originally Posted by MT4U

What about some "manual" invocation of the step size generation?

Not quite sure what you mean, you can already turn on output monitor and get the step sizes by turning the controls.

[MT4U]

Originally Posted by Geoff Waddington

Not quite sure what you mean, you can already turn on output monitor and get the step sizes by turning the controls.

I was thinking something like GenerateFocusedFXSTPFile that would write the good old *.stp file in the ZoneStepSizes directory.

If the process crashes or breaks something we will be prepared and we can restart everything.

[MixMonkey]

Originally Posted by Geoff Waddington

I am done with these crappy VST vendors.

I use UAD plugins exclusively for a reason.

The DSP sonics are excellent.

Just as important, the VST implementations are spot on.

In my testing I've never had a problem with step size calculations, and I find it very convenient.

So, here's the proposal.

As requested, there will be an option to disable auto step size generation.

However, even if it is enabled, the auto step size generation will only be allowed for certain vendors.

Since CSI stability is of utmost importance, as of this moment, the allowed vendors club consists of one member, UA.

Awaiting discussion/pushback...

I really wouldn't make any of this "automatic", in terms of initiation.

If someone wants to use CSI capabilities to scan a plugin, determining its parameter step sizes and output some kind of auto-generated FX.zon, that's a reasonable goal and probably achievable. Trying to automate the scanning of a users entire plugin collection and to have this process running continuously is going to end in tears and make life miserable for people that don't need it.

The implementation of the VST3 standard actually seems to require the inclusion of all the controllers on all the MIDI channels. Fortunately, most manufacturers see this for the nonsense that it is and ignore it. Some plugins have an indeterminate number of parameters at initiaton. Some manufacturers don't use evenly spaced steps.

My point is that it's a mugs game to try and second guess all this and it's better to run it on an ad-hoc basis where it's more easily controlled and understood.

To my mind, it would be more useful to encourage people to build maps using the eight rotaries and pushes that nearly all surfaces have and settle on a simple standard to flip the displays between rotaries and pushes. That way the maps can easily travel and all available controls can be identified with no additional documentation.

[Geoff Waddington]

Originally Posted by MixMonkey

I really wouldn't make any of this "automatic", in terms of initiation.

If someone wants to use CSI capabilities to scan a plugin, determining its parameter step sizes and output some kind of auto-generated FX.zon, that's a reasonable goal and probably achievable. Trying to automate the scanning of a users entire plugin collection and to have this process running continuously is going to end in tears and make life miserable for people that don't need it.

The implementation of the VST3 standard actually seems to require the inclusion of all the controllers on all the MIDI channels. Fortunately, most manufacturers see this for the nonsense that it is and ignore it. Some plugins have an indeterminate number of parameters at initiaton. Some manufacturers don't use evenly spaced steps.

My point is that it's a mugs game to try and second guess all this and it's better to run it on an ad-hoc basis where it's more easily controlled and understood.

To my mind, it would be more useful to encourage people to build maps using the eight rotaries and pushes that nearly all surfaces have and settle on a simple standard to flip the displays between rotaries and pushes. That way the maps can easily travel and all available controls can be identified with no additional documentation.

I hear ya'

However, you have been around long enough to remember very early on we envisioned Zones that were portable to many surfaces, it never really caught on.

As far as scanning individually, it seems that some plugins don't play nice, so I don't like that as much.

I think a nice middle compromise is to keep the FXParamStepValuesCache.txt file, but have it be a manual edit process.

We could build up step sizes over time for various plugin parameters and they would be available for any Zone definition, no matter what style.

[underboogiefunk]

Hi,

A workaround for making the hundreds digit disappear is to jump to bar 1001 using CTRL+J and then back to Bar 1.

I am trying to Update to Version 2.0 and looking for a zone file for the Icon Platform Nano to get started. Do you know where I can find something maybe?

Cheers

Originally Posted by rmusmeci

Dear CSI developers,

I very much appreciate your work. I really think the CSIs have great potential under the hood and will always get better with time.

Unfortunately I'm facing an annoying issue with the time display of my IconNano. When the measure indicator overtook a digit, this will not be erased when the time cursor goes back. For exampple, having a project with 110 measures, when the cursor reaches the measure say 107, when going back to the measure 25, the display will show 125 (as if the hundreds digit will become permanent).

The only way to reset the stuck digit is to reinitializing the CSI plugin opening the preferences or restarting Reaper.

Is there someone else facing this issue? Is there some workaround?

Thank a lot

[Geoff Waddington]

Ok, I'm starting to work through this method.

I'm going to write a program to take in the files from the old ZoneStepSizes folder (the ones Funkybot and I scanned a while back) and convert them to the FXParamStepValuesCache.txt file format.

That file will load on startup, and steps sizes will be available for any of the plugins in the FXParamStepValuesCache.txt file.

There will be no auto step size creation and, of course, any manually defined step sizes will override the file definitions.

[MT4U]

Originally Posted by Geoff Waddington

Ok, I'm starting to work through this method.

I'm going to write a program to take in the files from the old ZoneStepSizes folder (the ones Funkybot and I scanned a while back) and convert them to the FXParamStepValuesCache.txt file format.

That file will load on startup, and steps sizes will be available for any of the plugins in the FXParamStepValuesCache.txt file.

There will be no auto step size creation and, of course, any manually defined step sizes will override the file definitions.

Sounds perfect. Am I right to think we should then keep the FXParamStepValuesCache.txt file as slim as possible?
I mean: no need for step sizes of plug-ins we don't use, or parameters we don't use.

[Funkybot]

My suggestion would be create a new action called ToggleAutoStepSizes or a Reaper action CSI: Toggle parameter step size calculation. You run the action, you move the control it either a) writes to the ZoneStepSizes.txt or b) if it can't read the plugin, does nothing (but doesn't crash Reaper either).

This would ideally be done in addition to what Geoff is currently proposing and just add to the ZoneStepSizes.txt.

You could have a second action for ToggleAutoFXMapping. MM would never turn it on. I'd probably never keep it off as long as the crashes could be resolved. I'm pretty sure they can, the initial builds that mapped 8 params worked without crashing.

I feel like those are very "CSI ways" of dealing with things. Offers a combination of flexibility and simplicity. If you're Mixmonkey, you're turning both off and rolling you're own. If you're me, you'll roll your own for the important plugins and just roll the dice with whatever auto-maps for new plugins or stuff I rarely use and haven't mapped.

[Geoff Waddington]

Originally Posted by MT4U

Sounds perfect. Am I right to think we should then keep the FXParamStepValuesCache.txt file as slim as possible?
I mean: no need for step sizes of plug-ins we don't use, or parameters we don't use.

I'll do some load time tests once I have a decent file, I suspect with today's hardware it will be unnecessary to trim the file.

It only loads once at startup.

[Geoff Waddington]

Originally Posted by Funkybot

My suggestion would be create a new action called ToggleAutoStepSizes or a Reaper action CSI: Toggle parameter step size calculation. You run the action, you move the control it either a) writes to the ZoneStepSizes.txt or b) if it can't read the plugin, does nothing (but doesn't crash Reaper either).

That would be nice, except there is no indication CSI couldn't read the plugin, nor is there any indication of the impending crash.

Originally Posted by Funkybot

You could have a second action for ToggleAutoFXMapping. MM would never turn it on. I'd probably never keep it off as long as the crashes could be resolved. I'm pretty sure they can, the initial builds that mapped 8 params worked without crashing.

Not ready to completely give up on auto mapping yet.

Seems it worked better when CSI didn't query the param names, so we may fall back to that.

There is something else that has to be dealt with, even if we forego auto mapping.

If there is an old school Zone definition and a newer TCPFX style Zone definition, which one wins ?

We have so far ignored that possibility.

If we allow auto mapping, that further complicates the decision process.

My first thought is:
Old school beats all others
TCPFX beats auto

[MixMonkey]

Originally Posted by Geoff Waddington

However, you have been around long enough to remember very early on we envisioned Zones that were portable to many surfaces, it never really caught on.

There's no reason why FX.zon restricted to 8 rotaries, 8 pushes, 8 upper displays and 8 lower displays shouldn't be portable.

As far as scanning individually, it seems that some plugins don't play nice, so I don't like that as much.

So how does the bulk scanning differ? Does it not go through the plugins individually?

I think a nice middle compromise is to keep the FXParamStepValuesCache.txt file, but have it be a manual edit process.
We could build up step sizes over time for various plugin parameters and they would be available for any Zone definition, no matter what style.

I'm really struggling to get onboard with the white whale chase that is the determination of parameter step sizes. The great majority of stepped parameters still work with inc/dec rotaries. Those that don't aren't going to lend themselves to automatic FX.zon generation anyway. It seems to be a colossal amount of effort and disruption to the CSI ecosysytem for relatively small gains.

The most useful aspect of the step size determination process (in my mind) is that it identifies which parameters can be condsidered switches (simple toggles). This would then allow any subsequent automapping software to assign these to pushes rather than rotaries. Beyond that, in my opinion, the returns diminish rapidly.

[MixMonkey]

Originally Posted by Geoff Waddington

My first thought is:
Old school beats all others
TCPFX beats auto

I agree

[Funkybot]

I concur.

[Geoff Waddington]

Originally Posted by MixMonkey

I'm really struggling to get onboard with the white whale chase that is the determination of parameter step sizes. The great majority of stepped parameters still work with inc/dec rotaries.

It's all about responsiveness.

A typical example is a meter switch on a compressor -- Input, GR, Output.

Step sizes allow for a better feel, it just feels more like the rotary switch on the GUI as you turn it when you couple step sizes with tick count.

[Geoff Waddington]

New build is up.

CSI Support Files.zip

CSI Exp.zip

Converted to new step values approach.

You will need the support files which contain FXParamStepValuesCache.txt

That file contains the step sizes for over 1000 plugins and takes under a second to load at startup.

No more scanning, etc.

Much more direct, it is a file that we will update as new plugins that have steps become available.

As always, custom definitions in Zones override.

Also [ 0.0 ] should work again.

[MixMonkey]

Originally Posted by Geoff Waddington

You will need the support files which contain FXParamStepValuesCache.txt

That file contains the step sizes for over 1000 plugins and takes under a second to load at startup.

No more scanning, etc.

For the avoidance of doubt, does this mean that there's no more intereference in the loading of FX Zones to surfaces by CSI mechanisms seeking to harvest plugin data of various kinds?

Much more direct, it is a file that we will update as new plugins that have steps become available.

...and this updating takes place outside the user's Reaper/CSI enviroment?

[Geoff Waddington]

Originally Posted by MixMonkey

For the avoidance of doubt, does this mean that there's no more intereference in the loading of FX Zones to surfaces by CSI mechanisms seeking to harvest plugin data of various kinds?

Correct.

Originally Posted by MixMonkey

...and this updating takes place outside the user's Reaper/CSI enviroment?

Correct.

No more chasing white whales, just a quick load of a file at startup

Then it just resides in memory until a plugin requests step sizes.

A plugin will request step sizes only if it has no step sizes defined in the Zone -- aka you will never even hit that code

[MixMonkey]

Originally Posted by Geoff Waddington

No more chasing white whales, just a quick load of a file at startup

Then it just resides in memory until a plugin requests step sizes.

A plugin will request step sizes only if it has no step sizes defined in the Zone -- aka you will never even hit that code

That's awesome Geoff, thanks!

It may have been expectation bias, but I was feeling that FX.zon loading in the C4 was becoming sluggish.

It's always been the case that the first FX Zone load of any particular plugin in a project is slower than all its subsequent loads (I put this down to the first load being from disk and then the rest are from memory), but I was feeling a slowdown across the board recently.

[Geoff Waddington]

Originally Posted by MixMonkey

That's awesome Geoff, thanks!

It may have been expectation bias, but I was feeling that FX.zon loading in the C4 was becoming sluggish.

It's always been the case that the first FX Zone load of any particular plugin in a project is slower than all its subsequent loads (I put this down to the first load being from disk and then the rest are from memory), but I was feeling a slowdown across the board recently.

Please let me know if there is any change in load speed with this build and support files.

I had a look at the generated step size file on my machine, and it was getting downright ugly with bloat, so that may be the source of your load issues.

Make sure you replace your FXParamStepValuesCache.txt with the one in the support files.

Actually CSI hits the disk for every plugin Zone load, but the OS may be caching recently accessed files, so it might look like it's in memory.

[MixMonkey]

Originally Posted by Geoff Waddington

Please let me know if there is any change in load speed with this build and support files.

Will do.

Actually CSI hits the disk for every plugin Zone load, but the OS may be caching recently accessed files, so it might look like it's in memory.

It would definitely be worthwhile loading FX.zon into memory as they are accessed in the project. That way we get the benefit of the original load-everything-into-memory-at-start-up method but without the long start up delay.

[Geoff Waddington]

Originally Posted by MixMonkey

It would definitely be worthwhile loading FX.zon into memory as they are accessed in the project. That way we get the benefit of the original load-everything-into-memory-at-start-up method but without the long start up delay.

CSI does a scan of all plugins in each surface's Zone folder at startup.

It reads the first line and caches the name, alias, and file path.

Then it loads the "usual suspects", Home, Included, and Associated Zones.

The name and alias are all FXMenu needs for display.

When you push the Rotary Top, CSI uses the cached file path to load the full Zone contents, and configures the Zone for the current context - slot number, Track, etc.

On any modern machine the load step is on the order of microseconds, I've measured it.

There isn't really any measurable benefit to caching the Zone file, and it actually would complicate the parsing, so that's why it is the way it is.

So, although it would be theoretically worthwhile to cache, it isn't in practice.

[MT4U]

Originally Posted by Geoff Waddington

CSI does a scan of all plugins in each surface's Zone folder at startup.

It reads the first line and caches the name, alias, and file path.

Then it loads the "usual suspects", Home, Included, and Associated Zones.

The name and alias are all FXMenu needs for display.

When you push the Rotary Top, CSI uses the cached file path to load the full Zone contents, and configures the Zone for the current context - slot number, Track, etc.

On any modern machine the load step is on the order of microseconds, I've measured it.

There isn't really any measurable benefit to caching the Zone file, and it actually would complicate the parsing, so that's why it is the way it is.

So, although it would be theoretically worthwhile to cache, it isn't in practice.

Thanks!

Load times seem optimal here. Rotaries working like a charm.

Setting values still requires repeating the value inside the brackets.

[Geoff Waddington]

Originally Posted by MT4U

Thanks!

Load times seem optimal here. Rotaries working like a charm.

Setting values still requires repeating the value inside the brackets.

Thanks, good news.

I will continue to research the repeating value issue.

[Funkybot]

Question: how will we add to the ZoneStepSizesCache.txt file in the future if there's no scan mechanism? I've bought plugins since the last batch.

[Funkybot]

Originally Posted by Geoff Waddington

I am done with these crappy VST vendors.

I use UAD plugins exclusively for a reason.

The DSP sonics are excellent.

Just as important, the VST implementations are spot on.

Eh...UA does some really stupid sh*t too. For instance, they have hardwired MIDI CC assignments on their instruments. Why? What hardware are these CC's even hardwired to work with? Unclear. Why isn't there a MIDI learn system? Or no MIDI CC assignments at all? Why are some parameters on the GUI not exposed as automation parameters at all? Their Minimoog emulation doesn't even get the waveforms right, which is pretty basic.

The FX are quality. But their instruments are not up to their own high standards when it comes to things like that. But there are lots of great vendors out there that do FX and FX+instruments well. And I'm a UA fan overall. But there are other companies out there just as good or better.

But I digress...

[Geoff Waddington]

Originally Posted by Funkybot

Eh...UA does some really stupid sh*t too. For instance, they have hardwired MIDI CC assignments on their instruments. Why? What hardware are these CC's even hardwired to work with? Unclear. Why isn't there a MIDI learn system? Or no MIDI CC assignments at all? Why are some parameters on the GUI not exposed as automation parameters at all? Their Minimoog emulation doesn't even get the waveforms right, which is pretty basic.

The FX are quality. But their instruments are not up to their own high standards when it comes to things like that. But there are lots of great vendors out there that do FX and FX+instruments well. And I'm a UA fan overall. But there are other companies out there just as good or better.

But I digress...

Yup, agree, I was referring to the FX, although the only UA instrument I have (Waterfall) is excellent, not interested in any other instruments they offer though.

[Geoff Waddington]

Originally Posted by Funkybot

Question: how will we add to the ZoneStepSizesCache.txt file in the future if there's no scan mechanism? I've bought plugins since the last batch.

We'll just have to use the old fashioned method of engaging the output monitor and turning the knobs, then adding the results to the file.

[Funkybot]

Originally Posted by Geoff Waddington

We'll just have to use the old fashioned method of engaging the output monitor and turning the knobs, then adding the results to the file.

The scanning worked great on my M1 Mac, just was dodgy on Windows. And I understand and agree with MixMonkey's points around auto-scanning of any kind.

I don't want you to try to solve for it now, but once things are stable, I'd suggest, there might be a way to not throw the baby out with the bath water and have some kind of On-Demand scan method that works maybe one plugin at a time and writes to that file. Maybe it's a Reaper action like "CSI scan focused plugin for steps" or something like that. SWS adds a billion actions to Reaper, there's got to be a way for CSI to similarly add more Reaper actions without having to jump through crazy hoops. And since that file is global (not per-surface or per-page) there shouldn't be a question of where to write to.

But it makes more sense to table all that stuff and get all this other stuff sorted and stable for a while then revisit.

[Geoff Waddington]

Originally Posted by Funkybot

The scanning worked great on my M1 Mac, just was dodgy on Windows. And I understand and agree with MixMonkey's points around auto-scanning of any kind.

I don't want you to try to solve for it now, but once things are stable, I'd suggest, there might be a way to not throw the baby out with the bath water and have some kind of On-Demand scan method that works maybe one plugin at a time and writes to that file. Maybe it's a Reaper action like "CSI scan focused plugin for steps" or something like that. SWS adds a billion actions to Reaper, there's got to be a way for CSI to similarly add more Reaper actions without having to jump through crazy hoops. And since that file is global (not per-surface or per-page) there shouldn't be a question of where to write to.

But it makes more sense to table all that stuff and get all this other stuff sorted and stable for a while then revisit.

Totally agree.

[Funkybot]

Originally Posted by MT4U

In the button press, the value goes to 1.00, and then back to 0.00 on button release.

Same here.

It's like CSI's is expecting the number of steps defined for each press widget to match the number of values the widget transmits [ 1.0 0.0 ] and if not, it just transmits those widget values as param values.

Geoff should be easy to reproduce on your side. Just start with this...

Zone "Track"

Shift+RotaryPush|     TrackPan [ 0.5 ]

Then slowly press RotaryPush1, you see the TrackPan for that channel immediately jump to the max value [ 1.0 ] then if you release the button, you see the TrackPan drop to the min value [ 0.0 ]. But never does it go to 0.5.

Because it happens with TrackPan...I'm wondering if it's not directly related to step param files at all.

[Geoff Waddington]

Originally Posted by Funkybot

Same here.

It's like CSI's is expecting the number of steps defined for each press widget to match the number of values the widget transmits [ 1.0 0.0 ] and if not, it just transmits those widget values as param values.

Geoff should be easy to reproduce on your side. Just start with this...

Zone "Track"

Shift+RotaryPush|     TrackPan [ 0.5 ]

Then slowly press RotaryPush1, you see the TrackPan for that channel immediately jump to the max value [ 1.0 ] then if you release the button, you see the TrackPan drop to the min value [ 0.0 ]. But never does it go to 0.5.

Because it happens with TrackPan...I'm wondering if it's not directly related to step param files at all.

What happens if you use this instead:

Zone "Track"

Shift+RotaryPush|     TrackPan [ 0.5 0.5 ]

[Funkybot]

Originally Posted by Geoff Waddington

What happens if you use this instead:

Zone "Track"

Shift+RotaryPush|     TrackPan [ 0.5 0.5 ]

Works like a charm.

[Funkybot]

Originally Posted by MT4U

Sorry, Funkybot. I just saw your post and had a look.

I've been uploading a RAR file and the stash was renaming it to ZIP, since that was the first compressed format I uploaded.

I just up'ed a proper ZIP file

Still no worky here.

If I rename it .rar it opens. So I think the last upload didn't take. Or it's not a proper zip file.

[MixMonkey]

Originally Posted by Funkybot

Works like a charm.

I'm afraid this issue needs addressing as a matter of some urgency, as it has the capability to reset a parameter value to 0 (or whatever that represents) without the control being pressed.

Example: I have the VOS EQ mapped to a single C4 Zone. D1, D2 and D3 rotaries are the LF, MF and HF gains. The D1, D2 and D3 pushes are also the LF, MF and HF gains, set to send 0.5 when pushed (to reset the gain to 0)

Here's were it gets interesting. My C4 Home Zone has the FXMenu on row D. On a particular track, the VOS EQ is in the second slot and appears above rotary/press D2. If I select VOS EQ from the FXMenu (by pushing D2) the MF gain is instantly set to -18dB (aka 0) when the plugin maps to the C4.

If I move the VOSEQ to slot 1, it's the LF gain that is instantly set to -18dB.

I'm sure I don't have to point out this is a bit of a deal breaker

[Funkybot]

Originally Posted by MixMonkey

I'm afraid this issue needs addressing as a matter of some urgency, as it has the capability to reset a parameter value to 0 (or whatever that represents) without the control being pressed.

Example: I have the VOS EQ mapped to a single C4 Zone. D1, D2 and D3 rotaries are the LF, MF and HF gains. The D1, D2 and D3 pushes are also the LF, MF and HF gains, set to send 0.5 when pushed (to reset the gain to 0)

Here's were it gets interesting. My C4 Home Zone has the FXMenu on row D. On a particular track, the VOS EQ is in the second slot and appears above rotary/press D2. If I select VOS EQ from the FXMenu (by pushing D2) the MF gain is instantly set to -18dB (aka 0) when the plugin maps to the C4.

If I move the VOSEQ to slot 1, it's the LF gain that is instantly set to -18dB.

I'm sure I don't have to point out this is a bit of a deal breaker

Interesting...so the RotaryPush2 value of 1.0 [from the widget] properly engages the GoFXSlot immediately, so immediate it seems that the release value from the widget is zeroing out the plugin param? Yeah, that's bad.

BTW, I wasn't indicating that the [ 0.5 0.5 ] working was the solution. Just that, yeah, it works to reset the param to 0.5. There's definitely something wrong.

I'd definitely throw a "there be dragons here" warning on any users using these recent Exp builds in critical projects.

[MixMonkey]

Originally Posted by Funkybot

BTW, I wasn't indicating that the [ 0.5 0.5 ] working was the solution. Just that, yeah, it works to reset the param to 0.5. There's definitely something wrong.

I didn't infer that, I was just too lazy to find another post on the subject

I'd definitely throw a "there be dragons here" warning on any users using these recent Exp builds in critical projects.

That advice should apply to every EXP build, always Besides, we need as many people as possible using the main build in case there's anything we missed.