Crash on load when an extra menu is added to a VST3 plugin

[Synthdev]

I have a replicable crash when my plugin (with a large number of popup menus) is loaded in Reaper. It can be added into a new track, but this method can crash Reaper when the number of menus increases beyond a certain point, but even if successfully added, the resulting file will crash Reaper when loaded, every time.

The crash doesn't occur in the plugin, and (the memory exception?) is not caught by windows. This bug is likely very old, and I have had to work around it in irritating ways, by limiting the number of options. Right now, I can't add an important menu to one of the synth modules, so I'm stuck.

OTOH Ableton has no problems loading the plugin.

I can send the devs a copy of the plugin if required.

[vitalker]

I think it would be enough for them to get crash report.

[Synthdev]

Thanks, I will do that, if I can work out how to get one.

Update: I followed the instructions on the net, but, assuming I haven't done something stupid (which is quite possible!), windows isn't catching the exception :/

[Justin]

When you install REAPER you can have the installer configure REAPER to generate dump files on crash, after which you can zip up the dump file and send it our way.

... or yeah, send a build of the plug-in to support at cockos dot com, thanks!

[Synthdev]

Thanks Justin, I've sent the binaries, easier to work from imho.

Also, there's another (well known) bug, drag/drop file events are not being passed to the plugin, instead showing up on the track that the plugin belongs to.

Cheers!

[vitalker]

Originally Posted by Synthdev

Thanks Justin, I've sent the binaries, easier to work from imho.

Also, there's another (well known) bug, drag/drop file events are not being passed to the plugin, instead showing up on the track that the plugin belongs to.

Cheers!

What OS? Some plugins don't implement this function correctly, that's why it doesn't work. In most plugins it works okay.

[Justin]

Originally Posted by Synthdev

Thanks Justin, I've sent the binaries, easier to work from imho.

Also, there's another (well known) bug, drag/drop file events are not being passed to the plugin, instead showing up on the track that the plugin belongs to.

Cheers!

hmm haven't seen that email -- maybe send a link instead of attachments, if it was previously an attachment

[Synthdev]

Sorry about the delay.

The plugin format is vst3, platform Windows (10).

Weird re: the email not arriving, it was sent via pcloud, I'll get it to make a link instead.

[Synthdev]

Have the email/binaries come through?

BTW, it's the Wave editor window (the bottom module) that is connected to receive drag drop events, but is not receiving them, instead they turn up as a track event. I've checked to see if I have made any obvious mistakes, but it's just a few overrides which compile, and never get called, so there's not much room for error? That said, this is the VST3 sdk (does it qualify as an sdk? seems more like a ill specified framework to me?) we are talking about, which increases the room for error dramatically 😉

Thanks in advance!

[Justin]

Originally Posted by Synthdev

Have the email/binaries come through?

BTW, it's the Wave editor window (the bottom module) that is connected to receive drag drop events, but is not receiving them, instead they turn up as a track event. I've checked to see if I have made any obvious mistakes, but it's just a few overrides which compile, and never get called, so there's not much room for error? That said, this is the VST3 sdk (does it qualify as an sdk? seems more like a ill specified framework to me?) we are talking about, which increases the room for error dramatically 😉

Thanks in advance!

Got the email with the link. However, both builds work fine here. Is there anything I need to do to get the crash? I don't see any menu items added to the stock menu when I right click parameters...

Re: drag/drop, make sure to set your window WS_EX_ACCEPTFILES.

[Synthdev]

Hi Justin,

To replicate the crash:

1. Add morphic to a track

2. Save as a project

3. Load the project ... and build 6052 will crash (at least it does here, tested it just now. I'll reinstall Reaper and give it another go ...)

The only difference is on the LFG module, there is a "shots" menu in 6052.

Re: WS_EX_ACCEPTFILES, thanks so much, I'll give it a go!

[Justin]

Originally Posted by Synthdev

Hi Justin,

To replicate the crash:

1. Add morphic to a track

2. Save as a project

3. Load the project ... and build 6052 will crash (at least it does here, tested it just now. I'll reinstall Reaper and give it another go ...)

The only difference is on the LFG module, there is a "shots" menu in 6052.

Re: WS_EX_ACCEPTFILES, thanks so much, I'll give it a go!

Hmm, ok duplicated the crash. Have you tried running it in a debugger to see?

Tracing through it, it looks like we call IEditController::initialize(), and in your implementation of that function (or a function almost immediately called by that function), there is a very large (nearly 1MB) amount of stack space allocated, which is causing a stack overflow.

[Synthdev]

Thanks Justin,

That makes a lot of sense.

I did try running it in a debugger, it crashes Reaper itself ... I could never get it to load. Probably another form of the stack problem?

There is a Domain specific language that requires some recursion for hierarchical menus, and it uses std::string, which might be copying a lot of data. A net search reveals I might be able to create a thread with a custom stack size to interpret the DSL, or ...

Alternately, Reaper can be compiled with a larger stack (I normally use 8->64M in my apps, for a modern app the default 1M is probably quite small ... the bigger it is , the fewer problems there will be with third party plugins, until it fails ... silently!).

The crash is happening in the VST3 "SDK" before it gets to my code, so I might be able to override it, or modify the broken "SDK" ... I'll have to delve into the (ghastly) code.

Re: drag drop files, the "SDK" itself appears broken, the standalone player doesn't work, and the window initialization is ... dreadful 🤣

[schwa]

The libc++ std::string only uses 24 bytes of stack space, so either you have something else big on the stack in your recursive function, or the recursive function is getting called more often than it should. It's unlikely building the menus actually requires 1MB of space, on the stack or otherwise.

[Justin]

Post the code to your IEditController::initialize if you want us to take a look. 1MB stack is (or should be) plenty.

There is a Domain specific language that requires some recursion for hierarchical menus, and it uses std::string, which might be copying a lot of data. A net search reveals I might be able to create a thread with a custom stack size to interpret the DSL, or ...

Try it in a debugger, REAPER will not prevent you from debugging it. You will see that it's the stack frame of IEditController::initialize() (or a function that it calls and was inlined) that is large. It isn't an incremental thing with lots of recursion, it's literally almost 1MB of stack use in _one function_. Probably a trivial fix on your end. If you have some huge C++ object getting created on the stack 'type foo;' try changing it to 'type *foo = new type;' etc

[Synthdev]

Thanks all,

I'm going to track down (and if necessary post) the stack issues after a good nights sleep!

[Synthdev]

It works perfectly when the controller initialize function is called from a thread with a larger stack.

Thanks so much everyone!

[Justin]

Originally Posted by Synthdev

It works perfectly when the controller initialize function is called from a thread with a larger stack.

Thanks so much everyone!

I strongly suggest that you figure out why the stack use of that single function is so high and not use the thread hack. It is not high due to recursion, and creating a thread for this purpose could cause other issues for you.

[Synthdev]

Thanks Justin,

it looks like the issue might be in the (on the fly) UnitID allocation to menus/modules. I'm thinking about the easiest way to implement it in tail recursive form, it's always possible, but in this case, a bit tricky, because of a mutual recursion.

So for the moment, I'll live with the thread method. It works, and the thread is destroyed afterwards, and all the code called seems to be thread safe. In theory the compiler can refactor the recursion, but in this case it doesn't seem to be transforming the calls to jumps. Still, my code shouldn't be using that much stack, so there might be issues with the Windows code for hierarchical menus? Puzzling, but at least there is a workaround!

[Justin]

Originally Posted by Synthdev

Thanks Justin,

it looks like the issue might be in the (on the fly) UnitID allocation to menus/modules. I'm thinking about the easiest way to implement it in tail recursive form, it's always possible, but in this case, a bit tricky, because of a mutual recursion.

So for the moment, I'll live with the thread method. It works, and the thread is destroyed afterwards, and all the code called seems to be thread safe. In theory the compiler can refactor the recursion, but in this case it doesn't seem to be transforming the calls to jumps. Still, my code shouldn't be using that much stack, so there might be issues with the Windows code for hierarchical menus? Puzzling, but at least there is a workaround!

again, and I can't make this clear enough -- this issue is not due to recursion, the stack frame of the initialize() function itself is very large.

rather than theorizing about the cause and putting a band-aid on it, get to the root of it! if you can post the source to initialize() (or email it to support at cockos dot com if you want it not to be on the internet), I'd be happy to take a look.

[Synthdev]

Thanks Justin,

There is evidence there is a compiler bug relating to inline functions and scoping.

https://developercommunity.visualstu...cation/1060873

Another possibility is there are a lot of char string variables, and USTRINGs which are potentially created in the same scope, and putting each call in it's own scope might reduce the stack impact. If this doesn't work, introducing a string look up table will reduce the memory used, but even constructing the table will face similar problems! So fingers crossed, the scoping method works.

I'll update Visual C++ to the latest version and try it again, with a few code variations. The recursion seems like a red herring, because the recursion is not very deep.

[Synthdev]

Cheers Justin,

I solved it, though the debugger was little help, as it created even higher stack loads, so wouldn't pass the verifier. But commenting out blocks of code tracked it down, combined with implementing the MS compiler workaround.

There was a lot of copying going on in the recursive unit allocation functions, recoding with const std::string& instead of string fixed it.

In theory, these allocations shouldn't accumulate, but the "fixed" MS compiler bug looks like it isn't actually fixed, so even though each call is in it's own scope, it appears the stack load was increasing in the outer scope.