Sysex input bug (macOS)

[sockmonkey72]

Hi REAPER-gang, I am struggling with a sysex issue in macOS (MBP 2018, macOS 10.15) REAPER (6.36) and think I've determined the problem, but I'll need Cockos to confirm.

The issue: REAPER ignores some (but not all) sysex data coming from a particular hardware instrument. This is similar (or the same?) as the issue reported by @dupont in this thread: https://forums.cockos.com/showthread.php?t=225311 , but I am using a Yamaha DX7IID. REAPER sees/records all other MIDI data, including bulk sysex, from this instrument, but ignores sysex parameter changes (which are 7-byte messages). REAPER can also handle sysex parameter changes coming from other instruments in a similar fashion (like from a Yamaha SY99). Other DAWs (Logic, Digital Performer, Max) can accept/record the sysex, and MIDI Monitor.app indicates that the messages are arriving correctly.

Troubleshooting: swapped out cables & interface (no change), filtered out Active Sensing messages (no change). Using software like Max, I was able to rewrite the sysex messages (basically just pass them along without modification) -- and that worked.

Advanced Troubleshooting: I analyzed the CoreMIDI packet data arriving from the instrument (using my own software written in Obj-C) and discovered that the messages arriving from CoreMIDI are packetized like this for the 7-byte message 0xF0 0x43 0x10 0x00 0x6D 0x1C 0xF7:

Packet 1: 0xF0 0x43 0x10
Packet 2: 0x00 0x6D 0x1C
Packet 3: 0xF7

In this format, REAPER simply ignores the incoming sysex. Whether that's because it can't handle sysex broken up across multiple packets, or because the single byte 0xF7 at the end is confusing the parser, or something else entirely, I cannot say. But Cockos probably can.

The rewritten packet generated by Max puts the entire message into a single packet of 7 bytes (and REAPER immediately recognizes/handles it).

Anyway, I hope that this information serves to help the devs tighten up REAPER's MIDI/sysex implementation. Browsing through the forum, I've seen this class of gremlin come up several times over the years, so maybe it can finally be stomped...?

Happy to talk through this some more or provide additional troubleshooting or logs as required.

Thanks, and thanks for letting me onto the forum!

[schwa]

Ah, good bug report, thanks. We may be able to improve this.

[sockmonkey72]

Originally Posted by schwa

Ah, good bug report, thanks. We may be able to improve this.

Great! I just did a test on the 11-byte parameter changes sent by the Yamaha SY99 to see how CoreMIDI is handling that data. It looks like this:

2021-09-13 14:08:34.633985+0200 inPacket 0, l=3, d=0xF0 0x43 0x10
2021-09-13 14:08:34.635030+0200 inPacket 0, l=3, d=0x34 0x03 0x00
2021-09-13 14:08:34.635974+0200 inPacket 0, l=3, d=0x00 0x01 0x00
2021-09-13 14:08:34.636566+0200 inPacket 0, l=2, d=0x03 0xF7

which would suggest that the problem is the single 0xF7 at the end.

Hope that helps narrow it down.

[schwa]

If somebody had told me in 1985 that I would be dealing with DX7 compatibility in 2021...

[sockmonkey72]

Originally Posted by schwa

If somebody had told me in 1985 that I would be dealing with DX7 compatibility in 2021...

:-) lol

[schwa]

This should be fixed in the current +dev build, if you'd care to test it out.

https://forum.cockos.com/forumdisplay.php?f=37

[beingmf]

Does this relate to Windows as well?

[sockmonkey72]

Originally Posted by schwa

This should be fixed in the current +dev build, if you'd care to test it out.

https://forum.cockos.com/forumdisplay.php?f=37

Indeed, it is! Thanks so much.

[schwa]

Originally Posted by beingmf

Does this relate to Windows as well?

Yes, though there is probably not much midi hardware out there that would send sysex broken up that way.

[sockmonkey72]

Originally Posted by schwa

Yes, though there is probably not much midi hardware out there that would send sysex broken up that way.

Looks like the 3-byte packetization is a USB-MIDI thing: https://www.usb.org/sites/default/files/midi10.pdf, see Section 4.

I wonder aloud if the Windows class drivers do more work on incoming data, though (buffering sysex into a complete message before passing it along, for instance), which results in Windows users not experiencing this issue? The Nord Lead 2 issue described by @dumont was only present on macOS machines. If I get a chance, I'll try to determine what the Windows class driver is providing, just out of curiosity.

[beingmf]

Originally Posted by schwa

Yes, though there is probably not much midi hardware out there that would send sysex broken up that way.

If it exists, you can bet I have it