Cockos Incorporated Forums - View Single Post - ReaScript: New functions Get/SetAllMIDI for fast edits of thousands of events (DONE)

juliansader · 11-24-2016, 09:19 AM

I have updated a few of my MIDI scripts to write directly to the state chunk, and they are now usable even when the take contains hundreds of thousands of events.

However, this approach is not yet perfect: 'deferred' scripts that track mouse movement and update MIDI in realtime are still somewhat jerky, unlike REAPER's smooth native functions.

In the state chunk string, the MIDI data of a take is hidden away between lots of non-MIDI information (and also MIDI data of other takes in the item), and is encoded in a complicated way that requires some awkward -- and slow -- parsing. (Some details of the encoding can be read here and here.) For example, getting the PPQ position of en event requires adding the offsets of all prior events, and getting notation info requires base64 decoding.

The most time-consuming step seems to be that REAPER must process all the non-MIDI information in the state chunk -- as well as all the MIDI data of the non-active takes in the item -- whenever the state chunk with edited MIDI of the active take is uploaded via SetItemStateChunk.

(In my own scripts, for example, Lua can assemble and encode thousands of edited events during each cycle of reaper.defer(function) within a mere hundredth of a second, whereas reaper.SetItemStateChunk takes 0.2 seconds or more, making the movement of the MIDI events choppy and somewhat lagging behind mouse movement. The more takes in an item, the slower reaper.SetItemStateChunk gets.)

I am pretty sure that REAPER uses a more efficient representation of the MIDI data in its own internal processing: for example, arrays or tables with PPQ positions of each event.

It would be great if a new function MIDI_GetAllMIDI(take) could return only the MIDI data of one take, all the useful info that REAPER has already pre-calculated (such as PPQ positions), in whatever straightforward format is easiest for REAPER to encode and decode.

But even more importantly (since this function will be called multiple times, once during each defer cycle), a fast new function MIDI_SetAllMIDI(take) that uploads only a single take's MIDI (again, in whatever format would be easiest for REAPER to parse), so that REAPER does not need to process all the other irrelevant information in the state chunk.

11-24-2016, 09:19 AM
juliansader Human being with feelings Join Date: Jul 2009 Posts: 3,714	I have updated a few of my MIDI scripts to write directly to the state chunk, and they are now usable even when the take contains hundreds of thousands of events. However, this approach is not yet perfect: 'deferred' scripts that track mouse movement and update MIDI in realtime are still somewhat jerky, unlike REAPER's smooth native functions. In the state chunk string, the MIDI data of a take is hidden away between lots of non-MIDI information (and also MIDI data of other takes in the item), and is encoded in a complicated way that requires some awkward -- and slow -- parsing. (Some details of the encoding can be read here and here.) For example, getting the PPQ position of en event requires adding the offsets of all prior events, and getting notation info requires base64 decoding. The most time-consuming step seems to be that REAPER must process all the non-MIDI information in the state chunk -- as well as all the MIDI data of the non-active takes in the item -- whenever the state chunk with edited MIDI of the active take is uploaded via SetItemStateChunk. (In my own scripts, for example, Lua can assemble and encode thousands of edited events during each cycle of reaper.defer(function) within a mere hundredth of a second, whereas reaper.SetItemStateChunk takes 0.2 seconds or more, making the movement of the MIDI events choppy and somewhat lagging behind mouse movement. The more takes in an item, the slower reaper.SetItemStateChunk gets.) I am pretty sure that REAPER uses a more efficient representation of the MIDI data in its own internal processing: for example, arrays or tables with PPQ positions of each event. It would be great if a new function MIDI_GetAllMIDI(take) could return only the MIDI data of one take, all the useful info that REAPER has already pre-calculated (such as PPQ positions), in whatever straightforward format is easiest for REAPER to encode and decode. But even more importantly (since this function will be called multiple times, once during each defer cycle), a fast new function MIDI_SetAllMIDI(take) that uploads only a single take's MIDI (again, in whatever format would be easiest for REAPER to parse), so that REAPER does not need to process all the other irrelevant information in the state chunk. __________________ Scripts for advanced MIDI editing \| LFO Tool for MIDI editor and envelopes Video thumbnail scripts \| ReaScriptAPI extension Last edited by juliansader; 11-24-2016 at 10:23 AM.