Old 03-23-2019, 06:51 AM   #1
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default MIDI processing script runs very slowly, how to improve it?

MIDI processing script runs very slowly, how to improve it?
Shortly after I learned LUA, I wrote a script to compare the values of CC before and after, and delete the latter if they are the same. This script can work normally, but when the amount of data is large, it runs very slowly, and every processing is more than one minute. This should not be a normal situation. How can I improve it?

The code is as follows:

local editor=reaper.MIDIEditor_GetActive()

local take=reaper.MIDIEditor_GetTake(editor)

local ccidx=0

haha = reaper.MIDIEditor_OnCommand(editor,40214)
repeat

retval,selected,muted,ppqpos, chanmsg, chan, num, val = reaper.MIDI_GetCC(take, ccidx)

if (chanmsg==176 and num==11) then
reaper.MIDI_SetCC(take, ccidx, true, false, ppqpos, chanmsg, chan, num, val, noSortIn)
end

ccidx=ccidx+1

until retval==false
-- select CC11

local ccidx=-1

local temp=''

repeat
local integer = reaper.MIDI_EnumSelCC(take, ccidx)
local retval,selected,muted,ppqpos, chanmsg, chan, num, val = reaper.MIDI_GetCC(take, integer)
if (temp ~= val) then
reaper.MIDI_SetCC(take, integer,false, false, ppqpos, chanmsg, chan, num, val, noSortIn)
end
temp=val
ccidx = integer
until integer==-1
haha = reaper.MIDIEditor_OnCommand(editor,40667)
--delect
dangguidan is offline   Reply With Quote
Old 03-23-2019, 08:20 AM   #2
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 2,832
Default

Set noSortIn to true and call MIDI_Sort when you're done.

Code:
reaper.MIDI_SetCC(take, ccidx, true, false, ppqpos, chanmsg, chan, num, val, true)

-- rest of the script...

reaper.MIDI_Sort(take)
cfillion is online now   Reply With Quote
Old 03-23-2019, 04:51 PM   #3
Lokasenna
Human being with feelings
 
Lokasenna's Avatar
 
Join Date: Sep 2008
Location: Calgary, AB, Canada
Posts: 6,356
Default

FYI, when posting code samples you can use:
Code:
[code]
  ...code here...
[/code]
to keep things readable and preserve any indenting.
Lokasenna is offline   Reply With Quote
Old 03-24-2019, 10:07 PM   #4
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

Quote:
Originally Posted by cfillion View Post
Set noSortIn to true and call MIDI_Sort when you're done.

Code:
reaper.MIDI_SetCC(take, ccidx, true, false, ppqpos, chanmsg, chan, num, val, true)

-- rest of the script...

reaper.MIDI_Sort(take)
Thank you very much for your reply. What is the meaning of noSortIn? It seems that there is no detailed explanation on the official website. And is MIDI_Sort behind all the actions? What does this function do?
dangguidan is offline   Reply With Quote
Old 03-24-2019, 10:30 PM   #5
Lokasenna
Human being with feelings
 
Lokasenna's Avatar
 
Join Date: Sep 2008
Location: Calgary, AB, Canada
Posts: 6,356
Default

By default, Reaper internally sorts all of the notes in the MIDI item each time you change one of them - this is where you're hitting a performance issue.

The API documentation (see Reaper's Help menu) has a note on most of the MIDI functions:

Set noSort if setting multiple events, then call MIDI_Sort when done..
Lokasenna is offline   Reply With Quote
Old 03-24-2019, 11:09 PM   #6
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 8,057
Default

I suppose, workalke ReaScripts in EEL run a lot faster than in LUA. For more complex stuff, Python scripts will be able to take advantage from the many embedded native Library functions Python comes with. LUA seems perfect for rather simple stuff, though. I understand that these considerations is why Reaper decently supports all three scripting languages.

-Michael
mschnell is offline   Reply With Quote
Old 03-25-2019, 06:11 AM   #7
Lokasenna
Human being with feelings
 
Lokasenna's Avatar
 
Join Date: Sep 2008
Location: Calgary, AB, Canada
Posts: 6,356
Default

How is this a Lua issue? Reaper is the slow part.
Lokasenna is offline   Reply With Quote
Old 03-25-2019, 08:34 AM   #8
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 8,057
Default

Of course if indeed most of the time passes within Reaper API calls, and there are no long winding loops in the script code itself, the language does not matter

-Michael
mschnell is offline   Reply With Quote
Old 03-25-2019, 10:32 PM   #9
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

Quote:
Originally Posted by Lokasenna View Post
By default, Reaper internally sorts all of the notes in the MIDI item each time you change one of them - this is where you're hitting a performance issue.

The API documentation (see Reaper's Help menu) has a note on most of the MIDI functions:

Set noSort if setting multiple events, then call MIDI_Sort when done..
I tried to modify the code in this way, but the result didn't seem obvious.
dangguidan is offline   Reply With Quote
Old 03-25-2019, 10:38 PM   #10
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

In other scripts, MIDI_InsertCC runs very slowly when the number of processing is large. I have written CAL with similar functions in Cakewalk, and I have never encountered such a slow speed. Shouldn't this slow speed be Reaper's style?
dangguidan is offline   Reply With Quote
Old 03-26-2019, 04:08 AM   #11
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 2,804
Default

For fast editing of lots of MIDI events, you should use MIDI_GetAllEvts and MIDI_SetAllEvts.

The other functions such as MIDI_SetCC / MIDI_GetNote / etc are only intended for editing one or a few events at a time, for example if you want to trim the note under the mouse.

In order to use Get/SetAllEvts, you need to be familiar with Lua's string functions, as well as how REAPER stores MIDI events. There are several threads in this subforum that can help you with this, including Select notes before / after edit cursor.

The links in my signature below will take you to threads with GIFs that demonstrate the speed of Get/SetAllEvts.
juliansader is offline   Reply With Quote
Old 03-26-2019, 07:28 PM   #12
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

Quote:
Originally Posted by juliansader View Post
For fast editing of lots of MIDI events, you should use MIDI_GetAllEvts and MIDI_SetAllEvts.

The other functions such as MIDI_SetCC / MIDI_GetNote / etc are only intended for editing one or a few events at a time, for example if you want to trim the note under the mouse.

In order to use Get/SetAllEvts, you need to be familiar with Lua's string functions, as well as how REAPER stores MIDI events. There are several threads in this subforum that can help you with this, including Select notes before / after edit cursor.

The links in my signature below will take you to threads with GIFs that demonstrate the speed of Get/SetAllEvts.
Thank you for your advice. Get / SetAllEvts looks very powerful, but I've only been learning Lua for a month without a programming foundation, which is too difficult for me. Is there any other simple and feasible way?
dangguidan is offline   Reply With Quote
Old 03-26-2019, 09:11 PM   #13
mpl
Human being with feelings
 
mpl's Avatar
 
Join Date: Oct 2013
Location: Moscow, Russia
Posts: 2,542
Default

Quote:
Originally Posted by dangguidan View Post
Is there any other simple and feasible way?
No. Your script do unnesessary job since it loops events you need to modify twice. You can do something like:
- loop event from last to first
--- if selected and is CC11 then
------ remove event
--- end if
- end loop

You can modify flags of CC11 MIDI events, to make it only selected:
Code:
  for key in pairs(reaper) do _G[key]=reaper[key]  end   
  -------------------------------------------------------------------------
  function HandleMIDIData(take, exclude_msg_type)
    local tableEvents = {}
    local t = 0 -- Table key
    local gotAllOK, MIDIstring = MIDI_GetAllEvts(take, "")
    local MIDIlen = MIDIstring:len()
    local stringPos = 1 -- Position inside MIDIstring while parsing
    local offset, flags, msg
    local s_pack, s_unpack = string.pack, string.unpack            
    while stringPos < MIDIlen-12 do -- -12 to exclude final All-Notes-Off message
      offset, flags, msg, stringPos = s_unpack ("i4Bs4", MIDIstring, stringPos)
      if msg:len() > 1 and msg:byte(1)>>4 == 0xB and msg:byte(2) == 11 then
        if flags&1~=1 then flags = flags + 1 end
       else
        if flags&1==1 then flags = flags - 1 end
      end
      t = t + 1
      tableEvents[t] = s_pack("i4Bs4", offset, flags, msg)
    end
                
    MIDI_SetAllEvts(take, table.concat(tableEvents) .. MIDIstring:sub(-12))
    --MIDI_Sort(take)    
  end
  
  -------------------------------------------------------------------------  
  function main()
    local midieditor = MIDIEditor_GetActive()
    if not midieditor then return end
    local take =  MIDIEditor_GetTake( midieditor )
    if not take then return end
    Undo_BeginBlock()  
    HandleMIDIData(take, exclude_msg_type)
    Undo_EndBlock('Select CC11', 1)  
  end  
  
  main()
Or you can directly "clear" message of CC11 in one loop (not really a good way, you probably should not add empty message and take care just about next message offset):
Code:
  for key in pairs(reaper) do _G[key]=reaper[key]  end   
  -------------------------------------------------------------------------
  function HandleMIDIData(take, exclude_msg_type)
    local tableEvents = {}
    local t = 0 -- Table key
    local gotAllOK, MIDIstring = MIDI_GetAllEvts(take, "")
    local MIDIlen = MIDIstring:len()
    local stringPos = 1 -- Position inside MIDIstring while parsing
    local offset, flags, msg
    local s_pack, s_unpack = string.pack, string.unpack
    while stringPos < MIDIlen-12 do -- -12 to exclude final All-Notes-Off message
      offset, flags, msg, stringPos = s_unpack("i4Bs4", MIDIstring, stringPos)
      if msg:len() > 1 and msg:byte(1)>>4 == 0xB and msg:byte(2) == 11 then
        msg = ''
      end
      t = t + 1
      tableEvents[t] = s_pack("i4Bs4", offset, flags, msg)
    end
                
    MIDI_SetAllEvts(take, table.concat(tableEvents) .. MIDIstring:sub(-12))
    --MIDI_Sort(take)    
  end
  
  -------------------------------------------------------------------------  
  function main()
    local midieditor = MIDIEditor_GetActive()
    if not midieditor then return end
    local take =  MIDIEditor_GetTake( midieditor )
    if not take then return end
    Undo_BeginBlock()  
    HandleMIDIData(take, exclude_msg_type)
    Undo_EndBlock('Remove CC11', 1)  
  end  
  
  main()
Address any "thanks" for this stuff to JulianSader, it is mostly his code, slightly modified.
__________________
SoundCloud | MPL Scripts discussion | ReaPack | Donate

Last edited by mpl; 03-26-2019 at 09:21 PM.
mpl is offline   Reply With Quote
Old 06-27-2019, 10:30 PM   #14
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

Thank you for your patience and help!
I'm trying to use MIDI_GetEvt and MIDI_GetAllEvts, but how do I resolve the string msg, string buf?
dangguidan is offline   Reply With Quote
Old 06-28-2019, 05:50 AM   #15
lb0
Human being with feelings
 
Join Date: Apr 2014
Posts: 3,187
Default

Quote:
Originally Posted by dangguidan View Post
Thank you for your patience and help!
I'm trying to use MIDI_GetEvt and MIDI_GetAllEvts, but how do I resolve the string msg, string buf?
Not got an answer for this question - but with your original code - did you try encompassing the processing loop with:

reaper.MIDI_DisableSort(MediaItem_Take take)

...

reaper.MIDI_Sort(MediaItem_Take take)

I found the noSort parameter in the Set functions not as efficient - didn't make a huge improvement. I swapped to the (newish) MIDI_DisableSort API - and it became lightning quick. I still set the noSort parameter to true though.
__________________
Projects - Reascripts - Lua:
LBX Stripper | LBX Chaos Engine | LBX Floating FX Positioner | LBX SRD Smart Knobs
Donate via Paypal
lb0 is online now   Reply With Quote
Old 06-28-2019, 06:49 PM   #16
dangguidan
Human being with feelings
 
Join Date: Jan 2019
Posts: 106
Default

Thank you for your suggestion. I have used the new MIDI_DisableSort API. The problem of running speed has been greatly improved. When I wrote it, I found that some functions seem to need to set noSort to False faster.
Now I want to write some other scripts. I may use MIDI_GetEvt and MIDI_GetAllEvts, but I don't understand string msg, string buf at all, so I'll consult you in this post by the way.
dangguidan is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:12 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.