Old 05-01-2018, 04:30 AM   #1
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default MIDI scripting

Hello Reapers!

I was trying to dive into ReaScripting recently, doing some MIDI stuff. I found ExtremRaym's convenient ReaScript reference... But the rather sparse documentation makes it quite hard to understand what the functions do, sometimes. Especially what the input arguments are supposed to be...

I would like to, generally, modify the MIDI contents of one or more items/tracks. I thus played around with some of the functions and so far managed to change existing MIDI events.

For instance:
Code:
if reaper.CountSelectedMediaItmems(0) >= 1 then
   take = reaper.MIDIEditor_GetTake(reaper.MIDIEditor_GetActive())

   if take ~= nil then
      -- note stuff
      _,_,_,start,stop,chan,pitch,vel = reaper.MIDI_GetNote(take,0)
      -- display data, for instance the note end in minutes: stop/PPQ/BPM etc.
      
      -- cc stuff
      _,_,_,pos,_,chan,msg2,msg3 = reaper.MIDI_GetCC(take,0)
      -- display data, for instance the CC value in msg3 etc.
      reaper.MIDI_SetCC(take,0,nil,nil,nil,nil,nil,nil,66,nil) -- to set, e.g., the cc value to 66 etc
   end
end
But I don't know, for instance, how to add a new MIDI event.

Furthermore, could someone please explain to me how MIDI data is handled/accessed in ReaScripts in general? Just for giving me some ignition aid. (Or is there a basic tutorial on this topic - I couldn't manage to find one.)

How do, for example, GetNote(...) or GetCC(...) return the MIDI data? The returned data doesn't seem to be sorted? How to get all CC data of one specific CC, say #2?

I would be very glad for some information and insight!

Thanks, Lukas
LuCsa is offline   Reply With Quote
Old 05-01-2018, 05:50 AM   #2
Jason Lyon
Human being with feelings
 
Join Date: May 2016
Posts: 720
Default

In my experience, the challenge of working with MIDI via scripts is understanding MIDI's native format, rather than the actual manipulation of the data.

The attached might give you some basic ideas

Code:
--[[
 * ReaScript in Lua: Create two new tracks and split all MIDI takes from original into RH and LH on them at requested note. 
 * Original track is retained for safety but muted.
 * New RH and LH tracks are clones of original so inherit all its properties, including routing and plugin instances.
 * Instructions: Select track and execute script
 * Author: Jason Lyon
 * Uses code by XRaym and FnA
 * REAPER: 5.70
 * Extensions: None
 * Version: 1.3 2018-01-14
--]]

-- declare vars
local r = reaper
local trackid, numofitems, takeid, notcancelled, input, splitpoint
local LHtrackid, LHtakeid, RHtrackid, RHtakeid
local retval, trackname, itemcounter, workingitemid, split, MIDIstring, MIDIlen, allevents, stringPos, offset, flags, msg, mb1, pitch  

r.PreventUIRefresh(1)

if r.CountSelectedTracks(0) == 1 then -- error trap 1: continue if only one track is selected

-- info gathering from original track
trackid = r.GetSelectedTrack2(0, 0, 1)  
numofitems = r.GetTrackNumMediaItems(trackid)  

if numofitems > 0 then -- error trap 2: continue if items exist
 
takeid = r.GetActiveTake(r.GetTrackMediaItem(trackid, 0))  

if r.TakeIsMIDI(takeid) then -- error trap 3: continue if first item is MIDI

 -- get user input   
 notcancelled, input = r.GetUserInputs("MIDI Piano Splitter", 1, "Enter Split Note (C4 or 60)", "C4")
 
 -- function: convert possible text input to MIDI note
 local inputlen = string.len(input), notename, octave
      
 if inputlen == 2 and string.find(input, "%a%d") then -- input is of form eg "W2"
  notename = string.sub(input, 1, 1)
  octave = tonumber(string.sub(input, 2, 2)) end
 if inputlen == 3 and string.find(input, "%a[#]%d") then -- input is of form eg "j#8"
  notename = string.sub(input, 1, 2)
  octave = tonumber(string.sub(input, 3, 3)) end
 if notename then -- function error trap 1: continue if input is confirmed as above
 
 -- declare vars
 local octavetable, i, noteval
 
 notename = string.upper(notename)
 octavetable = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"}
 for i = 1, 12 do
  if notename == octavetable[i] then noteval = i - 1 end
 end
  
 if noteval then -- function error trap 2: continue if note name matches a table value
 input = (octave+1)*12 + noteval 
 
 end end --- close function error traps
 
if notcancelled and tonumber(input) then splitpoint = tonumber(input) else splitpoint = 0 end

if splitpoint > 0 and splitpoint < 127 then -- error trap 4: continue if user input valid

r.Undo_BeginBlock()
    
  -- create LH track
  r.Main_OnCommand(40062, 0) 
  LHtrackid = r.GetSelectedTrack(0, 0)
  LHtakeid = r.GetActiveTake(r.GetTrackMediaItem(LHtrackid, 0))
  retval, trackname = r.GetSetMediaTrackInfo_String(trackid, "P_NAME", "", false)
  r.GetSetMediaTrackInfo_String(LHtrackid, "P_NAME", trackname .. " — LH", true)  
    
  -- start loop to run through all MIDI items
  itemcounter = 0
  while (itemcounter < numofitems) do
   workingitemid = r.GetActiveTake(r.GetTrackMediaItem(LHtrackid, itemcounter))
    
   -- LH processing, delete notes over chosen point
   split = splitpoint - 1      
   retval, MIDIstring = r.MIDI_GetAllEvts(workingitemid, "")
   MIDIlen = MIDIstring:len()  
   allevents = {}
            
   stringPos = 1 -- position in MIDIstring while parsing through events 
   while stringPos < MIDIlen do
    offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos) 
    if msg:len() == 3 then mb1 = msg:byte(1)>>4
     if mb1 == 9 or mb1 == 8 then pitch = msg:byte(2)
      if pitch > split then msg = "" end
     end
    end
    table.insert(allevents, string.pack("i4Bs4", offset, flags, msg))
   end
  r.MIDI_SetAllEvts(workingitemid, table.concat(allevents))
   
  itemcounter = itemcounter + 1
  end
      
  -- relocate to orig
  r.SetOnlyTrackSelected(trackid)
    
  -- create RH track
  r.Main_OnCommand(40062, 0) 
  RHtrackid = r.GetSelectedTrack(0, 0)
  RHtakeid = r.GetActiveTake(r.GetTrackMediaItem(RHtrackid, 0))
  retval, trackname = r.GetSetMediaTrackInfo_String(trackid, "P_NAME", "", false)
  r.GetSetMediaTrackInfo_String(RHtrackid, "P_NAME", trackname .. " — RH", true)  
    
  -- start loop to run through all MIDI items
  itemcounter = 0
  while (itemcounter < numofitems) do
   workingitemid = r.GetActiveTake(r.GetTrackMediaItem(RHtrackid, itemcounter))
    
   -- RH processing, delete notes below chosen point
   split = splitpoint   
   retval, MIDIstring = r.MIDI_GetAllEvts(workingitemid, "")
   MIDIlen = MIDIstring:len()  
   allevents = {}
            
   stringPos = 1 -- position in MIDIstring while parsing through events 
   while stringPos < MIDIlen do
    offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos) 
    if msg:len() == 3 then mb1 = msg:byte(1)>>4
     if mb1 == 9 or mb1 == 8 then -- note-on/off MIDI event type
      pitch = msg:byte(2)
      if pitch < split then msg = "" end
     end
    end
    table.insert(allevents, string.pack("i4Bs4", offset, flags, msg))
   end
  r.MIDI_SetAllEvts(workingitemid, table.concat(allevents))
    
  itemcounter = itemcounter + 1
  end
    
  -- relocate to orig and mute
  r.SetOnlyTrackSelected(trackid)
  r.SetMediaTrackInfo_Value(trackid, "B_MUTE", 1)
    
r.Undo_EndBlock("Split MIDI track into RH and LH tracks", 0)    
end end end end -- close error traps

r.PreventUIRefresh(-1)
Jason Lyon is offline   Reply With Quote
Old 05-01-2018, 02:02 PM   #3
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

Quote:
Originally Posted by LuCsa View Post
But I don't know, for instance, how to add a new MIDI event.
The "insert" functions can be used: MIDI_InsertCC, MIDI_InsertNote, etc


Quote:
Furthermore, could someone please explain to me how MIDI data is handled/accessed in ReaScripts in general? Just for giving me some ignition aid. (Or is there a basic tutorial on this topic - I couldn't manage to find one.)
I am not aware of a tutorial, but it may be helpful to read through some scripts in ReaPack. Some of them have extensive comments. This thread may also be useful if you use the functions MIDI_Get/SetAllEvts: Select events in specific CC lane.


Quote:
Originally Posted by LuCsa View Post
How do, for example, GetNote(...) or GetCC(...) return the MIDI data? The returned data doesn't seem to be sorted? How to get all CC data of one specific CC, say #2?
The MIDI data is sorted and indexed by time (except when a script has temporarily shuffled the data, using the nosort option). To extract all CC2 events, you have to scan through all of them.
juliansader is offline   Reply With Quote
Old 05-03-2018, 01:40 PM   #4
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default

Hello!

Quote:
Originally Posted by Jason Lyon View Post
In my experience, the challenge of working with MIDI via scripts is understanding MIDI's native format, rather than the actual manipulation of the data.

The attached might give you some basic ideas

Code:
--[[
 * ReaScript in Lua: Create two new tracks and split all MIDI takes from original into RH and LH on them at requested note. 
 * Original track is retained for safety but muted.
 * New RH and LH tracks are clones of original so inherit all its properties, including routing and plugin instances.
 * Instructions: Select track and execute script
 * Author: Jason Lyon
 * Uses code by XRaym and FnA
 * REAPER: 5.70
 * Extensions: None
 * Version: 1.3 2018-01-14
--]]

-- declare vars
local r = reaper
local trackid, numofitems, takeid, notcancelled, input, splitpoint
local LHtrackid, LHtakeid, RHtrackid, RHtakeid
local retval, trackname, itemcounter, workingitemid, split, MIDIstring, MIDIlen, allevents, stringPos, offset, flags, msg, mb1, pitch  

r.PreventUIRefresh(1)

if r.CountSelectedTracks(0) == 1 then -- error trap 1: continue if only one track is selected

-- info gathering from original track
trackid = r.GetSelectedTrack2(0, 0, 1)  
numofitems = r.GetTrackNumMediaItems(trackid)  

if numofitems > 0 then -- error trap 2: continue if items exist
 
takeid = r.GetActiveTake(r.GetTrackMediaItem(trackid, 0))  

if r.TakeIsMIDI(takeid) then -- error trap 3: continue if first item is MIDI

 -- get user input   
 notcancelled, input = r.GetUserInputs("MIDI Piano Splitter", 1, "Enter Split Note (C4 or 60)", "C4")
 
 -- function: convert possible text input to MIDI note
 local inputlen = string.len(input), notename, octave
      
 if inputlen == 2 and string.find(input, "%a%d") then -- input is of form eg "W2"
  notename = string.sub(input, 1, 1)
  octave = tonumber(string.sub(input, 2, 2)) end
 if inputlen == 3 and string.find(input, "%a[#]%d") then -- input is of form eg "j#8"
  notename = string.sub(input, 1, 2)
  octave = tonumber(string.sub(input, 3, 3)) end
 if notename then -- function error trap 1: continue if input is confirmed as above
 
 -- declare vars
 local octavetable, i, noteval
 
 notename = string.upper(notename)
 octavetable = {"C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"}
 for i = 1, 12 do
  if notename == octavetable[i] then noteval = i - 1 end
 end
  
 if noteval then -- function error trap 2: continue if note name matches a table value
 input = (octave+1)*12 + noteval 
 
 end end --- close function error traps
 
if notcancelled and tonumber(input) then splitpoint = tonumber(input) else splitpoint = 0 end

if splitpoint > 0 and splitpoint < 127 then -- error trap 4: continue if user input valid

r.Undo_BeginBlock()
    
  -- create LH track
  r.Main_OnCommand(40062, 0) 
  LHtrackid = r.GetSelectedTrack(0, 0)
  LHtakeid = r.GetActiveTake(r.GetTrackMediaItem(LHtrackid, 0))
  retval, trackname = r.GetSetMediaTrackInfo_String(trackid, "P_NAME", "", false)
  r.GetSetMediaTrackInfo_String(LHtrackid, "P_NAME", trackname .. " — LH", true)  
    
  -- start loop to run through all MIDI items
  itemcounter = 0
  while (itemcounter < numofitems) do
   workingitemid = r.GetActiveTake(r.GetTrackMediaItem(LHtrackid, itemcounter))
    
   -- LH processing, delete notes over chosen point
   split = splitpoint - 1      
   retval, MIDIstring = r.MIDI_GetAllEvts(workingitemid, "")
   MIDIlen = MIDIstring:len()  
   allevents = {}
            
   stringPos = 1 -- position in MIDIstring while parsing through events 
   while stringPos < MIDIlen do
    offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos) 
    if msg:len() == 3 then mb1 = msg:byte(1)>>4
     if mb1 == 9 or mb1 == 8 then pitch = msg:byte(2)
      if pitch > split then msg = "" end
     end
    end
    table.insert(allevents, string.pack("i4Bs4", offset, flags, msg))
   end
  r.MIDI_SetAllEvts(workingitemid, table.concat(allevents))
   
  itemcounter = itemcounter + 1
  end
      
  -- relocate to orig
  r.SetOnlyTrackSelected(trackid)
    
  -- create RH track
  r.Main_OnCommand(40062, 0) 
  RHtrackid = r.GetSelectedTrack(0, 0)
  RHtakeid = r.GetActiveTake(r.GetTrackMediaItem(RHtrackid, 0))
  retval, trackname = r.GetSetMediaTrackInfo_String(trackid, "P_NAME", "", false)
  r.GetSetMediaTrackInfo_String(RHtrackid, "P_NAME", trackname .. " — RH", true)  
    
  -- start loop to run through all MIDI items
  itemcounter = 0
  while (itemcounter < numofitems) do
   workingitemid = r.GetActiveTake(r.GetTrackMediaItem(RHtrackid, itemcounter))
    
   -- RH processing, delete notes below chosen point
   split = splitpoint   
   retval, MIDIstring = r.MIDI_GetAllEvts(workingitemid, "")
   MIDIlen = MIDIstring:len()  
   allevents = {}
            
   stringPos = 1 -- position in MIDIstring while parsing through events 
   while stringPos < MIDIlen do
    offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos) 
    if msg:len() == 3 then mb1 = msg:byte(1)>>4
     if mb1 == 9 or mb1 == 8 then -- note-on/off MIDI event type
      pitch = msg:byte(2)
      if pitch < split then msg = "" end
     end
    end
    table.insert(allevents, string.pack("i4Bs4", offset, flags, msg))
   end
  r.MIDI_SetAllEvts(workingitemid, table.concat(allevents))
    
  itemcounter = itemcounter + 1
  end
    
  -- relocate to orig and mute
  r.SetOnlyTrackSelected(trackid)
  r.SetMediaTrackInfo_Value(trackid, "B_MUTE", 1)
    
r.Undo_EndBlock("Split MIDI track into RH and LH tracks", 0)    
end end end end -- close error traps

r.PreventUIRefresh(-1)
I have some very basic knowledge about the MIDI format, but certainly I'll need to check that out. Thanks for the ample code example. I understand most of the things, but naturally not the MIDI part. But before asking further questions, I will need to dig some more into Lua. Thanks for the input, though. I might get back to that example.


Quote:
Originally Posted by juliansader View Post
The "insert" functions can be used: MIDI_InsertCC, MIDI_InsertNote, etc

I am not aware of a tutorial, but it may be helpful to read through some scripts in ReaPack. Some of them have extensive comments. This thread may also be useful if you use the functions MIDI_Get/SetAllEvts: Select events in specific CC lane.

The MIDI data is sorted and indexed by time (except when a script has temporarily shuffled the data, using the nosort option). To extract all CC2 events, you have to scan through all of them.
These are some very useful hints, thanks a lot.
LuCsa is offline   Reply With Quote
Old 05-09-2018, 01:29 PM   #5
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default

Hey,

I'm back with another related question...

Quote:
Originally Posted by juliansader View Post
In this thread you stated that ReaScripts can not be used with takes in which MIDI notes overlap.

1) Why is this?
2) Are we talking about overlapping notes of one and the same pitch or two (or more) different pitches?
3) Why didn't you use the approach you pictured in the link (reading bytes etc) in this script but went for MIDI_SetNote() etc.?

Thanks a lot.
Lukas
LuCsa is offline   Reply With Quote
Old 05-09-2018, 01:53 PM   #6
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

Quote:
Originally Posted by LuCsa View Post
In this thread you stated that ReaScripts can not be used with takes in which MIDI notes overlap.

1) Why is this?
The way REAPER stores the data in MIDI items closely follows the standard specifications for MIDI files and the MIDI link communications protocol. In fact, you can use a plain .mid file as source for your MIDI items. (Item processing -> Convert active take MIDI to .mid file reference.)

In this protocol, notes are represented by two separate events (i.e. short sequences of bytes in a long stream): a note-on and a later note-off, each with pitch and channel information. In the case of overlapping notes, you get:

Note-on ... note-on ... note-off ... note-off.

Oops, which note-off goes with which note-on? The MIDI protocol makes no provision for pairing a note-on with a note-off, except "first come first served".

The MIDI editor can temporarily remember the correct pairing while you are working, but scripts do not have access to this temporary memory (and strangely, neither do REAPER's own MIDI functions such as Quantize or Humanize). The memory also gets lost during glueing or other actions.


Quote:
2) Are we talking about overlapping notes of one and the same pitch or two (or more) different pitches?
On the same pitch and channel.


Quote:
3) Why didn't you use the approach you pictured in the link (reading bytes etc) in this script but went for MIDI_SetNote() etc.?
The MIDI_Get/SetAllEvts functions did not yet exist at that time! MIDI_Get/SetAllEvts were added to the API because scripters such as me needed a faster way to edit MIDI en masse, rather than note-by-note.

The older MIDI_SetNote functions are easier to use for simple edits of a few notes. For more complicated edits of thousands of notes, MIDI_Get/SetAllEvts are much faster.

Last edited by juliansader; 05-09-2018 at 02:12 PM.
juliansader is offline   Reply With Quote
Old 05-10-2018, 04:32 AM   #7
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default

Thank you, thank you, thank you. Your answers are most insightful!
LuCsa is offline   Reply With Quote
Old 07-22-2018, 01:41 AM   #8
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default

Dear juliansander, I'm back with a couple of questions, after studying your "select all events of given CC" example. I just copied it below:

Code:
--[[
ReaScript name:  js_Select all MIDI events in specified lanes.lua
Version: 1.02
Author: juliansader
Website: http://forum.cockos.com/showthread.php?t=176878
REAPER: v5.30 or later
]]

--[[
  Changelog:
  * v1.0 (2017-01-09)
    + Initial Release
  * v1.01 (2017-01-09)
    + Changed it a little to allow selection from multiple lanes at the same time.
  * v1.02 (2017-01-10)
    + Exclude the final All-Notes-Off from analysis.
]]

---------------------------------------
-- USER AREA
-- Settings that the user can customize

-- Specify target lane types
local laneIsCC7BIT    = true
local laneIsCC14BIT   = false
local laneIsPITCH     = false
local laneIsCHPRESS   = false
local laneIsNOTES     = false
local laneIsSYSEX     = false
local laneIsTEXT      = false
local laneIsPROGRAM   = false
local laneIsBANKPROG  = false

-- In the case of 7-bit and 14-bit CCs, also specify the CC number
-- The 14-bit CC number is the MSB number + 256.  For example, Modwheel 14-bit is 1+256 = 257.
local targetCCnumber = 1

-- Add to selection, or replace selection?
local replace_selection = true

-- End of USER AREA

------------------------------------------------------------
------------------------------------------------------------
-- Check whether the required version of REAPER is available
if not reaper.APIExists("MIDI_GetAllEvts") then
    reaper.ShowMessageBox("This script requires REAPER v5.30 or higher.", "ERROR", 0)
    return(false) 
end

-- Get active take and item
editor = reaper.MIDIEditor_GetActive()
if editor == nil then 
    reaper.ShowMessageBox("No active MIDI editor found.", "ERROR", 0)
    return(false)
end
take = reaper.MIDIEditor_GetTake(editor)
if not reaper.ValidatePtr(take, "MediaItem_Take*") then 
    reaper.ShowMessageBox("Could not find an active take in the MIDI editor.", "ERROR", 0)
    return(false)
end
item = reaper.GetMediaItemTake_Item(take)
if not reaper.ValidatePtr(item, "MediaItem*") then 
    reaper.ShowMessageBox("Could not determine the item to which the active take belongs.", "ERROR", 0)
    return(false)
end

-- To check that there were no inadvertent shifts in event positions, the position of the final All-Notes-Off 
--    will be stored and checked at the end of the script.
sourceLengthTicks = reaper.BR_GetMidiSourceLenPPQ(take)

-- Get all the MIDI data in active take in bulk
local gotAllOK, MIDIstring = reaper.MIDI_GetAllEvts(take, "")

if not gotAllOK then

    reaper.ShowMessageBox("MIDI_GetAllEvts could not load the raw MIDI data.", "ERROR", 0)
    return false 

else

    local MIDIlen = MIDIstring:len()-12 -- Exclude final All-Notes-Off from being selected
    local tableEvents = {}
    local t = 0 -- Index in table. Curiously, myTable[t] = X is much faster than table.insert(myTable, X)
    local stringPos = 1 -- Position within MIDIstring.  
      
    -- Iterate through all events one-by-one   
    while stringPos < MIDIlen do
       
        local mustSelect = false
        local offset, flags, msg -- MIDI data that will be unpacked for each event
        offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos)        
      
        -- Check flag as simple test if parsing is still going OK
        if flags&252 ~= 0 then -- 252 = binary 11111100.
            reaper.ShowMessageBox("The MIDI data uses an unknown format that could not be parsed."
                                  .. "\n\nPlease report the problem:"
                                  .. "\nFlags = " .. string.format("%02x", flags)
                                  , "ERROR", 0)
            return false
        end
        
        -- Only need to check events that are not empty
        local msg1 = msg:byte(1)
        if msg1 then
        
            local eventType = msg1>>4
            
            if (laneIsCC7BIT and eventType == 11 and msg:byte(2) == targetCCnumber)
                                
            or (laneIsPITCH and eventType == 14)
                                    
            or (laneIsCC14BIT and eventType == 11 and (msg:byte(2) == targetCCnumber-224 or msg:byte(2) == targetCCnumber-256))
                                  
            or (laneIsNOTES and (eventType == 8 or eventType == 9))
                
            or (laneIsPROGRAM and eventType == 12)
                
            or (laneIsCHPRESS and eventType == 13)
                
            or (laneIsBANKPROG and (eventType == 12 or (eventType == 11 and (msg:byte(2) == 0 or msg:byte(2) == 32))))
                         
            or (laneIsTEXT and msg1 == 0xFF and not (msg2 == 0x0F)) -- Exclude REAPER's notation text events
                                    
            or (laneIsSYSEX and eventType == 0xF and not (msg1 == 0xFF))
        
            then
                mustSelect = true
            end
        end
        
        -- Re-pack and save in table
        if mustSelect then 
            t = t + 1
            tableEvents[t] = string.pack("i4Bs4", offset, flags|1, msg)
        elseif replace_selection then
            t = t + 1
            tableEvents[t] = string.pack("i4Bs4", offset, flags&0xFE, msg)
        else -- Keep selection status
            t = t + 1
            tableEvents[t] = string.pack("i4Bs4", offset, flags, msg)
        end

    end -- while stringPos < MIDIlen
    
    -- Copy the final All-Notes-Off to tableEvents
    t = t + 1
    tableEvents[t] = MIDIstring:sub(-12)
                
    -- Upload the edited MIDI into the take
    reaper.MIDI_SetAllEvts(take, table.concat(tableEvents))
    
    -- Check that there were no inadvertent shifts in event positions.
    if not (sourceLengthTicks == reaper.BR_GetMidiSourceLenPPQ(take)) then
        reaper.MIDI_SetAllEvts(take, MIDIstring)
        reaper.ShowMessageBox("The scripts caused inadvertent shifts in event positions.  Please report this bug.\n\nThe original MIDI data has been restored.", "ERROR", 0)
    end    
end

reaper.Undo_OnStateChange_Item(0, "Select all events in specified lanes", item)
My questions are both a bit more general and quite specific:

1) When you use
Code:
local MIDIlen = MIDIstring:len()-12
to exclude the final All-Notes-Off... why -12? Why not -16? I thought All-Notes-Off is a 2byte=16bit command?

2) From what I understand (given the format string in unpack()), the obtained data is saved in the following way: 4bytes signed integer (=offset), 1 unsigned char byte (flags) and a string whose length is given by an unsigned int of 4 bytes length (=the actual MIDI message? I guess it's expected to be 3-4 bytes long? Why do we need a 4byte-int "specifier" for this length?) Is that true? How do you know the format of that data string? How is this data actually encoded?

3) when you use
Code:
local msg1 = msg:byte(1)
I wonder: Does it give us the first 8 places of the string? In ones and zeros? Say the string is "1234567890"...would it give me "12345678"? Or just 8 bits, which would be a series of 8 ones and zeros which, depending on the size of each character in the string would be some bits of the encoded character? This question might be a bit confusing...but that is due to the fact that I am, indeed, confused. I hope you get what I'm aiming at and may enlighten me.

4) Why do you perform the
Code:
local eventType = msg1>>4
bitshift on the msg1? This I don't understand. Furthermore, when comparing this eventType variable against decimal numbers like 11 to check if it's CC-7bit information, for instance, does lua automatically know how to compare correctly? Or is there no difference to it whether I provide a number in dec or hex or whatever? I take it that e.g. 11 is just the dec representation of the binary expression of the actual event type?

5) Why do you suddenly use hex numbers (and not dec) when checking "laneIsTEXT" or "laneIsSYSEX"?

6) I don't understand the format in which the Reaper API function (getAllEvts) provides the flag information in the way it is "explained" in the reaper API doc... What does &1 and &2 mean there? Plus, in the re-pack part of your script, you use "flags|1" and "flags&0xFE" respectively. How is happening what you try to achieve? the 1 in the bitwise OR... is that of same length as flags? How do I know...? 0xFE is much longer? Help?

7) How does your consistency check work where you check if parsing is still going OK:
Code:
if flags&252~=0 then ......
I know this is a lot, but I'd be really grateful if you could shed some light on this! I really tried to get into it, but these things got me confused and it feels most of the problems come from not knowing how the data is actually managed and the doc is not specifically useful about that... Plus, I'm rather new to Lua...

Thanks a lot lot lot to you!
~Lukas
LuCsa is offline   Reply With Quote
Old 07-22-2018, 11:26 AM   #9
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

I'm pleased to see that more and more people are starting to use these Get/SetAllEvt functions.

The Get/SetAllEvts functions, as well as the | >> and & bitwise operations, have recently been discussed in two other thread, which explain the operations in helpful detail:
* Select events in specific CC lane
* Adapting my "nudge notes"


Quote:
When you use [...] to exclude the final All-Notes-Off... why -12? Why not -16? I thought All-Notes-Off is a 2byte=16bit command?
Despite the unusual name, All-Notes-Off is a normal CC event, so its message is 3 byte long. Adding the offset (4 bytes), flags (1 byte) and message length indicator (4 bytes), gives 12 bytes in total, in the format that REAPER encodes these events.


Quote:
Why do we need a 4byte-int "specifier" for this length?) Is that true? How do you know the format of that data string? How is this data actually encoded?
Different MIDI events can have messages of different lengths, so the Lua functions string.pack and string.unpack need to know how many bytes to pack or unpack.

When sending MIDI to a plugin or MIDI device, the 4-byte length specifier is not actually required, since MIDI plugins can deduce the expected length of the message from the message's first byte, which gives the event type.


Quote:
I wonder: Does it give us the first 8 places of the string? In ones and zeros? Say the string is "1234567890"...would it give me "12345678"? Or just 8 bits, which would be a series of 8 ones and zeros which, depending on the size of each character in the string would be some bits of the encoded character
The second option is correct: just 8 bits, equal to 1 byte. These Lua functions use single-byte encodings, so 8 bits is an entire character.



Quote:
5) Why do you suddenly use hex numbers (and not dec) when checking "laneIsTEXT" or "laneIsSYSEX"?
I use whatever format strikes me as the most readable, while I'm typing. Usually, for number that are larger than 15, hex is easier to read, since hexadecimal number are nicely aligned with bytes: two hex numbers for each byte. For example, if you see the hex number 0xFF0000, you immediately see that the first byte is FF (or 11111111 in binary) and the remaining two bytes are 0.


Quote:
I don't understand the format in which the Reaper API function (getAllEvts) provides the flag information in the way it is "explained" in the reaper API doc... What does &1 and &2 mean there?
If the rightmost (least significant) bit in flags is 1, it means that the event is selected; if it is 0, the event is unselected.
If the second-lowest bit is 1, it means that the event is muted; if it is 0, the event is nonmuted.

This means that there are four options:
* if flags is equal to 0 (or binary 00000000), the event is unselected and nonmuted. [EDIT: corrected]
* if flags is equal to 1 (or binary 00000001), the event is selected and nonmuted;
* if flags is equal to 2 (or binary 00000010), the event is unselected but muted.
* if flags is equal to 3 (or binary 00000011), the event is selected as well as muted.

&1 and &2 are bitwise "AND" with 00000001 and 00000010. If flags&1 is nonzero, the lower bit in flags must be 1. If flags&2 is nonzero, the second-lowest bit must be 1.


Quote:
How does your consistency check work where you check if parsing is still going OK:
if flags&252~=0 then ....
These lines are remnants from my very first scripts, and I should probably remove them since the test isn't really useful or reliable any more. Originally, the intend was to check whether something went wrong during packing or unpacking, and to check whether Cockos introduced new flags into the encoding. 252 = binary 11111100, so the quoted line checks whether all bits except the lower two bits are 0. In the current encoding, only the lower two bits are used as flags, so if flags&252 is not equal to 0, something went wrong.


Quote:
Plus, in the re-pack part of your script, you use "flags|1" and "flags&0xFE" respectively. How is happening what you try to achieve?
flags|1 is a bitwise "OR" with 00000001, which sets the leftmost bit equal to 1, thereby making the event selected.

flags&0xFE is a bitwise "AND" with 11111110, which sets the leftmost bit equal to 0, thereby deselecting the event.

Last edited by juliansader; 07-24-2018 at 02:52 AM.
juliansader is offline   Reply With Quote
Old 07-24-2018, 02:20 AM   #10
LuCsa
Human being with feelings
 
Join Date: Aug 2016
Posts: 87
Default

Dear Julian,

thank you so much for your time and detailed, diligent answers!! This helped so much and I learned a lot through further research!

Quote:
Originally Posted by juliansader View Post
This means that there are four options:
* if flags is equal to 0 (or binary --> 00000011 <--), the event is unselected ans nonmuted.
* if flags is equal to 1 (or binary 00000001), the event is selected and nonmuted;
* if flags is equal to 2 (or binary 00000010), the event is unselected but muted.
* if flags is equal to 3 (or binary 00000011), the event is selected as well as muted.
Am I correct that the first binary representation should be 00000000?

Thanks again!
Lukas

Last edited by LuCsa; 08-01-2018 at 01:47 AM.
LuCsa is offline   Reply With Quote
Old 07-24-2018, 04:13 AM   #11
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

Quote:
Originally Posted by LuCsa View Post
Am I correct that the first binary representation should be 00000000?
Oops, that was a copy/paste slip-up. 00000000 is correct.
juliansader is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 04:22 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.