Old 09-30-2015, 08:39 AM   #1
Human being with feelings
Join Date: Mar 2007
Posts: 549
Default MIDI to AUDIO encoder/decoder

How about encoding MIDI signals into AUDIO stream?
For example to transfer MIDI through ReaInsert or ReaNINJAM (if it survived ogg lossy compression)....

Then using decoder it would allow to restore MIDI data (from audio) to further refine music melodies... (audio-encoded-midi saved as part of ninjam multitrack sessions)

Anybody have an idea how to work it out?


P.S.: (I mean individual only MIDI in audio files (as transport containers) and not added (injected) in audiofiles already containing any real sound).

Last edited by akademie; 09-30-2015 at 11:15 AM. Reason: addition to describing the idea
akademie is offline   Reply With Quote
Old 10-14-2015, 07:17 PM   #2
Human being with feelings
Join Date: Oct 2015
Posts: 5
Default keep going

Hi .
I think it is better to use the packets struct of data , not the data ( buffer of chars ? ) itself .
( in the case on ninjam of course )
Perhaps it is what your P.S means ?

You talk about midi signals , but we aren't in audio man , we have only midi messages ^^ .
Little reminder :
MIDI messages are used by MIDI devices to communicate with each other.

Structure of MIDI messages:

MIDI message includes a status byte and up to two data bytes.
Status byte
The most significant bit of status byte is set to 1.
The 4 low-order bits identify which channel it belongs to (four bits produce 16 possible channels).
The 3 remaining bits identify the message.
The most significant bit of data byte is set to 0.
src :https://www.cs.cf.ac.uk/Dave/Multimedia/node158.html

I need to remember that part of the ninjam code , i would probably write a new midi packet class or struct , to go after the audio packets .

The first problem i can already see is the timestamping , because the audio data transfer is done every (x)milliseconds , the time needed to fullfill the audio buffer , but midi messages can't wait that time .
Well , this is perhaps a good thing , because all midi transfer could occur
just between 2 audioblocks transfers .
And ...Aaah , you were saying that we could use the audioblocks ( buffers ) to hold the midi data instead of audio ?
I could compare the size of a midi msg and the size of the audio buffers ,
to use them as containers ?
EDIT : To not reinvent the wheel or midi streaming , i would use a library that do the job for me :https://www.music.mcgill.ca/~gary/rtmidi/
Queued MIDI Input

The RtMidiIn::getMessage() function does not block. If a MIDI message is available in the queue, it is copied to the user-provided std::vector<unsigned char> container. When no MIDI message is available, the function returns an empty container. The default maximum MIDI queue size is 1024 messages. This value may be modified with the RtMidiIn::setQueueSizeLimit() function. If the maximum queue size limit is reached, subsequent incoming MIDI messages are discarded until the queue size is reduced.
We could have std::vector<unsigned char> Midicontainer at our disposal .
Wait , i just found the description of the ninjam protocol here :
Where audio blocks are used like that :
Server Download Interval Write (0x05)

Transfers audio data to clients. The payload layout is:

Offset Type        Field
0x0    uint8_t[16] GUID (binary)
0x10   uint8_t     Flags
0x11   ...         Audio Data
If the Flags field has bit 0 set then this download should be aborted.
What is your final goal ?

Edit : So you want to know how to pack midi data onto audio blocks ? ^^
In mpb.cpp line 805 , on ninjam client source project :
Net_Message *mpb_client_upload_interval_write::build()
  Net_Message *nm=new Net_Message;

  unsigned char *p=(unsigned char *)nm->get_data();

  if (!p)
    delete nm;
    return 0;
Here we learned that the format being used to store the audio data is :
unsigned char *p
And that is exactly the same way that Rtmidi uses for exemple , remember :
std::vector<unsigned char> Midicontainer
Midicontainer is a managed array while *p is also an array of same type ...

Then we see that the data is already stored in the Net_Message *nm :
unsigned char *p=(unsigned char *)nm->get_data();
If i look to that func into Net_Message class :
void *get_data() { return m_hb.Get(); }
i see that what is returned is of type :
WDL_HeapBuf m_hb;
Finally , WDL_HeapBuf is just :

  This file provides the interface and implementation for WDL_HeapBuf, a simple 
  malloc() wrapper for resizeable blocks.
    int m_granul;
    void *m_buf;
    int m_alloc;
    int m_size;
you follow me ?
a std::vector<> could be used instead of that class , and there could be :
std::vector<void*> audioData ;
std::vector<void*> MidiData ;

But the use std::vector is probably slower than direct access in memory , while speedy enough to make it ...
Ah ah ah , the manager of WDL_HeapBuf is somewhere slow also :

This file provides a simple class for a FIFO queue of bytes. It uses a simple buffer,
so should not generally be used for large quantities of data (it can advance the queue
pointer, but Compact() needs to be called regularly to keep memory usage down, and when
it is called, there's a memcpy() penalty for the remaining data. oh well, is what it is).

anyway , even if i keep that WDL_HeapBuf as container , i'll just have to define a new
class member for Net_Message :
... ( Net_Message body )
    		int m_parsepos;
		int m_refcnt;
		int m_type;
		WDL_HeapBuf m_hb;
                WDL_HeapBuf midi_hb;//my new buffer for midi streaming eh eh
then send an audio block OR midiblock , depending on a flag specifiying the nature
of the Net_message . So yeah , add a 'char' var in the private section of Net_Message ? :

... ( Net_Message body )
    		int m_parsepos;
		int m_refcnt;
		int m_type;
		WDL_HeapBuf m_hb;
                WDL_HeapBuf midi_hb;//my new buffer for midi streaming eh eh
                char typeFlag; // a=audio m=midi u=undefined etc ...
to be continued ...
sorry for that looooong post .

Last edited by ezee; 10-14-2015 at 09:25 PM. Reason: a good reason ^^
ezee is offline   Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -7. The time now is 11:44 PM.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.