How would you align audio and MIDI, given this situation?

[bhuether]

This is a follow up to a previous thread of mine (https://forum.cockos.com/showthread.php?p=2064190), now that I have better sense of what I might need to do.

I now have MPL's QuantizeTools installed as well as Eugen2777's transients to stretch markers script.

With these scripts, here is what I think can be done in theory, just need help putting all the pieces together:

Case 1: You have a MIDI track with some melody or whatever. Now you record it with guitar, for instance, and you played rubato style, so the audio is more free time than the MIDI. You now want to adjust the MIDI notes so that they align with the audio. The transients to stretch markers script can get you pretty much to point where you have stretch marker at each note transient in the recorded guitar. So you would have, say, 10 MIDI notes, and 10 stretch markers. I see in MPL's script how I can detect anchors and targets. So I would have 10 anchors (the stretch markers) and 10 targets (the MIDI note ons). But I can't get MPL's tool to automatically align those MIDI notes to the stretch markers. Ideas?

Case 2: Reverse of above. I have, say, the same MIDI and audio track as in Case 1. But now I want to do opposite - I want the stretch markers to automatically align to the MIDI note ons. Ideas?

Curious how you would accomplish this, and maybe there are other more suitable scripts for my task at hand?

thanks, Brian

[foxAsteria]

You don't need any scripts. All you need is an understanding of timebase, warp markers and the SWS/BR grid actions.

If you want to align the grid to audio, set timebase to time, run the action "ignore project tempo" on all MIDI items (otherwise they will always follow tempo and grid changes, and then use the BR actions to line the grid up to your recording. Now if you want to adjust the grid and have your items follow, switch timebase back to beats. These settings are made in Project Settings but can be overridden on a track or item level (I only ever use Project Settings).

[Eliseat]

Hi,

I tested MPL's Quantize tool and also couldn't get the result. I just made a setup with an audio drumloop and a midi pattern which almost matched the drumloop. But for some reason the script wasn't able to count the timestretch markers as expected. I had 10 markers corresponding to the 10 midi notes. The script always counted 7 stretchmarkers and put the last two in guide mode. In MPL's wiki I read that the stretch marker function is kind of unstable so ...

foxAsteria is right. This is easy done per hand and doesn't need much time. Sometimes I align a whole track in five minutes and have the benefit to be sure everything is at the right spot. The intention of MPL's script is pretty cool. But before I try several hours to achieve my goal I just do it myself. (Same with the love act )

I will post my find in MPL's forum and ask if maybe I have done something wrong.

Eli

[Eliseat]

Have to correct me. I didn't use MPL's newest Quantize tool. I just updated the Reapack archive and will test it again.

Edit: Now it worked as expected. I did just a simple test and everything was fine. It now detected the stretchmakers correctly and put them exactly at the MIDI note positions. The script version is 2.01.

Audio to MIDI notes.



MIDI to audio stretch markers.



Really great tool I have to say. Many thanks MPL.

[bhuether]

FoxAsteria,

Can you say in a bit more detail how to go about what you described? Reaper has various types of markers, just not sure what steps I need to take to arrive at desired result.

thanks,

Brian

[foxAsteria]

My mistake, not "warp markers" but stretch markers.

https://wiki.cockos.com/wiki/index.p..._grid_with_SWS

https://reaperblog.net/2015/04/video...marker-basics/

https://www.youtube.com/watch?v=rid4NpN8mqY

http://reapertv.co.uk/reaper-tutoria...rker-workflow/

The project timebase controls how media items, track envelopes, and project markers behave when the project tempo changes.

Timebase beats (position, length, rate):
Project elements will keep their position and length constant as measured in beats. Media item playback rate will speed up if the tempo is increased, or slow down if the tempo is decreased. This is the default project timebase.

Timebase beats (position only):
Project elements will keep their position constant as measured in beats, but media item length will remain constant as measured in seconds.

Timebase time:
Project elements will keep their position and length constant as measured in seconds. Positions and media item length will change as measured in beats, if the tempo is changed. This was the default project timebase prior to REAPER version 4.25.

Tempo and time signature change markers have timebase beats by default, regardless of the project timebase. This behavior can also be changed in File/Project Settings.

The timebase for individual tracks and media items can be set independently of the project timebase. MIDI media items always have timebase beats (position, length, rate). The timebase for individual MIDI media items can be locked to a specific tempo regardless of the project tempo, in the media item source properties.

[bhuether]

Eliseat, I also confirm it works, if the MIDI notes and audio notes are somewhat close. And I suppose one wouldn't have to worry about more extreme cases. On my end, even though I have 5 MIDI notes, it says 6 anchor notes. Maybe it is counting all MIDI messages including the note off which is the 6th message as I see in list editor.

Anyway, really cool stuff!

This raises the question - is this type of editing best done in this manner, or should I look more into what FoxAsteria said?

Some questions that come to mind: are we overlooking anything with this method? Would it be better to use sliced audio parts and have the boundaries overlap a bit and apply cross fades at boundary points? Or should I not even worry about that?

In any case, nice job MPL! Или, молодец!!

regards,

Brian

[foxAsteria]

Originally Posted by bhuether

This raises the question - is this type of editing best done in this manner, or should I look more into what FoxAsteria said?

Nice thing about Reaper (some don't like it though) is that there is always more than one way of doing things. Try different ways and see what suits you. I prefer manual for this type of thing, since you usually have to double check automated processes for error anyway. But mpl's tool is new and improved and he hasn't gotten much feedback on it, so go share your experiences with it on his thread:
https://forum.cockos.com/showthread.php?t=165672

[Eliseat]

Originally Posted by bhuether

... I have 5 MIDI notes, it says 6 anchor notes..

This is because it counts the start of the item as an anchor point too. So you have five notes and one item start point. The target item start point gets then shifted to fit the start point of the source.
No, it does not count the item position. It counts one more but it has no influence of the outcome. The more I test the better it gets. Really cool tool!

Which way to prefer? I never used mpl's quantize tool before so I always used the manual method. This looks like a lot of work but is in end effect easy to do. Okay, if you have songs with hundreds of tracks and 10 min long then I also would search for an automated solution.

The cool thing about the quantize tool is, you can get good results very quickly if everything is prepared correctly. For example: You record a grooving guitar and want to let a pizzicato string follow the same timing. You firstly have to make sure that EVERY note (transient) of your audio material has the right stretchmarker at it (per hand or guide). And you have to make sure you have the same count of MIDI notes in your target MIDI item. And you also have to make sure that the target notes are close enough at the source position to not shift to the next anchor accidentally. If everything is fine, you will have a lot of fun watching all the MIDI notes matching the audio peaks magically.

So you maybe need the same time preparing the tool or doing it by hand. Doing it manually needs focus and concentration but its more safe as you grab every single bar or beat to match it up. If you want to use the tool I would split the audio parts (four/eight bars?) to keep the overview. As we don't know how your project looks like we really can't give you more advice as showing you what's possible and letting you try and choose.

Good luck!

[mpl]

@Eliseat
Thanks for mentioning my script. A couple notes:
1. You have to push red button to calculate output positions, THEN turn knob (anyway seems I should add option to skip this step and calculate output on first knob touch).
2. If you align audio to audio, probably mpl_AlignTakes is better automated solution.
3. For QT, there is no need to have same count of anchor points and targets. You can limit target to search closer point only in defined area. Like, "don't move target stretch marker, if closer point far then 1.5beats".
4. For QT2.04 I fixed mismatch of anchor point/target count for green/blue buttons and green/blue test markers.

[bhuether]

Ok, here is how I am now currently doing this in fairly automated way. This assumes I already have MIDI track with corresponding notes to what I played on guitar, for instance.

1. Use Dynamic Split on audio track, with number of slices equal to number of notes on corresponding MIDI track. Then from there save stretch markers, and adjust manually so that the markers are precisely where I need them. With right Dynamic Split settings the amount of manual touch up is minimal.
2. Use Script: spk_Convert stretch markers to MIDI notes.
3. On the MIDI notes created in step 2 above, use Script: Lokasenna_Copy values from selected midi notes.lua
4. Still in the GUI from step 3 above, select the MIDI notes that I already had and choose Apply.

Voila!

Alternatively, if I didn't already have MIDI file corresponding to what I am playing on guitar, I could just go from step 2 above to created MIDI item and just adjust note pitches.

This is really useful. Still curious how others do this sort of stuff. But so far, I can't think of a more efficient way than steps above.

thanks, Brian