Old 03-17-2024, 12:27 PM   #1
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default Anyone using voice-to-MIDI conversion?

I can play a MIDI instrument -but- I can sing what I'm hearing in my head, MUCH easier than I can play it.

So, I'd like to create an orchestral piece, using my voice to create the MIDI notes rather than a keyboard or MIDI guitar.


This seems a really good way for non-skilled instrumentalists, to create their midi tracks.
Even folks with disabilities might be able to create, this way.
Presuming they can sing, on pitch.

Though I'm a fairly skilled guitarist, I'd sing an instrument line much different t hat I'd play it on a guitar, because it wouldn't be influenced by the licks and phrases I might gravitate towards.

And so, "singing" my tracks will make the result, a much more creative endeavor.
And not likely to sound like anything else I've written.

I assume ReaTune will work work fine for this.


Anybody else intrigued by this?
I'm curious about your thoughts, and especially your workflow!


Peace,

Keith
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 03-17-2024, 04:07 PM   #2
martifingers
Human being with feelings
 
Join Date: May 2011
Posts: 2,705
Default

Hi Keith. Here's my reaction - please bear in mind I am not a "trained" musician - The only formal education I have is classical singing lessons and an online course or two on music theory.

Yes, you would think voice to MIDI was a great way to go as it would seem to make a lot of sense (I loved your point about the guitar leading you into set patterns btw!) My impression is that this is not a widely used work flow at all. Now partly this may be that anyone wanting to do classical or any sort of orchestral arrangement probably have enough keyboard chops to be able to play into the DAW. (Guy Michelmore's channel for example seems to stress that you don't need to be an expert player to write complex pieces - well worth checking out BTW if you don't know of him.)

Contrariwise other genres rely on quite restricted instrumental textures and so voice to MIDI conversion is not really that crucial. I am pretty crap at piano but given enough time , a decent VST and lots of takes even I can probably do a reasonable/passable bass line, kalimba melody, organ backing etc. In the process I will hopefully audiate something that I want and indeed being able to "sing" the line is an essential part of the process.

The other problem is the conversion itself. Reatune will work for single ie monophonic lines but other software (including on line converters) are probably better - Melodyne springs to mind. I even seem to recall a crowd funded voice to MIDI app that was designed to work in real time. However no matter how good they are the resulting MIDI will probably need correction and in particular if you use the MIDI to generate musical notation you will almost certainly end up with something that is unreadable and that will need a lot of quantising, editing etc to make useable. I just looked at a video on transcribing jazz solos and the advice was to not even try to play it but that it would be much quicker for even an expert to write it out phrase by phrase.

All the above may change tomorrow of course if some bright person applies an AI solution! My feeling though is that, at the moment while perfectly possible, it is not a common work flow. However like I say, all the above just my impression and in any case you should not let anything stand in the way of you being creative with the technology. It is pretty amazing after all.
martifingers is offline   Reply With Quote
Old 03-17-2024, 04:47 PM   #3
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,110
Default

I tried it once (with imitone*) and as I noticed how bad my intonation actually is (I'm no trained singer) I gave up and went back to keyboard.

But maybe you have a better experience with it.

*a software where I took part in their kickstarter campaign 10 years ago and which is still in early beta stage as of today, but that's a different story. :/
nofish is offline   Reply With Quote
Old 03-17-2024, 06:29 PM   #4
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by martifingers View Post
Hi Keith. Here's my reaction...
Yes, you would think voice to MIDI was a great way to go as it would seem to make a lot of sense (I loved your point about the guitar leading you into set patterns btw!) My impression is that this is not a widely used work flow at all. Now partly this may be that anyone wanting to do classical or any sort of orchestral arrangement probably have enough keyboard chops to be able to play into the DAW. (Guy Michelmore's channel for example seems to stress that you don't need to be an expert player to write complex pieces - well worth checking out BTW if you don't know of him.)

Contrariwise other genres rely on quite restricted instrumental textures and so voice to MIDI conversion is not really that crucial. I am pretty crap at piano but given enough time , a decent VST and lots of takes even I can probably do a reasonable/passable bass line, kalimba melody, organ backing etc. In the process I will hopefully audiate something that I want and indeed being able to "sing" the line is an essential part of the process.

The other problem is the conversion itself. Reatune will work for single ie monophonic lines but other software (including on line converters) are probably better - Melodyne springs to mind. I even seem to recall a crowd funded voice to MIDI app that was designed to work in real time. However no matter how good they are the resulting MIDI will probably need correction and in particular if you use the MIDI to generate musical notation you will almost certainly end up with something that is unreadable and that will need a lot of quantising, editing etc to make useable. I just looked at a video on transcribing jazz solos and the advice was to not even try to play it but that it would be much quicker for even an expert to write it out phrase by phrase.

All the above may change tomorrow of course if some bright person applies an AI solution! My feeling though is that, at the moment while perfectly possible, it is not a common work flow. However like I say, all the above just my impression and in any case you should not let anything stand in the way of you being creative with the technology. It is pretty amazing after all.

Thanks for the reply.
I agree with your comments.

I wondering you and nofish were thinking of the same MIDI conversion program.
That would be a fun coincidence.


I'll likely find that my preferred workflow will be to start with chords.
But for melodies and single-lines, I'd really love to have a voice-to-MIDI option.

Rather than re-invent the wheel, I thought Id see if anyone is using that method, in their writing.


Oh...and yes, I've watched Mr. Michaelmore and he's great.
He's good at explaining things and he has a lot of fun, while he does.

Peace,
Keith
__________________
aka K8ch
(www.KeithHaydon.com)

Last edited by K8ch; 03-17-2024 at 06:34 PM.
K8ch is offline   Reply With Quote
Old 03-18-2024, 11:18 PM   #5
Winfield
Human being with feelings
 
Winfield's Avatar
 
Join Date: Jan 2007
Location: The Underground Bunker
Posts: 706
Default

Perhaps Neural Note is worth a look for you (it's free):
https://github.com/DamRsn/NeuralNote - only Win/Mac
-W
__________________
"if DAWs are religions, REAPER is atheism" - The big J
__________________
Windows 10x64 | Asus Z170-a i7, 32GB ram | RME-Digiface USB
Winfield is offline   Reply With Quote
Old 03-19-2024, 11:01 AM   #6
martifingers
Human being with feelings
 
Join Date: May 2011
Posts: 2,705
Default

Quote:
Originally Posted by nofish View Post
I tried it once (with imitone*) and as I noticed how bad my intonation actually is (I'm no trained singer) I gave up and went back to keyboard.

:/
Just as a test today I tried singing in an 8 bar melody using Neural Notes. I have to agree with nofish. Even with pitch correction the resulting MIDI had spurious notes, multiple notes where I just wanted one long one etc. etc. It took far longer to get a working part than playing it in via MIDI keyboard (with subsequent editing if necessary. I did not try but I reckon it was more effort than entering it note by note in Musescore - and of course the notation would be readable.

I hope other people have more success...
martifingers is offline   Reply With Quote
Old 03-19-2024, 02:15 PM   #7
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by martifingers View Post
Just as a test today I tried singing in an 8 bar melody using Neural Notes. I have to agree with nofish. Even with pitch correction the resulting MIDI had spurious notes, multiple notes where I just wanted one long one etc. etc. It took far longer to get a working part than playing it in via MIDI keyboard (with subsequent editing if necessary. I did not try but I reckon it was more effort than entering it note by note in Musescore - and of course the notation would be readable.

I hope other people have more success...
Thanks for sharing your results.
The first time I tried Neural Note I had the same results as you did.
But, I just spent a little time tweaking things and it seems to work pretty well.
Turning down "Note Sensibility" seemed to help a lot, though it sounded counter-intuitive

I'll futz with it, some more.

Thanks!
__________________
aka K8ch
(www.KeithHaydon.com)

Last edited by K8ch; 03-19-2024 at 07:21 PM.
K8ch is offline   Reply With Quote
Old 03-19-2024, 07:22 PM   #8
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by Winfield View Post
Perhaps Neural Note is worth a look for you (it's free):
https://github.com/DamRsn/NeuralNote - only Win/Mac
-W
Thanks for the suggestion, by the way.
With some tweaking, it may work itself into my workflow.

Thanks!
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 03-20-2024, 10:03 AM   #9
User41
Human being with feelings
 
Join Date: Aug 2020
Posts: 244
Default

Melodyne is the best Iíve found. The human voice is pretty complex, though. I think we are still a ways off from a good voice to MIDI conversion.
User41 is offline   Reply With Quote
Old 03-20-2024, 10:08 AM   #10
martifingers
Human being with feelings
 
Join Date: May 2011
Posts: 2,705
Default

Quote:
Originally Posted by K8ch View Post
The first time I tried Neural Note I had the same results as you did.
But, I just spent a little time tweaking things and it seems to work pretty well.
Turning down "Note Sensibility" seemed to help a lot, though it sounded counter-intuitive
I will try that, thanks. BTW another YT channel sort of like Guy Michelmore that may be of interest:https://www.youtube.com/channel/UCBq...nfP58vyzLMDSaA

A bit more demanding perhaps in terms of musical theory (I often have to stop and think what's going on) but she has a lot of interesting things to say.
martifingers is offline   Reply With Quote
Old 03-20-2024, 03:25 PM   #11
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by martifingers View Post
I will try that, thanks. BTW another YT channel sort of like Guy Michelmore that may be of interest:https://www.youtube.com/channel/UCBq...nfP58vyzLMDSaA

A bit more demanding perhaps in terms of musical theory (I often have to stop and think what's going on) but she has a lot of interesting things to say.

Thanks. I'm familiar with her, and though I don't read music I still learn a lot from her videos.
I appreciate the suggestion.
Marc Jovani (https://www.youtube.com/@CinematicComposing) is worth checking out, too.
You probably already know about him, tho...


Peace,
Keith
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 03-21-2024, 05:43 AM   #12
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by User41 View Post
Melodyne is the best I’ve found. The human voice is pretty complex, though. I think we are still a ways off from a good voice to MIDI conversion.
Thanks for chiming in!

Have you used Melodyne to do this?
Can you share your experience of it, and possibly compare it to any others you might've tried?
Just curious.
$249 for Melodyne seems like more than I care to spend since I wouldn't need it for anything else.


I agree. There's still a bit of editing to do, when trying to track a voice.

I've been playing guitar (MIDI too) for many years but I can't truly "play what I feel".
Not really.
I can only play licks and patterns that I've learned, and try to make it match the music.
But: I -CAN- sing the lines in a much more fluid and creative way than I can, playing my MIDI instrument.
So, I think this is a really worthwhile pursuit (for me).

Peace, Keith
__________________
aka K8ch
(www.KeithHaydon.com)

Last edited by K8ch; 03-21-2024 at 06:02 AM.
K8ch is offline   Reply With Quote
Old 03-21-2024, 08:08 AM   #13
Scoops
Human being with feelings
 
Join Date: Apr 2014
Posts: 102
Default

Quote:
Originally Posted by K8ch View Post
Thanks for chiming in!

Have you used Melodyne to do this?
Can you share your experience of it, and possibly compare it to any others you might've tried?
Just curious.
$249 for Melodyne seems like more than I care to spend since I wouldn't need it for anything else


Peace, Keith
Actually, I have used Melodyne for this.
I track my voice in Reaper, convert to midi in Melodyne, and use midi file to make harmony oooooo's and aaaaa's with various differnt vst's I have. It works quite well.

It is a bit on the pricey side, but it's one tool that's really worth it for me. If you're not going to use but once or twice, you probably have better options.

I do have a spare license of Melodyne essential. (PM me if interested) It does not do the conversion. You'd have to invest 150 bux or so to get the next level. The next package up from that does polyphony. It's way too cool.

Cheers
Scoops is offline   Reply With Quote
Old 03-21-2024, 05:30 PM   #14
martifingers
Human being with feelings
 
Join Date: May 2011
Posts: 2,705
Default

Quote:
Originally Posted by K8ch View Post
Marc Jovani (https://www.youtube.com/@CinematicComposing) is worth checking out, too.
You probably already know about him, tho...
No I didn't but it looks like one to add to the list. Thanks!
martifingers is offline   Reply With Quote
Old 03-22-2024, 08:48 PM   #15
User41
Human being with feelings
 
Join Date: Aug 2020
Posts: 244
Default

Quote:
Originally Posted by K8ch View Post
Thanks for chiming in!

Have you used Melodyne to do this?
Can you share your experience of it, and possibly compare it to any others you might've tried?
Just curious.
$249 for Melodyne seems like more than I care to spend since I wouldn't need it for anything else.

Peace, Keith
I sort of have used Melodyne for this purpose. As I said above, it works but not perfectly.

Melodyne is really good software though. I understand the price tag if you doní t have other uses for it. But it is one of only 2-3 pieces of software that I think you need to buy vs. rely on free alternatives. (Iím assuming of course that youíll use all of its functions).
User41 is offline   Reply With Quote
Old 03-23-2024, 07:16 PM   #16
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by User41 View Post
I sort of have used Melodyne for this purpose. As I said above, it works but not perfectly.

Melodyne is really good software though. I understand the price tag if you doní t have other uses for it. But it is one of only 2-3 pieces of software that I think you need to buy vs. rely on free alternatives. (Iím assuming of course that youíll use all of its functions).
That's true.
And, as much as I haven't admitted it, being able to fix a tiny flaw in an otherwise breathtaking vocal performance, is kinda nice option, too.

I think you and Scoops make -very- convincing arguments.
I think Melodyne Assistant is the way to go for voice-to-MIDI.

Thanks!!
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 03-24-2024, 10:29 AM   #17
S.L.I.P.
Human being with feelings
 
Join Date: Feb 2018
Posts: 335
Default

Here's a pretty cool video that Alejandro from Reapertips put out today. It's Audio to Midi using Reaper stock plugins:

Instantly convert AUDIO to MIDI in REAPER:

https://youtu.be/MbiQzNPIWWg?si=hwE9UgeDyHfiiYBV
__________________
New song, and video: The Lunatic Fringe Hijacked America

https://youtu.be/N0lb9Wt-acA?si=z7Dxj0lJnre6_y9E
S.L.I.P. is offline   Reply With Quote
Old 03-24-2024, 02:32 PM   #18
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by S.L.I.P. View Post
Here's a pretty cool video that Alejandro from Reapertips put out today. It's Audio to Midi using Reaper stock plugins:

Instantly convert AUDIO to MIDI in REAPER:

https://youtu.be/MbiQzNPIWWg?si=hwE9UgeDyHfiiYBV

Yep. Thanks for pointing it out, S.L.I.P.
It's a great video (as usual) but it's not "voice-to-MIDI", which is the focus here.
Still, it was worth watching and I appreciate your pointing it out!

Things that can be done w/Reaper, can often be done in a few different ways.
There's always something to learn!
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 03-24-2024, 04:50 PM   #19
S.L.I.P.
Human being with feelings
 
Join Date: Feb 2018
Posts: 335
Default

Quote:
Originally Posted by K8ch View Post
Yep. Thanks for pointing it out, S.L.I.P.
It's a great video (as usual) but it's not "voice-to-MIDI", which is the focus here.
Still, it was worth watching and I appreciate your pointing it out!

Things that can be done w/Reaper, can often be done in a few different ways.
There's always something to learn!
I don't know, but it seems like this technique should work for voice to midi, as long as the notes are sung correctly. It would be a similar procedure using Melodyne. You'd have to record your voice first, before Melodyne would be able to create a midi part.
__________________
New song, and video: The Lunatic Fringe Hijacked America

https://youtu.be/N0lb9Wt-acA?si=z7Dxj0lJnre6_y9E

Last edited by S.L.I.P.; 03-24-2024 at 05:05 PM.
S.L.I.P. is offline   Reply With Quote
Old 03-24-2024, 06:37 PM   #20
Philbo King
Human being with feelings
 
Philbo King's Avatar
 
Join Date: May 2017
Posts: 3,204
Default

I think you would create far more interesting music if you simply recorded the various voice parts. More samples? Boring.
Actual layered voice? Now that has potential!
__________________
Tangent Studio - Philbo King
www.soundclick.com/philboking - Audio streams
Philbo King is offline   Reply With Quote
Old 03-30-2024, 05:59 AM   #21
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by S.L.I.P. View Post
I don't know, but it seems like this technique should work for voice to midi, as long as the notes are sung correctly. It would be a similar procedure using Melodyne. You'd have to record your voice first, before Melodyne would be able to create a midi part.
Thanks again, SLIP.... Much appreciated.
I may just spring for the "Editor" version, later -just for the possibility of messing w/fingerpicked guitar.
Anyone know if it goes on sale, during the year?

Peace,
K8ch
__________________
aka K8ch
(www.KeithHaydon.com)

Last edited by K8ch; 03-30-2024 at 12:47 PM. Reason: correcttion
K8ch is offline   Reply With Quote
Old 03-30-2024, 06:52 AM   #22
mschnell
Human being with feelings
 
mschnell's Avatar
 
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 14,797
Default

There seems to be a bunch of new AI tools that can do that, I watched a lot of videos on voice replacement. Some seem go generate Midi on the fly.
mschnell is online now   Reply With Quote
Old 03-30-2024, 08:51 AM   #23
martifingers
Human being with feelings
 
Join Date: May 2011
Posts: 2,705
Default

We haven't mentioned Jam Origin here yet have we?

https://www.jamorigin.com/

I think I saw a review video where someone tried it with voice input but clearly it wasn't meant for that purpose.

But whatever I think I have found the perfect solution: https://www.youtube.com/watch?v=YsSRgIzAIyg
martifingers is offline   Reply With Quote
Old 03-30-2024, 11:47 AM   #24
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by mschnell View Post
There seems to be a bunch of new AI tools that can do that, I watched a lot of videos on voice replacement. Some seem go generate Midi on the fly.
I don't doubt it.
My guess is that it's not easy to incorporate into the Reaper workflow.
I also can't imagine AI developers really focus on pitch-detection, which the the cornerstone of this stuff.

Anyway, despite the promise of that AI can offer mankind, I don't trust them with it so I'd rather not even use it, when I can avoid it.
Yeah...it's probably more of a symbolic gesture than anything else.

Peace,
K8ch
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Old 04-03-2024, 05:02 AM   #25
RecycledInnovation
Human being with feelings
 
RecycledInnovation's Avatar
 
Join Date: Mar 2024
Location: Wherever brains are falling
Posts: 14
Default

Quote:
Originally Posted by K8ch View Post
Thanks again, SLIP.... Much appreciated.
I may just spring for the "Editor" version, later -just for the possibility of messing w/fingerpicked guitar.
Anyone know if it goes on sale, during the year?

Peace,
K8ch
According to Melodyne's website, even Essential works with any monophonic track, so depending on what you're playing on guitar, you don't necessarily need Editor. But it looks like Assistant is where audio-to-MIDI export becomes available, so you'd have to at least start there.

I picked up Essential for $49 (half off) at Sweetwater in 2020, then I was able to later upgrade to Assistant for $49 on Black Friday last year at Melodyne's website. I think $98 total was a great deal, given their normal web price is $249, and the software is excellent. I haven't found I need more than Assistant yet; I only use it with vocals right now, and the tracks I'm working with are monophonic. It's easy enough for me to have multiple FX windows open if I need to work with harmonies.

I'm not sure if they do big sales like that at other times of the year, but it's worth poking around and seeing. If you want to go for Editor, you could buy Assistant now, and upgrade during a sale later or when you find you need more.
__________________
Recycled Innovation on Bandcamp. I write music to capture the ideas that play in my head.
RecycledInnovation is offline   Reply With Quote
Old 04-03-2024, 05:52 AM   #26
K8ch
Human being with feelings
 
K8ch's Avatar
 
Join Date: Nov 2010
Location: New England
Posts: 693
Default

Quote:
Originally Posted by RecycledInnovation View Post
According to Melodyne's website, even Essential works with any monophonic track, so depending on what you're playing on guitar, you don't necessarily need Editor. But it looks like Assistant is where audio-to-MIDI export becomes available, so you'd have to at least start there.

I picked up Essential for $49 (half off) at Sweetwater in 2020, then I was able to later upgrade to Assistant for $49 on Black Friday last year at Melodyne's website. I think $98 total was a great deal, given their normal web price is $249, and the software is excellent. I haven't found I need more than Assistant yet; I only use it with vocals right now, and the tracks I'm working with are monophonic. It's easy enough for me to have multiple FX windows open if I need to work with harmonies.

I'm not sure if they do big sales like that at other times of the year, but it's worth poking around and seeing. If you want to go for Editor, you could buy Assistant now, and upgrade during a sale later or when you find you need more.

Hey...thanks for that great info.

As a matter of fact, I did buy "Assistant" - although not at the discount you got.
Wow!
I imagine that I'll be using Assistant in much the same way as you're using it.
So glad to hear it's working out for you.
You're absolutely right about needing Assistant to do the voice-to-MIDI properly as it seems to do the trick quite well.

I also appreciate knowing about the periodic sales because I think I'll want to upgrade.
When I'd seen videos about how easy it seems to tweak/change the individual notes in a fingerpicked acoustic guitar track, that just blew my mind.

I'll probably look into that a little more before I upgrade, but it seems really compelling!!

Thanks again!!
__________________
aka K8ch
(www.KeithHaydon.com)
K8ch is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 02:37 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.