Is this a thing yet? "Deep faking" someone's voice with a simple plugin. Like autotune but changing the voice instead of the tuning. Looking to put someone else's voice in a song and get sued.
As well as a.i. enabled plugins to make your output sound like whatever you train it to. Audio wise and performance wise.
There will be a day when you'll have perfect guitar simulation, perfect e.q. duplication, and performance execution tweaked to match anyone.
I'm really surprised something like this hasn't surface in a VST already. I certainly wouldn't invest $$$$ in a guitar sim right now, at some point using a GAN to match the preamp/eq/output/speaker/mic stages of a guitar amp will happen and the analog/digital argument will be over.
Vocalists will be in for a shock soon. Within 5 years you should be able to have "anyone" sing on your music.
Guitar playing is going to get messed up, as not just tuning but timing, vibrato and phrasing becomes tweakable. Fret buzz will be eliminated.
Personally I'm really looking forward to an a.i. trainable midi drum editor.
A.i. trainable eq. Mastering. As long as you have the right thing to train it on.
It will be the Really Big Deal in audio by 2022. And unfortunately allow more charlatans to appear to have talent. The hidden danger to humans: a.i. assisted activity masking true ability, spread across a few generations could have irreparable effects on the genome.
The computational power required to do what you are suggesting in real time in VST form just isn't there. I don't believe you are going to hear anything like that in VST form for a long time, at least not anything convincing.
Considering Deepfake even the best deep fake still looks odd and well, fake! - but to me the best are still comfortably better in some regards than the no expense hollywood CGI (see the jarringly odd Rachel FX in the otherwise visually excellent Blade Runner 2049, and the possibly worse video game style Grand Moff Tarkin in Rogue One).
Given a bit of time and effort the deep fake technique could have done a better job than than those premium efforts.
The computational power required to do what you are suggesting in real time in VST form just isn't there.
Initially it will be like Melodyne.
Quote:
I don't believe you are going to hear anything like that in VST form for a long time, at least not anything convincing.
I wouldn't be surprised if it happens before the end of next year. Max 3 years.
Quote:
Considering Deepfake even the best deep fake still looks odd and well, fake!
They're not all the same or trying to do the same thing. And it's not just about Deep Fake; there are a multitude of approaches happening simultaneously. What I'm referencing are the applications of applying styles via GANs and the generation methods that are effectively indistinguishable. The a.i. generated faces are well out of the uncanny valley. The Dadabots project should give everyone pause.
Quote:
no expense hollywood CGI (see the jarringly odd Rachel FX in the otherwise visually excellent Blade Runner 2049, and the possibly worse video game style Grand Moff Tarkin in Rogue One).
Tarkin was horrible but at least Rogue One was better than the abomination that was Last Jedi...
..but Deep Fake is just one basic use of techniques that are accelerating faster than Moore's law. Effectively my argument: you're able to do on your desktop over night what required millions of $ and months in Hollywood just a year ago.
...
Tarkin was horrible but at least Rogue One was better than the abomination that was Last Jedi...
Yes, I didn't like Rogue One but it was much, MUCH better than the Last Jedi.
Quote:
Originally Posted by chip mcdonald
..but Deep Fake is just one basic use of techniques that are accelerating faster than Moore's law. Effectively my argument: you're able to do on your desktop over night what required millions of $ and months in Hollywood just a year ago.
It's coming faster than you think.
The face mapping technology in Deep Fake is a neat trick, completely unrelated to audio techniques.
I think you are making some wild assumptions about how far technology will get in such a short time frame. Even the de-aging techniques used in The Irishman are pretty ropey in parts, looking like camp make up at times according to reviewers, and that's 2019 cost no object stuff.
The face mapping technology in Deep Fake is a neat trick, completely unrelated to audio techniques.
It's not a "trick" it's an algorithmic process. And the underlying technology of how it works is based on datasets, it doesn't matter whether it represents video or audio.
Quote:
I think you are making some wild assumptions about how far technology will get in such a short time frame.
I think you're like most people, and unaware of just how far a.i. has progressed. There are already programs that can clone voices, do deep fakes in real time. What you're calling unconvincing are bad examples, and it's merely a matter of motivation that keeps what I'm talking about from appearing on KVR as a plugin.
I would suggest there are developers right now that are in a curious period whereby they're holding their cards close to them while figuring out what to do, but once it starts it will happen as unbelievably fast as everything else in the a.i. field.
There are already programs that can clone voices, do deep fakes in real time...
That's what I thought I'd heard too. Any examples that are available to play with though? (Even if it's not in the form of a DAW plugin. I didn't really expect to see that!)
It's not a "trick" it's an algorithmic process. And the underlying technology of how it works is based on datasets, it doesn't matter whether it represents video or audio.
Okay you don't like the term "trick".
Right now the best I've seen so far is wide of the mark really - all of them are soft around the edges, and the faces don't fully blend, they look painted on and fuzzy. I would be happy to see some better examples.
I think you're like most people, and unaware of just how far a.i. has progressed. There are already programs that can clone voices, do deep fakes in real time. What you're calling unconvincing are bad examples, and it's merely a matter of motivation that keeps what I'm talking about from appearing on KVR as a plugin.
Cool, could you give some examples?
i've seen many of the video deep fakes, they are fun but it is very obvious a whole movie would have a lot of problems that would take you out of the experience.
Quote:
Originally Posted by chip mcdonald
I would suggest there are developers right now that are in a curious period whereby they're holding their cards close to them while figuring out what to do, but once it starts it will happen as unbelievably fast as everything else in the a.i. field.
On the contrary I suggest they would run as fast as possible (not walk) to Hollywood to beat the competition and would patent anything they possibly could. You can bet that numerous developers will get to a similar point at much the same time and thus no one can afford to risk sitting on anything.
The computational power required to do what you are suggesting in real time in VST form just isn't there. I don't believe you are going to hear anything like that in VST form for a long time, at least not anything convincing.
Oh, yes, it's available allright. Just not in VST form, but simple apps on several appstores. Most notably on a Chinese appstore. The one on the western side of things was removed after public outcry.
Quote:
Considering Deepfake even the best deep fake still looks odd and well, fake! - but to me the best are still comfortably better in some regards than the no expense hollywood CGI (see the jarringly odd Rachel FX in the otherwise visually excellent Blade Runner 2049, and the possibly worse video game style Grand Moff Tarkin in Rogue One).
Given a bit of time and effort the deep fake technique could have done a better job than than those premium efforts.
I've seen and heard pretty convincing examples from researchers. There's at least one site I remember that specialises in these: deepfake.com.
Mostly pics and video, not audio.
__________________
In a time of deceit telling the truth is a revolutionary act.
George Orwell
That's what I thought I'd heard too. Any examples that are available to play with though? (Even if it's not in the form of a DAW plugin. I didn't really expect to see that!)
I've been looking, talking to some people but everyone points me to other researchers who are using bespoke/customized front ends for their own tweaked processes.
There are a number of variations on the Deep Fake software, like DeepFaceLab, but nothing I'm aware of for audio.
Okay you don't like the term "trick".
Right now the best I've seen so far is wide of the mark really - all of them are soft around the edges, and the faces don't fully blend, they look painted on and fuzzy. I would be happy to see some better examples.
You realize this is early days for this technology - that you can run on your desktop? And that it is progressing faster than Moore's Law?
If you had casually looked at a 480p video 10 years ago with a good Deep Fake, unaware of the technology, you're saying you wouldn't have been fooled?
Quote:
i've seen many of the video deep fakes, they are fun but it is very obvious a whole movie would have a lot of problems that would take you out of the experience.
..... you're comparing what people do for fun on a desktop computer in a few hours to what Hollywood has the resources to do? You know about the Rogue One example: homemade desktop rendition from last year looks better than the Hollywood version just a few years previous. You think the technology is going to stand still for a decade?
I'd suggest any doubters watch the following video.
In this simple example 2 jazz audio recordings are converted to *classical* sounding. Not midi notes, not a composition, but the *sound*.
No, it's not perfect - it's a graduate student tinkering around. But one would be lying if one claimed nobody would be fooled into thinking the result wasn't an orchestra playing "classical music":
Now imagine a plugin that you can train to turn your tick-tacky upright piano recorded at your house into matching a Bosendorfer at Carnegie Hall, guitar into violin, croaky singing into Freddy Mercury.
You put the technology on all of your tracks, put it on the 2 bus to match Whatever Target Recording you want it to sound like - and it's not going to be like eq matching, or using IRs. It's going to be mind blowing.
Big problem is that it's going to confuse even more people into thinking they can "make music" or "play an instrument" because of it, and we'll enter an Uncanny Valley of fake music made by hacks who are poking at things and getting a realistically homogenized result. Digested by humans raised on monophonic music or no music at all, eventually raised on ersazt music.
But for the rest of us it will be great, this fiddly "getting sounds" business will be more like interior design than cutting sheetrock.
......and then 30 ppl will have jobs playing music. Great. Can't wait. The assault on musicianship continues. Make everything worth nothing. What an aspiration.
Not sure why anyone cares, everyone's been autotuning and quantizing bad performances to trick us into thinking people who can't play can for a couple of decades now - that's deep fake too. Which brings up a good point...
A close musician peer of mine and I were talking yesterday. I mentioned to him that when I record that I'm happy to leave that not quite perfect note in the performance - because when you hear it, you at least tend to realize that all the great-sounding stuff that surrounds that bad note it is real and actually occurred - when everyone is cookie-cutter perfecting everything after the fact, you can't really trust any of what you hear.
So in that regard, deep fake is just a tool man.
__________________ Music is what feelings sound like.
All I could find was randomly created realistic static faces, not animated. That could have been done more than a decade ago so what's the point here?
Quote:
Originally Posted by chip mcdonald
You realize this is early days for this technology - that you can run on your desktop? And that it is progressing faster than Moore's Law?
If you had casually looked at a 480p video 10 years ago with a good Deep Fake, unaware of the technology, you're saying you wouldn't have been fooled?
Irrelevant, we had Blu ray and HD TV 1080p more than ten years ago. Indeed in the UK we were used to 576 line Pal TV on CRT TVs. To be fair US television was very poor quality until everyone had HD and it became more or less a level playing field across the world.
I would say Deepfake could already be used to fit up a politician with low resolution doctored footage in 480p.
Quite another thing to do this in high enough quality for entertainment.
Quote:
Originally Posted by chip mcdonald
..... you're comparing what people do for fun on a desktop computer in a few hours to what Hollywood has the resources to do? You know about the Rogue One example: homemade desktop rendition from last year looks better than the Hollywood version just a few years previous. You think the technology is going to stand still for a decade?
No I don't think it will stand still, of course I don't. I've seen numerous videos precisely because it is so fascinating and seems a cool example of smart use of computer resources outperforming cost no object hollywood rendered CGI in some regards.
More importantly to the thread what I do think is that there seems to be something of a ceiling on what can be done in real-time within VSTs.
Going back to the off topic conversation: Hollywood CGI is all too often disappointingly unrealistic today a great deal of the time - see the new Terminator movie which amazingly looks worse than T2 much of the time. Probably becomes of the artful blend of practical effects with CGI in T2.
Ultimately it depends on the minimal quality people are prepared to tolerate. Maybe it will be like people that have never tasted real food preferring junk food?
Don't get me wrong, it has made amazing progress already. We aren't at the Running man stage yet, and certainly not in a VST!
Not sure why anyone cares, everyone's been autotuning and quantizing bad performances to trick us into thinking people who can't play can for a couple of decades now - that's deep fake too. Which brings up a good point...
A close musician peer of mine and I were talking yesterday. I mentioned to him that when I record that I'm happy to leave that not quite perfect note in the performance - because when you hear it, you at least tend to realize that all the great-sounding stuff that surrounds that bad note it is real and actually occurred - when everyone is cookie-cutter perfecting everything after the fact, you can't really trust any of what you hear.
So in that regard, deep fake is just a tool man.
I cannot remember which song or artist it was but anyway I remember a Rock producer from the 70s waxing lyrical about some famous guitar solo on a big hit that was actually elements of multiple takes seamlessly blended together (I think it was about twenty takes). It was then very, VERY hard even for the legendary guitarist to then repeat the quality of this amazing solo live.
IOW multi track studio recordings have almost always had an element of fakery about them.
IOW multi track studio recordings have almost always had an element of fakery about them.
I know, that's why every audio/video I've posted in collab over the last 18 months has been 100% live and as-is performance wise (there might be an overdub in one or two because I only have two hands LOL). Because I'm kind of sick of that very fakery - now, there is a "sensible" line here we can all be aware of - aka a creative comp of takes the player can actually play as a loose example or placing some reverb, maybe correcting that one flub... - vs creating a good performance out of pure shit.
Thusly, I get the tool thing, what I'm saying though is once we cross that line far enough and long enough, and never hold ourselves to any standards whatsoever, deep fake is the end-result. There will possibly be a day where we miss flaws, I already do because I think they are far more important to creative expression and being humans than most realize.
__________________ Music is what feelings sound like.
I'm of the flipside opinion about multi-track recording having an element of fakery because I play a lot of instruments, but can only play one at any given time, so my only course of action is to multi-track every part I play.
All the playing is 100% genuine and what you hear is what I played, but I consider myself more a composer/drummer and less a guitarist, bassist, banjo, mandolin picker. The stuff I play on instruments other than drums is also recorded in sections, because I compose and record at the same time, laying down 8/16/24 measures at a time. Drums I generally record last and in one single take, once the song has taken enough shape that there is something to play along with.
__________________
Glennbo
Hear My Music - Click Me!!!
--
I'm of the flipside opinion about multi-track recording having an element of fakery because I play a lot of instruments, but can only play one at any given time, so my only course of action is to multi-track every part I play.
I agree with that, like I said there's a sensible line there, I only have two arms and two legs and my toes can't make a G chord. And I'll be back to more studio'ey stuff at soon, just need a break and some sanity checks. I still generally abhor manufacturing turds into gold though.
__________________ Music is what feelings sound like.
... - vs creating a good performance out of pure shit.
That's the crux of the biscuit right there!
Saving that magical take by fixing the one truly awkward mistake vs settling for the stiff take with no mistakes is happiness and light.
Back to the fakery... I'm thinking that an "album" with some truly awkward stuff involving a few public figures and artists without their consent (and including some innocent bystanders) would be a great way to illustrate the emerging technology to average people.
Since I pay for YT and don't see ads, it's my primary method for ingesting what's out there. Almost everything I see that was done in a studio, I search out the artist performing live - preferably in intimate settings where there's little room for faking it - It's at least refreshing that much of the time the artist is capable. Wonder how much longer that will last.
__________________ Music is what feelings sound like.
Saving that magical take by fixing the one truly awkward mistake vs settling for the stiff take with no mistakes is happiness and light.
Right, there should be nothing wrong with suggesting that humanity make an effort to continue to aspire to that human element - manufactured perfection is fucking boring.
__________________ Music is what feelings sound like.
Right, there should be nothing wrong with suggesting that humanity make an effort to continue to aspire to that human element - manufactured perfection is fucking boring.
Imagine Chrissie Hynde with a healthy dose of Autotune!
I've always thought of multi-tracking as a means to capture performances or instances in time. Like Tom Petty said once in an interview about the studio capturing lightning in a bottle.
The use of plugins to fix up poor performances bugs me though. People need to f@*king learn how to play or sing and quit buying all the helper plugins.
__________________
Glennbo
Hear My Music - Click Me!!!
--
Is this a thing yet? "Deep faking" someone's voice with a simple plugin. Like autotune but changing the voice instead of the tuning. Looking to put someone else's voice in a song and get sued.
Not really a deep fake like you are referring to, but around 2000 I had a plugin called "Voice Tweaker" that could do a Barry White fake pretty well using it's "Maleify" preset. It had another preset called "Womanize" that did the opposite and would prolly need a name change for today's market.
__________________
Glennbo
Hear My Music - Click Me!!!
--
Apparently they're making a new movie with James Dean in it. I mean... CGI James Dean or deep faked James Dean.. not James Dean's rotted corpse The voice will be some voice actor doing his best James Dean impersonation, I presume.
I'd have to guess that we aren't far from technology existing where you could completely and convincingly fake video/audio of anyone, and that tech being used for even greater evils. How are we going to be able to verify authenticity of some "recorded" video? Say someone admitting to a crime, or a high ranking political official doing something that could incite a war, or implicate them (not that we need deep fakes for that!)
__________________ My Rig (also serves as my gaming PC): MSI Mag X570 Tomahawk Mobo, Ryzen R9 3900X, 32GB RAM, Samsung 960 Evo 500gb NVMe, Crucial 1TB NVMe, NVidia RTX 2080 Super, Arturia Minifuse 2, Nektar Impact LX25+ MIDI Controller Keyboard.
To expand this a bit further, how long until Pandora (or Spotify, whatever) is simply streaming us AI-generated content, that's better than what humans are creating?
I'm putting the over/under at 2030 for music.
2033 for video game content (i.e. not just random maps; whole epic story lines, new enemies, new graphic textures, new voice audio, all created on-the-fly).
2040 for AI-generated TV and movies.
My only real question is will humanity make it that long Sleep tight!
__________________
It helps if the hitter thinks you're a little crazy
- Nolan Ryan
To expand this a bit further, how long until Pandora (or Spotify, whatever) is simply streaming us AI-generated content, that's better than what humans are creating?
You're saying that like it's a bad thing. If AI's produce better music than people, I'm happy to listen to it. We'll just net universal basic income to make up for the lost jobs. Yang 2020!
I know, that's why every audio/video I've posted in collab over the last 18 months has been 100% live and as-is performance wise (
I'm starting to flip towards *not* worrying about "is this a major label-perfect execution?" in lieu of "this is what I did as a guitar player that has been playing for 30+ years".
Real humans that can *play*, not just try to be grid-bots, might be a rarity 10 years from now. As a guitar teacher I'm encountering more and more people that not only can't play anything with a shuffle or swung feel, they can't hear it at all: generations raised on beat detect.
Ersatz a.i. music is already here with Dadabots and Relentless Doppelganger, it will get better. But perhaps it will also tweak human's Uncanny Valley gauge as being "flat" and literally lifeless.
I'm starting to flip towards *not* worrying about "is this a major label-perfect execution?" in lieu of "this is what I did as a guitar player that has been playing for 30+ years".
Real humans that can *play*, not just try to be grid-bots, might be a rarity 10 years from now. As a guitar teacher I'm encountering more and more people that not only can't play anything with a shuffle or swung feel, they can't hear it at all: generations raised on beat detect.
On early Mahavishnu Orchestra albums in the 70s with Billy Cobham on drums, you could frequently hear him clipping the rim of a tom with a stick and other imperfections. Later they put Michael Walden in who was technically more perfected, but lacked the soul and fire of the raw performances Cobham played.
One thing that grates on my nerves are hiphop/urban/rap songs where a two measure sample is triggered over and over but on every round the trigger pad is hit slightly early so there is always a fraction of time missing. If you put it up to a metronome it would jump further and further in front of the beat.
__________________
Glennbo
Hear My Music - Click Me!!!
--
I'm starting to flip towards *not* worrying about "is this a major label-perfect execution?" .
+1000 - I certainly try to be as flawless as I can (=hard work instead of knob turning) and fix a thing here and there... but there's a point where I'd just be an impostor if I removed too much of the human element. I don't need someone else's studio but if I recorded in one, and they handed me a sliced and diced, over-compressed, auto-tuned master, I'd be furious.
PS: Everything good swings a little bit.
__________________ Music is what feelings sound like.
I'm starting to flip towards *not* worrying about "is this a major label-perfect execution?"
That's OK. The major labels aren't anymore either.
Heard anything out of them in the last 10 years that wasn't crude low budget? Obvious loops. Squashed dynamics. Autotune robot voice on everything. Everything sounds like it's always the same kid's Ableton Crash project or something. Some of these click track MIDI Tinkerbell projects wouldn't have made the cut for a b-side a few years back.
I think mistakes like an awkward sounding loop or an awkward robot effect on the voice are magnitudes more glaring than just a biffed note here or there or a little rhythmic stumble.
Of course there are still serious albums being recorded with technique and execution that are over the top. Perfect fidelity, great mix, surround sound... The pop stuff with the big tours and the videos and the budgets is all curiously low budget now though. Feels like opportunity.
Deep fake vocal plugin?
Linear Mode
