#VoCo. Adobe Audio Manipulator Sneak Peak with Jordan Peele | Adobe Creative Cloud

I remember when Adobe demoed this idea of being able to edit waveforms by the recognized text back in 2016 and it was pretty mind blowing for the time. ▶ this video EDIT: I could also definitely see Audapolis being useful if you could integrate it into a podcast's post processing flow (volume normalization, de-essing) by recognizing certain verbal tics and automatically removing them from the audio such as "ummmm...", etc.

There was Adobe Voco, which seemed kind of a forerunner: ▶ this video , link . It purportedly could edit speech like an audio editor and looked like a destroyer of authenticity. And then nothing was heard of it anymore. (Edit: Wikipedia says that VoCo takes “approximately 20 minutes of the desired target's speech”, and that it was a research prototype.) There was a thing called Tacotron from a team at Google, in 2018: link (In fact, the OP repo and the original CorentinJ/Real-Time-Voice-Cloning apparently rely on Tacotron.) And there was something from 2019: link , link The latter two seem to need more samples than pure real-time editing. Overall, to me a layman, this space appears quieter than ‘deep-faking’ videos. Which makes me wonder if I haven't missed something.

There's two ways to think of this as a deepfake: 1 - This is an actual video of the guy in his home, but they changed/synthesized the audio and then worked on the lip movements to make it match. 2 - This is a video of an actor impersonating the guy, possibly to the extent of impersonating his voice (although his timbre might make that a little tricky), and then they just deepfake the face on to the actor. An example of this is deeptomcruise on TikTok, something that you should treat yourself to if you haven't seen yet - link (first two today aren't great, here's a good one - link ? ) I'm not even convinced there's any alteration here, but even if there was both of the above could be possible. Adobe demoed something called VoCo in 2016 that never saw the light of day, not sure if there is something approximating this available today: ▶ this video

Thinking ahead - will Hollywood in 50 years time be actor based or digital rights based when your film is computer rendered and you pick your own actors that get digitally placed. Effectively reducing actors to DLC! Remember, technology to fake voices has already be done. ▶ this video Certainly interesting times ahead.

It's been 3 years since Adobe first showed off their voice impression and editing feature (unreleased): ▶ this video Though this is very impressive, it seems to take longer and longer to make those tiny improvements that make all the difference wrt believability. N.B. The most convincing TTS I've ever heard (predating Lyre by quite a bit) generated things like this: link

> and probably audio too I've grown to completely distrust audio recording ever since I found out about VoCo from Adobe[0]. I wonder how long until there's an open source alternative. [0] ▶ this video

The "Adobe Photoshop for voice" is quite amazing: ▶ this video (you can skip the first ~third to get to the demo)

I really recommend the Adobe demonstration of their VoCo speech audio "photoshopping" because of Jordan Peele's reaction: ▶ this video . More technology demo should have unscripted, sincere reactions like that.

I really recommend the Adobe demonstration of their VoCo speech audio "photoshopping" because of Jordan Peele's reaction: ▶ this video . More technology demos should have unscripted, sincere reactions like that.

Lyrebird has real tech capable of doing what they claim, whereas Adobe's demo was completely fake. ▶ this video "Wife" sounds exactly the same in both places. All they did was copy the exact waveform from one point to another, like an automated cut and paste in Audacity. Nothing is being synthesized. ▶ this video The word "Jordan" is not being synthesized. The speaker was recorded saying "Jordan" beforehand for this insertion demo and they're trying to play it off as though it was synthesized on the fly. That's incredibly dishonest. This is a scripted performance and Jordan is feigning surprise. ▶ this video Again, the phrase "three times" here was prerecorded. This was a phony demonstration of a nonexistent product. Reporters parroted the claims and none questioned what they witnessed. Adobe falsely took credit and received endless free publicity for a breakthrough they had no hand in by staging this fake demo right on the heels of the genuine interest generated by Google WaveNet. They were hoping they'd have a real product ready before anyone else. If Adobe had a real product then they'd have proven it with a demo as alarming, undeniable, and straightforward as Lyrebird's. Instead they relied on aesthetics and flashy, polished, deceptive performance art with famous actors and de facto applause tracks to cover up the fact that they have nothing.

This site has a "demo" section featuring only Soundcloud clips. Uses to much the present tense "In a world first, Montreal-based startup Lyrebird today unveiled" and "Record 1 minute [...] and Lyrebird can [..]Use this key to generate anything" but has no actual product or beta version. Adobe had a much more impressive sneak peek of a similar product called VoCo: ▶ this video

I agree and I've upvoted you, but I feel it's worth pointing out that Adobe's claim about their own progress in this field was fake news. ▶ this video "Wife" sounds exactly the same in both places. All they did was copy the exact waveform from one point to another. Nothing is being synthesized. ▶ this video The word "Jordan" is not being synthesized. The speaker was recorded saying "Jordan" beforehand for this insertion demo and they're trying to play it off as though it was synthesized on the fly. This is a scripted performance and Jordan is feigning surprise. ▶ this video The phrase "three times" here was prerecorded. This was a phony demonstration of a nonexistent product. Reporters parroted the claims and none questioned what they witnessed. Adobe falsely took credit and received endless free publicity for a breakthrough they had no hand in by staging this fake demo right on the heels of the genuine interest generated by Google WaveNet. I suppose they're hoping they'll have a real product ready by whatever deadline they've set for themselves. To be clear, I like Adobe and I think it's a cunning move on their part.

▶ this video "Wife" sounds exactly the same in both places, so all this did was copy the exact waveform from one point to another. Nothing is being synthesized. If this is all this app can do, it would be quicker and easier to do this manually. That little "guh" noise at the beginning of the first "wife" could also be manually cleaned up and pitch/formant shifted to sound more natural with respect to its position in the sentence. ▶ this video The word "Jordan" is not being synthesized. He was recorded saying "Jordan" beforehand for this insertion demo and they're trying to play it off as though this was synthesized on the fly. This is all a scripted performance. Jordan is phoning in his task of feigning surprise. ▶ this video Here they double down on their lie. The phrase "three times" was clearly prerecorded. If Adobe wanted to put the bare minimum effort into trying to convince anyone this was a real product that exists and is capable of synthesizing speech on the fly, then they'd toss a beachball around the audience and have them shout out words to type. This is a fraudulent demonstration of a nonexistent product and an audacious insult to everyone's intelligence. Adobe is falsely taking credit and getting endless free publicity for a breakthrough they had no hand in. They are stealing the hype recently generated by Google WaveNet and praying they'll have a real product ready by whatever deadline they've set for themselves.