Voice-Cloning as a Threat

Voice-Cloning as a Threat

By Janpha Thadphoothon

Not being an IT or AI expert, what you are about to hear from me might be something that you already know – AI can mimic or reproduce voice, or do voice cloning. I am sure you would agree with me that the pace at which artificial intelligence is developing is nothing short of astonishing, and sometimes, a little unsettling. As we know, the technology has advanced so rapidly that, as of today, distinguishing a machine-generated voice from an authentic human one is almost impossible. 

In my opinion, this isn't just a remarkable feat; it's a situation that veers sharply from what we consider normal, carrying implications we are only beginning to understand.

Fundamentally, voice cloning is a technology where Artificial Intelligence (AI) is used to create a synthetic, artificial copy of a person's voice.

I have tried to clone a voice of myself many years ago and found that the technology is viable. First of all, let me share a personal anecdote. It must have been several years back, when the whispers about AI-driven voice synthesis were just beginning to gain traction beyond niche tech circles. Intrigued, and as someone always curious about the intersection of language and technology – a natural consequence of being a language teacher, I suppose – I ventured into trying one of the early platforms. I uploaded samples of my own voice, meticulously following the instructions. And then ultimately, after some processing time, which felt like an eternity, I prompted the system. I typed in a simple phrase, something mundane like, "Hello, this is Janpha," and lo and behold, a voice that was unmistakably mine, yet not me, spoke those words. The fidelity wasn't perfect by today's standards, but it was good enough to send a shiver down my spine. It was a profound moment, I must admit that.

Created by Gemini AI, prompted by J.T.

Let me try explain you the technology. As far as I know, you can train using a dataset; the more data and the more compute power, the better. Then you ask the system to reproduce – you can type "I disagree with what you have said" or "You need money." As we know, the underlying principle, at least as I understand it from my readings and my certificate course in Generative AI with Large Language Models from DeepLearning.AI, isn't magic, though it often feels like it. Fundamentally, it is all about feeding the AI a substantial dataset of a target voice. This dataset consists of audio recordings, often hours of them, meticulously transcribed. The AI, typically a type of neural network, then learns the unique characteristics of that voice: the pitch, the timbre, the cadence, the subtle inflections, even the common speech patterns and filler words. The more high-quality data you provide, and the more computational power you throw at it, the more convincingly the AI can learn to "speak" in that voice. Once trained, you can simply provide text, and the AI will synthesize audio output in the target voice. Imagine typing, "The meeting has been rescheduled," or more alarmingly, "Transfer the funds immediately," and having it spoken in a voice your colleagues or family would implicitly trust.

Later on, I discovered that Google, for example, prohibited users (free users) to clone voice in the sandbox platform. This observation, I think, was quite telling. When a tech giant like Google, which is at the forefront of AI development, decides to put restrictions on such a feature, especially on its more accessible platforms, it signals a recognition of potential misuse. My gut tells me that this wasn't a decision taken lightly. It suggested that the gatekeepers of this technology were already grappling with the ethical implications.

There are risks and threats, of course. And this is where my initial intrigue gradually turned into a more profound concern. Like it or not, the world moves on, and with every technological leap, new challenges emerge. What we all know and agree upon is that powerful tools can be used for both good and ill. Voice cloning, in my opinion, is a quintessential example of this duality.

Let's be a bit more scientific, or at least systematic, in exploring these threats.

First of all, there's the obvious threat of impersonation for fraud. You may wish to picture this scenario: an elderly person receives a call. The voice on the other end is their grandchild, sounding distressed, claiming to be in trouble and urgently needing money. The voice is a perfect mimic. How many would hesitate, especially when emotions are heightened? The news has it that such scams, often called "vishing" (voice phishing), are already on the rise, even with less sophisticated voice manipulation. With high-fidelity voice cloning, this could become devastatingly effective. People say that the human ear, and the trust we place in familiar voices, are remarkably easy to exploit.

Secondly, I am sure you would agree with me that the potential for disinformation and fake news is massive. Imagine a cloned voice of a political leader appearing to endorse a controversial policy or make an inflammatory statement just before an election. Or a CEO seemingly announcing a catastrophic failure in their company, causing stock prices to plummet. Experts say that in an era already struggling with "fake news" in text and doctored images, deepfake audio could add a potent and harder-to-detect layer of deception. The saying "seeing is believing" has long been challenged; soon, "hearing is believing" might become equally fraught.

Thirdly, and this is something that particularly resonates with me as a language teacher, is the erosion of personal reputation and trust. Critics such as those focused on digital ethics would tell you that the ability to make anyone say anything could be weaponized for personal vendettas, blackmail, or severe harassment. Consider the psychological impact on the victim, who might struggle to prove their innocence against a recording that sounds exactly like them. It has perplexed me how we, as a society, will navigate a world where our own words can be so easily fabricated and turned against us.

What's more, there's the challenge to intellectual property and the creative arts. Voice actors, narrators, singers – their unique vocal talents are their livelihood. I'd like to entertain you with the idea that widespread, unauthorized voice cloning could devalue their work or see their voices used in ways they never consented to. Some argue for the creative possibilities, perhaps generating new performances by long-deceased artists. But some argue against it, citing the ethical nightmare of consent and compensation. It is my personal belief that we need to tread very carefully here.

Globally, the response to these emerging threats is still nascent. Different countries will undoubtedly adopt different regulatory approaches. In Thailand, for example, while awareness of cybercrime is growing, specific legislation addressing AI-generated voice impersonation might still be in its early stages. We often see technology outpace the legal frameworks designed to govern it. That's not all; even if laws are in place, the cross-border nature of the internet makes enforcement incredibly challenging.

My conviction is that education and awareness are paramount. As a language teacher, I often emphasize critical thinking when interpreting texts. Now, we must extend that critical thinking to auditory information. We need to cultivate a healthy skepticism, a habit of cross-referencing, and an understanding of the technological capabilities that exist. It is well known that literacy in the 21st century encompasses more than just reading and writing; it includes digital and media literacy.

I am not an expert in cybersecurity, but I have read somewhere that researchers are working on AI tools to detect AI-generated voices. This is a kind of technological arms race, where "good" AI is developed to fight "bad" AI. However challenging, I determine to make it clear that technology alone is unlikely to be the complete solution. The "democratization" of AI tools means that cloning capabilities are becoming more accessible, not just to large corporations or state actors, but to individuals with moderate technical skills and, potentially, malicious intent. They say that what was once the domain of specialized labs can now be achieved with off-the-shelf software or cloud-based services.

I notice that younger generations, often dubbed "digital natives," might be particularly vulnerable, paradoxically because they are so immersed in digital communication. Based on first impression, they might be quicker to trust digital interactions. However, I also have faith in their adaptability and their capacity to learn and navigate new digital terrains. Wisdom from the past hints that every generation faces its unique technological challenges and finds ways to adapt.

Let me introduce you to the notion of a "zero-trust" approach, but for audio. Perhaps, in high-stakes situations, we might need to move towards systems where voice alone is not sufficient for authentication or verification. Multi-factor authentication, video confirmation, or even pre-established code words could become more commonplace, even in personal communications, if the threat escalates. I know you would agree with me that this adds layers of complexity to our interactions, but it might be a necessary inconvenience.

Fundamentally, I would argue that the conversation around voice cloning is not just about technology; it's about ethics, responsibility, and the kind of society we want to build. It is my personal belief that developers of these powerful AI models bear a significant responsibility. They must proactively think about safeguards, responsible deployment, and the potential for misuse from the very inception of their creations. The decision by Google, which I mentioned earlier, to restrict access on its sandbox platform, was perhaps an early example of such corporate responsibility, however limited.

I somehow think that the genie is already out of the bottle. The technology exists, and it will continue to improve. Therefore, our efforts must focus on mitigating the risks. Some argue for stringent regulations, while others fear stifling innovation. Finding that balance is incredibly difficult. Nevertheless, it is my long-held belief that (though I could be wrong) we cannot afford to be passive observers. We need a multi-pronged approach involving technological safeguards, legal frameworks, public education, and a strong ethical compass guiding AI development and deployment.

What's more interesting is that the positive applications, though not the focus of this piece, do exist. For individuals who have lost their voice due to illness or injury, voice cloning offers a path to regaining a part of their identity. For creative industries, it can offer new tools for dubbing, character creation, or even personalized digital assistants that sound more natural and engaging. I like the idea of technology serving humanity in positive ways. As a matter of fact, AI has tremendous potential for good.

However, we must remain vigilant about the "threat" aspect. Those were the days when everything was simple, but that simplicity is often a veil of ignorance about underlying complexities. Like it or not, the world moves on, and we must move with it, armed with knowledge and caution.

Make no mistake, the capacity for AI to clone voices with increasing accuracy is a double-edged sword. One may ask what the ultimate impact will be. No one knows everything, but I would like to sound a note of informed caution. My gut tells me that we are still in the early stages of understanding the full societal implications of this technology. Accordingly, continuous dialogue, research, and proactive measures are essential.

While I marvel at the technological prowess behind voice cloning, like most people, I am also deeply concerned about its potential for misuse. It is a stark reminder that with great power comes great responsibility, as the saying goes. We, as individuals and as a society, need to be prepared. Having said that, I realize that fostering a culture of critical engagement with technology, rather than outright fear, is the most constructive path forward. 

Indeed, the future will be shaped by how we choose to navigate these powerful new tools. The past is the past; we must look to how we responsibly manage such innovations for a more secure future.

Janpha Thadphoothon is an assistant professor of ELT at the International College, Dhurakij Pundit University in Bangkok, Thailand. Janpha Thadphoothon also holds a certificate of Generative AI with Large Language Models issued by DeepLearning.AI.