Digital security
AI-based voice cloning can make things way too easy for scammers – I know because I tested it so you don’t have to learn about the risks the hard way.
November 22, 2023
•
,
6 minutes. read
The recent stealing my voice brought me to a new crossroads regarding how AI already has the potential to cause social disruption. I was so surprised by the quality of the cloned voice (and in this extremely clever, yet comical style from one of my colleagues) that I decided to use the same software for “nefarious” purposes » and see how far I could go. in order to steal from a small business – with permission, of course! Spoiler alert: it was surprisingly easy to make and took almost no time.
“AI will probably be the best or worst thing that can happen to humanity.” -Stephen Hawking
Indeed, since the concept of AI became more common in fictional films such as Blade Runner and The Terminator, people have wondered about the endless possibilities of what the technology could produce. However, it is only now, with powerful databases, growing computing power, and media attention, that we have seen AI reach a global audience in a way that is both terrifying and exciting. With technologies such as AI lurking among us, we are extremely susceptible to seeing creative and rather sophisticated attacks take place with damaging results.
Voice cloning escapade
My previous police roles instilled in me the mindset necessary to try to think like a criminal. This approach has very tangible and yet underestimated advantages: the more we think about it and even actions like a criminal (without becoming one), the better one can be protected. This is absolutely vital for staying up to date with the latest threats and predicting future trends.
So, to test some of AI’s current capabilities, I once again had to adopt the mindset of a digital criminal and attack a company ethically!
I recently asked a contact of mine – let’s call him Harry – if I could clone his voice and use it to attack his company. Harry agreed and allowed me to begin the experiment by creating a clone of his voice using readily available software. Luckily for me, getting my hands on Harry’s voice was relatively easy: he often records short videos promoting his business on his YouTube channel. So I was able to put together a few of these videos to create a good audio test bench. Within minutes, I had generated a clone of Harry’s voice, which sounded exactly like me, and I was then able to write anything and have it played in his voice.
To up the ante, I also decided to add authenticity to the attack by steal Harry’s WhatsApp account with the help of a SIM swap attack – again, with permission. I then sent a voice message from his WhatsApp account to his company’s finance director – let’s call her Sally – requesting a payment of £250 to a “new contractor”. At the time of the attack, I knew he was on a nearby island having a business lunch, which gave me the perfect story and opportunity to strike.
The voicemail stated where he was and that he needed the “floor plan guy” to be paid, and said he would send the bank details separately right after. This added the sound check of her voice to the voice message added to Sally’s WhatsApp feed, which was enough to convince her that the request was genuine. Within 16 minutes of the initial message, £250 was sent to my personal account.
I have to admit I was shocked at how simple it was and how quickly I was able to trick Sally into thinking Harry’s cloned voice was real.
This level of manipulation worked because of a compelling number of related factors:
- the CEO’s phone number verified it,
- the story I made up matched the events of the day, and
- the voicemail, of course, sounded like the boss’s.
During my debrief with the company, and upon reflection, Sally stated that she felt this was “more than sufficient” verification necessary to fulfill the request. Needless to say, the company has since added more safeguards to protect its finances. And sure enough, I refunded the £250!
WhatsApp Business Identity Theft
Steal someone’s WhatsApp account via a SIM swap attack might be a fairly time-consuming way to make an attack more credible, but it happens a lot more frequently than you might think. Yet cybercriminals don’t need to go that far to achieve the same result.
For example, I was recently the target of an attack that, on the surface, seemed credible. Someone had sent me a WhatsApp message claiming to be from a friend of mine who is an executive in an IT company.
The interesting dynamic here was that even though I was used to checking the information, this message arrived with the name of the linked contact instead of appearing as a number. This was particularly interesting because I hadn’t saved the number it came from in my contacts list and assumed it would always show up as a cell number rather than a name.
Apparently, they solved this problem simply by creating a WhatsApp Business account, which allows you to add any name, photo, and email address you want to an account and immediately make it look authentic. Add that to AI voice cloning and voilà, we have entered the next generation of social engineering.
Fortunately, I knew from the start that this was a scam, but many people could fall for this simple trick which could ultimately lead to the release of money in the form of financial transactions, cards prepaid cards or other cards such as the Apple Card, all of which are favorites of cyber thieves.
As machine learning and artificial intelligence advance by leaps and bounds and become more and more accessible to the general public in recent times, we are entering an era where technology is starting to help criminals more effectively than ever, including by improving all existing tools that help obscure criminals. the identity and location of the criminals.
Stay safe
Coming back to our experiences, here are some basic precautions business owners should take to avoid falling victim to attacks leveraging voice cloning and other shenanigans:
- Don’t take shortcuts in trade policies
- Check people and processes; For example, verify all payment requests with the person (allegedly) making the request and have as many transfers signed by two employees as possible.
- Stay abreast of the latest technology trends and update training and defensive measures accordingly
- Organize one-off awareness training for all staff
- Use multi-layered security software
Here are some tips to protect yourself against SIM swapping and other attacks aimed at separating you from your personal data or money:
- Limit the personal information you share online; if possible, avoid posting details such as your address or phone number
- Limit the number of people who can see your posts or other content on social media
- Beware of phishing attacks and other attempts to trick you into providing your sensitive personal data
- If your phone carrier offers additional protection on your phone account, such as a PIN or password, be sure to use it
- To use two-factor authentication (2FA), in particular an authentication application or hardware authentication device
Indeed, the importance of using 2FA cannot be underestimated – make sure you also enable it on your WhatsApp account (where it’s called two-step verification) and any other online account that offers it.