I saw something disturbing this morning.
One of my clients showed me an email. They use Gmail for their emails (on their own domain) and download them locally.
The email officially came from their company president, giving the purchasing department orders to immediately pay an invoice of around €20,000 to a new supplier in the UK. It included all the details and had the invoice attached as a PDF.
The worrying part is that the style and tone of the writing were exactly like their president's. However, the sender's address, while using the correct name, was a generic Gmail account. This immediately raised a red flag for the purchasing department, and they didn't fall for it. It was also easy for them to check because the president was in their office at that very moment.
Looking at the sender's address, it would have been simple for anyone to figure out what was happening, but many people don't.
The accuracy with which they (likely using an LLM) recreated the president's writing style is truly concerning.
