Although there are a few ways to mitigate the risk, the only way to block it is to get AI to differentiate instructions from data, which is impossible today.
"To address this class of risk, we use a defense-in-depth strategy with safeguards that block malicious instructions at multiple points and help keep tasks aligned with users’ requests,” Microsoft said.
Keep talks aligned with users’ requests?!? We’re doomed!
“We encourage customers to install the latest updates, use multiple layers of security protection, treat content from unknown sources with caution, and review AI-generated content before using or sharing it.”
No, no you don’t, you encourage customers to spend more to give copilot access to all of SharePoint and all of Exchange, so they can replace human competency, human expertise and long standing employees with copilot until they’re dependent on your ever-pricier subscription and you can raise prices until you recover your vast and incomprehensible LLM losses from your customers.
“Separating instructions from data may be part of the solution, but I think the distinction between data and instructions is not always clear in real-world workflows. For example, a user may ask an agent to arrange a business trip, requiring the agent to retrieve an email specifying the approved itinerary and a document containing the booking procedure,” Måløy [vulnerability researcher and discloser] said.
Or you could hire a PA, who wouldn’t spread the already live copilot worm.
A bunch of security experts note that distinguishing between instructions and data is a solved problem with SQL injection attacks, but completely unsolved in LLMs:
“None of them are rewarding that work commercially right now, so treat that as a multi-year research problem, not something a CISO should wait on.”
Is this a minor technical issue or a major problem? It’s a major problem.
Mike Wilkes, enterprise CISO at Aikido Security, said it would be difficult to overstate the potential problems from this situation.
Keep talks aligned with users’ requests?!? We’re doomed!
No, no you don’t, you encourage customers to spend more to give copilot access to all of SharePoint and all of Exchange, so they can replace human competency, human expertise and long standing employees with copilot until they’re dependent on your ever-pricier subscription and you can raise prices until you recover your vast and incomprehensible LLM losses from your customers.
Or you could hire a PA, who wouldn’t spread the already live copilot worm.
A bunch of security experts note that distinguishing between instructions and data is a solved problem with SQL injection attacks, but completely unsolved in LLMs:
Is this a minor technical issue or a major problem? It’s a major problem.
I like your display name
Well, I like yours too, but don’t remind my wife of that. She’s just not a fan of me flirting with other men.
I have good news for your wife, then!
…which is?
I have never heard a human write this, but I see Claude say it allllll the time
I’ve seen it used fairly often in cybersecurity communities, even before llm’s existed.
But I’ve never heard anyone write it either.
Maybe if more people used chalkboards…
After all, why wouldn’t Microsoft use copilot to wrote their press releases?
I mean, I see it all the time in like, military history books. But like, not in casual conversation or company statements.
My logic isn’t flawed. I’m not using a motte-and-bailey fallacy, I’m just using rhetorical defense-in-depth! 😂