When AI Starts Acting on Our Behalf

Part 5, and the finale, of our five-part series on the Privacy Leadership Workshop, hosted by Pyxos and the IAPP at the Riyadh Chamber of Commerce on June 22.

"That last panel had me jumping out of my seat." That was James Beriker, CEO of Pyxos, closing the day, and the panel he meant was this one. Tahir Latif, IAPP Country Leader for the UAE, KSA, and Qatar, moderated the closing session on agentic AI with a lineup spanning engineering, law, and research: Dr. Shaista Hussain, pathologist turned machine learning engineer and founder and CEO of SAIF CHECK, which builds compliance and security tooling for AI and quantum AI systems; Dr. Nisar Ahmad Zafar, CEO of Habtal, a sovereign AI company working with the defense and private sectors; Raza Rizvi, partner at Simmons & Simmons, where he leads the technology, data, and IP practice; Faysal A. Ghauri, Managing Partner at FANUUN BCG; and Dr. Mazhar Ali Bari, quantum physicist and visiting fellow at the University of Oxford, who was recruited onto the panel from the audience minutes before it began, after a scheduled panelist's flight fell through. "Be careful what you tell me," Latif warned the room, "because you may end up here unexpectedly."

Dr. Hussain opened by clearing the definitional fog, because the room's own definitions, gathered by a show of hands, varied widely. A large language model, she explained, is "a very speedy librarian" that retrieves from a knowledge base to answer questions. An agentic system is built on top of that model and "operates in a way that doesn't need regular, repetitive instructions: you can code a set of instructions, a task, and say act as an accountant, act as a lawyer, act as a DPO, retrieve what is needed, respond, justify your behavior." Agentic AI can plan, act, and rework autonomously, which is exactly what makes it both the opportunity and the risk.

Ghauri named the threat he argues the industry most underestimates: prompt injection, at the top of the OWASP risk list for two years running. An agent cannot reliably distinguish its owner's instructions from hidden instructions arriving through the same channel, an email, a document, a web page. "It's not a bug you can patch. It's structural: it reads your instructions and any hidden instructions from the same channel." He pointed to the documented EchoLeak-class vulnerability in a major enterprise copilot, where a crafted email could cause an agent to exfiltrate customer data with zero clicks from the user. His prescription reversed a twenty-year mistake in the making: "We spent twenty years building least privilege to protect our organizations. Then we integrated agentic AI and did the opposite, 180 degrees: we gave the agent human access, shared access, so you cannot trace which agent did what." The right model is agent-specific, task-based access that expires when the task ends; he cited market reporting that organizations granting excessive agent access are compromised at several times the rate of those enforcing least privilege.

Dr. Hussain pushed back on the fear itself, and the exchange was the panel's best. Agents can be given digital identities, "the same way you have a work ID number"; they can be traced end to end, with guardrails placed at every step of the architecture between request, retrieval, processing, and delivery. "There are tracking mechanisms and there are observability mechanisms. We are still the designers of these systems. That means we still have control as to how and where and when these things are deployed." The autonomy level is a spectrum and a design decision: humans fully in control of workflows at one end, near-autonomous operation reserved for low-risk, low-customer-impact contexts at the other. The discipline that matters is timing: assess, sandbox, and design the fail-safes before deployment, because retrofitting them afterward "is a bit of negligence and recklessness." The Kingdom's regulatory sandboxes, particularly in the financial sector, exist for exactly this. And she was precise about the human in the loop: it is not enough to have one; you must know "where a human is most strategically placed, with what responsibilities, without rubber-stamping and saying okay, okay, okay to everything."

Zafar drew the governance line where regulators will draw it. He is comfortable with agents that analyze and draft, "but when it comes to deciding on my behalf, I'm on the back foot," because delegation at machine speed, across resources no human can hold in view, changes the risk category. Yet abstention is not available: "Whether you like it or not, your organization and the environment around it are going to adopt agentic AI sooner or later. So why not be the first adopters, taking it head-on?" His non-negotiable: "You cannot make an agent accountable and put it in front of the regulator. If you are deploying any agent and you cannot nominate someone who is accountable, you should not be going agentic in any way, because if you leave that gap, the regulator is going to fill that gap with your name." His structural advice for higher-stakes contexts: break agentic workflows into small, siloed agents with narrow permissions and traceable handoffs, with a human as final decision-maker. And he repositioned the DPO in the process: in accelerator programs he has seen, most AI startups "don't have any clue what data they're collecting, storing, or using." The DPO who engages at design stage is not a brake but "the business enabler, the use-case enabler, and in most cases the key to your corporate ventures," because acquiring a company with a hidden data liability makes it the whole group's breach.

Rizvi took the legal frame to its hardest cases. Agentic AI joins the short list of technologies, he said, where law meeting technology "has been really messy": virtualization, sharding, blockchain. The linear world of a user prompting a model is analyzable; "with agentic, the bidirectional data flows when the model goes out and acts on your behalf create a huge world of complexity. You'll still have to determine who the controller is, but it's a much more complex exercise, with joint controllership issues, and the envelope of vulnerability is much broader." He flagged two commercial fronts already open: enterprises deploying agents are being asked to give representations, warranties, and indemnities over the data sets their agents traverse, and "there's a wave of litigation coming, it's already here in the US, around the provenance of data. That risk is being massively underappreciated." He was equally frank that safety representations from frontier model developers should not be taken at face value; the onus falls on deployers to verify. On the human in the loop, he set the bar the panel adopted, citing the direction of European guidance: oversight must be meaningful, able "to challenge, to interrogate, to make sure there's evidence, to be able to say: actually, no," and not "a token press of the approve button because it's a human finger with a fingerprint." His closing image for the profession was optimistic: with these tools, privacy teams will be working in "Iron Man suits."

The thread the room will remember came from Dr. Bari, who reframed the entire question through hifz al-'ird, the protection of a person's dignity, reputation, and private sphere in the Islamic tradition. "We talk about data protection as if it's just compliance, a box to tick, a penalty to avoid. Let's have a different frame. Are we protecting data because the law tells us to, or because the person behind the data has dignity and we are obligated to guard it? These two answers will give you very different systems. If privacy is a constraint, we do the minimum. We comply when watched. But if privacy is a purpose, we protect it even when no regulator is looking, and privacy by design stops being a checkbox. It becomes design intent itself, built from the inside out." This does not compete with the PDPL, he argued; it deepens it, and it aligns with the Riyadh Charter's move toward AI governed by human dignity. "The Kingdom is not playing catch-up, but perhaps leading this in a completely new direction." And the line that closed the day's thinking: "If you don't imagine your future, you're living somebody else's future."

That concludes our series on the Privacy Leadership Workshop. The conversation continues in the Pyxos masterclass series, which runs nearly every week with practitioners like these. To join the next session, or to be in the room for the next Riyadh event, visit pyxos.ai/masterclasses.

Key takeaways for DPOs

• Treat prompt injection as a structural property of agents, not a patchable bug; any agent that reads external content (email, documents, web) is an injection surface and must be scoped accordingly.
• Enforce least privilege at the agent level: dedicated agent identities, task-based access that expires on completion, and audit logs that can attribute every action to a specific agent, never shared human credentials.
• Name an accountable human for every agentic deployment before it goes live; if no one will own it, do not deploy, because the regulator will assign ownership for you.
• Do the assessment work pre-deployment: DPIA, sandbox testing, fail-safe and interrupt design; retrofitting controls after launch is negligence by another name.
• Re-run your controller-processor analysis for agentic workflows; bidirectional data flows across APIs create joint controllership exposure and data-provenance risk that contract teams are already pricing in.
• Get the DPO into AI projects at design stage and make the oversight meaningful: empowered to challenge, interrogate, and refuse, not to rubber-stamp at machine speed.

Our thanks to moderator Tahir Latif (IAPP) and panelists Dr. Shaista Hussain (SAIF CHECK), Dr. Nisar Ahmad Zafar (Habtal), Raza Rizvi (Simmons & Simmons), Faysal A. Ghauri (FANUUN BCG), and Dr. Mazhar Ali Bari.