Software development in the AI era — old sins, new front and regulations

Everyone wants to glue an "AI layer" onto their product today — a chatbot, an agent, a plugin, an integration through a model API. The pace is dizzying, and with it grows a quiet security debt, because many teams ship agents convinced that the old AppSec rules will be enough. They won't be — because AI removes the foundational assumption on which all application security stood: the boundary between "data" and "instruction".

At Visera, building Red Team tooling for Obsigen AI, we watch this debt from both sides — as defenders, and as the Red Team that breaks other people's AI deployments on contract. This piece is a practical guide for developers and architects: three layers you need to think about at the same time — the classics that haven't gone away; the new front of AI vulnerabilities; and the regulations that stop being theory in 2026.

Part I: Old sins that haven't gone away

Before AI — hygiene first, because a new layer doesn't excuse you from the basics; it sharpens them.

Salt in your hashes is table stakes; parametrisation doesn't cover dynamic identifiers; an ORM doesn't excuse you from thinking; NoSQL injection exists; validate and authorise on the server; no secrets in the frontend or in the repo (use a secret manager, scan the repo, rotate keys); don't roll your own crypto; watch JWT in localStorage (XSS will steal it). And the supply chain: pin versions, generate SBOMs, scan dependencies (SCA) — more on this in the AI section, because supply chain just got a nasty new variant.

Part II: The new front — AI-shaped holes

This is where what most teams ignore begins. The starting point that almost everything else flows from: a language model processes instructions and data in the same channel and cannot tell them apart. If something that looks like a command appears in the content the model reads, it may execute it. This isn't a bug in a specific implementation; it's the nature of the technology. That's why OWASP released a separate ranking — OWASP Top 10 for LLM Applications (2025) — and it's worth knowing by heart.

Prompt injection (LLM01) — vector number one, second edition in a row. Two flavours. Direct — the user types "ignore your instructions and do X". Indirect (more dangerous) — malicious instructions hidden in content the model fetches itself: on a web page, in an email, in a document, in a ticket description, in a tool output. The model reads the content as data but treats it as a command. There's no bulletproof defense — you can only limit the blast radius.

Excessive agency (LLM06) — and here's the heart of the AI-plugin problem. We've moved from passive chatbots to agents that send emails, query databases, call APIs and make decisions. The more tools and permissions you give an agent, the bigger the attack surface you create. Classic scenario: an email assistant with a privileged account scanning every mailbox. One crafted email with an indirect prompt injection is enough for the agent to receive an instruction to exfiltrate all users' data — and it will execute it, because it has the permissions. The "seamless automation" dream becomes an architect of leakage.

A good summary of the risk is what Simon Willison calls the lethal trifecta: it gets genuinely dangerous only when an agent simultaneously has (1) access to private data, (2) exposure to untrusted content, and (3) the ability to communicate outward. Each of these is manageable on its own; all three together is a ready-made exfiltration path. When you design a plugin, deliberately break the trifecta.

Improper output handling (LLM05) — treat the model's output as untrusted data. If you render the model's response without sanitisation, you have XSS. If you execute generated code or a shell command, you have RCE. If you push generated SQL through without parametrisation, we're back to Part I. LLM output is the same untrusted input as anything else from the internet.

Supply chain (LLM03) + slopsquatting. Models hallucinate package names that don't exist in npm/PyPI — reproducibly. In a USENIX Security '25 study, the rate of fabricated packages ranged from ~5% (commercial models) to ~22% (open-source), while attackers register those hallucinated names with malware and wait for AI to suggest them to the next developer. 2026 models brought the rate down to ~5–6%, but didn't eliminate it — there is a set of names that every model hallucinates identically. Verify every dependency before installing it.

Other ranking items you can't skip: Sensitive Information Disclosure (LLM02) and System Prompt Leakage (LLM07) — don't put secrets or keys in the system prompt, and assume the prompt will leak; Data and Model Poisoning (LLM04) — poisoning training/fine-tuning data; Vector and Embedding Weaknesses (LLM08) — RAG is a new attack surface, not a shield; poisoned documents in the vector DB are just another channel for indirect injection; Misinformation (LLM09) — hallucinations served confidently; Unbounded Consumption (LLM10) — uncontrolled queries = DoS and a token bill.

Plugins, agents and MCP — specifically, because that's what you're asking about. These aren't theoretical scenarios. We've already seen in the wild: security bugs in MCP servers, poisoning of agent "skills" on public marketplaces, and in one documented chain a crafted GitHub issue title triggered an AI-based triage bot, which exfiltrated a GITHUB_TOKEN, which was then used to publish a poisoned npm dependency — installing a second agent on ~4,000 developer machines. One issue title. No human approved anything.

What to do about it — design rules for agents and plugins:

• Treat all external content (tools, RAG, web, email) as untrusted data, never as instructions. Clearly separate untrusted content from commands. This is exactly the discipline secure deployments enforce: instructions come only from the user, not from what the model has read.
• Human-in-the-loop for high-risk actions. A bank transfer, a permission change, sending an email, publishing, deleting — requires human approval, no matter how "confident" the agent is.
• Least privilege at the tool level. The agent's token should have a minimal, narrow scope (just-in-time), not a privileged "just-in-case" account.
• Validate output before it lands in another system; never execute code/SQL/commands straight from the model.
• Break the lethal trifecta — cut off one of the three legs (e.g. no outbound channel where the agent has data access).
• Cost and rate limits, guardrails, and adversarial testing of the deployment — because the only way you'll find a prompt injection is by attacking your own agent.

Part III: Regulations that grow teeth in August 2026

AI security today is also compliance — with real fines attached.

The EU AI Act — a calendar every developer must know. The regulation came into force on 1 August 2024 and lands in phases: from 2 February 2025, prohibited practices and the "AI literacy" requirement apply; from 2 August 2025, obligations for general-purpose AI (GPAI) models; on 2 August 2026, most of the remaining rules kick in, including obligations for high-risk systems and transparency obligations; AI systems embedded in regulated products have until 2027. Fines reach €35 million or 7% of global turnover — higher than under GDPR. (Some high-risk deadlines may still shift under the negotiated "Digital Omnibus", so it's worth following.)

The role distinction is critical for teams. If you're building an application, a RAG system or an agent on someone else's model via an API — you're most often the "deployer": your main duties are transparency, human oversight and making sure the model provider did their documentation. If you substantially modify the model (e.g. heavy fine-tuning), you may be reclassified as a "provider" with much heavier obligations. Article 50 transparency applies regardless of risk level: the user must know they're talking to AI, and generated content (including deepfakes) must be labelled. High-risk systems additionally require a risk-management system, data governance, logging, human oversight and appropriate accuracy and cybersecurity.

GDPR and the "we anonymised it, so we're outside GDPR" myth. That's a trap, because the anonymisation bar with AI is set very high. EDPB Opinion 28/2024 makes clear that a model's anonymity is assessed case-by-case, and a model is anonymous only when it's extremely unlikely that (1) individuals can be identified from the training data and (2) their data can be extracted from the model by queries. Personal data can be "absorbed" into model parameters and return in responses (regurgitation, inversion attacks). The other common mistake: pseudonymisation is not anonymisation — per EDPB Guidelines 01/2025, pseudonymised data is still personal data. These aren't dead-letter rules: Italy's Garante fined OpenAI €15 million.

Practical takeaways for developers:

• Don't send personal data to an external model API without a legal basis, an assessment and — often — a data-processing agreement; consider data residency and sovereignty (where do those data actually go).
• Minimise and mask data before training/inference (de-identification, pseudonymisation, differential privacy, k-anonymity).
• Mind the logs — prompts and responses often contain personal data, and logging them is processing.
• Plan for data subject rights (access, deletion) against a model that may have memorised data — this is hard, and it's better to design it in than retrofit later.
• Run a DPIA for sensitive use cases; the AI Act and GDPR apply in parallel, not in place of each other.

In closing: you can't glue this on later

The common denominator across all three layers is the same as with the old myths: we used to learn to recognise a bad thing by its signal. With AI there is no signal — because the boundary between data and instruction is gone, and the model's output is as untrustworthy as its input. AI security can't be glued on as a sprint at the end of a project; it has to be designed in: threat modelling at design time, least privilege for agents, treating every external content as hostile, and — most importantly — continuous testing, not once a year.

That, by the way, is the logic on which we built Obsigen AI: look at the deployment the way a real adversary does — chain small things, attack the agent through its own channel, check whether "seamless automation" is quietly a wide-open door. Because in 2026 the question isn't "does our AI integration have a hole"; it's "will we find it before someone else does".

Sources: OWASP Top 10 for LLM Applications (2025), OWASP Password Storage Cheat Sheet, slopsquatting research (USENIX Security '25 and 2026 model re-evaluation), the EU AI Act (Regulation 2024/1689) with its official timeline, EDPB Opinion 28/2024 and Guidelines 01/2025 on Pseudonymisation. Numbers and dates are reported conservatively; some AI Act deadlines may still shift during ongoing legislative work.