A major shift is under consideration in the European Union that could fundamentally alter how artificial intelligence interacts with your personal information.
The European Commission’s proposed "Digital Omnibus" regulation seeks to amend parts of the General Data Protection Act (GDPR), including provision relating to the use of "legitimate interest" lawful basis for developing and operating AI systems.
Unsurprisingly, privacy organisations are sounding the alarm over what this means for individual privacy rights. These concerns are vital and deserve urgent attention. However, this policy debate also raises an important technical question that enterprise security teams must confront:
If AI is legally permitted to access more data, what stops it from accessing far more data than it actually needs?
A lawful basis may establish when personal data can be processed, but it does not establish which systems an AI application should access, what operations it should perform or which technical controls must prevent unauthorised activity. As legal frameworks evolve, organisations need to look beyond compliance requirements and examine whether their security architecture can enforce meaningful boundaries around AI data access.
The legal shift: Expanding AI’s appetite to personal data
Under the existing GDPR, organisations must establish an appropriate lawful basis before processing personal data. These include consent, contractual necessity, legal obligations, vital interests, public tasks and legitimate interests.
The European Commission's Digital Omnibus proposal, introduced in November 2025, seeks to provide greater legal clarity regarding the use of legitimate interests when developing and operating AI systems. This is significant because legitimate interests can, in certain circumstances, provide a lawful basis for processing personal data without obtaining individual consent.
However, relying on legitimate interests is not the same as having unrestricted permission to use personal information. Organisations must still consider necessity, proportionality, individuals' rights, applicable safeguards and other GDPR requirements, including purpose limitation and data minimisation.
The European Data Protection Board (EDPB) and European Data Protection Supervisor (EDPS) have raised concerns about parts of the proposed changes, particularly their implications for fundamental rights and data protection. In their February 2026 joint opinion, the EDPB and EDPS stated that introducing a specific GDPR provision on legitimate interests for AI development and deployment was unnecessary, as the existing GDPR framework already allows organisations to rely on legitimate interests under Article 6(1)(f) in certain circumstances. However, they recommended further clarification and safeguards should EU legislators decide to retain the proposed provision.
For enterprise security teams, however, the question extends beyond whether a lawful basis exists. If an organisation is legally permitted to process particular information, does that mean every AI application within its environment should be technically capable of accessing it? The answer should be no. Legal permission to process data does not eliminate the need for independent authorisation, least privilege, purpose-based restrictions and enforceable access controls. This is where the distinction between data protection compliance and security architecture becomes critical.
What does this mean for UK organisation?
Because of Brexit, these specific EU negotiations do not directly extend to the UK. The UK operates under its own UK GDPR framework, but British businesses cannot simply look away.
First, the EU GDPR can apply to UK-based organisations where their processing activities fall within its territorial scope under Article 3. This includes offering goods or services to individuals located in the EU or monitoring their behaviour within the EU. Importantly, the GDPR's application is not determined by EU citizenship alone. It also depends on whether the organisation has a business presence in the EU, whether the processing is connected to that presence, and the nature and circumstances of its activities..
Second, the UK is evolving its own data protection framework through the Data (Use and Access) Act 2025. The Act introduced a new recognised legitimate interests lawful basis under the UK GDPR, with relevant provisions coming into force on 5 February 2026. However, this is distinct from the existing legitimate interests basis and is limited to specified purposes, including crime prevention, safeguarding, public security and emergencies. It does not establish blanket permission for commercial AI development.
UK organisations may already be able to rely on the existing legitimate interests basis for certain AI-related processing activities, subject to the applicable legal requirements and assessments. For organisations operating across both jurisdictions, the challenge is therefore not simply whether the rules are becoming more permissive. It is whether their AI governance, data protection practices and security controls can demonstrate that personal data is accessed and processed only for authorised purposes. Regardless of which legal framework applies, technical access must remain subject to enforceable security boundaries.
Why traditional access controls fails in the age of AI
For decades, data security has relied on static boundaries. You put data in a database, write an Access Control List (ACL), and grant permissions based on a user's role.
AI completely breaks this model. Generative AI and Large Language Models (LLMs) do not interact with data through neat SQL queries; they ingest, index, and synthesise unstructured data: emails, Slack channels, internal wikis, and cloud storage into dense vector embeddings.
If an AI tool is granted a broad legal basis to access corporate data systems to "improve operational efficiency," a fundamental problem emerges. The AI becomes a super-user. Without strict boundaries, a standard employee asking a corporate chatbot a benign question could inadvertently trigger the retrieval of sensitive HR records, proprietary code, or customer PII that the AI absorbed during its unrestricted ingestion phase.
Implementing least privilege in the age of AI
To prevent AI from becoming a channel for unauthorised information disclosure, security teams must apply the principle of least privilege. In traditional IT, least privilege means giving a user or process the minimum access necessary to complete an authorised task. For AI systems, the same principle applies, but it must account for the different ways models, retrieval services and autonomous agents interact with data and enterprise resources.
Three layers deserve particular attention:
Data Ingestion and Classification: Just because an AI can legally process data under a legitimate-interest clause does not mean it should. Organisations must implement appropriate data discovery and classification processes to identify, minimise, exclude or protect personal data and sensitive intellectual property before information enters model-training or retrieval pipelines. Depending on the use case, controls may include pseudonymisation, redaction, tokenisation, retention restrictions and approval of datasets for particular AI purposes. Data classification should inform which information an AI application is permitted to process, rather than merely describing the sensitivity of data after it has already been ingested.
Identity and Contextual Authorisation: AI applications must enforce the effective access rights of the requesting user and the permissions of the AI workload itself.
If an employee does not have permission to view the company's Q3 financial drafts, the AI chatbot must be structurally incapable of retrieving or disclosing information from those files on that employee's behalf. Controls such as Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC) can help enforce restrictions based on identity, resource classification, business context and permitted operations. The important requirement is that authorisation is independently enforced by a trusted component, rather than relying on the model to decide what it should be allowed to access.
Workload Isolation and Access Boundary: Rather than assuming that every AI workload requires a separate model, organisations should isolate sensitive data, identities, tools and execution environments according to their security requirements. A single AI model may support multiple business functions safely when appropriate authorisation and isolation controls are enforced. Conversely, deploying separate models will not prevent inappropriate access if they share an overprivileged retrieval service or unrestricted credentials. Security boundaries should therefore be designed around data sensitivity, business function, execution privileges and trust relationships.
AI governance to runtime enforcement
Legal frameworks operate in months and years; security threats operate in milliseconds. Relying solely on static governance policies to manage AI data access is a recipe for a breach. Instead, organizations must deploy dynamic runtime controls.
Runtime controls act as an active, real-time security guard sitting between the user, the AI model, and the data store.
An effective runtime governance strategy relies on three pillars.
Prompt Inspection: Analysing user inputs in real time to block requests that attempt to trick the AI into accessing restricted data zones (prompt injection defense).
Output Guardrailling: Scanning the AI’s generated response before it is displayed to the user to ensure it does not contain leaked personal data, passwords, or unauthorized privileged information.
Continuous Auditing: Tracking the lineage of how an AI arrived at a specific answer, ensuring that security teams can audit exactly which data sources the AI accessed during runtime.
Who owns the risk when AI accesses your data
Before implementing technical controls, organisations need to establish what each AI system is authorised to do, which data it genuinely requires and who owns the associated risks. This is where risk management, security architecture and data governance must work together. Business and data owners should establish acceptable use, permitted processing purposes and accountability for the information being accessed. Security architects should identify relevant trust boundaries, assess potential attack paths and translate security requirements into enforceable architecture controls. Engineering teams are responsible for implementing those controls, while assurance activities should provide evidence that they have been tested and operate effectively. Risk owners must then determine whether remaining exposure falls within the organisation's risk appetite and whether additional treatment or formal acceptance is required.
For example, an enterprise may approve an AI assistant to retrieve internal operational documents but prohibit access to employee medical records or confidential financial information. That decision should be reflected in the application's authorisation policies, retrieval architecture, data classification and monitoring requirements. It should also be tested against realistic scenarios, including attempts to retrieve restricted information using otherwise valid identities and permissions. This creates a connection between the organisation's risk decisions and the technical evidence needed to demonstrate that those decisions are being enforced.
Moving from legal permission to active enforcement
The EU Digital Omnibus proposals raises important questions about how personal data may be used to develop and operate AI systems. But regardless of how the regulations evolve, one principle should remain unchanged:
Permission to process data is not permission to access everything.
A lawful basis establishes when personal data may be processed. Security architecture must determine how technical access is authorised, restricted, monitored and enforced. For organisations deploying AI applications and autonomous agents, this means looking beyond data governance policies and security posture assessments. Can the AI retrieve information the requesting user is not authorised to see? Can an agent bypass existing permissions through a broadly privileged service account? Can sensitive information cross a trust boundary without an independent policy decision? And most importantly, what evidence demonstrates that these actions would be prevented? These are questions organisations should answer before expanding AI's access to enterprise data. Because knowing what your AI can access is only the beginning. Proving what it cannot access is where security assurance begins.
If your organisation is moving from AI visibility to active runtime enforcement, I help translate security risks into architecture requirements, enforceable controls and measurable assurance.
Get in touch to discuss how to establish those boundaries.
Email: [email protected]
