AI investigations: risks, safety and data governance
Generative AI is under growing scrutiny: safety, child protection and data governance are becoming decisive criteria for adoption and control.
Generative artificial intelligence is entering a new phase: it is no longer enough to evaluate its capabilities; it is necessary to measure its impact on safety, child protection and data governance. When a conversational model is used at scale, every technical vulnerability quickly translates into operational, legal and reputational risk.
The case of an investigation launched by a state authority into an AI provider shows how the topic is no longer confined to labs or product teams. For decision makers, the question is not only whether a system works well, but whether it is supported by controls sufficient to prevent abuse, unintended exposure and uses incompatible with internal policies or applicable law.
Why regulatory pressure is growing
Authorities are shifting attention from performance and adoption toward responsibility, accountability and control of use cases. This change of perspective reflects a simple fact: generative models can produce useful content, but can also facilitate harmful behaviour if there are no effective barriers, contextual filters and escalation processes.
For companies, this means adoption cannot rest on an abstract assessment of the technology. A concrete reading of the risks is needed: access to data, retention of conversations, handling of sensitive inputs, protection of vulnerable users and verifiability of the answers the system gives.
Minors, sensitive content and product responsibility
The most delicate front concerns services used by the general public and minors. In these contexts, a chatbot is not just an interface: it is a point of contact that can amplify inappropriate content, reinforce dependency dynamics or fail to catch signs of psychological and behavioural risk.
Organisations that develop or integrate AI must therefore design specific controls for age, context and purpose. Among the most relevant measures are user classification, limits on exposed functionality, automated moderation with human review, secure logging and red-team testing focused on self-harm content, sexual abuse and manipulation.
What changes for those adopting AI in the enterprise
Even those who do not develop proprietary models are involved. A company that integrates an AI assistant into customer care, HR or internal support must know where data ends up, who can access it and how anomalous cases are handled. Responsibility does not end with the vendor contract: it also depends on the ability to govern configurations, permissions and workflows.
In practice, an adoption model with three levels of control is needed: risk governance, technical security and supervision of generated content. Without this framework, AI becomes a multiplier of exposure rather than of efficiency.
Data, national security and the technology supply chain
Another critical element is the origin and circulation of data. Concerns about possible access by hostile actors show that AI platforms are no longer judged only on service quality, but also on the robustness of the digital supply chain, data segregation and control of integrations.
For IT and security managers, this implies a review of asset inventory, data residency, encryption keys, privileged access controls and policies for sharing with third parties. The question is not whether AI is useful, but whether the architecture allows its impact to be limited in case of abuse or compromise.
From reputational risk to governance strategy
The message for the board is clear: AI cannot be managed as a simple software feature. It must be treated as a strategic capability with high regulatory impact, with executive sponsors, periodic audits and updated risk metrics. Organisations that move first build trust; those that delay risk more costly corrective interventions.
- Define clear usage policies for generative models and high-risk cases.
- Apply specific controls on minors, sensitive content and harmful prompts.
- Verify data flows, access, retention and third-party integrations.
- Introduce periodic security, bias and abuse testing before release.
- Align legal, security, compliance and product under a single governance framework.