Blog · Essay · AI security and compliance

AI security and compliance: Conducting a Risk Assessment for Generative AI in Regulated Industries

In a recent joint report, Cequence Security and KPMG examined how organisations can govern and secure agentic AI — a conversation that is now front‑page news in Singapore source. That report is a reminder that the hype around generative AI is now colliding with the hard reality of data‑privacy law, financial‑sector regulation, and health‑care compliance.

When I walked into a Singapore‑based fintech’s AI lab last month, the team had just rolled out a large‑language model (LLM) to draft customer communications. Within days, the regulator’s compliance officer flagged a single sentence that referenced a real client’s account number – a clear breach of PDPA. The root cause? No formal risk assessment before the model went live.

The incident illustrates a broader pattern: CEOs are eager to unlock generative AI’s productivity, but they often skip the very step that protects the business – a disciplined AI security and compliance risk assessment. Below is the execution‑first framework I use as an ISO 27001 Lead Auditor to turn that gap into a Monday‑ready checklist.


1. Why the usual “pilot‑and‑scale” approach breaks for regulated data

  1. Assumed data sanitisation – Many teams rely on post‑hoc redaction tools, assuming they will catch every PII leak. In practice, LLMs can hallucinate or re‑use fragments of training data that were never explicitly flagged.
  2. Model‑agnostic governance – Treating every AI model as the same ignores the unique attack surface of generative models (prompt injection, data extraction, model‑stealing).
  3. Compliance as a checkbox – Mapping a regulation to a single control (e.g., “encrypt data at rest”) does not address the lifecycle risk of generated outputs.

The result is a false sense of security that unravels the moment a regulator or a customer spots a leak.


2. The three pillars of an ISO 27001‑aligned AI risk assessment

2.1 Asset Identification & Classification

  • Data assets – Catalogue every data source the model consumes (training sets, real‑time feeds, prompt logs). Classify each according to regulatory sensitivity (PDPA, GDPR, HIPAA, MAS‑TRM). Use the same asset register you already maintain for IT systems.
  • Model assets – Record model version, vendor, training methodology, and any third‑party APIs. Note whether the model is “agentic” (i.e., makes autonomous decisions) because that triggers higher‑risk controls.

2.2 Threat & Vulnerability Mapping

  • Prompt‑injection vectors – Simulate malicious prompts that could cause the model to reveal protected data.
  • Data‑exfiltration via output – Run a “red‑team” script that queries the model with variations of known PII to see if it reproduces it.
  • Supply‑chain weaknesses – Verify the vendor’s own ISO 27001 certification and any third‑party libraries they embed.

2.3 Control Effectiveness Evaluation

  • Preventive controls – Input validation, context‑aware prompt guards, and fine‑tuning with differential privacy.
  • Detective controls – Real‑time monitoring of output logs for PII patterns, automated alerts to compliance officers.
  • Corrective controls – Incident‑response playbooks that include model rollback, audit‑trail preservation, and regulator notification timelines.

3. Monday‑ready checklist – What you can inspect this week

ActionOwnerQuick Win
Map all data feeds into the LLMData Engineering LeadPopulate a spreadsheet with source, classification, and retention policy.
Run a “PII‑leak test” on the modelAI EngineerUse a script that feeds 20 synthetic account numbers and checks output.
Validate vendor ISO 27001 scopeProcurement / SecurityRequest the vendor’s latest Statement of Applicability (SoA).
Create a model‑risk registerChief Risk OfficerAdd a row for each model version, risk rating, and mitigation status.
Draft an output‑monitoring rule setCompliance OfficerDefine regex patterns for regulated identifiers (e.g., NRIC, passport numbers).

Complete these items by the end of the week and you will have the foundation for a formal risk assessment.


4. Embedding ISO 27001 controls without slowing delivery

When I audited a health‑tech startup last year, the CTO feared that adding encryption and access‑control layers would double the time‑to‑market. The reality is that ISO 27001 controls can be layered as automation‑friendly policies:

  • Role‑based access for prompt‑logs – Use your IAM platform to grant read‑only access to compliance staff, while developers get write‑only rights.
  • Automated encryption at ingestion – Deploy a serverless function that encrypts every incoming data blob before it reaches the model.
  • Continuous compliance dashboards – Pull control‑status metrics from your SIEM into a single pane; the dashboard updates in real time, removing manual audit steps.

By treating controls as code, you keep the velocity of an agile AI team while satisfying ISO 27001 Annex A.


5. Governance hand‑off: From CEO to the operating model

A CEO’s responsibility ends at authorising the risk‑assessment process. The day‑to‑day governance belongs to a cross‑functional AI steering committee that includes:

  • Chief Information Security Officer (CISO) – Owns the threat‑model and control‑effectiveness reviews.
  • Chief Data Officer (CDO) – Ensures data‑classification consistency across the enterprise.
  • Head of Compliance – Aligns model‑output monitoring with regulator‑mandated reporting windows.
  • Product Lead – Guarantees that risk mitigations are baked into the product roadmap, not bolted on after a breach.

The committee meets bi‑weekly, reviews the model‑risk register, and escalates any “high‑risk” finding directly to the board. That cadence turns a one‑off audit into an ongoing governance rhythm.


6. FAQ

How often should I repeat the risk assessment?

A risk assessment is a living document. Re‑run the PII‑leak test with each new model version and any major data‑source change. At a minimum, schedule a full review quarterly.

What if my vendor does not have ISO 27001 certification?

Treat the lack of certification as a risk factor. Request a third‑party penetration test, and consider contractual clauses that require the vendor to obtain ISO 27001 within a defined timeline.

Can I rely on “privacy‑preserving” fine‑tuning alone?

Fine‑tuning reduces the likelihood of accidental data exposure, but it does not replace monitoring. Combine privacy‑preserving techniques with output‑scrubbing and audit logs.

How do I balance speed with compliance?

Automate controls wherever possible. Embedding encryption, access policies, and monitoring as code lets you ship features without manual gate‑keeping.

What’s the first thing I should tell my board about generative AI risk?

Present the three‑pillar framework (asset, threat, control) in a single slide, highlight any “high‑risk” assets, and request approval for the Monday checklist items.


7. Next steps – A soft invitation

If you see gaps in your current AI risk‑assessment process, I’m happy to walk through the checklist with you and tailor it to your industry’s regulator requirements. You can book a brief discovery call at https://calendly.com/rohan-girdhani/discovery-call or explore more on my homepage https://rohangirdhani.com/.


*Related reading:*

FAQ

How often should I repeat the risk assessment?

A risk assessment is a living document. Re‑run the PII‑leak test with each new model version and any major data‑source change. At a minimum, schedule a full review quarterly.

What if my vendor does not have ISO 27001 certification?

Treat the lack of certification as a risk factor. Request a third‑party penetration test, and consider contractual clauses that require the vendor to obtain ISO 27001 within a defined timeline.

Can I rely on “privacy‑preserving” fine‑tuning alone?

Fine‑tuning reduces the likelihood of accidental data exposure, but it does not replace monitoring. Combine privacy‑preserving techniques with output‑scrubbing and audit logs.

How do I balance speed with compliance?

Automate controls wherever possible. Embedding encryption, access policies, and monitoring as code lets you ship features without manual gate‑keeping.

What’s the first thing I should tell my board about generative AI risk?

Present the three‑pillar framework (asset, threat, control) in a single slide, highlight any “high‑risk” assets, and request approval for the Monday checklist items.

In the market

Headlines this post is responding to — not invented stats.