The complete Module 7 curriculum: Lesson 1 — key considerations in planning for AI deployment (own model vs third-party); Lesson 2 — governing the release, monitoring and maintenance of the AI model. Every topic from the sheet, with plain-words examples.
Before anything ships: who built it, who uses it, are your policies ready, and where does it run? The sheet's L1 planning considerations, with plain-words examples.
In-house team, third-party vendor, open-source community, or a foundation model provider?
Example: a bank deploys a credit model built in-house, but the LLM chatbot is from a vendor — different provenance, different accountability.Off-the-shelf vs fine-tuned vs fully custom — determines how much control you have over behaviour, bias, and security.
Example: a vendor chatbot with no fine-tuning access can't be re-trained to remove a discovered bias.Employees, customers, or the public — each audience changes the risk profile and disclosure duties.
Example: an internal copilot for staff vs a public-facing claims assistant have very different oversight needs.| Environment | What it is | Plain-words example |
|---|---|---|
| Cloud-based | Model runs on cloud infrastructure (AWS, Azure, GCP); scalable, pay-per-use | A customer chatbot runs on the cloud so it can scale to 100k concurrent users during sales season |
| On-premise | Model runs on the organisation's own servers; full data control | A hospital runs its diagnostic AI on-premise so patient data never leaves the building |
| Edge | Model runs on local devices — phones, sensors, cameras; low latency, offline | A factory defect-inspection model runs on the camera itself, flagging defects in milliseconds without network access |
| GenAI — fine-tuned, RAG? | Generative AI choices: fine-tune a foundation model, or use Retrieval-Augmented Generation to ground answers in your data | A legal assistant uses RAG so answers cite the firm's actual documents instead of hallucinating |
The platform, orchestration, and tool access that agents run on.
Example: an agent that books travel needs APIs to flights, hotels, and payments — each a new attack surface.Agents act autonomously — risk models must cover each action the agent can take.
Example: a refund agent's risk model limits it to refunds under $500; anything larger escalates to a human.The sheet's tiered guardrails for agentic systems.
Tier 1: foundational guardrails · Tier 2: risk-based guardrails · Tier 3: societal guardrails (below)| Tier | Guardrail type | What it covers | Plain-words example |
|---|---|---|---|
| Tier 1 | Foundational guardrails | Basic safety: the agent cannot exceed its permissions, access forbidden data, or act outside its purpose | The travel agent cannot book anything without a confirmed budget approval — a hard boundary baked into the system |
| Tier 2 | Risk-based guardrails | Scoped by risk level of the action: higher-risk actions need more oversight | Low-risk actions (answering FAQs) run free; high-risk actions (changing a delivery address) require human confirmation |
| Tier 3 | Societal guardrails | Broader societal impact: bias, manipulation, public harm, fundamental rights | A recruitment agent is reviewed for demographic fairness and cannot use protected attributes in decisions |
Map the legal landscape before deployment — PDPA, GDPR, EU AI Act, sectoral rules.
Example: a credit model must satisfy MAS fair-dealing guidelines plus PDPA consent rules.Consider alternatives — a different model, a simpler system, or no AI at all.
Example: a rules-based check may handle 90% of the use case with zero model risk.Write down what the system may and may not be used for.
Example: the model card states the triage tool is for screening, not diagnosis.The organisation's appetite for different harm types.
Example: the bank accepts credit-decision risk but not safety risk — tolerance is documented.Formal assessment of the deployment's risks, per the framework in place.
Example: a probability × severity matrix rates each failure mode of the chatbot.The model must pass testing before release — verification (built right) and validation (right system built).
Example: the fraud model is verified against specs and validated on live-shadow data before go-live.Version, containerise, and document the model for deployment.
Example: the model ships as a versioned container with its model card, config, and dependencies pinned.APIs, interfaces, and integration points that users actually touch.
Example: the chatbot's API is documented, rate-limited, and monitored from day one.Monitoring starts at deployment, not after problems appear.
Example: drift alerts, error rates, and escalation paths are armed before the first real user arrives.The sheet's trade-off analysis: risks and opportunities of deploying your own model, managing third-party risks, and evaluating vendor/open-source agreement terms.
You own the data risk — quality, sensitivity, and lawful basis are your responsibility.
Example: if training data contains special-category data, the company carries the full GDPR exposure.Complex models may be hard to explain to regulators and users.
Example: a deep neural net's credit decisions resist the simple explanations the regulator asks for.Provenance risk: where the data came from, and whether it was lawfully obtained.
Example: a model trained on scraped data carries legal risk even if the model performs well.Who owns the model, its training data, and its outputs — internally and with partners.
Example: an agency builds a model for a client — but who owns it when the contract ends?Mitigations fall on you: no vendor to share the burden.
Example: bias findings must be fixed by your team — there's no vendor patch to wait for.Higher governance burden: documentation, testing, monitoring, and audit trails.
Example: the regulator expects full technical documentation — the in-house team must produce it all.You may carry both hats: liable as provider (building it) and deployer (using it).
Example: under the EU AI Act, the in-house team is both provider and deployer of the credit model — double obligations.Full visibility of training data — no black-box provenance.
Example: the team can answer "what data trained this model?" from the lineage log in one screen.You have the documentation and control to satisfy regulators directly.
Example: the AIA, model card, and risk register are all produced in-house, on demand.No inherited supply-chain or backdoor risk from code you didn't write.
Example: a known open-source model had a backdoor — the in-house model isn't affected.Train exactly for your use case, data, and domain.
Example: the bank's fraud model is trained on its own transaction patterns — off-the-shelf models underperform on them.Context: third-party AI integrates into business operations — internal or external, including internal employee use of external tools.
| Risk | What it means | Plain-words example |
|---|---|---|
| Data lineage and traceability | You may not know where the vendor's training data came from | A vendor chatbot trained on scraped personal data — the customer's data flows into an unknown pipeline |
| Downstream issues | Vendor changes, model updates, or shutdowns ripple into your systems | The vendor retires a model version and the bank's chatbot suddenly behaves differently |
| Model output ownership and control — depending on licensing | Who owns the outputs, and what the licence lets you do with them | The licence forbids using outputs to train a competitor model — the team must check before fine-tuning |
| Data handling and security risks — depend on development environment | Your data may be processed in environments you don't control | Employees paste customer data into a public AI tool — it's now in the vendor's training pipeline |
| Quality and model performance may not fit the risk | The vendor's performance claims may not match your risk profile | A vendor claims 98% accuracy, but on the bank's demographic mix it drops to 71% — unfit for the risk |
| Term to evaluate | Why it matters | Plain-words example |
|---|---|---|
| Data considerations | How the vendor uses your data — training rights, retention, sub-processing | The contract must state the vendor cannot train on your customer data |
| Security / safety | Vendor's security posture, breach obligations, safety certifications | The vendor must notify the bank within 24 hours of any breach affecting its data |
| Potential for bias | Does the vendor disclose bias testing and subgroup performance? | The vendor's model card must include disaggregated performance by demographic group |
| Type of product | SaaS, API, on-prem license, open-source — each has different risk and control | An open-source model gives full control but zero support; a SaaS gives support but less control |
| Technical specs | Latency, throughput, integrations, and infrastructure requirements | The vendor's API must handle the bank's 5,000 requests-per-minute peak |
| Model performance results | Evidence of performance — benchmarks, test results, limitations | The vendor's benchmark scores are checked against the bank's own test data |
| Terms of use | Permitted uses, restrictions, liability caps, termination rights | The licence restricts use to internal purposes — deploying it for customers breaches the terms |
Lesson 2 governs what happens after release: periodic assessment, performance monitoring, incident response when it fails, and the downstream-consequence guidelines.
Continual checks of accuracy and quality against benchmarks (detailed next slide in the course).
Example: monthly scorecard of the fraud model's precision and recall against go-live baselines.The system is available and consistent when called upon.
Example: uptime above 99.5% and identical output for identical input, verified quarterly.Four tool types: audits, red teaming, threat modeling, security testing.
Example: an annual red-team exercise probes the chatbot for jailbreaks and data leaks.Accuracy drifting from baseline signals data or concept drift.
Example: fraud-model accuracy fell from 93% to 87% over a quarter — flagged for review.Odd outputs — distribution shifts, edge-case failures, or unexpected behaviour patterns.
Example: the pricing model suddenly quotes 10× normal for a product category — irregular decision.Data drift, concept drift, and label drift — the three drift types from Module 6.
Example: after a marketing campaign, the customer mix shifts and the churn model's inputs no longer match training.| Mitigation | What it means | Plain-words example |
|---|---|---|
| Use current best practices | Follow updated frameworks and standards, not last year's playbook | The team adopts the latest NIST AI RMF guidance in its monitoring playbook |
| Conduct red teaming exercises | Adversarial testing by a team that tries to break the system | A red team finds a prompt-injection path that makes the chatbot leak internal data — fixed before attackers find it |
| Consider bug bashing / bug bounties | Paid crowdsourced vulnerability discovery | The company runs a bug bounty for its AI products; a researcher earns $5,000 for a jailbreak report |
| Document using model cards and standard documentation | Keep model cards current as the system evolves | Every model version ships with an updated model card; the audit trail shows what changed and when |
| Snapshot the algorithm and its outputs | Freeze versions and output samples for forensics and comparison | Before a retrain, the team snapshots the current model and 1,000 sample outputs for later comparison |
| Monitor risks from third parties | Vendors and upstream models change under you — watch them | The vendor updates its foundation model; the team re-tests before allowing the new version into production |
| Drift | Definition | Plain-words example |
|---|---|---|
| Model drift | Overall degradation of model performance over time | The fraud model's F1 falls from 0.91 to 0.78 — the model as a whole has drifted |
| Concept drift | The relationship between inputs and outputs changes | "Free" used to mean spam; now a legitimate campaign uses it — the rule itself moved |
| Data drift | The distribution of input data changes | Card-present fraud was the norm; now 80% of fraud is online — the input mix shifted |
The plan exists for this moment — activate it, don't improvise.
Example: the chatbot leaks a customer's data; the incident playbook starts the clock on containment and notification.Know your escalation map: internal leadership, DPO, regulator, affected users.
Example: a PDPA notifiable breach must be reported to PDPC within 30 days — the team knows the route.The failure may originate upstream or downstream in the toolchain.
Example: the pricing model went wrong because a third-party data feed changed format — the incident spans the vendor too.The kill switch — a named human with a working stop mechanism.
Example: the operations lead has a single switch that takes the chatbot offline in 60 seconds, tested quarterly.Users distrust AI that was rolled out badly — reputational damage is real.
Example: a bank's buggy chatbot gave wrong answers for a week; customers now refuse to use it even after the fix.Users may over-trust the system's safety and share more than they should.
Example: a mental-health chatbot's disclaimer is buried — users believe it's confidential when it isn't fully.Second-order effects nobody planned for.
Example: an anti-fraud model blocks legitimate elderly customers' transactions, causing distress and lost trust.| Guideline | What it means | Plain-words example |
|---|---|---|
| Review potential downstream consequences early in R&D | Think about harm before the model is finished, not after | A facial-analysis project is assessed for misuse potential at the research stage — and redesigned |
| Categorise AI research and downstream consequences by risk level | Tier research by potential impact | Dual-use research is tagged high-risk; a benign recommender is tagged low-risk |
| Normalise discussions about downstream consequences of AI R&D | Make harm-checking a routine part of every project | Every project kickoff includes a 15-minute downstream-consequences discussion — standard agenda item |
| Be fully transparent and proactive in identifying negative downstream consequences | Surface harms early and publicly, don't bury them | The team publishes a known-limitations note when a model ships, including the biases they found |
| Develop common protocols for responsible product development, deployment, continuous improvement | Shared playbooks across the organisation | A single responsible-AI playbook covers every product: develop, deploy, improve — with updates in clear language, freely available |
The sheet's disclosure block — when you must tell people the AI is there:
| Obligation | What it means | Plain-words example |
|---|---|---|
| Under most laws, must disclose any AI being used | Transparency about AI use is a baseline across most data-protection and AI regimes | The bank's chatbot must tell customers they are talking to an AI, not a human |
| EU AI Act — provider and deployer requirements | Providers owe Art. 50 transparency duties (chatbots, deepfakes, emotion recognition); deployers of Annex III high-risk AI that make decisions about natural persons must inform those persons they are subject to the system (Art. 26(11)) | A hospital's triage AI must inform patients their care decisions are assisted by the system |
| Some use cases or contexts require disclosure | Even where the general law doesn't mandate it, the context may — sectoral rules, professional ethics, or contractual terms | A recruitment agency discloses AI screening to candidates because the industry code requires it |
| Disclosure may be required so users can exercise rights or legal protections | People need to know AI is involved to exercise appeal, contest, or data-protection rights | An applicant can only appeal an AI-driven rejection if they know it was AI-driven — disclosure unlocks the right |
The sheet lists six attributes of AI systems that cause failures in deployment — the same six failure modes from testing, now framed as ongoing deployment risks:
| Attribute | How it causes failure | Plain-words example |
|---|---|---|
| Brittleness | Small input changes break the model | A sticker on a stop sign flips the vision model's answer |
| Uncertainty | Overconfidence when the model should be unsure | 99% confidence on an ambiguous medical case |
| Hallucinations | Confident fabrication by generative models | A chatbot invents a refund policy that doesn't exist |
| Embedded bias | Training data skew surfaces in live decisions | A screening tool penalises group membership learned from history |
| Catastrophic forgetting | Fine-tuning destroys previously learned knowledge | After legal fine-tuning, the model forgets the maths it knew |
| False positives | Flags things that aren't real | 3× more legitimate transactions blocked as fraud |
AIAs, DPIAs, system audits, and model-card reviews are how organisations demonstrate they are answerable for their AI.
Example: the annual AI audit covers the AIA, bias tests, and compliance verification — the paper trail is the accountability.GDPR Art. 22: the right not to be subject to solely automated decisions with legal or similarly significant effects — with exceptions (contract, law, consent) and required safeguards (human intervention, right to contest).
Example: an AI-only loan rejection triggers Art. 22 rights — the applicant must be able to obtain human review and contest the decision.Governance tooling — automated inventories, drift monitoring, compliance dashboards — scales oversight and keeps pace with regulation.
Example: an automated AI-inventory tool updates risk scores nightly, so the compliance team sees drift before the regulator asks.Tap a card to flip it. Term on the front, definition + exam tip on the back. If your browser doesn't support 3D flips, the cards still work — they cross-fade instead.
TRUE / FALSE — every question drawn directly from the Module 7 sheet. Instant feedback, best score persists.
Tap each item as you master it. The goal: fill every box from memory, not recognition.