The complete Module 6 curriculum: Plan → Design → Data Collection & Prep → Model Development → Model Test & Evaluate → Monitor & Maintain → Decommission. Every topic from the course sheets, with plain-words examples so it sticks.
7 curriculum blocksL1 · L2 · L3 lessons3 LoD oversight5 V's of data prep6 testing types
PLAN→DESIGN→DATA COLLECTION & PREP→MODEL DEV→MODEL TEST & EVALUATE→DEPLOY→MONITOR & MAINTAIN→DECOMMISSION
Curriculum map: Lesson 1 (L1) = governing the planning & designing of AI models. Lesson 2 (L2) = governing data collection & use in AI design and development. Lesson 3 (L3) = governing the development of the AI model. This module follows the sheets exactly — nothing added, nothing dropped.
Scroll to begin ↓
1
Curriculum block 1 · Lesson 1 (L1)
Plan — Define, Scope & Govern
The planning stage decides whether AI is even the right answer, what problem it solves, and who is accountable. Governance failures here are baked in before a single line of code is written.
1.1 Define the business problem (L1)
Define business objectives and requirements — what outcome is the organisation actually trying to achieve?
Are there alternatives to solving the problem? — a rules engine, a human process, or no change at all may be better than AI.
Types of business problem:
Classification — assign inputs to categories (fraud / not fraud)
Regression — predict a continuous value (house price, risk score)
Recommendation — suggest items or actions (products, content)
Generation — produce new content (text, images, code)
In plain words: a bank wants to cut loan losses. The business problem is "predict which applicants will default" — that's a classification problem. The alternative question: could a simpler credit-score threshold do the job with less risk? If yes, AI may not be warranted.
1.2 Identify the use case (L1)
Identify the use case, mission, or gaps — what specific decision or task will the AI system perform, and what gap in the current process does it fill?
In plain words: "reduce customer churn" is a mission; the use case is "predict which subscribers will cancel within 30 days so retention staff can call them first."
1.3 Determine scope (L1)
Impact — how many people are affected, how consequential is the decision?
Effort — cost, time, data, and talent required to build and maintain it
Fit — does AI actually fit the problem and the organisation's capabilities?
Include compliance requirements — PDPA, GDPR, EU AI Act, sectoral rules (e.g., MAS for finance)
In plain words: a hospital's AI triage tool affects every patient (high impact), needs 18 months of clinical validation (high effort), and must satisfy medical-device regulation (compliance). Scope = the tool is justified only if impact × fit outweighs effort × compliance cost.
1.4 Evaluate data availability (L1)
Identify types of data accessible — internal records, public data, purchased data
Assess accuracy, sufficiency, and relevance to the identified use case
Explore additional data sources if necessary to ensure alignment with the AI system's requirements
In plain words: the churn model needs 12 months of subscription history. The team finds only 6 months exist, and billing data is full of gaps — so the plan must either source more data or narrow the use case. Data availability gates the project.
1.5 Governance existence structure (L1)
Determine if an AI governance framework exists; identify the individuals responsible
Appoint a champion — a named person who drives the governance agenda
Stakeholder engagement and feedback — map who is affected and give them a voice
Operational controls — answer these five questions:
Who has operational responsibility?
Who conducts audits and reviews?
Who responds to feedback and appeals?
Who elevates issues?
Who owns the kill switch?
Exam anchor — the five operational-control questionsThe sheet lists exactly five: operational responsibility, audits & reviews, feedback & appeals, issue elevation, and the kill switch owner. The exam tests that each control has a named person — not a committee, not "the team".[Module 6 sheet, p.430]
1.6 Human oversight of AI development — the 3 Lines of Defence
Line
Who
Role
Plain-words example
1st line
Management & product owners
Own the system day-to-day: design, build, operate, and first-line risk controls
The product owner of the fraud model owns its daily performance and escalates anomalies
2nd line
Risk & compliance
Oversee risk frameworks, set policies, challenge the first line, monitor compliance
Compliance reviews the model's PDPA basis and risk signs off the go-live
3rd line
Internal audit
Independent assurance — audits the whole governance system
Internal audit tests whether the model's sign-off records actually exist and were followed
In plain words — the 3 LoD: the players (1st line) run the game, the referees (2nd line) set and enforce the rules, and the match reviewers (3rd line) independently check that the referees and players both did their jobs.
2
Curriculum block 2 · Lesson 1 (L1)
Design — Assess Risk Before You Build
Design is where risk assessment happens before the build: algorithmic impact assessment, risk strategies, and governance across the whole data lifecycle.
2.1 Perform an algorithmic impact assessment (L1)
Algorithmic Impact Assessment (AIA) — a structured assessment of the potential impacts of an algorithmic system on individuals and groups, before deployment. The sheet flags the comparison: AIA vs PIA / DPIA.
AIA — Algorithmic Impact Assessment
PIA / DPIA — Privacy / Data Protection Impact Assessment
Focus
Impacts of the algorithm: fairness, bias, rights, societal effects
Impacts on personal data: privacy risks, lawful basis, data flows
Origin
Canada's Directive on Automated Decision-Making; emerging AI regimes
GDPR Art. 35; PDPC PIA practice
Trigger
Automated decision systems with significant impacts
Processing likely to result in high risk to individuals
Output
Impact level, safeguards, mitigation plan
Risk register, mitigations, sign-off
Exam tell
"Algorithm", "automated decision", "fairness"
"Personal data", "privacy", "Art. 35"
In plain words: the DPIA asks "what could go wrong with people's data?" The AIA asks "what could go wrong with people's lives?" A hiring model needs both — one for the résumé data, one for the career impact of the decisions.
2.2 Risk assessment strategies (L1)
Use case evaluation
Assess the specific use case's risk profile before building.
Example: a hiring tool is higher-risk than a product-recommendation tool — different scrutiny from day one.
Stakeholder mapping
Identify everyone affected, including non-users who are still impacted.
Example: a credit model affects applicants, guarantors, and even people whose data appears in training records.
Probability / severity harms matrix
Plot each harm by likelihood and impact to prioritise controls.
Example: "wrong loan rejection" = high severity, medium probability → top of the matrix → mandatory human review.
Risk mitigation hierarchy
Prefer eliminating risk over reducing it, and reducing it over transferring it.
Example: eliminate the risk by not using a sensitive attribute; reduce it with oversight; transfer residual risk with insurance.
Benchmarking
Compare against industry baselines and prior systems.
Example: the new fraud model's false-positive rate is benchmarked against the incumbent rules engine before go-live.
Pre-deployment pilots
Test in a controlled real-world setting before full rollout.
Example: the chatbot runs with 5% of customers for two weeks, with escalation metrics armed.
2.3 Governance across the AI data lifecycle
The sheet maps governance to every stage of the data lifecycle — not just training:
Recall below 80% for 3 days auto-flags the model for review
De-commissioning
Residual risk, data disposal, archiving, documentation
Retired model's training data is deleted per retention policy; the model card is archived
Exam anchor — governance is lifecycle-wideGovernance does not stop at training. The sheet explicitly spans training → evaluation → deployment → monitoring → de-commissioning. A question that treats governance as "just the training data" is wrong.[Module 6 sheet, p.430]
3
Curriculum block 3 · Lesson 1 (L1)
Data Collection & Preparation
Models are frozen data decisions. This block covers gathering the right data, lineage, quality, formats, the 5 V's of data prep, and the privacy toolkit.
3.1 Gather the right data (L1)
What data is required? — features and labels needed for the use case
How much is needed? — sufficiency for the model type and population
How is it collected? — lawful basis, consent, notices, collection methods
Where is it stored? — location, security, retention, access controls
In plain words: the churn model needs 12 months of subscription history (what), at least 100k records (how much), collected under the PDPA consent notice (how), stored in the EU region with access logging (where).
3.2 Data lineage and provenance
Data lineage and provenance track the flow of data — its history and source. For any record you can answer: where did it come from, under what basis, transformed how, by whom, when?
In plain words: lineage is the data's biography. When a regulator asks "where did this training record come from?", the lineage log answers in one screen — not after a three-week investigation.
3.3 Data quality considerations
Assess data for training algorithms: accurate, representative, non-biased, and collected with proper statistical sampling.
Quality factor
What it means
Plain-words example
Accurate
Data reflects reality; labels are correct
A mislabelled "not fraud" transaction teaches the model to miss fraud
Representative
Data covers the deployment population
A face-recognition model trained only on light skin fails on darker skin
Non-biased
No systematic skew against groups
Historical hiring data that under-represents women bakes the skew into the model
Statistical sampling
Data collected via sound sampling methods
A convenience sample of power users misrepresents the whole customer base
3.4 Data formats
Dimension
Options
Plain-words example
Structure
Structured (tables, rows, columns) vs unstructured (text, images, audio, video)
Structured: a spreadsheet of transactions. Unstructured: customer emails, support tickets, photos
Recency
Static (snapshot, doesn't change) vs streaming (continuous updates)
Static: a one-time census extract. Streaming: live sensor feeds from factory machines
3.5 Wrangling & preparing data — the 5 V's
Volume
How much data — scale of records and storage.
Example: 50 million transactions vs 5,000 — different pipelines, different governance.
Velocity
Update rate and speed of arrival.
Example: real-time fraud scoring needs streaming data; a quarterly churn model can use batch files.
Variety
Different types and formats of data.
Example: combining structured CRM rows with unstructured call transcripts and images.
Veracity
Accuracy and trustworthiness of the data.
Example: self-reported income is less trustworthy than bank-verified income.
Value
The right data for the outcome — relevance over volume.
Exam memory hook — the 5 V'sVolume (how much), Velocity (how fast), Variety (how many kinds), Veracity (how true), Value (how relevant). The sheet's exact definitions: Volume = how much; Velocity = update rate and rate; Variety = different types/formats; Veracity = accuracy/trustworthiness; Value = right data for outcome.[Module 6 sheet, p.430]
3.5b The 5 V's — match the scenario
Five scenarios, five V's. Pick the right one — instant feedback, best score persists.
In plain words: the team cleans the transaction data (removes 4,000 duplicate rows), labels 12,000 fraud cases, pseudonymises customer IDs, drops 30 fields the model doesn't need (minimisation), and trains with differential privacy so no single record can be inferred.
4
Curriculum block 4 · Lesson 3 (L3)
Model Development
The build stage: feature engineering, model training, system architecture, and the train/validate/test discipline. This is Lesson 3 (L3) territory — governing the development of the AI model itself.
4.1 Define model features (L3)
Ensure consistency between training and testing datasets — same features, same definitions, same preprocessing
Collaborate with subject matter experts — select relevant features and perform feature engineering with domain knowledge
Transform raw data into meaningful inputs — the essence of feature engineering
In plain words: for the churn model, the SME says "customers who log in less than twice a week churn" — so the team engineers a "login frequency" feature. The same feature must exist identically in training and test data, or the test results are meaningless.
4.2 Feature engineering — why it matters
Improve model performance
Better features beat bigger models — the right signal in the right shape.
Example: "days since last login" predicts churn better than raw login counts.
Reduce computational costs
Fewer, better features mean cheaper training and inference.
Example: dropping 200 redundant columns cuts training time by 60%.
Boost model explainability
Meaningful features are easier to explain than opaque engineered ones.
Example: "average order value" is explainable to a regulator; "feature_47" is not.
Exam anchor — the feature rulesThe sheet lists four: work with subject matter experts, use the same features for training and testing, avoid unnecessary features, and use feature flags for rollbacks (so a bad feature can be switched off without redeploying the model).[Module 6 sheet, p.430]
4.3 Model training
Evaluation metrics — define accuracy, precision, recall, F1, and bias metrics up front
Testing on new data — held-out data the model has never seen
Test and validate in real-world scenarios — pilots, shadow runs, edge cases
In plain words: the team trains the churn model, then scores it on 20,000 records it never saw during training. Then they run it in shadow mode on live traffic for two weeks before anyone acts on its predictions.
4.4 Determine system architecture (L3)
Select the model — balance accuracy vs interpretability; start with the most interpretable model that meets performance requirements
Train, test, validate — the three-way split:
Training data — to train the ML model (fit parameters)
Validation data — to fine-tune parameters and prevent overfitting before final evaluation using the test dataset
Testing data — to test the ML model's performance with new data, ensuring the model generalises well
Create challenger models — build alternative models and compare them against the incumbent before choosing
Split
Purpose (sheet definition)
Plain-words example
Training
To train the ML model
The model learns patterns from 70% of the data
Validation
To fine-tune parameters and prevent overfitting before final evaluation using the test dataset
The team tunes hyperparameters on 15% of the data, watching for the point where the model starts memorising
Testing
To test performance with new data to ensure the model generalises well
The final 15% is touched once — if the model scores well here, it generalises
In plain words — challenger models: instead of trusting one model, the team builds three: a logistic regression (interpretable), a gradient-boosted tree (accurate), and a challenger neural net. The champion is chosen on test-set performance and explainability — and the challengers stay ready to swap in if the champion degrades.
4.5 Training the AI model
Data representative, fair, compliant — the training data must reflect the population, avoid bias, and meet legal requirements
Model objectives — define what the model optimises, and what it should NOT optimise (reward misspecification risk)
Training vs fine-tuning — training from scratch on your data vs adapting a pre-trained foundation model
In plain words: the team fine-tunes a pre-trained language model on 5,000 curated support tickets (fine-tuning) rather than training from scratch — cheaper and faster, but they inherit the base model's biases and must re-evaluate.
4.6 Human oversight of AI development — 3 LoD (revisited)
The sheet repeats the 3 Lines of Defence at the model-development stage — the same structure applies throughout the lifecycle:
1st line — management & product owners
2nd line — risk & compliance
3rd line — internal audit
Exam noteThe 3 LoD appears in both the Plan block and the Model Dev block of the sheets — expect it in scenario questions about who approves, who challenges, and who audits.[Module 6 sheets, p.430–431]
5
Curriculum block 5 · Lesson 1 (L1)
Model Test & Evaluate
Testing is where claims become evidence. The sheet covers the six testing types, metric and threshold surveillance, and the system-audit checklist.
5.1 Test and validate the AI system (L1) — the six testing types
Brittleness
The model breaks or degrades on small input variations it wasn't trained on.
Example: a vision model misclassifies a stop sign when a sticker is added — the adversarial-example failure.
Uncertainty
The model is overconfident when it should be unsure — confidence scores don't match reality.
Example: a medical triage model says 99% confident on a case that's actually ambiguous.
Hallucinations
Generative models produce confident but fabricated content.
Example: a support chatbot invents a refund policy that doesn't exist.
Embedded bias
Bias baked in from training data or design choices surfaces in outputs.
Example: a résumé screener penalises "women's" club memberships — the Amazon 2018 failure.
Catastrophic forgetting
Fine-tuning on new data destroys previously learned knowledge.
Example: fine-tuning a model on legal text makes it forget how to do maths it previously handled.
False positives
The model flags things that aren't real — costly in fraud, safety, and moderation contexts.
Example: a fraud model blocks 3× more legitimate transactions than the old system.
Exam memory hook — the six testing typesBrittleness, Uncertainty, Hallucinations, Embedded bias, Catastrophic forgetting, False positives — "BUH-ECF". Each one is a failure mode the exam describes in a scenario; you name the type.[Module 6 sheet, p.431]
5.2 Metric and threshold surveillance
Establish measures — accuracy, precision, recall, F1 score, and bias metrics
Setting thresholds — performance benchmarks that trigger alerts, retraining, or rollback
Metric
What it measures
Plain-words example
Accuracy
Overall share of correct predictions
95% of fraud flags were right — but see precision/recall for the real picture
Precision
Of items flagged, how many are real
80% of flagged transactions were actually fraud — 20% were false alarms
Recall
Of real cases, how many were caught
The model caught 70% of actual fraud — 30% slipped through
F1 score
Harmonic mean of precision and recall
A single number balancing false alarms against missed cases
Bias metrics
Disparity across groups (e.g., equalized odds, demographic parity)
Approval rates for group A vs group B — a 15-point gap triggers review
In plain words — thresholds: the team sets "recall ≥ 85% and precision ≥ 60%" as the go-live bar, and "recall < 80% for 3 consecutive days" as the retraining trigger. Thresholds are set before deployment, not after the damage.
5.3 System audits
The sheet's audit checklist — six items that make up a full system audit:
Algorithmic impact assessments
Re-run or verify the AIA covers the deployed system's actual impacts.
Example: the audit confirms the hiring model's AIA was updated when the tool expanded to a new role type.
Bias and fairness testing
Independent subgroup analysis on live data.
Example: the auditor runs its own demographic breakdown rather than trusting the vendor's report.
Explainability and interpretability evaluations
Verify the model's decisions can be explained to stakeholders.
Example: can the team explain why a specific loan was rejected, in plain language?
Data governance and quality review
Check lineage, quality, and lawful basis of the data in use.
Example: the audit traces 50 random training records to their sources and consent bases.
Verification of compliance with regulations
Confirm the system meets applicable law (PDPA, GDPR, EU AI Act).
Example: the chatbot's Art. 50 AI disclosure is verified present and correct.
Confirmation of accountability and human oversight
Verify named owners and oversight controls actually operate.
Example: the audit confirms the kill-switch owner is named, trained, and reachable.
5.4 Test and validate — the discipline (L1)
Align testing to the use case — test what the system will actually do in its real context
Conduct repeatability assessments — same input, same output, every time (or documented variance)
Adversarial testing — probe with malicious or edge-case inputs
Threat modeling — identify how an attacker could abuse the system
Establish multiple layers of mitigation — defence in depth, not a single control
Review previous incidents — learn from what already went wrong, internally and industry-wide
Resources — high risk vs low risk — allocate testing depth proportionally to risk
In plain words: a high-risk credit model gets adversarial testing, threat modeling, and a full incident review before go-live; a low-risk product recommender gets a lighter but still documented test pass. Testing effort follows risk.
Deployment is the start, not the end. This block covers how to monitor, what to document, and how to communicate — to regulators and consumers.
6.1 Manage and monitor AI systems (L1)
How to monitor and maintain — the operational process, dashboards, and cadence
Understand purposes and risks — monitoring must track the system's intended purpose and its risk profile
Inventory of AI systems with risk score — every system listed, each with a risk rating that drives monitoring depth
Understand organisational security protocols — industry standards — align with ISO 27001, NIST, and sectoral baselines
In plain words: the bank's AI inventory lists 14 systems, each scored high/medium/low risk. The high-risk credit model gets daily drift monitoring and monthly audits; the low-risk recommender gets weekly checks. Security follows the same industry standards as the rest of the bank.
Example: the credit model's card states it is not validated for self-employed applicants — a documented out-of-scope use.
Create counterfactual explanations (CFE)
Explain a decision by showing what would have changed it: "if your income were $1,000 higher, you would have been approved."
Example: a denied applicant is told the single factor that would flip the decision — the GDPR Art. 22-style explainability tool.
Remediation of adverse impacts
Document how harms are corrected: appeals, manual review, compensation, retraining.
Example: the bank's remediation playbook covers wrong rejections: 24-hour manual review and retroactive correction.
Method of deployment
Record the platform used: cloud, on-site, or hybrid.
Example: the model runs on-premise for data-sovereignty reasons; the deployment method is documented for audit.
Exam anchor — counterfactual explanationsA CFE answers "what would need to change for a different outcome?" — the most practical explainability tool for individual decisions. The exam loves pairing it with model cards (system-level) as the two documentation artefacts.[Module 6 sheet, p.431]
6.3 Create a communication plan (L1)
Audience
Focus (sheet)
Plain-words example
Regulators
Consider compliance and disclosure obligations; ensure transparency and explainability; document data and risk classifications
The bank files its AIA, model card, and risk classification with the regulator before the credit model goes live
Consumers
Focus on transparency; communicate the functionality of the AI system
The chatbot tells users they are talking to an AI, what it can do, and how to reach a human
In plain words — two audiences, two messages: regulators get the evidence (compliance, risk classifications, explainability documentation). Consumers get the plain truth (this is AI, here's what it does, here's how to appeal).
The end of the lifecycle is governed too. The sheet lists ten de-commissioning concerns — each one an exam scenario waiting to happen.
Residual risk management
Risks don't vanish when the system is switched off — data, dependencies, and decisions live on.
Example: a retired credit model's past decisions still affect customers; appeals must still be handled.
Data disposal and retention
Delete or retain per policy and law — not all data can be deleted (regulatory retention).
Example: training data is deleted after 90 days, but transaction records are kept 7 years per MAS rules.
Model archiving
Preserve the model, its version, and its documentation for future reference and audits.
Example: the model binary, model card, and test results are archived so a 2028 audit can still inspect the 2026 system.
Documentation obligations
De-commissioning itself must be documented — who decided, when, and why.
Example: the decommissioning record notes the model was retired due to persistent drift and names the approver.
Communications
Tell affected parties the system is being retired — users, consumers, and regulators as needed.
Example: customers are emailed that the AI loan pre-approval service is ending and how to access alternatives.
Knowledge retention
Capture the team's knowledge before it walks out the door.
Example: the model owner writes a handover note covering quirks, failure modes, and lessons learned.
Security risks
Retired systems can be attacked — orphaned APIs, exposed data stores, forgotten credentials.
Example: the retired model's API endpoint is decommissioned, not just ignored — an unpatched endpoint is a breach waiting to happen.
Downstream dependencies
Other systems may depend on the model's outputs — map them before switching off.
Example: the churn model feeds the retention team's dashboard; retiring it without a replacement breaks the dashboard.
Third-party AI
If the system is a vendor product, de-commissioning involves the vendor: data return, deletion, contract exit.
Example: the vendor contract requires data deletion certificates within 30 days of termination.
Governance checklist
Run a formal checklist before, during, and after de-commissioning — the sheet's own governance tool.
Example: the checklist covers: approver named? data disposed? model archived? dependencies mapped? parties notified?
Exam anchor — de-commissioning is not "switch it off"The sheet's ten items are the exam's favourite "what did they forget?" scenario. The classic misses: downstream dependencies (something else still uses the output) and security risks (the orphaned endpoint).[Module 6 sheet, p.431]
8
Self-test · Curriculum-based exam traps
Eight Exam Traps
TRUE / FALSE — every question drawn directly from the Module 6 sheets. Instant feedback, best score persists.
9
Exam-day readiness
Before Thursday — The Checklist
Tap each item as you master it. The goal: fill every box from memory, not recognition.
✓Recite the 7 lifecycle blocks: Plan → Design → Data Collection & Prep → Model Dev → Test & Evaluate → Monitor & Maintain → Decommission.
✓Name the 4 business problem types: classification, regression, recommendation, generation — with one example each.