Free job simulation · AI and data
Machine learning engineer job simulation: train a model and catch target leakage
A machine learning engineer job simulation is a guided work exercise where you prepare data, train and evaluate a model, and document what you found, including mistakes such as target leakage that make results look better than reality. It is practice evidence, not employment. This version follows a member cancellation model for a simulated fitness chain, and ends with a release pack.
You join Tarn Valley Data Science Ltd (simulated company), a applied machine learning consultancy in Leeds. Days 1 to 5 are free, with about 10 h 30 min of tickets. Premium continues to day 20. It is practice, not employment.
Days 1 to 5 are free. You need a free account and no card. Create an account.
- Free days
- 1 to 5
- Free tickets
- about 10 h 30 min
- Project documents
- 6
- Certificate, optional
- £14.99
The project
You join Tarn Valley Data Science Ltd (simulated company)
Tarn Valley Data Science Ltd (simulated company) is a applied machine learning consultancy in Leeds. You help its team deliver a project for Pennine Fitness Clubs Ltd, at Member retention model for twelve clubs (ticket MLE-088).
You are the junior ML engineer on a member-cancellation model for a chain of fitness clubs. The Senior ML Engineer gives you a ticket, a training extract with planted problems, a plan with mistakes in it and a log of experiments, one of which scored 0.99. Over four weeks you count what is really in the data, prove where the too-good score came from, find that 97 percent accuracy is the baseline, choose a threshold with costs and a call capacity, handle a change request, and then deal with the model after release: a fairness gap, a changed door system, drift and a deletion request.
What you will do
- Profile a dataset for a datasheet and review a training plan for leakage, splits and the wrong metric
- Prove where a too-good AUC comes from and explain why 97 percent accuracy on 3 percent positives means nothing
- Choose a threshold with a cost matrix and the capacity of the people who act on the list
- Check a model by age band, write the model card honestly and refuse to hide a gap
- Read monitoring after release, find the upstream change that caused drift, and hand over with numeric retrain triggers
The people you deal with
- Head of Data Sciencecompany
- Senior ML Engineercompany
- Delivery Managercompany
- Client Retention Managerclient
- Client Operations Directorclient
- Client Data Ownerclient
- Platform Engineerclient
- Client Compliance Leadclient
The company, client and site are fictional. The people are fictional too.
Look inside
What the simulator looks like
The workplace, your tickets, the documents and the team chat, as a learner sees them.
Start in the workplace, with the team around you. 
Today: the morning email and your tickets for the day. 
Work: every ticket, by day, with its status. 
A ticket, written the way a colleague would write it. 
A project document with its number, revision and status. 
Ask a colleague. They are AI colleagues: they help you think, they do not hand out answers.
Screenshots from the simulator. Company, people and data are fictional. Portraits are AI-generated.
Meet the team
The colleagues you work with
You can read their messages and ask them questions. They are AI colleagues, clearly labelled. They help you think a ticket through; they do not hand out its answer.
- IMIngrid MortlockHead of Data ScienceSimulated colleague, company side
- RMRahul MenonSenior ML EngineerSimulated colleague, company side
- OPOwen PughDelivery ManagerSimulated colleague, company side
- RQRosemary QuayleClient Compliance LeadSimulated colleague, client side
- BRBethan ReesClient Retention ManagerSimulated colleague, client side
- MTMalcolm TindallClient Operations DirectorSimulated colleague, client side
- STSapna TrivediClient Data OwnerSimulated colleague, client side
- JPJoss PenningtonPlatform EngineerSimulated colleague, client side
Fictional people. Portraits are AI-generated.
What you get
What you walk away with
Build and check a member cancellation model: datasheet, leakage, metrics, thresholds, fairness and a release pack.
What you produce
Datasheet for the member dataset
Dataset documentation
Review comments on the feature spec and training plan
Design review
Leakage analysis of the 0.99 experiment
Experiment analysis
Experiment record and end-of-week status
Experiment log
Model card review
Governance reviewPremium
Impact note for a change request
Change controlPremium
Risk note: Map, Measure, Manage, Govern
Risk assessmentPremium
Release pack for the member cancellation model
CapstonePremium
Skills you practise
- Evaluation and metrics
- AI governance and documentation
- Machine learning modelling
- Written communication
- Applying standards
- Data cleaning and quality
- Working with clients and stakeholders
- Analysis and statistics
Roles to look for
- Machine Learning Engineer
- Junior Data Scientist
- Applied Machine Learning Engineer
- Data Scientist
117 live openings on the UK jobs board, checked 24 Oct 2026. See the jobs
Your proof
- A certificate anyone can verify. Optional, and included in Premium.
- An entry on your EDWartens profile under Work experience, marked as a job simulation.
- A work record with your skill scores, with Premium.
Free courses that go with it
What you can talk about at interview
- You can explain how you traced a score of 0.99 to leakage and what you changed.
- You can describe why accuracy misleads when only a few per cent of cases are positive.
- In Premium, you can walk through a threshold chosen from a cost matrix, a fairness check by age band and a release pack.
Day by day
Days 1 to 5, ticket by ticket
Self-paced. A day opens when you hand in the previous day's tickets. Every ticket is checked, and you see the model answer after it.
Day 1
about 1 h 50 min- Induction: team standards for machine learningQuiz · 40 min
- Datasheet: count what is really in members.csvTable · 1 h 10 min
Day 2
about 2 h 20 min- Review the feature spec and training planDocument review · 1 h 20 min
- Lab: data health check on the customer tableHand-in · 1 h
Day 3
about 2 h 20 min- EXP-015: prove where the 0.99 comes fromCalculation · 1 h 10 min
- Lab: train and evaluate without leakageHand-in · 1 h 10 min
Day 4
about 1 h 50 min- What does 97 % accuracy mean on 3 % positives?Calculation · 1 h 15 min
- "97 % is excellent. Switch it on tomorrow, and use deep learning."Decision · 35 min
Day 5
about 2 h 10 min- Lab: choose the metric and the thresholdHand-in · 1 h 10 min
- EXP-017 record and end-of-week statusWriting · 1 h
Premium: days 6 to 20about 23 h more
After day 5 the project goes on with a change request, site work, faults, a handover and a capstone reviewed by an engineer.
Day 6
about 1 h 40 min- Choose the threshold with the cost matrix and the call capacityTable · 1 h 40 min
Day 7
about 1 h 30 min- Review the draft model card v1.2Document review · 1 h 30 min
Day 8
about 1 h 50 min- Impact note for change request CR-02Writing · 1 h 50 min
Day 9
about 1 h 20 min- Lab: overfitting, tune the treeHand-in · 1 h 20 min
Day 10
about 1 h 10 min- Experiment log: gaps and candidatesTable · 1 h 10 min
Day 11
about 1 h 10 min- Measure the door system change in the training fileCalculation · 1 h 10 min
Day 12
about 1 h 30 min- Drift after release: list size, recall, precisionCalculation · 1 h 30 min
Day 13
about 1 h 20 min- Lab: fairness checkHand-in · 1 h 20 min
Day 14
about 1 h 20 min- Results by age band on our own modelCalculation · 1 h 20 min
Day 15
about 40 min- "Leave the age slice out of the board report"Decision · 40 min
Day 16
about 2 h- Risk note: Map, Measure, Manage, GovernWriting · 1 h 20 min
- A member asks us to delete her dataDecision · 40 min
Day 17
about 1 h 30 min- Review pull request 31: the v1.2 training pipelineDocument review · 1 h 30 min
Day 18
about 4 h- Capstone: the release pack for the member cancellation modelHand-in · 4 h
Day 19
about 1 h 10 min- Handover note to the platform teamWriting · 1 h 10 min
Day 20
about 50 min- A conversation about your own workViva · 50 min
Real documents
The documents you will work with
Each one has a number, a revision and an approver, the way a project document does. You read them, check them and write your own.
MLE-TKT-088
Ticket MLE-088: churn model v1 for the retention team
Rev ATicket
MLE-STD-001
Data science team standards (one page)
Rev CStandard
MLE-FRM-001
Problem framing: member cancellation within 30 days
Rev AProblem framing
MLE-DSH-001
Datasheet for the member snapshot dataset
Rev 0.2Datasheet
MLE-FSP-001
Feature specification and training plan v0.4
Rev AFeature spec
MLE-EXP-001
Experiment log
Rev CExperiment log
Documents that arrive in Premiumchange requests, records, handover
MLE-MCD-001
Model card v1.2: member cancellation model (EXP-017)
Rev AModel cardPremium
MLE-CHG-002
Change request CR-02: predict cancellation within 60 days and add the Family plan
Rev AChange requestPremium
MLE-FRM-001
Problem framing: member cancellation within 60 days (model v2)
Rev BProblem framingPremium
MLE-PCN-014
Platform change notice PCN-14: door system upgrade
Rev AChange noticePremium
MLE-MON-001
Monitoring plan and weekly monitoring file
Rev AMonitoring planPremium
MLE-PRQ-031
Pull request 31: training pipeline for model v1.2
Rev APull requestPremium
MLE-HOV-001
Release and handover pack requirements
Rev AHandover indexPremium
MLE-EXP-001
Experiment log, with the tree-depth runs
Rev DExperiment logPremium
Tools
In the industry, and here
| In the industry | Here |
|---|---|
| Python with scikit-learn and a notebook | Your own notebook or spreadsheet on the downloaded files; every answer can be found with either |
| Experiment tracking (run ids, parameters, metrics) | A numbered experiment log document and the experiment record you write |
| Model cards, datasheets and a risk register | Reviewed documents with numbers and revisions |
| Monitoring dashboard and ticket tracker | A weekly monitoring file and numbered tickets in the inbox |
Standards
The standards the work follows
- CRISP-DM
- Model Cards for Model Reporting
- NIST AI RMF 1.0
In the UK
The Machine Learning Engineer role in a UK setting
Who employs this kind of engineer: Data science teams in retail, finance, health, energy, leisure and the public sector, plus applied AI consultancies.
Where: Roles are advertised across the UK, with larger groups in London, Manchester, Edinburgh, Cambridge, Bristol and Leeds, and many are hybrid or remote. This is context, not a statement about vacancies.
UK standards and regulations to know about
UK GDPR and the Data Protection Act 2018
The rules for personal data. The ICO publishes guidance, including on AI and data protection.
Equality Act 2010
Protected characteristics such as age matter when a model affects people, which is why the fairness check by age band is in the project.
ICO guidance on AI and data protection
Explains fairness, transparency and accountability for AI that uses personal data.
Model cards
A common way to document what a model is for, how it was tested and where it fails.
These are named so that you recognise them. A job simulation does not teach them as a qualification, the tasks use simplified data, and you should always read the current edition of anything you rely on.
Your proof, optional
A certificate you can verify
The free days are free. The graded, verifiable certificate is an optional extra, and it is included in Premium.
- A graded, verifiable certificate with your score and band
- A public verification page and QR code
- One-click add to LinkedIn as a licence or certification
- Issued by WARTENS LTD. It is a training certificate, not an accredited qualification, and the company is simulated
Optional: £14.99 including VAT, once you pass.
Job Simulation Certificate
Your name
completed the Machine Learning Engineer job simulation at Tarn Valley Data Science Ltd (simulated company). Your score and band are shown on the real certificate.
- Number
- EDW-JS-0000
- Issued by
- WARTENS LTD
Not an accredited qualification. A verification page and QR code are printed on the real certificate.
Prepare
Free courses and labs that get you ready
Linked from the tickets that need them. You can read them before you start.
Free courses
- Machine Learning with Python
- Python for AI and Engineering Data
- Predictive Maintenance with Machine Learning
- AI Essentials for Work: ChatGPT, Claude, Gemini and Copilot
- Deep Learning with PyTorch for Engineers
- ISO 42001 and AI Governance
- Docker and Containers for Automation Engineers
A course that goes with this role
AI Module
A paid, self-paced course in AI and machine learning for industry: 83 lessons, 5 graded projects and career sessions. It is a separate product: you do not need it to finish the job simulation, and it does not lead to a job. The certificate is a certificate of completion, not a regulated qualification.
- Price
- £2,688 including VAT (£2,240 + VAT); or start with a £100 deposit
- Format
- Self-paced online, 12 months' access
About the Machine Learning Engineer role
Why your first score is too good
Target leakage happens when training data contains information that will not be available at prediction time. The model looks excellent in validation and fails in production.
In the simulation an experiment reports a score of 0.99, and you prove where it comes from. You then retrain without leakage in the hands-on exercise and see the honest figure. Fitting preprocessing only on training data, checking that each feature exists at prediction time and splitting carefully are the habits being drilled.
Write down the proof, not just the fix, so a reviewer can see how you found it.
Metrics tied to cost, not to a headline number
When only a small share of members cancel, a model that says nobody will cancel looks highly accurate and is useless. The simulation makes you work through that case with accuracy on rare positives.
You then choose a threshold with a cost matrix and the capacity of the team who call members, so the list the model produces is one people can act on. The costs are the learner's own table, and no figure is invented for you.
The decision is written down with its numbers, so that others can check it.
The model card and the risk note
A model card says what a model is for, what data it learned from, how it performed and where it should not be used. Reviewing a draft card, you check it against the numbers.
A fairness check by age band may show a gap, and a manager may suggest leaving it out of the board report. The correct response is to report it. You also write a risk note using the Map, Measure, Manage and Govern functions of the NIST AI Risk Management Framework, and handle a member's request to delete her data.
After release: drift and retrain triggers
Models decay when the world changes. A door system upgrade changes a feature in the training file, and the monitoring shows list size, recall and precision moving.
You find the upstream change that caused the drift and hand over with numeric retrain triggers. Reviewing a pull request for the training pipeline shows the engineering side of the role. The documents include a monitoring plan and an experiment log you keep tidy.
Retrain triggers written as numbers are easy to check later, which is why the handover asks for them.
Going further with run-to-failure data
For an engineering flavour, public run-to-failure datasets are a good next step after this simulation. Predictive maintenance is one of the most common machine learning projects in industry.
The AI Workbench lab and the linked free courses cover machine learning with Python and predictive maintenance. The related roles are the data analyst, the computer vision engineer and the LLM engineer.
Keep the model card, the leakage proof and the threshold decision. They are specific, checkable pieces of work, and a reviewer can read them in minutes to see how you think about evidence.
Keep going
Related job simulations and courses
Data Analyst
Clean real-looking data, answer the business question and present it.
Computer Vision Engineer
Specify, label and evaluate an inspection model on a production line.
AI and LLM Application Engineer
Build and evaluate an LLM assistant, then handle a prompt-injection incident.
- Data and AI job simulations
- Free courses
- Browse jobs
- Virtual work experience in the UK
- Job simulation questions
Content reviewed 19 Oct 2026; UK context added 21 Oct 2026. Practice, not employment.
Questions
Quick answers
What is target leakage?
It happens when training data contains information that will not be available at prediction time, so the model looks excellent in validation and fails in production. Reviewers find it easier to forgive a modest honest score than a perfect score that falls apart.
How do you detect data leakage?
Check for features derived from the target, confirm each feature exists at prediction time, and fit preprocessing on training data only. A pipeline helps enforce the last step.
What does an ML engineer do day to day?
They prepare data, train and evaluate models, document decisions and move models into use. The simulation covers data checks, training, evaluation, documentation and monitoring.
What should an ML portfolio project include?
A clear problem, honest validation, a metric chosen for the business cost, failure analysis and a short model card. Reviewers read these in minutes, so keep them short and checkable.
Is a machine learning job simulation worth it?
As practice, yes. As proof of experience, no. The strongest evidence is a documented project where you caught a problem such as leakage.
What maths and coding do I need?
Python and basic statistics. In the simulation every answer can be found with a notebook or with a spreadsheet on the downloaded files.
What is the difference between an ML engineer and a data scientist?
Data scientists tend to focus on analysis and modelling, and ML engineers on reliable training and deployment. Titles overlap, so check the job post.
What do machine learning engineers earn in the UK?
UK advertisements for machine learning engineers show a median salary of £65,000, with the middle half between £48,750 and £87,500, over the six months to 19 October 2026. This covers UK IT vacancies only, and pay in other countries varies.
Source: ITJobsWatch, Machine Learning Engineer (UK IT vacancies) (opens in a new tab), checked 19 Oct 2026.
Do I need to code?
No. The tasks are checked as numbers, decisions and writing, and the hands-on exercise is done in EDWartens Lab. Python helps with the CSV files, but a spreadsheet works for every calculation.
Is this a real company and a real model?
No. Tarn Valley Data Science and Pennine Fitness Clubs are made-up, the data is synthetic and generated with planted problems. The documents, tickets and situations follow what a junior ML engineer meets on a real project.
How long does it take?
The free week is about ten hours of work spread over five days. The full Premium edition runs over twenty working days and roughly thirty to forty hours, at your own pace.
Which libraries or cloud do I have to use?
None. The content is vendor-neutral. The answers are checked as numbers, so any tool that can count and rank works.
What do I get at the end?
A work record with your scores by task and skill, which shows on your EDWartens profile under Work experience, marked as a job simulation.
Is the Machine Learning Engineer job simulation free?
Yes, to start. Days 1 to 5 are free. The certificate (£14.99) and the Premium edition (£53.99) are optional, and both prices include VAT.
Does it count as work experience?
No. It is a training project at a simulated company, not employment, an internship or a placement. A job simulation is a realistic training project at a simulated company. It is not employment, an internship or a work placement, and it is not a job offer. Hours are active hours the learner logged on EDWartens UK.
A job simulation is a realistic training project at a simulated company. It is not employment, an internship or a work placement, and it is not a job offer. Hours are active hours the learner logged on EDWartens UK.
Start the Machine Learning Engineer job simulation today
Days 1 to 5 are free. No card to start. Practice, not employment.