Data & AI Project
Exploratory Data Analysis
Exploratory data analysis is where a data role really starts: loading a messy file, working out what is wrong with it, and finding what it can honestly support. The deliverable is not a notebook — it is a conclusion someone could act on, with the caveats stated.
The brief
Pick a real public dataset that has not been cleaned for you. Document what is wrong with it, fix what you can, explore it, and write up one clear finding with the evidence and the limitations.
What it demonstrates
That you can work with data as it actually arrives, and that you state uncertainty rather than hiding it.
What "done" looks like
Build all of these and the project is finished. Anything past that is in the stretch goals.
- A real dataset that has not been cleaned for you
- A documented list of what was wrong with it and how you handled it
- Univariate and relationship exploration
- Four to six charts that each answer something
- One clear finding, stated with its limitations
- A README that leads with the answer, not the method
How to build it
- 1
Pick a dataset with real problems
Government open data, an API export or your own scrape. Pre-cleaned competition data hides the work you are demonstrating.
- 2
Ask one question first
Decide what you are trying to find out before exploring. Aimless plotting produces notebooks, not conclusions.
- 3
Profile the data
Shape, types, missingness, duplicates, ranges. Write down what is wrong before changing anything.
- 4
Clean deliberately
Every decision — dropped rows, imputed values, corrected types — gets a reason recorded. Cleaning silently is how conclusions become unreproducible.
- 5
Explore
Distributions, then relationships. Watch for outliers that are errors and outliers that are the story.
- 6
Visualise the argument
Keep only the charts that support the finding. Label the axes, and never truncate one to exaggerate an effect.
- 7
Write the conclusion
Lead with what you found, then the evidence, then what would change your mind. The caveats are what make it credible.
Once it works
Only after the definition of done is met. Half-finished ambition reads worse than a small finished thing.
- Test a hypothesis formally rather than describing the pattern
- Automate the cleaning as a reproducible script
- Publish it as a short written piece rather than a notebook
Frequently Asked Questions
Should I use a Kaggle dataset?
Prefer one that has not already been cleaned. The famous Kaggle sets have had the hardest part done for you and have been analysed thousands of times, so they demonstrate very little. Government open data, an API export or a scrape shows far more.
What makes an EDA project good?
A conclusion. Twenty charts with no argument is a notebook, not an analysis. Pick one question, answer it, show the two or three charts that support the answer, and say what would change your mind.
How do I present it?
A README with the finding at the top and the charts inline, with the notebook linked below for anyone who wants the working. Nobody opens a notebook first — lead with the answer.