Pandas — tables in Python

A DataFrame is a table with named columns — like a spreadsheet, but in code and without the mouse.

What you want to do	Pandas
Read a file	pd.read_csv("data.csv")
Look at it	df.head(), df.info(), df.describe()
Select columns	df[["a", "b"]]
Filter rows	df[df.age > 15]
A new column	df["bmi"] = df.weight / df.height ** 2
Group	df.groupby("class").grade.mean()
Join tables	df.merge(other, on="pupil_id")
Sort	df.sort_values("grade", ascending=False)

Always start with df.info() and df.describe(). The first shows the row count, the column types and how many values are missing; the second shows the min, the max and the quartiles. Together they reveal most problems in five seconds — an age of 999, a column that came out as text instead of numbers, a thousand missing values.

import pandas as pd

df = pd.DataFrame({
    "pupil": ["Ada", "Bo", "Cim", "Dag", "Eve"],
    "class": ["7A", "7A", "7B", "7B", "7B"],
    "score": [82, 95, 67, None, 74],
    "hours": [12, 20, 8, 15, 10],
})

print(df.info())          # the row count, the types, non-null per column
print(df.describe())      # min, max, mean, quartiles for the numeric columns

# Filter — brackets are required round each condition, and & / | instead of and / or
print(df[(df.score > 70) & (df["class"] == "7B")])

# A new column
df["score_per_hour"] = df.score / df.hours

# Group and aggregate several metrics at once
print(df.groupby("class").agg(
    count=("pupil", "count"),
    mean_score=("score", "mean"),
    max_hours=("hours", "max"),
))
#        count  mean_score  max_hours
# class
# 7A         2        88.5         20
# 7B         3        70.5         15      ← the mean is over 2 values, None is skipped

# Joining — ALWAYS check the row count afterwards
attendance = pd.DataFrame({"pupil": ["Ada", "Bo", "Cim"], "attendance": [0.95, 0.80, 0.99]})
joined = df.merge(attendance, on="pupil", how="left", validate="one_to_one")
print(len(df), len(joined))        # 5 5 — the same count, as expected

# A common trap: a merge that accidentally duplicates rows
double = pd.DataFrame({"pupil": ["Ada", "Ada"], "test": [1, 2]})
print(len(df.merge(double, on="pupil", how="left")))    # 6 — one row became two!

Three traps that cost the most time:

Trap	Symptom	The right way
and/or in a filter	ValueError: truth value is ambiguous	& and `
Chained assignment	SettingWithCopyWarning, the change disappears	df.loc[condition, "col"] = value
A merge that duplicates	the row count grows silently	validate="one_to_one" or check len()

The last is the most dangerous because no error message appears. Check the row count after every merge — it is one line of code and saves hours.

A workflow that works every time you get a new file:

df = pd.read_csv("new_file.csv")

# 1. What does it look like?
print(df.shape)                       # (rows, columns)
print(df.dtypes)                      # did something come out as text that should be numbers?
print(df.isna().sum())                # missing values per column
print(df.duplicated().sum())          # exact duplicates

# 2. Are the values plausible?
print(df.describe())                  # min/max reveals 999, -1, 0 as «missing»
for c in df.select_dtypes("object"):
    print(c, df[c].nunique(), df[c].unique()[:5])   # typos, mixed formats

# 3. Only then: the analysis

Steps 1 and 2 take a minute and nearly always find something. Common findings:

• A column with dtype: object that should be float — there is a "-" or an "n/a" somewhere in it.
• A max of 999 or a min of −1 in an age column — somebody coded «missing» as a number.
• Category columns with 47 unique values when there should be 5 — typos and mixed capitalisation.
• More rows than expected after a merge.

Make it a habit before you write a single line of analysis. It is the same principle as looking at the data before training a model — and it is by far the most profitable minute in the whole project.