Common Pandas & NumPy Pitfalls

Analogy categoryAnalogies

Pandas and NumPy are forgiving enough to let incorrect code run without raising an exception, which makes their pitfalls more dangerous than a typical crash — the danger is a silently wrong result that ships into a report or a model. Most of these pitfalls come from a small number of root causes: ambiguity between views and copies, implicit index alignment, dtype coercion, and float precision. Recognizing the pattern behind each pitfall makes it far easier to spot in code review, even in a library or codebase you haven't seen before.

🏏

Cricket analogy: A scorer who miscounts extras without the umpire noticing ships a wrong final total into the record book; pandas' silent bugs work the same way — no crash, just a quietly wrong scoresheet passed downstream.

Chained Indexing and SettingWithCopyWarning

The single most common pandas pitfall is writing df[df.col > 0]['other_col'] = 1. This performs two separate __getitem__ operations, and pandas cannot always guarantee whether the intermediate result is a view into df or an independent copy — so the assignment may modify a throwaway copy, silently doing nothing to df. The fix is to collapse the selection and assignment into a single .loc[] call: df.loc[df.col > 0, 'other_col'] = 1, which pandas can resolve unambiguously in one step.

🏏

Cricket analogy: Writing df[df.runs>50]['out_type']='caught' filters the scorecard then tries to edit a possibly throwaway copy of it, so the dismissal type may never actually update; df.loc[df.runs>50,'out_type']='caught' does both steps in one unambiguous move.

python

import numpy as np
import pandas as pd

df = pd.DataFrame({'temp_c': [20, 35, 15, 40], 'alert': [False]*4})

# WRONG: chained indexing, may raise SettingWithCopyWarning and silently fail
df[df['temp_c'] > 30]['alert'] = True

# RIGHT: single .loc call, unambiguous
df.loc[df['temp_c'] > 30, 'alert'] = True

# WRONG: NumPy view surprise - modifying a slice modifies the original array too
arr = np.array([1, 2, 3, 4, 5])
view = arr[1:3]
view[0] = 99
print(arr)  # [1, 99, 3, 4, 5] - arr was mutated because view shares memory

# RIGHT: force an independent copy when mutation must not propagate
safe_copy = arr[1:3].copy()

Index Misalignment and dtype Coercion

When you perform arithmetic between two Series, pandas aligns them by index label before computing, not by row position — if the indexes don't match exactly, the result contains NaN for every unaligned label, which silently corrupts results if you assumed a positional alignment (common after a filter() or sort_values() left one Series with a different index order). Separately, assigning a single NaN into an integer column upcasts the entire column to float64, because NumPy's integer dtypes have no representation for missing values — this is why an apparently 'whole number' column can suddenly show .0 suffixes and comparisons like df['id'] == 5 can behave unexpectedly if 5 was silently read in as 5.0.

🏏

Cricket analogy: Adding two Series of runs indexed by player name only sums correctly if both lists list players in the same order; after sorting one by score, mismatched names produce NaN totals, and one missing score upcasts a whole runs column from int to float, adding stray .0s.

A useful mental model: pandas index alignment behaves like a database join on row labels for every binary operation between Series, not a positional zip — always reset or align indexes explicitly (.reset_index(drop=True) or .align()) before combining Series that came from different filtering or sorting operations.

Comparing floats for exact equality (df['ratio'] == 0.3) is a classic NumPy/pandas trap: floating-point arithmetic (e.g. 0.1 + 0.2) frequently produces values like 0.30000000000000004 that fail exact equality checks; use np.isclose() or a tolerance-based comparison instead.

Mutable Defaults and In-Place Confusion

Many pandas methods accept an inplace=True argument that modifies the DataFrame in place and returns None — a frequent bug is writing df = df.dropna(inplace=True), which sets df to None because inplace=True operations don't return the modified object. Relatedly, inplace=True does not always avoid a copy internally (pandas may still build a new block of memory and reassign it under the hood), so it is not a reliable performance optimization and is increasingly discouraged in favor of explicit reassignment: df = df.dropna(), which is both clearer and equally efficient in most cases.

🏏

Cricket analogy: Writing scorecard = scorecard.remove_extras(inplace=True) sets scorecard to None because inplace edits the sheet directly and hand back nothing; the fix is scorecard = scorecard.remove_extras(), which is just as reliable and clearer to read.

Chained indexing (df[mask]['col'] = x) risks silently writing to a throwaway copy; use a single .loc[] call instead.
NumPy basic slices return views that share memory with the original array — mutating one mutates the other.
Series arithmetic aligns by index label, not row position, producing NaN wherever labels don't match.
Introducing a NaN into an integer column silently upcasts the whole column to float64.
Exact float equality checks are unreliable; use np.isclose() for tolerance-based comparison.
inplace=True returns None, so df = df.method(inplace=True) is a bug, and it offers no reliable performance benefit.

Practice what you learned

1. What is wrong with `df[df['x'] > 0]['y'] = 1`?

2. Why does `arr[1:3][0] = 99` (basic slicing) also change the original NumPy array `arr`?

3. What happens when you write a single NaN value into an int64 pandas column?

4. Why is `df['ratio'] == 0.3` an unreliable check after computing `ratio` via floating-point arithmetic?

5. What does `df = df.dropna(inplace=True)` actually do?

Was this page helpful?

Pandas & NumPy Interview Questions

Pandas & NumPy Quick Reference

Topics covered

#Python#PandasNumPyStudyNotes#DataScience#CommonPandasNumPyPitfalls#Common#Pandas#NumPy#Pitfalls#StudyNotes#SkillVeris

Common Pandas & NumPy Pitfalls