Convert CSV to JSON in Python (2025 Guide)

November 2025 · 6 min read · Deep-dive companion to my Dev.to post

Why Not the Naive One-Liner?

json.dump(list(csv.DictReader(f))) looks fine until production. Here is what actually breaks:

• BOM (Byte Order Mark): Excel prepends \ufeff. First key becomes '\ufeffname' — every lookup fails silently.
• Empty values: A blank cell should be null, not the string "".
• Type coercion: "42" → 42, "3.14" → 3.14, but "007" stays a string (it's an ID).
• Whitespace: " active " → "active".
• Quoted delimiters: "Smith, John" contains a comma but is one field.

The Full Script

#!/usr/bin/env python3
"""CSV to JSON converter (zero dependencies).
Usage: python csv2json.py input.csv [-o out.json] [--indent 2]
"""
import csv, json, argparse

def coerce(value):
    """Convert a CSV string cell to the most specific Python type."""
    s = value.strip()
    if s == "":
        return None  # empty -> null
    # Integer (no leading zeros like 007)
    if s.lstrip("-").isdigit() and not (s.startswith("0") and len(s) > 1):
        return int(s)
    # Float
    try:
        f = float(s)
        if "." not in s:
            return int(f)
        return f
    except ValueError:
        pass
    # Boolean
    low = s.lower()
    if low in ("true", "false"):
        return low == "true"
    return s  # plain string

def convert(input_path, output_path=None, indent=2):
    with open(input_path, newline="", encoding="utf-8-sig") as f:
        reader = csv.DictReader(f)
        rows = []
        for row in reader:
            clean = {}
            for key, val in row.items():
                clean[key.strip()] = coerce(val) if val is not None else None
            rows.append(clean)
    result = json.dumps(rows, indent=indent, ensure_ascii=False)
    if output_path:
        with open(output_path, "w", encoding="utf-8") as out:
            out.write(result + "\n")
        print(f"OK {len(rows)} rows -> {output_path}")
    else:
        print(result)

if __name__ == "__main__":
    ap = argparse.ArgumentParser(description="Convert CSV to JSON")
    ap.add_argument("input", help="Path to input .csv file")
    ap.add_argument("-o", "--output", help="Output .json path (default: stdout)")
    ap.add_argument("--indent", type=int, default=2, help="JSON indent (default 2)")
    args = ap.parse_args()
    convert(args.input, args.output, args.indent)

Edge-Case Matrix

Worked Example

Input (employees.csv, Excel BOM):

id,name,active,salary
1,Alice,TRUE,75000.50
2,Bob,,42
3,  Charlie  ,false,

Output (--indent 2):

[
  {
    "id": 1,
    "name": "Alice",
    "active": true,
    "salary": 75000.5
  },
  {
    "id": 2,
    "name": "Bob",
    "active": null,
    "salary": 42
  },
  {
    "id": 3,
    "name": "Charlie",
    "active": false,
    "salary": null
  }
]

Running It

# Convert to stdout
python csv2json.py data.csv

# Save to file with custom indent
python csv2json.py data.csv -o clean.json --indent 4

# Pipe into jq for further processing
python csv2json.py data.csv | jq ".[0]"

When You Need More

• < 1M rows: the script above is sufficient.
• 1M–50M rows: switch to pandas.read_csv(...).to_json(orient="records") or polars.scan_csv() for lazy evaluation.
• Streaming (any size): write NDJSON (one object per line) if the file exceeds RAM.

Deliverable on OpenTask