Chapter 4: Files, Paths, CSV, JSON, and Data Cleanup

This chapter moves Python into the kind of automation work many teams actually need: reading files, cleaning rows, serializing JSON, and writing safer output pipelines instead of one-off copy-paste scripts.

Why This Chapter Exists In The OrderOps Python Project

This chapter moves Python into the kind of automation work many teams actually need: reading files, cleaning rows, serializing JSON, and writing safer output pipelines instead of one-off copy-paste scripts.

Inside OrderOps, this chapter shows up while operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. The goal is not to memorize one-off syntax. The goal is to make Python code readable enough to explain, safe enough to change, and grounded enough to discuss in an interview without sounding vague.

• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Milestone: build a cleanup pipeline that reads external data, normalizes it, and writes output without hiding row-level problems
• Interview lens: the next chapter teaches exceptions, logging, and debugging so cleanup jobs fail in ways humans can actually diagnose
• The chapter teaches Python fundamentals through one connected backend and automation story.

Path Objects Make File Intent Clearer Than Stringly-Typed Path Juggling

Use path objects so joins, names, and parent directories stay explicit and platform-safe.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Pathlib a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: Hand-built path strings are easy to get wrong and harder to review. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. Interviewers like when you pick the tool that makes intent obvious rather than the one that happens to work today.

• Use path objects so joins, names, and parent directories stay explicit and platform-safe.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: Hand-built path strings are easy to get wrong and harder to review.
• Interview lens: Interviewers like when you pick the tool that makes intent obvious rather than the one that happens to work today.

from pathlib import Path

report_path = Path("exports") / "orders.csv"
print(report_path.parent, report_path.name)

Reading Files Should Make Encoding And Ownership Obvious

Be explicit about how text is read so the script is honest about the boundary and easier to reason about later.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Reading Text a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: If the script hides encoding or file ownership assumptions, subtle environment issues become harder to explain. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. File boundaries are a good interview topic because they reveal whether you think beyond the happy path.

• Be explicit about how text is read so the script is honest about the boundary and easier to reason about later.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: If the script hides encoding or file ownership assumptions, subtle environment issues become harder to explain.
• Interview lens: File boundaries are a good interview topic because they reveal whether you think beyond the happy path.

from pathlib import Path

config_text = Path("config.txt").read_text(encoding="utf-8")
print(config_text.strip())

CSV Work Is About Row Contracts And Cleanup Discipline, Not Just Opening A File

Treat each row as external input that must be normalized before the rest of the workflow trusts it.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes CSV Parsing a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: Assuming every CSV row is well-formed usually delays failure until much later in the pipeline. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. Strong candidates talk about row contracts and validation, not only about the csv module itself.

• Treat each row as external input that must be normalized before the rest of the workflow trusts it.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: Assuming every CSV row is well-formed usually delays failure until much later in the pipeline.
• Interview lens: Strong candidates talk about row contracts and validation, not only about the csv module itself.

import csv

with open("orders.csv", newline="", encoding="utf-8") as handle:
    rows = list(csv.DictReader(handle))
print(rows[:1])

JSON Should Be Mapped Into Your Domain Deliberately Instead Of Leaking Everywhere

Parse JSON once, validate the shape, and map it into the structure your own code wants to reason about.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes JSON Parsing a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: Letting raw partner payloads flow unchecked through the codebase makes every later function depend on unstable external shape. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. Interviewers like to hear boundary translation because it sounds like real integration work.

• Parse JSON once, validate the shape, and map it into the structure your own code wants to reason about.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: Letting raw partner payloads flow unchecked through the codebase makes every later function depend on unstable external shape.
• Interview lens: Interviewers like to hear boundary translation because it sounds like real integration work.

import json

payload = json.loads('{"order_id":"ORD-1","subtotal":88.4}')
print(payload["order_id"], payload["subtotal"])

A Cleanup Pipeline Should Make Each Row Transformation Easy To Inspect

Normalize one field or one invariant at a time so bad rows can be rejected for a clear reason.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Normalization Pipelines a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: Trying to clean everything in one dense transformation step makes root causes harder to recover. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. Good operational engineers build pipelines that are explainable under pressure.

• Normalize one field or one invariant at a time so bad rows can be rejected for a clear reason.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: Trying to clean everything in one dense transformation step makes root causes harder to recover.
• Interview lens: Good operational engineers build pipelines that are explainable under pressure.

def normalize_row(row: dict[str, str]) -> dict[str, object]:
    return {
        "sku": row["sku"].strip(),
        "quantity": int(row["quantity"]),
    }

Writing Output Safely Matters Because Half-Written Files Become Real Incidents

Use safer write patterns when downstream consumers depend on the file existing in a coherent final state.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Safe Output Writes a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: If a process dies mid-write and the target file was already replaced, the next job may read corrupt output. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. Interviewers notice candidates who think about partially written artifacts and downstream consumers.

• Use safer write patterns when downstream consumers depend on the file existing in a coherent final state.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: If a process dies mid-write and the target file was already replaced, the next job may read corrupt output.
• Interview lens: Interviewers notice candidates who think about partially written artifacts and downstream consumers.

from pathlib import Path

target = Path("orders.json")
temp = target.with_suffix(".tmp")
temp.write_text("[]", encoding="utf-8")
temp.replace(target)

Cleanup Jobs Need Reports Humans Can Understand Without Reading The Source

Publish counts, rejects, and key summaries so operators can trust the job result and ask better follow-up questions.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Reporting and Auditability a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: A script that silently changes data without reporting what it did is operationally weak even if the code is correct. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. This sounds senior because it treats software as part of a real workflow, not only as local code execution.

• Publish counts, rejects, and key summaries so operators can trust the job result and ask better follow-up questions.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: A script that silently changes data without reporting what it did is operationally weak even if the code is correct.
• Interview lens: This sounds senior because it treats software as part of a real workflow, not only as local code execution.

cleaned = 245
rejected = 6
print(f"cleaned={cleaned} rejected={rejected}")

A Strong Mini Project Connects Input, Validation, Transformation, And Output In One Flow

Combine file reading, row normalization, and output writing into one coherent chapter project that can be tested later.

In OrderOps, operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on. That makes Import/Export Mini Project a real engineering concern instead of a trivia topic. It affects whether the script or service stays easy to trust when another engineer reads it six weeks later.

The common failure mode is straightforward: If each step works only in isolation, the learner never practices the end-to-end reasoning interviews often require. The stronger move is to make the rule explicit, keep the data shape visible, and leave a code path that is easy to narrate under interview pressure. A chapter project matters because it forces multiple Python basics to cooperate under one story.

• Combine file reading, row normalization, and output writing into one coherent chapter project that can be tested later.
• Project lens: operations staff are importing order files, cleaning malformed rows, and exporting summaries that downstream systems depend on
• Common pitfall: If each step works only in isolation, the learner never practices the end-to-end reasoning interviews often require.
• Interview lens: A chapter project matters because it forces multiple Python basics to cooperate under one story.

def export_summary(rows: list[dict[str, object]]) -> dict[str, int]:
    return {"processed": len(rows), "failed": 0}

Chapter Milestone And Interview Checkpoint

The milestone for this chapter is clear: build a cleanup pipeline that reads external data, normalizes it, and writes output without hiding row-level problems

That milestone matters because interview prep is not only about remembering Python features. It is about explaining why the code is shaped that way, what bug or maintenance cost the shape avoids, and what you would test before calling the work safe.

This chapter should end with two kinds of confidence. First, you should be able to write and read the code in context. Second, you should be able to explain the tradeoff behind it in plain engineering language.

• Milestone: build a cleanup pipeline that reads external data, normalizes it, and writes output without hiding row-level problems
• Healthy interview answers explain both code behavior and design intent.
• Good preparation means being able to trace a small example without guessing.
• Bridge to next chapter: the next chapter teaches exceptions, logging, and debugging so cleanup jobs fail in ways humans can actually diagnose

Chapter takeaway

Good file and serialization code is explicit about data shape, validation, and how output is written safely.