Reading and Writing Files in Python

Scripts live on files: configuration, logs, exports, downloaded data. The mechanics are short; the failures are mostly about encoding, newlines, and paths.

Environment

Examples are verified on Python 3.13. JSON and CSV support come from the standard library.

python -c "import sys; print(sys.version)"
# 3.13.x

Open with with

with closes the file when the block exits, including when an exception is raised:

with open("notes.txt", "r", encoding="utf-8") as f:
    content = f.read()

Without with you must call f.close() yourself, and the one path where you forget is the one that leaks a handle.

Modes:

Mode	Meaning	If the file is missing
"r"	read (default)	FileNotFoundError
"w"	write, truncate first	created
"a"	append	created
"x"	create exclusively	FileExistsError
"rb" / "wb"	binary read / write	same as r / w

Three ways to read text

with open("notes.txt", encoding="utf-8") as f:
    text = f.read()          # whole file; fine for small files

with open("notes.txt", encoding="utf-8") as f:
    for line in f:           # streaming; constant memory
        print(line.rstrip("\n"))

with open("notes.txt", encoding="utf-8") as f:
    lines = f.read().splitlines()   # list without trailing newlines

Prefer the second: a multi-hundred-megabyte log will not fit comfortably in memory, and line iteration costs nothing extra.

Writing:

with open("out.txt", "w", encoding="utf-8") as f:
    f.write("first line\n")
    f.writelines(["second\n", "third\n"])   # no newlines added for you

print(..., file=f) is often neater than manual \n handling.

pathlib instead of string concatenation

from pathlib import Path

p = Path("data") / "2026" / "notes.txt"

p.parent        # Path('data/2026')
p.name          # 'notes.txt'
p.stem          # 'notes'
p.suffix        # '.txt'
p.exists()      # False until created

The / operator works on every platform, so you never write "\\" yourself.

Common operations:

Path("data/reports").mkdir(parents=True, exist_ok=True)   # recursive, no error if present

Path("notes.txt").write_text("content", encoding="utf-8")
text = Path("notes.txt").read_text(encoding="utf-8")

for file in Path("data").glob("**/*.csv"):    # recursive match
    print(file)

Path.home()                        # user home
Path.cwd()                         # current working directory
Path(__file__).resolve().parent    # directory of this script

Path objects can be passed straight to open(), and the convenience methods (read_text, write_text, read_bytes) avoid the explicit with for one-shot operations.

JSON

Configuration and API payloads default to JSON:

import json

data = {"name": "Ada", "tags": ["python", "web"], "vip": False}

with open("config.json", "w", encoding="utf-8") as f:
    json.dump(data, f, indent=2)

with open("config.json", encoding="utf-8") as f:
    loaded = json.load(f)

• indent=2 keeps files readable and diff-friendly.
• ensure_ascii=False preserves non-ASCII characters instead of \uXXXX escapes. Use it whenever the file is meant for humans.
• json.dumps() / json.loads() are the string-based counterparts, useful for HTTP payloads and logs.

Type mapping: dict → object, list → array, str → string, int/float → number, True/False → true/false, None → null, and back again.

Two parsing gotchas:

json.loads("[]")     # []          (a list)
json.loads("{}")     # {}          (a dict)
json.loads('"hi"')   # 'hi'        (a string, not an error)

Do not assume the top level is a dict; validate the shape before indexing.

CSV

import csv

with open("users.csv", "w", encoding="utf-8", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["id", "name"])
    writer.writerow([1, "Ada"])

Reading by column name is usually what you want:

with open("users.csv", encoding="utf-8", newline="") as f:
    for row in csv.DictReader(f):
        print(row["name"])     # row is a dict keyed by the header

Writing dict rows:

with open("users.csv", "w", encoding="utf-8", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["id", "name"])
    writer.writeheader()
    writer.writerow({"id": 1, "name": "Ada"})

newline="" is required by the csv module: without it, Windows produces blank lines between rows because the writer’s \r\n is translated again.

Always give CSV a real parser. Splitting on commas breaks on quoted fields, embedded commas, and newlines inside values.

Common mistakes

1. Omitting encoding

The default encoding depends on the platform (often UTF-8 on Linux and macOS, a legacy code page on Windows). Reading UTF-8 text without specifying it eventually raises UnicodeDecodeError. Always pass encoding="utf-8" for text.

2. Writing CSV without newline=""

Blank lines on Windows; also a documented requirement of the csv module.

3. Relying on the current working directory

Relative paths resolve against where the process was started, not where the script lives:

BASE = Path(__file__).resolve().parent
config = BASE / "config.json"

4. Using "w" when you meant "a"

"w" truncates immediately, before you write anything. Use "a" to append, "x" to fail loudly if the file exists.

5. Creating paths with open but not their directories

open("data/out.txt", "w") fails if data/ does not exist. Call mkdir(parents=True, exist_ok=True) first.

6. Dumping custom objects with json.dump

json only knows built-in types. Convert to a dict, or pass default=:

json.dump(user, f, default=lambda o: o.__dict__, ensure_ascii=False)

7. Doing blocking file I/O in async code

A synchronous open().read() blocks the event loop. Large files belong in a worker thread (asyncio.to_thread) or a background process.

Summary

• Always use with, and always name the encoding.
• Stream large files line by line instead of read().
• Build paths with pathlib; anchor script-relative paths to Path(__file__).
• JSON: indent=2, ensure_ascii=False, validate the top-level type.
• CSV: newline="" on write, DictReader/DictWriter for named columns.

Next: handling the errors these operations raise.