Chapter 11: Files and Text Input/Output

Section 1

Files are how programs remember things after they stop running. Variables live only while a program runs, but a file on disk keeps its contents until something changes or deletes it. When you write a program that saves notes, settings, scores, or reports, you are usually writing to a file. When you load that information later, you are reading a file. Python makes this ordinary work feel simple, but the ideas underneath matter: a file is a sequence of bytes, a path tells the system where to find it, and opening a file creates a connection between your program and that storage. You do not work with the disk directly; you work through a file object that Python gives you.

Text files are the friendliest kind to start with. They store characters, one after another, using an encoding that maps characters to bytes. Python 3 treats text as Unicode by default, which means it can hold letters from many languages, digits, punctuation, and symbols. When you open a text file, you normally tell Python the encoding, most often UTF-8, so that reading and writing agree on how characters are stored. If two programs disagree about encoding, the same bytes can look like different characters, or reading can fail. Being explicit about encoding is a small habit that prevents a whole class of confusing errors later, especially when files move between computers.

Paths also deserve care. A relative path, such as notes.txt, means a file in the current working directory, which depends on where the program was started. An absolute path spells out the full location from the root of the system. Relative paths are convenient for small projects, but they can surprise you when a program is run from a different folder. It helps to think of the working directory as part of the program's environment, not as a fixed property of the code. For learning exercises, keeping data files beside your script and running from that folder keeps paths predictable.

The basic pattern for reading a whole text file uses open with mode "r", reads the contents, and then closes the file. Closing matters because the operating system limits how many files a program can hold open, and because data written to a file may sit in a buffer until the file is closed or flushed. Python's with statement handles closing automatically, even if an error happens in the middle. You will see with open(...) as f: again and again in this chapter, because it is the safe default. Get comfortable with it now, and file handling will feel routine rather than risky.

with open("notes.txt", "r", encoding="utf-8") as f:
    text = f.read()
print(text)

A useful mental model is to picture a file as a notebook that outlives the person writing in it. Your program opens the notebook, reads or writes a page, and then closes it so the next program can use it safely. That image also explains why two programs writing to the same file at once can interfere with each other: each expects the notebook to stay as it left it. For learning, work with one program and one file at a time, and you will avoid most of these tangles.

Check: Why is it safer to open files with a with statement than to open and close them by hand?

Section 2

The open function takes a path and a mode, and the mode decides what you are allowed to do. Mode "r" opens an existing file for reading and fails if the file does not exist. Mode "w" opens a file for writing, creating it if needed, but it erases any existing contents, so it must be used with care. Mode "a" opens a file for appending, creating it if needed and adding new data at the end without erasing what is already there. Mode "x" creates a new file and fails if one already exists, which is useful when you never want to overwrite by accident. There is also "r+", which allows both reading and writing, but it is less common in beginner code and easier to misuse.

Choosing the right mode is really choosing the right intention. Reading a configuration file calls for "r". Saving a fresh report calls for "w". Adding a line to a log calls for "a". Protecting an existing file from accidental replacement calls for "x". When you pick a mode deliberately, your code communicates its purpose to the next reader, and it fails loudly in situations where silently overwriting would be worse. A program that refuses to overwrite a precious file is not being difficult; it is being careful on your behalf.

Modes can be combined with a character that says whether the file is text or binary. Adding "t" means text, which is the default, while "b" means binary, where you work with bytes objects instead of strings. Text mode translates between characters and bytes using an encoding, and it also handles line endings in a portable way. Binary mode gives you the raw bytes exactly as stored, which is what you need for images, compressed data, or files whose format is not text. Most of this chapter works in text mode, because text files are where beginners build the habits that later transfer to binary work.

A common beginner mistake is opening a file in write mode just to check whether it exists, and thereby erasing it. Another is opening the same file for writing in two places and being surprised by the result. Keeping the mode close to the intention, and keeping file operations short and contained inside a with block, reduces both risks. If you are ever unsure which mode to use, pause and ask what should happen if the file already exists and what should happen if it does not. The answers usually point straight to the right mode.

with open("log.txt", "a", encoding="utf-8") as f:
    f.write("Session started\n")

Modes also protect you from yourself in small ways. Imagine a script that saves a report every time it runs. In write mode it replaces yesterday's report, which may be exactly what you want for a current snapshot. In append mode it grows a history instead, which suits a log. Naming the intention in the mode, and perhaps in a comment beside it, makes the difference visible to anyone reading the code later, including you in a few months.

Check: What is the difference between opening a file in "w" mode and opening it in "a" mode?

Section 3

Reading a whole file at once with read() is simple, but it is not always the best choice. For a small file, it is fine and convenient: you get one string holding everything, and you can search it, split it, or print it. For a very large file, however, loading everything into memory at once can be slow or wasteful, and sometimes it is simply unnecessary. If you only need to process the file line by line, reading it gradually is kinder to memory and often clearer to read. Python's file objects support iteration, so a for loop over an open file yields one line at a time, without loading the rest.

Line-by-line processing fits many real tasks: counting lines that contain a word, transforming each record, filtering out unwanted entries, or building a summary. The pattern is to open the file, loop over it, and handle each line inside the loop. Each line, as read, usually ends with a newline character, "\n", except possibly the last line. That trailing newline is part of the data returned, not an extra line, and you often remove it with rstrip("\n") or strip() before further work. Forgetting the newline is a classic source of double-spaced output, because print adds its own newline on top of the one already in the line.

You can also read a fixed number of characters with read(n), or a single line with readline(), or all lines into a list with readlines(). These methods give you finer control when a task needs it, such as reading a header before processing the rest, or peeking at the first line to decide what to do. In practice, read() for small files and iteration for larger ones cover most needs, and the other methods are there for the special cases. Knowing they exist matters more than using them every day.

When you iterate, remember that a file object is a one-way stream in the usual pattern: once you have read to the end, the position is at the end, and a second loop over the same object will find nothing left unless you move back to the start. That behaviour is natural for a stream, but it surprises beginners who expect a file object to behave like a list they can loop over repeatedly. If you need the lines more than once, store them in a list, or reopen the file, or move the position back with seek(0), a technique covered later in this chapter.

with open("notes.txt", "r", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n"))

This streaming style also shapes how you design programs. Instead of loading a whole dataset and then deciding what to do, you can decide as each line arrives, keeping only a count, a total, or the lines that match. That habit scales: the same loop that handles a hundred lines can handle a million, provided the work inside the loop stays simple. When a task genuinely needs random access to earlier lines, store the needed ones deliberately rather than loading everything by default.

Check: Why is iterating over a file line by line usually better than read() for a very large file?

Section 4

Writing text to a file is the mirror of reading it. The write() method takes a string and adds it to the file at the current position, which for a newly opened "w" or "a" file is the start or the end respectively. Unlike print, write() does not add a newline automatically, so you must include "\n" yourself when you want each item on its own line. This explicitness is useful: it means you control exactly what goes into the file, including blank lines, spacing, and separators. It also means a missing newline can join two records into one line, which is a common and instructive bug to meet early.

There is also writelines(), which takes an iterable of strings and writes them one after another. Despite its plural name, it does not add newlines either; each string must already end with one if you want line breaks. Many beginners expect writelines() to behave like printing a list, and are puzzled when the output runs together. The name is historical, and the behaviour is consistent once you know it: both write() and writelines() write exactly the characters you give them, nothing more.

When you write, Python may buffer the data, holding it in memory briefly before passing it to the operating system. Buffering makes writing faster, because it reduces the number of system calls, but it means that a crash before a flush could lose the most recent writes. Closing the file flushes the buffer, which is another reason the with statement is valuable: it guarantees the close, and therefore the flush, even when an error interrupts your code. You can also call flush() explicitly when another program needs to see the data immediately, such as a log file you are watching while the program runs.

A good exercise is to write a small program that saves a list of items, one per line, then reads the file back and prints the items with numbers. That round trip, write then read, is the heart of persistent storage. Once you can do it reliably, you can save user preferences, game scores, task lists, and simple reports, all with the same few lines. Later chapters and later formats will build on this foundation, but the file itself remains the same idea: characters on disk, written deliberately, read back faithfully.

items = ["apples", "bread", "cheese"]
with open("shopping.txt", "w", encoding="utf-8") as f:
    for item in items:
        f.write(item + "\n")

It is worth practising the write-then-read round trip until it feels boring, because boring reliability is the goal. Save a short list, close the file, reopen it, and check that what comes back matches what you wrote, including order and spelling. If something differs, the difference usually points to a missing newline, a wrong mode, or an encoding mismatch. Finding those three causes quickly is a skill you will reuse in every later chapter that touches data.

Check: Why must you add "\n" yourself when using write(), and what happens if you forget it?

Section 5

Every file ends, and every line ends, and handling those endings gracefully is part of file work. In text, lines are separated by newline characters. Different operating systems have historically used different conventions, but Python's text mode translates them to "\n" on reading and back to the platform convention on writing, so your code can usually think in terms of "\n" alone. When you split text yourself, splitlines() is handy: it breaks a string at line boundaries and drops the newline characters, handling several conventions without you having to think about them.

Trailing whitespace is another small detail with real consequences. A line read from a file may end with "\n", and may also carry spaces or tabs before it. Using strip() removes whitespace from both ends, which is often what you want when comparing or parsing a line, but it can also remove meaningful leading spaces, for example in indented text. Using rstrip("\n") removes only the newline, preserving the rest. Choosing between them is a matter of knowing your data: for simple word lists, strip() is fine; for formatted text, be more surgical.

Empty lines deserve a decision too. Should a blank line in the input be counted, skipped, or preserved? The answer depends on the task. A program counting records might skip blank lines, while a program copying a document must preserve them exactly. Making that decision explicitly, rather than letting it happen by accident, is a sign of growing care as a programmer. The same applies to a file that ends without a final newline: your code should not assume the last line ends with one.

Encoding errors are the line-ending problem's bigger cousin. If a file was written in a different encoding from the one you use to read it, Python may raise an error or produce strange characters. Sticking to UTF-8 on both sides, and being explicit about it, avoids most of this. When you receive a file from elsewhere and do not know its encoding, you may need to try UTF-8 first and investigate if it fails, rather than guessing silently. Files, like people, are easier to work with when everyone agrees on the language being spoken.

with open("notes.txt", "r", encoding="utf-8") as f:
    lines = [line.rstrip("\n") for line in f if line.strip()]
print(lines)

These details also matter when files travel between people. A file written on one computer and read on another should behave the same, and explicit encoding plus careful newline handling make that more likely. If you ever open a file and see replacement characters or odd symbols, treat that as a clue about encoding rather than as random corruption. Ask how the file was created, check the encoding you used, and try the most likely alternative before assuming the data is lost.

Check: What is the difference between strip() and rstrip("\n") when cleaning a line read from a file?

Section 6

Paths are not just strings; they name locations in a filing system that has its own rules. On different systems, the separator between folders differs, and a path that works on one system may fail on another. Python's pathlib module offers a modern, object-oriented way to work with paths: a Path object represents a path, and you can join parts with the / operator, ask for the file name, the parent folder, or the suffix, and check whether the path exists. Using pathlib instead of gluing strings together with separators makes code more portable and more readable, and it is the recommended approach for new code.

A Path can tell you useful things without opening the file: whether it exists, whether it is a file or a directory, and what its name and extension are. It can also create parent directories, list the contents of a folder, and open the file itself, returning the same kind of file object that open() gives you. This means you can start with a path, inspect it, and only then decide to read or write. For a learner, the shift from thinking of paths as raw strings to thinking of them as objects with behaviour is a small conceptual step with a large payoff in clarity.

The current working directory matters whenever you use a relative path. You can find it with Path.cwd(), and you can build a path relative to your script's own folder using Path(__file__).parent, which is more robust than assuming where the program was started from. That distinction becomes important as soon as a program is run from an editor, a terminal in another folder, or a scheduled task. Code that locates its data relative to its own location is far less brittle than code that assumes the working directory.

Folders themselves are part of file work. Before writing to a file inside a new folder, that folder must exist, or the open will fail. Path.mkdir(parents=True, exist_ok=True) creates a folder and any missing parents, and does nothing if it already exists, which makes it safe to call every time. Learning to create directories deliberately, rather than assuming they exist, prevents a common class of "file not found" errors that are really "folder not found" errors in disguise.

from pathlib import Path
data_dir = Path("data")
data_dir.mkdir(exist_ok=True)
path = data_dir / "notes.txt"
print(path, path.suffix)

Portable path handling becomes more valuable as projects grow beyond a single script. A project may have folders for data, output, and tests, and code that builds paths with pathlib reads almost like a description of that layout. It also makes refactoring easier: moving a folder means changing one base path rather than hunting through string concatenation. Even in tiny exercises, using a Path object trains you to think in locations and relationships, not just in characters.

Check: Why is building a path with pathlib usually better than joining folder names with a separator character by hand?

Section 7

Not every file operation succeeds, and file code should expect that. A file may not exist, a folder may be missing, permission may be denied, or a disk may be full. Python reports these situations by raising exceptions, such as FileNotFoundError and PermissionError, which are covered in more depth in the next chapter. For now, the key idea is that file code lives in the real world, where the filing system can say no, and a good program responds helpfully instead of crashing with a bare traceback for an ordinary problem like a missing input file.

A common pattern is to check whether a path exists before opening it, or to catch the exception and print a clear message. Checking first is intuitive, but it has a subtle weakness: the file could disappear between the check and the open, so the open can still fail. Catching the exception handles both cases with one mechanism, and it is the more robust habit. That does not mean checks are useless; they are excellent for giving early, specific feedback, such as telling a user which expected file is missing before doing other work.

User-facing messages deserve thought. When a program cannot find notes.txt, the helpful message names the file, says what was expected, and suggests what to do next, such as creating the file or checking the folder. A bare error, or a message that blames the user, helps nobody. As you write file programs, imagine the person running them: they may not know what a traceback means, but they can understand "I could not find notes.txt in this folder" and act on it.

Permissions are another real-world concern. A program can usually read files it owns, but system folders and other users' files may be off limits. Running a program with more privilege than it needs is not a solution; it is a risk. Instead, keep data files in folders the program is expected to use, and handle permission errors gracefully, explaining the problem in plain language. Defensive file handling is not pessimism; it is respect for the fact that your program shares a computer with other programs and other people.

from pathlib import Path
path = Path("notes.txt")
if not path.exists():
    print("notes.txt is missing; create it first.")
else:
    print(path.read_text(encoding="utf-8"))

Graceful failure also includes not destroying evidence. If a program cannot read its input, it should leave existing output files alone rather than overwriting them with an empty or partial result. A simple guard, such as reading and validating the input before opening the output for writing, prevents a missing input from becoming lost output. That ordering, validate first and write second, is a quiet form of care that users notice only when it saves them from an avoidable loss.

Check: Why is catching a file exception often more reliable than checking for the file first and then opening it?

Section 8

The pathlib module also provides convenient shortcuts for small files: read_text() reads an entire file into a string, and write_text() writes a string to a file, each with an encoding argument. These one-liners are excellent for short scripts and for learning, because they hide the open and close steps without hiding the intention. For larger files, or for work that processes data gradually, the open and iteration pattern from earlier sections remains the better choice, because it does not load everything at once.

Reading and writing whole files in one step makes certain tasks almost trivial: loading a configuration snippet, saving a short report, or replacing the contents of a small file. It also makes the code easy to review, because the whole operation fits on one line and the file, the encoding, and the data are all visible. The trade-off is control: you cannot easily process a huge file in pieces, handle lines as they arrive, or interleave reading and writing. Choosing between the shortcut and the streaming pattern is a judgement call that depends on the size of the data and the shape of the task.

Another useful habit is to separate the data you work with from the code that works with it. A program that reads its list of items from a file can be updated by editing the file, without touching the code. A program that hard-codes the same list must be edited and re-run for every change. Files are therefore not just storage; they are a way of making programs flexible. A quiz program that reads questions from a file, for example, can gain new questions without any code change at all.

When you use write_text(), remember that it behaves like opening in "w" mode: it replaces the existing contents. If you want to append with pathlib, you still use open in append mode, or read the old text and write the combined result, depending on size. There is no append_text() shortcut in the standard interface, and that absence is a small reminder that appending is a distinct intention from replacing, worth expressing explicitly rather than hiding behind a convenience.

from pathlib import Path
Path("greeting.txt").write_text("Hello, file world!\n", encoding="utf-8")
print(Path("greeting.txt").read_text(encoding="utf-8"))

Shortcuts are best used where their limits are understood. A one-line read is perfect for a settings snippet of a few lines, but it hides the question of what happens if the file is missing or malformed. Pairing a shortcut with a clear error message, or with a check that the file exists, keeps the code short without making it mysterious or fragile. In larger programs, the explicit open pattern often wins, because it shows exactly where reading happens and what happens afterwards.

Check: When is read_text() a good choice, and when should you prefer iterating over an open file instead?

Section 9

Copying, moving, and deleting files are operations on the filing system rather than on file contents, and Python provides them through modules such as shutil and pathlib. Copying a file duplicates its contents under a new name, leaving the original untouched. Moving a file changes its location or name, and may cross folders. Deleting removes the name from the folder, and on most systems the contents are not easily recovered afterwards. These operations deserve more caution than reading, because they change the user's files in ways that cannot always be undone.

A safe habit before any destructive operation is to confirm what will happen, especially in a program that deletes or overwrites. In a learning exercise, printing the path that would be deleted, and asking for confirmation, is a good stand-in for the safeguards real tools provide. In a script that runs unattended, destructive steps should be narrow, tested on copies first, and logged so that what happened can be reconstructed later. The goal is not to fear these operations, but to treat them as commitments rather than casual edits.

Renaming is often the safest way to "edit" a file's identity. Path.rename() changes a file's name, and moving it to another folder is the same operation with a different destination path. Backup strategies often rely on renaming or copying before overwriting: save the old version under a new name, then write the new version. That way, a mistake in the new version does not destroy the only copy of the old one. Many real programs follow this pattern, even if they hide it behind a "save" button.

Listing a folder's contents is the gentler sibling of these operations. Path.iterdir() yields the entries in a folder, and glob() patterns, such as "*.txt", select entries by name pattern. These tools let a program discover its input files instead of assuming their names, which is useful for batch processing: read every text file in a folder, summarise each one, and write a combined report. Discovery plus processing is a powerful combination, and it is built from the same simple pieces you have already learned.

from pathlib import Path
for path in Path(".").glob("*.txt"):
    print(path.name)

Batch work with discovered files also teaches a useful caution: patterns can match more than you expect. A glob for text files might pick up a backup, a draft, or a file you meant to keep private, so it is always very wise to print the list of files a batch program will process before it changes anything. That preview step turns a potentially surprising operation into a deliberate one, and it gives you a chance to narrow the pattern, rename a draft, or move stray files aside.

Check: Why should a program that deletes or overwrites files be more cautious than one that only reads them?

Section 10

A file's position is a hidden piece of state that explains several puzzling behaviours. When you open a file, the position starts at the beginning for reading, or at the end for appending. Each read or write moves the position forward by the amount of data handled. When the position reaches the end, further reads return nothing, not an error. You can ask for the current position with tell(), and you can move it with seek(), for example seek(0) to return to the start. These methods give you random access: the ability to jump to a particular place instead of always moving forward.

Random access is most useful for binary files and for structured formats where records have fixed sizes, but even in text work it explains why a second read after a first read finds nothing. If you need to read a file twice, you can seek back to the start, or reopen it, or store the contents. Each choice has a cost: seeking is cheap but only works on a still-open file, reopening is clear but repeats the open, and storing uses memory. For small files, storing is often simplest; for large ones, seeking or reopening may be better.

There is also a distinction between reading and writing in the same open file. A file opened for both reading and writing needs careful position management, because a write happens where the position currently is, and a read after a write may need a seek in between. Beginners are usually better off using separate opens for reading and writing, or reading everything, closing, processing, and then writing. That separation keeps the position logic simple and the intention clear, and it avoids a whole class of off-by-one surprises.

Understanding position also clarifies buffering. When you write, data may sit in a buffer until flushed; when you read, Python may read ahead into a buffer for efficiency. These details are normally invisible, and you do not need to manage them by hand, but knowing they exist helps you understand why closing a file matters and why a file that another program has just written may not yet show its newest data. The file object is an abstraction, and like all good abstractions, it hides complexity until you need to understand it.

with open("notes.txt", "r", encoding="utf-8") as f:
    first = f.read()
    print("position after read:", f.tell())
    f.seek(0)
    print("again:", f.readline().rstrip("\n"))

Position is also why mixing reading and writing in one session needs a plan. If you write a header, seek back, and then read, you are relying on the position being exactly where you think it is, which is easy to get wrong after buffering. A cleaner plan for most tasks is to finish one phase completely before starting the next: read all the input, close the file, transform the data in memory, and then write the output. Simplicity here is not a limitation; it is a design choice.

Check: After reading a file to the end, why does a second read return nothing, and how can seek(0) change that?

Section 11

When a program reads a file, it does not automatically understand the structure inside it. A text file may hold one item per line, a paragraph of prose, a table with separators, or a mixture of all three. Your job as the programmer is to decide what a line means and to turn it into data your program can use. That decision is called parsing, and it is one of the most practical skills in programming. A line such as Ada, 34, Lagos is just characters until you split it, name its parts, and convert the age to a number. The file does not do that work for you.

Splitting is the first tool for simple formats. The split method breaks a string at a separator you choose, such as a comma, and returns a list of pieces. If the separator is surrounded by spaces, you may need to strip each piece afterwards, because Ada, 34 splits into "Ada" and " 34", and that leading space matters when you compare or convert values. A small helper that strips every piece makes this pattern reliable. You should also decide what to do with lines that do not split into the expected number of pieces: skip them, report them, or treat them as errors. Ignoring malformed lines silently can hide data problems, while crashing on the first one can be too harsh for a forgiving tool.

Converting types is the second step. Numbers in a file arrive as text, so int or float must turn them into numbers before any arithmetic. That conversion can fail if the text is not a valid number, and handling that failure is part of parsing, not an afterthought. A robust parser checks the shape of each line, converts carefully, and keeps a record of lines it could not understand, so the user can fix the file rather than wonder where a record went. This discipline feels slow at first, but it prevents the silent corruption that is far harder to debug later.

Parsing also teaches you to respect the difference between data and presentation. A file that stores 34 is storing a number as text; a file that stores Age: 34 is mixing a label with data. Labels are helpful for humans reading the file directly, but they make parsing more work, because the program must remove them first. For files meant mainly for programs, a clean, regular layout with no extra decoration is easier to parse and easier to validate. For files meant for humans, a little decoration is worth the extra parsing effort, because readability matters too.

line = "Ada, 34, Lagos"
name, age_text, city = [part.strip() for part in line.split(",")]
age = int(age_text)
print(name, age, city)

Check: When parsing a comma-separated line, why should you strip each piece after splitting?

Section 12

Structured text often uses a header line to name its columns. A file might begin with name,age,city, followed by one record per line. The header tells both humans and programs what each column means, and a program that reads it can check that the columns are in the expected order before processing any data. Skipping the header by accident is a classic bug: the words name and age get treated as a person, and the conversion of age to a number fails. Reading the first line separately, or using a tool that understands headers, avoids that trap and makes the code's intention clear to the next reader.

When you write such a file, include the header deliberately and keep the column order stable. Changing the order without updating the reader breaks the contract between writer and reader, even though both programs still run. A stable format is a promise: this column will always mean this thing, in this position, in this form. Programs that keep that promise can evolve independently, because each side can rely on the other's behaviour. Breaking the promise silently, by reordering columns or changing a date format, is a common source of subtle bugs in real systems, where files pass between teams and tools.

Validation belongs at the boundary where data enters your program. Before you trust a record, check that it has the right number of fields, that numbers parse, and that required fields are not empty. A record that fails validation should not disappear quietly; it should be reported with its line number, so the file can be corrected. Line numbers are a small kindness that makes debugging far easier, because they let a person open the file, jump to the exact line, and see the problem. Without them, finding one bad record in a thousand lines is tedious work.

You should also think about what your program does with the records it accepts. Storing them in a list of dictionaries, with keys taken from the header, makes later code readable: record["name"] says more than record[0]. Dictionaries cost a little more memory than plain lists, but for the small files in learning exercises that cost is negligible, and the clarity is worth it. As files grow, you may move to dedicated tools, but the pattern remains the same: read, validate, name the fields, and only then compute.

header = "name,age,city"
records = []
for line in ["Ada,34,Lagos", "Bola,29,Abuja"]:
    values = [part.strip() for part in line.split(",")]
    records.append(dict(zip(header.split(","), values)))
print(records)

A header also makes a file self-describing for the next person who opens it in a spreadsheet or text editor. They can see at a glance what each column means without reading your code, which reduces misunderstandings when files are shared. When you design a format, therefore, spend a moment choosing clear, simple header names, avoiding spaces and ambiguous abbreviations, because those names will travel with the data wherever it goes.

Check: Why is it useful to store a parsed record as a dictionary keyed by the header names rather than as a plain list?

Section 13

Binary files store bytes rather than characters, and Python handles them with a different set of expectations. When you open a file in binary mode, reading returns a bytes object, and writing requires a bytes object, not a string. This strictness is helpful: it prevents accidental mixing of text and raw data, and it forces you to think about encoding explicitly when you do need to convert between the two. The methods encode and decode perform that conversion, with UTF-8 as the usual encoding for text that must become bytes.

Binary mode is necessary for images, audio, compressed archives, and any format whose bytes do not represent characters. It is also useful for text files whose encoding you want to control precisely, or for files you must copy without any translation of line endings. Copying a file in binary mode, reading chunks and writing them unchanged, preserves the original bytes exactly, which is why backup and transfer tools work in binary mode even for files that happen to contain text. Text mode, by contrast, may translate line endings, which is convenient for text but wrong for a faithful copy.

Chunked reading matters for binary files, because they can be large. Reading an entire image or archive into memory is wasteful when you only need to copy it or process it in pieces. The pattern is to read a fixed-size chunk, such as 8192 bytes, process or write it, and repeat until a read returns an empty bytes object, which signals the end of the file. This loop is the binary counterpart of line-by-line text iteration, and it uses the same idea: handle a manageable piece at a time, and let the loop end naturally when the data runs out.

You do not need to master binary formats to benefit from this section. The key lessons are the distinction between text and bytes, the need for explicit conversion, and the habit of chunked processing for large data. When you later meet formats such as images or compressed files, you will recognise the same pattern: open in binary mode, work with bytes, convert to text only when the bytes genuinely represent text, and keep chunks small enough to be comfortable in memory.

with open("data.bin", "wb") as f:
    f.write(b"hello")
with open("data.bin", "rb") as f:
    print(f.read())

Working with bytes also clarifies why some files cannot be opened as text at all. An image file contains byte sequences that do not correspond to printable characters, so decoding it as text produces nonsense or an error. That failure is not a flaw in the file; it is a signal that you chose the wrong mode. Matching the mode to the data, binary for raw bytes and text for characters, is therefore a basic correctness decision, not a stylistic preference.

Check: Why does writing to a binary file require a bytes object rather than an ordinary string?

Section 14

Standard input and output are files too, in a useful sense. When you call input, your program reads a line from standard input, which by default is the keyboard in a terminal, or piped data when the program is part of a larger command. When you call print, it writes to standard output, which by default is the terminal screen, but which can be redirected to a file by the shell. This uniformity means the same ideas of reading lines and writing text apply to interactive programs, to files, and to pipelines, with only the source or destination changing.

Redirecting output is a powerful testing habit. Running a program with its output sent to a file lets you inspect the result afterwards, compare it with an expected result, and share it with someone else. It also reveals whether your program prints extra blank lines, missing newlines, or debugging messages that should not be in the final output. A program that writes cleanly to standard output is easier to combine with other tools, because its output can become another program's input without cleaning.

Reading from standard input with a loop is another pattern worth learning. A program that reads lines until the end of input can process a file piped into it, or text typed by a user who signals the end with the terminal's end-of-file key. The loop ends naturally when there is no more input, just as a file loop ends at the end of the file. This design makes small tools composable: each tool does one job, reads from input, writes to output, and can be chained with others. The Unix tradition of small, composable tools rests on exactly this idea.

For beginners, the practical lesson is simpler: treat input and print as file operations with familiar defaults. When you need persistent storage, redirect or write to a named file; when you need interaction, use the defaults. As your programs grow, you may write to files directly, but the discipline of clean, line-oriented output will carry over. A program whose output is predictable, one record per line, is a program that other programs, and other people, can work with easily.

import sys
for line in sys.stdin:
    print(line.rstrip("\n").upper())

Pipelines also encourage you to keep each small program's own job narrow and its output clean. A tool that prints only its results, with no decorative banners, extra blank lines, or debugging chatter, can be chained with sorting, filtering, and counting tools without extra cleaning steps. If you later decide that a program needs a longer human-friendly report, you can add that formatting in a separate step, rather than mixing presentation into a tool that other programs depend on.

Check: How does redirecting a program's standard output to a file relate to writing to a file directly?

Section 15

Writing files safely means thinking about what happens if the program stops halfway through. A program that opens a file in write mode and then crashes leaves a partial file behind, and a later run may mistake that partial file for a complete one. One common safeguard is to write to a temporary file in the same folder, and then replace the target file with the temporary one only when the write has finished successfully. Replacing a file is usually a quick operation, so the window in which the target is in an inconsistent state is very small. This pattern is used by editors and installers for exactly this reason.

Another safeguard is to keep a backup before overwriting. Copying the existing file to a backup name, then writing the new version, means a failed write does not destroy the only good copy. Backups take disk space, and old backups need cleaning up, but for important data the trade is worthwhile. In learning exercises, the habit matters more than the mechanism: before a destructive operation, pause and ask what you would lose if it failed, and whether a copy would protect you. That question, asked consistently, prevents most regrettable overwrites.

Atomicity is the technical name for the idea that an operation either happens completely or not at all, with no half-finished state visible. True atomicity is hard to guarantee for large operations, but the temporary-file-and-replace pattern gets close for single-file writes. It does not protect against every failure, such as a disk error during the replace itself, but it eliminates the common case of a crash during writing leaving a truncated target. Understanding the limits of a safeguard is part of using it wisely, rather than trusting it blindly.

For small learning programs, these safeguards may feel like overkill, and that is fine. You do not need to implement temporary-file replacement for every exercise. What matters is recognising the situations where it is warranted: user data, configuration that would be painful to recreate, or any file whose loss would matter. When the stakes are low, a simple write is fine; when the stakes are high, add the safeguard, and explain in a comment why it is there.

from pathlib import Path
Path("draft.txt").write_text("new content\n", encoding="utf-8")
Path("draft.txt").replace("final.txt")
print(Path("final.txt").read_text(encoding="utf-8"))

The temporary-file pattern also helps when several programs might read the same file. Readers that open the file during a direct write could see a half-written version, whereas readers that open it before or after a quick replacement see either the old complete version or the new complete version. That consistency matters for configuration files and reports that other tools watch, because a half-written file can cause confusing errors far away from the program that wrote it.

Check: Why is writing to a temporary file and then replacing the target safer than writing directly to the target?

Section 16

Text encoding deserves a second look, because it is where file work most often goes wrong in practice. UTF-8 is the dominant encoding for new text, and it can represent every Unicode character, but older files may use other encodings, and a file with no encoding information cannot tell you which one it uses. When Python reads with the wrong encoding, it may raise a UnicodeDecodeError, or it may succeed but produce the wrong characters. The error is actually the friendlier outcome, because it tells you something is wrong; silent wrong characters can pass unnoticed into reports and databases.

When you encounter an encoding error, resist the urge to suppress it by ignoring errors or replacing characters without understanding the cause. Those options exist for cases where a few bad bytes are acceptable, such as recovering most of a damaged log, but they hide information. A better first step is to check how the file was created, what encoding the creator used, and whether a byte-order mark at the start gives a clue. For files you create, the answer is simple: always write UTF-8, and always say so explicitly in your code, so future readers do not have to guess.

The errors argument to open controls what happens on encoding problems. The default, strict, raises an error. Other values replace bad bytes with a placeholder or ignore them, which can be useful for recovery but should be a deliberate choice, not a default. If you use a lenient setting, log or report that data was lost or changed, so the user knows the output is not a faithful copy. Transparency about data loss is part of trustworthy programming, even when the loss is small.

Encoding also interacts with sorting and comparison. Two strings that look identical on screen may differ in their underlying characters, for example if one uses a composed character and the other uses a base character plus a combining mark. For most file exercises this subtlety will not arise, but it is worth knowing that visual sameness does not guarantee byte sameness. When exact matching matters, normalising text before comparison is a technique you will meet in more advanced work.

text = "café"
data = text.encode("utf-8")
print(data, data.decode("utf-8"))

A practical way to learn encoding is to experiment deliberately with a small file containing accented characters. Write it as UTF-8, read it back as UTF-8, and confirm the characters survive unchanged. Then try reading the same bytes with a different encoding, and observe how the characters change or fail. That controlled experiment turns an abstract warning about encodings into a concrete, memorable memory, which will help you recognise encoding problems quickly in real, messy, everyday files.

Check: Why is a UnicodeDecodeError often more helpful than silently replacing bad bytes when reading a file?

Section 17

Working with folders is a natural extension of working with files. A program may need to create an output folder, list the files in an input folder, or walk through a tree of folders to find every matching file. The pathlib module provides iterdir for one folder, and rglob for recursive searching through subfolders. Recursive searching is powerful, but it can also find far more files than you expect, including files in hidden folders or in folders you did not intend to process. Limiting the search by pattern, and by printing the results before acting on them, keeps recursive work under control.

Creating folders should be deliberate, and the parents and exist_ok arguments to mkdir make it safe to ensure a folder exists without failing if it already does. This is useful for output folders that a program creates on first run. It is less useful as a way to hide a mistake in a path: if a folder name is misspelled, creating it silently may scatter output into the wrong place. A balance is to create expected output folders, but to treat input folders differently, reporting a missing input folder as an error rather than creating an empty one that makes the program appear to succeed with no data.

Walking a folder tree also raises the question of order. The order in which entries are returned is not guaranteed to be sorted, and it may differ between systems. If your program's output should be predictable, sort the entries by name before processing them. Sorting is a small step that makes testing easier, because the same input always produces the same output order, and it makes reports easier to compare across runs. Predictability is a feature, not a luxury, in programs that process batches of files.

Finally, remember that folders, like files, can be renamed, moved, and deleted, and that deleting a folder with contents is a destructive operation. Python's tools for removing folders require the folder to be empty unless you use a recursive removal, which is deliberately a separate, more explicit step. That separation exists to protect you from accidental mass deletion, and you should respect it by testing folder operations on copies, and by confirming the target before any recursive delete.

from pathlib import Path
for path in sorted(Path(".").glob("*.txt")):
    print(path.name)

Recursive searching also requires care about symbolic links and deeply nested folders, which can make a walk take far longer than expected or revisit the same files repeatedly. For learning exercises, restrict a recursive search to a small test tree that you created yourself, and set a clear pattern so you know exactly what should be found. That discipline keeps exploration safe, and it mirrors how careful tools limit their scope before they start changing anything.

Check: Why should you sort folder entries before processing them when the output needs to be predictable?

Section 18

A useful file program often combines reading, transforming, and writing in a clear pipeline. For example, a program might read a list of names, remove duplicates, sort the result, and write it to a new file. Each stage is simple on its own, and the pipeline as a whole is easy to understand because data flows in one direction. When a stage fails, you can inspect its input and output separately, which makes debugging far easier than in a program where reading, transforming, and writing are tangled together in one loop. Separating stages is a design habit that scales to much larger programs.

Filtering is a common transformation. Given a file of records, keep only those that meet a condition, such as a score above a threshold or a city matching a name. The pattern is to read each line, test it, and write it only if it passes. The original file stays untouched, and the output file holds the filtered result. Keeping the input unchanged is a valuable safety property: if the filter was wrong, you can correct it and rerun, without having lost the original data. Programs that modify their input in place should do so only when that is the explicit intention, and with a backup.

Sorting is another transformation, but it requires all the data before output can begin, because the first output line depends on the whole input. That is why sorting programs typically read everything into a list, sort the list, and then write it out. This is a case where loading the whole file is justified, because the task genuinely needs a global view. Recognising which tasks need a global view and which can stream line by line is a key judgement in file programming, and it affects both memory use and program structure.

Counting and summarising are transformations that need almost no memory at all. A program can count lines, words, or matching records while reading, keeping only running totals. These summaries are often more useful than the raw data, because they answer a question directly: how many, how often, what is the total. A good summary program also reports what it counted, and over which file, so the numbers are interpretable without re-running it. Small, focused tools like these are the building blocks of larger data workflows.

names = ["Bola", "Ada", "Bola"]
unique = sorted(set(names))
with open("unique.txt", "w", encoding="utf-8") as f:
    for name in unique:
        f.write(name + "\n")

Pipelines become even clearer when each stage is a separate function with a descriptive name, such as read_records, filter_records, and write_records. A reader can then follow the main flow without getting lost in details, and each function can be tested on its own with small inputs. That separation is not just style; it is how large programs stay understandable, because a change to filtering does not require understanding the reading code in detail.

Check: Why is it safer for a filtering program to write to a new file rather than modifying its input file in place?

Section 19

File programs often need to handle more than one file at once, and Python's with statement can open several files in a single block. You can write with open("in.txt") as source, open("out.txt", "w") as target: and both files will be closed when the block ends, even if an error occurs. This is cleaner than nesting with blocks, and it makes the relationship between the files visible: this block reads from one and writes to the other. When the files are independent, separate blocks are fine, but when they form a pair, opening them together expresses that pairing.

Opening multiple files also requires thinking about failure order. If the input file cannot be opened, the output file should not be created or truncated before the error is known. Opening the input first, and only then opening the output, ensures that a missing input does not destroy an existing output. In a with statement with several opens, they happen left to right, so order matters and should be chosen deliberately. A program that opens its output first and then fails to open its input has already done damage, even though it never processed a line.

Another multi-file pattern is merging: reading several input files and writing their contents, in order, to a single output. This is useful for combining daily logs, chapters, or reports. The pattern is a loop over input paths, with an inner loop over lines, all writing to one open output file. Headers or separators between inputs can help the reader tell where one file ended and the next began. Without them, merged data can be hard to interpret, especially if the inputs do not end with a newline.

Splitting is the reverse: reading one large file and writing its contents across several output files, perhaps one per category or per day. Splitting requires managing several open output files at once, or opening and closing each one as needed. Opening all of them at once is efficient but uses more resources; opening and appending as needed is simpler to reason about but slower. For learning exercises, the append-as-needed pattern is usually clear enough, and it avoids the complexity of tracking many open files simultaneously.

with open("a.txt", "r", encoding="utf-8") as src, open("b.txt", "w", encoding="utf-8") as dst:
    for line in src:
        dst.write(line)

When merging files, it is also helpful to record where each line came from, especially if the inputs overlap or disagree. A simple prefix, or a header line naming the source file, can save hours of detective work later when a merged report contains a surprising value. Provenance, the record of where data came from, is a quiet quality that distinguishes a careful data program from one that merely concatenates bytes and hopes for the best.

Check: When opening an input and an output file together, why should the input be opened before the output?

Section 20

Testing file code requires a plan, because file code touches the world outside the program. A good approach is to create small, known test files in a temporary folder, run your program on them, and compare the output with what you expect. Temporary folders, created by the tempfile module or simply by using a dedicated test folder, keep test files separate from real data, so a bug in test code does not damage important files. After the test, the temporary folder can be removed, leaving no clutter behind. This separation is what makes file testing safe to repeat often.

You should test the edges, not just the happy path. Test an empty file, a file with one line, a file without a final newline, a file with blank lines, and a missing file. Each of these cases has surprised programmers in real projects, and each is cheap to test. A program that handles an empty file by producing an empty output, rather than crashing, is more trustworthy than one that only works on perfect input. Edge-case testing is not pessimism; it is a recognition that real files are messy, and that programs should meet them where they are.

Comparing output files is a practical testing technique. Instead of eyeballing results, write the expected output to a file and compare it with the actual output, either by eye in a small case or with a comparison tool in a larger one. If they differ, the difference itself is informative: a missing newline, an extra blank line, or a wrong order all show up as concrete differences, not vague feelings. This habit also creates a record of what correct behaviour looks like, which future changes can be checked against.

Finally, test code should clean up after itself, or at least keep its mess in one place. A test that leaves stray files in the project folder creates confusion for the next person, who cannot tell which files are real and which are test artefacts. Using a single test folder, with a clear name, and removing it when done, keeps the project tidy. Tidy test practice is part of professional file handling, not an optional extra, because the filing system is shared space, and leaving it clean is a courtesy to everyone who uses it.

from pathlib import Path
Path("test_input.txt").write_text("a\nb\n", encoding="utf-8")
print(Path("test_input.txt").read_text(encoding="utf-8"))

Testing with temporary folders also makes it easy to run the same test repeatedly without manual cleanup. Each run starts from a known state, creates its inputs, checks its outputs, and removes its folder, so a previous run cannot contaminate the next one. Repeatability is what turns a one-off experiment into a real test, because a test that passes only when run in a particular order, or only once, is not actually proving that the code works.

Check: Why is it important to test file code with an empty file and a file without a final newline, not just with perfect input?

Section 21

Reading configuration from a file is one of the most common uses of text input. A simple configuration file might hold one setting per line, such as theme = dark or limit = 20, and a program reads those lines, splits each at the equals sign, and stores the results in a dictionary. This pattern separates behaviour from code: changing the theme or the limit means editing the file, not the program. It also makes a program easier to test, because different configurations can be tried simply by swapping files.

A configuration parser needs to handle the realities of human-edited files. Lines may have spaces around the equals sign, blank lines may separate groups of settings, and lines starting with a marker such as # may be comments that should be ignored. A tolerant parser strips whitespace, skips blanks and comments, and reports lines that do not contain an equals sign, rather than crashing. Tolerance here does not mean accepting nonsense; it means accepting harmless variation in formatting while still catching genuine mistakes.

Defaults are an important part of configuration handling. If a setting is missing from the file, the program should fall back to a sensible default rather than failing, unless the setting is essential and has no safe default. Documenting which settings exist, what they mean, and what their defaults are, turns a mysterious file into a usable interface. A program that prints its effective configuration at startup, perhaps when asked, helps users see what it will actually do, rather than guessing from the file alone.

Types return as a question in configuration files, because every value arrives as text. A limit that looks like 20 must become an integer before it can be compared, and a flag that looks like true must become a boolean by an explicit rule you define. Converting with clear rules, and reporting a value that cannot be converted, keeps configuration predictable. A setting that silently stays as text when a number was expected can cause comparisons to behave strangely, and those bugs are hard to trace back to the file.

config = {}
for line in ["theme = dark", "limit = 20"]:
    key, value = [part.strip() for part in line.split("=", 1)]
    config[key] = value
print(config)

A configuration file also gives a program a memory of its user's preferences without requiring a database or a graphical settings screen. A user who edits a plain text file can change behaviour, share a working configuration with a colleague, or restore a previous setup by copying a file. That simplicity is why configuration files remain common even in an age of complex applications: they are transparent, editable, and easy to back up alongside the data they control.

Check: Why should a configuration parser skip blank lines and comment lines instead of treating them as settings?

Section 22

Log files are a special kind of text output: a chronological record of what a program did, written one event per line. Each line typically includes a time, a level such as INFO or ERROR, and a message describing the event. Logs are not primarily for users; they are for the person diagnosing a problem later, who needs to reconstruct what happened, in what order, and with what result. Writing clear log lines is therefore a form of communication with a future reader, who may be you, investigating a failure at an inconvenient time.

Appending is the natural mode for logs, because each run should add to the history rather than erase it. A log file that is overwritten on every run loses the very history that makes it useful. Over time, however, logs grow, and managing their size becomes a task in itself. Real systems rotate logs, moving the current file aside and starting a fresh one when it reaches a size or age limit. For learning programs, simply being aware that logs grow, and occasionally clearing or archiving them, is enough; the principle matters more than the mechanism.

What to log is a judgement call. Logging every trivial step produces noise that hides important events, while logging only errors leaves gaps when you need to understand the steps that led to an error. A useful rule is to log decisions, boundaries, and failures: when the program starts, when it reads or writes an important file, when it makes a choice that affects the result, and when something goes wrong. Those entries form a narrative that can be followed, rather than a flood of detail that must be searched.

Log messages should be specific and self-contained. A message such as Failed tells the reader almost nothing, while Failed to read notes.txt: file not found names the operation, the file, and the reason. Including the relevant values, without including sensitive information such as passwords, makes logs genuinely useful. Sensitive data should never be written to a log, because logs are often shared, copied, and kept for a long time, and a secret that enters a log is very hard to remove completely.

with open("app.log", "a", encoding="utf-8") as f:
    f.write("INFO: Program started\n")
    f.write("INFO: Read notes.txt\n")

Log files also become evidence when something goes wrong in a program that runs unattended, such as a scheduled task that runs overnight. Without a log, a failure leaves only its effects, and the cause must be guessed; with a log, the sequence of events can be traced, and the failing step identified. That diagnostic value is why even quite small programs benefit from a few well-chosen log lines, written consistently from the start. Keep this habit consistent in every project you build.

Check: Why is appending the natural file mode for a log file, and what problem does log rotation solve?

Section 23

CSV files, comma-separated values, look simple, but they contain traps that plain splitting does not handle. A field may itself contain a comma, in which case it is wrapped in quotes, and a quoted field may contain a quote character, represented by doubling it. A program that splits naively on commas will break such fields apart incorrectly, producing too many columns and corrupted data. Python's csv module understands these rules, and using it is almost always better than writing your own splitter for comma-separated data. This chapter introduces the idea; a later chapter covers data handling with CSV in depth.

The csv reader turns each row into a list of fields, handling quotes and embedded commas correctly, while the writer does the reverse, quoting fields that need it. Opening a CSV file for the csv module requires a small detail: use newline="" when opening, so that the module controls line endings itself. Without that detail, extra blank lines can appear on some systems, because both Python's text mode and the csv module try to handle newlines. It is a fiddly detail, but it is documented, consistent, and easy to copy correctly once you have seen it.

Even with the csv module, validation remains your job. A row may have too few or too many fields, a number may not parse, or a required field may be empty. The module gives you the fields faithfully; it does not judge whether they make sense. Checking row length, converting types carefully, and reporting the row number of a bad row are the same disciplines you practised with plain text parsing, applied to a format with stricter rules. A CSV reader that reports problems precisely is far more useful than one that silently drops bad rows.

Writing CSV data follows the same pattern in reverse: build rows as lists, and let the writer handle quoting. Do not build CSV lines by joining fields with commas yourself, because that reintroduces exactly the quoting problem the module solves. When your data contains commas, quotes, or newlines, the writer will quote it correctly, and another program reading the file with a proper CSV reader will recover the original fields exactly. That round-trip reliability is the whole point of using a standard format.

import csv
with open("people.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["name", "city"])
    writer.writerow(["Ada", "Lagos"])

The csv module also illustrates a broader lesson: use a standard library tool for a standard format, rather than reinventing its rules. Formats like CSV have edge cases that only become visible when real data arrives, with commas inside fields, quotes, and blank lines. A library that has handled those cases for years will be more reliable than a hand-written splitter, and it will make your code shorter and easier for others to trust.

Check: Why should you use the csv module rather than splitting on commas when reading a CSV file?

Section 24

Files can be large, and programs that process them should be mindful of memory and time. The simplest large-file strategy is streaming: read one line or chunk at a time, process it, write any output, and discard it before reading the next. Streaming keeps memory use roughly constant, no matter how large the file grows, because only a small piece is held at a time. It works for counting, filtering, transforming, and summarising, which cover a surprising share of real file tasks. When a task can stream, it usually should, because streaming scales gracefully.

Some tasks cannot stream, because they need a global view of the data. Sorting, finding a median, and removing duplicates across the whole file all require seeing everything before producing output, or at least keeping a substantial structure in memory. For those tasks, loading the file is justified, but it is worth asking whether a smaller structure would do: to remove duplicates, a set of seen values may be enough, without storing every record. To find the largest value, a single running maximum is enough. Thinking about the smallest state that answers the question is a powerful optimisation habit, and it often turns a memory-hungry program into a streaming one.

Time matters too, though less often for beginners. Reading a file line by line in Python is fast enough for files of ordinary size, and most file programs are limited by disk speed rather than by Python itself. Premature optimisation, such as writing complicated buffering code for a small file, usually costs more in clarity than it saves in time. The right order is to write a clear, correct program first, measure it on realistic data if it feels slow, and only then optimise the part that the measurement shows to be slow. Clear code that is fast enough beats clever code that is hard to trust.

Finally, be careful with programs that read and write the same file in one pass. If you open a file for writing, you truncate it before reading, so the input is gone. The safe pattern, already mentioned, is to read first, process, and then write, either to a new file or back to the original after reading has finished. This ordering rule prevents a whole class of data-loss bugs, and it is worth stating explicitly in code comments when a program overwrites its own input, so future readers understand that the ordering is deliberate.

count = 0
with open("notes.txt", "r", encoding="utf-8") as f:
    for line in f:
        if line.strip():
            count += 1
print("non-empty lines:", count)

Streaming also changes how you think about testing, because a streaming program can be tested on a small file and trusted, with reasonable confidence, on a much larger one, provided the per-line logic is correct. A program that loads everything, by contrast, may pass small tests and fail on large inputs for reasons unrelated to its logic, such as memory exhaustion. Testing with a moderately large file, therefore, is a useful check for programs that claim to handle real data.

Check: What is the smallest piece of state needed to count non-empty lines in a large file, and why does that allow streaming?

Section 25

Error handling for files is introduced here and developed in the next chapter, but a few patterns belong in this chapter because they shape how file code is written. The most common file errors are a missing file, a missing folder, a permission problem, and an encoding problem. Each has a distinct exception type, and catching the specific type lets you give a specific message. Catching every error with one broad handler, by contrast, hides the difference between a missing file, which the user can fix by creating it, and a permission problem, which may need a different action entirely.

A helpful pattern is to try the operation, catch the expected exceptions, and report them in plain language, while letting unexpected exceptions propagate so they are not hidden. This balance respects both the user, who gets a clear message for ordinary problems, and the developer, who still sees the traceback for genuinely surprising failures. Over-catching, where every error is swallowed and the program continues as if nothing happened, is a common beginner mistake that makes debugging much harder, because the program's later behaviour no longer reflects what actually went wrong.

Another pattern is to validate inputs before opening files, when validation is cheap. Checking that a filename is not empty, or that a folder exists, can give an early, clear error before any file is touched. Validation does not replace exception handling, because conditions can change between the check and the open, but it complements it by catching obvious mistakes early, with messages that can be more specific than a generic file error. The two techniques together, validate early and handle exceptions, cover both the predictable and the unpredictable cases.

Resource cleanup is the final piece. The with statement guarantees that files are closed, even when an exception occurs, which is why it appears in nearly every example in this chapter. Code that opens files without with must close them in a cleanup section that runs even on error, which is more code and more opportunity for mistakes. Using with consistently is therefore not just a style choice; it is an error-handling strategy that ensures resources are released no matter how a block ends.

try:
    with open("missing.txt", "r", encoding="utf-8") as f:
        print(f.read())
except FileNotFoundError:
    print("missing.txt was not found.")

File error handling also teaches a general lesson about program boundaries: wherever your program meets the outside world, whether a file, a network, or a user, it should expect the unexpected. Files disappear, disks fill, and permissions change, and none of those events means your program's logic is wrong. Handling them gracefully, with clear messages and safe cleanup, is what separates a prototype that works in ideal conditions from a tool that works in real ones.

Check: Why is it better to catch a specific exception like FileNotFoundError than to catch every possible error in one handler?

Section 26

When a program writes a file for a human to read, formatting matters as much as correctness. A report with clear headings, blank lines between sections, aligned columns, and a consistent date format is far easier to use than a dense block of values, even if both contain the same information. Python's string formatting, covered in earlier chapters, is the tool for this: you build each line as a formatted string, then write it. The file-writing code stays simple, because the formatting work happens in the string, before the write. Separating formatting from writing keeps both parts readable.

Alignment is a small detail with a large effect on readability. Columns that line up let the eye scan down a list of values quickly, while ragged columns force the reader to hunt for each value. Using a fixed width for a column, such as formatting a name in twenty characters, produces aligned output for ordinary values, though very long values will overflow the width and break alignment. For learning reports, fixed widths are a good compromise; for production reports, dedicated formatting tools handle overflow more gracefully. The principle is the same: think about the reader's eye, not just the data.

Numbers in reports also deserve formatting thought. A count is best shown as a plain integer, a money-like value with two decimal places, and a percentage with a clear sign, so the reader does not have to guess what a bare number means. Adding a unit or a label, such as items: 42 rather than just 42, removes ambiguity, especially when a report is read out of context months later. A report that explains itself, with headings and labels, is a report that remains useful after the program that made it is forgotten.

Finally, a report file should say when it was made and from what data, at least in a header line. That provenance helps a reader judge whether the report is current, and it helps a debugger reproduce it. A report without a date or source is a mystery; a report with a simple header such as Report for notes.txt, generated on a given date is a document. Even in exercises, adding a header line trains you to think of output files as artefacts that other people, and future you, will need to understand.

with open("report.txt", "w", encoding="utf-8") as f:
    f.write("Sales report\n")
    f.write(f"{'Item':<12}{'Count':>6}\n")
    f.write(f"{'Apples':<12}{42:>6}\n")

Formatting also helps when a report is compared across runs, because consistent layout makes differences stand out. If each report uses the same headings, column widths, and ordering, a reader can spot a changed value quickly, without wondering whether the layout itself changed. Consistency in output is therefore not mere polish; it is a tool for noticing change, which is often the whole purpose of generating a report in the first place.

Check: Why does a report file benefit from a header line stating its source and date, even in a small exercise?

Section 27

Sometimes a program needs to read a file that another program is still writing, such as a log file that grows while a service runs. Reading such a file is like reading a book that is still being written: you can read up to the current end, but more may appear later. A simple approach is to read the file, process what is there, and read again later, perhaps after a pause, picking up from where you left off. Keeping track of the position, or simply re-reading and skipping lines you have already seen, are both workable strategies for small tools, though each has trade-offs in efficiency and complexity.

Locking is the formal mechanism by which programs coordinate access to a shared file, but it is platform-dependent and beyond the scope of this chapter. For learning, the important lesson is awareness: when two programs share a file, each must assume the other may change it, and neither should assume it has exclusive control. Reading a file that is being written can yield a partial last line, because the writer may be in the middle of writing it. A program that treats a partial line as complete may misparse it, so checking for a trailing newline, or waiting for the line to complete, is a prudent step.

A related situation is a program that writes a file while another program reads it. If the writer replaces the file, a reader holding the old file open may continue reading the old contents, while a reader opening it fresh sees the new contents. This behaviour depends on the operating system and on how the replacement is done, and it can be surprising. The practical takeaway is to avoid designing systems where correctness depends on precise timing between a reader and a writer, unless you have a coordination mechanism. For single-program exercises, this complexity does not arise, and that simplicity is a feature to appreciate.

In short, shared files are a coordination problem, not just a storage problem. Most file programs in this book are single-program tools, where one program owns the file at a time, and that ownership makes reasoning simple. When you later build programs that share files, you will need to think about locking, atomic replacement, and partial reads, but you will be building on the same foundations: open, read, write, close, and respect the position and the encoding.

with open("app.log", "r", encoding="utf-8") as f:
    lines = f.readlines()
print("lines so far:", len(lines))

Shared-file awareness also applies to the simpler case of a user editing a file while your program reads it. A text editor may save a file by replacing it, which means a program reading the old file and a program opening the new file can see different contents at the same moment. That is usually harmless for a learning exercise, but it explains why an editor and a program can disagree about a file's contents, and why saving before running is a sensible habit.

Check: Why might reading a log file that is still being written give you a partial last line?

Section 28

Backups are a file habit, not just a tool. A program that creates important data should make it easy to copy that data somewhere safe, and a user who values their data should copy it regularly. From a programming perspective, the simplest backup is a copy of a file under a new name, perhaps with a date in the name, stored in a different folder. That copy protects against accidental overwriting and against a program bug that corrupts the original, though it does not protect against the loss of the whole computer. Real backup systems copy to separate devices or services, but the principle is identical: a second copy, made before it is needed.

Versioning is a step beyond backup. Instead of keeping one backup, a program or a workflow keeps several versions, so that a change made days ago can be undone even after several newer versions have been saved. Version control systems, covered in a later chapter, formalise this idea for code, tracking every change with a message and allowing any earlier version to be restored. For data files, a simple versioning scheme, such as keeping the last few dated copies, captures much of the benefit without any special tool. The key insight is that the ability to go backwards is valuable, and it must be built in before it is needed.

A program can help versioning by never silently destroying the previous version of a file it overwrites. Writing to a new file, keeping the old one, or moving the old one to a backup name are all ways to preserve history. Each approach costs disk space, and old versions eventually need pruning, but the cost of losing irreplaceable data is usually far higher. When you design a file-writing program, ask what happens to the previous version, and make the answer deliberate rather than accidental.

For learning, the practical habit is small: before you run a program that overwrites a file you care about, make a copy. That copy takes seconds, and it turns a potential disaster into a minor inconvenience. As you gain experience, you will automate backups and versioning, but the manual habit teaches the underlying value: data that exists in only one place is data at risk, and a programmer who respects data keeps more than one copy of what matters.

import shutil
shutil.copy("notes.txt", "notes-backup.txt")
print("backup created")

Backups also need testing, because a backup that has never been restored is only a hope, not a guarantee. Occasionally copying a backup back to a test folder and opening it confirms that the copy is complete and readable, and that you remember the steps to restore it. That small rehearsal is worth more than a shelf of untested copies, because it turns a backup from a file you assume is good into a file you know is good.

Check: How does keeping dated copies of a file differ from keeping just one backup, and why might that matter?

Section 29

Text files are also a bridge to other tools. A program can write a file that a spreadsheet, a text editor, a web browser, or another program can open, and that interoperability is a major reason text formats remain popular. A plain text report can be read by anyone, without special software. A comma-separated file can be opened in a spreadsheet for analysis. A file with one item per line can be sorted, counted, and filtered by command-line tools that have existed for decades. Writing text that other tools can consume multiplies the value of your program, because its output does not stay trapped inside Python.

Interoperability imposes discipline on format choices. If another tool expects one record per line, your program must not put two records on one line, or break a record across lines without a quoting rule. If a spreadsheet expects a header row, omitting it makes the data harder to use. Thinking about the consumer of your file, whether human or program, and matching its expectations, is part of file programming, not an afterthought. A format is a contract, and contracts work best when both sides honour them.

Plain text also has limits, and recognising them helps you choose the right format for a task. Text cannot efficiently store binary data, it does not enforce types, and it does not describe its own structure beyond what headers and conventions provide. For complex data, structured formats such as JSON, covered in a later chapter, add type information and nesting, while databases add querying and integrity. Choosing text when it fits, and a richer format when it does not, is a judgement that grows with experience, and it starts with understanding what text does well: simple, readable, portable records.

A good exercise is to write a program whose output you then use in another tool, even a simple one. Write a list to a file, open it in a text editor, and check that it looks as expected. Or write a CSV file and open it in a spreadsheet, if one is available. That round trip through another tool reveals formatting problems that Python alone might not show, such as a missing header or an encoding issue, and it builds confidence that your files are genuinely usable, not just correct in isolation.

with open("words.txt", "w", encoding="utf-8") as f:
    for word in ["apple", "banana", "cherry"]:
        f.write(word + "\n")

Interoperability also means your files may outlive your program, which is a reason to choose formats that are widely understood. A plain text file written today can still be read decades later with almost any tool, whereas a proprietary binary format may require software that no longer exists. Longevity is not always the top priority, but for notes, records, and reports that matter over time, a simple, open format is a gift to your future self.

Check: Why does writing output as plain text make a program's results more useful to other tools and people?

Section 30

File permissions and ownership are part of the filing system that programs inherit, even when they do not manage them directly. On most systems, each file has an owner, and permissions that say who may read, write, or execute it. A program runs with the permissions of the user who started it, so it can only read files that user may read, and only write to folders that user may write to. When a program fails with a permission error, it is usually because it is trying to cross one of those boundaries, not because the file itself is broken. Understanding that helps you diagnose the error correctly.

For learning programs, the practical rule is to keep data files in folders you own and control, such as your project folder or a documents folder, and to avoid writing to system folders that require special privilege. Writing to a system folder may seem to work if you run with elevated privilege, but it creates files that ordinary runs cannot update, and it risks damaging the system. Programs should ask for no more privilege than they need, and users should be wary of programs that demand more. Least privilege is a security principle that applies to file access as much as to any other resource.

Permissions also matter for files you share. A file you create may be readable by others on the same computer, depending on default permissions, and a file containing private notes or personal data should be stored where others cannot read it, or with restricted permissions. Python does not change default permissions by itself, so the protection comes from where you put the file and how the system is configured, not from the code. Being aware of that, and choosing sensible locations for sensitive files, is part of responsible programming, even before you learn to set permissions explicitly.

Finally, remember that deleting a file does not necessarily erase its contents from the disk immediately; it removes the name that points to those contents, and the space may be reused later. For ordinary files this distinction does not matter, but for sensitive files it means that deletion alone is not secure erasure. Secure deletion is a specialised topic, and the lesson here is simply not to assume that deleting a sensitive file makes it unrecoverable. Keep sensitive data out of files you do not need, and treat file storage as a place where data persists in ways that deserve thought.

from pathlib import Path
path = Path("notes.txt")
print("exists:", path.exists())

Permissions also remind you that a program's view of the filing system is limited by who runs it, not just by what the code says. Two users running the same program may see different files, succeed on different operations, and receive different errors, all because their permissions differ. Testing a program as the user who will actually run it, in the folder where it will actually run, is therefore more informative than testing it only in a privileged or specially prepared environment.

Check: Why should a program keep its data files in folders the user owns, rather than writing to system folders?

Section 31

A task list stored in a file is a good first project for file skills, because it combines reading, writing, and updating in a simple, useful form. Each task can be one line, perhaps with a marker for done or pending, such as [ ] Buy milk or [x] Buy milk. Adding a task appends a line, listing tasks reads the file, and completing a task requires reading all lines, changing one, and writing the file back. That read-modify-write cycle is the pattern behind many small data programs, and it is worth practising until it feels natural.

The read-modify-write cycle has a subtle risk: if the program crashes between reading and writing, no change is saved, which is safe, but if it writes a partial file, data can be lost. Using the temporary-file pattern from earlier, or at least writing to a new file and then replacing the old one, reduces that risk. For a learning task list, a straightforward write-back is acceptable, but knowing the safer pattern means you can upgrade when the data matters more. The difference between a toy and a tool is often just that kind of care.

Designing the line format is also part of the project. A format that is easy for a human to read, such as a checkbox marker at the start, is also easy for a program to parse, because the marker is in a fixed position. A format that buries the status in the middle of free text is harder for both. Choosing a format that serves both readers, human and program, is a recurring theme in file work, and a task list is a gentle place to practise it. When the format is clear, the code that reads and writes it stays clear too.

Finally, a task list program should handle the case where the file does not exist yet. On first run, there are no tasks, and the program should treat that as an empty list, not as an error. Creating the file on first write, or simply handling the missing-file case when listing, gives the program a friendly first-run experience. That small detail, an empty list rather than a crash, is often the difference between a program that feels finished and one that feels fragile.

tasks = ["[ ] Buy milk", "[x] Read chapter"]
with open("tasks.txt", "w", encoding="utf-8") as f:
    for task in tasks:
        f.write(task + "\n")

A task list also shows why a program should confirm a change before discarding the old version. If completing a task rewrites the file, a user who completes the wrong task by mistake needs a way to undo it, whether that is a backup copy or an explicit uncomplete command. Designing for mistakes, not just for correct use, is a sign of a mature tool, because users will always make slips, and a program that forgives them is trusted more.

Check: In a task-list program, why does completing a task require reading the whole file and writing it back?

Section 32

A notes program is similar to a task list, but its records are longer and less structured. A note might be a title, followed by several lines of text, followed by a blank line that separates it from the next note. Parsing such a file means reading lines, grouping them into notes, and deciding where one note ends and the next begins. The blank-line separator is a simple convention that humans understand intuitively, and that programs can implement by accumulating lines until a blank line appears, then saving the accumulated note and starting a new one.

Searching notes is a natural feature to add. A program can read all notes, check whether a search word appears in the title or body, and print only the matching notes. Because notes are text, searching is straightforward, though case sensitivity matters: a search for python will not match Python unless you normalise both sides, typically by comparing lowercased versions. That normalisation is a small step, but it makes search feel helpful rather than pedantic, and it illustrates how a tiny transformation can change the user experience.

Editing a note follows the same read-modify-write pattern as completing a task, but with larger units. The program reads all notes, finds the one to edit, replaces its text, and writes all notes back. If notes are stored as blocks separated by blank lines, writing them back means joining the blocks with blank lines, preserving the format. A mistake in the joining logic can merge two notes or split one, so testing the round trip, read then write without changes, and checking that the file is unchanged, is a valuable test. A program that can round-trip its own format without corruption has a solid foundation.

Notes also raise the question of titles and identifiers. If two notes have the same title, how does the program know which one to edit? Giving each note a number when listing it, and editing by number, solves the ambiguity without requiring unique titles. That numbering is a user-interface choice as much as a file-format choice, and it shows how file design and program design influence each other. The file stores the notes; the program provides a way to refer to them unambiguously.

notes = ["Shopping\nBuy milk\n", "Ideas\nLearn Python\n"]
for i, note in enumerate(notes, start=1):
    print(f"{i}. {note.splitlines()[0]}")

Notes also benefit from a simple export: writing all notes, or the matching notes, to a single readable file that can be printed or shared. Export turns a program's internal storage into a document, and it tests whether the file format is truly self-explanatory. If an exported file looks confusing without the program, the format may need clearer titles, separators, or headings, which will improve the program itself as well.

Check: Why is a blank line a useful separator between notes in a plain text notes file?

Section 33

Counting words, lines, and characters in a file is a classic exercise that teaches streaming, string handling, and summary reporting in one small program. Counting lines is easy when iterating, because each iteration is one line. Counting words requires splitting each line on whitespace, and counting characters requires measuring the length of each line, with a decision about whether the newline character counts. Those decisions should be stated in the program's output, so a user knows what the numbers mean. A count without a definition is less useful than it appears.

Word counting also introduces the question of what counts as a word. Splitting on whitespace treats hello, and hello as different words, because the comma is attached to the first. For a simple count, that may be acceptable, but for a frequency count, where you want to know how often each word appears, punctuation and capitalisation need normalising. Stripping punctuation and lowercasing before counting gives a cleaner frequency table, though it also loses information, such as the distinction between a sentence-starting capital and a proper name. Every normalisation is a trade, and the right trade depends on the question being asked.

Frequency counting uses a dictionary to map each word to its count, incrementing as each word is seen. That dictionary is the program's running state, and it grows with the number of distinct words, not with the size of the file, which is usually manageable. After reading, the program can sort the words by frequency and print the most common ones, which often reveals the topic of a text at a glance. This simple technique, counting and sorting, underlies many text-analysis tools, from spell checkers to search engines, in much more elaborate forms.

A word-count program is also a good place to practise handling command-line arguments or user input for the filename, rather than hard-coding it. A program that can count any file the user names is more useful than one that only counts a fixed file, and it teaches the separation between the program's logic and the specific data it runs on. That separation, logic in code and data in files, is the same principle that makes configuration files and task lists flexible, applied to a new kind of tool.

from collections import Counter
words = "the cat sat the cat".split()
print(Counter(words).most_common(2))

Word counting also leads naturally to thinking about language: a text with many repeated words may be a list, a text with a wide vocabulary may be prose, and a text with unusual tokens may contain codes or names. Those observations are the beginning of text analysis, where simple counts reveal structure. Even without advanced tools, a frequency table can tell you what a document is about, which words dominate, and where to look next.

Check: Why should a word-frequency program normalise capitalisation and punctuation before counting?

Section 34

Sorting a file's lines is a useful transformation that illustrates the difference between streaming and whole-file processing. To sort, you must read all lines, sort them, and write them out, because the first output line cannot be known until every input line has been seen. Python's sorted function makes the sorting itself trivial, but the surrounding decisions matter: should the sort be case-sensitive, should blank lines be kept or dropped, and should the trailing newlines be preserved or re-added on writing. Each decision changes the output, and each should be made deliberately.

Case sensitivity is a common surprise in sorting. In the default ordering, uppercase letters sort before lowercase letters, so Banana comes before apple, which looks wrong to a human reader expecting alphabetical order. Sorting by a lowercased key, using sorted(lines, key=str.lower), gives the expected human order, while preserving the original capitalisation in the output. That distinction, between the key used for ordering and the value that is output, is a general sorting technique, and it appears in many contexts beyond files, wherever display order should differ from raw order.

Stable sorting is another concept worth knowing. Python's sort is stable, meaning that items that compare equal keep their original relative order. For lines that are identical, that does not matter, but for records sorted by one field, stability preserves the order of records that tie on that field, which can be meaningful. Stability is a guarantee you can rely on, and it makes sorting predictable in ways that matter when a file is sorted repeatedly or by successive keys.

Sorting also provides a way to find duplicates: after sorting, duplicate lines sit next to each other, so a single pass can detect them. That technique is more memory-efficient than keeping a set of all seen lines in some cases, though for ordinary files a set is simpler. The point is not that one method is always better, but that sorting transforms a global problem, finding duplicates anywhere in a file, into a local problem, comparing neighbours, which is easier to reason about and to implement correctly.

lines = ["banana\n", "Apple\n", "cherry\n"]
for line in sorted(lines, key=str.lower):
    print(line.rstrip("\n"))

Sorting also raises the question of stability across runs: a program that sorts the same input twice should produce the same output twice, which Python's deterministic sort guarantees. That repeatability matters for testing, because a test that compares output files will fail if the order changes randomly. Deterministic behaviour, where the same input always gives the same output, is a property worth preserving deliberately in every transformation you write, because it makes testing, comparison, and debugging far more reliable over time, even as the program grows and changes daily.

Check: Why does sorting lines by a lowercased key give a more human-expected order than the default sort?

Section 35

Merging and splitting files are operations that combine the skills of reading, writing, and iteration. Merging takes several input files and produces one output, in a defined order, perhaps with a separator or header between inputs. Splitting takes one input and produces several outputs, perhaps one per category, per day, or per fixed number of lines. Both operations are common in data preparation, where files arrive in pieces and must be combined, or arrive as one large file and must be divided for processing or sharing.

When merging, order and provenance matter. The order of inputs determines the order of the output, and a reader of the merged file should be able to tell, if it matters, where each part came from. Adding a short header before each input's contents, naming the source file, is a simple way to preserve provenance, and it costs only a few lines. Without such headers, a merged file can be a puzzle, especially if the inputs have similar content or overlapping records. A merged file that explains itself is more useful than one that merely concatenates.

When splitting, the key decision is the rule that assigns each line to an output file. The rule might be a field value, such as a city name, or a count, such as every thousand lines to a new file, or a pattern in the line itself. Whatever the rule, it should be consistent and documented, so that the split can be understood and, if needed, reversed by merging. A split that cannot be explained is hard to trust, because there is no way to check whether a line went to the right file without re-running the rule.

Both operations should preserve the input files unchanged, writing only to new outputs. That safety property means a mistake in merging or splitting does not destroy the originals, and the operation can be corrected and rerun. It also means the program's inputs and outputs are clearly separated, which makes testing easier: run the program, compare the outputs with expected results, and know that the inputs are still intact for the next test run.

with open("merged.txt", "w", encoding="utf-8") as out:
    for name in ["a.txt", "b.txt"]:
        try:
            with open(name, "r", encoding="utf-8") as f:
                out.write(f.read())
        except FileNotFoundError:
            print(f"Skipped missing {name}")

Merging and splitting also teach you to think about reversibility: a good split can be undone by a merge, and a good merge preserves enough information to be split again if needed. Operations that lose information, such as dropping headers or discarding source names, make reversal impossible. Keeping the information needed to reverse an operation, even if you do not plan to reverse it, is a form of future-proofing that costs little and can save a great deal.

Check: When merging files, why is it helpful to add a header naming the source file before each input's contents?

Section 36

Reading a file in chunks, rather than by lines, is useful when the data is not line-oriented. Binary files, long records without newlines, and files where you want to control memory precisely all benefit from chunked reading. The pattern is to call read(size) repeatedly, with a size such as 4096 or 8192, until it returns an empty result, which signals the end of the file. Each chunk is processed or written before the next is read, so memory use stays bounded by the chunk size, regardless of file size. This is the same streaming idea as line iteration, applied to a different unit.

Chunk boundaries rarely align with meaningful units. A chunk may end in the middle of a word, a number, or a multi-byte character, and a program that processes chunks independently must handle those splits. For copying, splits do not matter, because chunks are written unchanged and reassembled by concatenation. For parsing, splits matter a great deal, and the program must carry over the partial unit from one chunk to the next. That carry-over logic is a common source of bugs, which is why line iteration is preferred whenever the data is line-oriented: it guarantees that each unit is complete.

Choosing a chunk size is a trade between efficiency and memory. Very small chunks mean many read operations, which is slow; very large chunks use more memory, but reduce the number of operations. For ordinary programs, a size in the range of a few thousand bytes is a sensible default, and the exact value rarely matters much. The important choice is to chunk at all, rather than loading everything, when the file may be large. A program that chunks is a program that scales, at least in memory, to files far larger than the ones used in testing.

Chunked reading also appears in network and compression work, where data arrives in pieces and must be processed as it arrives. Learning the pattern with files gives you a mental model that transfers: read a piece, handle it, repeat until done, and handle the final partial piece correctly. That model, simple as it sounds, underlies a great deal of systems programming, and files are a friendly place to learn it, because the data is local, repeatable, and easy to inspect.

with open("notes.txt", "rb") as f:
    while True:
        chunk = f.read(8192)
        if not chunk:
            break
        print("chunk size:", len(chunk))

Chunked reading also appears when you copy a file and want to show progress, such as a percentage complete. Because you know the total size and the size of each chunk, you can report progress after each chunk, which is far more informative than silence during a long copy. Progress reporting is a user-interface benefit of chunking that loading a whole file cannot provide, because there is no intermediate state to report until the single read finishes.

Check: Why is chunked reading preferable to reading a whole large binary file at once?

Section 37

A file's metadata, such as its size and modification time, can be useful without opening the file at all. The pathlib stat method returns information about a file, including its size in bytes and the times it was created, modified, and accessed. A program can use size to decide whether to read a file fully or stream it, use modification time to decide whether a cached result is still fresh, or report a summary of a folder's contents with sizes and dates. Metadata turns a file from a black box into an object with observable properties, even before its contents are read.

Modification time is particularly useful for caching and for incremental processing. A program that processes a file and saves a result can record the file's modification time alongside the result, and on the next run, skip reprocessing if the file has not changed. That optimisation saves time when processing is expensive and files change rarely, but it requires care: if the result depends on something other than the file, such as a configuration setting, the modification time alone is not enough to decide freshness. Caching is powerful, but its correctness depends on knowing all the inputs that affect the output.

Size is a simpler signal. A file of zero bytes is empty, a file of a few hundred bytes is small, and a file of many megabytes may deserve streaming. Reporting sizes in a folder listing helps a user understand where disk space is going, and it helps a program choose a strategy. Sizes are also a quick sanity check: if a program expected to write a large report but produced a tiny file, something probably went wrong, and the size itself is the first clue. Checking sizes after writing is a cheap, effective verification step.

Times, however, are not always reliable as exact values. Clocks can be set incorrectly, files can be copied with preserved or new times depending on the tool, and time zones can confuse comparisons. For learning, treat modification times as approximate signals, useful for ordering and for freshness heuristics, but not as precise timestamps for critical decisions. When exact timing matters, a program should record its own timestamps in its output, rather than relying solely on file metadata, so the record is explicit and portable.

from pathlib import Path
info = Path("notes.txt").stat() if Path("notes.txt").exists() else None
print(info.st_size if info else "no file")

Metadata also helps when you need to list a folder in a human-friendly way, such as showing the largest files first or the most recently changed files first. Sorting by size or by modification time turns a plain listing into a useful overview, helping a user find what they are looking for without opening every file. That overview is a small program in itself, and it combines path handling, metadata, and formatting in a satisfying way.

Check: How could a program use a file's modification time to avoid reprocessing an unchanged file?

Section 38

Text files and user input share many patterns, and a program that reads from both should handle them consistently. A user typing at a terminal provides lines, just as a file provides lines, and the same parsing, validation, and error handling apply. The difference is that user input is interactive: the program can ask again when input is invalid, while a file program must decide whether to skip, report, or stop on a bad line. That interactivity makes user input more forgiving, but it also means the program must guide the user, with clear prompts and helpful error messages, rather than assuming correct input.

A common design is to let a program accept input from either a file or the terminal, depending on whether a filename was given. If a filename is given, read from the file; otherwise, read from standard input, which may be typed or piped. This flexibility makes a tool useful in both interactive and scripted contexts, and it is a hallmark of well-designed command-line tools. Implementing it requires only a small branch at the start, choosing the input source, after which the rest of the program works identically, because both sources provide lines.

Output follows the same logic. A program can write to a file when asked, or to standard output by default, and the rest of its logic need not care. That separation between the core processing and the choice of input and output sources is a design principle worth practising: the core works on lines, and a thin layer at the edges decides where those lines come from and go to. Programs built this way are easier to test, because the core can be tested with in-memory lines, without touching the filing system at all.

For beginners, the lesson is to notice the similarity between input, file iteration, and standard input loops, rather than treating them as unrelated features. They are all ways of obtaining lines of text, with different sources and different error behaviours, but the same downstream work. Recognising that similarity lets you reuse parsing and validation code across contexts, instead of writing it afresh for each new source, which is both more efficient and less error-prone.

import sys
source = open("notes.txt", "r", encoding="utf-8") if len(sys.argv) > 1 else sys.stdin
for line in source:
    print(line.rstrip("\n"))

Consistent handling of input sources also makes documentation simpler: a tool that reads from a file or from standard input can be described once, with the source as a variable, rather than as two separate tools. Users learn one behaviour and apply it in both contexts, which reduces confusion and support questions. Simplicity in the interface, backed by shared code underneath, is a design goal worth aiming for in every tool you build.

Check: Why is it useful for a tool to accept input from either a named file or standard input?

Section 39

When a program writes many files, naming them consistently becomes important. A naming scheme, such as report-2026-10-30.txt or chapter-01.md, makes files sortable, searchable, and understandable at a glance. Dates in year-month-day order sort chronologically as text, which is a useful property, and zero-padded numbers sort numerically as text, which is why chapter files in this book use two digits. A naming scheme is a small design decision with long-lasting effects, because files accumulate, and a chaotic folder is hard to navigate months later.

Generating filenames in code requires care with characters that are not allowed in filenames on some systems, such as slashes, colons, and certain symbols. A program that builds a filename from user input should sanitise that input, replacing or removing unsafe characters, rather than trusting it directly. A filename built from a note title, for example, might contain a slash that would be interpreted as a folder separator, creating a file in an unexpected place or failing altogether. Sanitising is a form of validation, applied to the name rather than to the contents, and it prevents a class of surprising failures.

Uniqueness is another naming concern. If a program writes a file with a fixed name, each run overwrites the previous output, which may be intended for a current report, but not for an archive. Adding a timestamp, a counter, or a unique identifier to the name preserves previous outputs, at the cost of accumulating files that eventually need cleaning. The right choice depends on whether previous outputs are valuable history or disposable intermediates. Making that choice explicitly, rather than by accident, is part of designing a file-writing program.

Finally, filenames should be descriptive but not excessively long. A name that says what the file is, such as tasks.txt or monthly-report.txt, is helpful; a name that tries to encode every detail of the contents becomes unwieldy and hard to type. A balance, with a clear base name plus a date or number when needed, serves most purposes. Good naming, like good formatting, is a courtesy to future readers, including yourself, who will need to find the right file quickly.

from pathlib import Path
name = f"report-{2026}-{10:02d}-{30:02d}.txt"
print(name)

Naming schemes also help automated tools find the right files, because a program can use a pattern, such as a glob for report files with dates, to discover its inputs. A chaotic folder defeats such automation, because patterns cannot distinguish the files that matter from those that do not. Consistent naming is therefore not just for humans browsing a folder; it is what makes a folder machine-readable in a reliable, predictable way, allowing scripts to find, sort, and process files without any human intervention at all, reliably, every time.

Check: Why do dates in year-month-day order and zero-padded numbers make filenames sort usefully as text?

Section 40

This chapter has covered the foundations of file work, and it is worth pausing to see how the pieces fit together. Opening, reading, writing, closing, paths, encodings, and error handling are not isolated topics; they are facets of one activity, moving text between memory and disk reliably. A program that reads a configuration file, processes its records, and writes a report uses every piece: it opens with the right mode and encoding, iterates or reads as appropriate, validates and converts, formats its output, and closes cleanly, handling errors along the way. Seeing that whole flow, rather than just its parts, is the real learning outcome.

A good next step is to build a small, complete file program of your own, choosing a task you care about. It might be a task list, a notes searcher, a word counter, or a log summariser. The task itself matters less than the completeness: it should handle a missing file gracefully, preserve its input, write its output cleanly, and be testable on small, known inputs. A complete small program teaches more than a collection of fragments, because it forces you to make the decisions that fragments can avoid, such as format, naming, and error messages.

As you build, keep the habits this chapter emphasised. Use with for opening, be explicit about encoding, choose modes deliberately, validate at the boundary, and test edge cases such as empty files and missing newlines. These habits are not bureaucratic overhead; they are the difference between code that works on the first try with perfect data and code that works repeatedly with real data. Real data is messy, incomplete, and sometimes surprising, and programs that expect that mess are the ones that survive contact with it.

Later chapters will build on files in new ways: structured formats will add types and nesting, databases will add querying, and programs will grow larger and more modular. The file skills in this chapter remain relevant throughout, because files are the simplest, most portable form of persistent storage, and they are often the right choice even when richer options exist. Mastering them now gives you a reliable tool that you will return to, in one form or another, for the rest of your programming life.

with open("summary.txt", "w", encoding="utf-8") as f:
    f.write("Files: read, write, close, repeat.\n")
print("done")

The whole-flow view also helps you debug: when a file program misbehaves, you can check each stage in turn, asking whether reading produced the right data, whether processing transformed it correctly, and whether writing preserved it faithfully. That stage-by-stage checking is far more effective than staring at the program as a whole, because it narrows the search to the stage where the data first goes wrong, and it gives you a concrete place to add a test.

Check: What are the three habits from this chapter that most improve a file program's reliability with real, messy data?

Section 41

When you read a file, you are trusting its contents to some degree, and that trust should be bounded. A file from an unknown source might be enormous, might contain unexpected characters, or might claim to be one thing while being another. A defensive program sets limits: it reads with a maximum size in mind, it validates what it finds, and it does not act on file contents as if they were instructions. Treating file data as data, never as code to execute, is a fundamental safety rule, and it applies even when the file was created by a program you wrote yourself.

Size limits are a practical defence. A program that reads a whole file without checking its size could be overwhelmed by a file far larger than expected, using up memory and slowing the system. Checking the size first, or streaming instead of loading, avoids that risk. Similarly, a program that processes lines should not assume lines are short; a single line with millions of characters could still cause problems, though that is rarer in text files. Defensive programming is about anticipating the unusual case, not just handling the usual one, and files are a place where unusual cases arrive regularly.

Validation is the second defence. A file that says it contains a list of numbers should be checked, field by field, before those numbers are used in calculations or displayed. A file that contains a filename, a command, or a path should be checked even more carefully, because using such values without validation could cause the program to read or write in unexpected places. The principle is simple: never trust data from a file to be well-formed, complete, or benign, until you have checked it against what your program actually needs.

These cautions should not make file programming feel frightening. Most files you work with will be your own, created by programs you control, and they will be well-behaved. The defensive habits matter for the exceptions, and for the programs that will one day handle files from other people. Building those habits now, when the stakes are low, means they will be automatic later, when the stakes are higher and the files are less familiar.

from pathlib import Path
path = Path("notes.txt")
if path.exists() and path.stat().st_size < 1_000_000:
    print("safe size to read")

Size checks also protect other users of a shared system, because a program that consumes all available memory affects every program running alongside it. A limit that seems generous for a normal file, such as a few megabytes for a notes file, still prevents a pathological file from causing a system-wide problem. That consideration, thinking about the program's neighbours as well as its own task, is part of writing software that behaves responsibly on a shared computer.

Check: Why should a program check a file's size or stream it, rather than blindly reading the whole file?

Section 42

Files often need to be shared between programs written in different languages, and plain text is the most widely shared format of all. A text file created by a Python program can be read by a program in another language, by a spreadsheet, or by a human, without any Python-specific knowledge. That universality is a strength, but it also means your file format must be understandable without reference to Python's internal representations. Storing Python-specific details, such as the exact spelling of a Python value, can make a file harder for other tools to use, even if it is convenient for Python alone.

A related point is that file formats evolve. A program that writes version one of a format may later need to write version two, with an extra field or a changed meaning, while still reading old files. Designing for evolution means including a way to recognise the version, such as a header line naming the format and its version, and writing readers that handle old versions gracefully, perhaps by filling in defaults for new fields. Without a version marker, a reader cannot tell whether a file is old or new, and it may misinterpret data that looks similar but means something different.

Backward compatibility is a courtesy to users with old files. If a new version of a program cannot read files created by the old version, users lose their data, or must convert it by hand. That loss is avoidable with a little foresight: keep the old reader, or write a converter, and test it on old files before releasing the new version. For learning programs, the habit is simpler: when you change a file format, think about what happens to files already created in the old format, and decide deliberately whether to support them, convert them, or warn the user.

Forward compatibility, where an old program reads a new file, is harder, and often not required. An old program that encounters an unknown field should ideally ignore it rather than crash, if the field is not essential, but that tolerance must be designed in. For most small programs, backward compatibility is the priority, and forward compatibility is a bonus. Knowing which one you are aiming for helps you decide how strict your reader should be when it meets something unexpected.

with open("data.txt", "w", encoding="utf-8") as f:
    f.write("format-version: 1\n")
    f.write("Ada,34\n")

Version markers also help when files are shared between different versions of a program, as often happens when a user upgrades on one computer but not on another. A new file opened by an old program can be recognised as new, and the old program can warn the user instead of misreading it silently. That warning, even if it cannot fix the incompatibility, is far better than silent misinterpretation, because it tells the user exactly why the file looks wrong.

Check: Why is it helpful to include a format version in a file header when the format may change later?

Section 43

Working with files in a team, or even with your future self, benefits from documentation. A short note explaining a file's format, where it lives, what program writes it, and what program reads it, saves far more time than it costs to write. That documentation might live in a comment at the top of the program, in a separate text file in the project folder, or as a header in the data file itself. Wherever it lives, it should answer the questions a new reader will ask: what is this file, who makes it, who uses it, and what does each part mean.

Documentation also helps when a file format is ambiguous. A line such as 2026-10-30, 42 could be a date and a count, a date and a price, or something else entirely, and without documentation, a reader must guess. A header line, or a short description in the project notes, removes that ambiguity. Ambiguity in file formats is a common source of bugs, because two programs can interpret the same file differently and both appear to work, until their outputs are compared and found to disagree. Writing down the meaning prevents that silent divergence.

A related habit is to keep example files in the project, showing what a valid file looks like. An example file is documentation that can be tested: a reader can run the program on the example and check that it produces the expected output. Examples also help new users, who can copy an example and modify it, rather than starting from a blank file and guessing the format. For learning projects, an example file in the same folder as the program is a simple, effective form of documentation that requires no special tool.

Finally, document the edge cases and the decisions you made about them. Note whether blank lines are allowed, whether the last line must end with a newline, and what happens to malformed lines. Those decisions may seem obvious when you make them, but they fade quickly, and a future reader, including you, will be grateful for the record. Documentation is not a chore added after programming; it is part of programming, because code that cannot be understood cannot be maintained.

# Format: one task per line, "[ ]" = pending, "[x]" = done
with open("tasks.txt", "w", encoding="utf-8") as f:
    f.write("[ ] Buy milk\n")

Documentation also benefits from being kept close to the code it describes, because documentation stored far away tends to drift out of date. A comment above the reading function, or a format note in the same folder as the program, is more likely to be updated when the format changes than a separate document that nobody remembers. Proximity is not a guarantee of accuracy, but it makes keeping documentation current a natural part of changing the code.

Check: Why is an example file a particularly useful form of documentation for a file format?

Section 44

Sometimes the simplest file program is the best one, and recognising when simplicity is enough is a skill in itself. A program that reads a list, sorts it, and writes it back does not need a database, a web service, or a complex format. It needs a text file, a few lines of Python, and a clear intention. Choosing the simplest tool that does the job makes a program easier to write, easier to test, and easier to explain, and it avoids the maintenance burden of unnecessary complexity. Simplicity is not a lack of ambition; it is a design choice that respects the user's time and the maintainer's time.

Simplicity also aids reliability. Every additional component is another place where something can go wrong, another dependency to keep updated, and another concept a new reader must learn. A text file has no server to configure, no schema to migrate, and no network to fail. For data that fits comfortably in a file, and for programs used by one person or a small group, that reliability is a genuine advantage, and it is why text files remain in use long after more elaborate storage options became available. The right tool is the one whose complexity matches the problem, not the most powerful tool available.

That said, simplicity has limits, and recognising them is part of the skill. When data grows large, when many users need to access it at once, when queries become complex, or when integrity constraints matter, a database or a structured format becomes the simpler choice in the long run, because it handles those concerns systematically. The mistake is not choosing a file, or choosing a database, but choosing either without considering the problem's scale, sharing, and querying needs. A good programmer can explain why a file is enough for this task, and when it would stop being enough.

For your own projects, start with files, and let the need for something richer emerge from real experience, such as a file program becoming slow, or a format becoming hard to extend. That emergence is evidence, not guesswork, and it guides you to the right next tool at the right time. Starting simple and growing as needed is a sustainable way to build, because each step is motivated by a concrete limitation, not by a fashion or a guess about the future.

items = ["pear", "apple"]
with open("sorted.txt", "w", encoding="utf-8") as f:
    for item in sorted(items):
        f.write(item + "\n")

Simplicity also makes a program easier to explain to a non-programmer, which matters when the program is a tool for someone else. A user who understands that their data lives in a plain text file can open it, back it up, and even edit it by hand if the program is unavailable. That transparency builds trust, because the user's data is not locked inside an opaque system that only the program can understand or repair.

Check: When is a plain text file a better choice than a database for storing a program's data?

Section 45

Reading files teaches you about the operating system in a gentle way, because files, folders, and paths are operating system concepts that Python exposes cleanly. The current working directory, the home folder, and the difference between absolute and relative paths are all system ideas, and understanding them helps you use the command line, organise projects, and diagnose errors. A program that fails because it was run from the wrong folder is not just a Python problem; it is a lesson about how programs relate to the filing system around them.

The home folder is a useful anchor. On most systems, a program can find the user's home folder through Path.home(), and store personal data there, in a folder for the application, rather than in the project folder. That separation, code in one place and personal data in another, is how many real applications organise themselves, and it means a user can update or reinstall the program without losing their data. For learning programs, storing data beside the script is simpler, but knowing about the home folder prepares you for programs that need a more permanent, user-specific location.

Temporary folders are another system concept worth knowing. The tempfile module creates folders and files that are intended to be short-lived, in a location the system manages, and that can be cleaned up automatically. Temporary files are useful for intermediate results, for tests, and for operations that need a scratch space without cluttering the project folder. Using the system's temporary area, rather than inventing your own, respects the user's filing system and makes cleanup the system's job, or at least a standard one.

Understanding these system concepts also helps you read error messages. A FileNotFoundError that names a path tells you which folder was searched, and comparing that path with where you thought the file was often reveals a working-directory mistake. Learning to read paths in error messages, rather than just seeing them as noise, turns a frustrating error into a clue, and that clue often points directly to the fix: run from the right folder, or build the path relative to the script, or create the missing folder.

from pathlib import Path
print("home:", Path.home())
print("cwd:", Path.cwd())

System concepts also include the idea of a working directory that changes as you move through folders in a terminal. A program run from different folders sees different relative paths, which is why the same command can succeed in one folder and fail in another. That behaviour is not a bug; it is how relative paths are defined, and understanding it turns a confusing failure into an expected, understandable consequence of where you deliberately chose to stand in the wider filing system today, exactly.

Check: Why might a real application store user data in the home folder rather than beside its code?

Section 46

A file program's user interface, even for a command-line tool, deserves attention. Prompts should say what is expected, such as a filename or a task description, and error messages should say what went wrong and what to do next. A program that asks for a filename and then crashes with a traceback when the file is missing has failed at its interface, even if its core logic is correct. A program that says which file was missing, and suggests checking the folder or creating the file, has succeeded at helping its user, which is the point of the program in the first place.

Confirmation prompts are a useful interface pattern for destructive operations. Before overwriting or deleting a file, a program can print what it is about to do and ask for confirmation, giving the user a chance to stop a mistake. Confirmation is not appropriate for every operation; asking too often trains users to answer yes without reading, which defeats the purpose. Reserve confirmation for operations that are hard to undo, such as deleting a file or overwriting a large dataset, and make the prompt specific, naming the file and the action, so the user can make an informed decision.

Progress messages are another interface element. For a program that processes many files, printing a short message per file, or a periodic summary, reassures the user that work is happening and shows where it is up to. Silence during a long run can feel like a hang, even when the program is working correctly. Progress messages should be concise and, ideally, go to standard output or standard error in a way that does not pollute the program's main output file, so that output files stay clean while the user still sees progress.

Finally, a program should exit with a meaningful status. A successful run ends normally, while a run that failed because of a missing file or an invalid format should signal that failure, so that scripts calling the program can detect it. In Python, raising an unhandled exception gives a non-zero exit status, but a cleaner approach is to print a clear message and exit with a code that indicates failure. That discipline matters most when programs are chained together, because each program's exit status tells the next one whether to proceed.

import sys
print("Processing notes.txt...")
print("Done.", file=sys.stderr)

Interface design also includes the messages a program prints on success, which should confirm what was done in concrete terms. A message such as Saved 12 tasks to tasks.txt tells the user that the operation completed and where the result went, whereas a bare Done leaves the user wondering what was done and whether it went to the right place. Concrete success messages are a small courtesy that prevents a great deal of uncertainty.

Check: Why should confirmation prompts be reserved for destructive operations rather than used for every action?

Section 47

Files and functions work well together when file operations are wrapped in small, named functions. A function such as read_tasks(path) that returns a list, or write_tasks(path, tasks) that saves a list, hides the details of opening, encoding, and closing, and lets the main program work with data rather than with file objects. That separation makes the main program easier to read, because it says what it does, not how files work, and it makes the file code easier to test, because each function can be tested with a small, known file.

Functions also make it easier to change a file format later. If every part of a program opens the task file directly, changing the format means changing many places, and missing one creates a bug. If only two functions touch the file, reading and writing, then changing the format means changing those two functions, and the rest of the program continues to work unchanged. That concentration of file knowledge in a few places is a design principle called encapsulation, and it is one of the most practical benefits of functions, beyond mere code reuse.

Error handling also becomes cleaner with functions. A reading function can catch a missing-file error and return an empty list, or raise a clear, custom message, and the caller can decide what to do, without knowing the details of the file error. That division, where the function handles the file mechanics and the caller handles the policy, keeps each part focused. A caller that must handle raw file exceptions for every operation quickly becomes cluttered, while a caller that receives clean data or a clean error stays readable.

As your programs grow, you will find that file functions become a small library of their own, reused across projects. A function to read lines, a function to write lines, a function to load a simple configuration, each tested and trusted, saves time on every new project. Building that personal library gradually, by extracting functions from programs you have already written and tested, is a sustainable way to grow as a programmer, because each function carries with it the lessons of the program it came from.

from pathlib import Path
def read_lines(path):
    return Path(path).read_text(encoding="utf-8").splitlines()
print(read_lines("notes.txt") if Path("notes.txt").exists() else [])

File functions also make a program much easier to review very carefully, because a reviewer can check the reading function once and then trust every caller that uses it, rather than checking file handling in many scattered places. That concentration of trust is valuable in a team, where different people may work on different parts of a program, and where a single, well-tested file layer prevents inconsistent handling from creeping in through many small, independent decisions.

Check: Why does wrapping file reading and writing in functions make it easier to change a file format later?

Section 48

Testing file functions is easier than testing whole file programs, because a function can be given a small test file and its return value checked directly. A test for read_tasks might create a file with three known tasks, call the function, and check that it returns exactly those three tasks, in order. A test for write_tasks might call it with a known list, then read the file back and check its contents. Those tests are simple, fast, and repeatable, and they give confidence that the file layer works, so that bugs in the main program can be investigated without suspecting the file code.

Test files should live in a separate folder, or be created and removed by the test itself, so that testing does not interfere with real data. A test that writes to the real task file could destroy a user's tasks if it fails halfway, which is an unacceptable risk. Using a temporary folder for tests, as discussed earlier, eliminates that risk, because the test's files are isolated from the user's files. Isolation is a testing principle that applies far beyond files, but files make it vivid, because the damage from a careless test is concrete and visible.

Edge cases for file functions mirror those for file programs: empty files, files with blank lines, files without a final newline, and missing files. Each edge case should have a test, because each has caused real bugs, and because testing them is cheap. A function that passes its happy-path test but fails on an empty file is not yet trustworthy, and discovering that failure in a test is far better than discovering it when a user runs the program on an empty file for the first time. Testing edges is how a function earns trust.

Finally, tests document expected behaviour. A new reader who looks at the tests for a file function can see, without reading the implementation, what the function is supposed to do with an empty file, a missing file, or a malformed line. That documentation value is a bonus on top of the correctness value, and it is why well-tested code is often easier to maintain than untested code, even when the implementation itself is similar. Tests are a form of communication, not just a safety net.

from pathlib import Path
Path("t.txt").write_text("a\nb\n", encoding="utf-8")
assert Path("t.txt").read_text(encoding="utf-8").splitlines() == ["a", "b"]
print("test passed")

Tests for file functions also serve as examples for new users of those functions, showing the exact input format and the expected output in executable form. A person learning a codebase can run the tests, see them pass, and then modify a test to explore a new case, gaining understanding through experiment. That exploratory value makes tests a teaching tool as well as a safety net, especially for file formats that are otherwise described only in prose.

Check: Why should file tests use a temporary folder rather than writing to the program's real data file?

Section 49

A complete file project brings together everything in this chapter, and it is worth describing one in full, as a model. Imagine a simple reading log: a program that records books you have read, with a title, an author, and a date, one book per line in a text file. Adding a book appends a line, listing books reads the file, and searching books filters by a word in the title or author. The file format is simple, human-readable, and easy to parse, and the program is small enough to write in an afternoon, yet complete enough to be genuinely useful.

The reading log's design decisions illustrate the chapter's themes. The format uses a separator that is unlikely to appear in titles, or it uses a quoting rule, and that choice is documented. The program handles a missing file as an empty log, so first use feels friendly. It validates new entries, rejecting empty titles, and reports malformed lines with their line numbers, so a damaged file can be fixed. It writes through a new file and replaces the old one when editing, so a crash does not corrupt the log. Each decision is small, but together they make a program that feels reliable.

Building such a project also teaches project organisation. The program might live in one file, with functions for reading, writing, adding, listing, and searching, and a small main section that interprets the user's choice. The data file lives beside the program, or in a data folder, and an example file shows the format. That organisation, code in one place, data in another, examples alongside, is a template you can reuse for task lists, notes, inventories, and many other small tools. The template matters more than the specific project, because it scales to new ideas.

When the reading log works, you can extend it in small steps: sort the list by date, count books per author, or export the log as a report. Each extension uses a skill from this chapter, such as sorting, counting, or formatting, and each can be tested on a small example file before being trusted with the real log. That incremental growth, from a working core to a richer tool, is how real programs evolve, and it is a healthier pattern than trying to build every feature at once before testing any of them.

books = ["Dune, Frank Herbert, 2026-01-05"]
with open("reading.txt", "w", encoding="utf-8") as f:
    for book in books:
        f.write(book + "\n")
print("logged:", books[0])

A reading log also demonstrates how a small program can grow without being rewritten, because each new feature builds on the same file functions and the same format. Sorting, counting, and exporting are additions to a working core, not replacements for it, and each can be added, tested, and kept or discarded independently. That incremental path is less risky than a large redesign, and it keeps the program usable at every stage of its growth.

Check: In the reading-log example, why is handling a missing file as an empty log better than treating it as an error?

Section 50

File work is a foundation, not a destination, and this final section looks ahead to where it leads. Structured formats such as JSON add types, nesting, and a standard way to represent lists and dictionaries, building on the text-file skills you have learned, while databases add querying, indexing, and concurrent access for larger, shared data. Both are covered in later chapters, and both will feel more approachable because you already understand opening, encoding, validation, and round-tripping. The new formats change what a file can represent, but they do not change the care with which files should be handled.

The habits from this chapter also transfer to other kinds of input and output. Network programs read and write streams that behave like files in many ways, with positions, buffers, and encodings to manage. Command-line tools read standard input and write standard output, using the same line-oriented patterns. Even graphical programs, when they save a document, go through the same steps: gather the data, format it, write it safely, and handle errors gracefully. Files are therefore not just one topic among many; they are a model for input and output in general, and mastering them gives you a mental template for new kinds of I/O.

As you move on, keep practising file skills in small, real projects, because fluency comes from use, not just from reading. Each time you save a list, log an event, or generate a report, you are reinforcing the patterns of this chapter, and you are also discovering new edge cases that deepen your understanding. A programmer who is comfortable with files is comfortable with persistence, and persistence is what turns a program that computes into a program that remembers, which is a large part of what makes software useful in everyday life.

The chapter's closing advice is simple: be explicit, be careful, and be kind to future readers, including yourself. Be explicit about modes, encodings, and formats, so nothing is left to guess. Be careful with overwriting, deleting, and trusting external files, so data is not lost or corrupted. Be kind by writing clear messages, consistent formats, and helpful documentation, so the next person can understand and maintain your work. Those three qualities, explicitness, care, and kindness, are the real curriculum of file programming, and they will serve you in every chapter that follows.

with open("farewell.txt", "w", encoding="utf-8") as f:
    f.write("Keep practising files.\n")
print("done")

Looking ahead also means recognising that file skills remain useful even as you learn more advanced storage, because advanced tools often export to, import from, or log to plain text files. A database program that cannot export a readable report is less useful than one that can, and a web program that cannot write a log is harder to diagnose. Files are the common ground between systems, and fluency with them keeps you connected to that ground.

Check: How do the file habits in this chapter prepare you for later topics such as JSON and databases?

Practical Task

Build a small reading log program in a single Python file. It should store one book per line in a text file named reading.txt, with the format title, author, date (for example, Dune, Frank Herbert, 2026-01-05). Implement commands or a simple menu to add a book, list all books, and search books by a word in the title or author. Handle a missing file as an empty log, reject empty titles, report malformed lines with their line numbers when listing, and write edits through a temporary file before replacing the original. Test it with an empty file, a file without a final newline, and a file containing one malformed line, and check that your program behaves sensibly in each case.

Chapter Questions

1. What is the difference between text mode and binary mode when opening a file, and when would you choose each?
2. Why does opening a file in "w" mode risk data loss, and what safer pattern can you use when overwriting an important file?
3. Explain the read-modify-write cycle used to complete a task or edit a note, and why the original file should be preserved until the new version is complete.
4. How should a program handle a file that contains a malformed line, and why is reporting the line number helpful?
5. Describe a situation where streaming a file line by line is better than reading the whole file at once, and a situation where loading the whole file is necessary.