Reading Files in Python: A Guide to File I/O and Context Managers

Previously in this course, we built a Project: Statistics Processor where we managed data in memory. Today, we advance that project by learning to persist and retrieve that data from disk using file I/O.

Until now, our programs have been "ephemeral"—once you close the script, the data disappears. To build real-world applications, your programs must be able to read and write information to external storage.

The Open Function and File I/O

In Python, interacting with the file system starts with the built-in open() function. This function returns a "file object," which acts as a bridge between your script and the data stored on your hard drive.

The basic syntax is: file_object = open("filename.txt", "r")

The second argument, "r", stands for read mode. It tells Python you only intend to look at the data, not change it. While this works, it is dangerous if you forget to close the file afterward. If your program crashes before file_object.close() is called, the file could remain "locked" by the operating system, leading to memory leaks or data corruption.

Using the Context Manager (The with Statement)

To solve the resource management problem, we use a context manager. By using the with keyword, Python automatically handles the opening and closing of the file for us, even if an error occurs while the file is being read.

Consider this example:

PYTHON
# The CE9178">'with' statement creates a context manager
with open("data.txt", "r") as file:
    content = file.read()
    print(content)

# Once the indented block ends, the file is automatically closed.

Using the with statement is the industry standard for file I/O because it guarantees that resources are released immediately after the task is finished.

Reading File Content

Once the file is open, you have several ways to extract data depending on your needs:

1. read(): Reads the entire file into a single string. Useful for small configuration files.
2. readline(): Reads a single line. Useful for processing logs one entry at a time.
3. readlines(): Reads all lines into a list, where each element is one line of the file.

Worked Example: Reading a Data Log

Imagine your CLI tool needs to read a stats.txt file that contains one number per line. Here is how you would process that:

PYTHON
def read_log_file(filepath):
    total = 0
    with open(filepath, "r") as file:
        # Loop through the file object directly to read line-by-line
        for line in file:
            # We strip whitespace like newline characters(\n)
            number = int(line.strip())
            total += number
    return total

# Usage
# Assuming CE9178">'stats.txt' contains:
# 10
# 20
# 30
result = read_log_file("stats.txt")
print(f"The total from the file is: {result}")

Hands-on Exercise

Create a text file named notes.txt in your project folder and add three lines of text to it. Write a Python script that uses with open(...) to read the file and print each line prefixed with a line number (e.g., "1: [first line content]").

Common Pitfalls

• Forgetting the mode: If you don't specify "r", open() defaults to read mode, but explicitly stating it makes your code more readable.
• File Not Found: If the file path is incorrect, Python raises a FileNotFoundError. We will cover how to catch these errors in a future lesson on Exception Handling, but for now, ensure your file exists in the same directory as your script.
• Encoding issues: If your file contains special characters (like emojis or non-English letters), always specify the encoding: open("file.txt", "r", encoding="utf-8").

Frequently Asked Questions

• Why does with close the file? It triggers a special method that instructs the operating system to release the file handle, preventing system-level errors.
• Should I use read() for large files? No. read() loads the entire file into RAM. For massive files, iterate over the file object (as shown in the worked example) to process them line-by-line.

Recap

In this lesson, we moved beyond memory-only programming. You learned that file I/O is the foundation of data persistence, that the open() function provides the connection, and that the context manager (with statement) is the safest way to manage those connections.

Up next: Writing to Files, where we will learn how to save your program's data permanently.