Python Directory and Files Management

Why This Matters

In this comprehensive Python lesson, we delve into the essential topic of managing directories and files using Python. By mastering these skills, you will be able to:

1. Store data persistently between sessions
2. Organize your code into modular files for better maintainability
3. Share your projects with others easily
4. Automate repetitive tasks involving reading, writing, or manipulating files
5. Debug and understand the behavior of your programs by inspecting their input and output files
6. Gain a competitive edge in exams, interviews, and real-world programming scenarios

Prerequisites

Before diving into Python's directory and file management, you should have a solid understanding of:

1. Basic Python syntax and data structures (variables, loops, functions)
2. How to run a Python script from the command line or an integrated development environment (IDE)
3. Understanding of relative and absolute paths
4. Familiarity with common file formats like .txt, .csv, .json, etc.
5. Basic understanding of exceptions and error handling in Python

Core Concept

Python provides several built-in modules for handling files and directories: os, os.path, and shutil. In this section, we will explore each module in detail and provide examples to help you grasp the concepts better.

Using the os module

The os module offers a wide range of functionalities related to the operating system, including file and directory management. Here are some examples:

import os

Check if current working directory exists

if not os.path.exists('.'):

print("Current working directory does not exist!")

Create a new directory

os.mkdir('my_directory')

Remove a directory (and its contents)

os.rmdir('my_directory')

### Using the os.path module

The `os.path` module provides functions to manipulate paths and perform various file system operations. Here are some examples:

import os

import os.path

Get the absolute path of the current working directory

abs_path = os.path.abspath('.')

print(abs_path)

Join two or more paths

joined_path = os.path.join('my', 'directory', 'file.txt')

print(joined_path)

### Using the shutil module

The `shutil` module offers higher-level functions for handling files and directories, such as copying, moving, and deleting files. Here are some examples:

import shutil

Copy a file

shutil.copy('source_file', 'destination_directory')

Move or rename a file

shutil.move('source_file', 'new_name')

Delete a file

shutil.rmtree('my_directory') # Deletes the directory and its contents recursively

Worked Example

Let's create a simple Python script that:

1. Creates a new directory called my_files
2. Writes some text to a file named file.txt inside the created directory
3. Reads the content of the file and prints it
4. Deletes the directory and its contents
5. Copies another file named backup.txt to the my_files directory
6. Moves the file.txt from the my_files directory to a new location called processed_files

import os
import shutil

Create a new directory

os.mkdir('my_files')

Join the path to the new directory and the file name

file_path = os.path.join('my_files', 'file.txt')

backup_path = 'backup.txt'

processed_path = 'processed_files'

Open the file in write mode ('w') and write some text

with open(file_path, 'w') as f:

f.write("Hello, world!")

Copy a backup file to the my_files directory

shutil.copy(backup_path, 'my_files')

Open the file in read mode ('r') and print its content

with open(os.path.join('my_files', file_path), 'r') as f:

print(f.read())

Move the file from my_files to processed_files

shutil.move(os.path.join('my_files', file_path), os.path.join('processed_files'))

Remove the directory and its contents

shutil.rmtree('my_files')

Common Mistakes

1. Forgetting to close a file after opening it (use with open(...) as f:)
2. Using relative paths incorrectly, resulting in files not being found
3. Trying to delete a directory that is not empty (use shutil.rmtree() or manually remove its contents before deleting)
4. Not handling exceptions when working with files and directories
5. Forgetting to import the necessary modules (os, os.path, shutil)
6. Misusing file modes when opening a file for reading or writing (e.g., using 'r' instead of 'rb' for binary files)
7. Failing to check if a file or directory exists before attempting to manipulate it
8. Not properly escaping special characters in paths, especially on Windows systems

Practice Questions

1. Write a Python script that creates two subdirectories named images and videos, respectively, in the current working directory.
2. Given a list of files, write a function that moves all the files from their current location to a new directory called archive.
3. Write a Python script that reads the contents of multiple text files located in a specific directory (specified by the user) and prints the total number of words across all files.
4. Implement a function that checks if a given file exists, and if it does not exist, creates it with some default content.
5. Write a script to recursively find all .txt files under a specified directory and its subdirectories, and concatenate their contents into a single text file called all_contents.txt.
6. Implement a function that copies a directory (and its contents) from one location to another while preserving the original directory structure.
7. Write a script to find all empty directories under a specified directory and its subdirectories, and delete them recursively.
8. Create a function that renames multiple files in a directory by appending a timestamp to their names.
9. Implement a script that moves all .jpg files from the current working directory to a new directory called images, while maintaining the original file names.
10. Write a Python script that reads a CSV file, processes its data, and writes the results to a new CSV file in a different location.

FAQ

What is the difference between os, os.path, and shutil?

• os offers low-level functionalities for interacting with the operating system, including file and directory management.
• os.path provides functions to manipulate paths and perform various file system operations.
• shutil offers higher-level functions for handling files and directories, such as copying, moving, and deleting files.

How can I read a file line by line in Python?

You can use a for loop with the built-in readlines() method to read a file line by line:

with open('file.txt', 'r') as f:
for line in f:
print(line)

How can I write to a file and append lines instead of overwriting it?

To append lines to a file, use the 'a' mode when opening the file:

with open('file.txt', 'a') as f:
f.write("New line!\n")

How can I read binary files in Python?

To read a binary file, use the 'rb' mode when opening the file:

with open('file.bin', 'rb') as f:
data = f.read()

How can I handle exceptions when working with files in Python?

You can use a try-except block to catch and handle exceptions that might occur while working with files:

try:
with open('file.txt', 'r') as f:
data = f.read()
except FileNotFoundError:
print("File not found!")

How can I check if a file or directory exists in Python?

You can use the os.path.exists() function to check if a file or directory exists:

import os
if os.path.exists('file.txt'):
print("File exists!")
elif os.path.isdir('directory'):
print("Directory exists!")
else:
print("Neither file nor directory exists!")