Python - Iterators and Generators

Iterators and Generators Python Code

title: Contents 
style: nestedList # TOC style (nestedList|inlineFirstLevel)
minLevel: 1 # Include headings from the specified level
maxLevel: 4 # Include headings up to the specified level
includeLinks: true # Make headings clickable
debugInConsole: false # Print debug info in Obsidian console

Overview

Sources:

• **

Code Snippets

Bad Example (Memory Intensive)

def process_large_file(filename):
    all_lines = open(filename).readlines() # Yikes! Loading everything at once  
	results = []  
	for line in all_lines:  
		# Do some processing on each line  
		results.append(process_line(line))  
	return results

Good Example (Generator)

def process_huge_file_generator(filename):
    with open(filename) as f:
        for line in f:
            yield process_line(line)  # Process on demand, no giant lists
 
# Using the generator
for result in process_huge_file_generator("massive_log.txt"):
    # Do something with each result

Details

The generator approach processes your file line-by-line, yielding results as needed. Your memory usage stays sane, even with massive datasets. This is a core principle behind how Google handles enormous amounts of data — efficiency at scale.

To enhance the code even more, you can chain generators together to create elegant data processing pipelines.

See Also

• Python Map of Content
• Python Code
• Tool - Python
• Tool - Python Flask

---

Appendix

Note created on 2024-04-15 and last modified on 2024-04-15.

Backlinks

LIST FROM [[Python - Iterators and Generators]] AND -"CHANGELOG" AND -"04-RESOURCES/Code/Python/Python - Iterators and Generators"

---

(c) No Clocks, LLC | 2024