Using Python generators for streaming data

Python Toolbox

Hugo Bowne-Anderson

Data Scientist at DataCamp

Generators for the large data limit

  • Use a generator to load a file line by line
  • Works on streaming data!
  • Read and process the file until all lines are exhausted
Python Toolbox

Build a generator function

  • sequence.py
def num_sequence(n):
    """Generate values from 0 to n."""
    i = 0
    while i < n:
        yield i
        i += 1
Python Toolbox

Let's practice!

Python Toolbox

Preparing Video For Download...