THINK FIRST·CODE LATER

← All labs

Follow API pagination safely

Problem

Simulate a paginated API. Standard input: the allowed requests per minute, then one JSON page per line (in any order) like {"page": 1, "next": 2, "results": [{"id": 1, "pm25": 42}, …]}; the last page has "next": null.

Starting at page 1, follow next links:

  • if a page was already visited, print loop detected at page N, stopping; if it does not exist, print page N missing (HTTP 404), stopping;
  • collect records, deduplicating by id (keep first-seen order).

Then print requests: 3, records received: 6, unique records: 5, ids: 1, 2, 3, 4, 5 and polite schedule: one request every 2 s -> at least 4 s in total (60/rate seconds between requests, (requests − 1) gaps; numbers in :g format).

Data for this lab

Your program reads its data from standard input. Download the file, then give it to your program as shown below. Start with data set 1: its expected output is given.

v4-3-data-1.txt

30
{"page": 2, "next": 3, "results": [{"id": 3, "pm25": 40}, {"id": 4, "pm25": 38}]}
{"page": 1, "next": 2, "results": [{"id": 1, "pm25": 42}, {"id": 2, "pm25": 35}]}
{"page": 3, "next": null, "results": [{"id": 4, "pm25": 39}, {"id": 5, "pm25": 30}]}
What your program must print for v4-3-data-1.txt
requests: 3, records received: 6, unique records: 5
ids: 1, 2, 3, 4, 5
polite schedule: one request every 2 s -> at least 4 s in total

Run your program first, then open this and compare line by line: your output must contain these lines exactly.

How to give the file to my program

Terminal (macOS, Linux, or Windows Command Prompt), in the folder that holds both files:

python my_lab.py < v4-3-data-1.txt

Windows PowerShell (the default terminal of VS Code on Windows) does not understand <:

Get-Content v4-3-data-1.txt | python my_lab.py

PyCharm: Run → Edit Configurations → tick Redirect input from and choose the file.

Jupyter: put this at the top of the cell that holds your program:

import sys
sys.stdin = open("v4-3-data-1.txt", encoding="utf-8")

Write it here, in Jupyter or in your IDE, then paste it. Run it on the sample input yourself before comparing. Your code stays in your browser — it is never sent to or stored on the server.