THINK FIRST·CODE LATER

← Interactive Information Visualization
Chapter 4 · Week 4

Getting Data: Files, Databases, APIs and Web Scraping

Answered 0/30 Correct 0
Sign in to save progress across devices
Q1

What is the difference between WHERE and HAVING in SQL?

Q2

Which query finds stations that have no departing trip?

Q3

Trips: (S1→S2, 8 min), (S1→S2, 10), (S2→S1, 9), (S3→S1, 25), (S1→S3, 22), (S3→S3, 5), (S2→S1, 11). S1 and S2 are in Lucheng, S3 in Ouhai. Grouping by the district of the start station, what are the trip count and average duration for Lucheng?

Q4

Why should large aggregations (e.g., trips per district over 10 million rows) usually be computed in the database?

Q5

Which code is safe against SQL injection when station comes from a user's input?

Q6

Which pandas function returns the result of an SQL query as a DataFrame?

Q7

An API answers with HTTP status 429. What does it mean and what should your client do?

Q8

What happens with requests.get(url) if the server accepts the connection but never answers?

Q9

An API allows 30 requests per minute. What is the minimum time to fetch 500 pages one after another?

Q10

Why should a paginating client deduplicate records by id?

Q11

Where should an API key for the CityPulse data portal be stored?

Q12

What is GBFS?

Q13

A scraper selects the data table with soup.find("table", class_="x7f2"). Why is this fragile?

Q14

What does td.get_text(strip=True) return for <td> 1 </td>?

Q15

A bike-station table on a web page contains a row <td colspan="4">Maintenance notice…</td> among normal 4-cell rows. What should a robust scraper do?

Q16

A page shows a table in the browser, but requests.get returns HTML without the table. What is the most likely reason and the best first step?

Q17

What is robots.txt?

Q18

robots.txt contains User-agent: * with Disallow: /admin/ and Crawl-delay: 5. May the crawler "CityPulseStudentBot" fetch /stations, and how fast?

Q19

robots.txt contains a group User-agent: BadBot with Disallow: /. What may BadBot fetch?

Q20

A public website allows crawling in robots.txt, but its terms of service forbid automated collection, and pages contain users' names and photos. What is the right decision?

Q21

Which is the best source for a large set of film data for a non-commercial student project?

Q22

Which practice makes data acquisition reproducible?

Q23

An AI assistant wrote a scraper loop for page in range(1, 1000): requests.get(url + str(page)) with no delay, no timeout and no User-Agent. Which problems does it have?

Q24

Which HTTP status code indicates that your credentials were accepted but you are not allowed to access the resource?

Q25

In SQLite and in Python's sqlite3, how do you write a placeholder for a parameter?

Q26

A server answers HTTP 503 several times in a row. What is the recommended retry strategy?

Q27

pd.read_html(url) returns a list of DataFrames from a page's <table> elements. What must you still do?

Q28 Short answer

Write SQL (with the tables stations(station_id, name, district) and trips(trip_id, start_station, end_station, duration_min)) for: (a) trips and average duration per district of the start station, most trips first; (b) routes (start→end, excluding round trips) with at least 2 trips; (c) stations with no departing trip. Explain the role of each JOIN type you use.

Q29 Short answer

Design a robust client for a paginated CityPulse readings API (30 requests/minute, pages linked by next). List the safeguards you would implement and estimate how long downloading 120 pages takes.

Q30 Short answer

A classmate wants to scrape all public reviews (with user names and profile photos) of restaurants near campus to visualize "where students eat". Discuss the legal and ethical issues and propose an approach you would accept as a supervisor.