THINK FIRST·CODE LATER

← All labs

ROC, precision–recall and choosing a threshold

Problem

Input: target_recall R, then a CSV actual,score.

Print 10 cases, 5 positives; ROC AUC 0.720, average precision 0.794 (scikit-learn roc_auc_score, average_precision_score). Then for each distinct score t, from highest to lowest (predict positive if score ≥ t): threshold 0.45: recall 0.80, precision 0.67, false-positive rate 0.40. Finally highest threshold reaching recall 0.8: 0.45 (precision 0.67) (the first threshold in that order with recall ≥ R).

Data for this lab

Your program reads its data from standard input. Download the file, then give it to your program as shown below. Start with data set 1: its expected output is given.

v13-2-data-1.txt

target_recall 0.8
actual,score
1,0.95
1,0.85
0,0.80
1,0.70
0,0.55
1,0.45
0,0.30
0,0.20
1,0.15
0,0.05
What your program must print for v13-2-data-1.txt
10 cases, 5 positives; ROC AUC 0.720, average precision 0.794
threshold 0.95: recall 0.20, precision 1.00, false-positive rate 0.00
threshold 0.85: recall 0.40, precision 1.00, false-positive rate 0.00
threshold 0.80: recall 0.40, precision 0.67, false-positive rate 0.20
threshold 0.70: recall 0.60, precision 0.75, false-positive rate 0.20
threshold 0.55: recall 0.60, precision 0.60, false-positive rate 0.40
threshold 0.45: recall 0.80, precision 0.67, false-positive rate 0.40
threshold 0.30: recall 0.80, precision 0.57, false-positive rate 0.60
threshold 0.20: recall 0.80, precision 0.50, false-positive rate 0.80
threshold 0.15: recall 1.00, precision 0.56, false-positive rate 0.80
threshold 0.05: recall 1.00, precision 0.50, false-positive rate 1.00
highest threshold reaching recall 0.8: 0.45 (precision 0.67)

Run your program first, then open this and compare line by line: your output must contain these lines exactly.

How to give the file to my program

Terminal (macOS, Linux, or Windows Command Prompt), in the folder that holds both files:

python my_lab.py < v13-2-data-1.txt

Windows PowerShell (the default terminal of VS Code on Windows) does not understand <:

Get-Content v13-2-data-1.txt | python my_lab.py

PyCharm: Run → Edit Configurations → tick Redirect input from and choose the file.

Jupyter: put this at the top of the cell that holds your program:

import sys
sys.stdin = open("v13-2-data-1.txt", encoding="utf-8")

Write it here, in Jupyter or in your IDE, then paste it. Run it on the sample input yourself before comparing. Your code stays in your browser — it is never sent to or stored on the server.