THINK FIRST·CODE LATER

← All labs

Canary analysis: promote or roll back?

Problem

A new StudyBuddy release is rolled out in stages. At each stage the canary (new version) and the baseline (old version) serve traffic at the same time. Decide automatically whether to continue.

Input: lines stagePercent baselineRequests baselineErrors canaryRequests canaryErrors canaryP95ms baselineP95ms until the end of input.

For each stage compute error rates (errors / requests, as percentages). The canary fails the stage if:

  • its error rate is greater than 2 × the baseline's error rate and greater than 0.5% (absolute), or
  • its p95 latency is more than 1.2 × the baseline p95.

Output per stage:

Stage 5%: baseline 0.40% errors, canary 0.50% errors, p95 310 vs 300 ms -> PROMOTE

or -> ROLLBACK (errors), -> ROLLBACK (latency) or -> ROLLBACK (errors, latency). Rates with two decimals. After the first ROLLBACK, stop processing and print Result: rolled back at X%. If every stage passes, print Result: fully released.

If a group has 0 requests, its error rate is 0.

Input:

1 10000 40 100 0 290 300
10 9000 36 1000 5 305 300
50 5000 20 5000 60 320 300
100 0 0 10000 50 300 300

Output:

Stage 1%: baseline 0.40% errors, canary 0.00% errors, p95 290 vs 300 ms -> PROMOTE
Stage 10%: baseline 0.40% errors, canary 0.50% errors, p95 305 vs 300 ms -> PROMOTE
Stage 50%: baseline 0.40% errors, canary 1.20% errors, p95 320 vs 300 ms -> ROLLBACK (errors)
Result: rolled back at 50%

Data for this lab

Your program reads its data from standard input. Download the file, then give it to your program as shown below. Start with data set 1: its expected output is given.

s13-2-data-1.txt

1 10000 40 100 0 290 300
10 9000 36 1000 5 305 300
50 5000 20 5000 60 320 300
100 0 0 10000 50 300 300
What your program must print for s13-2-data-1.txt
Stage 1%: baseline 0.40% errors, canary 0.00% errors, p95 290 vs 300 ms -> PROMOTE
Stage 10%: baseline 0.40% errors, canary 0.50% errors, p95 305 vs 300 ms -> PROMOTE
Stage 50%: baseline 0.40% errors, canary 1.20% errors, p95 320 vs 300 ms -> ROLLBACK (errors)
Result: rolled back at 50%

Run your program first, then open this and compare line by line: your output must contain these lines exactly.

How to give the file to my program

Terminal (macOS, Linux, or Windows Command Prompt), after compiling:

java Main < s13-2-data-1.txt

Windows PowerShell does not understand <:

Get-Content s13-2-data-1.txt | java Main

IntelliJ IDEA: Run → Edit Configurations → Redirect input from. Or simply paste the content of the file into the console when the program waits for input.

Write it here or in your IDE, then paste it. Compile and test it yourself before comparing. Your code stays in your browser — it is never sent to or stored on the server.