Create the first version of Buddy's evaluation for CPS 2231 (Java).
- Write 12 golden items in a table: id, category, the question, what a correct answer must contain (key facts), the source that must be cited (e.g., "Lecture 5, slide 12" — invent plausible ones), and the expected behaviour (
answer,answer with hints only,refuse,refuse + redirect to board). Include at least: 5 normal questions, 2 out-of-scope questions, 2 graded-exercise questions, 1 question in Chinese, 1 prompt-injection attempt, 1 question containing personal data. - Define four metrics with formulas and release thresholds.
- Explain how you will judge correctness (who/what, calibration).
- Write two metamorphic tests for Buddy.
- Describe how the set will grow over the semester.