THINK FIRST·CODE LATER

← All labs

An experiment: does prompting with the schema help?

Problem

Design (and, if you have access to an AI assistant, run on a small scale) an experiment comparing three ways of asking an AI tool for CityPulse charts: (P1) the question only; (P2) the question plus the data schema (columns, types, units, example values); (P3) P2 plus the task in visualization vocabulary ("compare distributions…").

  1. State the hypotheses.
  2. Define the question set (at least 20 questions of varying difficulty), the gold standard and the procedure (model, temperature, repetitions).
  3. Define the metrics (validity, schema errors, exact/component/execution accuracy, design-lint issues, human ratings).
  4. Describe how you will analyse the results and the threats to validity.
  5. If you ran it: report your results in a table and discuss them.

Work it out on paper, in a document or here, then compare with the model answer. Your answer stays in your browser — it is never sent to or stored on the server.