Building PRISM: For Evaluating language models

The question

Take a multiple-choice question. You can ask it directly, phrase it as an instruction, or wrap it in a conversational frame. The meaning stays the same, and a model should land on the same answer each time. Often it doesn’t. A model might get it right under one phrasing and wrong under another, follow the answer format once and ignore it the next time, or give the same wrong answer across every variation.

That last case is the one that stuck with me. When a model agrees with itself, that doesn’t mean it is correct.

This became PRISM, short for Prompt Reliability Through Intelligent Semantic Multiplexing. I wanted to see how small language models that run locally respond to prompts that mean the same thing, and whether accuracy alone captures that behaviour. So I looked at correctness, instruction compliance, answer recovery, agreement, and prompt sensitivity together. It started as my own project and later became our university minor project.

How the pipeline works

PRISM keeps the question fixed, changes how it is presented, and compares the answers under one evaluation protocol.

We use two science datasets: ARC-Challenge, with hard multiple-choice science questions, and SciQ, with questions written for educational assessment. The benchmark takes 200 questions from each and asks every question in five formulations, labelled P0 to P4. The meaning stays the same while the wording or instruction framing changes.

Models run through Ollama, so the experiments work on ordinary hardware without paid APIs. The set is Llama 3.2 3B, Gemma 3 4B, Mistral 7B, and Phi-4 Mini (3.8B). Later result records also include Qwen 2.5 in 0.5B and 3B sizes. These models don’t represent every small language model. The goal is to compare how they behave under the same setup.

Fair comparison needs consistent answer extraction and scoring. A response that ignores the requested format shouldn’t automatically count as a wrong answer, so instruction compliance is measured separately from correctness.

• Accuracy: how often the answer is correct.
• Conditional accuracy: accuracy over the response subset the protocol defines.
• Answer recovery: whether the model gets the right answer under another formulation after an initial miss.
• Instruction compliance: whether the response follows the requested output format.
• Question majority accuracy: whether the most common answer across formulations is correct.
• Agreement: how consistently the model answers the same question.
• Prompt sensitivity: how much responses vary across formulations.
• Answer unanimity: how often all formulations give the same answer.
• Prompt-invariant incorrect rate: how often the model gives a stable but wrong answer.

What the results show

These figures come from the public PRISM results repository, one row per model and dataset.

Source: PRISM benchmark results. Values come from the published summary. The repository also holds additional runs that still need validation.

Agreement and correctness are different measures

Qwen 2.5 3B reached 95.5% agreement on ARC-Challenge but only 78.0% accuracy, with a 14.5% prompt-invariant incorrect rate. It often agreed with itself, and some of those agreed answers were wrong. Consistency tells you something about a model, but it doesn’t tell you the answers are right.

Reliability depends on the dataset

Across these runs, SciQ gave higher accuracy and agreement and lower prompt sensitivity than ARC-Challenge. Phi-4 Mini scored 93.1% on SciQ and 80.8% on ARC-Challenge, and its prompt-invariant incorrect rate was 3.5% against 9.0%. Results depend on the task as well as the model, so one dataset isn’t enough.

A single score can hide failures

Phi-4 Mini had high accuracy but surprisingly low instruction compliance, 10.1% on ARC-Challenge and 10.9% on SciQ. Before drawing any conclusion, I need to check output parsing, model behaviour, and the evaluation protocol.

A note on the data

The public repository also contains zero-valued records. A zero doesn’t necessarily mean the model failed every question. It may come from a failed evaluation or an output-processing problem. So I don’t treat every published row as an equally valid result. The summary is good for exploring patterns, but a rigorous analysis needs verified run identifiers, evaluation completeness, prompt-level records, and a checked scoring pipeline. A benchmark is only as good as the evidence behind it, and checking that evidence is part of the research.

From solo experiment to minor project

I began with one question: how much does an answer change when the same question is presented differently? Then I realised a model-output spreadsheet wouldn’t be a real contribution. We needed a reproducible pipeline, consistent metrics, readable results, documentation, and a way for other people to run the experiments.

We submitted the minor-project proposal in May and defended it in August, working as a team. My friends and I built the pipeline, a website, and a desktop application for running and simulating the benchmark. Once other people had to use it, working on my machine stopped being enough. Someone else had to be able to understand the methodology, reproduce the evaluation, inspect the results, and run the software without rebuilding my environment.

Mentorship and teamwork

Our professor, Pukar Karki, guided us as the project grew. He helped us turn a research idea into a structured university project, and it was useful to have someone to talk through direction, implementation choices, documentation, and milestones with.

My teammates changed how I approached it too. Building a research pipeline is one job. Building something a group can maintain, explain, demonstrate, and share is another. I’m grateful to our professor for his guidance and to my teammates for taking the project well beyond its first form.

The pipeline, the website, and the desktop app

We wanted PRISM to be something other people could run, so it grew along three paths.

The research pipeline runs the evaluations, processes model responses, calculates metrics, and produces structured results. The documentation website explains the motivation, methodology, and metrics, and how to read the findings. The desktop application lets users run the pipeline and look at benchmark outputs without touching every part of the research code.

Building these taught me that explaining research is part of the technical work. A result is far more useful when someone else can see how it was produced and check the evidence.

Open source

The research pipeline is on GitHub, and the benchmark results live in a separate repository. The desktop app and documentation give people other ways in if they want to explore the work. The results repository has a review process: submitted runs enter a queue, a maintainer checks them, and they are then synced into the published results, which makes the public record easier to trace than a pile of unverified outputs.

Open code and results make it easier to spot problems, compare evaluations, and improve the method. They don’t make every result correct or every conclusion general, but they make scrutiny and reproduction practical, and I want my research to allow both.

What I learned

Which question you measure matters. A leaderboard built on accuracy answers a different question from a benchmark built on prompt sensitivity and stability.

Metrics need to be read together. High agreement can sit alongside wrong answers, and instruction-following problems can complicate an otherwise strong result.

Infrastructure is part of research quality. Reproducible runs, traceable results, validation, and careful handling of malformed outputs all limit what you can responsibly conclude.

And projects improve when they’re built for other people to use. The pipeline, website, and desktop app each cover a different part of making the research understandable, testable, and accessible.

What comes next

PRISM is still a learning process for me, and we haven’t solved language-model reliability. I want to validate the questionable runs, analyse prompt-level behaviour, add more models and tasks, and work out when a stable answer is a correct one rather than a stable error. I also want to keep improving the tools that make this work easier to reproduce and inspect.

The original question hasn’t changed: if we ask a model the same thing in different ways, how much should we trust the answer when the wording changes, and what should we conclude when it doesn’t?

Reliability can’t be reduced to one number. We need to see whether a model is right, how its behaviour shifts, what it follows, and where its consistency can mislead us.