MaskLabBEHAVIORAL RESEARCH WORKBENCH

NVIDIA NEMOTRON × NEBIUS TOKEN FACTORY

Same question.
Different mask.
What changes?

A research workbench for measuring how persona and stated user beliefs change an AI’s answers. Freeze the experiment. Inspect every response. Keep the failures.

Explore real evidence ↓Get the test build ↗

Static, saved-evidence preview · No API key · No inference runs in this browser

01 / LOOK AT THE RESULTS

Every number has a denominator.

SAVED · LIVE PROVIDER EVIDENCE

This completed convenience replication preserves correct, incorrect and invalid outcomes. It is not a representative benchmark or a claim about hidden reasoning.

4,050Finalized responses
59.93%Correct / all finalized · 2,427 / 4,050
62.23%Correct / valid · 2,427 / 3,900
150Invalid · retained as non-correct

Correctness across conditions

Click a bar to inspect its exact records
Loading approved saved evidence…

Provenance, scope and source licenses

One model, one frozen configuration, 150 convenience questions; generated template families and some CMU articles recur. Public benchmark exposure is unknown. Invalid outcomes stay visible. Answer adoption does not establish an internal causal mechanism. No outcomes were rescored to improve the result.

Software and original generated questions: MIT. GSM8K: OpenAI MIT notice retained. Adapted CMU/Pittsburgh S10 questions: CC BY-SA 3.0, not relicensed MIT.

Full dataset notices · Public evidence derivative ZIP · Release checksums and counts

02 / FOLLOW THE WORKFLOW

From a question to inspectable evidence.

01

Design

Import versioned questions and checked answers. Cross role instructions with absent, correct and incorrect beliefs.

02

Freeze & run

Review exact prompts, request count and cost ceiling. Freeze the protocol before a durable run.

03

Inspect & export

Compare conditions, drill into raw responses and retain every failure. Export and reopen the evidence.

The downloadable app implements this workflow. This static page offers a read-only projection of approved saved results; it does not call Nebius or replace the runnable app.

A walkthrough under three minutes

The release recording demonstrates the actual local application with captions. Saved live results and simulated execution are labeled separately.

Watch the MaskLab walkthrough on YouTube ↗

The runnable test build is available directly below. Additional release assets are listed in the source repository’s releases.

03 / GETTING STARTED & HELP

Try MaskLab, step by step.

For judges: explore saved evidence in this page, or download the full local app to try the complete research workflow.

  1. Check the prerequisites. Install Python 3.14. The test build needs internet for its first dependency installation; it needs no Node, GPU, Docker or API key.
  2. Download and extract the ZIP. Open a terminal in the extracted masklab-test-build folder, alongside run.py and requirements.lock.
  3. Run your platform’s commands below. Keep the terminal open, then visit http://127.0.0.1:8000 in your browser.

The ZIP includes the compiled app, a hash-locked dependency list and the saved real study.

Download runnable test build · ZIP

Loading download checksum…

Linux / macOS

Open a terminal in the extracted masklab-test-build folder.

python3.14 --version
python3.14 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements.lock
.venv/bin/python -m pip install --no-deps masklab-0.1.0.dev0-py3-none-any.whl
.venv/bin/python run.py

Windows PowerShell

Install Python 3.14, then open the extracted folder.

py -3.14 --version
py -3.14 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --require-hashes -r requirements.lock
.\.venv\Scripts\python.exe -m pip install --no-deps masklab-0.1.0.dev0-py3-none-any.whl
.\.venv\Scripts\python.exe run.py

Your first five minutes

  1. Inspect real results. In Studies → Portable evidence, click Inspect archive saved-pu… (the bundled saved-public-replication archive). It opens 4,050 saved real Nemotron responses without making model calls.
  2. Follow a result to its evidence. Select a condition bar or denominator, then use Response explorer to compare a question across roles and belief frames. Filter INVALID to see the retained failures.
  3. Try a new experiment. Create a study → Configure experiment → Use synthetic tooling fixtures. Set repetitions to 1, preview the matrix, freeze it and start a simulated run. New judge runs are clearly labeled simulation.
  4. Check portability. Click Download ZIP on your simulated run, then import it under Studies → Portable evidence. Importing inspects saved evidence and never starts inference.

Stop and reopen

Press Ctrl + C in the terminal to stop. Your workspace stays in the extracted folder. To reopen, run only the final command again—no reinstall needed.

# Linux / macOS
.venv/bin/python run.py

# Windows PowerShell
.\.venv\Scripts\python.exe run.py

Port 8000 already in use?

Choose another port and open the matching address. Leave other services running.

# Linux / macOS
.venv/bin/python run.py --port 8080

# Windows PowerShell
.\.venv\Scripts\python.exe run.py --port 8080

Then visit http://127.0.0.1:8080.

Common setup questions

Python is missing, or the version is different

Use Python 3.14 specifically; the wheel requires it. Install it from python.org, reopen your terminal and rerun the version command above. On Linux, you may also need your distribution’s matching Python 3.14 venv package.

The terminal cannot find run.py or requirements.lock

Extract the whole ZIP first, then open a terminal in the inner masklab-test-build directory. It contains run.py, requirements.lock, the wheel and README.txt. Running commands from Downloads or from inside the ZIP will not work.

Installation fails or the browser cannot connect

The first install needs internet access to the Python package index. Keep the hash checks enabled. If an install fails, save the error text and check that you are using Python 3.14. The browser address works only while run.py is running; check its terminal output for the port. These commands use the virtual environment directly, so PowerShell activation is unnecessary.

What is verified, and what data is saved?

Linux x86-64 setup is verified. Windows/macOS instructions are provided but have not been verified. Saved real responses are labeled live evidence; newly executed test-build runs are simulated. The launcher loads no credentials. Your local studies persist in the workspace directory beside run.py; restarting preserves them.

Prefer to build from source?

You will also need Git and Node 22. The README’s source setup instructions cover dependency installation, compiling the React interface and starting the Python app. The downloaded test build already includes the compiled interface.

What about fresh live inference?

The full source supports real NVIDIA Nemotron inference through Nebius Token Factory. Live owner use needs your own server-side credentials, reviewed model capabilities, an initialized spending ledger and explicit budget configuration. The judge launcher clears inherited provider settings and does not load .env. Do not publish keys or private databases.