FAQ

A run shared from darkmux shows as submitted. When a second machine gets a matching result on the same eval, model and weights, it shows as reproduced. Runs far outside a cell’s spread, and anything the checks cannot decide, go to a maintainer instead of the tables.

Only runs with the same eval version, darkmux minor version, model weights hash and quantization. Hardware, operating system, engine and context length are what you compare across. A run that does not report one of those keys is shown on its own, never mixed into a row.

The tables show whether the task passed, the time, tokens per second and peak memory, and how many runs each number comes from. The pass range is a 95% Wilson interval, so a row with few runs shows a wide range. Rows are listed by model and machine, never ranked.

Active time is a run’s wall clock minus the pauses darkmux takes on purpose: rests that let the machine cool, from its thermal governor, and pauses that save battery. On a hot machine those pauses can be a large share of the wall clock, and a model that takes more turns takes more of them, so leaving them out keeps the time about the model. The chip slowing itself down under heat is not a pause, so it stays in. Tokens per second counts generation time only, so pauses never lower it. With a free account, each run also shows its wall clock and how long darkmux paused.

darkmux reads the machine when you share, so a fact that changed after the run, such as an OS update, a new engine runtime or re-downloaded weights, is left out as not reported rather than guessed.

The eval it ran, the darkmux version, a hardware class, the engine and model details, the measurements and the trajectory. It carries no machine names, user names or file paths: sharing refuses a run that contains them, rather than quietly removing them.

A run that never reached a model, one where the harness failed rather than the model, one with no verdict, and one that passed before darkmux checked that the work was done. A run from a chip other than Apple Silicon is refused too.

darkmux officially supports Apple Silicon only, under macOS or Omarchy Linux. On macOS it also records power, thermal state and GPU load from Apple's own counters. On Omarchy it records the package's power and energy, but not yet GPU load or thermal state. A fork may run darkmux on other hardware, but this site shows only what darkmux officially supports.