benchpack registry snapshot

Generated 2026-10-03T13:57:50+00:00 from staging-pi-20261003.sqlite. Static, read-only export over indexed compact rows.

Machine-readable snapshot: snapshot.json.

Browse Filters

Quick views: Django Resume one-shot results passing runs failed runs NVIDIA Django wrap pack

Runs

idlabelimportedrowspacksadaptersmodelendpointsruntimemodel metadataquantizationmodel artifactmodel revisionmodel sha256hostcomparisonrepo commitoperatingrun.jsonl sha256
12026-04-28-mlx-lm-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen2.5 0.5B Instructhttp://localhost:8080/v1/chat/completions——————Apple M5 Max (64 GB)———1a75463cc086d35787b2957d0a3b3d52499ce99536dd58bedee1b115ce374cf2
22026-04-28-mlx-lm-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen2.5 0.5B Instructhttp://localhost:8080/v1/chat/completions——————Apple M5 Max (64 GB)———868121e67c2b7a5f2a169661414c0de8bbbf4a68f0133e8c143b28a3ec311e7f
32026-04-29-llama-server-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M5 Max (64 GB)———8a23c8f3e0e2ebb1490822f4571cc18f6524870e4a2dabe57a4e5880f9a45d69
42026-04-29-llama-server-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M5 Max (64 GB)———ed795551960a1e4c88f1c6c66c163640dc923abfd5fe7a085f00d707b8ca5cd8
52026-04-30-llama-server-desktop-django-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M5 Max (64 GB)———2ca61c497e8ba76cb9c94369d49485cbc069db11268eeeae95951e0e571b8523
62026-05-03-m4-max-llama-qwen25-05b-190636-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M4 Max (128 GB)———bfff2514d5534c33dc494f5fed80b4599e4c78169c6178dfd230ead9023a59e5
72026-05-03-m4-max-llama-qwen25-05b-190636-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M4 Max (128 GB)———bee931cf81a26acac8440288568153dbc7d1a69ca8dcc5aa3849af31ea3793cf
82026-05-03-m4-max-llama-qwen25-05b-190636-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M4 Max (128 GB)———542511f03d4cc6a2373fc868d19a7563d281e8d006ab63dcc217ad3cf8e4a262
92026-05-03-m4-max-llama-qwen25-05b-190636-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8081/v1/chat/completions——————Apple M4 Max (128 GB)———98dee1608317ea389a6c0e315bb330f56a934debbd08d150d9f0a2b77f65f8c6
102026-05-03-m4-max-mlx-qwen25-05b-185824-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8080/v1/chat/completions——————Apple M4 Max (128 GB)———1e6e1caa89e527865c7b40913edf91063549b36794d143dd537d285ed778c213
112026-05-03-m4-max-mlx-qwen25-05b-185824-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8080/v1/chat/completions——————Apple M4 Max (128 GB)———d7c1489b282bd86892cbb20b721949ed691eb59e887e944275215ae5d827a7a6
122026-05-03-m4-max-mlx-qwen25-05b-185824-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8080/v1/chat/completions——————Apple M4 Max (128 GB)———5ebcf409eb4c686cc31939567cac3b43591693e119929920fca3f36ff3407e2b
132026-05-03-m4-max-mlx-qwen25-05b-185824-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen2.5 0.5B Instructhttp://127.0.0.1:8080/v1/chat/completions——————Apple M4 Max (128 GB)———5cf5c75cc27f057f549c7e5b8b8acebecf7465d5855b02651b4750ba06491d15
142026-05-03-m4-max-qwen3coder-184938-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——————Apple M4 Max (128 GB)———b6dec7420aab1c0accf65e1950225811d223c246aaaf831ecb61ec1acb74225e
152026-05-03-m4-max-qwen3coder-184938-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——————Apple M4 Max (128 GB)———27865294ce16ff9ea1975f48e2c51daee97faafe86eeac3293e81df0fbdcbdde
162026-05-03-m4-max-qwen3coder-184938-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——————Apple M4 Max (128 GB)———3e5d2b25b70c8d63877781195bfc8941ed12dcd0d255ac465414955f822a061d
172026-05-03-m4-max-qwen3coder-184938-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——————Apple M4 Max (128 GB)———bd9728b4bba8ad01eb74f1c2f2183b5d31f57ea89f564b7935619744687f0d15
182026-05-05-m4-qwen36-dense-llamacpp-154321-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8083/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-27b-q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled2bdacdc23b40e2c9db1d649c4169b247992362c0b667de7f95f191ec4dfd63f4
192026-05-05-m4-qwen36-dense-llamacpp-154321-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8083/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-27b-q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled53b99d66b44236a2dba662de7513afac4bdd6cd89c7218a3d70864d7bac90383
202026-05-05-m4-qwen36-dense-llamacpp-154321-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8083/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-27b-q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controllede4397caed9ab98638e4e01b140c9e17116fbe64705720d2925ee0d964dc7422d
212026-05-05-m4-qwen36-dense-llamacpp-154321-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 27Bhttp://127.0.0.1:8083/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-27b-q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled14cfe8d6f64f2cab901d5e4e859f050b00f97e8450f9b68f34e803277631ec2c
222026-05-05-m4-qwen36-dense-llamacpp-smoke-1542552026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8083/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-27b-q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled50bd4db36ab7d5cf84bc811ffd796979907902c86b9e21249355ad19d4edc18c
232026-05-05-m4-qwen36-dense-mlx-155943-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8085/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-27B-4bit4bit———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled54d59baf80e90c374eb8a369864c3b0264eb085e87df6267a89a54b95c1b148e
242026-05-05-m4-qwen36-dense-mlx-155943-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8085/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-27B-4bit4bit———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledaa0602642dbf3fd0829e61be616eadd3faa29f40c5147932849582609378ab6d
252026-05-05-m4-qwen36-dense-mlx-155943-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8085/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-27B-4bit4bit———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled9ffbeb873f7441f0c8ad0e7033cb1bc749918a9b2af3b788256480126358c60e
262026-05-05-m4-qwen36-dense-mlx-155943-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 27Bhttp://127.0.0.1:8085/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-27B-4bit4bit———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledb3c8b5da25d632e4a30896286548a26bf84de7fd93fc709fc68b2d8997f3ce0c
272026-05-05-m4-qwen36-dense-mlx-smoke-1559162026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:8085/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-27B-4bit4bit———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled6c2ce10ddec0d2460fa60a76baf45765a96bca5d29e9f67b8aa141e8d30bc5cd
282026-05-05-m4-qwen36-dense-ollama-155051-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0ollama-generateQwen3.6 27Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-27b-gguf:q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled90ad79e13be51f0a006fe3640081adfc77b2a40e5f5856754aceb0d0a24fd724
292026-05-05-m4-qwen36-dense-ollama-155051-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0ollama-generateQwen3.6 27Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-27b-gguf:q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled026b15e71216fc30839332bd3830d60b02a7d7b73840d8a869ecfb8e128c4895
302026-05-05-m4-qwen36-dense-ollama-155051-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0ollama-generateQwen3.6 27Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-27b-gguf:q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled91e4cf517ae21103651c30c72bd6269a46c66a391910075e523aa11dbd04b6f0
312026-05-05-m4-qwen36-dense-ollama-155051-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5ollama-generateQwen3.6 27Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-27b-gguf:q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled80e8f160825ccab575109815fb17a2c1b3099192514b862b0ef4ed739f2d70d7
322026-05-05-m4-qwen36-dense-ollama-smoke-1548082026-09-02T10:33:15+00:001smoke-chat 0.1.0ollama-generateQwen3.6 27Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-27b-gguf:q4kmQ4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled49e90ace6ec080cebe40afc29f9fe6234503a44c73f4c72d759e76bbcca85890
332026-05-05-m4-qwen36-moe-llamacpp-153657-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled7211cd7b8b961809e21be8aa2fd0f0795b60b5fdc4ca311b25a5d050ef15b919
342026-05-05-m4-qwen36-moe-llamacpp-153657-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlleddf24db7e293761efa0727c27243691cefcdf9302c72ceec5a928f7410fe4338e
352026-05-05-m4-qwen36-moe-llamacpp-153657-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled580935de10ef745e30bd192e688bc6b17ad4c8019e8ae8f844cc4160e437aa04
362026-05-05-m4-qwen36-moe-llamacpp-153657-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled9236044cfc59cd87955da876a9f566f4e9bdd2e6b88959bd61de402e3c4af626
372026-05-05-m4-qwen36-moe-llamacpp-reasonoff-smoke-1536342026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled68d40ce43d482210491dfdc3c2eb3c85c4d19f1bc248e44a543c5f5593bec5cf
382026-05-05-m4-qwen36-moe-llamacpp-smoke-1531592026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; 9020 (a4701c98f)qwen36-35b-a3b-q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled8bff4f37e6ba08d69594af9718d6f29c5b4b492a29b33fb0dacf79f0457e5d28
392026-05-05-m4-qwen36-moe-mlx-155645-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8084/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-35B-A3B-MLX-MXFP4MXFP4———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledbed80bfd65eeba7726c474d9b13531c9bed9185d00e421f2f7390dd67da257b0
402026-05-05-m4-qwen36-moe-mlx-155645-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8084/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-35B-A3B-MLX-MXFP4MXFP4———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledf0506fad86fc7d9110b41f1f0b76609fa5f9b513ff5f221fa68e126c85ae5ab7
412026-05-05-m4-qwen36-moe-mlx-155645-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8084/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-35B-A3B-MLX-MXFP4MXFP4———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledd07c1c550bd141c596e7dd189cc672c935bf928daacbf2e66e9bdba3ab093cae
422026-05-05-m4-qwen36-moe-mlx-155645-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8084/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-35B-A3B-MLX-MXFP4MXFP4———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled89e7fe52de3d91a6ab0a61320f5f0ea7ac3a71683fe2c41b7a040b15d99d81b9
432026-05-05-m4-qwen36-moe-mlx-smoke-1556192026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 35B-A3Bhttp://127.0.0.1:8084/v1/chat/completionsmlx-lm server; 0.31.3/Users/jochen/models/mlx/Qwen3.6-35B-A3B-MLX-MXFP4MXFP4———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled2aa0eba95785bc47b2fa875e2f4bdb793df4f7e96408d0bf905ad01109b25142
442026-05-05-m4-qwen36-moe-ollama-154846-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0ollama-generateQwen3.6 35B-A3Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-35b-a3b-gguf:q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledd3e86295d7d842b4c96d8e627af3bce80fdf4cf4ee2f4fb101cffdf0e42621cd
452026-05-05-m4-qwen36-moe-ollama-154846-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0ollama-generateQwen3.6 35B-A3Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-35b-a3b-gguf:q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled743de0b33cd443a29be1d771425a15f60370c7622af2e504cc2d89415115f787
462026-05-05-m4-qwen36-moe-ollama-154846-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0ollama-generateQwen3.6 35B-A3Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-35b-a3b-gguf:q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlled328b12888209d930071aa141c5c4c24c3842ca987a6bef4ad41e3b3d2fce15cf
472026-05-05-m4-qwen36-moe-ollama-154846-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5ollama-generateQwen3.6 35B-A3Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-35b-a3b-gguf:q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledfad0ff56345ef618f6af7e9d51f11491feb09f31571ec55f04c32d34de82b560
482026-05-05-m4-qwen36-moe-ollama-smoke-1547472026-09-02T10:33:15+00:001smoke-chat 0.1.0ollama-generateQwen3.6 35B-A3Bhttp://localhost:11434/api/generateollama; server 0.20.5; client 0.23.0qwen36-35b-a3b-gguf:q4kmUD-Q4_K_M———Apple M4 Max (128 GB)——not captured; not captured; not intentionally controlledfac9f37161c20ba207a1735689d363494e2e076a4d8a12d8cccede2159e2c3c8
492026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled2b036d49a5e9d6a4803bfed19bfa2177ebd1c161b7703c719870f93f3d511c8b
502026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled9a7a841b225dcc6730182be72572aebf9f0efe1e9869636954696889c999efaa
512026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlledcab70fecc1304a84cf8ef614fe58df90df7609cf4f38e68bb1a70c6e4b28ee22
522026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled727e46e9f5053eedb541ef2140b12db9622bb19c702f3b1120ef9535ef291516
532026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-server2acd1b3not captured; not captured; no intentional benchmark load; existing host background state not otherwise controlledde3de00b332e27ae95daeea0e58aa39fc9f3f9e00b2f5a1768517baac4bb7f7a
542026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-server2acd1b3not captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled4d0dff2fa49c393a80cb44dcf25b7128158d1117eccffa26e81380639dee0e46
552026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-server2acd1b3not captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled22433417300a34eac6de9e7e1dd0c71765898db2ebf465591308de1c6735cb7f
562026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-server2acd1b3not captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled9eaaf591d1b198c2db991ad03faa619418428270bfdfbd36c0f72f11eeec5fd6
572026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-1039282026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled66ded72dd97702092ce901aa22b1c622174b3cd0cf85d2d10674a0a977ae1306
582026-05-07-m4-max-gemma4-llama-reasoning-off-smoke-20260507-1038562026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:8081/v1/chat/completionsllama-server; version: 9030 (a09a00e50); built with AppleClang 21.0.0.21000099 for Darwin arm64; http://127.0.0.1:8081/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705Apple M4 Max (128 GB)strict-same-gguf-llama-servera82fb3fnot captured; not captured; no intentional benchmark load; existing host background state not otherwise controlled2b2ccf37b7568ff154e1935ffc11e1dfa310e7bfbc7f4a6eed09ead04f47c94c
592026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-dashboard-regression-external-agent2026-09-02T10:33:15+00:001django-dashboard-regression-fix-external-agent 0.1.1ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——Q4_K_M———Apple M4 Max (128 GB)exploratory-direct-edit-external-agent86d3194 docs: record endpoint correctness v020 validationnot intentionally controlled; not intentionally controlled; no intentional benchmark load besides local Ollama/Codex workflow83cd20917322de9ee3db923f2cb9f38038970b55a77995dacd646a2c668071a5
602026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-patch-external-agent2026-09-02T10:33:15+00:001patch-from-failure-external-agent 0.1.1ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——Q4_K_M———Apple M4 Max (128 GB)exploratory-direct-edit-external-agent86d3194 docs: record endpoint correctness v020 validationnot intentionally controlled; not intentionally controlled; no intentional benchmark load besides local Ollama/Codex workflowd0244bd1751bfc077a48a2a00165d05364cad35809fa76a6a888354bb6eb147b
612026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-python-regression-external-agent2026-09-02T10:33:15+00:001python-regression-fix-external-agent 0.1.1ollama-generateQwen3 Coder 30B-A3Bhttp://localhost:11434/api/generate——Q4_K_M———Apple M4 Max (128 GB)exploratory-direct-edit-external-agent86d3194 docs: record endpoint correctness v020 validationnot intentionally controlled; not intentionally controlled; no intentional benchmark load besides local Ollama/Codex workflow1310606d40d1045fccc217584d90d4ba214e5c11bb486c52c62db8381f9718e5
622026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-endpoint-correctness2026-09-02T10:33:15+00:001endpoint-python-correctness 0.2.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpack11e6273fd238f4982423c68573a2055bcf368ad41658b0da763637c722d907f6
632026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpackd608b1a3fec1ba5261f492b9cf076c75a24f07cf427579cc42a4278c429e248e
642026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpack092881ce190eb716e92c967580bda824df322e9f896f9b5b5b838cf5f0ef16d6
652026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpack236c343a41303208679df25b6be2db309312652757fb15a05633ba7e74364812
662026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpack3d80ccdf204feffd68b94f25dd97a448cd9abf26ad8905f052fd087ec3745a61
672026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9080 (9f5f0e689), AppleClang 21.0.0.21000099, Darwin arm64; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0Apple M4 Max (128 GB)strict-same-gguf-llama-server38f2017not explicitly controlled; not explicitly captured; no intentional benchmark load besides llama-server and benchpack56dab960eb9565548bae5eaaec775f8ca1c1946e39ce970a2230116ae8efd88e
682026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——d8cd7376ad41d0eb369f6d8edbbca0bc27f1df61ed5769d153820b5b3471bd3d
692026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——b9efa76bb23233a37bc2cc977fea86af373e35532834c464c799635174c1680e
702026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——d888d85a7c9df68378e267eabaafef3ffabd25bdac9dd38b0c1fa7b19f8ee781
712026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——25f76672c7e40e4c2f86629f3754e0bbd8cd1cc7469c5885f46f557d0ef3c761
722026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-endpoint-correctness2026-09-02T10:33:15+00:001endpoint-python-correctness 0.2.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——cf6953b31b421101e5649378914471a8b097e563e97e31f03f52fa00f7f58a69
732026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——6e838b3042e170c907296380debf890d568eeec5296e31a0ff4a8a04e03c13f1
742026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9110 (ef22b3e4a); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcApple M4 Max (128 GB)strict-same-gguf-llama-server——15e0c9bf6abc70f6314988f77c0e949decc39c027becf981bb397f69af1cee8d
752026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:18011/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18011/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server65baa818dd6bdbb2e3d7764bf182fad35e26bf49Pre-window nvidia-smi reported about 11.99 W while production vLLM was resident.; Pre-window GPU temperature was 38 C; post-run temperature and memory were checked before documenting the run.; Production llm.service was intentionally stopped for an exclusive GPU window; no public strict-lane exposure was configured.34e5504b457457cef62016a60ec926a4f572d99e3dde32419b50a795e590d4d1
762026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:18011/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18011/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server65baa818dd6bdbb2e3d7764bf182fad35e26bf49Pre-window nvidia-smi reported about 11.99 W while production vLLM was resident.; Pre-window GPU temperature was 38 C; post-run temperature and memory were checked before documenting the run.; Production llm.service was intentionally stopped for an exclusive GPU window; no public strict-lane exposure was configured.7ae24172941595e52bb3c108ebbb1deddf8548583606b6ba2847976fbd24a4ba
772026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatGemma 4 E2Bhttp://127.0.0.1:18011/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18011/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-serverb1e62c0live host wall power not separately metered; GPU power captured by nvidia-smi snapshots; GPU temperature captured before and after with nvidia-smi; no intentional thermal load beyond production baseline; production llm.service stopped only for exclusive GPU window; llm-mgmt and caddy remained activea3518e7aced2b71db491b094ef3181922800d71f923230cf9ba35852640c738b
782026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatGemma 4 E2Bhttp://127.0.0.1:18011/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18011/v1google/gemma-4-E2B-itQ4_K_Mbartowski/google_gemma-4-E2B-it-GGUF; google_gemma-4-E2B-it-Q4_K_M.ggufb5e99bd964eaacc27ba484bb2eb3e9f6160b9143b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-serverb1e62c0live host wall power not separately metered; GPU power captured by nvidia-smi snapshots; GPU temperature captured before and after with nvidia-smi; no intentional thermal load beyond production baseline; production llm.service stopped only for exclusive GPU window; llm-mgmt and caddy remained activea9e3ecdbe4d1f4a40fa5c2b30029130892dde809f5f6b86aef2648d39fe52727
792026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; capture in run notes when starting exclusive GPU window; production llm.service stopped only during the strict-lane exclusive GPU window91b1457944bc139a762fa513f0e0253db8fdb7c0ea09a3349154adab1296eddd
802026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; capture in run notes when starting exclusive GPU window; production llm.service stopped only during the strict-lane exclusive GPU windowafc87ed10583bd5b24f168fea5fcb73cda0a67493aea81e6213d0850eda66d0d
812026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; capture in run notes when starting exclusive GPU window; production llm.service stopped only during the strict-lane exclusive GPU windowa2fbdfb28c7ac7c46665d43c7f66437d5b6f6db25b824243f212f08920555e27
822026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; capture in run notes when starting exclusive GPU window; production llm.service stopped only during the strict-lane exclusive GPU windowef85043031c730c1aae7d130fad8e8ccd828b3295809a261205298b260c73069
832026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-endpoint-correctness2026-09-02T10:33:15+00:001endpoint-python-correctness 0.2.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; pre-window GPU temperature 38 C; production llm.service stopped only during the strict-lane exclusive GPU windowaa2d22a0f50bf3455024066bedd4e261bc0136ca79eb701e855bf934d9c34b0d
842026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18082/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18082/v1Qwen/Qwen3.6-27BQ4_K_Munsloth/Qwen3.6-27B-GGUF; Qwen3.6-27B-Q4_K_M.gguf—5ed60d0af4650a854b1755bd392f9aef4872643dc25a254bc68043fa638392a0NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server38f2017not explicitly controlled; pre-window GPU temperature 38 C; production llm.service stopped only during the strict-lane exclusive GPU windowe12238e237257e3452e1cf23fafced2d5c8c8ad22d25fc063e15d665f03f9fb9
852026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patch2026-09-02T10:33:15+00:001patch-from-failure 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——fa22c961b456dffd7595e4edc4e74a6afbe7cbc04e88b18c1068687e670cd5f4
862026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——4963169dd0da0559d255352c8ac8997984dac35b12fc381248f443c6b78873d5
872026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——306f25a90ab8acd75aed50c6fdc17aeddc217c81926ba6ea3388c0c80affba3c
882026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrap2026-09-02T10:33:15+00:002desktop-django-wrap 0.1.5openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——90b30c41a37b349b30c4905647f0b86f1bbbd6e00bf251c1593ce77bc6446ad8
892026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtime2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——5419cd0756190ab16948fb69f94dfe84b32ca6b828b2593ca76a2d767a625816
902026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smoke2026-09-02T10:33:15+00:001smoke-chat 0.1.0openai-chatGemma 4 26B-A4Bhttp://127.0.0.1:18083/v1/chat/completionsllama-server; 9030 (a09a00e50); http://127.0.0.1:18083/v1—Q4_K_Mggml-org/gemma-4-26B-A4B-it-GGUF; gemma-4-26B-A4B-it-Q4_K_M.gguf—88f4a13b0bb95f031a7fad973e10854122fb67ebc34d214d39a2f65053046abcNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)strict-same-gguf-llama-server——6c43aa96a6c6a581fe56b44fd44af351ae79accdf49cc2230f42667afce259e7
912026-06-28-product-offer-pricerunner-pi-gpt552026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatopenai-codex/gpt-5.5http://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callopenai-codex/gpt-5.5————Apple M4 Max (128 GB)———db1f39c2108cc9c1be52f375a82f549ae243280af2c65cb99cee29b836ed6c83
922026-09-01-m4-max-laguna-xs21-llamacpp-q4km2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatlaguna-xs21-q4kmhttp://127.0.0.1:18088/v1/chat/completions——————Apple M4 Max (128 GB)———e694e05662794564168ca66bd983e4374a297fe5aaf34fe1617e8a538574963d
932026-09-01-m4-max-qwen38-27b-llamacpp-q4km2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.8 27Bhttp://127.0.0.1:18086/v1/chat/completions——————Apple M4 Max (128 GB)———03c7f32c6d5c06f79ce280cae39b0eefcda773461a6ec497e15e4450e9bcb9d9
942026-09-01-m4-max-qwen38-27b-mlx-4bit2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.8 27Bhttp://127.0.0.1:18087/v1/chat/completions——————Apple M4 Max (128 GB)———ca19842285e963c55aac756fde93586cd67a7e353930c77055f9badcf8a55d88
952026-09-01-m5-max-laguna-xs21-llamacpp-q4km2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatlaguna-xs21-q4kmhttp://127.0.0.1:18088/v1/chat/completions——————Apple M5 Max (64 GB)———49c102821294f5a1fd773ac8afd56d8b7c5e46b48d259aa2fa1a07b584d1f1ed
962026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerun2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatlaguna-xs21-q4kmhttp://127.0.0.1:18088/v1/chat/completions——————Apple M5 Max (64 GB)———5fcb95af94188b9d009021ed3af5efa12e7308109fe31fc992a0d74d0a089b98
972026-09-01-m5-max-qwen38-27b-llamacpp-q4km2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.8 27Bhttp://127.0.0.1:18086/v1/chat/completions——————Apple M5 Max (64 GB)———171dfb1d1c882f4c97a620a7e027a149c3fbee631849611790ff61d6e05be07c
982026-09-01-m5-max-qwen38-27b-mlx-4bit2026-09-02T10:33:15+00:009runtime-sweep 0.1.0openai-chatQwen3.8 27Bhttp://127.0.0.1:18087/v1/chat/completions——————Apple M5 Max (64 GB)———555032109d2b46429ec43afccf7fb507fb28a03e88fa92e8db56a26ed39ea189
99product-offer-pi-gpt55-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatopenai-codex/gpt-5.5http://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; pi CLI with openai-codex/gpt-5.5; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callopenai-codex/gpt-5.5————Apple M4 Max (128 GB)———712a9a26998d5242ce4d9537b6e2b0a631c27279571151c08017a892a9fbbd67
100product-offer-pi-gpt55-large-pairwise-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatopenai-codex/gpt-5.5http://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; pi CLI with openai-codex/gpt-5.5; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callopenai-codex/gpt-5.5————Apple M4 Max (128 GB)———0765362a07e3c6d90b940a9686bd62c86195ddddac26af9b4c395a6abefde88e
101product-offer-pi-qwen36-27b-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; pi CLI with ollama/qwen36-27b-gguf:q4km; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callollama/qwen36-27b-gguf:q4km————Apple M4 Max (128 GB)———83fafe559b2011a6790c2eb6c85c0a7dd3e189b43935c9e060ba6e591567bf66
102product-offer-pi-qwen36-27b-llamacpp-high-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatplaceholder-adapterhttp://127.0.0.1:18089/v1/chat/completions9820 (3fc4e1052); http://127.0.0.1:18083/v1—————Apple M4 Max (128 GB)———ac58e9f3ab58956616c6476af1f69cc79506808c80ddfb49bfb082be3d612897
103product-offer-pi-qwen36-27b-tools-128k-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; pi CLI with ollama/qwen36-27b-tools-128k:bench; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callollama/qwen36-27b-tools-128k:bench————Apple M4 Max (128 GB)———3ebdf81b12a714b8756ac28417074e3289788f8f2127e2012a93c1a384ba5f2e
104product-offer-pi-qwen36-27b-tools-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completionsbenchpack external-agent placeholder adapter plus Pi; pi CLI with ollama/qwen36-27b-tools:latest; http://127.0.0.1:18089/v1 placeholder adapter for required pre-task adapter callollama/qwen36-27b-tools:latest————Apple M4 Max (128 GB)———1a98ab3917345188e0bee393a008de44521a47c3a480776a8a33ea1069f6b821
105product-offer-pi-qwen36-llamacpp-high-large-pairwise-202606272026-09-02T10:33:15+00:002product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completions9820 (3fc4e1052); http://127.0.0.1:18083/v1llamacpp-qwen36/qwen36-27b-q4km-highQ4_K_M———Apple M4 Max (128 GB)———769dc867a04c0ac11c9ef6e0d5a41f678de33346a38a69eed84dd7acd5ebb9b5
106product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-final-202606272026-09-02T10:33:15+00:001product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completions9820 (3fc4e1052); http://127.0.0.1:18083/v1llamacpp-qwen36/qwen36-27b-q4km-highQ4_K_M———Apple M4 Max (128 GB)———099a04a477cd5753f5739e962d9ae05cc0a49eec94105cd4e1095b350bfaea21
107product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-rerun-202606272026-09-02T10:33:15+00:001product-offer-matching 0.1.0openai-chatQwen3.6 27Bhttp://127.0.0.1:18089/v1/chat/completions9820 (3fc4e1052); http://127.0.0.1:18083/v1llamacpp-qwen36/qwen36-27b-q4km-highQ4_K_M———Apple M4 Max (128 GB)———a88b1d620830e3c06ccfc2a0bdad7f25b054b1f7c53fc823beaabc2e0bc54454
1082026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtime2026-09-02T15:59:40+00:009runtime-sweep 0.1.0openai-chatQwen3.8 27Bhttp://127.0.0.1:18084/v1/chat/completionsllama-server; 9030 (a09a00e50), GNU 13.3.0, Linux x86_64, CUDA; http://127.0.0.1:18084/v1 through an SSH local forwardQwen/Qwen3.8-27BQ4_K_Mggml-org/Qwen3.8-27B-GGUF; Qwen3.8-27B-Q4_K_M.gguf0669b98607d47046c7c2b3f801011d54a08cfccf31629f53165ab6a7dad8c9847dcfd1fdf55829dac1e6e748f4a68581b0033d34NVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)same-gguf-remote-inference76bc7bb7ecc1605f90f4fc0faa5e18267c581deanvidia-smi sampled during the run; nvidia-smi sampled during the run; production llm.service stopped for an exclusive GPU windowd012bbc0756d638d9920d1a9d6ea9f488fe57b7f774d12621433ed54cf3e33e1

Comparison Matrix

runpackcasehostruntimemodelquantizationrowsokscoring passedwall med sTTFT med sdecode TPS medtotal TPS medprompt tok medcached tok medoutput tok med
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0longApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.370.06406.38345.01811.00—128.00
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.350.06419.80347.09376.00—120.00
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0shortApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.260.06436.67299.11104.00—77.00
2026-04-28-mlx-lm-smokesmoke-chat 0.1.0capitalApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—111/10.68——11.6841.00—8.00
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0longApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.200.01411.57398.54811.00—81.00
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.300.01410.07402.01376.00—119.00
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0shortApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—33—0.280.01411.82403.76104.00—114.00
2026-04-29-llama-server-smokesmoke-chat 0.1.0capitalApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—111/10.26——30.8741.00—8.00
2026-04-30-llama-server-desktop-django-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—110/10.880.03402.68386.97648.000.00342.00
2026-04-30-llama-server-desktop-django-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M5 Max (64 GB)—Qwen2.5 0.5B Instruct—110/11.230.27402.87312.83569.000.00384.00
2026-05-03-m4-max-llama-qwen25-05b-190636-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/10.51——318.40199.0023.00161.00
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.250.01335.28325.92811.00810.0081.00
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.360.01334.90329.17376.00375.00119.00
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.350.01333.66327.85104.00103.00114.00
2026-05-03-m4-max-llama-qwen25-05b-190636-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—111/10.20——40.8141.000.008.00
2026-05-03-m4-max-llama-qwen25-05b-190636-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/11.080.06334.33315.99648.000.00342.00
2026-05-03-m4-max-llama-qwen25-05b-190636-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/11.200.06338.01320.02569.000.00384.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/10.74——344.77199.0024.00256.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.410.06362.45313.60811.00810.00128.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.370.05382.64327.46376.00375.00120.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—33—0.250.05395.31311.73104.00103.0077.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—111/10.53——15.1641.000.008.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/11.170.10358.34327.56648.0026.00384.00
2026-05-03-m4-max-mlx-qwen25-05b-185824-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)—Qwen2.5 0.5B Instruct—110/11.150.11366.57332.68569.0023.00384.00
2026-05-03-m4-max-qwen3coder-184938-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—110/11.21—103.5177.04179.00—93.00
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—33—1.17—99.4291.69790.00—107.00
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—33—1.30—101.0793.68355.00—122.00
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—33—1.23—102.5294.4083.00—116.00
2026-05-03-m4-max-qwen3coder-184938-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—111/119.07—66.460.4220.00—8.00
2026-05-03-m4-max-qwen3coder-184938-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—110/12.14—100.3773.28627.00—157.00
2026-05-03-m4-max-qwen3coder-184938-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)—Qwen3 Coder 30B-A3B—110/11.77—100.6167.19548.00—119.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M110/13.63——15.97194.000.0058.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—6.020.1621.8021.25806.00802.00128.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—5.880.1522.3521.78364.00360.00128.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—4.850.1522.3621.6687.0083.00105.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/10.67——11.9824.000.008.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/110.363.0121.9015.54657.000.00161.00
2026-05-05-m4-qwen36-dense-llamacpp-154321-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/19.932.6021.9516.21579.000.00161.00
2026-05-05-m4-qwen36-dense-llamacpp-smoke-154255smoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/11.01——7.9224.000.008.00
2026-05-05-m4-qwen36-dense-mlx-155943-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit110/13.12——18.56194.000.0058.00
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit33—4.920.1827.0026.02806.00805.00128.00
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit33—4.930.1826.8525.95364.00363.00128.00
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit33—3.020.1827.0425.4687.0086.0077.00
2026-05-05-m4-qwen36-dense-mlx-155943-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit111/10.62——12.8024.0023.008.00
2026-05-05-m4-qwen36-dense-mlx-155943-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit111/17.582.8626.7116.63657.000.00126.00
2026-05-05-m4-qwen36-dense-mlx-155943-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit111/18.672.5026.8919.14579.000.00166.00
2026-05-05-m4-qwen36-dense-mlx-smoke-155916smoke-chat 0.1.0capitalApple M4 Max (128 GB)mlx-lm serverQwen3.6 27B4bit111/10.99——8.1124.000.008.00
2026-05-05-m4-qwen36-dense-ollama-155051-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M110/11.13——0.88183.00—1.00
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M33—9.39—16.4213.63795.00—128.00
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M33—9.79—16.4613.08353.00—128.00
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M33—8.52—16.3615.0276.00—128.00
2026-05-05-m4-qwen36-dense-ollama-155051-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M111/12.21—16.5313.5512.00—30.00
2026-05-05-m4-qwen36-dense-ollama-155051-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M110/127.11—16.3114.16646.00—384.00
2026-05-05-m4-qwen36-dense-ollama-155051-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M110/126.86—16.1614.30568.00—384.00
2026-05-05-m4-qwen36-dense-ollama-smoke-154808smoke-chat 0.1.0capitalApple M4 Max (128 GB)ollamaQwen3.6 27BQ4_K_M111/114.02—16.562.1412.00—30.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M110/11.08——53.56194.000.0058.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M33—1.850.0470.7769.35806.00802.00128.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M33—1.760.0474.1872.54364.00360.00128.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M33—1.880.0467.3865.9887.0083.00124.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M111/10.21——38.1224.000.008.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M111/12.660.5871.8656.05657.000.00149.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M111/12.690.5473.8258.84579.000.00158.00
2026-05-05-m4-qwen36-moe-llamacpp-reasonoff-smoke-153634smoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M111/10.55——14.4524.000.008.00
2026-05-05-m4-qwen36-moe-llamacpp-smoke-153159smoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 35B-A3BUD-Q4_K_M110/11.14——56.2122.000.0064.00
2026-05-05-m4-qwen36-moe-mlx-155645-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP4110/10.93——62.64194.000.0058.00
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP433—1.420.1399.7290.44806.00805.00128.00
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP433—1.440.1397.3288.66364.00363.00128.00
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP433—1.120.12100.6789.8187.0086.00101.00
2026-05-05-m4-qwen36-moe-mlx-155645-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP4111/10.37——21.9124.0023.008.00
2026-05-05-m4-qwen36-moe-mlx-155645-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP4111/12.080.5693.6068.32657.000.00142.00
2026-05-05-m4-qwen36-moe-mlx-155645-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP4111/12.130.5594.5770.06579.000.00149.00
2026-05-05-m4-qwen36-moe-mlx-smoke-155619smoke-chat 0.1.0capitalApple M4 Max (128 GB)mlx-lm serverQwen3.6 35B-A3BMXFP4111/11.46——5.4824.000.008.00
2026-05-05-m4-qwen36-moe-ollama-154846-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M110/16.12—45.5241.86183.00—256.00
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M33—3.29—45.7638.95795.00—128.00
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M33—3.36—45.6138.05353.00—128.00
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M33—3.16—45.0540.5676.00—128.00
2026-05-05-m4-qwen36-moe-ollama-154846-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M111/11.22——————
2026-05-05-m4-qwen36-moe-ollama-154846-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M110/16.41—45.6539.01646.00—250.00
2026-05-05-m4-qwen36-moe-ollama-154846-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M110/19.42—45.3640.77568.00—384.00
2026-05-05-m4-qwen36-moe-ollama-smoke-154747smoke-chat 0.1.0capitalApple M4 Max (128 GB)ollamaQwen3.6 35B-A3BUD-Q4_K_M111/16.12——————
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M110/10.65——119.15191.004.0078.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.680.01138.82136.51799.00798.0093.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.880.02140.94136.99356.00351.00121.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.610.02140.34135.1784.0079.0083.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/10.11——69.8021.000.008.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/11.160.27139.02106.62677.006.00124.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/11.090.26140.91107.45596.006.00117.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M110/10.65——119.48191.004.0078.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.670.01141.02138.63799.00798.0093.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.880.02141.75137.83356.00351.00121.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.600.02142.55137.1984.0079.0083.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/10.11——71.1821.000.008.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/11.150.27141.03107.79677.006.00124.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/11.090.26141.31107.72596.006.00117.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.670.01141.28138.35799.00798.0093.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.890.03140.53136.38356.00351.00121.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M33—0.610.02141.18135.5884.0079.0083.00
2026-05-07-m4-max-gemma4-llama-reasoning-off-smoke-20260507-103856smoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverGemma 4 E2BQ4_K_M111/10.30——26.4421.000.008.00
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-dashboard-regression-external-agentdjango-dashboard-regression-fix-external-agent 0.1.1fix-dashboard-regressionsApple M4 Max (128 GB)—Qwen3 Coder 30B-A3BQ4_K_M110/111.32—96.1583.841468.00—949.00
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-patch-external-agentpatch-from-failure-external-agent 0.1.1fix-greetingApple M4 Max (128 GB)—Qwen3 Coder 30B-A3BQ4_K_M111/111.83—102.857.86346.00—93.00
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-python-regression-external-agentpython-regression-fix-external-agent 0.1.1fix-task-summaryApple M4 Max (128 GB)—Qwen3 Coder 30B-A3BQ4_K_M110/15.58—100.0783.73957.00—467.00
2026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregationApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/122.88——18.09962.000.00414.00
2026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/10.77——10.3924.000.008.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/14.62——13.64363.000.0063.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—5.860.1522.4221.84806.00802.00128.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—5.990.1521.9421.39364.00360.00128.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M33—4.960.1521.9221.1987.0083.00105.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/10.68——11.8224.000.008.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/110.192.9522.2615.81657.000.00161.00
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverQwen3.6 27BQ4_K_M111/19.832.5622.1516.38579.000.00161.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patchpatch-from-failure 0.1.0fix-greetingApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M111/11.00——61.79373.004.0062.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—1.470.0489.9587.25803.00798.00128.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—1.390.0491.6989.08360.00355.00124.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—0.730.0492.2687.5888.0083.0064.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M111/10.24——33.9425.000.008.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-contextApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M111/11.980.5792.5365.70681.006.00130.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-smallApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M111/11.560.5492.9060.99600.006.0095.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregationApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M110/13.46——67.841029.004.00235.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0longApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—1.450.0491.0788.26803.00798.00128.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0mediumApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—1.380.0492.7690.02360.00355.00124.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0shortApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M33—0.720.0494.1889.1388.0083.0064.00
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smokesmoke-chat 0.1.0capitalApple M4 Max (128 GB)llama-serverGemma 4 26B-A4BQ4_K_M111/10.22——37.0325.000.008.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0longNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M33—0.800.01119.38117.33799.00798.0094.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0mediumNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M33—0.900.02120.58117.50356.00351.00106.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0shortNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M33—0.700.02122.14118.4984.0079.0083.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-smokesmoke-chat 0.1.0capitalNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M111/10.22——37.1721.000.008.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-patchpatch-from-failure 0.1.0fix-greetingNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M110/10.72——108.09191.004.0078.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-wrapdesktop-django-wrap 0.1.5wrap-plan-contextNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M111/11.180.11118.60107.41677.006.00127.00
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-wrapdesktop-django-wrap 0.1.5wrap-plan-smallNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 E2BQ4_K_M111/11.070.19118.1497.01596.000.00104.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patchpatch-from-failure 0.1.0fix-greetingNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/15.18——12.17363.000.0063.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0longNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M33—8.940.1214.5114.31806.00802.00128.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0mediumNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M33—8.860.1214.6414.44364.00360.00128.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0shortNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M33—6.370.1214.7314.4487.0083.0092.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smokesmoke-chat 0.1.0capitalNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/10.72——11.1524.000.008.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-contextNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/111.791.3714.5012.81657.000.00151.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-smallNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/111.781.2314.5012.99579.000.00153.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregationNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/131.07——13.78962.000.00428.00
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-smokesmoke-chat 0.1.0capitalNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.6 27BQ4_K_M111/10.73——10.9324.000.008.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patchpatch-from-failure 0.1.0fix-greetingNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M111/11.16——53.38373.004.0062.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0longNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—1.860.0671.1468.91803.00798.00128.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0mediumNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—1.780.0573.8471.93360.00355.00128.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0shortNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—0.870.0475.8372.5188.0083.0063.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smokesmoke-chat 0.1.0capitalNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M111/10.31——25.5125.000.008.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-contextNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M111/12.400.4671.6757.95681.006.00139.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-smallNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M111/12.260.4772.2257.06600.006.00129.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0longNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—1.860.0671.1368.87803.00798.00128.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0mediumNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—1.780.0573.9371.93360.00355.00128.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0shortNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M33—0.870.0476.0072.6688.0083.0063.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smokesmoke-chat 0.1.0capitalNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverGemma 4 26B-A4BQ4_K_M111/10.18——45.1925.000.008.00
2026-06-28-product-offer-pricerunner-pi-gpt55product-offer-matching 0.1.0cluster-pricerunner-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—110/10.00——608.071.00—1.00
2026-06-28-product-offer-pricerunner-pi-gpt55product-offer-matching 0.1.0cluster-pricerunner-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—110/10.00——1096.741.00—1.00
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0longApple M4 Max (128 GB)—laguna-xs21-q4km—33—1.150.02113.59111.76802.00801.00128.00
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—laguna-xs21-q4km—33—0.690.04120.35113.65370.00365.0078.00
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0shortApple M4 Max (128 GB)—laguna-xs21-q4km—33—0.580.04121.24113.7097.0092.0066.00
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0longApple M4 Max (128 GB)—Qwen3.8 27B—33—6.110.1321.4120.94806.00802.00128.00
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—Qwen3.8 27B—33—6.100.1321.4620.99364.00360.00128.00
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0shortApple M4 Max (128 GB)—Qwen3.8 27B—33—5.170.1321.3520.6987.0083.00107.00
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0longApple M4 Max (128 GB)—Qwen3.8 27B—33—5.140.1925.9024.90806.00805.00128.00
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0mediumApple M4 Max (128 GB)—Qwen3.8 27B—33—5.090.1926.0925.12364.00363.00128.00
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0shortApple M4 Max (128 GB)—Qwen3.8 27B—33—4.310.1926.5025.3287.0086.00109.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0longApple M5 Max (64 GB)—laguna-xs21-q4km—33—1.220.02107.03105.17802.00801.00128.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—laguna-xs21-q4km—33—1.11——115.01370.00365.00128.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0shortApple M5 Max (64 GB)—laguna-xs21-q4km—33—0.99——129.2997.0092.00128.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0longApple M5 Max (64 GB)—laguna-xs21-q4km—33—1.430.0290.9989.21802.00801.00128.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—laguna-xs21-q4km—33—1.14——112.61370.00365.00128.00
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0shortApple M5 Max (64 GB)—laguna-xs21-q4km—33—1.02——125.6697.0092.00128.00
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0longApple M5 Max (64 GB)—Qwen3.8 27B—33—5.670.0822.9222.59806.00802.00128.00
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—Qwen3.8 27B—33—5.520.0823.5323.20364.00360.00128.00
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0shortApple M5 Max (64 GB)—Qwen3.8 27B—33—4.480.0724.2923.9187.0083.00107.00
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0longApple M5 Max (64 GB)—Qwen3.8 27B—33—4.270.1631.1529.97806.00805.00128.00
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0mediumApple M5 Max (64 GB)—Qwen3.8 27B—33—4.260.1631.2230.04364.00363.00128.00
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0shortApple M5 Max (64 GB)—Qwen3.8 27B—33—3.620.1631.6730.0987.0086.00109.00
product-offer-pi-gpt55-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—111/10.00——655.181.00—1.00
product-offer-pi-gpt55-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—110/10.00——864.711.00—1.00
product-offer-pi-gpt55-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—110/10.00——551.291.00—1.00
product-offer-pi-gpt55-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus Piopenai-codex/gpt-5.5—110/10.00——1058.581.00—1.00
product-offer-pi-qwen36-27b-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——564.111.00—1.00
product-offer-pi-qwen36-27b-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——832.121.00—1.00
product-offer-pi-qwen36-27b-llamacpp-high-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)—placeholder-adapter—110/10.00——586.871.00—1.00
product-offer-pi-qwen36-27b-llamacpp-high-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)—placeholder-adapter—110/10.00——962.581.00—1.00
product-offer-pi-qwen36-27b-tools-128k-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——460.651.00—1.00
product-offer-pi-qwen36-27b-tools-128k-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——817.741.00—1.00
product-offer-pi-qwen36-27b-tools-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——573.901.00—1.00
product-offer-pi-qwen36-27b-tools-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)benchpack external-agent placeholder adapter plus PiQwen3.6 27B—110/10.00——1053.321.00—1.00
product-offer-pi-qwen36-llamacpp-high-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)—Qwen3.6 27BQ4_K_M110/10.00——739.691.00—1.00
product-offer-pi-qwen36-llamacpp-high-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-rustApple M4 Max (128 GB)—Qwen3.6 27BQ4_K_M110/10.00——868.871.00—1.00
product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-final-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)—Qwen3.6 27BQ4_K_M110/10.00——716.651.00—1.00
product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-rerun-20260627product-offer-matching 0.1.0pairwise-real-small-pythonApple M4 Max (128 GB)—Qwen3.6 27BQ4_K_M110/10.00——655.521.00—1.00
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0longNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.8 27BQ4_K_M33—9.950.1713.0912.86806.00802.00128.00
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0mediumNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.8 27BQ4_K_M33—9.900.1713.1512.92364.00360.00128.00
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0shortNVIDIA RTX 4000 SFF Ada Generation (20 GB VRAM)llama-serverQwen3.8 27BQ4_K_M33—7.970.1813.2312.9387.0083.00103.00

Case Metrics

runpackcaserowsokprompt rowsprompt mediancached rowscached medianprefill rowsprefill median
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0long333811.000—314475.75
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0medium333376.000—36245.15
2026-04-28-mlx-lm-runtimeruntime-sweep 0.1.0short333104.000—31703.63
2026-04-28-mlx-lm-smokesmoke-chat 0.1.0capital11141.000—0—
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0long333811.000—3120958.26
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0medium333376.000—364642.76
2026-04-29-llama-server-runtimeruntime-sweep 0.1.0short333104.000—318814.73
2026-04-29-llama-server-smokesmoke-chat 0.1.0capital11141.000—0—
2026-04-30-llama-server-desktop-django-wrapdesktop-django-wrap 0.1.5wrap-plan-context111648.0010.00118793.21
2026-04-30-llama-server-desktop-django-wrapdesktop-django-wrap 0.1.5wrap-plan-small111569.0010.0012074.09
2026-05-03-m4-max-llama-qwen25-05b-190636-patchpatch-from-failure 0.1.0fix-greeting111199.00123.000—
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0long333811.003810.003116940.93
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0medium333376.003375.00360860.30
2026-05-03-m4-max-llama-qwen25-05b-190636-runtimeruntime-sweep 0.1.0short333104.003103.00317181.44
2026-05-03-m4-max-llama-qwen25-05b-190636-smokesmoke-chat 0.1.0capital11141.0010.000—
2026-05-03-m4-max-llama-qwen25-05b-190636-wrapdesktop-django-wrap 0.1.5wrap-plan-context111648.0010.00110915.43
2026-05-03-m4-max-llama-qwen25-05b-190636-wrapdesktop-django-wrap 0.1.5wrap-plan-small111569.0010.0018907.68
2026-05-03-m4-max-mlx-qwen25-05b-185824-patchpatch-from-failure 0.1.0fix-greeting111199.00124.000—
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0long333811.003810.00314740.91
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0medium333376.003375.0037108.90
2026-05-03-m4-max-mlx-qwen25-05b-185824-runtimeruntime-sweep 0.1.0short333104.003103.0031989.65
2026-05-03-m4-max-mlx-qwen25-05b-185824-smokesmoke-chat 0.1.0capital11141.0010.000—
2026-05-03-m4-max-mlx-qwen25-05b-185824-wrapdesktop-django-wrap 0.1.5wrap-plan-context111648.00126.0016435.77
2026-05-03-m4-max-mlx-qwen25-05b-185824-wrapdesktop-django-wrap 0.1.5wrap-plan-small111569.00123.0015332.03
2026-05-03-m4-max-qwen3coder-184938-patchpatch-from-failure 0.1.0fix-greeting111179.000—1932.39
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0long333790.000—376626.80
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0medium333355.000—333696.40
2026-05-03-m4-max-qwen3coder-184938-runtimeruntime-sweep 0.1.0short33383.000—37975.31
2026-05-03-m4-max-qwen3coder-184938-smokesmoke-chat 0.1.0capital11120.000—152.08
2026-05-03-m4-max-qwen3coder-184938-wrapdesktop-django-wrap 0.1.5wrap-plan-context111627.000—11289.58
2026-05-03-m4-max-qwen3coder-184938-wrapdesktop-django-wrap 0.1.5wrap-plan-small111548.000—11191.99
2026-05-05-m4-qwen36-dense-llamacpp-154321-patchpatch-from-failure 0.1.0fix-greeting111194.0010.000—
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0long333806.003802.0035156.87
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0medium333364.003360.0032437.34
2026-05-05-m4-qwen36-dense-llamacpp-154321-runtimeruntime-sweep 0.1.0short33387.00383.003570.36
2026-05-05-m4-qwen36-dense-llamacpp-154321-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-dense-llamacpp-154321-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.001218.57
2026-05-05-m4-qwen36-dense-llamacpp-154321-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.001222.64
2026-05-05-m4-qwen36-dense-llamacpp-smoke-154255smoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-dense-mlx-155943-patchpatch-from-failure 0.1.0fix-greeting111194.0010.000—
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0long333806.003805.0034493.64
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0medium333364.003363.0032036.54
2026-05-05-m4-qwen36-dense-mlx-155943-runtimeruntime-sweep 0.1.0short33387.00386.003491.91
2026-05-05-m4-qwen36-dense-mlx-155943-smokesmoke-chat 0.1.0capital11124.00123.000—
2026-05-05-m4-qwen36-dense-mlx-155943-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.001229.75
2026-05-05-m4-qwen36-dense-mlx-155943-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.001231.51
2026-05-05-m4-qwen36-dense-mlx-smoke-155916smoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-dense-ollama-155051-patchpatch-from-failure 0.1.0fix-greeting111183.000—1180.88
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0long333795.000—3549.94
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0medium333353.000—3191.22
2026-05-05-m4-qwen36-dense-ollama-155051-runtimeruntime-sweep 0.1.0short33376.000—3138.13
2026-05-05-m4-qwen36-dense-ollama-155051-smokesmoke-chat 0.1.0capital11112.000—144.51
2026-05-05-m4-qwen36-dense-ollama-155051-wrapdesktop-django-wrap 0.1.5wrap-plan-context111646.000—1195.23
2026-05-05-m4-qwen36-dense-ollama-155051-wrapdesktop-django-wrap 0.1.5wrap-plan-small111568.000—1202.14
2026-05-05-m4-qwen36-dense-ollama-smoke-154808smoke-chat 0.1.0capital11112.000—118.23
2026-05-05-m4-qwen36-moe-llamacpp-153657-patchpatch-from-failure 0.1.0fix-greeting111194.0010.000—
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0long333806.003802.00321799.00
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0medium333364.003360.00310168.98
2026-05-05-m4-qwen36-moe-llamacpp-153657-runtimeruntime-sweep 0.1.0short33387.00383.0031983.44
2026-05-05-m4-qwen36-moe-llamacpp-153657-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-moe-llamacpp-153657-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.0011123.31
2026-05-05-m4-qwen36-moe-llamacpp-153657-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.0011062.68
2026-05-05-m4-qwen36-moe-llamacpp-reasonoff-smoke-153634smoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-moe-llamacpp-smoke-153159smoke-chat 0.1.0capital11122.0010.000—
2026-05-05-m4-qwen36-moe-mlx-155645-patchpatch-from-failure 0.1.0fix-greeting111194.0010.000—
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0long333806.003805.0036118.59
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0medium333364.003363.0032832.95
2026-05-05-m4-qwen36-moe-mlx-155645-runtimeruntime-sweep 0.1.0short33387.00386.003696.58
2026-05-05-m4-qwen36-moe-mlx-155645-smokesmoke-chat 0.1.0capital11124.00123.000—
2026-05-05-m4-qwen36-moe-mlx-155645-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.0011170.50
2026-05-05-m4-qwen36-moe-mlx-155645-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.0011050.42
2026-05-05-m4-qwen36-moe-mlx-smoke-155619smoke-chat 0.1.0capital11124.0010.000—
2026-05-05-m4-qwen36-moe-ollama-154846-patchpatch-from-failure 0.1.0fix-greeting111183.000—1670.11
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0long333795.000—32288.59
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0medium333353.000—3862.50
2026-05-05-m4-qwen36-moe-ollama-154846-runtimeruntime-sweep 0.1.0short33376.000—3415.95
2026-05-05-m4-qwen36-moe-ollama-154846-smokesmoke-chat 0.1.0capital110—0—0—
2026-05-05-m4-qwen36-moe-ollama-154846-wrapdesktop-django-wrap 0.1.5wrap-plan-context111646.000—1868.31
2026-05-05-m4-qwen36-moe-ollama-154846-wrapdesktop-django-wrap 0.1.5wrap-plan-small111568.000—1829.47
2026-05-05-m4-qwen36-moe-ollama-smoke-154747smoke-chat 0.1.0capital110—0—0—
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-patchpatch-from-failure 0.1.0fix-greeting111191.0014.000—
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0long333799.003798.00370476.94
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0medium333356.003351.00314817.46
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-runtimeruntime-sweep 0.1.0short33384.00379.0033683.39
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-smokesmoke-chat 0.1.0capital11121.0010.000—
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-wrapdesktop-django-wrap 0.1.5wrap-plan-context111677.0016.0012498.04
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1058-wrapdesktop-django-wrap 0.1.5wrap-plan-small111596.0016.0012305.09
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-patchpatch-from-failure 0.1.0fix-greeting111191.0014.000—
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0long333799.003798.00370594.47
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0medium333356.003351.00314678.95
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-runtimeruntime-sweep 0.1.0short33384.00379.0033680.53
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-smokesmoke-chat 0.1.0capital11121.0010.000—
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-wrapdesktop-django-wrap 0.1.5wrap-plan-context111677.0016.0012496.94
2026-05-07-m4-max-gemma4-llama-reasoning-off-4pack-20260507-1116-wrapdesktop-django-wrap 0.1.5wrap-plan-small111596.0016.0012309.08
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0long333799.003798.00357450.16
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0medium333356.003351.00313811.25
2026-05-07-m4-max-gemma4-llama-reasoning-off-runtime-20260507-103928runtime-sweep 0.1.0short33384.00379.0033430.95
2026-05-07-m4-max-gemma4-llama-reasoning-off-smoke-20260507-103856smoke-chat 0.1.0capital11121.0010.000—
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-dashboard-regression-external-agentdjango-dashboard-regression-fix-external-agent 0.1.1fix-dashboard-regressions1111468.000—11321.70
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-patch-external-agentpatch-from-failure-external-agent 0.1.1fix-greeting111346.000—11078.93
2026-05-09-m4-max-codex-oss-direct-edit-20260509-162514-python-regression-external-agentpython-regression-fix-external-agent 0.1.1fix-task-summary111957.000—11352.95
2026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregation111962.0010.000—
2026-05-09-m4-max-qwen36-27b-llamacpp-strict-preflight-20260509-220411-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patchpatch-from-failure 0.1.0fix-greeting111363.0010.000—
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0long333806.003802.0035304.76
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0medium333364.003360.0032418.51
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0short33387.00383.003583.75
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.001222.52
2026-05-10-m4-max-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.001226.07
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patchpatch-from-failure 0.1.0fix-greeting111373.0014.000—
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0long333803.003798.00318378.28
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0medium333360.003355.0039084.58
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0short33388.00383.0032310.64
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smokesmoke-chat 0.1.0capital11125.0010.000—
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-context111681.0016.0011187.03
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-small111600.0016.0011121.19
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregation1111029.0014.000—
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0long333803.003798.00317985.88
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0medium333360.003355.0038795.53
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0short33388.00383.0032285.61
2026-05-12-m4-max-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smokesmoke-chat 0.1.0capital11125.0010.000—
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0long333799.003798.00357586.96
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0medium333356.003351.00315482.70
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-runtimeruntime-sweep 0.1.0short33384.00379.0034002.76
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-154814-smokesmoke-chat 0.1.0capital11121.0010.000—
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-patchpatch-from-failure 0.1.0fix-greeting111191.0014.000—
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-wrapdesktop-django-wrap 0.1.5wrap-plan-context111677.0016.0016065.93
2026-05-07-hetzner-gex44-gemma4-llama-strict-gguf-20260507-161956-wrapdesktop-django-wrap 0.1.5wrap-plan-small111596.0010.0013107.86
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-patchpatch-from-failure 0.1.0fix-greeting111363.0010.000—
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0long333806.003802.0036568.55
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0medium333364.003360.0033086.33
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-runtimeruntime-sweep 0.1.0short33387.00383.003698.38
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-context111657.0010.001478.24
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-fourpack-20260510-131758-wrapdesktop-django-wrap 0.1.5wrap-plan-small111579.0010.001471.27
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-endpoint-correctnessendpoint-python-correctness 0.2.0fix-inventory-aggregation111962.0010.000—
2026-05-10-hetzner-gex44-qwen36-27b-llamacpp-strict-preflight-20260509-234711-smokesmoke-chat 0.1.0capital11124.0010.000—
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-patchpatch-from-failure 0.1.0fix-greeting111373.0014.000—
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0long333803.003798.00313819.58
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0medium333360.003355.0037791.30
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-runtimeruntime-sweep 0.1.0short33388.00383.0032319.63
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-smokesmoke-chat 0.1.0capital11125.0010.000—
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-context111681.0016.0011483.53
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-fourpack-20260512-wrapdesktop-django-wrap 0.1.5wrap-plan-small111600.0016.0011264.00
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0long333803.003798.00313554.75
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0medium333360.003355.0037570.86
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-runtimeruntime-sweep 0.1.0short33388.00383.0032305.02
2026-05-12-hetzner-gex44-gemma4-26b-a4b-llamacpp-strict-preflight-20260512-smokesmoke-chat 0.1.0capital11125.0010.000—
2026-06-28-product-offer-pricerunner-pi-gpt55product-offer-matching 0.1.0cluster-pricerunner-python1111.000—0—
2026-06-28-product-offer-pricerunner-pi-gpt55product-offer-matching 0.1.0cluster-pricerunner-rust1111.000—0—
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0long333802.003801.00346115.26
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0medium333370.003365.0039686.93
2026-09-01-m4-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0short33397.00392.0032681.28
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0long333806.003802.0036042.70
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0medium333364.003360.0032738.82
2026-09-01-m4-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0short33387.00383.003657.91
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0long333806.003805.0034149.30
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0medium333364.003363.0031939.49
2026-09-01-m4-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0short33387.00386.003450.86
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0long333802.003801.00337858.61
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0medium333370.003365.000—
2026-09-01-m5-max-laguna-xs21-llamacpp-q4kmruntime-sweep 0.1.0short33397.00392.000—
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0long333802.003801.00334736.35
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0medium333370.003365.000—
2026-09-01-m5-max-laguna-xs21-llamacpp-q4km-rerunruntime-sweep 0.1.0short33397.00392.000—
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0long333806.003802.0039736.99
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0medium333364.003360.0034710.69
2026-09-01-m5-max-qwen38-27b-llamacpp-q4kmruntime-sweep 0.1.0short33387.00383.0031209.38
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0long333806.003805.0034925.74
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0medium333364.003363.0032239.04
2026-09-01-m5-max-qwen38-27b-mlx-4bitruntime-sweep 0.1.0short33387.00386.003529.40
product-offer-pi-gpt55-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-gpt55-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-gpt55-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-gpt55-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-27b-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-27b-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-27b-llamacpp-high-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-27b-llamacpp-high-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-27b-tools-128k-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-27b-tools-128k-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-27b-tools-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-27b-tools-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-llamacpp-high-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-llamacpp-high-large-pairwise-20260627product-offer-matching 0.1.0pairwise-real-small-rust1111.000—0—
product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-final-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
product-offer-pi-qwen36-llamacpp-high-large-pairwise-python-rerun-20260627product-offer-matching 0.1.0pairwise-real-small-python1111.000—0—
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0long333806.003802.0034644.84
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0medium333364.003360.0032160.59
2026-09-02-hetzner-gex44-qwen38-27b-q4km-20260902-runtimeruntime-sweep 0.1.0short33387.00383.003483.88

Agent Wrap One-Shot Runs

The Outcome column is the recorded benchmark result; Node tests and smoke are supporting verification checks.

This curated agent-workflow dataset has no host facet. For local-runtime prompt-pack results, select Comparison Matrix and the desktop-django-wrap pack.

outcomemodelharness/providerthinkingtimefilesinsertionsdeletionsnode testssmoketargetartifactnotes
PASSGPT-6.1 SolPi / OpenAI subscription / established full-accessthinking low, established full-access359.90s
6m00s agent		37833314154health 200; root 302 → resume 200django-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt61sol-pi-openai-sub-low-20261003/
notesFresh Pi0.99.2 established-full-access; OpenAI ChatGPT subscription OAuth provider openai, not paid API/OpenRouter. One sample, no ranking claim. Agent 359.9s, full 369.246124s; verifier not separately timed. Input including cache/cached/output 914801/868992/6295; native Pi uncached input 45809, cache is input subset. Prior zero-token openai-codex auth failure remains separate infrastructure-invalid, not a model FAIL. Effort from explicit argv/provider mapping; native JSON does not expose server effort. Provenance and clean logs: results/agent-wrap-oneshot/gpt61sol-pi-openai-sub-low-20261003/provenance.json.
PASSGPT-5.6 SolPi / OpenAI Codexthinking off, neutral runner362.90s
6m03s		36610014439HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt56sol-pi-django-resume-030-off/
notesNew fastest row in the dataset: 362.9s beats GPT-5.5 via Pi thinking off (431.7s) by 16%. Pi 0.84.4 through the built-in openai-codex provider, model gpt-5.6-sol, thinking off, --pi-mode json (JSON transcript kept for forensics; the GPT-5.5 Pi cell used text mode). 38 turns and 68 tool calls (27 read, 21 bash, 14 write, 6 edit). Smallest diff of the 2026-09-01/02 cells (36 files, +6100/-144) with the leanest Django side (89-line base_settings.py, 46-line middleware.py); 39 model-authored Node tests against 53 for GPT-5.5 Pi off. Verified its own dev smoke and redirect chain but did not boot its own packaged smoke; the verifier's independent packaged smoke passed. D-040 override in effect, empty dist/, no workaround. Host caveat: another session's Flash-Next llama-server was resident (13-14% free memory) and generating for its own qwen38-flashnext-pi-llamacpp-128k-medium-rerun2 wrap cell at 74-107% CPU throughout, 1-minute load average 3-19, so the wall time is an upper bound.
PASSGPT-5.6 SolCodex CLI / yoloreasoning low, neutral runner362.90s
6m03s		37819914153HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt56sol-codex-yolo-django-resume-030-low/
notesPassed at 362.9s, tying the GPT-5.6 Sol Pi thinking-off cell to the tenth of a second for fastest row in the dataset, 17% faster than the GPT-5.5 Codex low rerun (438.3s), and passing on the first attempt where GPT-5.5 low had failed once (491.9s) before its rerun. 53 model-authored Node tests, the same count as every GPT-5.5 Codex cell; 37 files, +8199/-141, with a compact Django side (62-line base_settings.py, 41-line desktop.py) and no packaging CI workflow, which it called optional for the wrapping scope. Codex CLI 0.151.0, model id gpt-5.6-sol, model_reasoning_effort=low set explicitly. Verified its own dev smoke, redirect chain, token probes, and packaged settings; did not boot its own packaged smoke; the verifier's independent packaged smoke passed. D-040 override in effect, empty dist/, no workaround. Host caveat: another session's Flash-Next llama-server was resident (13-14% free memory) and generating for its own qwen38-flashnext-pi-llamacpp-128k-medium-rerun2 wrap cell at roughly 70-80% CPU for most of the cell, 1-minute load average 8-21, so the wall time is an upper bound.
PASSGPT-5.5Pi / OpenAI Codexthinking off, neutral runner431.70s
7m12s		35776714053HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-pi-django-resume-030-off/
notesFresh Pi/off retry completed normally and passed packaged smoke; fastest current-target pass.
PASSGPT-5.5Codex CLI / yoloreasoning low, neutral rerun438.30s
7m18s		378179053HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-codex-yolo-django-resume-030-low-rerun1/
notesFresh Codex low rerun passed packaged smoke; fastest direct Codex CLI current-target pass.
PASSGPT-6.1 SolCodex CLI / established full-accessreasoning low, established full-access473.30s
7m53s agent		42862414054health 200; root 302 → resume 200django-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt61sol-established-low-20261003/
notesFresh established-full-access profile; one sample, no ranking claim. Registry time is agent 473.3s, not full runner 482.252439s; verifier not separately timed. ChatGPT subscription, Codex 0.160.0. Native input/cached/output 1658709/1580416/11583; cached is subset of input. Earlier confined/blocked/supplemental records remain separate. Exact provenance and clean logs: results/agent-wrap-oneshot/gpt61sol-established-low-20261003/provenance.json.
PASSClaude Opus 5Claude Code / yoloeffort low, neutral runner535.20s
8m55s		36691713941HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus5-claude-yolo-django-resume-030-low/
notesPassed at 535.2s, 49% faster than Opus 4.8 low (1053.0s) and the fastest direct Claude Code/yolo row so far, ahead of Fable 5.1 medium (737.5s). Explicit model id claude-opus-5 on Claude Code 2.1.257 with the D-040 Electron override in effect. 41 model-authored Node tests (Opus 4.8 low: 53). Authored a Go menu for back/forward/overview navigation and an app-data-copy desktop dev mode because example/db.sqlite3 is tracked; deliberately added no packaging CI workflow and said so in its docs. Did not boot its own packaged smoke; the verifier's independent packaged smoke passed. The first launch at 23:13 hit the Claude account session limit after 188.6s with 28 files changed and was discarded as infrastructure noise; this row is the forced retry launched 00:57. Host: the first ~3 minutes ran on a quiet host (80% free memory); from 01:00 another session's Flash-Next llama-server was resident again (10-11% free memory) and generating for its own wrap cell at 80-180% CPU, with the 1-minute load average reaching 101 as this cell finished, so the wall time is still an upper bound.
PASSDeepSeek V4.1 FlashPi / OpenRouterthinking low, neutral runner560.00s
9m20s		33700913940HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/deepseek-v41-flash-pi-openrouter-django-resume-030-low/
notesFastest cell of the 2026-09-10 DeepSeek V4.1 Flash Pi/OpenRouter campaign at 560.0s, with 28,303 reasoning tokens and the cheapest spend of the four ($0.1838). 57 turns, 78 tool calls (51 bash, 14 write, 8 edit, 5 read), 18.0 MB transcript. 40 model-authored Node tests. Self-verified with three `just desktop-dev-smoke` runs and two `npm --prefix electron test` runs; did not boot its own packaged smoke, which the verifier then passed.
PASSDeepSeek V4.1 FlashPi / OpenRouterthinking high, neutral runner576.00s
9m36s		34757613853HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/deepseek-v41-flash-pi-openrouter-django-resume-030-high/
notesSecond-fastest DeepSeek V4.1 Flash cell of the 2026-09-10 Pi/OpenRouter campaign at 576.0s, with the most reasoning of the four (34,487 tokens) and 53 Node tests. 75 turns, 95 tool calls (76 bash, 14 write, 4 edit, 1 read), 21.1 MB transcript, $0.1986. Three dev smokes and two Node test runs; the verifier's independent packaged smoke passed.
PASSGPT-5.6 SolCodex CLI / yoloreasoning medium, neutral runner595.70s
9m56s		39841714253HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt56sol-codex-yolo-django-resume-030-medium/
notesFirst GPT-5.6 Sol Codex CLI cell. Passed at 595.7s, 20% faster than GPT-5.5 Codex medium (741.9s) with a comparable diff (39 files, +8417/-142 vs +7401/-140) and 53 model-authored Node tests (GPT-5.5 medium: 55) plus 4 new Django tests in tests/desktop_runtime_test.py. Codex CLI 0.151.0, model id gpt-5.6-sol, model_reasoning_effort=medium set explicitly on the command line (the host's ~/.codex/config.toml default is high). Verified its own dev smoke, redirect chain, token behaviour, and packaged settings through the test client; did not boot its own packaged smoke; the verifier's independent packaged smoke passed. D-040 override in effect, empty dist/, no workaround. Host caveat: another session's Flash-Next llama-server was resident (13-14% free memory) and generating for its own qwen38-flashnext-pi-llamacpp-128k-medium-rerun2 wrap cell at 74-107% CPU throughout, 1-minute load average 3-21, so the wall time is an upper bound.
PASSClaude Opus 4.8Pi / OpenRouteroff612.40s
10m12s		36751313553HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-opus48-openrouter-main-off/
notesFastest Opus 4.8 current-target pass; highest approximate OpenRouter cost among successful hosted rows.
PASSGPT-5.6 SolCodex CLI / yoloreasoning high, neutral runner635.90s
10m36s		43853214154HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt56sol-codex-yolo-django-resume-030-high/
notesCompletes the GPT-5.6 Sol Codex sweep at 3 for 3 (low 362.9s, medium 595.7s, high 635.9s), 18% faster than GPT-5.5 Codex high (776.2s). 54 model-authored Node tests (GPT-5.5 high: 53) and the largest GPT-5.6 Sol diff (43 files, +8532/-141): a 177-line cross-platform packaging workflow with checksum publishing, a docs/dev/desktop.txt guide, and a Sphinx build run with warnings treated as errors. Codex CLI 0.151.0, model id gpt-5.6-sol, model_reasoning_effort=high set explicitly. Verified its own dev smoke, redirect chain, edit view, token probes, packaged static serving, and both media-bootstrap probes; did not boot its own packaged smoke; the verifier's independent packaged smoke passed. D-040 override in effect, empty dist/, no workaround. Host caveat: another session's Flash-Next llama-server was resident (14-18% free memory) and generating intermittently for its own qwen38-flashnext-pi-llamacpp-128k-medium-rerun2 wrap cell at 0-122% CPU, 1-minute load average 3-16, so the wall time is an upper bound.
PASSGPT-6 AstraPi / OpenAI subscription / established full-accessthinking medium, established full-access667.50s
11m08s agent		41794414054health 200; root 302 → resume 200django-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt6astra-pi-openai-sub-medium-20261003/
notesFresh Pi0.99.2 established-full-access; OpenAI ChatGPT subscription OAuth provider openai, not paid API/OpenRouter. One sample, no ranking claim. Agent 667.5s, full 674.916760s; verifier not separately timed. Input including cache/cached/output 1970456/1899776/11857; native Pi uncached input 70680, cache is input subset. Prior zero-token openai-codex auth failure remains separate infrastructure-invalid, not a model FAIL. Effort from explicit argv/provider mapping; native JSON does not expose server effort. Provenance and clean logs: results/agent-wrap-oneshot/gpt6astra-pi-openai-sub-medium-20261003/provenance.json.
PASSGPT-5.5Codex CLI / yoloreasoning low669.90s
11m10s		39838513753HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-gpt55-codex-yolo-main-low/
notesEarlier direct Codex low pass from the legacy artifact root; slower than the fresh Pi/off and Codex-low rerun rows.
PASSGPT-5.5Pi / OpenAI Codexthinking high701.20s
11m41s		35804914353HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-gpt55-main-high/
notesClean Pi/GPT-5.5 pass; parsed unique-turn usage estimate about $4.92.
PASSClaude Fable 5.1Claude Code / yoloeffort low, neutral runner702.90s
11m43s		40795213857HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/fable51-claude-yolo-django-resume-030-low/
notesPassed at 702.9s, 33% faster than Opus 4.8 low (1053.0s) and slightly faster than Fable 5.1 medium (737.5s); slower than Opus 5 low (535.2s). Explicit model id claude-fable-5-1 on Claude Code 2.1.257. 57 model-authored Node tests plus 13 new Django tests in tests/desktop_test.py; authored a Go menu for back/forward/overview navigation and a placeholder icon set. D-040 note: unlike the other three Claude 5 cells, this one reported that electron could not start after npm install skipped Electron's postinstall and copied the 275 MB Electron 40.8.5 dist plus path.txt from the starter checkout, so it measures workaround ability alongside wrapping ability; the other cells ran on the empty dist via the ELECTRON_OVERRIDE_DIST_PATH override. It also wrote a Claude Code auto-memory note about that workaround under ~/.claude/projects/ for the target path. Did not boot its own packaged smoke; the verifier's independent packaged smoke passed. The first launch at 23:11 hit the Claude account session limit after 361.6s with 32 files changed and was discarded as infrastructure noise; this row is the forced retry launched 00:57. Host: quiet for the first ~3 minutes (80% free memory); from 01:00 another session's Flash-Next llama-server was resident again (10-12% free memory) and generating for its own wrap cell, with the 1-minute load average at 90-101 during this cell's last three minutes, so the wall time is an upper bound.
PASSGPT-5.5Codex CLI / yoloreasoning high730.20s
12m10s		39848414053HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-gpt55-codex-yolo-main-high/
notesSlightly slower than Codex low reasoning on this single pair; broader target-side docs/tests.
PASSClaude Fable 5.1Claude Code / yoloeffort medium, neutral runner737.50s
12m18s		43829413460HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/fable51-claude-yolo-django-resume-030-medium/
notesFirst Claude 5-family cell on this benchmark and the fastest direct Claude Code/yolo row so far (737.5s vs Opus 4.8 medium 871.0s). Explicit model id claude-fable-5-1 on Claude Code 2.1.257 with the D-040 Electron override in effect. Authored a Go menu module (app-menu.cjs, with tests) for back/forward/overview navigation plus a Django-side tests/desktop_shell_test.py; 60 model-authored Node tests is the most of any Claude Code/yolo row. The agent skipped its own packaged smoke per the skill's unattended-run guidance; the verifier's independent packaged smoke passed. Host caveat: the first ~6.5 minutes overlapped another session's 744s sustained-generation load test against a resident ~49 GB Flash-Next llama-server (1-minute load average 159 two minutes after launch, 10-14% free memory, swap in use), and its last ~5-7 minutes overlapped the same session's qwen38-flashnext-pi-llamacpp-128k-medium-rerun1 local-model wrap cell (launched 23:06:34) generating on that server, so treat the wall time as an upper bound.
PASSGPT-5.5Codex CLI / yoloreasoning medium, neutral runner741.90s
12m22s		39740114055HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-codex-yolo-django-resume-030-medium/
notesModerate reasoning pass; slower than low rerun and slightly faster than neutral high.
PASSDeepSeek V4.1 FlashPi / OpenRouterthinking medium, neutral runner760.20s
12m40s		34754913653HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/deepseek-v41-flash-pi-openrouter-django-resume-030-medium/
notesMiddle of the 2026-09-10 DeepSeek V4.1 Flash Pi/OpenRouter campaign at 760.2s with 30,806 reasoning tokens and 53 model-authored Node tests, matching the GPT-5.5 Pi count. 65 turns, 108 tool calls (49 read, 38 bash, 10 write, 11 edit), 17.5 MB transcript, $0.2116. The only DeepSeek cell that ran its own `smoke:packaged`; it also ran three dev smokes and three Node test runs.
PASSGPT-5.5Codex CLI / yoloreasoning high, neutral runner776.20s
12m56s		38838013853HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-codex-yolo-django-resume-030-high/
notesFresh neutral-runner repeat. Passed packaged smoke; slower than same-session Codex low failed run.
PASSClaude Opus 5Claude Code / yoloeffort high, neutral runner850.30s
14m10s		47838313458HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus5-claude-yolo-django-resume-030-high/
notesCompletes the Opus 5 sweep at 3 for 3. Passed at 850.3s, 25% faster than Opus 4.8 high (1131.3s) and level with Opus 5 medium (855.8s); Opus 5 low (535.2s) remains the fastest of the three. Explicit model id claude-opus-5 on Claude Code 2.1.257 with the D-040 Electron override in effect (empty dist/, no workaround). 58 model-authored Node tests (Opus 4.8 high: 33), the largest diff of the Opus 5 cells: a separate navigation.cjs module behind the Go menu, an SVG brand icon with generated icon set, a cross-platform CI workflow, and a fix for the project's missing STATICFILES_DIRS that had left example/static/css/custom.css returning 404. Did not boot its own packaged smoke; the verifier's independent packaged smoke passed. The first launch at 23:16 hit the Claude account session limit within 3.1s with no files changed and was discarded as infrastructure noise; this row is the forced retry launched 01:06 after Opus 5 low in the same lane. Host: throughout the cell another session's Flash-Next llama-server was resident (10-14% free memory) and generating for its own wrap cell at 90-145% CPU, with the 1-minute load average between 5 and 133, so the wall time is an upper bound.
PASSClaude Opus 5Claude Code / yoloeffort medium, neutral runner855.80s
14m16s		41707513633HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus5-claude-yolo-django-resume-030-medium/
notesPassed at 855.8s, marginally faster than Opus 4.8 medium (871.0s) and 16% slower than the Fable 5.1 medium cell launched concurrently (737.5s). Explicit model id claude-opus-5 on Claude Code 2.1.257 with the D-040 Electron override in effect. 33 model-authored Node tests, fewer than Opus 4.8 medium (53) and equal to Opus 4.8 high (33). Also authored a Go menu (app-menu.cjs, with tests), staged the backend with the bundled Python 3.13 and re-ran the Django-side checks inside the stage bundle; it did not boot its own packaged smoke, and the verifier's independent packaged smoke passed. Same host caveat as the Fable 5.1 medium cell: the first ~6.5 minutes overlapped another session's 744s sustained-generation load test against a resident ~49 GB Flash-Next llama-server, and its last ~5-7 minutes overlapped the same session's qwen38-flashnext-pi-llamacpp-128k-medium-rerun1 local-model wrap cell (launched 23:06:34) generating on that server, so treat the wall time as an upper bound.
PASSClaude Opus 4.8Claude Code / yoloeffort medium, neutral runner871.00s
14m31s		34748713453HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus48-claude-yolo-django-resume-030-medium/
notesFastest direct Claude Code/yolo row so far; first launch hit transient API 500 before retry.
PASSGPT-6 AstraCodex CLI / established full-accessreasoning medium, established full-access911.70s
15m12s agent		52902414556health 200; root 302 → resume 200django-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt6astra-established-medium-20261003/
notesFresh established-full-access profile; one sample, no ranking claim. Registry time is agent 911.7s, not full runner 919.065475s; verifier not separately timed. ChatGPT subscription, Codex 0.160.0. Native input/cached/output 2144901/2043264/20005; cached is subset of input. Earlier confined/blocked/supplemental records remain separate. Exact provenance and clean logs: results/agent-wrap-oneshot/gpt6astra-established-medium-20261003/provenance.json.
PASSClaude Fable 5.1Claude Code / yoloeffort high, neutral runner935.40s
15m35s		40841513258HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/fable51-claude-yolo-django-resume-030-high/
notesCompletes the Fable 5.1 sweep at 3 for 3 (low 702.9s, medium 737.5s, high 935.4s). Slowest Claude 5 cell, yet 17% faster than Opus 4.8 high (1131.3s) and 27% faster than Sonnet 4.6 high (1276.6s); 10% slower than Opus 5 high (850.3s). Explicit model id claude-fable-5-1 on Claude Code 2.1.257 with the D-040 Electron override in effect (empty dist/, no workaround). 58 model-authored Node tests plus 19 new Django desktop tests in a 358-line tests/desktop_test.py, the most Django-side coverage of any cell; also fixed the project's missing STATICFILES_DIRS, built the Sphinx docs, and shipped no icon or CI workflow by stated choice. Wrote a Claude Code auto-memory note about the host's Electron binary environment under ~/.claude/projects/ for the target path (same rerun hazard as the Fable 5.1 low cell). Did not boot its own packaged smoke; the verifier's independent packaged smoke passed. The first launch at 23:17 hit the Claude account session limit within 3.5s with no files changed and was discarded as infrastructure noise; this row is the forced retry launched 01:08 after Fable 5.1 low in the same lane. Host: throughout the cell another session's Flash-Next llama-server was resident (10-14% free memory) and generating for its own wrap cell at ~90-125% CPU, with the 1-minute load average between 5 and 133, so the wall time is an upper bound.
PASSGPT-5.5Pipy / openai-codexthinking off, native tool loop1003.50s
16m44s		35733914153HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/pipy-gpt55-django-resume-030-off/
notesFirst Pipy native-tool-loop row; passed packaged smoke, slower than Pi/off and direct Codex rows.
PASSGLM-5.2Pi / OpenRouterthinking high1014.00s
16m54s		27669613540HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-glm52-main-high/
notesHigh thinking was faster than GLM thinking off on this single run; estimated cost about $1.51.
PASSClaude Opus 4.8Claude Code / yoloeffort low, neutral runner1053.00s
17m33s		38767913553HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus48-claude-yolo-django-resume-030-low/
notesFirst direct Claude Code/yolo row; low effort is the closest no/low-thinking lane.
PASSGLM-5.2Pi / OpenRouterthinking off1126.40s
18m46s		31668912931HTTP okdjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-glm52-main-off/
notesSlow GLM pass; lowest approximate hosted cost among successful rows.
PASSClaude Opus 4.8Claude Code / yoloeffort high, neutral runner1131.30s
18m51s		34668813633HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/opus48-claude-yolo-django-resume-030-high/
notesPassed, but slower than Claude Code low effort and same-session Codex high.
PASSClaude Sonnet 4.6Claude Code / yoloeffort low, neutral runner1136.10s
18m56s		35731213853HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/sonnet46-claude-yolo-django-resume-030-low/
notesFastest Sonnet 4.6 row; first launch hit transient Claude 529 before retry.
PASSClaude Sonnet 4.6Claude Code / yoloeffort high, neutral runner1276.60s
21m17s		34730913353HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/sonnet46-claude-yolo-django-resume-030-high/
notesPassed; slower than Sonnet low but faster than Sonnet medium.
PASSDeepSeek V4.1 FlashPi / OpenRouterthinking off, neutral runner1371.00s
22m51s		33700013637HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/deepseek-v41-flash-pi-openrouter-django-resume-030-off/
notes2026-09-10 DeepSeek V4.1 Flash Pi/OpenRouter campaign, the first DeepSeek evidence on this benchmark; all four thinking cells passed. Thinking off was the slowest cell at 1371.0s, 2.4x its own thinking-low lane, despite emitting zero reasoning tokens: 96 turns, 108 tool calls (83 bash, 14 write, 7 edit, 4 read), 13.5 MB transcript, $0.2265 OpenRouter spend. It still ran `just desktop-dev-smoke` four times and `npm --prefix electron test` five times, so the extra wall time is exploration, not unverified output. 37 model-authored Node tests, the fewest of the four; did not boot its own packaged smoke, which the verifier's independent run then passed.
PASSClaude Sonnet 4.6Claude Code / yoloeffort medium, neutral runner1554.80s
25m55s		33721313747HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/sonnet46-claude-yolo-django-resume-030-medium/
notesSlowest successful current-target pass so far; fewer model-authored Node tests than low/high.
PASSQwen3.8 Flash-NextPi / llama.cpp localreasoning_effort medium, set explicitly2488.10s
41m28s		34681413634HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-flashnext-pi-llamacpp-128k-medium/
notesFastest local open-weight pass and the first qwen4exp (Flash-Next MoE) cell; needed a source-built llama.cpp (build 200, 774ee0e) at 131072 context with the n-gram table offloaded to CPU. Ran 3 smokes in 54 bash calls and wrote a 34-test harness. Not a controlled comparison with the 27B rows: different binary, context, quant family, and port. Two earlier attempts were terminated externally at ~29.5 min and are discarded, not recorded (D-042); this pass came from a detached relaunch.
PASSQwen3.8 Flash (Alibaba hosted)Pi / OpenRouter (Alibaba)reasoning.effort medium, verified in request body3368.70s
56m09s		41894813660HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-flash-pi-openrouter-django-resume-030-medium/
notesHosted counterpart of the local Qwen3.8-Flash-Next lane: OpenRouter pinned to the Alibaba provider (`provider.only=[Alibaba]`, verified in the request body) with `reasoning.effort=medium`. Ran with the D-040 Electron environment exported, matching the local Flash-Next lane, so the wall time is model/serving-bound rather than Electron-recovery-bound. 176 turns and 172 tool calls (bash 123, read 11, write 30, edit 19); 137,969 output tokens including 53,521 reasoning tokens; $0.53 OpenRouter spend. 60 model-authored Node tests, the most of any Qwen cell here; self-verified and passed the verifier's independent packaged smoke (`/health/` 200, `/` 302, `/resume/` 200). Raw decode measured separately at ~49-54 tok/s versus ~30 tok/s for local Flash-Next on M4 Max. Runtime-and-format comparison, not artifact parity: hosted Alibaba Qwen3.8 Flash (1M context, snapshot 20260826) versus local unsloth UD-IQ4_XS at 131072 context.
PASSQwen3.8 27BPi / llama.cpp localreasoning_effort medium, set explicitly3701.50s
61m42s		36802513540HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-q8-pi-llamacpp-256k-medium/
notesQ8_0 control for the medium lane, same Homebrew server and flags as the Q4_K_M cells. Passed on its first attempt, 31 minutes faster than the Q4_K_M pass, with 8 smokes in 64 bash calls. Wrote a 40-test harness rather than the 53-test one every Q4_K_M cell produced. One observation each way: shows the pass is not an artifact of Q4_K_M, not that Q8_0 is more reliable.
PASSQwen3.8 Flash-NextPi / llama.cpp localreasoning_effort medium, set explicitly3737.70s
62m18s		367032334HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-flashnext-pi-llamacpp-128k-medium-rerun2/
notesSecond replication cell for the Flash-Next pass, launched detached (D-042). Exited cleanly in 62.3 min with 34/34 tests and a served packaged app; Flash-Next at reasoning_effort=medium is 3-for-3, the first local open-weight lane whose pass replicated. 102 tool calls, 3 smoke invocations, no electron-builder call. Ran concurrently with a claude-yolo cell from another session on the same host, with free memory at 9-18% and swap up to 12.5 GB.
PASSQwen3.8 27BPi / llama.cpp localreasoning_effort medium, set explicitly5570.00s
92m50s		36762212053HTTP okdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-pi-llamacpp-256k-medium/
notesFirst pass by a local open-weight model on this benchmark; exited cleanly rather than timing out. Decisive variable was reasoning_effort=medium, not the model or the quant. Did not replicate: the confirmation cell qwen38-pi-llamacpp-256k-medium-rerun1 failed the packaged smoke, so the medium lane is 1-for-2.
PASSQwen3.8 Flash-NextPi / llama.cpp localreasoning_effort medium, set explicitly7200.00s
120m00s		38750413753HTTP ok (verified after the runner timeout)django-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-flashnext-pi-llamacpp-128k-medium-rerun1/
notesFirst replication cell for the Flash-Next pass, launched detached (D-042). The runner's 7200s timeout killed pi (pi_exit=124), but the verifier found a served packaged app with 53/53 tests, so the gate records PASS - the first pass at the cap in this table. About 30 of the 120 minutes went to a single `npx electron-builder --dir`, which auto-discovered the host's Developer ID identity and deep-signed the app (with its embedded Python runtime) file by file with Apple timestamping; reproduced afterwards, not a hang and not the model's doing. No earlier passing cell had invoked electron-builder. 178 tool calls, 11 smoke invocations.
FAILGPT-5.5Codex CLI / yoloreasoning none, neutral runner471.40s
7m51s		337377053Smoke faildjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-codex-yolo-django-resume-030-none/
notesDirect-Codex no-reasoning comparison for Pi/off; failed on Electron install integrity before HTTP checks.
FAILGPT-5.5Codex CLI / yoloreasoning low, neutral runner491.90s
8m12s		348141253Smoke faildjango-resume 0.3.0/mainresults/agent-wrap-oneshot/gpt55-codex-yolo-django-resume-030-low/
notesFast agent phase, but independent smoke failed on Electron install integrity before HTTP checks.
FAILClaude Opus 4.8Pi / OpenRouterthinking high734.30s
12m14s		32338413253Smoke faildjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-opus48-openrouter-main-high/
notesPackaged backend leaked a starter dependency: ModuleNotFoundError: desktop_django_starter.
FAILLaguna XS 2.1Pi / llama.cpp localenable_thinking=true (binary toggle, no effort levels; normalized as high)1910.10s
31m50s		17840477335smoke exit 0 with zero HTTP checks (false success)django-resume 0.3.0/mainresults/agent-wrap-oneshot/laguna-xs21-pi-llamacpp-256k-think/
notesFirst non-Qwen local open-weight cell and first MoE cell. Fastest self-terminating run and largest diff in the table, but zero executions of its own smoke; will-quit handler hardcodes app.exit(0), so the startup failure (unstaged backend) reported success and only the verifier's app_served gate caught it. Counterfactual: one npm-script path fix stages the backend and boots Django, but the Electron shell issued no HTTP requests across two manual smokes.
FAILLaguna XS 2.1Pi / llama.cpp localenable_thinking=false (binary toggle, no effort levels)2144.40s
35m44s		5626114052013045verifier smoke hung to 200s cap, all HTTP counters 0django-resume 0.3.0/mainresults/agent-wrap-oneshot/laguna-xs21-pi-llamacpp-256k-off/
notesThinking-off A/B partner to the thinking-on cell. Behavioral flip: this lane DID self-verify - ran stage-backend, its own packaged smoke (exit 1), 45/45 node tests and 182 Django tests - but never got a green packaged smoke and declared victory anyway; the verifier smoke then hung to the 200s cap. Diff is inflated: 5,590 of 5,626 files are un-gitignored .stage/ staging output; the authored diff is ~36 files.
FAILQwen3.8 27BPi / llama.cpp localreasoning_effort medium, set explicitly3567.30s
59m27s		33753713653smoke exit 1, ModuleNotFoundError example.packaged_settingsdjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-pi-llamacpp-256k-medium-rerun1/
notesConfirmation cell for the medium-lane PASS; did not replicate. Exited cleanly in 59.5 min with 53/53 node tests, staged the backend, but Django died booting packaged mode on a settings module the staging step never ships (ModuleNotFoundError: example.packaged_settings) - the campaign's one-pathology signature: two components disagreeing about an interface on a never-executed path.
FAILQwen3.8 27BPi / llama.cpp localthinking off, chat-template kwarg7200.00s
120m00s		387749353Smoke faildjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-pi-llamacpp-256k-off/
notesFirst local open-weight Qwen3.8 row; killed by the 7200s runner timeout. Staging failed because .stage/backend/src was not a Python project. No transcript captured (--pi-mode unset), see D-039.
FAILQwen3.8 27BPi / llama.cpp localthinking high7200.00s
120m00s		37770813653Smoke faildjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-pi-llamacpp-256k-high/
notesKilled by the 7200s runner timeout. Its own generated stage-backend.cjs threw a TypeError because it assumed fs.cpSync passes a Dirent to filter. It never ran its own packaged smoke once, and stopped editing files with 51% of the budget unspent. A post-hoc counterfactual correcting only that two-line signature made the packaged app serve, so the distance to pass was a single defect.
FAILQwen3.8 27BPi / llama.cpp localthinking off, Electron pre-warmed7200.00s
120m00s		46760813853Smoke faildjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-pi-llamacpp-256k-off-prewarmed/
notesFirst cell where stage-backend succeeded, clearing the step that killed runs A and B. It did attempt self-verification and deadlocked on its own invented desktop-dev-smoke target, which passes the smoke flag as an env var while its launcher reads argv; the packaged smoke separately hung because its exit is bound solely to did-finish-load with no failure path.
FAILQwen3.8 27BPi / llama.cpp remote CUDAreasoning_effort medium, set explicitly7200.10s
120m00s		0000not run; no electron/package.json produceddjango-resume 0.3.0/mainresults/agent-wrap-oneshot/qwen38-hetzner-pi-llamacpp-q4km-256k-medium/
notesRTX 4000 SFF Ada Q4_K_M fit test with inference on the Hetzner GEX44 and the unchanged Pi harness, target checkout, and verifier on the M4 Studio over an SSH local forward. All 64 model layers fit on the GPU, but exact 262144 context required an 8.5 GiB q8_0 KV cache in system RAM (--no-kv-offload). The run spent the full two-hour cap reading and planning, made 56 inspection calls but no write/edit call, and left the clean target unchanged. Completed-generation decode fell from 7.50 tok/s at 2.3K context to 2.12 tok/s at 91.7K; the final canceled prompt contained 93.3K tokens. Eleven transient client connection errors automatically retried while the server remained healthy. Treat this as a split-host CUDA inference result, not a full agent-host comparison.
INTERRUPTEDGPT-5.5Pi / OpenAI Codexthinking off1240.10s
20m40s		37825914053Smoke faildjango-resume 0.3.0/maindesktop-django-starter/.bench-qwen36/results-oneshot-gpt55-main-off/
notesStopped because raw JSON stream grew to 8.7 GB while the run remained in a self-debug loop.

Raw Report

Open raw report.md for a portable Markdown version of the indexed benchmark results.