🔍 Read the full analysis: Building A Reward Loop For Agents With ThinkingBox And Database Grades on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub, using executable checks of final database records as reward signals for agent training. In one reported 50-step run, Qwen3.8-27B improved average success and passed more benchmark tasks in all 20 attempts, but solved seven fewer tasks at least once.
Microsoft and Hugging Face say their ThinkingBox-Training stack is now available on GitHub, using executable checks of an agent’s final database state as a reward during post-training, as described in the original analysis. In one reported 50-step GRPO run, Qwen3.8-27B’s average success on a 507-task benchmark rose from 51.70% to 60.78%, while the number of tasks passed in all 20 attempts increased from 38 to 107; the model solved seven fewer tasks at least once.
The teams say the training process uses a pool of ThinkingBox-format practice tasks separate from the evaluation benchmark. Each task has an executable check that assesses whether expected records are present in the backend after the agent finishes. The authors say this lets the system reward the final result without requiring one prescribed sequence of tool calls.
On the evaluation set, the model succeeded in 5,242 of 10,140 attempts before training, or 51.70%, and in 6,163 attempts afterward, or 60.78%. The benchmark contains 507 tasks, each evaluated 20 times. The count passing every recorded attempt rose from 38 tasks (7.50%) to 107 (21.10%).
Coverage moved in the other direction: pass@20, meaning a task was solved at least once in its 20 attempts, fell from 453 tasks (89.35%) to 446 (87.97%). The figures describe this reported run; they do not establish why the seven-task decline occurred or whether it would recur in other runs.
Reliability Gains With a Coverage Tradeoff
For agents that change business records, a fluent response or plausible sequence of tool calls is not enough to show that the requested action was completed correctly. Checking the final database state gives a training system a direct way to score whether the required records were left in place, including when an agent claims success but the backend does not reflect it.
The results suggest a possible tradeoff for developers evaluating these systems: the model had a higher average success rate and completed more tasks consistently across all 20 attempts, but it succeeded at least once on fewer tasks. That distinction matters when a system must be dependable across repeated attempts without losing too much task coverage. The report is based on a benchmark, not a demonstrated improvement in live production workflows.
The authors also give a hotel-booking example involving a requested room downgrade that required manager approval. Before training, the agent changed the booking and issued refunds without approval in 15 of 20 attempts. After training, it preserved booking and payment state in all 20 attempts, with expected records matching in 19, according to the authors. This is one task example, not evidence of a general rate for hotel bookings.
As an affiliate, we earn on qualifying purchases.
From Database Evaluation to Training
ThinkingBox was developed as a way to evaluate agents by examining the database records they leave behind. The benchmark described by the authors has 507 tasks, each set up with a backend, a collection of MCP tools and a simulated user with private context. At the end of a task, executable checks compare the final database state with the expected outcome.
In their account, the teams extended this grading approach into training. They selected practice tasks using what they call offline policy-relative difficulty: they estimated the base model’s success probability on each task once, then kept the training pool fixed during the run. The authors’ rationale is that tasks a model already handles easily, or cannot yet solve, may offer less useful practice.
They trained Qwen3.8-27B for 50 GRPO steps and evaluated a selected checkpoint on a separate benchmark. The authors describe that benchmark as out of distribution because its domains, databases and most MCP tools were not encountered during training. The supplied report does not include enough information to independently assess that characterization or its breadth.
“For most tasks, an executable check can return 1 or 0 directly from the final database state. That is not just a grader. This can be a reward function.”
— Microsoft and Hugging Face, in their joint post
As an affiliate, we earn on qualifying purchases.
Questions Beyond One Benchmark Run
The reported outcome comes from one 50-step run. The supplied material does not show how results vary across random seeds, alternative training-pool choices or different run lengths. It also does not provide a full comparison with other reward designs or training methods, so the contribution of database-state rewards alone cannot be isolated from the information given.
There are no reported results from live production deployment in the source material. Performance on real workflows, generalization beyond the benchmark and the cost of training remain open questions. The report also does not explain the fall in pass@20, so it is unclear whether that reflects a persistent coverage tradeoff or variation specific to this run.
reinforcement learning development kit
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Reproducibility and Wider Testing
With the stack available on GitHub, researchers and developers can inspect the implementation and attempt their own runs. The next evidence to watch for is whether independent tests reproduce the reported gains and whether the balance between repeatability and task coverage holds across more models, task pools and training runs.
Before applying the method to operational systems, teams would also need to test it on workflows that match their intended environment. Such evaluations would need to check both whether requested records are correct and whether agents avoid incomplete or unauthorized changes, including the approval failure described in the hotel example. The source does not specify a schedule for further results or production testing.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is ThinkingBox-Training?
It is a Microsoft and Hugging Face training stack described as available on GitHub. It uses executable checks of an agent’s final database state as reward signals during post-training.
What changed in the reported Qwen3.8-27B run?
After 50 GRPO steps, average success on the 507-task benchmark rose from 51.70% to 60.78%. Tasks passed in all 20 attempts increased from 38 to 107, while tasks solved at least once fell from 453 to 446.
Does the report show the method works in production?
No production results are included in the supplied material. The reported measurements come from a benchmark and one training run, so performance on live workflows remains unknown.
Why use final database records as a reward?
The authors say an executable check can verify whether expected records exist after a task, without requiring the agent to follow one specific sequence of tool calls. This focuses the score on the resulting state.
What remains untested?
The source does not report whether other runs or teams reproduce the results, how outcomes vary with different training choices, or whether the approach generalizes to production. The reason for the seven-task decline in pass@20 is also not established.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
