Memdoor Workflow: Multi-Agent Qwen3.8-27B Models Outperform Fable 5
A five-agent Memdoor harness achieves superior coding results on LiveCodeBench Hard benchmarks.
1Multi-Agent Programming Framework
Memdoor implements a multi-agent workflow using five Qwen3.8-27B models acting as planners, ideators, solvers, and managers. These agents operate from fresh contexts, communicating exclusively through files on disk rather than chat history. This architecture replicates the GVS5H method, enabling specialized roles to tackle complex programming tasks with high precision.
2Benchmarking Against Fable 5
Rebuilt with exact author prompts, the Memdoor workflow achieved a mean score of 9.8 out of 10 across the first ten LiveCodeBench Hard problems. This performance surpasses Fable 5, which scored 8.0 in the same evaluation. The results demonstrate that multi-agent coordination significantly enhances model output on difficult coding challenges.
3Operational Cost and Efficiency
The evaluation involved 50 runs, costing approximately $35.79 for 15.3 million output tokens. Each run took a median of 40 minutes, with the longest session lasting four hours. By utilizing OpenRouter across various hosts like Modal and CoreWeave, the system maintained consistent performance despite occasional mid-run host switching requirements.
4Workflow Execution Mechanics
The workflow utilizes a DAG structure defined in YAML files, where each step requires specific inputs and produces outputs. Agents follow a strict ledger discipline, ensuring that ideators and solvers work from a shared state. This method eliminates human intervention, relying on automated shell scripts to verify test success.
5Technical Configuration Parity
The system maintains strict parity with the original GVS5H harness, including temperature settings of 0.3 for planning and 0.2 for management. It utilizes a 128K reply cap and specific top_p/top_k configurations. These parameters ensure that the model's reasoning remains consistent with the authors' original research findings during the execution.
6Addressing Model Contamination
The contests used for benchmarking date from March 29 to April 6, 2025. Because the model may have encountered these problems during its training phase, contamination cannot be definitively ruled out. However, the harness relies on hidden tests to validate solutions, providing a more rigorous assessment than public sample evaluations.
7Workflow Integration and Deployment
Memdoor allows users to deploy workflows by cloning the repository and placing task files into a local directory. The system supports various backends, including local models via Ollama or vLLM, and cloud providers like OpenRouter. Users define tasks through task.md files, enabling rapid iteration and testing of custom programming problems.
8Future Optimization Potential
Despite high success rates, two specific gaps remain in the current loop. These issues are fixable by implementing brute-force checks against small inputs or retaining the last program that passed samples. Addressing these minor logic flaws could further improve the reliability of the multi-agent system without requiring fundamental method changes.