IN Brief:
- EdgeRunner-Camo was developed from 1.4 million CamoGPT conversations containing 17.7 million user and model messages.
- EdgeRunner reports a 37% overall reduction in error rate across its held-out military task set.
- Domain specialisation is intended to support smaller models that can run locally at the tactical edge.
EdgeRunner AI and the US Army Artificial Intelligence Integration Center have developed a military-specific language model using historical CamoGPT interactions, reporting a 37% reduction in error across a held-out set of Army-related tasks.
EdgeRunner-Camo was trained using data drawn from 1.4 million CamoGPT conversations containing 17.7 million user and model messages. The work was carried out through a Cooperative Research and Development Agreement between EdgeRunner and the Army centre and is intended to demonstrate how internal military data can be used to create smaller, domain-specific models.
CamoGPT provides an unusually concentrated training source because its users were already asking Army-related questions. The platform was first deployed in 2024 and had reached around 75,000 users by mid-2025, giving researchers access to a much more specialised interaction history than a general-purpose public chatbot would produce.
EdgeRunner did not use the complete dataset without filtering. The development team assessed conversations for military relevance, personally identifiable information, training quality, subject category, and whether the answer depended on retrieval or tools unavailable to a standalone model. Non-military conversations, PII, and responses dependent on unavailable external information were removed before subsequent processing.
The remaining material covered personnel and organisation, logistics and supply, training and doctrine, defence policy and strategy, weapons and platforms, and tactics and operations. Positive-feedback conversations were withheld to form the test set, allowing the team to compare the new model with answers that CamoGPT users had previously marked favourably.
That evaluation method has limitations that matter when reading the reported 37% result. The researchers assume that an answer receiving positive user feedback can act as the reference answer, and a separate judge model then determines whether EdgeRunner-Camo’s response sufficiently matches it. The benchmark therefore measures improvement against a curated set of historical interactions rather than establishing that every resulting answer is objectively correct.
Performance also varied significantly between categories. EdgeRunner reports a 16% reduction in error for logistics and supply and 57% for training and doctrine, with the overall held-out test set producing the 37% figure. Those differences are more informative than the headline average because military users will encounter task types with different levels of ambiguity and factual sensitivity.
The model was trained using on-policy distillation. EdgeRunner used Gemma 4 E4B as the student model and Gemma 4 31B as the teacher, with the smaller model generating candidate answers and receiving training signals derived from the teacher model’s probability distribution. The team evaluated progress every 1,000 steps and selected the 4,000-step checkpoint for the full test.
All data processing and model training for the project was conducted in an IL5-certified government cloud environment. That requirement reflects the sensitivity of the source material even where the resulting model is intended for wider deployment, because internal military interactions can contain controlled information that should not be mixed casually with a commercial development environment.
The engineering objective is not simply improved answer quality. EdgeRunner argues that military domain specialisation can reduce model size by an order of magnitude or more while retaining useful task performance, lowering the compute and memory burden required to run AI locally.
That has a direct deployment consequence. A large frontier model normally assumes access to substantial server infrastructure, while tactical users may be operating on laptops, rugged computers, or edge devices with limited power and intermittent or unavailable connectivity. A smaller model capable of running locally can continue functioning where cloud access is impractical.
Local deployment also changes the security architecture. Prompts and documents do not necessarily have to leave the user’s device for inference, but the model weights, retrieval data, software packages, and update process become part of the fielded system. Defence users still need controls around access, logging, configuration, model updates, and validation.
Smaller models do not eliminate hallucination or uncertainty either. The reported error-rate reduction is comparative, and EdgeRunner describes the work as an initial experiment. Planned development includes additional supervised fine-tuning, on-policy distillation, grounded optimisation, preference optimisation, tool calling, and agentic retrieval.
The project nevertheless illustrates a route from internal AI adoption to purpose-built model development. An organisation that has accumulated millions of real user interactions can use them to identify recurring task types, create evaluation material, and train a model around its own workflows rather than relying entirely on generic training data.
That approach will require careful governance as more defence organisations build their own datasets. Historical user feedback can contain errors or biases, while sensitive data need to be separated from material suitable for broader training. The quality of the resulting model will depend on those filtering and evaluation processes as much as on the number of conversations available.
EdgeRunner-Camo now provides a measurable Army-specific case study rather than a claim that local AI has solved the wider problem. Its next test is deployment: whether the performance seen in a controlled held-out benchmark survives on constrained tactical hardware, with real users, imperfect data access, and the operational pressures that make edge computing attractive in the first place.
