GMKtec Says Its Halogen Engine Triples Token Speed on the EVO-X5 Pro Without Changing the Chip

GMKtec says its EVO-X5 Pro mini PC can sustain around 75 tokens per second when running Qwen3.8-Flash-Next through its Halogen inference engine, against roughly 23 tokens per second for a general-purpose engine on the same model and the same underlying hardware. In a post on its own news blog, the company frames the result as a software story rather than a silicon one, saying AMD provides the hardware and that GMKtec has not modified the silicon or changed the physical memory bandwidth.

According to GMKtec, Halogen is a specialized inference engine designed to optimize AI workloads on AMD's Strix Halo platform, and it takes a deliberately narrow approach rather than the broad compatibility of a general-purpose engine. The company says Halogen targets AMD Strix Halo using the gfx1151 GPU architecture, targets the Qwen3.8-Flash-Next model family, and uses hardware-specific kernels and model-aware execution paths. GMKtec describes the trade-off as a choice between a Swiss Army knife and a precision-engineered tool, with the specialized engine focusing its resources on the exact hardware and workload it is designed to run.

GMKtec says the first optimization is removing unnecessary abstraction. The company says that because a general-purpose engine such as llama.cpp must accommodate a wide range of GPUs and computing environments, its implementation stays flexible across architectures and execution requirements, while Halogen can develop kernels specifically for the gfx1151 architecture and tune performance-critical operations around the platform's compute resources, memory behavior and cache hierarchy. GMKtec says that reduces reliance on generic execution paths and lets small improvements compound into a meaningful increase in generation speed.

The second optimization, according to GMKtec, is execution tuned for Mixture-of-Experts models. The company says Qwen3.8-Flash-Next uses a MoE architecture with close to 180 billion total parameters, routing each input through a selected subset of specialized experts rather than activating every parameter for every token. GMKtec says expert routing, weight access and memory movement all affect performance, and that on a platform where memory bandwidth is a critical resource unnecessary data movement becomes particularly expensive. The company says Halogen is designed around the target model's execution patterns to optimize expert routing and data movement, with the goal of moving less unnecessary data and keeping the hardware working efficiently. GMKtec also says Halogen uses a 5.53 bits-per-weight quantization configuration, which it describes as compact weight storage without forcing the entire model into a uniform 4-bit representation.

GMKtec says the third technique is speculative decoding, in which a smaller draft component proposes several candidate tokens in advance and the main model verifies them in a more parallelized step, so tokens that pass verification can be accepted together. The company says the result depends on how accurately the draft component predicts the main model's output, and that for workloads with repetitive or predictable output patterns, such as code completion, structured text generation and document rewriting, the technique can be especially effective. In GMKtec's internal testing, the company says Halogen reached a peak of 94 tokens per second in repetitive-content scenarios, and it notes that this is a peak result for a specific workload rather than a universal generation speed.

GMKtec says the combined result on the EVO-X5 Pro, which it says is equipped with up to 192GB of unified memory, is around 75 tokens per second sustained generation, up to 2,000 tokens per second prompt processing and up to 94 tokens per second peak generation in repetitive-content scenarios, all from its internal testing. The company says these figures describe different stages or conditions of inference and should not be treated as directly interchangeable, and that fast prompt processing helps when submitting a long document, a large codebase or an extensive conversation history while fast sustained generation makes the response itself feel more immediate. GMKtec says actual performance varies with model settings, context length, quantization and workload, and that local inference also gives users greater control over where their data is processed.

Source: GMKtec (https://www.gmktec.com/blogs/news/same-chip-different-ai-speed-how-gmktec-evo-x5-pro-reaches-75-tokens-per-second).