Korea split AI inference across GPU and NPU. Benchmark the handoff, not the brochure

THE SIGNAL IN ONE SENTENCE

Two chips are being asked to share one answer. SK Telecom, NVIDIA and South Korean AI-chip company Rebellions announced a collaboration in Seoul on November 9 to develop and validate a rack-scale system that connects NVIDIA infrastructure with Rebellions' next-generation Rebel R200 neural processing units. The planned design uses NVIDIA NVLink Fusion and the MGX rack architecture. SK Telecom intends to deploy the resulting systems in its AI data center and test whether they can support its own models and public-facing services. The interesting idea is not merely that a GPU and an NPU will occupy the same rack. It is that they may split the work of producing a response. The companies describe a disaggregated inference design. NVIDIA GPUs would handle prefill, the stage where a model processes the user's prompt and builds the initial internal state. Rebellions NPUs would handle decode, the stage where the model generates the response token by token. That division makes intuitive engineering sense. Prefill can demand a large burst of parallel computation, especially when the prompt is long. Decode repeatedly moves model state and produces relatively small pieces of output. Different processor designs may be better at those different shapes of work. May be is doing important work in that sentence. The announcement is a development plan, not a field result. Rebellions is still developing the R200. NVIDIA says R200 systems are expected to be ready for deployment in 2028. The companies have not published an end-to-end benchmark for this GPU and NPU combination, a production service-level result, an energy measurement or a failure-recovery study. So the useful question is not whether both chips are fast on their favorite tasks. The useful question is whether the service remains fast, efficient and correct after one chip hands the work to the other. That handoff is where a tidy architecture diagram meets the mildly vindictive physics of a data center. The scheduler must send each request to the right prefill worker. The system must preserve the model state needed for decode. Data may need to cross an interconnect. The decode worker must be available, compatible and ready at the right moment. If one side is overloaded, the other side can sit idle. If the handoff stalls, a spectacular prefill score will not save the person watching a cursor blink. The benchmark therefore has to begin at the request and end at the final token. Start with time to first token. That measures how long a user waits before the response begins. Then measure time per output token, total response time and throughput under realistic concurrency. Publish the median, but also publish the 95th and 99th percentiles. Averages have a charming habit of hiding the queue that ruins Monday morning. Test short prompts, long prompts, short answers and long answers. Test single users and crowded services. Test the actual Korean and multilingual workloads the system is supposed to serve. Test mixture-of-experts models if those are part of the promise. Record model size, context length, batch size, precision, software versions, rack configuration, power limits and cooling conditions. Then measure the handoff itself. How long does it take from the end of prefill on the GPU to the start of decode on the NPU? How much data moves? Does the scheduler copy state, rebuild it or keep it accessible through shared infrastructure? What happens when the destination is busy? How often does a request wait, retry, fall back or fail? Which metrics distinguish an accelerator problem from a network, memory, scheduler or software problem? The public announcements do not answer those implementation questions. That is fine at the beginning of a development program. It is not fine at procurement time. Energy needs the same discipline. The companies aim to improve power efficiency, and specialized inference hardware can be valuable in a data center where electricity, cooling and rack capacity are real constraints. But a chip-level efficiency claim is not the same as an efficient service. Count the whole request: both accelerators, interconnects, host processors, memory, networking, idle capacity and cooling. Report joules per completed request and joules per generated token, alongside latency and quality. If the design saves accelerator power but requires more duplicate capacity, extra transfers or a complicated fallback pool, the rack may tell a less flattering story than the component sheet. Correctness belongs in the benchmark too. Moving a model between processor types should not quietly change its output quality beyond the accepted numerical tolerance. Compare the disaggregated system with a reference implementation on the same model, prompts, decoding settings and evaluation set. Check structured output, tool calls, Korean language performance, long-context behavior and safety policies, not just general text similarity. For public services, the failure test matters as much as the speed test. Pull an NPU worker out of service. Saturate the handoff. Delay the interconnect. Restart a scheduler. Feed a request that the NPU runtime does not support. Verify whether the system rejects, queues, reroutes or resumes the request. Make sure a retry does not duplicate a tool call or transaction. Preserve a trace that shows which processor handled each stage and why a fallback occurred. This is where South Korea's sovereign AI goal gets practical. The project connects a Korean-designed inference processor, SK Telecom's data-center operations and models, and NVIDIA's rack and interconnect ecosystem. That can expand the market for domestic AI silicon and give Korean operators more choices inside a global infrastructure stack. It is not technological independence, and the announcement does not claim that it is. The design still depends on NVIDIA technology, shared rack standards, software compatibility, supply chains and the successful delivery of Rebellions' future hardware. Sovereignty here is better understood as greater domestic capability and bargaining power, not a magical severing of every dependency. That makes the dependency map part of the product. Operators should know which components, drivers, runtimes, compilers, interconnects and management tools come from which supplier. They should know what can be replaced, what data leaves the system, which failures require vendor support and how long a service can keep running if one dependency becomes unavailable. The collaboration is worth watching because inference is becoming the expensive, repetitive part of putting AI into ordinary services. If South Korea can pair domestic accelerators with an established rack ecosystem and prove better economics on real workloads, it could create a credible path for local chips to reach production. But the proof is not the memorandum. It is the receipt from the rack. Publish the workload. Publish the handoff delay. Publish the tail latency, completed-request energy, quality comparison, failure behavior, fallback path and total cost. Let public and private buyers see what each component contributed and what the seam took away. Two processors can share an answer. The benchmark has to show whether the user benefits from the arrangement.