aiDAPTIV ^TM

更快的 AI 推理性能与更大规模的 LLM 训练，全程在本地部署（On-Prem）私有环境中完成

Dynamic MoE

Run a more capable model locally

A larger MoE model may deliver better results on the target task without activating all of its parameters for every token.

What MoE means

A Mixture of Experts, or MoE, model replaces some dense model layers with a bank of specialized expert networks. In a dense model, every parameter in a layer participates in every token’s computation. In an MoE model, a router evaluates the current token and selects only a small number of relevant experts, often called Top-K experts. Only those selected experts perform computation for that token. This lets an MoE model have far more total parameter capacity than it activates at any one moment. Depending on the model and workload, that may improve coding, reasoning, tool use, multilingual performance, or domain-specific results without requiring the active compute of an equivalently sized dense model.

Dynamic MoE reduces the memory footprint

Keep the full model in aiDAPTIV Cache Memory and swap only the experts each token needs into unified memory,

Traditional approach

Without aiDAPTIV, the whole AI model must load
into unified memory.

aiDAPTIV approach

aiDAPTIV offloads the full MoE model to cache and swaps only the needed experts into memory.

Run the same MoE model in roughly half the unified memory – larger models become practical on AI PCs.

The catch? Sparse compute does not mean small memory.

Only a few experts may compute for a given token, but the full expert pool still needs to be available somewhere.

A model with hundreds of billions of total parameters may activate only a small fraction per token. Its active compute requirement can be manageable. Its full parameter footprint may still exceed available GPU or unified memory.

That is the MoE memory problem.

Make larger MoE models practical on local systems

In supported integrations, Pascari aiDAPTIV™ keeps active experts near compute while extending the available expert pool across GPU memory, system memory, and cache memory.

This can make a larger sparse model viable on a client system, workstation, edge deployment, or server that could not otherwise hold the full expert pool in GPU or unified memory.

Locality decides much of the experience

Some tasks repeatedly use a similar set of experts. When that happens, more of the required expert working set can remain close to compute.

Other tasks jump rapidly across domains, languages, coding styles, tools, or reasoning modes. That can require more different experts over a shorter period, which increases movement through the memory system.

This is the central performance tradeoff.

Questions to ask before using dynamic MoE

Does the task repeatedly use a similar expert working set?

Is larger model capability worth occasional expert-loading delays?

How consistent does token-to-token latency need to be?

Does the target runtime support the required expert-management behavior?

Interactive AI is not automatically excluded. Suitability depends on expert locality, platform configuration, and the latency consistency the experience requires.

Disclaimer: When the router selects an expert already resident near compute, the model can continue with minimal additional data movement. When it selects an expert that must be brought forward from system memory or cache memory, computation waits for that expert to arrive.

The outcome depends on platform memory bandwidth, cache memory throughput, PCIe or interconnect bandwidth, model architecture, router behavior, expert locality, prompt diversity, and runtime implementation. A larger MoE is worth using only when its added capability matters to the task. aiDAPTIV expands model capacity; it does not guarantee the latency of a fully GPU-resident model.

aiDAPTIV TM