Hunyuan-A13B released

Posted by kristaller486@reddit | LocalLLaMA | View on Reddit | 144 comments

From HF repo:

Model Introduction

With the rapid advancement of artificial intelligence technology, large language models (LLMs) have achieved remarkable progress in natural language processing, computer vision, and scientific tasks. However, as model scales continue to expand, optimizing resource consumption while maintaining high performance has become a critical challenge. To address this, we have explored Mixture of Experts (MoE) architectures. The newly introduced Hunyuan-A13B model features a total of 80 billion parameters with 13 billion active parameters. It not only delivers high-performance results but also achieves optimal resource efficiency, successfully balancing computational power and resource utilization.

Key Features and Advantages

Compact yet Powerful: With only 13 billion active parameters (out of a total of 80 billion), the model delivers competitive performance on a wide range of benchmark tasks, rivaling much larger models.

Hybrid Inference Support: Supports both fast and slow thinking modes, allowing users to flexibly choose according to their needs.

Ultra-Long Context Understanding: Natively supports a 256K context window, maintaining stable performance on long-text tasks.

Enhanced Agent Capabilities: Optimized for agent tasks, achieving leading results on benchmarks such as BFCL-v3 and τ-Bench.

Efficient Inference: Utilizes Grouped Query Attention (GQA) and supports multiple quantization formats, enabling highly efficient inference.