arXiv 2412.19437
DeepSeek-V3 Technical Report
By DeepSeek-AI, Aixin Liu, et al.
Published 2024-12-27
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2. Furthermore, DeepSeek-V3 pioneers an auxiliary-loss-free strategy for load balancing and set…