Recently, the research consortium coordinated by the German Association for Artificial Intelligence officially released a new open-source large language model - Soofi S30B-A3B. This achievement not only marks a significant progress in Europe's sovereign AI infrastructure construction, but also injects fresh blood into the open-source model field that pursues high performance and efficiency.

The model has a clever architecture design. Soofi S adopts a resource-efficient "Mixture of Experts" architecture, combining Mamba-2 with standard attention layers. It has a total parameter count of 31.6 billion, but during actual generation, each token activates about 3.2 billion parameters. This design significantly reduces its computational cost, making it closer to small models, while showing excellent processing speed and generation throughput under high load scenarios. Especially when handling long texts, its performance advantages are particularly evident - under a context of 40,000 tokens, its generation throughput reaches about 8 times that of traditional dense models of the same scale, and it maintains extremely high stability even as the context increases.

image.png

In terms of training data strategy, Soofi S shows a clear preference for German. During the 27 trillion token training process, the research team intentionally increased the proportion of German data. From 7.2% in the first stage to 15.3% in the second stage, and integrated multi-source corpora including newspaper articles, Wikipedia, and technical documents, enabling it to perform outstandingly in dealing with complex tasks in German environments.

Benchmark test results prove its leading strength. Soofi S stands out among open-source models in comprehensive tests in English and German, surpassing well-known models such as OLMo332B and Apertus70B. Especially in code generation, professional reasoning, and German regional knowledge tests, Soofi S demonstrates top-level performance. However, the research team also objectively pointed out that there is still room for improvement in the model's extraction tasks for ultra-long contexts and German mathematical reasoning.

image.png

The implementation of this project relies on solid infrastructure support. The entire training process was completed on the Deutsche Telekom Industrial AI Cloud in Munich, using 512 Nvidia B200 GPUs. As part of the European IPCEI-CIS project, the model is entirely powered by renewable energy, and the model weights, training evaluation code, and detailed data lists will be gradually opened source. This move not only practices the concept of open and transparent R&D, but also provides a reliable technological foundation for European industrial application testing. With the release of this model, the research team is actively seeking industry partners to jointly explore its application prospects in areas such as technical document processing and code generation.