With the rapid development of artificial intelligence technology, large models have shown great application potential in many fields. Especially in the automotive-related industry, the trend of intelligence and digitalization is becoming increasingly obvious, and the demand for Chinese LLMs is also increasing. Recently, CTTL issued the "Multi-modal LLM Capability Certificate" to the "Autohome Cangjie Multi-modal LLM" of Autohome Intelligent Internet (Beijing) Technology Co., Ltd., marking the official implementation of the evaluation system of the large model capability of the automotive service platform and providing a new evaluation standard for the application of AI technology in the automotive industry.

In the current rapid development of the automobile industry, the advancement of intelligent driving, Internet of Vehicles and other technologies has brought consumers a new car purchase and use experience. However, despite the promising application of large models in these fields, the existing benchmarks do not adequately cover the needs of the automotive consumer sector. The evaluation standards of general capabilities can no longer meet the evaluation needs of large model capabilities in the fields of automobile sales platforms, and there is a lack of targeted evaluation systems.

In order to fill this gap, the CTTL formulated FG-Z18-0007-02 "Detection Method for LLM of Automobile Service Platform", and carried out a comprehensive performance test of the "Autohome Cangjie Multi-modal LLM" based on this method. The detection method not only covers the general capabilities of large models, but also includes multiple vertical domain dimensions such as domain capabilities, multimodal capabilities, and security and compliance, ensuring the comprehensiveness and scientificity of the evaluation. The "Autohome Cangjie Multi-modal Large Model" performed well in this evaluation, which met the requirements of the "Multi-modal Large Model Capability" of the automotive service platform formulated by the CTTL. This achievement not only proves the application value of the large model in the field of automotive services, but also demonstrates the strength of Automotive Intelligence Internet (Beijing) Technology Co., Ltd. in AI technology research and development.

As an authoritative testing organization, CTTL will continue to improve the evaluation system and scheme of the large model of the automotive service platform in the future. By focusing on industry cooperation, we will jointly formulate more comprehensive and scientific dedicated scenario evaluation standards to accurately measure the performance of large models in the industry. At the same time, the CTTL will continue to launch and update more evaluation programs, screen out excellent products for the industry in a professional and fair evaluation way, and guide the entire industry to develop in a healthy and sustainable direction.

CTTL sincerely invites more automotive service companies to participate in the research of special standards and industry benchmarking, and jointly contribute to the sustainable development of China's automotive service AI industry. Through such cooperation, we can ensure that the development of large-scale model technology is more in line with the actual needs of the industry, and at the same time, it can also promote the technological progress and innovation of the entire automotive service industry.

Consultation contact:

Mr. Zhang

zhangxiaoyu@caict.ac.cn