Super easy to use ARM embedded system AI framework

OPEN AI LAB was established in December 2016 and was initiated by ARM China United Industry Partner. It is committed to promoting deep cooperation in the entire industry chain of chips, hardware, algorithms and software, and accelerates the deployment of artificial intelligence industrial applications and the expansion of application scenarios. Tengine is a lightweight, modular, high-performance neural network inference engine optimized for ARM embedded devices. Perfect support for ARM platform, support for ARM Cortex CPU, ARM Mali GPU, ARM DLA, third-party DSP, is compatible with "Almighty King".

Developers can use Tengine to exhaust the mainstream framework model and embedded operating system, as well as heterogeneous scheduling of various hardware in the platform, making full use of hardware computing power. In addition, Tengine provides common AI application algorithms, including image detection, face recognition, and speech recognition. If you don't understand AI, you can run AI applications. Tengine also supports a variety of common convolutional neural networks, including SqueezeNet, MobileNet, AlexNet, ResNet, etc., support layer fusion, 8-bit quantization and other optimization strategies. And the performance of the Arm CPU is fully exploited by calling the HCL library optimized for different CPU microarchitectures.

Tengine的“发动机”——HCL计算库

Some people say that NCNN is a high-performance neural network forward computing framework optimized for mobile phones. From the beginning of design, the deployment and use of mobile phones are deeply considered. In the past, NCNN did, but there was no Tengine at the time.

Now, as the plug-in for Tengine, the HCL Compute Library is the fastest ARM CPU NN computing library and supports dynamic 8-bit quantization. The accuracy is almost the same, which is 2-3 times faster than NCNN. Memory usage is reduced by a third.

*Tengine performance data A72, A53 performance based on RK3399 platform test, A17 performance based on RK3288 platform test

NCNN data is September 7, 2018

HCL INT8 quantitative calculation accuracy change

* Using the ILSVRC2012_VAL data set to test 5000 images to get the quantitative accuracy test results are as follows

    As an AI application framework, Tengine not only provides an inference API interface for upper-layer applications, but also supports mainstream models and formats. It also defines hardware interfaces in the lower layers, adapts various computing modules of embedded platforms, and supports multiple AI application algorithms and calls at the same time. Calculation module calculation. Therefore, the launch of Tengine has a milestone significance for the AI open source ecosystem. With Tengine, application and algorithm vendors can get rid of complex embedded platform adaptation work, and use HCL to directly exploit the full potential of hardware; with Tengine, the chip factory can stand on the shoulder of Arm embedded AI ecosystem, the fastest The speed of the chip is brought to market and deployed in edge AI applications. As the two major tools developed by OPEN AI LAB for edge AI application deployment, Tengine+HCL will help the embedded artificial intelligence industry to accelerate forward and contribute to the ultimate realization of all things.