This course introduces the hardware circuit design and basic environment set up, as well as provides some simple development examples and some basic Deep learning examples.

Milk-V Duo is an ultra-compact embedded development platform based on CV1800B. It has small size and comprehensive functionality, it is equipped with dual cores and can run linux and rtos systems separately, and has various connectable peripherals.

• Scalability: The Milk-V Duo core board has various interfaces such as GPIO, I2C, UART, SDIO1, SPI, ADC, PWM, etc.
• Diverse connectable peripherals: The Milk-V Duo core board can be expanded with various devices such as LED, portable screens, cameras, WIFI and so on.

Course features:

• The content materials are rich and complete, including development board hardware design, peripheral interface instructions, basic environment set up method, and sample code scripts.
• The learning path is scientifically reasonable, starting from the introduction and basic usage of the development board, and then leading to pratical projects to fully utilize the development board and provide reference for users' own development.
• The pratical projects are rich, and the course provides many examples of practical code usage and function demonstrations. Different functions can be implemented by simply modifying and combining the code.

欢迎加入大模型课程！本课程将带你深入了解什么是大模型，并帮助你掌握应用这些强大模型的技能。无论你是对深度学习领域感兴趣，还是想在实际项目中应用大模型，本课程都将为你提供宝贵的知识和实践经验。

大模型是指具有巨大参数量和复杂结构的深度学习模型。这些模型在处理大规模数据集和复杂任务时表现出色，如图像识别、自然语言处理、语音识别等。大模型的兴起引发了深度学习领域的巨大变革，并在许多领域取得了突破性的成果。

在本课程中，你将学习大模型的基本概念和原理。我们将详细介绍LLM（Large Language Models）的基础理论、发展历程、常用的大模型和不断发展的Prompt和In-context learning技术。随着课程的深入，我们将进行大模型实战应用。你将学习到如何部署Stable Diffusion和ChatGLM2-6B等备受关注的大模型到算能自研的最新一代深度学习处理器BM1684X。SOPHON BM1684X, 是算能面向深度学习领域推出的第四代张量处理器，算力可达32TOPS，支持32路高清硬解码和12路高清硬编码，可用于深度学习、机器视觉、高性能计算等环境。

无论你是想在学术界深入研究大模型，还是在工业界应用这些技术，本课程都将为你提供坚实的基础和实践能力。准备好迎接大模型的挑战了吗？让我们一起探索这个令人着迷的领域吧！

This course introduces the hardware circuit design and peripheral resource utilization methods of Shaolin Pi, as well as provides tutorials on using the hardware acceleration interface of Deep learning and some basic Deep learning examples.

"Shaolin Pi" is a development platform based on BM1684 with about 20 TOPS computing power. It has good hardware scalability based on the Mini-PCIe interface, a rich ecosystem, and various connectable peripherals.

• Scalability: The Mini-PCIe of the "Shaolin Pi" core board can be converted into various interfaces such as WiFi, 4G, Bluetooth, GPIO, M2 interface, USB, RJ45, SATA, SFP, HDMI, and CAN.
• Diverse connectable peripherals: The "Shaolin Pi" core board can be expanded with various devices such as portable screens, keyboards, mice, cameras, headphones, and VR. Users can DIY a full-scenario Linux workstation on the "Shaolin school" and practice various Deep learning experiments to their heart's content.

Course features:

1. The content materials are rich and complete, including development board hardware design, peripheral interface instructions, development board upgrade process, and sample code scripts.

2. The learning path is scientifically reasonable, starting from the introduction and basic usage of the development board, deepening the understanding of the development details through the learning of the internal system architecture and code, and finally leading to practical projects to fully utilize the development board and provide reference for users' own development.

3. The practical projects are rich, and the course provides many examples of practical code usage and function demonstrations. Different functions can be implemented by simply modifying and combining the code.

Code download link: https://github.com/sophgo/sophpi-shaolin

Note: The model conversion part can refer to the SE5 development series courses.

This course introduces the hardware circuit design and peripheral resource operation methods of the CV1812H development board from the "Huashan Pi" series. It also provides tutorials on using Deep learning hardware acceleration interfaces and some basic Deep learning examples.

Huashan Pi (CV1812H development board) is an open-source ecological development board jointly launched by TPU processor and its ecological partners. It provides an open-source development environment based on RISC-V and implements functions based on vision and Deep learning scenarios. The processor integrates the second-generation self-developed deep learning tensor processor (TPU), self-developed intelligent image processing engine (Smart ISP), hardware-level high-security data protection architecture (Security), speech processing engine, and H.264/265 intelligent encoding and decoding technology. It also has a matching multimedia software platform and IVE hardware acceleration interface, making Deep learning deployment and execution more efficient, fast, and convenient. The mainstream deep learning frameworks, such as Caffe, Pytorch, ONNX, MXNet, and TensorFlow (Lite), can be easily ported to the platform.

Course Features

1. Rich and complete content materials, including hardware design of the development board, SDK usage documents, platform development guides, and sample code scripts.

2. Scientific and reasonable learning path. The course introduces the development board and basic routines, and then delves into the internal system architecture and code learning to understand the development details. Finally, practical projects are introduced to fully utilize the development board, which can also serve as a reference for users to develop on their own. 

3. Suitable for different audiences. For users who want to quickly use the development functions, the course provides many code samples for use and function display, which can be easily modified and combined to achieve different functions. For enthusiasts or developers in related industries, the course also provides detailed SDK development usage guidelines and code sample analysis documents, which can help users to gain in-depth understanding. 

4. long-term maintenance of the course. In the future, we will launch more development courses to communicate with developers and grow together.

Course contents

Link to the open-source code for the Huashan Pi development board：https://github.com/sophgo/sophpi-huashan.git

The deep neural network model can be trained and tested quickly and then deployed by the industry to effectively perform tasks in the real world. Deploying such systems on small-sized, low-power Deep learning edge computing platforms is highly favored by the industry. This course takes a practice-driven approach to lead you to intuitively learn, practice, and master the knowledge and technology of deep neural networks.The deep neural network model can be trained and tested quickly and then deployed by the industry to effectively perform tasks in the real world. Deploying such systems on small-sized, low-power Deep learning edge computing platforms is highly favored by the industry. This course takes a practice-driven approach to lead you to intuitively learn, practice, and master the knowledge and technology of deep neural networks.

The SOPHON Deep learning microserver SE5 is a high-performance, low-power edge computing product equipped with the third-generation TPU processor BM1684 developed independently by SOPHGO. With an INT8 computing power of up to 17.6 TOPS, it supports 32 channels of Full HD video hardware decoding and 2 channels of encoding. This course will quickly guide you through the powerful features of the SE5 server.  Through this course, you can understand the basics of Deep learning and master its basic applications.

Course Features

1. One-stop service 

All common problems encountered in SE5 applications can be found here.

 • Provide a full-stack solution for Deep learning micro servers

 • Break down the development process step by step, in detail and clearly

 • Support all mainstream frameworks, easy to use products

2. Systematic teaching 

It includes everything from setting up the environment, developing applications, converting models, and deploying products, as well as having a mirrored practical environment.

• How is the environment built? 

• How is the model compiled? 

• How is the application developed? 

• How are scenarios deployed?

3. Complete materials

The course includes video tutorials, document guides, code scripts, and other comprehensive materials. 

• Rich video materials 

• Detailed application guidance 

• Clear code scripts 

Code download link: https://github.com/sophon-ai-algo/examples

4. Free cloud development resources 

Online free application for using SE5-16 microserver cloud testing space 

• SE5-16 microserver cloud testing space can be used for online development and testing, supporting user data retention and export 

• SE5-16 microserver cloud testing space has the same resource performance as the physical machine environment 

Cloud platform application link: https://account.sophgo.com/sign_in?service=https://cloud.sophgo.com&locale=zh-CN

Cloud platform usage instructions: https://cloud.sophgo.com/tpu.pdf

There are many types of intelligent robots, and the most widely used ones are wheeled mobile robots, mainly used for indoor or warehouse patrol, planet exploration, teaching, scientific research, and civilian transportation. In this course, the intelligent car obtains video information through the built-in camera (visual sensor), recognizes the surrounding environment, and realizes autonomous navigation and obstacle avoidance in a small space based on sensors such as lidar and inertial measurement unit (IMU). This course takes a practical approach to guide you to intuitively learn robot operating system (ROS) and use Shaolin Pi development board to build an intelligent car vision application platform. Through programming the intelligent car in practical exercises, you will master the basic knowledge and application of Deep learning.

The Shaolin Pi development board is a high-performance, low-power edge computing product equipped with the third-generation TPU processor BM1684 independently developed by Algorithmic Ability, with INT8 computing power of up to 17.6 TOPS. It supports hardware decoding of 32 full HD videos and encoding of 2 channels. The Shaolin Pi development board has flexible peripheral configuration, supporting 3 mini-PCIe and 4 USB interfaces, as well as DC power supply and Type-C power supply. According to the needs of different scenarios, the board can achieve optimal configuration, reasonable cost, optimal energy consumption, and optimal function selection. This course will help you quickly master the powerful features of the Shaolin Pi development board. Through this course, you will not only be able to master the basics of the Robot Operating System (ROS) and Deep learning, but also understand the basic applications of Deep learning.

Course Features

1. One-stop Service

All common issues related to KT001 intelligent car can be found here.

• Provides a full-stack solution for KT001 intelligent car.
• Comprehensively explains the basic concepts and practical applications of ROS.
• With practical application as the core, it explains a large number of computer vision case studies, such as image processing based on OpenCV, object detection based on YOLOv5, multi-object tracking based on DeepSort, face detection based on RetinaFace and face recognition based on ResNet, as well as the implementation principles and methods of action recognition based on TSM.

2. Systematic Teaching

From product introduction to environment building, and then to visual application.

• What is the composition of the intelligent car?
• How is the intelligent car assembled?
• How is the environment built?
• How is the application developed?

3. Complete Materials

The course includes video tutorials, document guides, code scripts, etc., which are detailed and rich.

• Abundant video materials.
• Detailed application guidance.
• Clear code scripts.

Code download link: https://github.com/sophgo/sophon_robot

Course Catalogue

Multimedia, commonly understood as the combination of "multi" and "media," refers to the integration of media forms such as text, sound, images, and videos. In recent years, there has been a surge in emerging multimedia applications and services, such as 4K ultra-high-definition, VR, holographic projection, and 5G live streaming.

Multimedia and Artificial Intelligence

Deep Learning is based on multimedia technologies, such as image processing and recognition, audio processing and speech recognition, and so on. This course is based on the BM1684 Deep learning processor, which has a peak performance of 17.6 TOPS INT8 and 2.2 TFTOPS FP32, and supports 32-channel HD hardware decoding. It demonstrates the core capabilities of a processor: computing power + multimedia processing power.

Key Technologies and Indicators for Intelligent Multimedia

Key technologies include coding and decoding technology, image processing technology, and media communication technology. Key indicators include the number of decoding channels, frame rate, resolution, level of richness of the image processing interface, latency, and protocol support.

This course will focus on introducing the three aspects of image processing technology, coding and decoding technology, and media communication technology. Through a combination of theory and practice, students will learn about intelligent multimedia related theories for artificial intelligence and quickly master basic practical methods.

Related GitHub links

sophgo_ffmpeg: https://github.com/sophgo/sophon_ffmpeg

sophgo_opencv: https://github.com/sophgo/sophon_opencv

Deep learning compilers act as a bridge between frameworks and hardware, achieving the goal of developing code once and reusing various computational processors. Recently, Altran has also open-sourced its self-developed TPU compilation tool—TPU-MLIR (Multi-Level Intermediate Representation). TPU-MLIR is an open-source project that focuses on Deep learning processor TPU compilers. The project provides a complete toolchain that converts pre-trained neural networks of various frameworks into binary files (bmodel) that can be efficiently operated in TPU, to achieve more efficient inference. This course is driven by practical exercises and aims to lead everyone to intuitively understand, practice, and master the SOPHON Deep learning processor TPU compiler framework.

The current TPU-MLIR project has been applied to the latest generation of artificial intelligence processor BM1684X developed by SOPHON, which, together with the processor's high-performance ARM core and corresponding SDK, can achieve rapid deployment of deep learning algorithms. The course content will cover the basic syntax of MLIR and implementation details of various optimization operations in the compiler, such as graph optimization, int8 quantization, operator splitting, and address allocation.

Compared to other compilation tools, TPU-MLIR has the following advantages:

1. Simple and convenient

Users can quickly get started by reading the development manual and included examples to understand the model conversion process and principles. TPU-MLIR is designed based on the current mainstream compiler tool library MLIR, and users can also use it to learn the application of MLIR. The project has provided a complete toolchain, and users can directly complete the model conversion work quickly through the existing interface, without the need to adapt to different networks themselves.

2. Universal

TPU-MLIR currently supports TFLite and ONNX formats, and these two formats of models can be directly converted into bmodels that TPU can use. What if it is not one of these two formats? In fact, ONNX provides a set of conversion tools that can convert models written in mainstream deep learning frameworks on the market to ONNX format, and then continue to convert them into bmodel.

3. Precision and efficiency coexist

During the model conversion process, there may be precision loss. TPU-MLIR supports INT8 symmetric and asymmetric quantization, which greatly improves performance while combining the original development company's Calibration and Tune technologies to ensure high precision of the model. Not only that, TPU-MLIR also uses a large number of graph optimization and operator splitting optimization technologies to ensure efficient operation of the model.

4. Achieving ultimate cost-effectiveness and creating the next generation of Deep learning compilers

To support graphic computing, each operator in the neural network model needs to develop a graphic version; to adapt to TPU, each operator should have a TPU version. In addition, some scenarios require adapting products of different models of the same computational processor, and each time they need to be manually compiled, which will be very time-consuming. Deep learning compilers aim to solve the above problems. TPU-MLIR's series of automatic optimization tools can save a lot of manual optimization time, enabling models developed on the RISC-V to be smoothly and free of charge ported to TPU to obtain the best performance and price ratio.

5. Comprehensive information

The course includes Chinese and English video teaching, document guidance, code scripts, etc., with abundant video materials, detailed application guidance, and clear code scripts. TPU-MLIR stands on the shoulders of MLIR giants to create it, and now all the code of the entire project has been open-sourced and made available to all users for free.

Code Download Link: https://github.com/sophgo/tpu-mlir

TPU-MLIR Development Reference Manual: https://tpumlir.org/docs/developer_manual/01_introduction.html

The Overall Design Ideas Paper: https://arxiv.org/abs/2210.15016

Video Tutorials: https://space.bilibili.com/1829795304/channel/collectiondetail?sid=734875"