You can get a lot of innovation from running machine learning inference on mobile devices, but what if you could too? train your models on mobile devices? What would you invent if you could refine your models at the edge of the network?
With version 2.1 of the Qualcomm Adreno OpenCL ML SDK, you can get started with new mobile applications that use both inference and training. With the SDK, you can now fine-tune and improve your existing models at the edge of the network, so you don’t have to travel back and forth to the cloud and back. In addition to using a model that has already been trained, you now have the option to refine by updating the weights through training strides.
It’s a new way to bring the computing power to the data instead of commuting the data to and from the computing power.
What can you do with mobile training?
When we released OpenCL ML SDK 1.1, we took advantage of the acceleration built into the OpenCL driver in our Android image for faster inference on our Adreno GPU. SDK 2.1 now uses accelerated operations in the OpenCL driver for the inference, back-propagation, and weight update phases of training.
Of course, this is different from the resource-intensive training you would provide on a high-performance workstation or in the cloud. It assumes that you start with a trained model and that the training on the device updates the model, rather than training it from scratch.
If you’re already using OpenCL for inference on mobile, then training on mobile is the logical next step on your development path. You can implement it at the network edge in several use cases:
Federal Learning – Let’s say your mobile app collects sensitive data like photos, and for security you want to keep the data out of the network as much as possible. How can you keep your model up to date without constantly sending all new data to the cloud for training? With Open CL ML SDK 2.1 and the Adreno GPU, you can instead run federated learning to run the training loops locally and collect updates to the model parameters. When it suits you, you can send those updates only to the cloud.
Transferring learning – In this use case, you start with a cloud-trained model like MobileNet. Then you update the model by adjusting the weights for specific layers through edge training. This enables transfer of model knowledge between data domains.
Personalization – This is similar to transferring learning and can apply to use cases such as video conferencing. For example, imagine you want a model that can identify the participant who is speaking at a particular moment, so you can blur their background. You achieve this through live updates of weights of specific layers during the video conference. Under the Hood – Tensor Batch 1 Approach and Back Propagation
Training consists of a loop that takes an entry, performs a forward stride, calculates the loss, performs a backward stride, and updates the weights. The loop continues until the loss begins to decrease to an acceptable level. OpenCL ML SDK 1.1 has the forward pass enabled, which is the inference part of machine learning. Now SDK 2.1 enables the backward stride, with accelerated operations for the back propagation and weight update phases of training.
Of course, trying to train on a mobile device with typical batch sizes of 32 or more tensors would put a load on the memory on the device. Therefore, the operations in SDK 2.1 have a tensor batch size of 1. The smaller memory footprint of the Tensor Batch 1 approach is well suited for mobile.
The diagram below illustrates the training flow with this approach.
Gradients are collected in each iteration of the minibatch and the weights are updated at the end of the minibatch. That is mathematically equivalent to training with a tensor batch size equal to the mini batch size.
The training is done using a 32-bit floating point data type. All tensor values and arithmetic are floating point. For models with batch normalization layers, you should keep the statistics from the model’s original training. However, trainable batch normalization parameters, such as scale and bias, can be updated.
In future versions of OpenCL ML, we expect the training options to expand beyond tensor batch size 1.
Download OpenCL ML SDK 2.1 now!
Remember how impressed you were when machine learning inference at the network edge was finally possible? This is your chance to join the training on the fringe early on.
Here’s what you’ll find in the SDK:
Specification for the OpenCL extension, cl_qcom_ml_ops, which introduces a new set of CL API calls, data structures, and tokens for specifying and queuing machine learning operations Open CL ML developer’s guide to model training Sample applications to show how to use the SDK can use The Generate Graph Model Tool to convert TensorFlow protobuf frozen models (.pb) or TensorFlowLite (.tflite) models into a TensorFlow Graph Model representation The Graph Model to QFP16/32 Tool to extract the weight tensor as .qfp16 and .qfp32 file types
We think many of you will find SDK 2.1 useful for fine-tuning on the edge. Download OpenCL ML SDK 2.1 now and let us know your usage scenarios and suggestions for improvement.
Qualcomm Adreno is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.
Qualcomm Inc. this content published on December 13, 2021 and is solely responsible for the information contained therein. Distributed by Public, unedited and unaltered, on December 13, 2021 19:05:10 UTC.
Reveal now 2021
All news about QUALCOMM, INC. Analyst Recommendations About QUALCOMM, INC. Turnover 2022 39 486 M – – Net profit 2022 10 588 M – – Net cash 2022 3 885 M – – P/E ratio 2022 19.8x Return 2022 1.48 % Capitalization 206 B 206 B – EV / Sale 2022 5.12x EV / Sales 2023 4.63x Number of employees 45 000 Free-Float 99.9% Duration : Auto.2 months3 months6 months9 months1 year2 years5 years10 yearsMax. Period: DayWeek
Trends in Technical Analysis QUALCOMM, INC.
Short TermMedium TermTrendsBullishBullishBullish
Income statement Evolution
Average Consensus BUY Number of Analysts 32 Last Close Price $183.88 Average Price Target $198.14 Spread / Average Price Target 7.76%