It looks like it’s peak season for chip announcements, and Oppo just unveiled its first piece of custom silicon at its Inno Day 2021 — dubbed MariSilicon X. This isn’t a full-fledged SoC like Qualcomm’s Snapdragon 8 Gen 1 or Google Tensor, but rather a custom image signal processor. , or imaging NPU, infused with Oppo’s machine learning smarts.
According to Oppo, algorithms are leading the way for innovations in mobile photography, but the computational capabilities of SoC machine learning are currently the bottleneck for better images. Google would probably agree, after dropping Qualcomm’s Snapdragon in favor of its own custom silicon. As such, the MariSilicon X is built to power Oppo’s AI imaging algorithms for stills and up to 4K 30fps HDR video.
Also see: The best camera phones money can buy
If you’re looking for some numbers, the MariSilicon X features 18 TOPS (trillion operations per second) of int8 or 72 TOPS of int4 number crunching performance, as well as an energy efficiency of 11.6 TOPS per watt. On its own, TOPS is a rather meaningless statistic because it doesn’t tell you anything about the workload. However, Oppo also gave us a real example. The AI noise reduction algorithm on the Snapdragon 888 powered Oppo Find X3 Pro runs at just 2 fps and requires 1,693 mW of power. The same algorithm on MariSilicon X runs at 40fps and consumes 797mW.
That’s 20x the performance and less than half the power, at least in this example. All very impressive sounding, but what does this ISP actually do differently?
Inside MariSilicon X
Ryan-Thomas Shaw / Android Authority
At a lower level, the 6nm MariSilicon X combines traditional image signal processing (ISP), a neural processing unit (NPU), and memory components into a single chip built specifically for AI imaging.
MariSilicon X provides a 20-bit HDR image processing pipeline that can run AI algorithms directly on RAW data rather than later in the compression pipeline. Oppo states that this results in a remarkable improvement in 8dB signal-to-noise (SNR) versus processing after RGB and YuV encoding.
MariSilicon X moves AI processing from the lossy domain to the lossless RAW domain.
Interestingly, the chipset also offers separate RGB and white pixel data channels for RGBW sensors, as it was shown by Oppo earlier this year. Again, this results in an SNR improvement of 8.6 dB versus merging this sensor-level pixel data for delivery through a single pipeline. You can think of this as a bit like the days of separate RGB and monochrome cameras, but housed in a single sensor. Oppo calls this RGBW Pro mode and also claims that using this technique it is a 1.7x improvement in texture quality.
In a nutshell, Oppo’s ISP applies machine learning image and video processing to RAW data directly from the image sensor, resulting in less noise for more beautiful photos. This massive throughput is made possible in part thanks to a special memory subsystem in the MariSilicon X. In other words, data doesn’t have to leave the chip. Oppo has developed a custom memory architecture that is essentially a fast cache for the NPU. The NPU is also coupled with its own dedicated external DDR memory with a bandwidth of 8.5 GB/s for lower latency in loading and saving compared to traditional system memory. This also ensures that the bandwidth is not consumed by other smartphone applications.
The net result is a powerful, energy-efficient image and machine learning processor built to extend the broader capabilities of a smartphone SoC with unique computational photography and videography.
New Use Cases for the Oppo Find Series
Robert Triggs / Android Authority
Ultimately, device usage scenarios matter most. Oppo notes that MariSilicon X will power advanced noise reduction capabilities, enhanced nighttime snaps and 4K videography, real-time live previews and 4K ultra-dynamic range video.
As we identified earlier in the year, custom image processors with AI smarts are quickly becoming a major smartphone trend, especially in the Chinese market. Vivo and Xiaomi already have their own custom ISPs, while Google Tensor takes things a step further with a strong emphasis on machine learning via a semi-custom full SoC. The common thread is that photography is an important part of modern smartphones and is ripe for differentiation and unique selling points. Especially when it comes to advanced machine learning algorithms. Advance in the space of the photography algorithm and a brand could just launch a must-have smartphone.
Read more: Why custom image processors are the next battleground for mobile photography
We’ll wait to get our hands on a handset packing MariSilicon X before jumping to conclusions. The chip will debut in the next-gen Oppo Find flagship smartphone slated for Q1 2022.