The Stories Behind OPPO’s Latest Imaging Technologies

Written by the OPPO Research Institute

Coffee, colleagues, and creation – these are the three words that Howard Zhang, head of the OPPO Japan Research Center, would use to describe the Center’s early days. Back in 2017, when OPPO’s Yokohama office was just starting out, Zhang often used to find himself working in a nearby café when he needed some caffeine-induced inspiration, particularly in the early morning as the clean table tops and smell of fresh coffee helped to sharpen his senses.

“The OPPO Japan research center was built from scratch, so there was plenty of decisions to be made,” said Zhang. “Me, Jun Luo and Numata Hajime – the other two original team members – the three of us could always been found in coffee shops preparing for work, and just as importantly, meeting other cool people who wanted to create cool tech with us.”

From these humble beginnings, the OPPO Japan Research Center has kept this youthful spirit alive while managing to grow at a bewildering speed. Today, four years later, the Center is the source of many of OPPO’s industry-leading innovations – technologies that eventually find their way to the four corners of the globe where they help users capture the beauty of life with ease.

OPPO Japan Research Center

From day to night
2019 was a big year for tech, most noticeably with the introduction of 5G to worldwide markets.

Thanks to its ultra-fast data speeds, the fifth-generation communication technology is seen as crucial technology for ushering in new entertainment and communication mediums, for example short video. Recognizing the implications of this, and knowing that short video would soon replace photo as the must-share content on social media, the OPPO Japan Research Center was quick to begin its preliminary research into what would eventually become OPPO’s Ultra Night Video algorithm.

This didn’t come without its challenges, however. Capturing video on smartphone is a significantly more complex process than capturing photos, let alone doing so in dark, nighttime scenes. There’s also the fact that providing real-time previews for video capture requires much higher processing power.

Perfecting a smartphone function or algorithm in lab conditions is a much easier job than guaranteeing its ability to perform in the diverse, multifaceted scenarios we come across in our everyday lives. As could be expected then, the team wasn’t short of challenges to overcome. In one such case, the team found that the prototype device being used to implement the Ultra Night Video algorithm wasn’t ready, leading them to conduct further research in partnership with other OPPO R&D teams from around the world.

The team is testing the Ultra Night Video algorithm

Progress was quickly disrupted again as COVID-19 restrictions prevented the team from working together in the lab. Under these more complicated working conditions, the team had to discuss and adjust the algorithm online. With many late-night sessions, the team found plenty of opportunity to test the algorithm in real-world conditions.

The end result of this day-to-night research is an Ultra Night Video algorithm capable of recording every possible moment in a user’s life, creating brilliant images that make full use of light and shadow.

The algorithm achieves this by combining multiple image frames to reduce noise and optimize picture quality, raising the signal-to-noise ratio to nearly 3Db as a result. By increasing light intake by 74.4% and by improving image purity by 33.7% in extreme low-light conditions of 5 lux, Ultra Night Video is able to record steady, high-quality nighttime videos at 1080P@30fps. Ultra Night Video was first implemented on the OPPO Reno5, where it was integrated into the AI Highlight Video feature. When using AI Highlight Video, if the algorithm detects that the user is trying to record in a nighttime environment, it will automatically apply Ultra Night Video to deliver the best video quality. So, whether the user is taking pictures on a winter evening walk, or they are at a party in a restaurant celebrating and recording unforgettable moments – Ultra Night Video has every situation covered.

(Left) Standard mode (Right) AI Highlight Video

This year, the OPPO Japan Research Center has further updated AI Highlight Video for the Reno6 series. When shooting at night, the new AI Highlight Video on Reno6 Pro can effectively improve video clarity by up to 40.2% versus Reno5 Pro, while color and brightness have all improved, resulting in a clearer, brighter, and more vividly colored portrait videos.

Stand in the spotlight
As short video sharing apps have become such a global hit, more and more people are no longer satisfied with just watching videos – they want to be part of conversation themselves. This inevitably leads them to ask: what do I need in order to create a good video or vlog?

First off, the subject in the video should be clear, in a way that allows their unique personality to standout no matter where they are. This insight was the inspiration behind the OPPO Japan Research Center developing the Hair-level Portrait Matting algorithm used in OPPO’s Bokeh Flare Portrait mode. The high precision algorithm ensures that even the subject’s hair is carefully separated from the background, providing more creative space for users to express themselves in.

Researchers are sharing inspirations

The team was heavily inspired by the cinematic bokeh effect often seen in movies. Previously, to achieve this type of effect, you would have needed professional equipment and even specialist post-production. But the OPPO Japan research center was able to use its expertise in AI to create a similar effect, eventually bringing it to the Bokeh Flare Portrait on the Reno6 Series.

With one simple click, the OPPO Reno6 starts using deep learning to conduct a real-time, frame-by-frame semantic understanding of the portrait subjects. This allows it to accurately distinguish and segment the subjects and the background in a 360-degree environment during both day and night. Even if the subject is turning, moving, facing side on or facing away from the camera, the precise Hair-level Portrait Matting algorithm can always find them. Through this, a bokeh-style effect can be applied to the background with natural and smooth transitions between the background and subject layers, cresting a truly cinematic bokeh effect in real-time. 

Perfect separation of subject and background taken by the OPPO Reno6

This year, the OPPO Reno6 Series became the first smartphone series to achieve movie-quality bokeh using computational photography. Maybe, you’re a budding photographer or a soon-to-be social media star. With Bokeh Flare Portrait, every user has the power to tell their stories with more character and individuality.

Chasing the next frontier
Building on the momentum of its previous achievements, the OPPO Japan Research Center is now chasing the next frontier of imaging technology.

Allowing users to take long-range shots that remain sharp and clear has been a long-term trend in the development of smartphone imaging technology, and it’s a development that OPPO has been heavily involved in itself. The OPPO Japan Research Center is now exploring more advanced Continuous Optical Zoom technology that combines hardware innovations and optimized algorithms to deliver a camera zoom system comparable to those found on professional cameras. Such technology will allow users to take images of whatever they want to capture, no matter how far away it is. The Continuous Optical Zoom technology may even eventually allow the multi-camera zoom setups currently used in smartphones to be replaced by a single camera system in the future.

OPPO Japan Research Center is working on combination of hardware innovations and optimized algorithms

OPPO has long believed that, when it comes to smartphone imaging systems, hardware guarantees the lower limits of image quality, but the upper limits can only be reached through innovation in advanced algorithms. For the OPPO Japan Research Center, this means figuring out how the smart use of deep learning, computational vision, and other advanced processing techniques can be used to remove the need for complex and cumbersome hardware, delivering the same professional image quality with greater portability and ease of use.

Beyond imaging technology, the team is also working to develop new application scenarios in areas such as IoT, digital twins, and human-machine interaction in ways that allow all users to further unleash their imagination.

Small improvements lead to big breakthroughs

It’s another beautiful morning in Yokohama. As Zhang steps into the bright office with his signature cup of coffee in time for the morning’s meeting, he looks around and sees that his hopes for the team have come true: this cool team has indeed created cool tech.

Researchers are working in the bright office

Along with the Japan Research Center, each member of OPPO’s 10,000-strong R&D staff – wherever they are in the world – knows that OPPO’s goal is to make small improvements every day. OPPO research centers in the U.S., Shenzhen, and Beijing are currently dedicated to exploring exciting new technologies like 5G, AI, IoT, and big data, to help develop OPPO’s multi-portal smart device ecosystem.

Over time, these seemingly small advances are what drives OPPO’s technological breakthroughs. The common thread tying them all together is OPPO’s aim to create, develop, and refine technologies that make the lives of its users easier, every day.

About OPPO R&D
Driven by both user needs and cutting-edge technologies, OPPO has been building an ever-growing R&D team with more than 10,000 researchers worldwide. OPPO currently runs six Research Institutes in Beijing, Shanghai, Shenzhen, Dongguan, Silicon Valley (United States), and Yokohama (Japan), as well as five R&D Centers located in Shenzhen, Dongguan, Chengdu, Xi’an, and Hyderabad (India). Through its global research network, OPPO strives to create the best innovations for its users around the world.