With features like multi-view video, free-view video, and VR, today's video market is rich with opportunities for carriers.
By Wang Zhiyuan & Zhang Xiaodong, Huawei
With more than 365 million connected 5G devices in China and 960,000 5G base stations, China is the world's largest 5G market. While 5G users generate twice as much data as 4G users, they use similar services to 4G users.
That means to increase the 5G user base, the industry must create new 5G services that demonstrate different, improved experiences.
So, do differentiated new service scenarios already exist?
Dynamic visuals for new free and immersive experiences
Huawei believes that new-concept video, which brings a differentiated experience to users through dynamic visuals, such as free-viewpoint video, multi-view video, and VR, is set to experience explosive development in the 5G era.
Research shows that human vision can perceive information in four dimensions: space, color, shape, and moving objects. Improving quality in each of these four dimensions can significantly improve our visual experiences. For example, upgrading resolution from HD to 4K allows viewers to see the shape of an object more clearly, increasing the color gamut from BT.709 to BT.2020 yields far richer colors, and upgrading frames per second from 25 to 50 allows users to enjoy higher-quality dynamic visuals.
Similarly, if video perception quality is improved in terms of dynamic visuals, users will gain a better sense of space and a more immersive experience. To distinguish new-concept video from other types of video, China's National Radio and Television Administration has included multi-view video and free-viewpoint video within the scope of new-concept video.
On May 11, 2019, the Sound of Music Gala adopted a large-scale, multi-view live broadcasting model for the first time ever (see Fig 1). Huawei worked with a carrier, 4K Garden, and South New Media to simultaneously deliver users videos from multiple viewing angles, giving a better view of performers as users could freely switch angles.
Fig 1: Multi-view video from the Sound of Music Gala
In terms of application, multi-view video overhauls the traditional model where viewing angles are changed by a director, instead delivering a more personalized experience.
As more than four channels of video need to be transmitted at once, the network bandwidth required by multi-view video is more than four times higher than regular video, while the typical bit rate exceeds 30 Mbit/s. The average download speed of 4G networks in China is about 23 Mbit/s. So, if users watch multi-view video on a 4G network, freezing is very likely. However, if they watch the video on 5G networks with a downlink bandwidth of more than 1 Gbit/s, the viewing experience is extremely smooth.
In mobile scenarios, multi-view video depends heavily on the transmission capabilities of 5G networks.
Free-viewpoint video is another form of new-concept video service, which is more advanced than multi-view video. On November 29, 2019, China Unicom, Huawei, and zhibo.tv ran a live broadcast of the 2019 ITTF Men's World Cup for table tennis, using a free-viewpoint model for the first time. Multiple cameras onsite collected data in real time, allowing users to watch the matches by seamlessly switching between multiple viewing angles, including front, side, and 45 degrees. Without missing a single moment of the match, viewers could freely change their view.
The typical bit rate of free-viewpoint video exceeds 30 Mbit/s. Much like multi-view video, free-viewpoint video requires the high bandwidth of 5G networks to provide the desired service experience.
Scenarios such as free-viewpoint video and multi-view video generally need to carry information from more dimensions than regular video. For example, multi-view video requires data to be transmitted from more than four viewing angles, while free-viewpoint video requires the transmission of redundant data that is used to rotate viewing angles.
This is typical of new-concept video. It delivers differentiated experiences, but the transmission of more information means higher network requirements, which 4G networks struggle to meet.
So what progress has been made?
Industry recognition and popularity
Since the first large-scale adoption of multi-view video during the live broadcast of the Sound of Music Gala, joint efforts have been made to promote the service, which has gradually become widely accepted by the industry.
On August 4 and August 17, 2019, Tencent Interactive Entertainment partnered with Huawei and a carrier to adopt a multi-view model during the live broadcast of the King of City competitions during the 7th Honor of Kings World Champion Cup. E-sports provide rich content, both individual and team, and particularly suit multi-view video for an excellent user experience.
On October 1, 2019, during the live broadcast of the gala celebrating the 70th anniversary of the founding of the People's Republic of China, a number of companies including China Central Television (CCTV), Migu Video, and Kuaishou, worked together to introduce multi-view video technology. They provided various angles, including from the air, ground, far away, and close up, delivering a full and personalized viewing experience.
On December 22, 2019, China's National Center for the Performance Arts teamed up with Huawei to apply multi-view video technology at a ceremony celebrating the 12th anniversary of its founding. Viewers could watch the live stream from multiple angles, and focus on the singer, conductor, choir, or pianist.
Huawei has also adopted multi-view video technology to serve the live broadcasts of many other customers such as the CBA preseason games broadcast by Qingdao TV Station and China Unicom; the "Yellow River into the Sea" symphony concert broadcast by Shandong Radio and Television; iQiyi's Shout out for Love Gala; and the New Year's Concert of Guangzhou Opera House.
Telecom carriers and Huawei are not the only ones looking to drive the wide adoption of multi-view video technology. TV stations, cable networks, opera houses, and Internet OTT players are also all enthusiastic about this technology. It can be used for concerts, sports and e-sports competitions, and other major events. We believe that multi-view video technology will be applied in many more domains in the future, bringing more personalized viewing experiences to audiences and overhauling the traditional viewing model.
With the increasing popularity of multi-view video in industries, one major technical pain point is the difference between encoders and network transmission. This often makes it impossible to synchronize the different viewing angles, especially in scenarios like concerts and sports.
To solve this issue, Huawei has introduced frame synchronization technology in its video 3.0+ solution. Multiple channels of video can be synchronized and displayed on a user's device at a frame-level error of below 40 ms, delivering a smooth viewing experience when users switch viewing angles.
With the large-scale rollout of 5G, carriers are playing a key role in promoting the commercial use of free-viewpoint video.
China Unicom and Huawei have worked together to deliver a free-viewpoint live broadcast of two table tennis world championships. China Telecom and Huawei partnered to live broadcast the boxing event Fight to the End (see Fig 2). Migu Video introduced free-viewpoint video during its live broadcast of the CBA All-Star Weekend. Carriers such as China Mobile Zhejiang, China Unicom Henan, and China Telecom Shaanxi have also collaborated with Huawei to explore free-viewpoint video applications.
Fig 2: Free-viewpoint live broadcast of Fight to the End
Dong Dawu, vice president of China Unicom Video Technology Company, highlighted carriers' enthusiasm to promote free-viewpoint video during a speech. "With the commercial rollout and provisioning of large-bandwidth, low-latency 5G services, users expect 5G to meet their demand for new experiences that 3G and 4G networks cannot deliver such as ultra-HD video, free-viewpoint video, and multi-view video," he said. "Carriers must closely keep up with user demand and create better user experiences."
Alongside carriers, TV stations and other media outlets are also interested in applying free-viewpoint video. Hunan TV's Dance Smash 2 is a fantastic example. On October 10, 2020, Dance Smash 2 premiered with the special show "Storm Moment". A 360-degree video precisely showed the posture and movements of any contestant from any angle, helping judges to score.
To deliver the best possible viewing experience, Huawei and Hunan TV developed a "space-time condensation system" based on the free-viewpoint model. Powered by AI algorithms, the system supports intelligent fast-focus, the "butterfly" lift shooting array, zoom in and out, and multi-focus, which delivered amazing 3D visual effects for the first time in the industry.
In addition to Hunan TV, other Chinese TV stations have introduced free-viewpoint video in their programs, including Guangdong TV’s Stunning Acrobatics, Chongqing TV's and Shandong TVs spring festival galas, and CCTV's Year of the Ox and Lantern Festival galas. As well as entertainment shows, Huawei collaborated with CCTV to combine skeleton tracking, virtual viewing angles, and AI algorithms with free-viewpoint technology at China's 14th National Games held this year. This delivered new and smart experiences to the audience watching the games (see Fig 3). We believe that free-viewpoint video will be applied at even more major sporting events.
Fig 3: Sprinting speeds during China's 14th National Games 100m final
TV stations are incredibly enthusiastic about introducing free-viewpoint video production systems, and we believe that these production systems will become essential tools for TV stations and other media outlets in the future.
To adapt to the latest industry trends, Huawei has promoted the wide adoption of new-concept video and applied its expertise to the standardized product Huawei Envision MediaCube. Highly integrated and simple to use, the system supports features such as fast focus, space-time condensation, dynamic flow, intelligent multi-focus, virtual viewing angles, 8K field of view (FOV), and frame synchronization. The system also supports the one-stop processing of free-viewpoint video, multi-view video, and 8K VR video, making front-end content production of new-concept video more efficient and convenient.
Industry collaboration over the past two years has driven new-concept video services into the large-scale application phase. Boosted by 5G, these services are expected to ramp up industry productivity and cover more users.
Collaboration models
Partners involved in the free-viewpoint video production system aim to achieve different goals: TV stations want to improve the quality of their programs, while carriers hope to deliver differentiated experiences via the 5G network.
These goals aren’t in conflict. For example, for Dance Smash 2, the space-time condensation system was used to produce TV clips for live satellite broadcasts, as well as video clips that allow users to freely change viewing angle. Video clips from the show have been uploaded to the MangoTV app, which are marked and can provide users with an even more extensive viewing experience (see Fig 4). In the near future, carriers will provide featured content like these video clips for users on both large and small screen channels to meet diverse viewing needs.
This will allow free-viewpoint videos produced by TV stations and other media outlets to serve multiple purposes. They will not only be used as part of TV shows, but also meet carriers' requirements to deliver a differentiated service experience through 5G networks.
Performing arts, sports, and other types of entertainment companies have taken a similar approach. For example, to make breakthroughs in the post-pandemic era, China National Opera & Dance Drama Theater went online, partnering with Huawei to shoot and produce new-concept video for offline performances and broadcast them featuring free-viewpoint video, multi-view video, and 8K VR video. The videos are available through other channels such as HUAWEI Video and carriers' 5G smartphone, IPTV, and VR services.
To motivate industry players to produce more new-concept videos, carriers are taking measures to shape the market.
For example, China Mobile Zhejiang has started offering new-concept video as featured content for SVIP 5G users, including in 5G packages, and offering rewards for partners that provide featured content through 5G networks.
Such initiatives have motivated content providers to deliver more exciting and differentiated 5G content, which in turn drives carriers to develop robust 5G services and provide better returns for content providers, creating a model of shared success through collaboration.
In July 2021, 10 departments within the Chinese government, including the Ministry of Industry and Information Technology, Cyberspace Administration of China, National Development and Reform Commission, Ministry of Education, Ministry of Finance, and Ministry of Culture and Tourism, jointly released the Set Sail Action Plan for 5G Applications (2021–2023). The plan pledges support for deploying new-concept video in typical scenarios such as cloud-based broadcasting and viewing online games where 5G is available.
With the support of policies and efforts of all industry players, we believe that new-concept video will experience explosive growth in the 5G era.
