Delivering 3D immersive content – such as a fast-moving football game or live concert – to individual viewers has huge commercial potential, but remains highly challenging for providers. The difficulties are linked primarily to the large volumes of high-quality data involved, differences between networks, and user terminal requirements, such as set-top boxes and routers.
The difficulties are multiplied when viewers are mobile as well as in fixed positions, scattered across different locations, and watching content on different types of devices – and also when demand is high. The challenge is exacerbated when those users wish to have a shared experience while watching something remotely at the same time.
The 'Remote Collaborative Real-Time Multimedia Experience over the Future Internet' (ROMEO) project is tackling the compression and hybrid synchronised 3D media delivery aspects of the puzzle, while seeking to prepare networks for future high-bandwidth services. Solutions in both areas could clear the way for live 3D streaming to a screen or mobile terminal near you, accompanied by spatial audio (3D sound) for a realistic experience, shared simultaneously by others with similar interests.
The viewing experience is optimised in such a way that the broadcaster will deliver the main 3D stereoscopic content (e.g. the football game viewed from one angle) while simultaneously streaming the match from other viewpoints using peer-to-peer (P2P) technology. The P2P delivery is ensured thanks to scalable and multi-description coding of the content against problems like congestion, as well as through the use of ROMEO’s multi-cast, multi-tree structure for distributing high-quality additional views.
To deliver content simultaneously and avoid long latencies (signal delays), ROMEO foresees delivering content from a master peer to other peers in the collaborative network. To keep costs down, the project is also working with equipment virtualisation to ensure peers at the periphery of the network are well served. The team is creating new software functionalities, usually deployed in the home or elsewhere, so that they can be placed on non-bespoke edge servers.
The project has so far combined DVB-T2 broadcast access network technologies with a P2P distribution system that operates over wired and wireless links, including Wi-Fi and an LTE emulator for high-speed mobile data transmission. Fixed users can receive the P2P content as well as the DVB-T2 content, whereas mobile users are served by DVB-T2, their own cellular operators and Wi-Fi hot-spot connections.
The default setting for fixed users is the delivery of complete 3D multi-view content with multiple audio object streams using a DVB-T2 network and P2P connection simultaneously. When the network detects deteriorating ‘quality of experience’ (QoE), adjustments are made within users’ terminals. Mobile users can also retrieve the DVB-T2 service at any time, but have the hot-spot service as the priority setting and switch to their own LTE service if hot-spots are not available.
An audio-visual communication overlay has been created to bring remote collaborators together so that they may enjoy the streamed 3D media together – the live streaming 3D media is received and consumed by users at the same time with no detectable latency or delay. Components of the system have been successfully tested by ROMEO. A full working demonstrator is planned to carry out final validation and assessment of all system modules.
ROMEO plans to have the tools in place to introduce European citizens – at home and on the move – to new, 3D media at an affordable price by the time the project ends in September 2014. New 3D multi-view video and spatial audio coding and optimisation techniques, dynamic ways to measure 3D video and audio quality and estimate user experience, and new MPEG compression standards will open up a new market for consuming, producing, commercialising and sharing 3D content.