Why does Social Brothers choose WebRTC?

WebRTC and HoloLens

Social Brothers uses WebRTC for Bureau Veritas' remote assistance application. Bureau Veritas inspects ships all over the world. That is, their inspectors sometimes fly all over the world for a ten-minute inspection. Not only does this cost the inspector time and money, shipping can also miss out on millions. After all, the ship must remain stationary until the inspector is on site.

At Social Brothers we have been working with the HoloLens for a while. We have built an application for Bureau Veritas ourselves to meet this issue of Bureau Veritas and to enable remote inspections. The idea behind our remote Assistanceapplication is that there is someone on the ship wearing the HoloLens. The Bureau Veritas inspector sits remotely behind his own desk and can monitor the ship via the camera image of the HoloLens and communicate with the wearer of the HoloLens. For more information about remote assistance, read our blog about it.

In addition, the application has the option to exchange audio and to exchange extra data, such as the position of the carrier of the HoloLens in the engine room of the ship. This allows the remote inspection to be performed faster and more efficiently. WebRTC turned out to be the best technology for the communication protocol, which is why we started using it for this proof-of-concept. WebRTC makes it possible to connect two devices in a very simple way and to get a stable audio and video connection.

Fast connection

WebRTC is a simple way to stably connect two (or more) devices. Both devices let the server know that they want to connect. They exchange P2P data (so-called handshake) and then the connection is started. This is separate from the server: it is only used for the initial contact. Sending custom data back and forth between the two devices is made possible by the RTCDataChannel API. The technology behind this is available as a regular JavaScript API. For various examples about the possibilities you can look here.

The WebRTC protocol provides a fast connection. If the server is the connection, one sends some to the server and the other gets it. That is an extra step that causes delays. Compare it to a conversation: it is easier to talk directly to the person to whom you want to convey a specific message, than it is to have a conversation through an intermediary. By skipping the extra step in the digital communication process, P2P has up to six times faster connection times. The video quality is better and you have more natural online conversations because there is no delay.

Secure connection

Because there is no server in between, WebRTC is in principle also extra safe, because it is more difficult for a man-in-the-middle attack to take place that captures certain communications. Of course, you have to trust the person you connect with. We therefore verify via a websocket technique whether you are connecting with the right person. When you connect to the server, socket.io creates a room from the server. That room will be given a separate name, so you can only join if you know the name of the room. That way you know who's on the other side.

Data channels at WebRTC

Another advantage of the WebRTC protocol is the data channels. These make it possible to send other data back and forth besides speech and images. For the Bureau Veritas application, this is the position of the HoloLens in the engine room. The inspector has a 3D model of the engine room on his computer screen, next to the video image from the HoloLens. The position and rotation of the HoloLens is constantly transmitted to the inspector through the data channel. We use that data to ensure that the 3D model shows exactly the position of the HoloLens wearer.

The inspector sees both the 3D model and the video. The video is important for an inspection of a dent, but the 3D model is useful for the overall picture. This is much less jerky than the camera image that suffers, as it were, from the movements of the wearer and the limited network speed of the HoloLens.

Flexibility

webRTC's data channels are very flexible: you can send everything and format it the way you want. This gives us the opportunity to keep the data as small as possible. This is important given the slow network on the ship. Another advantage of webRTC is that the system itself determines the video quality based on the connection. Furthermore, through the handshake we can ensure that we exchange the correct information and no more than that. The inspector should see the video of the HoloLens wearer, but not the other way around. By smartly sending the right data, you can make better use of the ship's limited bandwidth.

Want to know more?

If you want to stay informed of developments in the IT field, sign up for our newsletter or take a look at our other blogs† We also warmly recommend us complete handbook on Augmented Reality .

Do you want to know more about webRTC and the possibilities to use it for remote assistance, please contact contact up with Social Brothers.