Browsed by
Tag: WebRTC

Google Meet and Noise Canceling Solution with Artificial Intelligence

Google Meet and Noise Canceling Solution with Artificial Intelligence

Because of the pandemic period, you know that web video conferencing systems have become very popular with working from home. Although Zoom is dominating the market, Google is also one of those who want to increase it’s market share. It aims to add new features to its platform day by day. Thanks to noise canceling feature recently added to Google Meet, Google seems to have made an innovation in this area.

In April 2020, Google announced that Meet’s noise canceling feature is available for G Suite Enterprise and G Suite Enterprise for Education. We should also point out that the father of the idea of this feature is G Suite Product Management Director Serge Lachapelle. Serge Lachapelle has worked on video conferencing technologies for 25 years, (13 of which are on Google) and he is quite experienced.

Beginning of the Project

Basically, this project starts with acquisition of Limes Audio in January 2017. The main idea arises from the difficulties experienced in the meetings held with the participants in different time zones (sounds of children and pets of home workers, breakfast sounds, etc.).

How to Prevent Noise in Google Meet?

Maybe there are those who use it, some headphones and smartphones have noise canceling mechanisms that use multiple microphones. This method basically works by extracting the sound signal received from far end microphone than the sound signal received from main microphone. The feature offered in Google Meet is implemented using a completely cloud-based infrastructure and machine learning, independent of user device.

A machine learning model (denoiser) needs to be trained to find out what is speech and what is not speech, to understand the difference between noise and speech, and then to filter only speech. Serge and his team use their own meetings to train the model, then the algorithm is matured by Youtube videos which includes many people and then manual verification methods. Ultimately, the system can intelligently filter background distractions such as dog barking, pen clicking and much more.

As you can see in the video below, while talking, Serge shows how the feature works by making noise with things like a nut bag, a pen and a glass-spoon. Non-routine noises are heard loud as soon as they start, but over time these noises are damped:

Google Meet Noise Cancellation Demo

As you can appreciate, your voice needs to be listened by Google in order to use a noise canceling system with artificial intelligence. The voice encrypted by the user is decoded and analyzed in Google data centers, and the filtered voice is also encrypted and transmitted to the users. Although the analysis of the sound by listening is a question mark on the part of the user, it is stated that this analysis is made only within denoiser. I think that it is very important to realize these transactions in a short time, especially for real-time communications.

To activate this feature, just click the three dots on the bottom right during the meeting and activate Noise cancellation from Settings:

Google Meet Noise Cancellation Ayarı
Google Meet Noise Cancellation Settings

Last Words

The method of processing data in the center (in the cloud), which is a method that Google loves, and leaving the user side more lean, has also emerged in another area. For now, this service, which is still limited to G Suite Enterprise and G Suite Enterprise for Education customers, will be available to all Google Meet users soon. In the future, I think that Google may offer it to other service providers as a cloud service.

How to Install Jitsi Meet on Ubuntu

How to Install Jitsi Meet on Ubuntu

PS: If you need professional assistance about installing & configuring Jitsi Meet, you can contact me via contact link.

Jitsi Meet is a very usable and simple WebRTC based open-source multi-platform video conferencing solution. It can be even cloud based solution or you can install it on your premises. In this blog post, I will explain how to install Jitsi server on your Ubuntu based linux platform.

Installing Jitsi Meet is very easy if you want to install it on Ubuntu linux platform. In this guide you can find how to install Jitsi Meet on Ubuntu by using .deb packages.

I assume that you can install Ubuntu linux and I will continue from that point.

First, let’s install base packages like sudo & ssh, so set that up first. Log in from console as root, then install the necessary packets.

apt-get install -y ssh sudo ufw apt-transport-https

Add your non-root user (mine is ferikci) to /etc/sudoers file.

 ferikci  ALL=(ALL:ALL) ALL 

Now you can continue with your user by using sudo commands.

(Optional) Enable UFW firewall and open the needed ports:

ufw allow in ssh
ufw allow in http
ufw allow in https
ufw allow in 10000/udp
ufw enable

I have to warn you that if you are connected to your linux machine via SSH, enter “ufw enable” command after entering “ufw allow in ssh” command, otherwise you may lose your current SSH connection.

If you will use your Jitsi server with a hostname, please be sure that /etc/hosts file includes your hostname:

127.0.0.1 localhost jitsi.test.local

Now re-login with your non-root user via SSH for the rest of the setup.

Add the Jitsi GPG key.

wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | sudo apt-key add -

On Ubuntu systems, Jitsi requires dependencies from Ubuntu’s “universe” package repository. To do this, add the universe repos with the following command:

apt-add-repository universe

Add the Jitsi repository and update apt

sudo sh -c "echo 'deb https://download.jitsi.org stable/' > /etc/apt/sources.list.d/jitsi-stable.list"
sudo apt-get -y update

Install Jitsi-Meet

Now you’re ready for Jitsi server installation. Use the command below to install jitsi-meet with dependencies:

sudo apt-get -y install jitsi-meet

You will be asked your hostname but do not only write your hostname, you MUST write as FQDN, otherwise you will encounter with problems. By the way, be sure that the FQDN can be addressable with your DNS server (Or you can insert the FQDN to your host file.).

Jitsi Hostname Configuration

After that you will be asked for certificate. In this installation I will use self-signed SSL certificate, so select the first option.

Jitsi SSL Certificate Configuration Menu

The installation will be completed after a while and it will put you to the command prompt. Reboot your linux machine:

sudo reboot

Now it’s time to connect to your video conference GUI. Use https://FQDN to go to the main page of Jitsi server:

https://jitsi.test.local

You will see a greeting page with a room name input field. Just enter a room name and click Go button.

Jitsi Meet Greeting Page

That’s it! You can add more participants with the same procedure or by using URL https://FQDN/roomname

https://jitsi.test.local/testroom
Finally Jitsi Meet is Alive!

Running Jitsi Meet Behind a NAT

If you wish to use your Jitsi server behind a NAT, you must configure your router to forward the following ports to your Jitsi Meet server:

  • 80/TCP
  • 443/TCP
  • 10000/UDP

Next you have to add following lines to /etc/jitsi/videobridge/sip-communicator.properties file:

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=[INTERNAL.IP.ADDRESS]
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=[PUBLIC.IP.ADDRESS]

For example, here is my configuration:

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=192.168.1.20
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=95.9.74.32
Free Video Conference Service For Coronavirus (COVID-19)

Free Video Conference Service For Coronavirus (COVID-19)

It’s a good time to work / collaborate remotely without traveling, or video chat with your loved ones to prevent the spread of the world-threatening Coronavirus (COVID-19). You can do this with the video conferencing server that I have presented to everyone in the link below.

You do not need to register or enter any user information. You can easily create a video call room by clicking the link below:

https://fatiherikci.com/videocall/

After creating and joining a video call room, you can select and copy the page URL in your browser and share it with other participants so they can join the same room. 🙂

Note: This page works best with Chrome / Chromium and Opera.

How to Install & Configure TURN Server (coTURN)

How to Install & Configure TURN Server (coTURN)

This blog page covers how to install and configure coTURN server for your SIP or WebRTC projects (like Jitsi Meet) to allow users behind restrictive firewalls or proxies to connect.

What is TURN?

TURN stands for Traversal Using Relays around NAT. Similar as STUN, it is a network protocol / packet format (IETF RFC 5766) used to assist in the discovery of paths between peers on the Internet. It differs from STUN in that it uses a public relay to transfer packets between peers. TURN is used to exchange media packets when no other option is available. So that it consumes server resources and has an increased latency due to the extra hop in peer to peer connection.

The time when you must use TURN is when one of the peers is behind a symmetric NAT and the other is behind either a symmetric NAT or port-restricted NAT. The frequency of cases where a relay is necessary is around %10 of the overall connections, since STUN is enough for most cases.

Install coTURN Server

Audio / Video based services requires a wide range of UDP ports to be available for WebRTC. In some network restricted sites, such as those behind NAT or a firewall that restricts outgoing UDP connections, users may be unable to make outgoing UDP connections to your media server.

TURN protocol is designed to allow UDP communication flows to bypass NAT or firewalls by forcing the client to connect to the TURN server, and then force TURN server to connect to the destination on their behalf.

Using a TURN server under your control improves the success of connections to your multimedia application and also improves user privacy, since it acts like a proxy so that peers will no longer be sending their IP address information to a public STUN server.

Required Hardware

TURN protocol is not really CPU or memory intensive. Additionally, since it’s only used during connection setup (for STUN) and as a fallback for users who would otherwise be unable to connect, the bandwidth requirements aren’t particularly high. For a moderate number of connections, a single small VPS configuration is usually enough. Here you can find my reccomendations to install coTURN:

  • At least two vCPUs
  • 4GB Memory.
  • 20GB HDD. SSD can be used, but not mandatory.
  • The most important thing is the networking performance.
    • Low jitter (less than 30ms)
    • Low latency (less than 150ms)
    • Enough bandwidth to handle relayed media streams in both directions.

Having the server behind NAT (like on Amazon EC2) is OK, but all incoming UDP and TCP connections on any port (TCP 80 & 443, UDP 3478, 10000-20000) must be forwarded to coTURN server and not firewalled.

Required Software

I recommend using a minimal server installation of Debian with netinst or Ubuntu. Since coTURN software uses port TCP 443, the server which coTURN will be installed cannot have any other web applications running.

coTURN is already available in the Debian and Ubuntu repositories and it can be installed with apt-get:

$ sudo apt-get update
$ sudo apt-get install coturn

Please note that coTURN will not start automatically until the configuration is finished. You can find the configuration tasks in below.

DNS Entry For coTURN

You need to setup a fully qualified domain name (FQDN) that resolves the external IP address of your coTURN server. You’ll use this domain name to generate a TLS certificate.

Generating TLS Certificates

You can use certbot to generate free TLS certificates from Let’s Encrypt. To setup certbot, enter the following commands on your coTURN server:

$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot

Note: If you face with add-apt-repository command not found error, please use sudo apt-get install software-properties-common command to install the necessary packets.

You can then run a certbot command like the following to generate the certificate, replacing turn.fatiherikci.com with the domain name of your TURN server:

$ sudo certbot certonly --standalone --preferred-challenges http \
    --deploy-hook "systemctl restart coturn" \
    -d turn.fatiherikci.com

Current versions of the certbot command set up automatic renewal by default. Note that when certbot renews the certificate, it will restart coTURN service, so coTURN will start to use the updated certificate files. This will cause an interruption on any ongoing TURN connections. You may change the certbot renewal schedule or disable automatic renewal if you like.

Configure coTURN

coTURN configuration is stored in /etc/turnserver.conf file. There are a lot of options and all of them are documented in comments in that file. I include a sample configuration below with comments as the recommended settings, also with notes in places where customization is needed.

You can replace the contents /etc/turnserver.conf with the file below and make these changes:

  • Replace turn.fatiherikci.com with the hostname of your TURN server
  • Change the values in bold with your choices.

You can see an example config file below:

server-name=turn.fatiherikci.com
realm=turn.fatiherikci.com
cert=/etc/letsencrypt/live/turn.fatiherikci.com/cert.pem
pkey=/etc/letsencrypt/live/turn.fatiherikci.com/privkey.pem
fingerprint 
listening-ip=0.0.0.0 
external-ip= 1.2.3.4/192.168.0.1 #or just write the external ip 
listening-port=3478 
min-port=10000 
max-port=20000 
log-file=/var/log/turnserver.log 
verbose 
user=myusername:mypassword 
lt-cred-mech

You can now start the COTURN service with this command:

$ systemctl start coturn

Running coTURN as a Service

The Debian / Ubuntu package for coTURN requires that you edit a file to enable at startup. Edit /etc/default/coturn file and uncomment the following line:

TURNSERVER_ENABLED=1

That’s it! coTURN install is complete. Now you have an up and runnning TURN server!

Testing Your TURN Server

To test your coTURN server, you can use Trickle-Ice testing tool. Go to trickle-ice webpage and enter following:

STUN or TURN URI : turn:Your Public IP Address:3478
TURN username: Username
TURN password: Password

Then click Add Server button and then click Gather Candidates button. If everything works well, you should see Done as final result.

What Is WebRTC?

What Is WebRTC?

We’ve been hearing the name WebRTC a lot lately. In fact, WebRTC, which has been in use since 2011, is not a new technology but is a technology that provides simultaneous media communication (audio and video). The most important feature of WebRTC, which has many advantages, is that it can work directly on many popular browsers without requiring additional software.

WebRTC stands for Web Based Real Time Communication. Multimedia applications can be designed using HTML5 and Javascript APIs.

We can define the communication format used in WebRTC as peer-to-peer. This communication is directly between peers, so you don’t need any media servers. WebRTC is free and has a BSD license, so you can develop WebRTC applications for free. (For example, you can experience a video conference virtual room with WebRTC at this link)

WebRTC Supported Browsers

Nowadays, the following browsers support WebRTC:

  • PC & MAC
    • Microsoft Edge 12+
    • Google Chrome 28+
    • Mozilla Firefox 22+
    • Safari 11+
    • Opera 18+
    • Vivaldi 1.9+
  • Android
    • Google Chrome 28+
    • Mozilla Firefox 24+
    • Opera Mobile 12+
  • iOS
    • MobileSafari / WebKit (iOS 11+)
  • Chrome OS
  • Firefox OS
  • BlackBerry 10
  • Tizen 3.0

WebRTC Components

There are 3 main components in WebRTC:

1. MediaStream API

The MediaStream API provides user access to the camera, microphone or screen using javascript.

2. RTCPeerConnection API

The RTCPeerConnection API provides NAT traversal, codec processing, mutual SDP negotiation, media transmission, and secure connection functions between peers.

3. RTCDataChannel API

The RTCDataChannel API provides the functionality of establishing bidirectional data transfer channels between peers.

Establishing Peer-to-Peer Connection

Signaling is a process that forms the connection between peers. It can be achieved by WebSocket, XMPP, SIP or any other mechanism. WebRTC technology utilizes protocols such as RTP, STUN, SIP and ICE.

WebRTC Signaling Process

Session Description Protocol (SDP)

Also known as SDP, it is a protocol used to communicate media capabilities (voice codecs, IP and port information, etc.) between peers before establishing a connection and to meet each peer at a common point.

Interactive Connectivity Establishment (ICE)

ICE is a framework for the NAT traversal mechanism. ICE collects all available candidates (local IP addresses, STUN return IP addresses, and transmitted IP addresses – TURN). All collected addresses are then sent to remote peers via SDP.

STUN Server

The STUN server enables peers to find public IP addresses, the types of NAT they use, and the relationship between the Internet-side port information associated with the local port information specified by NAT.

TURN Server

When STUN usage is not possible, it is used to transmit media streams over a TURN server (you may think of it as a proxy).

WebRTC is not always peer-to-peer (P2P), but in multiple communication situations (eg video conferencing), different solutions are available. Let’s take a look at these.

Multi-Point Communication Types

1. Mesh

In the mesh network, all peers send their streams separately to other connected peers directly on the network.

All Peers Communicate With Each Other in Mesh Topology

Since this structure is completely distributed, there is no need to have any media servers in the center. The disadvantage of the mesh structure is the use of high bandwidth. In a multi-video call using a mesh structure, if each user generates a 1 Mbps stream, the amount of data sent and received per user will be 4 Mbps in each direction.

2. SFU

SFU stands for Selective Forwarding Unit. An SFU receives incoming media streams from all users and then decides which users to send to.

SFU Transfers Media To All Peers Separately

In this model, each user transmits their own generated media stream to the SFU server. The SFU server can send whoever wants the stream. In this way, bandwidth is used more effectively. Similar with the mesh example above, if each user generates a 1 Mbps stream, the total outgoing data amount per user will be 1 Mbps and the total incoming data amount will be a maximum of 4 Mbps.

3. MCU

MCU stands for Multipoint Conferencing Unit. An MCU receives incoming media streams from all users, decodes them, creates a new layout, and sends it to all users as a single stream.

MCU Combines Media of All Peers & Sends a Single Stream to Peers

The difference of this structure from SFU is that a single combined stream will be sent to each user and the total transmission and reception amount per user will be 1 Mbps in each direction. The disadvantage of this structure, as you can imagine, is the high cost of the MCU with a high processing power in the center.