How does AI assist video communication?

The software TrueConf AI Server is an AI (machine learning) based solution for logging (speech recognition) in conferences. It enables automatic audio recognition from your events held in TrueConf Server / TrueConf Enterprise and saves it as text with speaker separation. Additionally, it offers the option to create summaries for the transcriptions, allowing you to choose the type of text you need. Users can upload any file (both audio and video recordings) to obtain transcriptions not only for TrueConf events.

Integration in full access mode with audio transmission for recognition is only available with TrueConf Server 5.5.0+ and TrueConf Enterprise 1.6+. For more details, see the integration section.

Thanks to seamless integration with TrueConf Server and TrueConf Enterprise solutions, users can access protocols through their video conferencing server without needing to log in to a separate service. This enhances user convenience and reduces the number of different web pages users need to access. We recommend using the video conferencing server’s personal area to manage protocols, although it is still possible to log in via the personal area of TrueConf AI Server.

The solution is offered as software for installation on the customer's infrastructure (on-premise option).

In the future, the terms decryption, transcription, speech recognition are the same, and denote the process of analyzing an audio conference to obtain its transcript (protocol).

Key features

• Automatic speaker detection for correct attribution of each remark.

• Generating a complete transcript of the conference with the division of phrases for each participant (including terminals in meeting rooms).

• Ability to select the main language for conference audio track recognition.

• Displaying timestamps for individual phrases in the protocol.

• Automatic punctuation.

• Automatic capitalization at the beginning of sentences.

• Correct speech recognition when speakers use different languages.

• Correct recognition of not only common speech words but also rarely used ones.

• Recognition of known abbreviations.

• Correct recognition of numbers and dates and their recording in digital format.

• Successful recognition of audio tracks in challenging acoustic conditions: noise, quiet speech, simultaneous speaking by multiple participants.

• Filtering out filler words during recognition, such as "um", "ah", etc.

• The ability to recognize audio on both NVIDIA GPU and x86-64 CPU architecture. The company TrueConf strongly recommends using the option with a discrete GPU for better recognition performance.
• Support for multiprocessor systems when processing audio on the CPU.

• Storing conference audio track and recognized transcript.

• The ability to manually delete any audio recording along with the transcript.

• No limitations on the number of recognizable conferences due to the formation of a task queue.

• Support for audio processing on systems with both single and multiple NVIDIA GPUs.

• Web interface for managing recognized conferences.

• Authorization on the AI server using a user account from the video conferencing server, including when the customer uses an LDAP directory service, such as Active Directory, on their side.

• The ability to integrate the AI server with one or more video conferencing servers.

• Display of ongoing conferences' information: name, Conference ID, start time, and duration.

• Built-in web interface player for playing back the audio file of a conference or call.

• Ability to configure decryption access for both conference participants and any arbitrary user of the video conferencing server.

• Real-time display of conference recognition status.

• Export decryption results in text format (.txt) or table format (.csv).

• Export the conference's shared audio track.

• Notification sent to a conference moderator. This notification indicates that the conference recording was successfully saved on the side of the transcription server.

• Notification sent to a conference moderator. This notification indicates that transcription was successfully completed and a link to the result was provided.

• Notification sent to the users of the video conferencing server. This notification indicates that the users were allowed to view the conference transcript.

• Summarization of the transcribed conference.

• Uploading any audio or video file by the AI server user, whether it's a conference TrueConf or any other recording: detecting different speakers in the uploaded audio stream by their voices and distributing their remarks in the transcript.

• Text search across all transcripts available to the user.

• Users can choose the type of summary depending on their needs, for example, they can create a task list or key event takeaways.

• The system administrator can set custom rules (types of summarization) that users can select if they want to create a summary of a conference.

How it works

Full functionality with TrueConf AI Server requires TrueConf Server version 5.5.0 or later. A read-only connection is available for TrueConf Server version 5.4.5 and older.

The possibility of integration in full access mode with the AI server is determined by:

• with additional licensing of the extension for TrueConf Server;

• your TrueConf AI Server license, which determines the number of video communication servers that can be connected to AI.

The following algorithm is used:

1. Installed on a physical or virtual machine TrueConf Server (or TrueConf Enterprise for large customers).

2. TrueConf AI Server is installed on a separate physical machine (virtual machines are not supported because AI operations require the closest possible interaction with the hardware).

3. A connection between the video conferencing server and the AI server is established using an integration key. In this case, multiple TrueConf Server can be connected to a single instance of the AI server.

4. According to the TrueConf Server settings, the conference audio recording is sent to TrueConf AI Server.

5. Speech recognition and transcript creation are in progress. This can be done automatically immediately after the event ends or launched manually by the conference owner in the AI server's web interface. The owner can also share the original audio and the resulting text with any participant and even another user of their own or federated TrueConf Server.

6. A user can create a summary of the selected type based on the generated transcript.

7. In addition to transmitting audio from TrueConf Server, you can upload any media file with an audio track and generate transcripts for it.