Gemelo.ai Voice Plugin is a UPM compatible package

Installation in Unity Editor

Install the package via the menu Window/Package Manager.

Add package from Git URL:

https://github.com/charactr-platform/gemelo-unity-plugin.git#v1.1.0

API Configuration

Before you use the Plugin, you have to initialize it first with your account's credentials. To get them, please log in to the Client Panel.

📘

Use Tools/Gemelo.ai Voice/Configuration tool menu in Unity Editor to provide API access keys first!

Text to speech convert C# API reference (Convert API)

Voice Conversion request

var request = new ConvertRequest()
{
  Text = "This is sample text to convert using Gemelo API",
  VoiceId = 151 //Liam
};

//Note: API configuration needs to set before making requests
using (var convert = new Convert())
{
  var audioClip = await convert.ConvertToAudioClip(request);
}

Text to speech streaming C# API reference (Streaming API)

📘

Please refer to AudioStreamingManager class how to use our streaming technology in runtime.

AudioStreamingManager is the faster way to test and iterate over text to voice streaming.

TTS Simplex Streaming

Simplex streaming allows you to send the text to the server and receive the converted audio response as a stream through WebSockets. It greatly reduces the latency compared to the traditional REST endpoint and it's not limited by the text's length.

To use a custom request, create aIEnumerator return type method, that will yield when AudioClip will be ready.

On Update method we are consuming incoming data from the WebSocket connection until buffering is completed.

using System.Collections;
using Gemelo.Voice;
using Gemelo.Voice.Audio;
using Gemelo.Voice.Streaming;
using UnityEngine;

public class TestStreaming : MonoBehaviour
{
    private const string Text = "Hello world! From Gemelo.ai Voice Plugin!";
    private const int VoiceId = 151;
    
    private DefaultAudioStreamingClient _client;
    
    // Start is called before the first frame update
    void Start()
    {
        //Note: API configuration needs to set before making requests
        var configuration = Configuration.Load();
        var url = Configuration.STREAMING_API + $"?voiceId={VoiceId}";
        
        var audioParameters = new AudioParameters();
        //Maximum duration of AudioClip buffer, in seconds
        audioParameters.SetMaxLenght(5); 
        audioParameters.SetAudioDataType(AudioDataType.MP3);
        
        _client = new DefaultAudioStreamingClient(url, configuration, audioParameters);
        
        StartCoroutine(CreateTtsRequest(Text));
    }
    
    private IEnumerator CreateTtsRequest(string text)
    {
        _client.Connect();
        _client.SendConvertCommand(text);
        yield return new WaitUntil(() => _client.Initialized);
        
        //AudioClip can be played in AudioSource now.
        var audioClip = _client.AudioClip;
        
        Debug.Log("AudioClip initial length: "+ _client.AudioLength);
        
        yield return new WaitUntil(() => _client.BufferingCompleted);
        Debug.Log("AudioClip full length: "+ _client.AudioLength);
    }

    private void Update()
    {
        if (!_client.BufferingCompleted)
            _client.DepleteBufferQueue();
    }
}

🚧

MP3 Streaming can be used only with 44100 sample rate (default value).

TTS Duplex Streaming

This more advanced technique allows you to start a two-way, WebSockets-based stream, where you can asynchronously send text and receive the converted audio. You control when the stream ends.

🚧

Duplex is not supported currently in Unity Plugin.

Examples

Our Unity Plugin comes with a bunch of examples to help you start working with it. Please, refer to the Unity Plugin README@GitHub if you wish to run them.