Cerence - NLU Component¶

The NLU component takes a speech output, performs speech recognition and analyses the result with a pre-trained NLU model and returns the result as a token to stepDP.

Setting up Cerence¶

First you need to model your model in Cerence Studio. When you are satisfied with the model, you can build and deploy it in Cerence. In the last step, you will receive your credentials, which you will need in the next step.

Adding Cerence to the Dialog¶

In your main class, you create a variable for Cerence and initialise Cerence with the credentials you received in the previous step.

public class MyDialog extends Dialog 
{
    public static de.dfki.step.cerence.NLUController cerence;

    public MyDialog() 
    {
        try {
            cerence = new NLUController(this);
            cerence.InitCerenceMix("wss://autotesting-azure-use2-cvt1-voice-ws.nuancemobility.net/v2",
                    "NMDPTRIAL_julian_wolter_dfki_deXXXXXXXXXXXXXXXXXXX",
                    "86376F1D3BF1805A1ED8CXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXDA8A97CA4F3B7830B57AB3C2DB8DD1571DF",
                    "AXXX_CXXXX",
                    "eng-USA");
            cerence.StartAudioServer("0.0.0.0", 50002);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

In this example, the audio server is started on port 50002. Cerence expects a TCP stream with the audio data in OPUS format on this port. This data is later forwarded to the web service.

To control Cerence, it makes sense to build a web interface for it. We recommend the web interface from the SmartHome example, which you can find here. Remember to add the class of the web interface in the entry point of your application.

@SpringBootApplication(scanBasePackages = {"de.dfki.step.web", "de.dfki.step.example.cerence.web"})

In the future, it is planned to equip stepDP with a general system that can manage audio streams, so that the last step is no longer necessary.