Handle user input with ASR

A code snippet that shows how to handle a user input with Automatic Speech Recognition (ASR). The user says their input at the prompt and their input is acknowledged via a speech-to-text message.

Example

Prerequisites

Create an application

A Nexmo application contains the required configuration for your project. You can create an application using the Nexmo CLI (see below) or via the dashboard. To learn more about applications see our Nexmo concepts guide.

Install the CLI

npm install -g nexmo-cli

Create an application

Once you have the CLI installed you can use it to create a Nexmo application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID in the example below. It will also create private.key in the current directory which you will need in the Initialize your dependencies step

Nexmo needs to connect to your local machine to access your answer_url. We recommend using ngrok to do this. Make sure to change demo.ngrok.io in the examples below to your own ngrok URL.

nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Install dependencies

npm install express body-parser
Initialize your dependencies

Create a file named asr.js and add the following code:

Copy to Clipboard
const app = require('express')()
const bodyParser = require('body-parser')

app.use(bodyParser.json())

View full source

Write the code

Add the following to asr.js:

Copy to Clipboard
const onInboundCall = (request, response) => {
  const ncco = [{
      action: 'talk',
      text: 'Please say something',
    },
    {
      action: 'input',
      eventUrl: [`${request.protocol}://${request.get('host')}/webhooks/asr`],
      speech: {
        endOnSilence: 1,
        language: "en-US",
        uuid: [request.query.uuid]
      }
    }
  ]
  response.json(ncco)
}

const onInput = (request, response) => {
  const speech = request.body.speech.results[0].text

  const ncco = [{
    action: 'talk',
    text: `You said ${speech}`
  }]

  response.json(ncco)
}

app
  .get('/webhooks/answer', onInboundCall)
  .post('/webhooks/asr', onInput)

app.listen(3000)

View full source

Run your code

Save this file to your machine and run it:

node asr.js

Prerequisites

Create an application

A Nexmo application contains the required configuration for your project. You can create an application using the Nexmo CLI (see below) or via the dashboard. To learn more about applications see our Nexmo concepts guide.

Install the CLI

npm install -g nexmo-cli

Create an application

Once you have the CLI installed you can use it to create a Nexmo application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID in the example below. It will also create private.key in the current directory which you will need in the Initialize your dependencies step

Nexmo needs to connect to your local machine to access your answer_url. We recommend using ngrok to do this. Make sure to change demo.ngrok.io in the examples below to your own ngrok URL.

nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Install dependencies

Install-Package Nexmo.Csharp.Client
Import dependencies

Create a file named AsrController.cs and add the following code:

Copy to Clipboard
using Nexmo.Api.Voice.AnswerWebhooks;
using Nexmo.Api.Voice.EventWebhooks;
using Nexmo.Api.Voice.Nccos;
using Nexmo.Api.Utility;

View full source

Write the code

Add the following to AsrController.cs:

Copy to Clipboard
[HttpGet("[controller]/webhooks/answer")]
public IActionResult Answer()
{
    var host = Request.Host.ToString();
    //Uncomment the next line if using ngrok with --host-header option
    //host = Request.Headers["X-Original-Host"];

    var request = WebhookParser.ParseQuery<Answer>(Request.Query);
    var eventUrl = $"{Request.Scheme}://{host}/webhooks/asr";
    var speechSettings = new SpeechSettings { Language = "en-US", EndOnSilence = 1, Uuid = new[] { request.Uuid } };
    var inputAction = new MultiInputAction { Speech = speechSettings, EventUrl = new[] { eventUrl } };

    var talkAction = new TalkAction { Text = "Please speak now" };

    var ncco = new Ncco(talkAction, inputAction);
    return Ok(ncco.ToString());
}

[HttpPost("/webhooks/asr")]
public async Task<IActionResult> OnInput()
{            
    var input = await WebhookParser.ParseWebhookAsync<MultiInput>(Request.Body, Request.ContentType);
    var talkAction = new TalkAction();
    talkAction.Text = input.Speech.SpeechResults[0].Text;
    var ncco = new Ncco(talkAction);
    return Ok(ncco.ToString());
}

View full source

Prerequisites

Create an application

A Nexmo application contains the required configuration for your project. You can create an application using the Nexmo CLI (see below) or via the dashboard. To learn more about applications see our Nexmo concepts guide.

Install the CLI

npm install -g nexmo-cli

Create an application

Once you have the CLI installed you can use it to create a Nexmo application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID in the example below. It will also create private.key in the current directory which you will need in the Initialize your dependencies step

Nexmo needs to connect to your local machine to access your answer_url. We recommend using ngrok to do this. Make sure to change demo.ngrok.io in the examples below to your own ngrok URL.

nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Install dependencies

composer require slim/slim:^3.8

Write the code

Add the following to index.php:

Copy to Clipboard
$app = new \Slim\App;

$app->get('/webhooks/answer', function (Request $request, Response $response) {
    $uri = $request->getUri();
    $ncco = [
        [
            'action' => 'talk',
            'text' => 'Please say something'
        ],
        [
            'action' => 'input',
            'eventUrl' => [
                $uri->getScheme().'://'.$uri->getHost().':'.$uri->getPort().'/webhooks/asr'
            ],
            'speech' => [
                'endOnSilence' => 1,
                'language' => 'en-US',
                'uuid' => [
                    $request->getQueryParams()['uuid']
                ],
            ]
        ]
    ];

    return $response->withJson($ncco);
});

$app->post('/webhooks/asr', function (Request $request, Response $response) {
    $params = $request->getParsedBody();
    $text = $params['speech']['results'][0]['text'];

    $ncco = [
        [
            'action' => 'talk',
            'text' => 'You said ' . $text,
        ]
    ];

    return $response->withJson($ncco);
});

$app->run();

View full source

Run your code

Save this file to your machine and run it:

php -t . -S localhost:3000

Prerequisites

Create an application

A Nexmo application contains the required configuration for your project. You can create an application using the Nexmo CLI (see below) or via the dashboard. To learn more about applications see our Nexmo concepts guide.

Install the CLI

npm install -g nexmo-cli

Create an application

Once you have the CLI installed you can use it to create a Nexmo application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID in the example below. It will also create private.key in the current directory which you will need in the Initialize your dependencies step

Nexmo needs to connect to your local machine to access your answer_url. We recommend using ngrok to do this. Make sure to change demo.ngrok.io in the examples below to your own ngrok URL.

nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Install dependencies

gem install sinatra sinatra-contrib rack-contrib
Initialize your dependencies

Create a file named answer-inbound-call-with-asr.rb and add the following code:

Copy to Clipboard
require 'sinatra'
require 'sinatra/multi_route'
require 'rack/contrib'

use Rack::JSONBodyParser

View full source

Write the code

Add the following to answer-inbound-call-with-asr.rb:

Copy to Clipboard
before do
  content_type :json
end

route :get, :post, '/webhooks/answer' do
  [
    {
      action: 'talk',
      text: 'Please say something'
    },
    {
      action: 'input',
      eventUrl: ["#{request.base_url}/webhooks/asr"],
      speech: {
        endOnSilence: 1,
        uuid: [params[:uuid]],
        language: 'en-US'
      }
    }
  ].to_json
end

route :post, '/webhooks/asr' do
  [{
    action: 'talk',
    text: "You said #{params["speech"]["results"][0]["text"]}"
  }].to_json
end

route :post, '/webhooks/event' do
  puts params
end

set :port, 3000

View full source

Run your code

Save this file to your machine and run it:

ruby answer-inbound-call-with-asr.rb

Try it out

Call your Nexmo Number. When the call is answered you will be asked to say a message. When you are finished, you will then hear your message repeated back to you via speech-to-text.