Localization of Voice Overlay Sample

#1

The voice overlay sample of Jovo for Web works perfectly. I love it because I could design a multi-modal interaction of VUI integrated with GUI. I want to run it in Japanese setting. Could advice which files should be modified? I tried to change some files, but unfortunately I didn’t get it work in Japanese.

#2

I was able to the overlay sample to hear Japanese. However it doesn’t speak anything. Where should I update configuration of speech output?

#3

Hey! Great that you’re using the Overlay starter, would love to take a look at the result when you’re ready to share something!

Let me know if you run into any additional problems!

#4

Thank you for your response.

I would appreciate any help.

I am developing a VUI app for old aged people who are not fluent with IT and GUI.
The app is on WEB which is used from Smartphone browser.
It is friendly to old aged people because of VUI, large character box tiles, no menu, no icons, no install, no update, and no ad.
I used Python server, websocket, and WEB speech API at client, and create the app from scratch (neither Actions builder nor Amazon lex).

Then I found Jovo for web.

Current VUI community has a bias for VUI-only UI, because it emerged from smart speaker devices. But I believe both VUI and GUI have each advantages and unique roles. UI design should aim to integrate best of GUI/VUI from the beginning.
Interface among human is essentially multi-modal, and human machine interface should be the same.
I found this Voice overlay best matches with my objectives as a testbed to design GUI/VUI integrated UX.

I also plan to teach VUI+GUI design to IT college students. It looks to me this voice overlay looks an excellent tool for education purpose too, if I can succeed to make it work in my native language and prepare some sample apps.

I have set up: “npm install --save @nlpjs/lang-ja”.

And I have changed app.ts lines:
webPlatform.use(new NlpjsNlu(),);
into
webPlatform.use(new NlpjsNlu({language: [‘ja’]}),);

I hear Japanese. It changes background as expected with my Japanese instructions. But it doesn’t speak anything, and keep being silent.
And jovo debugger is saying ‘NOT CONNECTED’ while my console of ‘jovo run’ shows request/response as expected. Its log is as follows.

Request - 2021-03-28T07:32:41.915Z
{
“version”: “3.4.0”,
“type”: “jovo-platform-web”,
“request”: {
“id”: “c7610920-c3d7-4b2d-b088-2fd9d68a159d”,
“timestamp”: “2021-03-28T07:32:41.861Z”,
“type”: “TRANSCRIBED_TEXT”,
“body”: {
“text”: “明るく”
},
“locale”: “ja”,
“data”: {}
},
“context”: {
“device”: {
“type”: “BROWSER”,
“capabilities”: {
“AUDIO”: true,
“HTML”: true,
“TEXT”: true
}
},
“session”: {
“id”: “93d86c8c-bcfb-405f-bb3e-08ca17f42282”,
“data”: {},
“new”: true,
“lastUpdatedAt”: “2021-03-28T07:30:48.714Z”
},
“user”: {
“id”: “05aeb627-1618-4549-b504-ae014bd22242”,
“data”: {}
}
}
}

<<<<< Response - 2021-03-28T07:32:41.946Z
{
“version”: “3.4.0”,
“actions”: [
{
“plain”: “明るく します”,
“ssml”: “明るく します”,
“type”: “SPEECH”
},
{
“type”: “CUSTOM”,
“command”: “set-theme”,
“value”: “明るく”
}
],
“reprompts”: [],
“user”: {
“data”: {}
},
“session”: {
“data”: {},
“end”: true
},
“context”: {
“request”: {
“nlu”: {
“intent”: {
“name”: “SwitchThemeIntent”
}
}
}
}
}

Request - 2021-03-28T07:32:48.056Z
{
“version”: “3.4.0”,
“type”: “jovo-platform-web”,
“request”: {
“id”: “853c2b4e-0c6b-49bd-aa92-071f06b1d451”,
“timestamp”: “2021-03-28T07:32:48.052Z”,
“type”: “TRANSCRIBED_TEXT”,
“body”: {
“text”: “暗く”
},
“locale”: “ja”,
“data”: {}
},
“context”: {
“device”: {
“type”: “BROWSER”,
“capabilities”: {
“AUDIO”: true,
“HTML”: true,
“TEXT”: true
}
},
“session”: {
“id”: “fe2e758c-2918-4e50-9263-11276b8d5a59”,
“data”: {},
“new”: true,
“lastUpdatedAt”: “2021-03-28T07:32:41.948Z”
},
“user”: {
“id”: “05aeb627-1618-4549-b504-ae014bd22242”,
“data”: {}
}
}
}

<<<<< Response - 2021-03-28T07:32:48.066Z
{
“version”: “3.4.0”,
“actions”: [
{
“plain”: “暗く します”,
“ssml”: “暗く します”,
“type”: “SPEECH”
},
{
“type”: “CUSTOM”,
“command”: “set-theme”,
“value”: “暗く”
}
],
“reprompts”: [],
“user”: {
“data”: {}
},
“session”: {
“data”: {},
“end”: true
},
“context”: {
“request”: {
“nlu”: {
“intent”: {
“name”: “SwitchThemeIntent”
}
}
}
}
}

Request - 2021-03-28T07:33:06.401Z
{
“version”: “3.4.0”,
“type”: “jovo-platform-web”,
“request”: {
“id”: “e1125908-371c-4a34-befe-d4f7ba19d3be”,
“timestamp”: “2021-03-28T07:33:06.395Z”,
“type”: “TRANSCRIBED_TEXT”,
“body”: {
“text”: “明るく”
},
“locale”: “ja”,
“data”: {}
},
“context”: {
“device”: {
“type”: “BROWSER”,
“capabilities”: {
“AUDIO”: true,
“HTML”: true,
“TEXT”: true
}
},
“session”: {
“id”: “2f7149d2-7927-4029-ad56-c69f461cfd47”,
“data”: {},
“new”: true,
“lastUpdatedAt”: “2021-03-28T07:32:48.068Z”
},
“user”: {
“id”: “05aeb627-1618-4549-b504-ae014bd22242”,
“data”: {}
}
}
}