Problem with WebhookVerified

RHaecker · 2020-04-17 10:13:29 UTC

Hi,

I created an Alexa Skill using Jovo 2.2. It worked well in the development environment, so I wanted to deploy it to a server (I’m using a windows server vm). Which worked fine as well, until I tried to verify the requests from alexa. I tried following the instructions at https://www.jovo.tech/docs/hosting/express-js .
But then I get the following errors:

Error -----------------------------------------------------------

Message:
Unexpected token u in JSON at position 0

Stack:
SyntaxError: Unexpected token u in JSON at position 0
at JSON.parse ()
at BasicLogging.requestLogger (C:\USU\voice-server\node_modul
work\src\middleware\logging\BasicLogging.ts:82:30)
at App.emit (events.js:198:13)
at Middleware.run (C:\USU\voice-server\node_modules\jovo-core
re.ts:91:19)
at App.handle (C:\USU\voice-server\node_modules\jovo-core\src
07:41)
at App.handle (C:\USU\voice-server\node_modules\jovo-framewor
413:17)
at Webhook.post (C:\USU\voice-server\index.js:28:14)
at Layer.handle [as handle_request] (C:\USU\voice-server\node
ess\lib\router\layer.js:95:5)
at next (C:\USU\voice-server\node_modules\express\lib\router
13)
at Route.dispatch (C:\USU\voice-server\node_modules\express\l
te.js:112:3)

WARN: Jovo instance is not available. ON_ERROR doesn’t work here

and:

Error -----------------------------------------------------------

Message:
Unexpected token u in JSON at position 0

Stack:
SyntaxError: Unexpected token u in JSON at position 0
at JSON.parse ()
at BasicLogging.requestLogger (C:\USU\voice-server\node_modul
work\src\middleware\logging\BasicLogging.ts:82:30)
at App.emit (events.js:198:13)
at Middleware.run (C:\USU\voice-server\node_modules\jovo-core
re.ts:91:19)
at App.handle (C:\USU\voice-server\node_modules\jovo-core\src
07:41)
at App.handle (C:\USU\voice-server\node_modules\jovo-framewor
413:17)
at Webhook.post (C:\USU\voice-server\index.js:28:14)
at Layer.handle [as handle_request] (C:\USU\voice-server\node
ess\lib\router\layer.js:95:5)
at next (C:\USU\voice-server\node_modules\express\lib\router
13)
at Route.dispatch (C:\USU\voice-server\node_modules\express\l
te.js:112:3)

WARN: Jovo instance is not available. ON_ERROR doesn’t work here

This happens only when I include the alexa-verifier-middleware. But to certify the skill I need the requests to be verified, if I understood that correctly.

My index.js looks like this:
‘use strict’;

const { WebhookVerified: Webhook, ExpressJS, Lambda } = require(‘jovo-framework’);
const { app } = require (’./app.js’);

// ------------------------------------------------------------------
// HOST CONFIGURATION
// ------------------------------------------------------------------

// ExpressJS (Jovo Webhook)
if (process.argv.indexOf(’–webhook’) > -1) {
const port = process.env.JOVO_PORT || 3001;

Webhook.jovoApp = app;

Webhook.listen(port, () => {
    console.info(`Local server listening on port ${port}.`);
});

Webhook.post('/alexa', async (req, res) => {
	await app.handle(new ExpressJS(req, res));
});

}

// AWS Lambda
exports.handler = async (event, context, callback) => {
await app.handle(new Lambda(event, context, callback));
};

And I’m starting the app by calling:
node index.js --webhook

I tried including the alexa-verifier-middleware in the way described on the npm page (https://www.npmjs.com/package/alexa-verifier-middleware), but that didn’t work either. I didn’t get any errors anymore, but I also didn’t get any response from the app back to alexa.

I’m lost here, any idea what I can try or do?

AlexSwe · 2020-04-13 15:09:10 UTC

Hey @RHaecker

Does it fail in certification/validation? Or also in the simulator/device?

I tried to reproduce it. But it works on my machine. (win 10)

Could you provide the request object?

Add console.log in the /webhook post handler

Webhook.post('/webhook', async (req, res) => {
        console.log(req.body);
        await app.handle(new ExpressJS(req, res));
    });

RHaecker · 2020-04-14 02:51:34 UTC

Hey Alex,

thanks for taking the time to look into my problem.

The error occures already when I test the app. I tried adding the console.log you recommended, but the it is not executed, because the error is thrown before it. If I remove the “WebhookVerified” and use “Webhook” instead, it outputs the requestbody, but then also the application works and reponds as expect.
So for some reason, when I add, WebhookVerified instead of Webhook, like it is described in the mentioned tutorial, the code inside the post mapping is not executed, but an error occures (I guess the error occures while the verifying of the request is done, before the mapped code is executed).

Is there a working demo somewhere, so I can compare what I did, to a working version?

Again, thanks for your time.

AlexSwe · 2020-04-14 07:24:58 UTC

That’s weird.The error in BasicLogging.ts:82:30 is triggered by a function in app.handle(...). Could you remove await app.handle(new ExpressJS(req, res)); for a moment and just print req.body or even req

I tried it on an existing project of mine. And it worked.

Could you try to debug it in the alexa-verifier-middleware package?

/node_modules/alexa-verifier-middleware/

RHaecker · 2020-04-14 13:05:25 UTC

So I removed the app.handle function and logged the req.body and then also req to the console. req.body is undefined, but the req object is logged, but due to it’s length I can’t easily copy it from the console. Which parts are relevant if the body is not there?

AlexSwe · 2020-04-14 15:21:42 UTC

Let’s start with the following.

req.rawBody
req.headers
req._body

RHaecker · 2020-04-15 08:19:28 UTC

+++ here comes the req.body: +++
undefined
+++ before was the req.body: +++
+++ here comes the req.rawBody: +++
undefined
+++ before was the req.rawBody: +++
+++ here comes the req.headers: +++
{ host: ‘mysubdomain.mydomain.de’,
‘content-type’: ‘application/json; charset=utf-8’,
accept: ‘application/json’,
‘accept-charset’: ‘utf-8’,
signature:
‘lWTP6DYRZUh//G0Grb0H/mPKW0BpLy0+nfvf6OP6KU7ofhXL6MA5UXVM74bSp98OyOJBxqHIQmYw
vb98SxisDIah1l6qW/eAvtBLPPtKKulYpALVrq9CznTj/9OiK2tBR+N/61i1O7wrtCxDMf7guF7CNidZ
nCHOzHUjDmg2Ky1GLGn81v0R/zz4xgTputCruxc79rzJ68h4yBhbuRSDifJO3f0Sa+ZrAS19j2znU99j
GCK+FJhOmZfpbUrNLMfQSHm0aOa/13+BkMa8//vTarfvsZpxaTwiiBQwYBIC6zhWDv0rbxB67zoKoDdn
NJuVz9L1FB8tlP/66r8JFLEmmw==’,
signaturecertchainurl: ‘https://s3.amazonaws.com/echo.api/echo-api-cert-7.pem’
,
‘user-agent’: ‘Apache-HttpClient/UNAVAILABLE (Java/1.8.0_242)’,
‘x-forwarded-for’: ‘myIP, myIP’,
‘x-forwarded-host’: mysubdomain(.)mydomain(.)de, mysubdomain(.)mydomain(.)de’,
‘x-forwarded-server’: ‘mysubdomain(.)mydomain(.)de, mysubdomain(.)mydomain(.)de’,
connection: ‘Keep-Alive’,
‘content-length’: ‘3733’ }
+++ before was the req.headers: +++
+++ here comes the req._body: +++
undefined
+++ before was the req._body: +++

I had to change the domains, so the linkparser wouldn’t confuse them with real links, because as a new user I can only put 2 links in a post.

Just to be sure, so i didn’t make any typos or mistakes on my part (because a lot of this is undefined), here is the part of the index.js for the logging:
Webhook.post(’/alexa’, async (req, res) => {
console.log("+++ here comes the req.body: +++");
console.log(req.body);
console.log("+++ before was the req.body: +++");
console.log("+++ here comes the req.rawBody: +++");
console.log(req.rawBody);
console.log("+++ before was the req.rawBody: +++");
console.log("+++ here comes the req.headers: +++");
console.log(req.headers);
console.log("+++ before was the req.headers: +++");
console.log("+++ here comes the req._body: +++");
console.log(req._body);
console.log("+++ before was the req._body: +++");
//await app.handle(new ExpressJS(req, res));
});

AlexSwe · 2020-04-20 10:47:17 UTC

Hmmm, never experienced such a behavior. Is it a Win Server issue?

Could you try it locally with ngrok?

node index.js --webhook
ngrok http 3000

RHaecker · 2020-06-12 14:43:00 UTC

First of all, sorry I didn’t reply for such a long time, some other things came up and I was waiting for a new server.
So what I did in the meantime is:

Upgrading to version 3 (at least I think I did)
Getting a new Server (Win Server 2019) and installing a new environment (now using node 14.4.0, I hope this is a working version, but I can change it)

I also tried to do some debugging using chrome and vs code. The Problem is, that I don’t really find the point where the error is caused. If I set the break point inside the post-function of Webhook, the request is already lacking parts like the body (I checked, it is there if I use the Webhook instead of WebhookVerified).
This is the console.log of the request in chrome (finally more readable):

I will try and look into ngrok now maybe I can see something there. But in the meantime, maybe these information are useful in figuring out what happens there.

Is there somewhere a tutorial with exact steps to follow when deploying to windows? I want to ensure, that the error is not on my part…

Thanks for your time and best regards
Raphael

RHaecker · 2020-06-18 14:38:05 UTC

Ok, I have found the issue and I will admit, it is mostly my fault.

So I did some experimenting with doing the verification my self, but I couldn’t get the application to set the correct http responsecode (400) and I was looking around on what happens there, when I realized, the url path “/webhook_alexa” is hardcoded for the verified server.

So the problem was that I didn’t use this. Sorry for wasting your time and thanks that you looked into this. I would suggest making this point a bit clearer in the docs to avoid issues like this.

AlexSwe · 2020-06-18 14:37:53 UTC

No problem! Glad it’s working now!