Feature Proposal: Conversational Components


#6

I see that. So there will be a substantial config.js file with buttons and knobs to customize?

There needs to be a way to customize without updating the .js files under src for the component. Otherwise updating the component via npm from v1 to v2 will wipe out developer modifications.


#7

Should also allow for developers to organize their code into components even if they don’t want to publish on npm. Corporate-wide components and even project-level components.


pinned globally #8

#9

Yes, I believe this would be necessary, at least for larger components.

Agreed!

Oh yes, definitely. We’ve seen this internally that we do a lot of copy-paste between projects we’re working on


#10

I love this idea! It will be ready next week sometime yes? :stuck_out_tongue:


#11

Haha, great to hear that @natrixx!

What types of components would you like to see?

Here are a few ideas:


#12

Those all sound like great ideas! I would likely even use the collect zip one.

I currently have a google sign in workflow setup for my google action, and I think that would make a good component.

I also wonder about something for dialog delegation intents, but I have barely scratched the surface of that topic on a personal project, so perhaps its not super viable.


#13

I’m thinking about this as well, if a component could be created for slot filling. This would require more configuration from the users though, maybe even at the moment of delegating to the component, not on a config.js level.

Would it make sense to have the delegate method call look like this?

this.delegate('COMPONENT_NAME', options);

And the options would then differ for each component? Maybe some components don’t need an ON_COMPLETE_INTENT, others might need more data to work with (which slots to fill, for example).

It could then look like this for the phone number:

this.delegate('COMPONENT_PHONE_NUMBER', { onComplete: 'CompletedIntent' });

#14

yea, that seems like it would work to me!


#15

I’d love to see a Google Calendar Integrated Scheduling component

We’ve barely scratched the surface of the topic, but we’re designing a skill for our own needs, we are meeting with local businesses and schedule via email frequently. Voicefirst would use this component and then also set up our customers with the same capabilities.

“If you’d like to schedule a meeting with us, just ask Alexa.”


#16

That’s a great idea @rjolayolay :+1:

I could see more components like this:

  • Schedule a meeting
  • Be connected to/get a call from customer support
  • Report a bug

#17

By the way, the concept of something like “conversational modules” was also discussed in this great blog post by Jeff Smith almost 1.5 years ago:


#18

Yeah, I like this idea a lot. Kind of the voice equivalent of React UI components. I think this would move us closer to having “patterns” that become the expected/accepted way that common VUI/VUX things are done. So, it’s not just better for developers - it’s better for users too.


#19

I was just talking to @jan about a conversational component that acts as a list selector where a user can select from several choices. This is already implemented in Google Actions as a list selector, where users can select an element by saying its name or Ordinal value. You can then access the selected element in the ON_ELEMENT_SELECTED() handler.

However this only words on Google Assistant devices that have a screen, so no Google Home. You can have a component take a list of choices and return the users choice, letting them select an it by saying its name, or by ordinal value. This would be useful if the user needs to select a song, article, or podcast title.

This is essentially already implemented on Google Assistant, except it does not work on surfaces with no screen. I’d love to hear if someone already knows how to implement a list selector on surfaces with no screen.


#20

Had a look at the pull request for CC (https://github.com/jovotech/jovo-framework/commit/215fe5e8f3e56203abbc29ae27195747433a2c7a) and thought of a few things.

Good work @Kaan_Kilic!

Thoughts and Questions:

  1. Is upppercase (PHONE_NUMBER) the convention or could I as a developer create a component called PhoneNumberUSComponent?
  2. The delegate/completed handler pair makes sense: this.delegate('PHONE_NUMBER', 'CompletedIntent'); paired with CompletedIntent() {}
  3. Would the pair work inside a state? Do you foresee a situation where the delegate can be in one state (or no state) and the completed handler in another state?
  4. Is the component name used by the class, in the useComponents statement, in the call to delegate, off of this.$components and elsewhere just convention? How is it enforced?
  5. In app’s config.js, can other component values be set and used throughout the component? For example, the count of fails before the component switches to sequence mode?
  6. Does the config for the component get merged into the config for the app? In this example, the IntentMaps?
  7. Will the component configs honor stages? config.js, config.local.js, config.dev.js, config.test.js, config.prod.js?
  8. Should the namespacing of the component translations follow a format similar to ‘org-component-name’ (ex: jovo-component-phonenumber or JovoComponentPhoneNumber or MyCorpComponentPhoneNumber or MyCorpPhoneNumber)
  9. Will the component model also follow the main Jovo model (not platform) with the ability to be named en.json and then have it properly merge out into en-US, en-GB, etc. based on the setting in project.js?
  alexaSkill: {
    nlu: {
      name: 'alexa',
      lang: {
        en: ['en-US', 'en-GB', 'en-IN', 'en-CA', 'en-AU'],
      },
    },
  1. Why does the component module use AMAZON.Number for the COMPONENT_PHONE_NUMBER_PhoneNumberIntent instead of AMAZON.PhoneNumber? (Oh, it is US-only)
  2. How will differences in available built-in intents or slots be handled in a component that is to be used in multiple locations?
  3. Having every component have a START handler as its entry point make sense.
  4. Should every intent for a component be in a STATE? For example, the root of the PHONE_NUMBER component has Yes and No intents. Using this component in a skill that also has Yes and No intents would be problematic. Maybe the START handler is outside of a STATE but everything else is in one?
  5. I currently have an implementation that has 2 different intents for numbers. One uses AMAZON.Number slot type and the other AMAZON.PhoneNumber. Because of Alexa’s lack of Intent Context, I have to include both intents in the state when asking for a phone number because sometimes one is chosen over the other. This same thing can happen with AMAZON.Time. All will need to be included that are also included in the main component because the language model is one.
  6. Any STATE in a component should have an Unhandled and FallbackIntent (maps to AMAZON.FallbackIntent). Often a StopCancelIntent (maps to AMAZON.StopIntent and AMAZON.CancelIntent) is also useful.
  7. When I have handled phone number in the past, I have checked if it is valid and sent the user back to re-enter before asking to confirm: Ask, Validate, Confirm
  8. I also handle the US situation when a user gives a 7-digit phone number instead of 10, it is assumed they forgot to say the area code. They are then prompted for the area code. This will be different in different countries.
  9. I could see US_PHONE_NUMBER and INTL_PHONE_NUMBER components
  10. Is the component outside the main src tree because the idea is that component would be separate npm modules?
  11. Can we use removeState, followUpState, toStateIntent, and toStatelessIntent in a component?

There will likely be more feedback going forward.

I really like what you are doing here and am being so detailed because I care about the framework and this feature.

@Kaan_Kilic you have done a great service getting this initial version out.

Thanks!

Mark


#21

Thanks for the feedback Mark!

There is no convention. You can name components as you like.

I think there has to be made some small tweaks for it to work in a state, which should be fairly easy to implement. As for one of them being inside a different state, I am not sure about the amount of work it would take. I also don’t really see a scenario in which it would be useful. The typical scenario I have in my head would be: Inside State A, delegate to a component. Receive the response inside the CompletedIntent inside the same state and route to what ever state you want from that point on. Do you have one in your mind?

There is no naming convention. You have the possibility to specify the component’s name inside it’s name property (which might actually better fit into the config now that I think about it). If there was no name specified the framework will use the constructor’s name (in the example’s case PHONE_NUMBER). The component’s name will be used to reference the component from that point on, whether in delegate or with this.$components. It’s not currently implemented, but if you try to access it with a wrong name, you will simply receive an error.

Yes, that’s possible.

Example:

module.exports = {
    components: {
        PHONE_NUMBER: {
            numberOfFails: 3
        }
    }
};

You can access the component’s config, outside and inside the component, using this.$components.componentName.config

The config inside the component is used as a default configuration provided by the dev. The changes you want to make to that default config will be made inside the app’s root config file, just like the example above.
The changes made in the root config are merged into the component’s default config.

The intentMap is handled a little different. The component’s intentMap is never merged into the app’s one. If a component is active, it’s intentMap will be used, else the app’s one.

Yes, since you make changes to the component’s config using the root config files, the staging feature you already use will work here as well.

Not implemented yet, but should be possible.

Difficult to answer, since it would be different for each component (e.g. amount of work to abstract it for every location).

Yes, I agree. In the updated version everything is inside a state.

I can see, how that could be a problem, but I’d say that’s a problem that can’t be addressed on framework/component side. Do you have an idea?

Yes, I agree. In the case of Stop and Cancel, the current idea is to send out a response (specifying the status as rejected) and routing to the onCompletedIntent at which point the dev can take over.

I prefer Ask, Confirm, Validate, but I’m open for discussion.

Good idea. Might implement that in a further version.

If one went that route, it might make sense to split each component into multiple for each location, e.g. US_PHONE_NUMBER is its own component and DE_PHONE_NUMBER as well.

There’s no real reason, as far as I know.

Yes, should be possible.

Thanks again for the feedback! There should be an updated version of the phone number example out soon.


#22

If given permission, I can get the user’s mobile phone number using:

const mobileNumber = await this.$alexaSkill.$user.getMobileNumber();

This can then be used in the PHONE_NUMBER component to first validate (does it exist) and then use as a default value:

USER: Send me a text
ALEXA: Would you like to send the info to 555.867.5309?
USER: No
ALEXA: What phone number should I use?
USER: 555.555.5555
ALEXA: You want to send the text to 555.555.5554. Correct?
USER: No
ALEXA: What's the phone number?
...

There needs to be a way to pass the default phone number.

Option 1

// options param, completion handler
const options = {
  defaultPhoneNumber: '555.867.5309'
};

this.delegate('PHONE_NUMBER', 'CompletedIntent', options);

Option 2

// options param only
const options = {
  defaultPhoneNumber: '555.867.5309',
  completedHandler: 'CompletedIntent'
};

this.delegate('PHONE_NUMBER', options);

Option 3

// pass to $components
this.components$.PHONE_NUMBER.$params.defaultPhoneNumber = '555.867.5309';
this.delegate('PHONE_NUMBER', 'CompletedIntent');

Which option do you think is better?


#23

I like this one best! Maybe not every component will have a completedHandler, so making this part of the options makes sense to me


#24

You can use an Entity for this. You can build an entity with all the known values of possibilities you would use and create an intent the receives only that value.

But, I highly recommend you, that for no screen devices, to not prompt more than two options to the user. It can turn to be complicated to both the user and you.


#25

@renatoalencar I agree that overloading the user with options makes for a poor experience. However sometimes giving a list of choices is appropriate. For example listing products or article titles.

Allowing the user to select an item by name instead of ordinal value improves this experience. However, when you are pulling data from an API it may be hard to list all the possible entities since they can change frequently and have highly varied values. Alexa has support for Dynamic Entities which might do the trick, however I haven’t found something like that for Google Assistant.