In this blog post I will show you how to make your own personal assistant (think Siri, Cortana, Alexa) in R, very quickly.
This will be done in three steps:
- Get R to recognise your voice and convert it to text
- Set up a system which uses text from (1) as a query to Google or pre-defined functions
- Return the action or answer to the app
Standing on the Shoulders of Giants
I must mention this leverages the awesome work done by Yihue Xie using his app which allows you to edit a graph by talking to it.
Step 1: Voice Input
The app will need a key phrase to recognise so it knows what text to process - I will call mine Albert.
I decided to have two functions in my app:
- Have custom calls which allow user-defined functions such as answering a personal question, sending an email, going to a website, running some simulations, etc.
- Get Google to tell you answers to your questions
For this example, for my custom calls, I have one personal question, and one action. I have used
if statements but in reality questions, answers and actions should be written to a data frame.
My custom calls are as follows:
As you can see, if I ask it for my next appointment, it tells me where and when it is. If I ask for a suggestion of something cool, it opens a browser and takes me to R-Bloggers.
If the question asked isn’t in the list of custom calls, it simply asks Google and hopes what is returned is sensible:
Don’t be perplexed by getting information from Google!
The above custom functions attempt to get the initial results from Google - a very crude version of Summly, if you will. The functions are defined as follows.
As mentioned, they are very crude, so please improve on them if you can! They simply use the
XML packages to help you get answers.
Now, we can ask it a question and get an answer. Cool!
Here’s how you do it:
- Make sure you have shiny, RCurl and XML packages installed.
- Save the app.R and init.js files into a single folder.
- Open app.R, and run the app
- The browser will open, and use the keyword "Albert" before asking a question. Try these:
- "Albert, where is the Eiffel Tower?"
- "Albert, when is my next appointment?"
- "Albert, show me something cool!"
This version listens to your questions and presents answers in text on screen. The obvious next step is to be spoken back to - watch this space for part 2 of this blog.
Full code below (save both files in the same folder):