Tutorial - How to build a Facebook Messenger bot using Django, Ngrok
archived on 2016-04-29
This article is not maintained and the information here is stale.
2016 has been hailed as the year of bots with Facebook, very recently launching their own Messenger platform. This lets businesses (Facebook pages) automatically talk to users/customers to answer questions ranging from simple to complex. The platform is currently in beta but with close to a billion active users, it is by far the biggest messaging platform out there. Naturally, as a person who works a lot with NLP, I had to get my feet wet by building a small toy project. I will list all the basic steps to get a simple keyword based bot powered by Django/Python working on Facebook messenger.
What you need to know - Python; Django (only if you intend to use databases later on)
1. First things first - Facebook App & its page!¶
To build a Facebook messenger bot, we must first create a new Facebook app. To do this, lets head to the Facebook developer site, create a new app and fill out the relevant details to get our App ID. You can select "Apps for Messenger" as its category. This will prepopulate the required products for the app in the dashboard.
You should be redirected to the dashboard that looks like this
Next, our app must be associated with a Facebook page since it is the page that interacts with the users. If you don't already have a page, go ahead and create one. You can spice up your page with pictures and relevant details or you can leave it as is for now. The page doesn't have to be published for the bot to work.
To access the Facebook messaging API, our app will need a page access token. Go ahead and click on the "Messenger" tab on the control panel under "Products". Then, select the page that the app will link to and a token will be generated. This token will later be used by your app to send messages to users!
2. Django server setup¶
This tutorial shows you how to use Django as the backend server. This can be replaced by any other framework.
Lets create a new virtual environment, install Django, create a new project and run the development server.
mkvirtualenv yomamabot
pip install django
django-admin.py startproject yomamabot
cd yomamabot
python manage.py runserver
Now you must have your django server running at 127.0.0.1:8000. Go to the URL and you must see the "It worked!" page. Django projects are made up of small apps that perform specific functions. So lets create an app for our Facebook messenger bot by running
This must now create an app in the project directory with the relevant files. Go to the settings.py file and add this app by appending the app name 'fb_yomamabot' into the INSTALLED_APPS variable.
# yomamabot/yomamabot/settings.py
# ..... other settings .....
INSTALLED_APPS = [
'django.contrib.admin',
# ...
'django.contrib.staticfiles',
'fb_yomamabot',
]
Next, lets set up the URLs. In the main urls.py file, add this line.
# yomamabot/yomamabot/urls.py
from django.conf.urls import url, include
from django.contrib import admin
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^fb_yomamabot/', include('fb_yomamabot.urls')),
]
This means, when a request like 127.0.0.1:8000/fb_yomamabot/<rest-of-the-url> comes to the server, it strips away 127.0.0.1:8000/fb_yomamabot/ and forwards <rest-of-the-url> to the urls.py file in the fb_yomamabot app directory to handle it. So lets create a urls.py file in our fb_yomamabot app directory and leave it empty for now.
3. Webhook - time for your app and Facebook to hook up!¶
"A webhook is a way for an app to provide other applications with real-time information". In this case, Facebook provides our app with real-time information i.e. whenever someone sends our page a message.
The first part is to decide exactly "where" our app and Facebook should "hook up"! Lets create a URL and tell Facebook about it so it can send updates there. Later, we can write code to handle these incoming requests. Keep in mind that we want this URL to be secret, unless you want intruders to know where we hook up!
Generate a long random sequence for our url. In a python interpreter, run this
import os, binascii
print binascii.hexlify(os.urandom(25))
Here is one such sequence, 66d2b8f4a09cd35cb23076a1da5d51529136a3373fd570b122. Lets define our webhook URL in urls.py file and create a view to handle it
# yomamabot/fb_yomamabot/urls.py
from django.conf.urls import include, url
from .views import YoMamaBotView
urlpatterns = [
url(r'^66d2b8f4a09cd35cb23076a1da5d51529136a3373fd570b122/?$', YoMamaBotView.as_view())
]
To ensure everything is working well, update our views.py file to look like this
# yomamabot/fb_yomamabot/views.py
from django.views import generic
from django.http.response import HttpResponse
# Create your views here.
class YoMamaBotView(generic.View):
def get(self, request, *args, **kwargs):
return HttpResponse("Hello World!")
Now when you go to the following URL, you should see "Hello World!".
Next, we have to tell Facebook about our webhook and specify the URL. However, Facebook can only access web URLs. Since we are working locally, Facebook cannot reach us. Also, Facebook requires the endpoint to run over https!
Ngrok to the rescue!¶
For development purposes, we will use Ngrok that sets up secure tunnels to our localhost i.e. Ngrok gives web accessible URLs and tunnels all traffic from that URL to our localhost! Easy, peasy! Go to Ngrok's download page, download the zip file, unzip and simply run the command
This should open up a terminal panel that looks like this. Ngrok also provides a neat web interface at 127.0.0.1:4040 to look at all the requests that are coming in!
Its that easy! Now any outside computer can reach your localhost server at https://36096fff.ngrok.io which means so can Facebook! So lets set up the webhook for Facebook. Go to your app dashboard and click on Messenger (where you created your page access token previously). Click on "Setup Webhooks" right below "Token Generation" and fill the details as shown.
Note: In Django 1.10+, make sure you add the ngrok url to "ALLOWED_HOSTS" list in your
settings.pyfile. For this example,ALLOWED_HOSTS = ['36096fff.ngrok.io']
The "Verify Token" can be any value you wish. This is what Facebook will send along with a challenge token. Your server should make sure the "Verify token" matches and respond back with the challenge token. If you click on "Verify and Save" now, it won't work because our server currently responds with a string "Hello World!". Lets update our views.py file.
# yomamabot/fb_yomamabot/views.py
class YoMamaBotView(generic.View):
def get(self, request, *args, **kwargs):
if self.request.GET['hub.verify_token'] == '2318934571':
return HttpResponse(self.request.GET['hub.challenge'])
else:
return HttpResponse('Error, invalid token')
Now click on "Verify and Save" and your webhook is setup! You should see the green tick. Next, select the page you want & your app to be subscribed to and click on Subscribe. Facebook will now push any page events, in real time, to the URL you just created as POST requests.
(Initial release expected you to manually send a POST request with your Page access token using curl to subscribe your app to page events)
4. Send and Receive messages¶
Now that you are subscribed, Facebook will make POST requests every time the selected page event is triggered. Here is a brief description of what each even means. Feel free to look at the full API reference for more details.
messages- This is the primary type of event we will be interested in. This event is triggered every time someone sends your page a message.message_deliveries- Delivery reports for the messages you send.messaging_postbacks- Facebook allows you to send more than text. It allows templates with buttons that, upon being clicked/tapped, can send the specified payload back to your webhook. A simple example is a template with 2 buttons to subscribe or unsubscribe. When the user taps any of them, the postback event is triggered and Facebook sends the payload you set for the button, back to you.messaging_optins- This is used for authentication and is trigged when someone uses Facebook's Send-to-Messenger Plugin that may reside on your website as an entry point.
For this simple toy example, we will restrict ourselves to handling only messages. Now lets write some code to simply echo a users message back.
Receive a message - First, in our views.py, lets update the post function to print the received message. Since django POST methods require csrf tokens by default, lets exempt the dispatcher method to handle Facebook callbacks. Update your views.py as shown below.
# yomamabot/fb_yomamabot/views.py
# import required modules
class YoMamaBotView(generic.View):
# The get method is the same as before.. omitted here for brevity
@method_decorator(csrf_exempt)
def dispatch(self, request, *args, **kwargs):
return generic.View.dispatch(self, request, *args, **kwargs)
# Post function to handle Facebook messages
def post(self, request, *args, **kwargs):
# Converts the text payload into a python dictionary
incoming_message = json.loads(self.request.body.decode('utf-8'))
# Facebook recommends going through every entry since they might send
# multiple messages in a single call during high load
for entry in incoming_message['entry']:
for message in entry['messaging']:
# Check to make sure the received call is a message call
# This might be delivery, optin, postback for other events
if 'message' in message:
# Print the message to the terminal
pprint(message)
return HttpResponse()
Now open your Facebook page, click on message and type something. If everything goes well, it should look like this
Send a message - To send messages, you need to make POST calls using Facebook's Graph API using your page access token. The json body will include the user's id and the message payload. You can simply send text or use templates with images, lists, links etc. For details, look at Facebook's Send API reference. For our toy project, we will only send back text.
To echo the recevied text back, we will use the excellent requests package to make POST calls. Lets create a simple helper function that will make this POST call with the received text.
# yomamabot/fb_yomamabot/views.py
# This function should be outside the BotsView class
def post_facebook_message(fbid, recevied_message):
post_message_url = 'https://graph.facebook.com/v2.6/me/messages?access_token=<page-access-token>'
response_msg = json.dumps({"recipient":{"id":fbid}, "message":{"text":recevied_message}})
status = requests.post(post_message_url, headers={"Content-Type": "application/json"},data=response_msg)
pprint(status.json())
Now in our post function, below the pprint(message) line, call this function with the sender id and the received message.
# yomamabot/fb_yomamabot/views.py
def post(self, request, *args, **kwargs):
# .....code omitted for brevity......
if 'message' in message:
# Print the message to the terminal
pprint(message)
# Assuming the sender only sends text. Non-text messages like stickers, audio, pictures
# are sent as attachments and must be handled accordingly.
post_facebook_message(message['sender']['id'], message['message']['text'])
Go ahead and try it out. Send a few messages and it should echo back!
5. More than echo!¶
Now that we have out end to end pipe line set up, we can go ahead an start implementing some basic functionality into the bot. Since our bot is supposed to tell Yo Mama jokes, lets create a simple dictionary (for now) that has some jokes indexed by category.
jokes = {
'stupid': ["""Yo' Mama is so stupid, she needs a recipe to make ice cubes.""",
"""Yo' Mama is so stupid, she thinks DNA is the National Dyslexics Association."""],
'fat': ["""Yo' Mama is so fat, when she goes to a restaurant, instead of a menu, she gets an estimate.""",
""" Yo' Mama is so fat, when the cops see her on a street corner, they yell, "Hey you guys, break it up!" """],
'dumb': ["""Yo' Mama is so dumb, when God was giving out brains, she thought they were milkshakes and asked for extra thick.""",
"""Yo' Mama is so dumb, she locked her keys inside her motorcycle."""]
}
Next, lets update our post_facebook_message function to search for one of the three keywords and send a joke to the user. If the keyword is absent, it'll respond with a failure.
# yomamabot/fb_yomamabot/views.py
def post_facebook_message(fbid, recevied_message):
# Remove all punctuations, lower case the text and split it based on space
tokens = re.sub(r"[^a-zA-Z0-9\s]",' ',recevied_message).lower().split()
joke_text = ''
for token in tokens:
if token in jokes:
joke_text = random.choice(jokes[token])
break
if not joke_text:
joke_text = "I didn't understand! Send 'stupid', 'fat', 'dumb' for a Yo Mama joke!"
post_message_url = 'https://graph.facebook.com/v2.6/me/messages?access_token=<page-access-token>'
response_msg = json.dumps({"recipient":{"id":fbid}, "message":{"text":joke_text}})
status = requests.post(post_message_url, headers={"Content-Type": "application/json"},data=response_msg)
pprint(status.json())
Now when you chat, it will look like this.
6. A personal touch!¶
You can use Facebook's API to query basic user information from the user id. You can get the user's first name, last name and a link to their profile pic. Append the following code soon after the joke_text is set.
# yomamabot/fb_yomamabot/views.py
def post_facebook_message(fbid, recevied_message):
# ...... code that selects and assigns the joke_text ....
user_details_url = "https://graph.facebook.com/v2.6/%s"%fbid
user_details_params = {'fields':'first_name,last_name,profile_pic', 'access_token':'<page-access-token>'}
user_details = requests.get(user_details_url, user_details_params).json()
joke_text = 'Yo '+user_details['first_name']+'..!' + joke_text
# ..... code to post message ...
Now the bot uses the user's name in every text as seen below.
Tip: You would want to store the user details in a database instead of calling the API every single message!
7. What next? Build, test, iterate!¶
This is a barebone bot that is only intended to demo the Facebook messenger platform. The real usefulness of the bot lies in its ability to understand what the user sends and how well it answers the user's questions. This is an active research area and computers are far from being good at understanding natural language. However, bots are still useful when the information space is fairly small and the questions follow some specific format.
For the toy application, the obvious next step is to use a database filled with Yo Mama jokes. You should also store user information and keep track of jokes you've already sent them. Also, if you have subscription for daily jokes, you could keep track of that and have a cronjob that runs a python manage.py command daily, that sends a joke to every subscribed user.
Facebook messenger platform is still in beta and before your bot is available to the public, they must approve it! This process is expected to take several weeks currently and understandably so. A few bad bot experiences might frustrate the users and shun them away from about the platform and bots in general. Guardian already published this strongly-worded piece "Please, Facebook, don't make me speak to your awful chatbots" that has pointed out common frustrations and issues with not-so-good bots. Here is another piece from TechCrunch "Facebook’s new chatbots still need work". This highlights the importance of a good initial experience for your users when interacting with your bot.
Test, Test, Test!¶
While your bot approval might take several weeks, you don't have to wait for that time to have real users interacting with your bot! You can add test users from your dashboard. Go to your "Roles" tab from the dashboard and then click on "Add testers". You can now add anyone who is your Facebook friend. For users who aren't but are interested to test your bot, you can add them using their fbid (numeric id) or their username. A Facebook username is NOT the display name. You can get it from the URL of the person's profile page.
8. Submit for approval¶
Once you are happy with your application, deploy it onto your servers. Since https is a requirement, you could either buy a certificate or use letsencrypt to get a free one. Here is a tutorial from Digital Ocean on how to obtain an SSL certificate with letsencrypt and turn on https for nginx.
Then you can submit your app to Facebook for approval! Head to the Messenger tab and click on "Request permissions". For messenger bots, your app at least needs the "pages_messaging" permission.
Once you select the permissions and Add them, you will be redirected to the App Review tab. If you have a red exclamation as shown below, it means that your app is currently ineligible for submission.
For the App details, at a bare minimum, you will need an App Icon and a privacy policy URL. You can submit any 1024x1024 image for your App icon and you can use an excellent service by IUbenda that generates Privacy policies for Facebook apps. Once you, enter this, head to the App Review where you should click on Edit Notes and fill out the details to submit your app for approval.
For the approval process, you must agree to their terms which includes not spamming your users with ads and promotional offers (understandable!) and submit a screencast of interaction with your bot. Here is the screencast of a slightly better version of Yo Mama Bot that I submitted.
2021 Update: My bot has been inactive for many years now!
9. Some tips.!¶
- Facebook provides a bot engine called Wit.ai that parses input text and converts it into structured information that you can then use to set responses. There are also several commercial and non-commercial information extraction software that could be used to this end. You could also use free NLP tools like NLTK, Stanford coreNLP and build your own query processor.
- Move away from words and use word vectors instead! Funny and Joke are two unrelated words for a computer if it is just text. So are words like stupid and dumb. Using word vectors from Google's word2vec or Stanford's GloVe, you can capture the relative closeness between any two words which eases the constraint on users to use the exact word a bot expects!