Create Twitter Image Recognition Bot with Serverless and AWS

TIP: Check out their tutorial — it’s really awesome and will help you to understand what’s what in serverless world.

In this tutorial, I will show you how to build a Twitter Bot that will receive a tweet with an attached image, recognize what’s on that image (if it’s an animal) and respond with the correct response. For example, if you’ll tweet an image of a giraffe, the bot will use our serverless architecture and almost immediately respond to you with something like this — “Hey, on your image I can see a giraffe!”.

In this example, you’ll use Amazon Web Services — AWS. It has dozens of great cloud services, but you will use only a few — S3 bucket, Lambda Functions, API Gateway and Image Rekognition. Check out this fancy flowchart to see how it’s all going to work together.

Now, when you successfully applied for API access, on your developer account, go to Dev Environments and create an environment for Account Activity API. Make note of Dev environment label, because you’ll need it later.

Now, the way Account Activity API works might look a little confusing at first, but it’s actually pretty straightforward. Here are the steps required to make it work:

To handle all these requests, we will create a Twitter Controller Class.

First of all, let’s create all properties that we’ll need to use in our methods:

All properties that we’re going to pass in a constructor, will be stored in a serverless.env.yml file in the project root directory. I’ll get back to that later.

Now, let’s take a look at methods that will handle communication with Twitter API.

To start working with Serverless we need to install it (duh!). Open your terminal and install the framework globally.

After that, navigate to your project folder and run :

This command will create a default node.js + AWS configuration file. The yaml file that was generated contain lots of commented code. We won’t need it here, so go on and remove it. The only things we care about now is this:

After setting up the credentials, you could just start adding configuration details. But there’s one more thing I want to mention. Normally, Serverless will default your profile name and AWS region you’re using, but if you have multiple profiles on your machine (private, work etc.), it’s a good practice to define it in serverless.yaml file like that:

TIP: In your command line you can use a shortcut — instead of ‘serverles …’ you can simply type ‘sls …’. Pretty neat.

Like I’ve mentioned before, for storing our keys, tokens and other variables, we will create serverless.env.yml file in the root folder. It should look like that:

First five of them, we’ve mentioned before, while creating App in Twitter Dev Account. There’s also an URL base in here, just to keep all variables in one file. We will create Twitter Challenge Response Check URL later on, with Serverless Framework and AWS.

With env file in place, you can inject variables into your code, by placing them in serverless.yml file. We can do it like that:

By adding variables as environment object in provider, we’re able to access them in any function that we’re going to define in a serverless configuration file. We could also pass it separately in each function, but I’ll show that example later in the tutorial.

Now, let’s get to the main part of our project — lambda functions. Let’s start with defining first of them in our config file.

So, what happened here is, we create the first lambda function called handleCrc. Body of the function is going to live in the handler method. In the events section, you define when that function meant to be invoked. As you can see, after sending GET request to our endpoint — twitter/webhook/handleapi, handleCrc function is going to run. That’s the basic way of creating lambda functions configuration in the Serverless Framework. There are multiple options to define events, like for example — image has been uploaded to S3 Bucket, new data has been added to database etc.

Let’s see how your function actually looks like, and what is it doing.

You’re going to use Crypto library to encode response from Twitter API. As you can see it’s pretty straightforward. You need to pass Twitter Challenge Response Check token, and your Twitter Consumer Secret to encodeCRC function and return the result. Notice that we’re getting our secret from process.env object. We can access it like that thanks to previously defining it in serverless.yml file.

Now you can deploy your serverless build to obtain the Twitter Challenge Response Check URL that we’re going to need later on.

To deploy our function, simply run serverless command from our project directory:

That will create new AWS CloudFormation template, and upload your functions into S3 bucket. If everything went fine, you should see something like this:

Here you can find all info about your stack: stage, stack name, endpoints, uploaded functions etc. The endpoint is the thing that you should care about right now. Like I’ve mentioned before, you’re going to need that URL address to pass Twitter Challenge Response Check. Copy and paste it into your serverless.env.yml file.

Now, let’s add lambda functions that are going to be responsible for registering webhook and user subscription. You’re going to follow the same pattern for defining functions in the config file.

Actual bodies of these functions are really straightforward. Basically, you invoke appropriate method from TwitterController class, that we’ve created earlier.

Certainly no magic in here. You create a new instance of a class, pass all the credentials, and run the functions. That’s it. Let’s redeploy our application with:

You should see a ‘report’ (similar to the one we’ve received after the first deploy), with your endpoints URLs. Now you’ve got everything to actually register your webhook.

You can literally just paste the endpoints into the browser’s address bar. Let’s do it with registerWebhook first. But before we do that, let’s see how we can actually monitor our functions.

If you run that in your terminal, you’ll get a logs report of the last invocation of your function. You can optionally tail the logs and keep listening for new logs by passing this option:

NOTE: That will work only if your functions have been invoked at least once before.

Now you can go to your registerWebhook endpoint URL in the browser. After that, go to terminal and run logs. If you everything is fine, you should see a message:

Repeat the same steps for registerSubscription function. Great! You just registered your twitter webhook.

Since now, any activity on your twitter account will trigger a POST request with all the data about that activity. To see the data, you have to create a lambda function that will handle that request.

Let’s analyse that and see what is actually happening here:

In handleTweet.js file:

NOTE: At line 18 we’re creating a filename from variables — tweet ID and user name and some dashes/underscores. We’re doing it like that to easily get these variables in later part.

In uploadImage.js file:

Handle image upload to S3

Now, you want to set up a lambda function that will fire, every time a new image is uploaded to our bucket. To do it we need to add some config to servereless.yml

Let’s take a look at respondToTweetWithImage function.

Let’s analyse that and see what is actually happening:

And that’s it. You’ve managed to create a Twitter Bot that will automatically recognize the image, and respond with a correct message, accordingly to posted picture. I encourage you to play even more with the functionality — recognize different type of images, create more specific messages etc. This example was just a brief overview of serverless, and how you can build stuff with it without almost zero backend knowledge.

If you have any comments or think something might be wrong, please send me a message or leave a comment.

Add a comment

What is your view?

Send

Related posts:

Abnehmen yokebe

Abnehmen leicht gemacht mit Yokebe. Jetzt im günstigen Sparset bestellen. 10% Neukundenrabatt. Schnelle Lieferung. Kauf auf Rechnung möglich. Goldene Tipps damit du Einfach und Schnell Abnehmen… Read more

GAME PLAY

Mech Master is a first-ever 3D Turn-based Mecha Blockchain game, with strategic RPG elements, where you can earn tokens through skillful gameplay and smart trading. Mech Master is not only an… Read more

Something New Chapter 1

I felt all dried up. Finished. Which was really stupid. I’m only 58, I have a long way to go. Yes, I’m older, but there are plenty of older vibrant folks out there. That’s when I met Dave. I wasn’t… Read more