Node-RED Hosting in the Cloud With a Twitter Feed Analyzer
Node-RED can serve as the glue for a tweet analysis platform, using the cloud to host your data while tweets are measured for their contents' tone.
Join the DZone community and get the full member experience.
Join For Freedue to applications and components in the global iot ecosystem getting more and more interrelated, it becomes essential to be capable of wiring them together to achieve automation. advanced connectivity of systems allows us to eliminate repetitive programming tasks and therefore free time for research and development.
one of the tools to make different systems interact with each other is node-red – a simple programming tool for binding online services, hardware devices, and apis in an interactive and visually appealing fashion. it provides a browser-based editor and a wide set of nodes to build and deploy various functional flows.
within this article, we’ll take a look at a solution we’ve implemented for a quick node-red delivery and will deploy a sample flow of a twitter streaming data analyzer to store and alert upon posts on any theme you are interested in.
a simple way to run node-red instance within a cloud
at jelastic, the ready-to-go node-red development environment is provided as a single-click installation package. it is available within the jelastic jps collection on github alongside with other preconfigured solutions and could be installed on any platform of 4.9.5 jelastic version and higher.
all the required resources and settings for running node-red on jelastic are declared within the package manifest.jps file, where the main points to consider are:
- nodered/node-red-docker : docker image with the latest tag is used as a solution base
- the created node-red instance is assigned 16 dynamic cloudlets , which enable it to scale vertically up to 2 gib of ram and 6.4 ghz of cpu.
- the automatically attached endpoint allows to access node-red the admin panel through a randomly allocated port via shared load balancer (eliminating the necessity to purchase a public ip address)
so, let’s bring it to life – log into your
jelastic cloud
account and proceed with the following steps.
within the main dashboard page, click import at the top and switch to the url tab. here, insert link to the corresponding manifest.jps file:
https://github.com/jelastic-jps/node-red/blob/master/manifest.jps
click
import
to continue.
in the opened installation dialog box:
- type environment as the name to be used as its internal hostname.
- optionally, specify environment display name (so-called alias , to be shown within the dashboard and ssh terminal)
- select an environment region (if multiple ones are available)
click
install
and wait a bit to be shown a notification message on the process completion.
click
open in browser
to access the node-red development environment and start creating your first project (so-called
flow)
.
we’ll learn how to interact with this wizard through a hands-on example of a twitter content tracker – building such a useful and practical tool will also help you to get started with node-red as quick as possible.
deploying twitter feed analyzer as a node-red flow
so, the application we are going to build is intended to search through twitter feeds based upon a specified criteria (keyword, hashtag, and/or username), conduct sentiment analysis of observed posts, and process them accordingly. depending on the message's tone, action should be:
- upon detecting negative content: alert via email message.
- for positive and neutral results: store a mention within a dedicated file inside a container for each type of characteristic separately
for analysis, we’ll use #cloud , #cloudcomputing , and #paas hashtags as a base for the corresponding hosting area investigation. obviously, you are free to specify any other search parameters depending on your requirements – e.g. your company name.
the whole application structure will look like below:
here, the following component types (nodes) are involved:
- twitter – used to authenticate your twitter account and specify search keywords.
- sentiment – designed to analyze posts and return a corresponding value (negative, positive or neutral).
- switch – intended to evaluate sentiment scores and take a corresponding action according to the defined tone.
- two file nodes – provided to store positive and negative messages.
- email – allows receiving instant alert messages with negative feedback.
altogether, such a flow enables you to quickly react to negative feedback and, simultaneously, to learn about user experience, best practices, news, and events within neutral and positive posts. this way, building such a tool will help you to get relevant information in time and stay tuned to what’s happening in the market.
so, below, we’ll consider the parameters that should be adjusted for each of these node types and consequently implement the above workflow.
twitter node
within the node-red flow editor, expand the
social
section and drag and drop the
twitter
icon to place it on the sheet. this will be the very base of our application.
double-click the picked node and specify the following information in the opened dialog box:
- twitter id – click on the pencil icon to authorize the app with your twitter account login and password ; after this operation successfully passes, select add to finish
- search – choose an area for conducting search, for example, all public tweets
- for – specify a search term (keyword, hashtag, and/or username); multiple entries сould be separated with spaces (for the bot to look for their joint mention) or commas without spaces (to detect separate terms)
-
name
– optionally, give some custom name to the node (or leave the default one)
select done above to proceed.
tip:
for more comprehensive feedback analysis, append an additional
debug
node to the main
twitter
one so that fetched posts can be reviewed directly through the editor.
as a result, records of all semantic types will be available within the sidebar
debug
tab, being stored within its default
msg.payload
log.
sentiment node
now, expand the
analysis
section and choose a
sentiment
node – it will analyze found posts and return a corresponding
sentiment
value to determine whether a tweet is negative, positive, or neutral.
this component requires no additional adjustments unless you’d like to change its name (otherwise, double-click this block, specify the preferred denomination, and click
done
).
switch node
now let’s add another
switch
element from the
function
section. it is intended to evaluate the received sentiment score and take a corresponding action according to the defined tone.
tip: the average sentiment score ranges from -5 (which means a very negative tweet) to +5 (a very positive feedback), with 0 being a neutral value. scores outside this range are considered as extreme values.
to adjust the switch node configs, double-click it and provide the following details in the opened dialog box:
- name – pick a custom node name (or leave the default one)
-
property
– specify the
msg.sentiment.score
value and
+add
the following rules to determine which sentimental value should be considered as
neutral
,
positive,
or
negative
respectively:
- == 0
- > 0
- < 0
select
checking all rules
at the bottom and click
done
.
file nodes
now, we are going to set up logging of the observed neutral and complimentary messages. but before the corresponding node insertion, you need to create files for such mentions to be stored in – thus, open the
config
manager for your node-red container and enter the desired destination directory.
here, select new file within options menu, type a name for it (e.g. neutral ) and press enter . repeat these actions to create another file for positive tweets.
note: your application should have sufficient permissions to access and write to the created files. to ensure this, connect to your node-red container via ssh gate as follows:
- generate an ssh keypair.
- add your public ssh key to the dashboard.
- access your account via ssh protocol.
once the connection is established, run the
chown node-red {filename}
command within the destination directory.
then, expand
storage
wizard section and drag the comprised
file
node to the sheet.
double-click this block to adjust the appropriate parameters:
- filename – specify a full path to a destination storing file.
-
action
– select
append to file
to add new tweets to the end of a file. optionally, tick the following additional options are available for this settings:
- add newline (\n) to each payload? – for a new line to be added to every message (can be turned off if required)
- create directory if it doesn’t exist? – for a new directory to be created if it doesn’t exist
- name – enter the index number of the required switch node condition so that tweets with the corresponding semantic value will be written to this file (e.g. 1 for neutral posts in this example)
click
done
to confirm. repeat these operations to configure one more
file
node that will process and save
positive
tweets.
email node
the last required
email
component could be located under the
social
section.
place it within the work area and double-click to provide the following details:
- to – type an email address the alerts on negative tweets should be sent to
- server & port – leave the default values ( smtp.gmail.com host and 465 port number correspondingly)
- userid & password – provide credentials to an email account from which the notifications will be sent
- name – type the index number of the corresponding switch node condition according to the post type you’d like to receive the alerts on (in our case, semantic range for negative tweets was defined third)
note: to ensure that the email node will definitely work properly, your email account (i.e. the one alerts will be sent from) should meet the following requirements:
- have the access to less secure apps setting turned on
- being protected with a strong 8-character password (containing uppercase/lowercase letters, numbers, and symbols)
when you are finished, click
done
.
note: due to inherent twitter api limits, you could be shown the rate limit hit warning, appearing under the twitter node, if changing nodes settings and/or clicking deploy too often. this does not affect the actual flow operability – just wait a minute for this warning to be replaced with your search keyword(s) instead.
checking results
so, your self-made twitter bot has started to work, tracking the current posts and processing the information you’ve asked about. subsequently, results of this analysis could be checked as follows:
-
for
neutral/positive
mentions –
click
config
for the node-red instance within your jelastic dashboard and navigate to the appropriate file; its content can be reviewed just through the embedded file manager:
-
for
negative
tweets – just track new incomings within your email box
that’s it! the node-red flow is now up and running to analyze streaming data from twitter.
Published at DZone with permission of Tetiana Markova, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments