This is for the Node-js-Denver-Boulder Meetup <3 Cheers!
Miss part 1? Check it out here.
Let’s begin …
Before adding additional functionality to the Node Twitter Sentiment Analysis application, we need to refactor the code. Frankly, there are some mistakes that were made on purpose to highlight an issue that many new developers overlook when first working with Node.
Remember this function from index.js in the routes folder:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
Essentially we’re grabbing the user inputted data, pulling tweets based on the inputs, and then calculating the sentiment of those tweets. The timeout is necessary because of how Node works. Because Node is asynchronous, functions do not block other functions from running. Without the 5 second time-out, the next function will append the results to the DOM without waiting for the function to finish running. Essentially, nothing is appended. Make sense?
Put another way, when functions run that are blocking, they wait there for the result to come back before another function fires. Node, on the other hand, will continue executing the code that comes after it (because it’s functions are non-blocking(, then jump back when the result is available.
So, why won’t a timeout work then?
Again, the code has a function that sends the results in 5 seconds, regardless as to the execution state of the call to twitter. What happens though, if we run the program without a network connection? Or if Twitter is down? Or if we pulled in 10,000 tweets instead of 20?
It’s still going to return results after 5 seconds. This is not what we want, obviously. So, how do we fix it? There’s a number of different methods, none of which fully solve it in an elegant manner. In this post, we’ll look at:
Thanks to Manish Vachharajani for developing the code for this example.
One solution is to use the Async. This is often the go-to solution, since the syntax is simple, it’s totally straightforward, and it uses call backs. In fact, in order to use Async, you must follow the convention of providing the callback as the last argument of the Async function. Thus, for users used to callbacks, this is an extremely easy solution.
Start by installing the package:
In our code we will be using the
map() helper method, which takes an array, a filter function, and a callback. The filter function is an async function that takes a callback.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Test it out here.
Basically, we have an array of names, in lower case, which we are converting to uppercase, then outputting via a
console.log. Let’s say that another function depended on the results of
getInfo was long-running, then the other function could fire before
getInfo returned the results. Thus, the need to suspend the function until the results are returned.
We just need to update the index.js file in the “routes” folder:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105
What’s going on?
Let’s look at the specific changes:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
We pass in the
choices array, the
getAndScoreTweets() function (which handles the calculating of sentiment), then the results are serialized and sent back to the client.
async.map() suspends the
getAndScoreTweets() function until it’s done running. Thus, the results are not sent back to the client until Sentiment is done.
async.map() allows you to do a long delay operation on each array element because of the fact that the mapped function must call “callback” - which happens in the
Check out the final code here: https://github.com/mjhea0/node-twitter-sentiment-async
Thanks to Richard Lucas for developing the code and writing the following explanation.
$.Deferred object. They are very similar.
What are promises (from the Q documentation)
If a function cannot return a value or throw an exception without blocking, it can return a promise instead. A promise is an object that represents the return value or the thrown exception that the function may eventually provide. A promise can also be used as a proxy for a remote object to overcome latency.
Here are some great resources for learning more about promises
- Promises A+ Spec
- Q Library
- Promisesjs.org - Great introduction
- Promises by Nodeschool.io
- Promises in Node.js
- Using Promises with Q
- Using jQuery Deferreds - Book from O'Reilly
Here the deferred pattern was used, which goes something like this:
1 2 3 4 5 6 7 8 9
You then go on to using the
1 2 3 4 5
How they were implemented
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
The search function. Note the promise chain:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Generators are the new kid on the block, but they look the most promising. Essentially, they make it easy to suspend/pause a function then resume it with the
… also …
As of Node v0.11.3, you must use the
--harmony_generatorsflag for running applications that contain generator examples in order to enable ES6 experimental features - e.g.,
node --harmony_generators app.js.
Let’s look at a quick example.
1 2 3 4 5 6
Next, you can call the function with this line:
Finally, you can generate an object with the returned values by calling
So, how do we add this to our Sentiment project? I’m not sure. :)
1 2 3 4 5 6 7 8
So, how do we ad this to our Sentiment project? I’m not sure. :)
This isn’t a method of handling the non-blocking function issue, but it instead shows how easily update the front end. We are using Async again to address the function issue. Check out the code here.
Thanks to Aaron Vandrey for developing the code and writing the following explanation.
From [10 things to know about KnockoutJS on day one])http://www.knockmeout.net/2011/06/10-things-to-know-about-knockoutjs-on.html)“:
Observables are functions. The actual value and subscribers to the observable are cached internally by the function. You set an observable’s value by passing the new value as the only argument to the function and you read the value by passing no arguments.
We can use these functions to read the values from the form directly, hide and expose DIVs and change text on the screen.
From the KnockoutJS data-binding page:
Knockout’s declarative binding system provides a concise and powerful way to link data to the UI. It’s generally easy and obvious to bind to simple data properties or to use a single binding. … A binding consists of two items, the binding name and value, separated by a colon.
Server Side Code
Combining the functions in our main.js (more on this later), on the client side, with Knockout’s declarative data-binding syntax, we can set up the Jade template in the manner shown below.
In the original Jade template there are placeholder DIVs set up that we then use jQuery to interact with - to display the error messages and results. We also used jQuery to update the styles applied to the DIVs. Since we are using data binding in this example, we will go ahead and set up the DIVs for errors and results and have their HTML and styles in the DOM at all times. Then using the “visible” data binding on the DIVs we can hide and expose them as needed. In the example below we have a couple of data-bind attributes that KnockoutJS will use to handle the two-way communication from the View to the ViewModel and vise-versa.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
In the highlighted text we can see just a few of the many data-binding possibilities.
submit binding will handle both the “click” event of the submit button as well as a user hitting the “enter” key. In the background KnockoutJS will also perform a “preventDefault” so that the form does not attempt to submit the form to the server.
value binding will update the ViewModel with the values entered into the text boxes. A form submit is not needed to consume these values, though in this case we are using a form submit. Alternatively we could use KnockoutJS to
subscribe to the change event for these form values and begin our processing when our inputs passed validation.
text binding will both display values in the View propagated from the ViewModel, as well and send values from the View back to the ViewModel.
enable binding will disable the submit button when the ViewModel reports back to the View that it has results back from the Twitter Sentiment Analysis.
Client Side Code
applyBindings method to enable all the two-way data binding goodness.
1 2 3 4 5 6
In order to grab the two choices from the form we write a small method that will take use the KnockoutJS observable’s ‘no parameter’ signature to return the values.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
The error handling will remain the same, however in the data-binding example we set the value of our
error() observable. The act of setting the value of the error observable causes it to change from being a “falsey” value to being a “truthy” value, which cause the
visible data binding to also change from
visible = false to
visible = true. This changes the visibility of the DIV formatted for error reporting as well as set the text of the specific error we encountered.
If no errors are encountered on subsequent submissions we can set up the array we need in the call to Twitter. We also blank out the
error() observable that will hide the error reporting DIV and also set the
isProcessing() observable to true which will expose the “processing” animation.
We finish up processing the results. This logic to this is essentially unchanged, however, it is shown here to further exemplify how values are set and retrieved in KnockoutJS.
1 2 3 4 5 6 7 8 9 10
The logic required to turn off the “processing” animation, expose the DIV formatted to successful results, and display the results are achieved by manipulating more observables. The
isProcssing() observable is set to false to hide the animation, the
hasResults() observable is set to true to expose the results DIV and finally, by setting the
results() observable to some friendly copy we let the user know the outcome of the sentiment analysis. When writing this value out the page we use the
html binding rather than the
text binding so that we can inject HTML into the copy we are writing to the screen. If the
text binding had been used, rather than the
html binding, the HTML would have been encoded and we would have had the literal string
<strong> written to the screen - which obviously is not what we want in this case.
After submitting this code it we determined that the data-binding could have been used even better by not having an error DIV and a results DIV. By taking advantage of the
css binding and a KnockoutJS
computed observable (an observable that can watch multiple observables and return one value) the Bootstrap class could have easily been changed from
success and the title and copy changed using existing observables.
shouldShowMessages is a computed observable that will return true if either we have an error or if we have results, otherwise it will return false. Similarly,
messageType is a computed observable that will return “error” unless we have successfully received results, at which point it will return “success”.
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
It should be noted that most data-bindings will make a call to
ko.utils.unwrapObservable() behind the scenes. This allows us to make the data-bind safely on both observables and non-observables. However, if you take a look at where the
messageType observable is used you will notice that we are referencing the observable as a function (with parentheses). This is because we are accessing the observable inside an expression.
Thanks to John Rosendahl for help with writing the intro.
Pull requests are welcomed/encouraged/needed. Enjoy!