A few weeks ago I posted the article “Boost the Performance of an AngularJS Application Using Immutable Data”. It shows how to speedup your AngularJS application when having a lot of bindings to big data collections. The idea behind the optimization is quite simple - create a new collection when the data changes. This way you can reduce the watchers execution from
In the post I did simple profiling using the built-in
Date but it didn’t give enough information in exactly which cases it is more suitable to use immutable data and when you should bet on the standard collections. It also didn’t include any information about the garbage collection, although we know that using immutable collections will eventually lead to highly intensive memory management. All the code for the benchmarks and the charts generation is available in my GitHub account.
On ng-conf 2015, Jeff Cross, gave a talk about Benchpress, which fits perfectly in my profiling purposes. Basically, benchpress uses the WebDriver for Selenium, through
protractor to profile your application. It can run multiple samples of the code you want to profile in order to take the mean and give you a meaningful result. It also includes a few output values, most useful in our case are:
scriptTime- the execution time of your script
gcTime- time required for garbage collection of given sample
Benchpress can render the output of the benchmarks as tables, printed onto the
stdout or save it into json files.
With every collection you can have the following primitive operations:
- Create collection
- Read collection
- Update collection
- Delete collection
*But what if:
So, if we strictly want to perform given action only when the structure of the collection or the data inside it have changed (not only the reference) we will need to loop over both collections (old one and new one) and check whether they contains the same data.*
If we delete, update or create additional entries in the collection, this is considered as update. If we use immutable data, a new collection will be returned on each such manipulation:
That said, we can build a sample script, which will be used for profiling. Initially we will create a single collection and attach watchers to it. Once this is done,
protractor will start clicking a button, which will update a random element of the collection. Here is our
SampleCtrl, which is responsible for initialization and handling the update requests:
And the template…
There are three main variables, which has the strongest impact over such benchmark in AngularJS:
- type of the collection (mutable, immutable)
- size of the collection
- count of the bindings added to the collection
In order to get clear understanding of what bindings count and collection size should make us prefer
Immutable.js over the standard collections, I run the benchmarks with the following collections’ sizes:
5, 10, 20, 50, 100, 500, 1000, 2000, 5000, 10000, 100000 and bindings count:
5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100.
Application with 100 bindings might sounds not realistic to you but keep in mind that AngularJS adds bindings for (almost) all directives in your templates (
ng-model, etc.). Each
$digest loop will lead to traversal of your data structures. For more information on this topic review the AngularJS source code or watch “Sasqwatch is Real” by William Scott Moss.
So far so good. The only thing left is to configure benchpress but this is pretty much a straightforward process. You can find my configuration scripts here and Jeff’s here. Note that it is not required to use benchpress with Angular, you can use it with any other framework or even without one.
After you’ve successfully configured benchpress and start protractor you should see something like:
After we run the benchmarks and set the output directory, all the logs will be saved in there, in json format. What we can do is to aggregate the raw results (for example get the mean) and output them in another file or print them on the screen. This seems fine but visual representation is always better. For rendering the data as charts I used node.js with
node-canvas. Later I found that there’s a “hacked” version of Chart.js, which could be run in node. The glue code for rendering benchpress logs onto Chart.js charts seems generic enough and it may get handy to anyone else, so next couple of weeks I may publish it as npm module.
Exploring the Results
The results contain the script running time and the garbage collection time, mostly because of the immutable data.
This metric will have the strongest impact on the application’s performance, since the script time is single threaded. On the charts below are illustrated the script metrics concerning the script running time of the benchmarks above. The horizontal axis (x-axis) shows the bindings count and the vertical axis (y-axis) illustrates the script running time (in ms).
When having a collection with 5 entries the running time of the script using immutable list is almost double the running time of a standard array. When the bindings counts get bigger there are no significant changes in the time required for running the scripts (both immutable and standard array).
Basically when we have 100 bindings (+1 with the watcher added by
ng-repeat) the AngularJS change detection mechanism needs to perform 5 * 100 iterations (in the worst case) in order to verify that the current value of the array is equals to the previous one for the standard array and a 100 comparisons (one for each binding using
===) in order to detect changes in the immutable data. Since we have overhead caused by the creation of new immutable list on change, the running time of the script using immutable data is bigger.
There are no significant changes in the results of the benchmark, which runs script with array/list with 20 entries. The only noticeable difference here is that the running time of the script using standard array gets significantly bigger when we increase the bindings count. Again, when we have 100 bindings and 20 element array we get 20 * 100 iterations in order to detect change, compared to 20 * 5 with only 5 bindings.
This is the first benchmark, which results show better performance of the immutable list compared to the standard one. In case of 100 bindings the immutable list wins!
Here the competition gets ruthless! We see how much bigger the running time of the script using standard array gets when we have large amount of bindings. Anyway, still the standard array looks like the winner.
It looks like the supreme champion in this benchmark is the immutable array! You still have to watch out whether you have only a few bindings to the collection but definitely if you have more than 10 you should consider using immutable data.
After 20 bindings the running time of the script using standard array gets crazy big! You can see how it gets bigger and bigger with every next binding, although the running time of the script using immutable list stays constant. I’d definitely recommend you to use immutable data with such big collections.
Garbage Collection Time
This metric indicates the time required for garbage collection. I’m exposing this metric as secondary one because the user may not notice directly the impact of the garbage collection time during her experience using our application. It is interesting that the information we got from the benchmarks doesn’t imply any significant advantage of the standard array compared to the immutable list.
I will not provide further explanation for the following sections because I think they are self explanatory. However, if you have any troubles understanding them or you think they deserve further discussions you’re welcome to drop a line as a comment or reach me over e-mail (take a look at the About section above).
Some of the ideas brought to us by the functional paradigm, such as: pure functions, referential transparency, immutable data, etc. constantly find broader and broader applications in building user interface. As we explored in details in the two parts of this blog post on using immutable data with AngularJS, immutability could be quite helpful when we have a lot of bindings on a big data collection. However, in the general use case I’d recommend usage of standard arrays.
The immutable data helps us speedup the watchers from
O(n) (looping over the whole data structure) to
O(1) (comparing only references, since on change a new immutable data structure will be created) but it also has its drawbacks:
- overhead of the creation of the new data structure on change
- overhead of the performed garbage collection (which wasn’t such a big deal, as show above)
But can’t we build something, which takes the best of both worlds? Can’t we build a specialized data structure used for super fast data binding, which doesn’t require creation of new instance on change but still allows
O(1) check for change? This is what I’m going to explore in the last blog post of this series.
Parts of the series:
- Boost the Performance of an AngularJS Application Using Immutable Data - Part 1
- Boost the Performance of an AngularJS Application Using Immutable Data - Part 2
- Even Faster AngularJS Data Structures