javascript dependency management

Hello fellow geek,

A while back, I wrote an article about javascript dependency management, explaining my discomfort with most options out there. I tried filling the void with something meaningful by creating useuses; a gulp plugin that adds zero extra bytes to your combined files but does add meaning to your individual files.

I came to realise, that this plugin was flawed in a couple of ways, so naturally I’ve fixed those flaws. And thus I stand here today to proudly present a new module that takes a different approach but still uses the same solution.

@uses

But before I do, I’d like to explain what @uses is supposed to mean, why I think it’s better and how it should be applied.

What’s @uses?

@uses is an annotation I’ve been using for a while now. It allows me to keep track of what dependencies my files have easily.

How do I use it?

Here’s what it looks like:

As you can see, it unobtrusively informs the developer about the dependencies being used by this particular file. So using it is really simple: just maintain the dependencies of your file in the annotations, and you’re already doing it.

So, why is it better?

The arguments I’ve presented in my previous blog post still stand. In summary:

  • It doesn’t add extra bytes
  • It doesn’t force a dependency upon you
  • It doesn’t use weird tags, or magic. Just an annotation.
  • I adds value to your files.

Improvements

Like I mentioned earlier, I made quite some improvements to the module. Here are the biggest ones.

It’s not a grunt plugin anymore

The biggest flaw was that it forced you to use gulp. There was no standalone version, or alternative for let’s say gulp.~ Useuses itself is now a standalone module that can be used via the CLI, or programmatically.

This means it can be used in grunt, gulp, or any environment for that matter.

It’s faster

I’ve made improvements in the file searches, sorting of dependencies and making sure there aren’t any duplicates. These improvements have made this module a lot faster.

It’s lighter

The module does its job by utilising streams now, not buffering the entire file, for every file. This means that its memory footprint is a lot smaller.

Using it

You can find the module here:

Github: https://github.com/SpoonX/useuses
NPM: https://www.npmjs.org/package/useuses

Sails.js count blueprint

I’ve been in need of a count blueprint for a while now, so I decided to just go ahead and make one. IF you want to use it, feel free. Here’s the code:

Documentation for everything

Hello fellow geeks!

Today I’d like to share a nice piece of software that has boosted my productivity by 200%. It’s a tool that allows you to download documentation from a lot of libraries, frameworks, programming / scripting languages, stack overflow, packagist and more. It’s called Dash (for mac) and Zeal docs for Windows / linux. For this post I will be talking about Dash specifically, but most of it also applies to zealdocs.

What it does

Like they say themselves:

Dash gives your Mac instant offline access to 150+ API documentation sets.

It integrates with the system (shortkey) and a lot of editors / other systems. As an example, in your editor, you can select some text (for instance, Angular’s $http) and push the shortkeys for Dash (defaults to cmd+shift+d), which will bring up the documentation on $http for angular.js. Are you writing php, and are you unsure on what strstr or stristr do? Select that, press the shortkeys and boom, documentation.

Like I said, it has documentation for a lot of things. Node.js, PHP, javascript, ruby, Zend Framework 2, Laravel, and much, much, much (I can keep going like this) much more.

It also supports Packagist and automatically builds documentation for all packages you wish to use, if there aren’t any. And it supports stackoverflow by downloading answered questions which you can search.

Download

It’s offline, fast, has everything in one place and doesn’t force you to wait for the browser. It’s just cool. Check it out:

Mac: Dash

Windows/Linux: Zeal docs

Javascript censoring text

Yesterday I released a new module. I thought about it for a bit and decided to write about it.
The reason for this, is that I want to elaborate the reason for building this module, and at the same time perhaps supply a use-case to you.
I’m firmly against censoring anyone or anything, so I hope this module, if at all, gets used for the good.

Why I built it

We’re running and providing some very high-traffic communications services, one of them being chat.
The chat doesn’t allow you to share contact information (for security purposes) and so we needed a way to both sensor occurrences, and be notified whenever it happened.
When receiving notifications we also wanted to see the offending matches highlighted, and for that reason, I’ve added the highlight functionality.

The module was built with performance (regex, rather complex can become a slowing factor on high traffic messaging systems) and ease of use in mind.
It’s not rocket science, so it doesn’t need a freakishly large and complicated engine.

And thus, behold, the censoring module was born.

First, some code

Code speaks more than… Me. So here’s an example:

What it does

The module is compact, but does what it should do very well. What it does:

  • Supply a default set of filters
  • Supply the possibility to censor words
  • Supply the possibility to highlight matches (found abuses)
  • Detect words, even if they’ve b33n 13373d…
  • or s.p.l-i*t up.

What it doesn’t

To make sure you don’t get the wrong idea, I want to clear up some (already) misconceptions.

  • It doesn’t censor a predefined set of words.
  • It doesn’t actually censor by default.
  • It doesn’t swap words.

Where can I find the module?

Github: https://github.com/SpoonX/Censoring

NPM: https://www.npmjs.org/package/censoring

Installation is simple:

npm install --save censoring

Q&A: sails.js deploy app and run on port 80

Question

<sickyjim> hi all, I’m trying to run an app on port 80 instead of 1337 but when I change the port in config/env/production, the app launches but I can’t reach it from a browser :/

Answer

This could be caused by a couple of things. We’ll go through each and every one of them. If it’s still not working for you, leave a comment down below and I’ll update the list with more checks.

Port 80 < 1024

When running an application on a port < 1024, you’ll need su access. These ports are “special”, and thus protected behind sudo.

Now please don’t run your application on sudo. In stead, run it on any port > 1024, and in your firewall forward port 80 to the chosen port. For this example I’ve chosen the default port 1337.

Port in use

Something else might be using port 80. Common ones are:

  • Apache – sudo apachectl stop
  • Nginx – nginx -s stop
  • Skype – preferences > Advanced: Change Incoming connections port

If all else prevails, run the following command (for your OS) to find out what’s running on port 80:

MacOS: sudo lsof -i ':80'

Linux: netstat -tulpn | grep :80

SPI protection

It might also be SPI firewall protection blocking your application from functioning. You can read more about that here: SPI Firewall protection on router.

Q&A: Deploying your sails.js application

Question

<ebsarex> hi, I just finished developping a sailsjsapp and would like to deploy it on a dedicated server of mine, does anyone know about a guide to do so or something like this ?

Answer

Yes. The deployment guide for sails can be found here, in the docs but also here on github if you prefer that.

If you’re using a specific hosting provider such as Modulus, NodeJitsu, OpenShift or one of the many others, you can take a look here as well.

Q&A: Sails.js get model definition for enum

Question

<leolrrj> I’ve enum at my model and I want to retrieve it values… there’s a way?

Answer

Sails stores the models under sails.models. For those of you that have the models option set to true in the globals.js config, User.find() could also be written as sails.models.user.find().

Once initialised, the model becomes an object where one of its properties is definition. So in order to retrieve the model’s definition, and in this specific case, the values for the enum, you could simply use sails.models.user.definition.enumfield to retrieve the values.

Survival guide: Many websocket connections

When working with sails.js, or indeed any software that accepts websocket connections, you have to be careful you don’t start rejecting requests. It’s all about finding the perfect balance between CPU, memory and file descriptors. Unfortunately, there is no such thing as a “one size fits all” solution in this case, because it all depends on your application.

I’ll try to highlight the importance of certain aspects by providing examples with them. I’ll explain why, when and how they’re important, providing possible solutions with them as well. For this article I’ll assume you’re using sails.js, but the suggestions given in this article apply to any application. Let’s get to it.

Scaling

Scalability is something to always keep in mind. You can read more about scaling sails.js here.

Horizontal scaling

You can read about this more by reading this article. To me, this is one of the most important things to keep in mind while setting up your server architecture.

The most obvious way to make scaling possible, to me, is by using redis as the session store and pubsub adapter. That way, you’re not memory-bound and you can simply add instances and VMs, where you apply load balancing.

System scaling

This is a pretty simple thing to set up. Using PM2 you’ll be able to scale your application over multiple cores on a single machine. Sometimes, an extra core is cheaper that an extra VM. This all depends on how much memory you’ll need to add, and how much CPU power is actually being used.

Resources

It’s crucial to know exactly how much memory you need, and to keep an eye on how much you’re using. If you have a memory leak, or a sudden increase in activity your application might crash (which can obviously be recovered using forever.js) your application might crash, and we don’t want that to happen. In general, you should be careful with what you store in memory. Just as with memory, it’s also equally as important to keep an eye on your CPU power.

File descriptors

File descriptors, as taken from wikipedia:

In computer programming, a file descriptor (FD) is an abstract indicator for accessing a file.

Everything, even network connections use FDs. By default, the limits set for them are quite low making it quite probable you’ll find yourself looking at errors that only show up in your /var/log/messages log, and don’t make a whole lot of sense. Capping this limit, will cause connections to drop, communications to fail and basically ensures chaos.

Viewing the limits

There are two types of FD limits: system limits and user limits. The system limits can be viewed by running:

the user limits can be viewed by signing in as the user (su username) and running for the hard limit:

And for the soft limit:

Changing the limits

To change the system limits, run:

Open up vi /etc/sysctl.conf and add the following at the end of the file: fs.file-max = 200000.

Finally, run sysctl -p.

For the user, open the file vi /etc/security/limits.conf and add the following lines (replacing username with the username you’re using):

Save the file, and done.

The right limits

Figuring out what the right number is for your server depends on your application. If you notice you’re capping on file descriptors, with plenty of bandwidth, memory and CPU left it’s probably safe to increase the number by quite a bit.

Sails.js enable redis sockets

Socket.io allows you to use redis for sockets. Because sails.js uses socket.io, it consequently also allows you to set up redis for sockets. Enabling this behaviour is relatively easy.

To enable this, open up config/sockets.js and look up this block:

Now uncomment the options, configure the adapter and you’re done.

Q&A: sails.js enable jsonp

Question

This is a question I had myself. Why, if I call /foo?callback=bar do I not get a jsonp response?

Answer

Out of the box, sails doesn’t enable jsonp support. In order to enable this, you have to change config/blueprints.js and add jsonp: true.

©SpoonX 2014