The Startup Stack

The Startup Stack

The Startup Stack

Fork me on GitHub

Monitoring in the-startup-stack is provided by Sensu.

Sensu can monitor basically anything and it has a very intuitive DSL (Ruby). You can verify KPIs, you can monitor speed, disk, CPU and even less obvious stuff like a server trailing behind in deployment.

Sensu also has loads of community driven plugins that you can checkout.

Sensu is built on a messaging architecture, driven by RabbitMQ. Here's an overview of the sensu architecture.

On by default

Sensu is part of the base role which comes on every server, the base role includes the sensu_client role.

For very server you bootstrap that gets the base role, you will have monitoring on by default and every check you define will run on that server asw well.

The Sensu Recipe


The default recipe for sensu will install and define the client service, install community plugins and configure what's needed.

Each of the clients for sensu needs a subscription, by default the recipe will include some.

Those subscriptions include:

Lets take a backend instance for example. The backend instance will have backend as the stack_role, production as the chef environemt.

This means, that this instance will subscribe to all checks that include the backend and production as well as the instance hostname which is prod-be1 for example.

Of course the default all and the strings defined in node['sensu']['subscriptions'] as well.


The default recipe defined some attributes


The checks recipe comes with some default checks

Sample checks

Checking if your sitemap is fresh

Many sites generate the sitemap.xml in a cron job. Making sure that file is fresh on a specific server (Every other server proxies to this one to get the fresh one).

sensu_check "check_sitemap_fresh" do
  command "check-mtime.rb -f /mnt/data/your-app/current/public/sitemap.xml -c 90000"
  handlers ["slack"]
  interval 3600
  subscribers ["specific-hostname-for-server"]
  additional(:notification => "Sitemap did not run for 25 hours")

Checking if a specific process is running

sensu_check "check_warm_cache" do
  command "check-procs.rb -p warm_cache.rb -C 1"
  handlers ["slack"]
  interval 600
  subscribers ["specific-hostname-for-server"]
  additional(:notification => "Warm cache is down")


Sensu comes with many plugins (You can browse the Sensu Community Plugins).

The plugins recipe will flatten all the plugins and handlers into a single directory to make sure you don't need the full path in order to call the DSL (Trust me, it will make your life easier).

Often, you would want to make custom plugins for yourself as well. If you do, here's what you'll need to add to the plugins recipe.

 stack_plugins_directory = "/opt/sensu-stack-plugins"

git stack_plugins_directory do
  repository ""
  reference "master"
  action :sync

ruby_block "flat stack plugins" do
  block do
    ["plugins", "handlers"].each do |dirname|
      Dir.glob("#{stack_plugins_directory}/#{dirname}/**/*").each do |name|
        new_file_name = "#{node['sensu']['directory']}/#{dirname}/#{File.basename name}"
        File.symlink name, new_file_name unless File.symlink?(new_file_name)

Bootstrapping a sensu server

$ export MY_IP=`curl -s | sed -e "s/.*Current IP Address://" -e "s/<.*$//"`">`
$ terraform plan -var key_name=production -var your_ip_address=$MY_IP -target=aws_instance.sensu

Once you review the plan and it makes sense, you can apply your changes to the stack.

$ terraform apply -var key_name=production -var your_ip_address=$MY_IP -target=aws_instance.sensu
$ bin/knife bootstrap $SERVER_IP -r "role[base],role[sensu_server]" -E production -x ubuntu --sudo

That's it, now you have a sensu server and you are ready to start monitoring your servers and services.

Edit This Page | Page History