Collins_shell

Good times with the collins shell

In this section of the recipes guide we'll go through some of the most common or useful examples to get you up to speed with the collins shell.

IPMI Attach

Collins stores the IPMI data for assets which might be usful for occasionally doing something not supported via the collins web UI, like attaching to the IPMI console. Collins shell

Protip Several collins shell commands support an exec option which will execute the specified command for all assets matching the query.

collins-shell asset get TAG \
  --exec='IPMI_PASSWORD={{ipmi.password}} ipmitool -I lanplus -E -U {{ipmi.username}} -H {{ipmi.address}} sol activate'

Notice the use of mustache templates in the exec option. You can use any available asset tags in the mustache template and they will expand to whatever the value happens to be for that particular asset.

The Console

The collins shell console provides an interactive environment for working with your assets. When you're inside the collins console, you're working in a Ruby environment so you have complete access to all the language features of Ruby.

Start a console session

> collins-shell console

Change your context to a PRIMARY_ROLE. Since PRIMARY_ROLE is a tag, and the console knows this, you will get context sensitive information about primary roles

collins / > cd /PRIMARY_ROLE

Now that you're in the PRIMARY_ROLE context, use ls to see what tag values are available.

collins /PRIMARY_ROLE > ls

We're interested in assets with a PRIMARY_ROLE of DEVEL, so let's change context again.

collins /PRIMARY_ROLE > cd DEVEL

Now let's again use ls to list items in this context, but since we know these are assets, let's also format them so they include the hostname and status and only show assets formatted like this that have the phrase 'blake'.

collins /PRIMARY_ROLE/DEVEL > ls --format='{{hostname}} {{status}} {{tag}}' --grep=blake

We have found the asset we're interested to it, let's change context again

collins /PRIMARY_ROLE/DEVEL > cd sl-91016

Now that we're finally in an asset context, the prompt tells us this by ending the prompt with an asterisk. Once you're in an asset context you have a number of new commands and methods available to you.

collins /PRIMARY_ROLE/DEVEL/sl-91016* > ls
collins /PRIMARY_ROLE/DEVEL/sl-91016* > cat -b
collins /PRIMARY_ROLE/DEVEL/sl-91016* > cat /var/log/messages
collins /PRIMARY_ROLE/DEVEL/sl-91016* > cat /var/log/NOTE
collins /PRIMARY_ROLE/DEVEL/sl-91016* > asset.created.to_s
collins /PRIMARY_ROLE/DEVEL/sl-91016* > power?
collins /PRIMARY_ROLE/DEVEL/sl-91016* > reboot!

Context paths are relative, like in a regular shell. Let's check out another asset and get its status

collins /PRIMARY_ROLE/DEVEL/sl-91016* > cd ../sl-102313
collins /PRIMARY_ROLE/DEVEL/sl-102313* > stat

Power Status

Occasionally it is useful to perform batch operations on a large number of hosts. In this recipe we'll use the collins console, the interactive collins shell interface, to find a set of hosts and check the power status on all of them.

Since the collins console is just a Ruby environment, we can use standard Ruby classes to do our bidding. In the example below we fetch all assets matching a particular hostname, grab only the ones that are allocated, then check the power status of those assets.

collins / > ls /HOSTNAME/.*blake.* | {|array| array.map{|a| [a.tag,a.hostname,a.status]}}
collins / > hosts = _
collins / > hosts.select do |host|
collins / *   host[2] == 'Allocated'
collins / * end.map do |host|  
collins / *   [host[0], host[1], collins_client.with_asset(host[0]).power_status]
collins / * end

In the first line we fetch all assets with a HOSTNAME matching the regular expression .*blake.*. The pipe (|) in the line is pry specific and will just feed the results of the ls command to a Ruby block. We map each of the matching assets to an array which contains the tag, hostname, and status of the asset.

In the next line we assign the results of the map to a variable named hosts. Note the use of the underscore, this is a pry specific thing.

After this we select assets with a status of Allocated, then map those assets to their power status.

Protip You can get a sense for what methods are available to you by using tab completion.

LLDP Linecard

Last week a coworker asked me how he could find all assets on a particular line card of a switch. In this recipe we use the LLDP_PORT_DESCRIPTION attribute which will allow us to match the name of the line card (in this case ge-12). We combine that with the LLDP_CHASSIS_NAME to restrict the search to a specific switch. The command is below.

collins-shell asset find --selector=LLDP_PORT_DESCRIPTION:ge-12 LLDP_CHASSIS_NAME:switch01 SIZE:100

Cql

Extreme times with the collins query language

Collins Query Language (CQL) is a lightweight DSL similar to the WHERE clause of a typical SQL query. It is designed to allow freeform advanced search queries constructed out of arbitary boolean expressions on asset attributes.

Currently, CQL can be used to search for assets as well as asset logs. Asset log searching is still in beta and is only availble via a specialized API endpoint.

CQL Syntax

CQL's syntax is a mix of standard SQL and Solr's native query format

Data Types

Attribute values follow a simple type system with the following defined types:

String (currently includes dates and ip addresses)
Integer
Double
Boolean

When new Attributes are created, they default to String type. Currently there is no way to change this, but most CQL features including range queries work on strings just as well as numeric types.

Simple Key/Values

The fundamental components of CQL queries are key/value matches:

TAG = "my_asset_tag"

All keys and string values are case insensitive and extra whitespace is ignored. When values consist of a single word (any string not containing spaces, quotes, or parentheses), quotes can be omitted

All are valid queres:

tag = my_asset_tag

TAG="my_asset_tag"

tag =     "MY_ASSET_TAG"

Almost every piece of data that Collins stores about an asset can be searched, including hardware data and unmanaged tags:

HOSTNAME = "db.foo.net"
DISK_STORAGE_TOTAL = 500107862016
SOME_BOOLEAN_TAG = false

When searching on attributes that may contain multiple values, for example servers with multiple NIC's will have multiple values for MAC_ADDRESS, Collins will return assets that have at least one value that matches the searched value.

Fuzzy Searching String Values

CQL allows for fuzzy searching via wildcards and (very) limited regex support. Currently full-text searching and other Lucene-style text searching is not enabled for most values, but is planned for a future release

Find assets that have any value set for a tag

This will return any asset that has a hostname

hostname = *

and this will return any asset that does not have a hostname:

NOT hostname = *

When string values are enclosed in double quotes, Collins will search for exact matches. When single-word values are unquoted, Collins will search for values that contain the search value as a substring

Find an asset with a specific hostname

hostname = "web-a63fe82c.foo.bar.com"

hostname = ^web-a63fe82c.foo.bar.com$

Find all assets whose hostname contains "foo"

hostname = foo

hostname = *foo*

Find assets whose hostname starts with "web"

hostname = web*

hostname = ^web

Find assets whose hostname ends with ".bar.com"

hostname = *.bar.com

hostname = .bar.com$

While most fields default unquoted values to double-ended wildcards (eg foo is converted to *foo*), some fields default to exact match searching. Currently these fields are TAG, IP_ADDRESS, and IPMI_ADDRESS. This is will be configurable in the next Collins release.

Range Queries

Instead of searching a single value, CQL can search on a range of values by specifying lower and upper bounds:

Find all assets with CPU speed between 0.8 and 2.4Ghz

CPU_SPEED_GHZ = [0.8, 2.4]

Find all assets with at least 250GB of disk storage

DISK_STORAGE_TOTAL = [268435456000, *]

Find all assets with at most 4 memory banks

MEMORY_BANKS_TOTAL = [*, 4]

MEMORY_BANDS_TOTAL <= 4

Find assets with more than 1 disk

NUM_DISKS = (1,*]

NUM_DISKS > 1

The following two queries are equivalent

CPU_COUNT = *

CPU_COUNT = [*, *]

Numeric and date values use their natural ordering. Ip addresses and other strings use lexicographical ordering.

Logical Connectives

CQL permits arbitrary boolean expressions of key/values using the standard AND, OR, and NOT connectives

TAG = 10016 OR TAG = 10033

HOSTNAME = db-* AND (STATUS = unallocated OR STATUS = provisioned)

NOT (STATUS = allocated OR MAC_ADDRESS = 04:7d:7b:*)

When parentheses are omitted, CQL follows the standard order of operations, where NOT is evaluated first, followed by AND and OR.

ATTR_A = A OR NOT ATTR_B = B AND ATTR_C = C

is equivalent to

ATTR_A = A OR ((NOT ATTR_B = B) AND ATTR_C = C)

Sorting and Limiting

Any single-valued attribute can be sorted, either ascending or descending.

How to use CQL

in the Ruby Collins client, the search method performs CQL queries:

The method signature:

    def search query, size = 50, sort = "ASC", sort_field = "tag"

Example:

client.search "HOSTNAME = web-* AND status = Allocated", 50, "ASC", "HOSTNAME"

In the API and in Collins shell, CQL is part of the asset find operation:

curl --basic -u blake:admin:first 'http://localhost:9000/api/assets?query=hostname%20%3D%20web*%20AND%20status%20%3D%20allocated'

./collins_shell asset find --selector="query:hostname = web* AND status = allocated" --size=50 --sort=ASC --sortField=hostname

Puppet terminus

Use Collins as your puppet node classifier and fact terminus

At Tumblr we use puppet to manage our running systems. Typically with puppet you map nodes (hosts) to puppet classes by hostname, or by regular expression. Once you have more than a few hosts you may want to use a node classifier instead.

In this recipe, we'll give you the code to turn Collins into your puppet source of truth. In addition to resolving Collins assets into node classes for puppet, all collins tags for the specified asset are available in your puppet code. This makes it trivial to do things like tag an asset with a USE_ALT_LOGGING=true tag and have puppet do something different based on it.

Looks up the asset by hostname in Collins. If the asset is found, we use the nodeclass tag on that asset as the puppet nodeclass. We also merge in all the tags found on that asset into a puppet parameter named 'collins'. That is, in puppet there is no a variable you can reference as $collins, e.g. if $collins['USE_ALT_LOGGING'] .... Note that the config file referenced as /var/db/collins.yaml expects it to be a hash with keys server, username, password, and environment where server/username/password are collins specific and environment is the puppet environment.

#!/opt/ruby-1.9.2/bin/ruby
require 'collins_client'
require 'uri'

class CollinsTerminus
  attr_reader :config, :collins

  def initialize
    @config  = YAML.load_file '/var/db/collins.yaml'
    @uri = URI::HTTP.build(:host => @config['server'], :port => 8080)
    @collins = Collins::Client.new :host     => @uri,
                                   :username => @config['username'],
                                   :password => @config['password'],
                                   :timeout  => 300,
                                   :strict   => true
  end

  def manifest fqdn
    node = collins.find(:hostname => fqdn, :details => true).first
    nodeclass = node.nodeclass || 'basenode2' rescue 'basenode2'

    manifest  = {
      'environment' => config['environment'],
      'classes'     => {nodeclass => nil},
    }

    if node
      manifest.merge!({
        'parameters'  => {
          'collins' => node.extras['ATTRIBS'].values.first,
        },
      })
    end

    manifest.to_yaml
  end

end

puts CollinsTerminus.new.manifest ARGV.first

Centos

Detailed description on how to get collins up and running on Centos 6.5

Dependencies

This will be all the dependencies needed.

wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
rpm -i rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
yum install -y ruby mysql-server unzip wget redhat-lsb
yum install -y git --enablerepo rpmforge-extras

wget http://libslack.org/daemon/download/daemon-0.6.4-1.x86_64.rpm
rpm -i daemon-0.6.4-1.x86_64.rpm

You need Oracle Java JRE and JCE.

Download JRE from here: Java JRE 6
Download JCE from here: Java JCE 6
Get the files onto your box and run the following:


mkdir /usr/java
cd /usr/java
# replace with your rpm name
rpm -ivh ~/jre.rpm
# replace with your zip name
unzip JCE.zip
cd UnlimitedJCEPolicy
mv -f * /usr/java/latest/lib/security

Preparing mysql

chkconfig mysqld on
service mysqld start
/usr/bin/mysql_secure_installation

Setup Security

Add iptables rules for collins, or turn off iptables if you don't need it on the box.

Remember to disable SELinux if you don't want to use that.

Install Collins

Collins should be installed under /usr/local/collins/current

adduser collins
cd ~
wget http://tumblr.github.io/collins/releases/collins-1.3.0.zip
cd /usr/local
unzip ~/collins-1.3.0.zip
cd collins
# create current dir or make a symlink.
mkdir current
mv * current
cd current
./scripts/setup
# Run the commands the setup script tells you.

Now remember to edit your production.conf file to have what you need.

collins_conf_dir = "/usr/local/collins/current/conf"

To start Collins on boot:

cp /usr/local/collins/current/scripts/collins.sh /etc/init.d/collins
chmod +x /etc/init.d/collins
chkconfig --add collins
chkconfig collins on

Collins Recipes