Aerospike quick start with Node and Docker

Aerospike is a high-performance (flash-optimized), distributed key-value NoSQL database. I was playing around with it and figured I would share my "quick-and-dirty" getting started notes using the Node client and Docker to run the database.

Prerequisites

Docker

Find installation instructions for your platform here. You will want to install the Docker Engine and Docker Machine.

If you're on Windows or a Mac, you can install everything you need by installing Docker Toolbox. If you're on a Mac, you will also need to separately reinstall VirtualBox due to a bug with Docker Toolbox 1.8.1b.

VirtualBox

The docker daemon runs on Linux, so Windows and Mac users will need VirtualBox for running a lightweight linux distribution to host it. Docker Toolbox installs it for you, but if you're on a Mac, you will need to reinstall the latest VirtualBox after installing Docker Toolbox 1.8.1b due to a bug with the bundled VirtualBox version. The latest VirtualBox test build (5.0.x revision 102322) works fine.

Aerospike Server

Once you have docker on your system, you can run the Aerospike server in a container.

The toolbox installation should have created a docker machine called default. Docker machines are linux hosts for the docker daemon and the containers that it starts up. You can list the docker machines on your system with the following command:

$ docker-machine ls

If default is stopped, you can start it as follows:

$ docker-machine start default

Once the machine is running, you can get the Aerospike server docker image used to create a container (remember, containers are just processes that have very sophisticated isolation features).

Aerospike has an official repository on Docker Hub where an automated build process keeps the image up-to-date with the latest version.

The simplest method to get the Aerospike server up and running is the following command to pull the image and start a container running with all of the server ports exposed in the container forwarded to the host machine (which is not your localhost, but the docker machine running on it).

$ docker run -d --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike/aerospike-server

Remember, these ports are exposed on the host and mapped to the container -- but the host is the docker machine, which on a Windows or Mac is running as a VirtualBox virtual machine. Therefore, the IP address is not 127.0.0.1, but the IP address that was assigned to the machine. You can obtain the address with the following command:

docker-machine ip default (or whatever you named your machine)

After the required images are pulled to your local docker host, you can confirm that the aerospike server is running in a container with the following command:

$ docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                              NAMES  
a9f2dbd870eb        aerospike/aerospike-server   "/usr/bin/asd --foreg"   18 seconds ago      Up 17 seconds       0.0.0.0:3000-3003->3000-3003/tcp   aerospike

Aerospike Tools

Finally, you will want to install the Aerospike tools from here.

One of the tools is a command line interface, symlinked to /usr/bin/cli. Taking an example from Aerospike's documentation, use the cli to create an object with the key Aerospike in the test namespace that is part of the default configuration with three fields (name, address, and email):

HOST="$(docker-machine ip default)"

$ cli -h $HOST -n test -o set -k Aerospike -b name -v "Aerospike, Inc."
# succeeded: key= Aerospike  set=   bin= name  value= Aerospike, Inc.
$ cli -h $HOST -n test -o set -k Aerospike -b address -v "Mountain View, CA 94043"
# succeeded: key= Aerospike  set=   bin= address  value= Mountain View, CA 94043
$ cli -h $HOST -n test -o set -k Aerospike -b email -v "info@aerospike.com"
# succeeded: key= Aerospike  set=   bin= email  value= info@aerospike.com

You query for the record like this:

$ cli -h $HOST -n test -o get -k Aerospike
# {'email': 'info@aerospike.com', 'name': 'Aerospike, Inc.', 'address': 'Mountain View, CA 94043'}

And finally, delete the record like this:

$ cli -h $HOST -n test -o delete -k Aerospike
delete succeeded: key= Aerospike  set=  

You can find docs on the cli tool and supported commands here. As noted, it is really intended for basic validation only since it creates a connection for each transaction. For building applications, use one of the supplied client libraries.

Node Client

Once you've verified the Aerospike server is running, you can create a simple Node.js test application.

The Aerospike node client is available on npm. Add the package and save the dependency in your package.json file with the following command:

$ npm install --save aerospike

Unfortunately, the package will not successfully build with io.js versions >= 3. I use nvm for switching node versions, and tested the client with v2.5.0, available here.

$ nvm use iojs-v2.5.0

Aerospike provides sample applications on its GitHub page and in the repo's examples directory. You can find documentation here.

Creating an Aerospike client object

var aerospike = require('aerospike');

var config = { ... };

var client = aerospike.client(config);  

The client object will be used for subsequent Aerospike operations.

The client configuration

We created a single Aerospike container previously. The IP address for the container was obtained as follows (for the default docker machine):

$ docker-machine ip default
192.168.99.100

So the client configuration looks like this:

var config = {  
  hosts: [
    { addr: "192.168.99.100", port: 3000 }
  ]
}

If you had created a cluster, it would look something like this:

var config = {  
  hosts: [
    { addr: "192.168.99.100", port: 3000 },
    { addr: "192.168.99.101", port: 3000 },
    { addr: "192.168.99.102", port: 3000 }
  ]
};

You can augment the configuration object with other settings:

var config = {  
  hosts: [
    { addr: "192.168.99.100", port: 3000 },
    { addr: "192.168.99.101", port: 3000 },
    { addr: "192.168.99.102", port: 3000 }
  ],
  policies = {
    // default timeout for all operations is 100ms
    timeout: 100 
  }
  log : {
     level: aerospike.log.INFO,
     file : fd  // fd  opened by the application using fs.open()
  }
};

You can find more details here.

Client operations

Details on client operations can be found here.

Data is added with objects representing records, explained here. Some operations allow metatdata to be provided with records, as described here.

The Query class provides the interface for all query operations. A query is created with a Statement that has attributes governing query execution for everything from selecting and filtering data, aggregation (map-reduce jobs written as Lua user defined functions), and full database scans.

Working on the client

If you want to check out the or even contribute to the client source, you can fork and clone it from here.

Make sure you are running a version of node/io.js < 3. Change to the clone directory and run:

$ npm install

To verify the tests pass, make sure to use the IP address for the machine hosting your aerospike container:

$ OPTIONS="-h $(docker-machine ip default)" npm test
97 passing (5s)
1 pending