Applications for Tarantool, Part 1: Stored Procedures

First Steps

I’m going to walk you through the creation of a Tarantool application that implements an API for registering and authenticating users. It offers the following capabilities:

• Registration and authentication via email in three easy steps: creating an account, confirming the registration, and setting the password.
• Registration with social network credentials (FB, Google+, VK, etc.).
• Password recovery.

For an example of a stored procedure for Tarantool, we’ll take a look at the first step of email registration: getting a confirmation code. To make it more interactive, you can check out this GitHub page and follow along.

Installing Tarantool

You can find detailed installation instructions for various operating systems. For example, to install Tarantool on Ubuntu, you’ll need to run the following script by pasting it into your console:

curl http://download.tarantool.org/tarantool/1.7/gpgkey | sudo apt-key add -
release=`lsb_release -c -s`

sudo apt-get -y install apt-transport-https

sudo rm -f /etc/apt/sources.list.d/*tarantool*.list
sudo tee /etc/apt/sources.list.d/tarantool_1_7.list <<- EOF
deb http://download.tarantool.org/tarantool/1.7/ubuntu/ $release main
deb-src http://download.tarantool.org/tarantool/1.7/ubuntu/ $release main
EOF

sudo apt-get update
sudo apt-get -y install tarantool

Let’s check that the installation has been successful by typing tarantool and entering the interactive administrator console.

$ tarantool
version 1.7.5-30-g960aad6
type 'help' for interactive help
tarantool>

This is where you can try your hand at Lua programming. If you’re not familiar with Lua, here’s a short tutorial to get you started.

Registration via Email

It’s time to take it one step further and write our first script, which creates a space that will hold all users. A space is analogous to a table in a relational database. Data itself is stored in tuples (arrays holding records). Each space must have one primary index and can have several secondary indexes. Indexes can be defined on a single or on multiple fields. Below is the space scheme for our authentication service:

As you can see from the image, we’re using indexes of two types: HASH and TREE. A HASH index allows finding tuples by a fully matching primary key and must be unique. A TREE index supports non-unique keys, enables searches by the first part of a composite index, and lets us streamline key sorting since key values within an index are ordered.

The session space holds a special key (session_secret) used for signing session cookies. Storing session keys allows logging users out on the server side, if necessary. A session also has an optional link to the social space. This is necessary for validating the sessions of those users who log in with social network credentials (we check the validity of a stored OAuth 2 token).

Writing the Application

Before we start writing the application itself, it’s worth taking a look at the structure of the project:

tarantool-authman
├── authman
│ ├── model
│ │ ├── password.lua
│ │ ├── password_token.lua
│ │ ├── session.lua
│ │ ├── social.lua
│ │ └── user.lua
│ ├── utils
│ │ ├── http.lua
│ │ └── utils.lua
│ ├── db.lua
│ ├── error.lua
│ ├── init.lua
│ ├── response.lua
│ └── validator.lua
└── test
 ├── case
 │ ├── auth.lua
 │ └── registration.lua
 ├── authman.test.lua
 └── config.lua

Paths specified in the package.path variable are used for importing Lua packages. In our case, packages are imported relative to the current directory, that is tarantool-authman. However, if necessary, import paths can easily be extended as follows:

-- Prepending a new path with the highest priority
package.path = “/some/other/path/?.lua;” .. package.path

Before creating the first space, let’s put all the needed constants into separate models. We need to give each space and each index a name. It’s also necessary to specify the order of fields in a tuple. For example, here’s what the authman/model/user.lua model looks like:

-- Our package is a Lua table
local user = {}

-- The package has the only function — model — that returns a table
-- with the model’s fields and methods
-- The function receives configuration in the form of a Lua table
function user.model(config)
  local model = {}

  -- Space and index names
  model.SPACE_NAME = ‘auth_user’
  model.PRIMARY_INDEX = ‘primary’
  model.EMAIL_INDEX = ‘email_index’

  -- Assigning numbers to tuple fields
  -- Note thatLua uses one-based indexing!
  model.ID = 1
  model.EMAIL = 2
  model.TYPE = 3
  model.IS_ACTIVE = 4

  -- User types: registered via email or with social network
  -- credentials
  model.COMMON_TYPE = 1
  model.SOCIAL_TYPE = 2

  return model
end

-- Returning the package
return user

When handling users, we’ll need two indexes: unique by ID and non-unique by email address. When two different users register with social network credentials, they may be assigned the same email address or even no email address at all. As for the users who registered the regular way, our application will make sure their email addresses are unique.

The authman/db.lua package contains a method for creating spaces:

local db = {}

-- Importing the package and calling the model function
-- The config parameter is assigned a nil (empty) value
local user = require(‘authman.model.user’).model()

-- The db package’s method for creating spaces and indexes
function db.create_database()

  local user_space = box.schema.space.create(user.SPACE_NAME, {
              if_not_exists = true
  })
  user_space:create_index(user.PRIMARY_INDEX, {
              type = ‘hash’,
              parts = {user.ID, ‘string’},
              if_not_exists = true
  })
  user_space:create_index(user.EMAIL_INDEX, {
              type = ‘tree’,
              unique = false,
              parts = {user.EMAIL, ‘string’, user.TYPE, ‘unsigned’},
              if_not_exists = true
  })
end

return db

UUID will serve as a user ID, and we’ll be using a HASH index for full-match searches. The index for searches by email will be made up of two parts: (user.EMAIL, ‘string’) — the user’s email address, and (user.TYPE, ‘unsigned’) — the user’s type. As a reminder, the types were defined in the model a bit earlier. A composite index enables searches not only by all the fields but also by the first part of the index. Therefore, we can search by email address only (without the user type).

Let’s enter the interactive administrator console and try to use the authman/db.lua package.

$ tarantool
version 1.7.5-30-g960aad6
type ‘help’ for interactive help
tarantool> db = require(‘authman.db’)
tarantool> box.cfg({})
tarantool> db.create_database()

Great, we’ve just created the first space. One thing to keep in mind here: before calling box.schema.space.create, you need to configure and run the server via the box.cfg method. Now, we can perform some simple actions within the space we’ve created:

-- Creating users
tarantool> box.space.auth_user:insert({‘user_id_1’, ‘example_1@mail.ru’, 1})
 — -
- [‘user_id_1’, ‘example_1@mail.ru’, 1]
…
tarantool> box.space.auth_user:insert({‘user_id_2’, ‘example_2@mail.ru’, 1})
 — -
- [‘user_id_2’, ‘example_2@mail.ru’, 1]
…
-- Getting a Lua table (array) with all the users
tarantool> box.space.auth_user:select()
 — -
- — [‘user_id_2’, ‘example_2@mail.ru’, 1]
 — [‘user_id_1’, ‘example_1@mail.ru’, 1]
…

-- Getting a user by the primary key
tarantool> box.space.auth_user:get({‘user_id_1’})
 — -
- [‘user_id_1’, ‘example_1@mail.ru’, 1]
…

-- Getting a user by the composite key
tarantool> box.space.auth_user.index.email_index:select({‘example_2@mail.ru’, 1})
 — -
- — [‘user_id_2’, ‘example_2@mail.ru’, 1]
…

-- Changing the data in the second field
tarantool> box.space.auth_user:update(‘user_id_1’, {{‘=’, 2, ‘new_email@mail.ru’}, })
 — -
- [‘user_id_1’, ‘new_email@mail.ru’, 1]
…

Unique indexes restrict the insertion of non-unique values. If you need to create some records that may already be in a space, use the upsert (update/insert) operation. You can find the full list of available methods in the official documentation.

Let’s extend the user model with the capability to register users:

function model.get_space()
    return box.space[model.SPACE_NAME]
  end

  function model.get_by_email(email, type)
    if validator.not_empty_string(email) then
      return model.get_space().index[model.EMAIL_INDEX]:select({email, type})[1]
    end
  end

  -- Creating a user
  -- Fields that are not part of the unique index are not mandatory
  function model.create(user_tuple)
    local user_id = uuid.str()
    local email = validator.string(user_tuple[model.EMAIL]) and
                  user_tuple[model.EMAIL] or ‘’
    return model.get_space():insert{
                  user_id,
                  email,
                  user_tuple[model.TYPE],
                  user_tuple[model.IS_ACTIVE],
                  user_tuple[model.PROFILE]
    }
  end

  -- Generating a confirmation code sent via email and used for
  -- account activation
  -- Usually, this code is embedded into a link as a GET parameter
  -- activation_secret — one of the configurable parameters when
  -- initializing the application
  function model.generate_activation_code(user_id)
    return digest.md5_hex(string.format(‘%s.%s’,
                  config.activation_secret, user_id))
  end

The code snippet below uses two standard Tarantool packages — uuid and digest — and one user-created package — validator. Before you can use them, they need to be imported:

-- standard Tarantool packages
local digest = require(‘digest’)
local uuid = require(‘uuid’)
-- Our application’s package (handles data validation)
local validator = require(‘authman.validator’)

When declaring variables, we’re using the local operator, which limits their scope to the current block. Otherwise, these variables will be global, which is what we want to avoid because of potential name collisions.

Now, let’s create the main package — authman/init.lua — which will hold all of the API methods:

local auth = {}

local response = require(‘authman.response’)
local error = require(‘authman.error’)
local validator = require(‘authman.validator’)
local db = require(‘authman.db’)
local utils = require(‘authman.utils.utils’)

-- The package returns the only function — api — that configures and
-- returns the application
function auth.api(config)
  local api = {}
  -- The validator package contains checks for various value types
  -- This package sets the default values as well
  config = validator.config(config)

  -- Importing the models for working with data
  local user = require(‘authman.model.user’).model(config)

  -- Creating a space
  db.create_database()

  -- The api method creates a non-active user with a specified email
  -- address
  function api.registration(email)
    -- Preprocessing the email address — making it all lowercase
    email = utils.lower(email)

    if not validator.email(email) then
      return response.error(error.INVALID_PARAMS)
    end

    -- Checking if a user already exists with a given email
    -- address
    local user_tuple = user.get_by_email(email, user.COMMON_TYPE)
    if user_tuple ~= nil then
      if user_tuple[user.IS_ACTIVE] then
        return response.error(error.USER_ALREADY_EXISTS)
      else
        local code = user.generate_activation_code(user_tuple[user.ID])
        return response.ok(code)
      end
    end

    -- Writing data to the space
    user_tuple = user.create({
      [user.EMAIL] = email,
      [user.TYPE] = user.COMMON_TYPE,
      [user.IS_ACTIVE] = false,
    })

    local code = user.generate_activation_code(user_tuple[user.ID])
    return response.ok(code)
  end

  return api
end

return auth

Great! Now users can create accounts.

tarantool> auth = require(‘authman’).api(config)
-- Using the api to get a registration confirmation code
tarantool> ok, code = auth.registration(‘example@mail.ru’)
-- This code needs to be sent to a user’s email address so that they
-- can activate their account
tarantool> code
022c1ff1f0b171e51cb6c6e32aefd6ab

That’s it for now. The next article will be about using ready Tarantool packages, networking, and implementing OAuth 2 in tarantool-authman.

Thanks for reading and stay tuned!