Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Applications for Tarantool, Part 3: Testing and Deployment

DZone's Guide to

Applications for Tarantool, Part 3: Testing and Deployment

In this article, we’re going to take a look at quality assurance in Tarantool applications.

· Database Zone ·
Free Resource

Discover Tarantool's unique features which include powerful stored procedures, SQL support, smart cache, and the speed of 1 million ACID transactions on a single CPU core!

A Tarantool application is a collection of stored procedures used as an API. Data is processed alongside the stored procedures, which allows for a significant performance boost. However, maintaining the stored procedures can easily become a nightmare.

So, in this article, we’re going to take a look at quality assurance in Tarantool applications. Specifically, we’ll talk about testing, deployment to production, using connectors, and migrating a data schema.

If you’ve missed parts 1 and/or 2 of this tutorial series, they can be found here and here. And let me briefly remind you that all of the examples I’m using in the series are based on an application for user authorization called tarantool-authman.

Testing

Testing is just one of many ways of improving the quality of your application.

With regards to stored procedures, functional tests are a good choice for keeping an API up to date. In Tarantool, testing is done using a built-in module called tap, which provides several methods for checking and comparing Lua objects. Let’s test a couple of user registration scenarios that both use auth.registration(email).

local case = {}

local tap = require(‘tap’)
local response = require(‘authman.response’)
local error = require(‘authman.error’)
local db = require(‘authman.db’)
local config = require(‘test.config’)

-- Initializing the application for testing
local auth = require(‘authman’).api(config)
local test = tap.test(‘registration_test’)

-- Clearing the space before each test run
function case.before() db.truncate_spaces() end
function case.after() end

function test_registration_success()
  local ok, code
  ok, code = auth.registration(‘test@test.ru’)

  -- Verifying the registration is successful and the returned value is a string
  test:is(ok, true, ‘test_registration_success user created’)
  test:isstring(code, ‘test_registration_success code returned’)
end

function test_registration_user_already_exists()
  local ok, code, got, expected, user
  ok, code = auth.registration(‘test@test.ru’)

  -- An API method for activating a user with a password
  ok, user = auth.complete_registration(‘test@test.ru’, code, ‘password’)

  -- Verifying the value corresponds to the expected error
  -- The method returns two values, so to simplify the check, the call is wrapped in a Lua table initialization
  got = {auth.registration(‘test@test.ru’), }
  expected = {response.error(error.USER_ALREADY_EXISTS), }
  -- is_deeply checks for matching keys and values in the two Lua tables
  test:is_deeply(got, expected, ‘test_registration_user_already_active’)
end

case.tests = {
  test_registration_success,
  test_registration_user_already_exists
}

return case

Now, we’re going to write a short script for running the tests. Using the before and after methods will make the whole testing process easier.

-- Launching Tarantool
box.cfg {
  listen = 3331,
}

local TEST_CASES = {
 ‘test.case.registration’,
}

function run()
  for case_index = 1, #TEST_CASES do
    local case = require(TEST_CASES[case_index])

    for test_index = 1, #case.tests do
    -- Running a particular test
      case.before()
      case.tests[test_index]()
      case.after()
    end
  end
end

run()

The results are as follows:

$ tarantool test/authman.test.lua
…
TAP version 13
ok — test_registration_success user created
ok — test_registration_success code returned
ok — test_registration_user_already_active

Deploying an Application Instance

Our application is now ready and tested, so it’s time to deploy it to production.

To manage Tarantool instances, we’ll be using the built-in tarantoolctl utility. Let’s first create an application instance, add a user, and grant the user some privileges.

box.cfg {
  listen = 3331;
}

local function bootstrap()
  box.schema.user.create(‘my_user’, {password = ‘123’})
  box.schema.user.grant(‘my_user’, ‘read,write,execute’, ‘universe’)
end

box.once(‘init_user’, bootstrap)

config = {
  -- Application configuration
}

-- Declaring a global auth variable
auth = require(‘auth’).api(config)

Now, we’ll create a symlink to our file from the instances.enabled directory:

sudo ln -s /etc/tarantool/instances.available/auth.lua /etc/tarantool/instances.enabled/auth.lua

Next, let’s launch tarantoolctl and see if we can connect to the Tarantool instance and use the methods defined in the application:

$ tarantoolctl start auth
$ tarantoolctl enter auth
connected to unix/:/var/run/tarantool/auth.control

unix/:/var/run/tarantool/auth.control> ok, user = auth.registration(‘ivanov@mail.ru’)
unix/:/var/run/tarantool/auth.control> ok, user = auth.complete_registration(‘ivanov@mail.ru’, user.code, ‘123’)
unix/:/var/run/tarantool/auth.control> user
— -
- is_active: true
  email: ivanov@mail.ru
  id: 8cd27d26–3974–43d6-a2b2–87202664753d
…

If anything went wrong, we can take a look at the logs located in /var/log/tarantool/auth.lua. For more detailed information on server administration, refer to the official documentation.

Connector

One of the advantages of stored procedures is that they provide a common interface to services written in different programming languages.

Consider this example where Tarantool is used asynchronously via a Python connector:

import asyncio
import asynctnt

async def create_user():
  tnt_connection = asynctnt.Connection(
  host=’127.0.0.1', port=’3367', username=’my_user’, password=’123’
  )

  # Asynchronous user registration
  user_data = await tnt_connection.call(
  ‘auth.registration’, [‘example@mail.ru’, ]
  )
  await tnt_connection.disconnect()

# Calling the user creation logic
loop = asyncio.get_event_loop()
loop.run_until_complete(create_user())

Note that it’s also possible to communicate with a Tarantool instance from another Tarantool instance by using the built-in net.box module.

Data Migration

For applications in production, a common task is updating data schemas. For example, suppose that users need to have a new gender attribute.

To accomplish this, we need to create a script that changes all of the data within the auth_user space. This script can be added to the application initialization logic and the box.once function will suppress all subsequent calls.

local fiber = require(‘fiber’)

local function migrations()

  -- Performing migration only once
  box.once(‘20171101_add_gender’, function ()
    local counter = 0
    -- Iterating over the space reduces RAM consumption
    -- Only the current object is held in memory
    for _, tuple in box.space.auth_user:pairs(
      nil, {iterator=box.index.ALL}
    ) do
      local user_tuple = tuple:totable()
      -- Writing new data to the table
      user_tuple[4] = get_user_gender()
      box.space.auth_user:replace(user_tuple)

      counter = counter + 1
    end
  end)
end

migrations()

Even though the migration can take a long time, the main execution thread doesn’t get locked. This is because the replace method does an implicit yield. To explicitly free the main thread, it’s necessary to call fiber.sleep(0). More information on this topic is available in the Tarantool documentation.

What's Next?

Tarantool is constantly evolving.

Here are some additional topics that are worth taking a look at if you’re planning to develop your own Tarantool services:

  •  Sharding and replication. The vshard module.

  •  SQL syntax support in Tarantool. Optimizing stored procedures with SQL.

  •  The Vinyl on-disk storage engine. Vinyl vs. Memtx, i.e. “What do I do if I run out of RAM?”

  •  Optimizing and profiling stored procedures written in C.

Also, let me reiterate that all of the code examples above are taken from the tarantool-authman application, which is also actively growing and can now be used as an OAuth2 server.

Discover Tarantool's unique features such as powerful stored procedures, SQL support, smart cache, and the speed of 1 million ACID transactions on a single CPU.

Topics:
tarantool ,deployment ,lua ,tutorial ,database ,software testing ,qa

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}