Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Tarantool 101, Guide 3: Spawning With Fibers and Channels

DZone's Guide to

Tarantool 101, Guide 3: Spawning With Fibers and Channels

In this tutorial, we will write a program with a single producer fiber and multiple consumer fibers because it's better to use fibers than Lua coroutines.

· Database Zone ·
Free Resource

Compliant Database DevOps and the role of DevSecOps DevOps is becoming the new normal in application development, and DevSecOps is now entering the picture. By balancing the desire to release code faster with the need for the same code to be secure, it addresses increasing demands for data privacy. But what about the database? How can databases be included in both DevOps and DevSecOps? What additional measures should be considered to achieve truly compliant database DevOps? This whitepaper provides a valuable insight. Get the whitepaper

Fibers are a unique Tarantool feature —"green threads" or coroutines that run independently of operating system threads. Fibers nicely illustrate Tarantool's theoretical grounding in the actor model, which is based on the concept of a number of light processes that cooperatively multitask and communicate with one another through messaging. An advantageous feature of Tarantool fibers is that they include local storage.

Fibers vs. Lua Coroutines

Tarantool is programmed using Lua, which has its own coroutines, but due to the way that fibers interface with Tarantool's asynchronous event loop, it is better to use fibers rather than Lua coroutines. Fibers work well with all of the non-blocking I/O that is built into Tarantool's application server and fibers yield implicitly to other fibers, whereas this logic would have to be added for coroutines.

Channels

Essential to fibers are channels, which manage communication between fibers. Fibers get messages from and put messages into channels.

Our Program: Overview

In this tutorial, we will write a program with a single producer fiber and multiple consumer fibers. The producer fiber will generate tasks in our case HTTP calls and will place them as messages into a channel. Each consumer fiber will then take a task out of the channel and execute it. Note that we could easily add multiple producers but have kept it to one for simplicity's sake.

We will work piecemeal in the Tarantool console (into which entire functions can be pasted), but you can see the code in its entirety at the end of the article.

Fiber Basics

Our program will create new fibers, but in fact, everything in Tarantool happens in a fiber — so some are automatically created. Let's verify this. Make sure you have installed the most recent stable version of Tarantool from this link and then start the console with:

tarantool

Next enter:

box.cfg{}    --Tarantool's essential startup command

And:

fiber = require("fiber")  --import the fiber module

To get a list of running fibers, type:

fiber.info() 

To see the currently scheduled fiber, use:

fiber.self() 

As you can see, when the console starts, we are in a fiber named interactive.

Writing Our Program

Our program is quite simple, with only three functions: a template function for the single producer fiber, a template function for the consumer fibers, and the main function, where the fibers are actually created.

We begin our program by importing the module that will allow our consumer fibers to make HTTP calls:

http = require("http.client").new()

Next, we create our channel, which can hold eight messages at a time:

our_channel = fiber.channel(8)  

Finally, we specify the number of consumer fibers:

desired_fibers = 8     

After that, we can define the template function that will be used to create our producer fiber. As mentioned above, it loops through and puts eight different API-calling tasks into our channel:

function producer_fiber()
  for i = 1,desired_fibers,1 do
    task = http:request('GET','https://jsonplaceholder.typicode.com/posts/' .. i)
    our_channel:put(task)
  end
end

Next, we'll set up the template function that will be used to make our consumer fibers, passing in it so that we have a record of our fibers' creation order:

function consumer_fiber(i)
  fiber.sleep(math.random(1,4))
  task = our_channel:get()
  print('hello from consumer number ' .. i)
  print(task.body)
  if our_channel:is_empty() then
    os.exit()
  end
end

The consumer fibers first sleep a random number of seconds between one and four, then fetch a task from a channel. Next, they print their respective numbers and payloads. We are randomly sleeping the consumer fibers so that they will finish out of order, which more clearly demonstrates their concurrent nature. Finally, each consumer fiber checks to see if the channel is empty and will exit the program if it is; this means that the consumer fiber that finishes last will exit the program.

Note that fiber.sleep() is an explicit form of yielding to other fibers (a key aspect of cooperative multitasking) but as mentioned above, there are also implicit yields in Tarantool: every data change operation or network access causes a fiber yield and each statement that goes through the Tarantool client also causes a fiber yield.

Now, let's write the main function of the program:

function main()
  for i=1,desired_fibers,1 do
    fiber.create(consumer_fiber, i)
  end
  fiberp = fiber.create(producer_fiber)
end

The main function loops to create the eight consumer fibers, then creates the single producer fiber.

Finally, we call the program:

main()

As you can see from the output, the fibers print their respective numbers and payloads in random order, and the final one exits the program.

I hope that you now have a sense of how to start using fibers in Tarantool. Fibers can be used for tasks that are long-running and/or that require non-blocking execution (video games often use coroutines, for example). There are a couple of other fiber features like conditional variables that will be covered in a future article. And, of course, please contact us should you like to learn more about Tarantool.

The Code in Its Entirety

box.cfg{}

fiber = require("fiber")
http = require("http.client").new()

our_channel = fiber.channel(8)  
desired_fibers = 8

function producer_fiber()
  for i = 1,desired_fibers,1 do
    task = http:request('GET','https://jsonplaceholder.typicode.com/posts/' .. i)
    our_channel:put(task)
  end
end

function consumer_fiber(i)
  fiber.sleep(math.random(1,4))
  task = our_channel:get()
  print('hello from consumer number ' .. i)
  print(task.body)
  if our_channel:is_empty() then
    os.exit()
  end
end

function main()
  for i=1,desired_fibers,1 do
    fiber.create(consumer_fiber, i)
  end
  fiberp = fiber.create(producer_fiber)
end

main()

Compliant Database DevOps and the role of DevSecOps DevOps is becoming the new normal in application development, and DevSecOps is now entering the picture. By balancing the desire to release code faster with the need for the same code to be secure, it addresses increasing demands for data privacy. But what about the database? How can databases be included in both DevOps and DevSecOps? What additional measures should be considered to achieve truly compliant database DevOps? This whitepaper provides a valuable insight. Get the whitepaper

Topics:
tarantool ,coroutines ,channels ,database ,tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}