Over a million developers have joined DZone.
Platinum Partner

Happybase Connection Pooling

· DevOps Zone

The DevOps Zone is brought to you in partnership with New Relic. Improving the performance of your app is easy with New Relic's SaaS-based monitoring.

Wrote a simple connection pool for Happybase using socketpool:

import time
import random
import contextlib
import happybase
from socketpool import ConnectionPool
from socketpool.conn import TcpConnector

class HappybaseConnectionPool(object):
    ''' singleton to share a connection pool per process '''

    _instance = None

    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super(HappybaseConnectionPool, cls).__new__(cls, *args, **kwargs)
        return cls._instance

    def __init__(self, host, **options):
        options['host'] = host
        if not hasattr(self, 'pool'):
            self.pool = ConnectionPool(
                max_size=options.get('max_size', 10),

    def connection(self, **options):
        return self.pool.connection(**options)

    def table(self, table_name):
        with self.pool.connection() as connector:
            yield connector.table(table_name)

class HappybaseConnector(TcpConnector):

    def __init__(self, host, port, pool=None, **kwargs):
        self.host = host
        self.port = port
        self.connection = happybase.Connection(self.host, self.port)
        self._connected = True
        # use a 'jiggle' value to make sure there is some
        # randomization to expiry, to avoid many conns expiring very
        # closely together.
        self._life = time.time() - random.randint(0, 10)
        self._pool = pool
        self.logging = kwargs.get('logging')

    def is_connected(self):
        if self._connected and self.connection.transport.isOpen():
                # isOpen is unreliable, actually try to do something
                return True
        return False

    def handle_exception(self, exception):
        if self.logging:
            print exception

    def invalidate(self):
        self._connected = False
        self._life = -1

    def open(self):

    def close(self):

    def __getattr__(self, name):
        if name in ['table', 'tables', 'create_table', 'delete_table',
                'enable_table', 'disable_table', 'is_table_enabled', 'compact_table']:
            return getattr(self.connection, name)
            raise AttributeError(name)

You can get a single pool object from anywhere in your stack with the following:

pool = HappybaseConnectionPool('localhost', '9090')
with pool.connection() as connection:

Note: happybase may be adding their own connection pool shortly.

The DevOps Zone is brought to you in partnership with New Relic. Know exactly where and when bottlenecks are occurring within your application frameworks with New Relic APM.


Published at DZone with permission of Chase Seibert , DZone MVB .

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}