Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Diving into Couchbase Index Replicas

DZone's Guide to

Diving into Couchbase Index Replicas

· Database Zone ·
Free Resource

RavenDB vs MongoDB: Which is Better? This White Paper compares the two leading NoSQL Document Databases on 9 features to find out which is the best solution for your next project.  

With Couchbase Server 4.x, customers used to create Equivalent Indexes to satisfy the twin requirements of keeping the indexes highly available and to load balance the N1QL queries. What this meant was that the exact same index definition was used to create indexes with different names. What’s in a name you might ask… a rose is a rose is a rose :)

 create index index1 on bucket(field1);

create index index2 on bucket(field1); 

N1QL queries could hit either of the two indexes based on the load seen by each of these indexes; and also, if either of the indexes were to go down, the other index would continue to serve the query traffic. There were a few challenges with equivalent indexes, namely:

  1. You had to manually create indexes with different names

  2. There was no simplified scheme for distributing the indexes across the chosen index nodes. When index nodes were added or removed, it was a manual operation to create such index copies again.

  3. Queries using the USE Index hint or Prepared Statements used to fall apart if the index  node was lost, due to the close affinity with the multiple index names.

With Couchbase Server 5.0, you can transition from using Equivalent Indexes to Index Replicas. With Index Replicas, you need to fire the index definition only once, but state either the number of replicas you want or the nodes on which they have to be placed, and thereafter, Couchbase takes care of all the work of apportioning the query traffic and also rebalancing suitably when nodes are added or removed. When enough replicas remain in the system(i.e, if not all nodes containing the index replicas are down), Index Service ensures that the indexes remains online even during system failures. If an index scan fails in one of the replicas, then the scan will be tried in another replica. As soon as any of the replicas are created, the index service starts sending the query traffic.

 create index index1 on bucket(field1) with {“num_replica”:2} 

IF you use the num_replica parameter while creating the index, the index service creates additional replicas as mentioned in the parameter. So, if the num_replica were 2, then there are 3(num_replica + 1) copies of the index(i.e, 2 extra copies). Do note:

  • if there are lesser nodes compared to the num_replica, then the create_index fails with a readable message stating that indexer node count is not enough to create the index with the replica count specified. Hence, it is advisable that number of indexer nodes be greater than the num_replica specified in the create index statement.

  • Couchbase picks the nodes that have fewest indexes; and does not place more than one copy of the index(i.e, replica) on the same node.

  • If index nodes are spread into server groups(Rack Zone Awareness), then Couchbase spreads out the replicas to as many server groups as possible. If the replica count is more than the number of server groups, then some of the server groups will have more replicas than the rest.

If you want to manually specify the nodes on which to create the indexes, then it is possible by replacing the num_replica parameter with the ‘nodes’ parameter.

 create index index1 on bucket(field1) with {“nodes”:[“1.2.3.4:8091”, “1.2.3.5:8091”, “1.2.3.6:8091”]} 

It is advisable that while using this syntax with the ‘nodes’ parameter specified, you choose the nodes that are on multiple server groups; this syntax gives the flexibility of choosing the nodes, but unlike the syntax wherein num_replica is used, Couchbase Index Service does not override the supplied node settings by the user.

All the replicas continue to receive the query traffic; that is, there is no concept of a master-slave in index replicas and all the index replicas are active. This helps in load balancing.

Moving Index Replicas

If the index(along with its replicas) is spread out on the nodes : “10.10.10.1:8091”, “10.10.10.2:8091”, “10.10.10.3:8091” And if you want to manually move a replica (say 10.10.10.2:8091) to a different node, the following REST call (to be fired on any of the indexing nodes) gets the job done:

 curl -u user:password 10.10.10.1:9102/moveIndex -d  '{"index" : "def_city", "bucket" : "travel-sample", "nodes" : ["10.10.10.1:8091","10.10.10.4:8091","10.10.10.3:8091"]}'

Couchbase knows that only the replica on “10.10.10.2:8091″ needs to be moved to “10.10.10.4:8091″ and does not touch the replicas on “10.10.10.1:8091″ and “10.10.10.3:8091″; and hence only one replica is moved.

Move index does not change(increase or decrease) the count of replicas; the number of nodes to be mentioned in the ‘nodes’ parameter in this REST call has to match with the number of nodes on which the replicas are already spread out.

Drop Index

Drop index statement drops all replicas. Till all the replicas are dropped, the Index remains online. 

Index Replicas are supported for both Memory Optimized Indexes(MOI) and GSI(Plasma) in Couchbase Server 5.0.


Get comfortable using NoSQL in a free, self-directed learning course provided by RavenDB. Learn to create fully-functional real-world programs on NoSQL Databases. Register today.

Topics:
couchbase ,index ,indexing ,couchbase server

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}