Learn how to use the service registration and discovery services in ZooKeeper to manage microservices when refactoring from an existing monolithic application.
Distributed transaction tracing is a useful way to monitor or evaluate your microservices architecture for performance, particularly when measuring end-to-end requests.
In this article, excerpted from the book Docker in Action, I will show you how to open access to shared memory between containers. Linux provides a few tools for sharing memory between processes running on the same computer. This form of inter-process communication (IPC) performs at memory speeds. It is often used when the latency associated with network or pipe based IPC drags software performance below requirements. The best examples of shared memory based IPC usage is in scientific computing and some popular database technologies like PostgreSQL. Docker creates a unique IPC namespace for each container by default. The Linux IPC namespace partitions shared memory primitives like named shared memory blocks and semaphores, as well as message queues. It is okay if you are not sure what these are. Just know that they are tools used by Linux programs to coordinate processing. The IPC namespace prevents processes in one container from accessing the memory on the host or in other containers. Sharing IPC Primitives Between Containers I’ve created an image named allingeek/ch6_ipc that contains both a producer and consumer. They communicate using shared memory. Listing 1 will help you understand the problem with running these in separate containers. Listing 1: Launch a Communicating Pair of Programs # start a producer docker -d -u nobody --name ch6_ipc_producer \ allingeek/ch6_ipc -producer # start the consumer docker -d -u nobody --name ch6_ipc_consumer \ allingeek/ch6_ipc -consumer Listing 1 starts two containers. The first creates a message queue and starts broadcasting messages on it. The second should pull from the message queue and write the messages to the logs. You can see what each is doing by using the following commands to inspect the logs of each: docker logs ch6_ipc_producer docker logs ch6_ipc_consumer If you executed the commands in Listing 1 something should be wrong. The consumer never sees any messages on the queue. Each process used the same key to identify the shared memory resource but they referred to different memory. The reason is that each container has its own shared memory namespace. If you need to run programs that communicate with shared memory in different containers, then you will need to join their IPC namespaces with the --ipc flag. The --ipc flag has a container mode that will create a new container in the same IPC namespace as another target container. Listing 2: Joining Shared Memory Namespaces # remove the original consumer docker rm -v ch6_ipc_consumer # start a new consumer with a joined IPC namespace docker -d --name ch6_ipc_consumer \ --ipc container:ch6_ipc_producer \ allingeek/ch6_ipc -consumer Listing 2 rebuilds the consumer container and reuses the IPC namespace of the ch6_ipc_producer container. This time the consumer should be able to access the same memory location where the server is writing. You can see this working by using the following commands to inspect the logs of each: docker logs ch6_ipc_producer docker logs ch6_ipc_consumer Remember to cleanup your running containers before moving on: # remember: the v option will clean up volumes, # the f option will kill the container if it is running, # and the rm command takes a list of containers docker rm -vf ch6_ipc_producer ch6_ipc_consumer There are obvious security implications to reusing the shared memory namespaces of containers. But this option is available if you need it. Sharing memory between containers is safer alternative to sharing memory with the host.
[This article was written by Sveta Smirnova] Like any good, thus lazy, engineer I don’t like to start things manually. Creating directories, configuration files, specify paths, ports via command line is too boring. I wrote already how I survive in case when I need to start MySQL server (here). There is also the MySQL Sandbox which can be used for the same purpose. But what to do if you want to start Percona XtraDB Cluster this way? Fortunately we, at Percona, have engineers who created automation solution for starting PXC. This solution uses Docker. To explore it you need: Clone the pxc-docker repository:git clone https://github.com/percona/pxc-docker Install Docker Compose as described here cd pxc-docker/docker-bld Follow instructions from the README file: a) ./docker-gen.sh 5.6 (docker-gen.sh takes a PXC branch as argument, 5.6 is default, and it looks for it on github.com/percona/percona-xtradb-cluster) b) Optional: docker-compose build (if you see it is not updating with changes). c) docker-compose scale bootstrap=1 members=2 for a 3 node cluster Check which ports assigned to containers: $docker port dockerbld_bootstrap_1 3306 0.0.0.0:32768 $docker port dockerbld_members_1 4567 0.0.0.0:32772 $docker port dockerbld_members_2 4568 0.0.0.0:32776 Now you can connect to MySQL clients as usual: $mysql -h 0.0.0.0 -P 32768 -uroot Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 10 Server version: 5.6.21-70.1 MySQL Community Server (GPL), wsrep_25.8.rXXXX Copyright (c) 2009-2015 Percona LLC and/or its affiliates Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> 6. To change MySQL options either pass it as a mount at runtime with something like volume: /tmp/my.cnf:/etc/my.cnf in docker-compose.yml or connect to container’s bash (docker exec -i -t container_name /bin/bash), then change my.cnf and run docker restart container_name Notes. If you don’t want to build use ready-to-use images If you don’t want to run Docker Compose as root user add yourself to docker group
written by tim brock. scatter plots are a wonderful way of showing ( apparent ) relationships in bivariate data. patterns and clusters that you wouldn't see in a huge block of data in a table can become instantly visible on a page or screen. with all the hype around big data in recent years it's easy to assume that having more data is always an advantage. but as we add more and more data points to a scatter plot we can start to lose these patterns and clusters. this problem, a result of overplotting, is demonstrated in the animation below. the data in the animation above is randomly generated from a pair of simple bivariate distributions. the distinction between the two distributions becomes less and less clear as we add more and more data. so what can we do about overplotting? one simple option is to make the data points smaller. (note this is a poor "solution" if many data points share exactly the same values.) we can also make them semi-transparent. and we can combine these two options: these refinements certainly help when we have ten thousand data points. however, by the time we've reached a million points the two distributions have seemingly merged in to one again. making points smaller and more transparent might help things; nevertheless, at some point we may have to consider a change of visualization. we'll get on to that later. but first let's try to supplement our visualization with some extra information. specifically let's visualize the marginal distributions . we have several options. there's far too much data for a rug plot , but we can bin the data and show histograms . or we can use a smoother option - a kernel density plot . finally, we could use the empirical cumulative distribution . this last option avoids any binning or smoothing but the results are probably less intuitive. i'll go with the kernel density option here, but you might prefer a histogram. the animated gif below is the same as the gif above but with the smoothed marginal distributions added. i've left scales off to avoid clutter and because we're only really interested in rough judgements of relative height. adding marginal distributions, particularly the distribution of variable 2, helps clarify that two different distributions are present in the bivariate data. the twin-peaked nature of variable 2 is evident whether there are a thousand data points or a million. the relative sizes of the two components is also clear. by contrast, the marginal distribution of variable 1 only has a single peak, despite coming from two distinct distributions. this should make it clear that adding marginal distributions is by no means a universal solution to overplotting in scatter plots. to reinforce this point, the animation below shows a completely different set of (generated) data points in a scatter plot with marginal distributions. the data again comes from a random sample of two different 2d distributions, but both marginal distributions of the complete dataset fail to highlight this separation. as previously, when the number of data points is large the distinction between the two clusters can't be seen from the scatter plot either. returning to point size and opacity, what do we get if we make the data points very small and almost completely transparent? we can now clearly distinguish two clusters in each dataset. it's difficult to make out any fine detail though. since we've lost that fine detail anyway, it seems apt to question whether we really want to draw a million data points. it can be tediously slow and impossible in certain contexts. 2d histograms are an alternative. by binning data we can reduce the number of points to plot and, if we pick an appropriate color scale, pick out some of the features that were lost in the clutter of the scatter plot. after some experimenting i picked a color scale that ran from black through green to white at the high end. note, this is (almost) the reverse of the effect created by overplotting in the scatter plots above. in both 2d histograms we can clearly see the two different clusters representing the two distributions from which the data is drawn. in the first case we can also see that there are more counts from the upper-left cluster than the bottom-right cluster, a detail that is lost in the scatter plot with a million data points (but more obvious from the marginal distributions). conversely, in the case of the second dataset we can see that the "heights" of the two clusters are roughly comparable. 3d charts are overused, but here (see below) i think they actually work quite well in terms of providing a broad picture of where the data is and isn't concentrated. feature occlusion is a problem with 3d charts so if you're going to go down this route when exploring your own data i highly recommend using software that allows for user interaction through rotation and zooming. in summary, scatter plots are a simple and often effective way of visualizing bivariate data. if, however, your chart suffers from overplotting, try reducing point size and opacity. failing that, a 2d histogram or even a 3d surface plot may be helpful. in the latter case be wary of occlusion.
in the previous blog post about solrcloud we’ve talked about the situation when zookeeper connection failed and how solr handles that situation. however, we only talked about query time behavior of solrcloud and we said that we will get back to the topic of indexing in the future. that future is finally here – let’s see what happens to indexing when zookeeper connection is not available. looking back at the old post in the solrcloud – what happens when zookeeper fails? blog post, we’ve shown that solr can handle querying without any issues when connection to zookeeper has been lost (which can be caused by different reasons). of course this is true until we change the cluster topology. unfortunately, in case of indexing or cluster change operations, we can’t change the cluster state or index documents when zookeeper connection is not working or zookeeper failed to read/write the data we want. why we can run queries? the situation is quite simple – querying is not an operation that needs to alter solrcloud cluster state. the only thing solr needs to do is accept the query, run it against known shards/replicas and gather the results. of course cluster topology is not retrieved with each query, so when there is no active zookeeper connection (or zookeeper failed) we don’t have a problem with running queries. there is also one important and not widely know feature of solrcloud – the ability to return partial results. by adding the shards.tolerant=true parameter to our queries we inform solr, that we can live with partial results and it should ignore shards that are not available. this means that solr will return results even if some of the shards from our collection is not available. by default, when this parameter is not present or set to false , solr will just return error when running a query against collection that doesn’t have all the shards available. why we can’t index data? so, we can’t we index data, when zookeeper connection is not available or when zookeeper doesn’t have a quorum? because there is potentially not enough information about the cluster state to process the indexing operation. solr just may not have the fresh information about all the shards, replicas, etc. because of that, indexing operation may be pointed to incorrect shard (like not to the current leader), which can lead to data corruption. and because of that indexing (or cluster change) operation is jus not possible. it is generally worth remembering, that all operations that can lead to cluster state update or collections update won’t be possible when zookeeper quorum is not visible by solr (in our test case, it will be a lack of connectivity of a single zookeeper server). of course, we could leave you with what we wrote above, but let’s check if all that is true. running zookeeper a very simple step. for the purpose of the test we will only need a single zookeeper instance which is run using the following command from zookeeper installation directory: bin/zkserver.sh start we should see the following information on the console: jmx enabled by default using config: /users/gro/solry/zookeeper/bin/../conf/zoo.cfg starting zookeeper ... started and that means that we have a running zookeeper server. starting two solr instances to run the test we’ve used the newest available solr version – the 5.2.1 when this blog post was published. to run two solr instances we’ve used the following command: bin/solr start -e cloud -z localhost:2181 solr asked us a few questions when it was starting and the answers where the following: number of instances: 2 collection name: gettingstarted number of shards: 2 replication count: 1 configuration name: data_driven_schema_configs cluster topology after solr started was as follows: let’s index a few documents to see that solr is really running, we’ve indexed a few documents by running the following command: bin/post -c gettingstarted docs/ if everything went well, after running the following command: curl -xget 'localhost:8983/solr/gettingstarted/select?indent=true&q=*:*&rows=0' we should see solr responding with similar xml: 0 38 *:* true 0 we’ve indexed our documents, we have solr running. let’s stop zookeeper and index data to stop zookeeper server we will just run the following command in the zookeeper installation directory: bin/zkserver.sh stop and now, let’s again try to index our data: bin/post -c gettingstarted docs/ this time, instead of data being written into the collection we will get an error response similar to the following one: posting file index.html (text/html) to [base]/extract simpleposttool: warning: solr returned an error #503 (service unavailable) for url: http://localhost:8983/solr/gettingstarted/update/extract?resource.name=%2fusers%2fgro%2fsolry%2f5.2.1%2fdocs%2findex.html&literal.id=%2fusers%2fgro%2fsolry%2f5.2.1%2fdocs%2findex.html simpleposttool: warning: response: 5033cannot talk to zookeeper - updates are disabled.503 as we can see, the lack of zookeeper connectivity resulted in solr not being able to index data. of course querying still works. turning on zookeeper again and retrying indexing will be successful, because solr will automatically reconnect to zookeeper and will start working again. short summary of course this and the previous blog post related to zookeeper and solrcloud are only touching the surface of what is happening when zookeeper connection is not available. a very good test that shows us data consistency related information can be found at http://lucidworks.com/blog/call-maybe-solrcloud-jepsen-flaky-networks/ . i really recommend it if you would like to know what will happen with solrcloud in various emergency situations.
In 2015, the middleware world focuses on two buzzwords: Docker and Microservices. Software vendors still sell products such as an Enterprise Service Bus (ESB) or Complex Event Processing (CEP) engines. How is this related? Docker is a fascinating technology to deploy and distribute modules (middleware, applications, services) quickly and easily. Most people agree that Docker will change the future of software development in the next years. I will do another blog post about how Docker is related to TIBCO and how you can deploy and distribute Microservices with Docker and TIBCO products such as TIBCO EMS and BusinessWorks 6 easily. Microservices is NOT a technology, but a software architecture style. Many people say that Microservices kill the Enterprise Service Bus (ESB) because Microservices use smart endpoints and dumb pipes. I had a talk at the Microservices Meetup in Munich in June 2015. Most attendees were surprised, why TIBCO shall be relevant for Microservices. I heard that question in several customer meetings, too. This was the main motivation for this talk. I want to share the slide deck and video recording of the talk with you… Abstract: Why use TIBCO for Microservices? Microservices are the next step after SOA: Services implement a limited set of functions. Services are developed, deployed and scaled independently. Continuous Integration and Continuous Delivery control deployments. This way you get shorter time to results and increased flexibility. Microservices have to be independent regarding build, deployment, data management and business domains. A solid Microservices design requires single responsibility, loose coupling and a decentralized architecture. A Microservice can to be closed or open to partners and public via APIs. This session discusses the requirements, best practices and challenges for creating a good Microservices architecture, and if this spells the end of the Enterprise Service Bus (ESB). Key messages of the talk: Microservices = SOA done right Integration is key for success – the product name does not matter Real time event correlation is the game changer Slide Deck from Microservices Meetup in Munich, Germany Here is the slide deck: Microservices = Death of the Enterprise Service Bus (ESB)? from Kai Wähner Video Recording on Youtube The session was recorded (thanks to the guys from AutoScout24). Here is the Youtube upload: https://youtu.be/wMDHUTmUsKg Looking forward to your feedback… Is the ESB dead or not? If no, what kind of ESB (or better said in 2015: Service Delivery Platform) do you use? If yes, how to you implement “ESB features” in your projects? “Simple” REST services and server-code under the hood, or how else?
It's a big week with Dockercon going on, and we have some great updates. At the show, we are demoing UrbanCode Build and Deploy build containers, storing them in registries, and deploying them out through test environments and production across hybrid clouds. Check out this quick overview from the team: For a deep dive on any of it, find the guys at the IBM booth at Dockercon. They'll be happy to show you!
This days I’m reading about Microservices. The idea is great. Instead of building a monolithic script using one language/framowork. We create isolated services and we build our application using those services (speaking HTTP between services and application). That’s means we’ll have several microservices and we need to use them, and maybe sometimes change one service with another one. In this post I want to build one small container to handle those microservices. Similar idea than Dependency Injection Containers. As we’re going to speak HTTP, we need a HTTP client. We can build one using curl, but in PHP world we have Guzzle, a great HTTP client library. In fact Guzzle has something similar than the idea of this post: Guzzle services, but I want something more siple. Imagine we have different services: One Silex service (PHP + Silex) use Silex\Application; $app = new Application(); $app->get('/hello/{username}', function($username) { return "Hello {$username} from silex service"; }); $app->run(); Another PHP service. This one using Slim framework use Slim\Slim; $app = new Slim(); $app->get('/hello/:username', function ($username) { echo "Hello {$username} from slim service"; }); $app->run(); And finally one Python service using Flask framework from flask import Flask, jsonify app = Flask(__name__) @app.route('/hello/') def show_user_profile(username): return "Hello %s from flask service" % username if __name__ == "__main__": app.run(debug=True, host='0.0.0.0', port=5000) Now, with our simple container we can use one service or another use Symfony\Component\Config\FileLocator; use MSIC\Loader\YamlFileLoader; use MSIC\Container; $container = new Container(); $ymlLoader = new YamlFileLoader($container, new FileLocator(__DIR__)); $ymlLoader->load('container.yml'); echo $container->getService('flaskServer')->get('/hello/Gonzalo')->getBody() . "\n"; echo $container->getService('silexServer')->get('/hello/Gonzalo')->getBody() . "\n"; echo $container->getService('slimServer')->get('/hello/Gonzalo')->getBody() . "\n"; And that’s all. You can see the project in my github account.
Continuing my Spring-Cloud learning journey, earlier I had covered how to write the infrastructure components of a typical Spring-Cloud and Netflix OSS based micro-services environment - in this specific instance two critical components, Eureka to register and discover services and Spring Cloud Configuration to maintain a centralized repository of configuration for a service. Here I will be showing how I developed two dummy micro-services, one a simple "pong" service and a "ping" service which uses the "pong" service. Sample-Pong microservice The endpoint handling the "ping" requests is a typical Spring MVC based endpoint: @RestController public class PongController { @Value("${reply.message}") private String message; @RequestMapping(value = "/message", method = RequestMethod.POST) public Resource pongMessage(@RequestBody Message input) { return new Resource<>( new MessageAcknowledgement(input.getId(), input.getPayload(), message)); } } It gets a message and responds with an acknowledgement. Here the service utilizes the Configuration server in sourcing the "reply.message" property. So how does the "pong" service find the configuration server, there are potentially two ways - directly by specifying the location of the configuration server, or by finding the Configuration server via Eureka. I am used to an approach where Eureka is considered a source of truth, so in this spirit I am using Eureka to find the Configuration server. Spring Cloud makes this entire flow very simple, all it requires is a "bootstrap.yml" property file with entries along these lines: --- spring: application: name: sample-pong cloud: config: discovery: enabled: true serviceId: SAMPLE-CONFIG eureka: instance: nonSecurePort: ${server.port:8082} client: serviceUrl: defaultZone: http://${eureka.host:localhost}:${eureka.port:8761}/eureka/ The location of Eureka is specified through the "eureka.client.serviceUrl" property and the "spring.cloud.config.discovery.enabled" is set to "true" to specify that the configuration server is discovered via the specified Eureka server. Just a note, this means that the Eureka and the Configuration server have to be completely up before trying to bring up the actual services, they are the pre-requisites and the underlying assumption is that the Infrastructure components are available at the application boot time. The Configuration server has the properties for the "sample-pong" service, this can be validated by using the Config-servers endpoint - http://localhost:8888/sample-pong/default, 8888 is the port where I had specified for the server endpoint, and should respond with a content along these lines: "name": "sample-pong", "profiles": [ "default" ], "label": "master", "propertySources": [ { "name": "classpath:/config/sample-pong.yml", "source": { "reply.message": "Pong" } } ] } As can be seen the "reply.message" property from this central configuration server will be used by the pong service as the acknowledgement message Now to set up this endpoint as a service, all that is required is a Spring-boot based entry point along these lines: @SpringBootApplication @EnableDiscoveryClient public class PongApplication { public static void main(String[] args) { SpringApplication.run(PongApplication.class, args); } } and that completes the code for the "pong" service. Sample-ping micro-service So now onto a consumer of the "pong" micro-service, very imaginatively named the "ping" micro-service. Spring-Cloud and Netflix OSS offer a lot of options to invoke endpoints on Eureka registered services, to summarize the options that I had: 1. Use raw Eureka DiscoveryClient to find the instances hosting a service and make calls using Spring's RestTemplate. 2. Use Ribbon, a client side load balancing solution which can use Eureka to find service instances 3. Use Feign, which provides a declarative way to invoke a service call. It internally uses Ribbon. I went with Feign. All that is required is an interface which shows the contract to invoke the service: package org.bk.consumer.feign; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.springframework.cloud.netflix.feign.FeignClient; import org.springframework.http.MediaType; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.ResponseBody; @FeignClient("samplepong") public interface PongClient { @RequestMapping(method = RequestMethod.POST, value = "/message", produces = MediaType.APPLICATION_JSON_VALUE, consumes = MediaType.APPLICATION_JSON_VALUE) @ResponseBody MessageAcknowledgement sendMessage(@RequestBody Message message); } The annotation @FeignClient("samplepong") internally points to a Ribbon "named" client called "samplepong". This means that there has to be an entry in the property files for this named client, in my case I have these entries in my application.yml file: samplepong: ribbon: DeploymentContextBasedVipAddresses: sample-pong NIWSServerListClassName: com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList ReadTimeout: 5000 MaxAutoRetries: 2 The most important entry here is the "samplepong.ribbon.DeploymentContextBasedVipAddresses" which points to the "pong" services Eureka registration address using which the service instance will be discovered by Ribbon. The rest of the application is a routine Spring Boot application. I have exposed this service call behind Hystrix which guards against service call failures and essentially wraps around this FeignClient: package org.bk.consumer.service; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.bk.consumer.feign.PongClient; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Service; @Service("hystrixPongClient") public class HystrixWrappedPongClient implements PongClient { @Autowired @Qualifier("pongClient") private PongClient feignPongClient; @Override @HystrixCommand(fallbackMethod = "fallBackCall") public MessageAcknowledgement sendMessage(Message message) { return this.feignPongClient.sendMessage(message); } public MessageAcknowledgement fallBackCall(Message message) { MessageAcknowledgement fallback = new MessageAcknowledgement(message.getId(), message.getPayload(), "FAILED SERVICE CALL! - FALLING BACK"); return fallback; } } Boot"ing up I have dockerized my entire set-up, so the simplest way to start up the set of applications is to first build the docker images for all of the artifacts this way: mvn clean package docker:build -DskipTests and bring all of them up using the following command, the assumption being that both docker and docker-compose are available locally: docker-compose up Assuming everything comes up cleanly, Eureka should show all the registered services, at http://dockerhost:8761 url - The UI of the ping application should be available at http://dockerhost:8080 url - Additionally a Hystrix dashboard should be available to monitor the requests to the "pong" app at this url http://dockerhost:8989/hystrix/monitor?stream=http%3A%2F%2Fsampleping%3A8080%2Fhystrix.stream: References 1. The code is available at my github location - https://github.com/bijukunjummen/spring-cloud-ping-pong-sample 2. Most of the code is heavily borrowed from the spring-cloud-samples repository - https://github.com/spring-cloud-samples
CORE
