DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Data Engineering Topics

article thumbnail
Finding Dependency in Stored Procedure
Introduction Here in this article we are trying to discuss about the finding reference object within stored procedure and also finding the calling procedure references. Hope you like it and it will be informative. What We Want Developers are writing several stored procedure almost every day. Sometimes developers need to know about the information such as what object is used within the stored procedure or from where (SP) the specified stored procedure call. This is the vital information for the developer before working on a particular stored procedure. Here we are representing a pictorial diagram to understand the nature of implementation. Now we have to answer some question 1. What are the DB Object used in Stored Procedure1 and there type. 2. In case of Store Procedure3 which procedure calls the Store Procedure3 So we are not going to read the Stored Procedure to find the answer. Suppose the each procedure have more than 3000 line. How We Solve the Answer To solve the answer first we take the example and create an example scenario to understand it. -- Base Table CREATE TABLE T1 (EMPID INT, EMPNAME VARCHAR(50)); GO CREATE TABLE T2 (EMPID INT, EMPNAME VARCHAR(50)); GO --1 CREATE PROCEDURE [dbo].[Procedure1] AS BEGIN SELECT * FROM T1; SELECT * FROM T2; EXEC [dbo].[Procedure3]; END GO --2 CREATE PROCEDURE [dbo].[Procedure2] AS BEGIN EXEC [dbo].[Procedure3]; END GO --3 CREATE PROCEDURE [dbo].[Procedure3] AS BEGIN SELECT * FROM T1; END GO Now we are going to solve the question What are the DB Object used in Stored Procedure1 and there type. sp_depends Procedure1 In case of Store Procedure3 which procedure calls the Store Procedure3 SELECT OBJECT_NAME(id) AS [Calling SP] FROM syscomments WHERE [text] LIKE '%Procedure3%' GROUP BY OBJECT_NAME(id); Hope you like it.
July 3, 2015
by Joydeep Das
· 12,237 Views
article thumbnail
Ramesh Shivakumaran Gulftainer records 8 growth in container volume to achieve 6.4 Million Teus in 2014
16 Apr 2015 In a year defined by international expansion and investments in new infrastructure to enhance operational efficiency, Gulftainer recorded robust growth across its entire terminal portfolio. Iain Rawlinson, Group Commercial Director of Gulftainer said: “The positive growth recorded by Gulftainer across its terminals globally underlines the confidence of our partners in our ability to meet their requirements efficiently. Our extensive network and technological expertise are the strengths that have enabled us to expand our footprint to new locations. We continuously invest in enhancing our infrastructure, thus boosting reliability, operational efficiency and productivity.” He added: “The growth in volume achieved throughout our terminals is strong testament to the expertise and dedication of our employees and the strong productivity levels we are able to achieve on a consistent basis. In the dynamic global trade routes linking Asia and Europe, our terminals today play an increasingly significant role. Even as we expand and grow our business, we also remain committed to the communities we serve in by creating new jobs and supporting the domestic economy.” In global markets, Gulftainer’s Saudi terminals recorded impressive growth with Northern Container Terminal accounting for 1.9 million TEUs, sustaining previous-year trends, while Jubail Container Terminal (JCT) noted a growth of 22 per cent to over 396,000 TEUs. The total volume at the Saudi terminals was over 2.29 million TEUs. Gulftainer’s Umm Qasr terminal also accomplished a significant growth of 46 per cent in 2014, while the Recife terminal in Brazil marked a growth in volume of 7 per cent. Gulftainer’s UAE terminals recorded a total volume of 3.8 million TEUs in line with the all-round growth in business. The company marked another significant milestone, with the Sharjah Container Terminal (SCT) surpassing 400,000 TEUs in annual throughput for the very first time. Operations at SCT were energised by the positive growth in global trade and the arrival of new services, such as UASC’s Gulf India Service (GIS1), which now connects Sharjah with Sohar in Oman, Mundra in India and Karachi in Pakistan. The addition of this service represented a significant development for Sharjah and boosted the national carrier’s volumes through SCT last year. The only fully fledged operational container terminal in the UAE located outside the Strait of Hormuz, Khorfakkan Container Terminal (KCT) has today emerged as one of the most important transshipment hubs for the Arabian Gulf, the Indian Sub-continent, the Gulf of Oman and the East African markets. Further strengthening the operations at KCT, Gulftainer has received and commissioned new state-of-the-art Ship to Shore (STS) and Rubber Tyred Gantry (RTG) cranes that will further increase overall performance and productivity. This enhanced infrastructure marks an investment of over US$60 million. Gulftainer has set an ambitious target to triple the volume over the next decade through organic growth across existing businesses, exploring green field opportunities and potential M&A activities.
July 2, 2015
by Androcles Buckley
· 726 Views
article thumbnail
Using HA-JDBC with Spring Boot
This is a really simple way to provide high-availability with failover and load balancing to any Java backend using JDBC and Spring Boot .
July 2, 2015
by Lieven Doclo
· 23,310 Views · 1 Like
article thumbnail
GULFTAINER SURPASSES 400,000 TEU MILESTONE AT SHARJAH CONTAINER TERMINAL IN 2014
Gulftainer, a privately owned, independent terminal operating and logistics company, marked another significant milestone with the Sharjah Container Terminal (SCT) surpassing 400,000 TEUs (Twenty Foot Equivalent Units) in annual throughput during 2014. SCT has again recorded double-digit growth compared to last year’s volumes. The achievement was reached with an impressive safety record under challenging conditions including space constraints. Iain Rawlinson, Group Commercial Director of Gulftainer said that the professional approach of Gulftainer’s management, along with consistently high productivity levels, was a driving force behind the Terminal’s success. “SCT has always marketed itself as ‘The Flexible Alternative’ and the individual attention we extend to our customers offers us an advantage over competitors.” The 400,000th unit was discharged from Mag Container Lines’ vessel, ‘Mag Success’, one of the Terminal’s regular callers, which considers Sharjah as her base port. Speaking on behalf of Mag Line’s CEO, BDM Jamal Saleh congratulated the Terminal for its achievement. He said: “The announcement today reflects how Gulftainer and MCL have grown together over the years and, in partnership, managed to reach this target. The continuous support, flexibility and excellent operational performance MCL receives from Gulftainer, both operationally and logistically, has contributed greatly to this achievement.” The milestone was achieved on the shift of Duty Superintendent Mehmood Malik, the longest serving employee at over 38 years at the Terminal and part of the team when the first TEU crossed the quay. Mehmood has witnessed several records and milestones and recalls handling 2,500 TEUs in 1976: “At that time we could not imagine reaching the levels of throughput we have today, so this is a very special moment for me.” SCT, which is managed and operated by Gulftainer on behalf of the Sharjah Port Authority, has the honour of being the site of the first container terminal in the Gulf, commenced operations in 1976. SCT is located in the heart of Sharjah and is an ideal gateway for import and export cargo with direct links throughout the Gulf, Asia, Europe, Americas and Africa. The strong performance of the Sharjah economy has supported the growth of many of SCT’s customers, enabling them to increase their throughput and contribute to a record year for the Terminal. The relationships built with our customers have been strengthened by the joint efforts of Gulftainer’s sales and marketing team and the high levels of service and operational efficiency at the terminal, “When looking at the Sharjah market, the dedicated team at SCT listen to and address the many requirements of our diverse and interesting customer base,” said Iain Rawlinson. SCT’s figures have been further boosted with the arrival of new services throughout the year, including UASC’s Gulf India Service (GIS1), which now connects Sharjah with Sohar in Oman, Mundra in India and Karachi in Pakistan, which has boosted in the national carrier’s volumes through SCT in November and December. Gulftainer’s current portfolio covers UAE operations in Khorfakkan Port and Port Khalid in Sharjah as well as activities at Umm Qasr in Iraq, Recife in Brazil, Jeddah and Jubail in Saudi Arabia and in Tripoli Port in Lebanon, which will be operational in April 2016. It also marked another milestone in 2014 with its expansion to the US by signing a long-term agreement to operate the container and multi-cargo terminal at Port Canaveral in Florida. With a current handling activity of over 6 million TEUs, the company has set an ambitious target to triple the volume over the next decade through organic growth across existing businesses, exploring green field opportunities and potential M&A activities.
July 2, 2015
by Tirill Malmin
· 776 Views
article thumbnail
Crowdsourcing our way to better food hygiene
The last few years has seen a tremendous boom in the number of sources online relaying information about restaurant quality. Whether it’s review sites or more general social media, there is no shortage of feedback on how people have found a particular restaurant. I wrote a few years ago about a project from the University of Rochester that aimed to mine Twitter for mentions of eating out, with the hope of producing a detailed and comprehensive map of food hygiene standards throughout restaurants in New York. The system, called nEmesis, analyzed millions of tweets, and was on the hunt for people sharing an attack of food poisoning after visiting a restaurant. You might think, or hope at least, that this would be a relatively small number, but over a four month period they found 480 such mentions in New York City alone from a total of 23,000 restaurant visitors. What’s more, the data collected correlated well with public health data on those diners. Crowdsourcing food hygiene A recent Harvard led project is hoping to provide similar assistance to the Boston food hygiene authorities by providing more intelligent information for the authorities to base their inspection checks on. Rather than using Twitter for data however, the Harvard project is turning to the review website Yelp. They have launched a NetFlix style competition to create an algorithm that can search through the ratings of restaurants in Boston and produce recommendations for which restaurants warrant a visit from the hygiene police. The competition, organized by the data company DrivenData, will see the raw data posted online and then an army of data scientists charged with solving the puzzle. The founders observed that whilst the collection of machine readable data was now mandated by the government, there was a literacy problem that rendered much of that data left dormant and unused. Bringing data science to the masses And so the competition was born to try and make data science affordable for organizations with a clear social need but no budget to afford what are still very expensive skill sets. Of course, the food hygiene challenge is but one of the challenges on the DrivenData website, with the venture coming along way from their first challenge to make a better algorithm for improving spending in schools. The organization try and ensure that whatever winning entries emerge from the competitions receive support and help to grow and improve. The winner of that initial competition, for instance, eventually turned their algorithm into a software tool for schools to use. The eventual aim is to establish a community of data scientists that are happy to deploy their talents for socially worthwhile endeavors. “Our mindset has grown; we want to solve the big-picture data literacy and data capacity problems in the social and public sectors,” the creators say. “We think competitions are a great mechanism to do that right now, but our goal is to do more, to serve that community in other ways.” Suffice to say, challenges have come a long way from their beginnings in the 18th century when the UK government launched such a competition to help find longitude more easily. The likes of the X Prize has taken them to newfound heights, and it’s great to see organizations like DrivenData apply the concept to more manageable challenges. Of course, they aren’t the only organization seeking to make algorithms more accessible. I wrote last year about the Algorithmia social network, which aims to connect up organizations with lots of data with algorithms that are being under-utilized. The aim is that this match up will create not just new insights but extra profits. Data science is undoubtedly a burgeoning field, and it’s one with a great many exciting developments in it. Original post
July 2, 2015
by Adi Gaskell
· 902 Views · 1 Like
article thumbnail
Using Camel, CDI Inside Kubernetes With Fabric8
Learn about how to integrate Apache Camel and Fabric8 into an existing Kubernetes CDI service.
July 2, 2015
by Ioannis Canellos
· 19,747 Views · 1 Like
article thumbnail
SolrCloud: What Happens When ZooKeeper Fails – Part Two
in the previous blog post about solrcloud we’ve talked about the situation when zookeeper connection failed and how solr handles that situation. however, we only talked about query time behavior of solrcloud and we said that we will get back to the topic of indexing in the future. that future is finally here – let’s see what happens to indexing when zookeeper connection is not available. looking back at the old post in the solrcloud – what happens when zookeeper fails? blog post, we’ve shown that solr can handle querying without any issues when connection to zookeeper has been lost (which can be caused by different reasons). of course this is true until we change the cluster topology. unfortunately, in case of indexing or cluster change operations, we can’t change the cluster state or index documents when zookeeper connection is not working or zookeeper failed to read/write the data we want. why we can run queries? the situation is quite simple – querying is not an operation that needs to alter solrcloud cluster state. the only thing solr needs to do is accept the query, run it against known shards/replicas and gather the results. of course cluster topology is not retrieved with each query, so when there is no active zookeeper connection (or zookeeper failed) we don’t have a problem with running queries. there is also one important and not widely know feature of solrcloud – the ability to return partial results. by adding the shards.tolerant=true parameter to our queries we inform solr, that we can live with partial results and it should ignore shards that are not available. this means that solr will return results even if some of the shards from our collection is not available. by default, when this parameter is not present or set to false , solr will just return error when running a query against collection that doesn’t have all the shards available. why we can’t index data? so, we can’t we index data, when zookeeper connection is not available or when zookeeper doesn’t have a quorum? because there is potentially not enough information about the cluster state to process the indexing operation. solr just may not have the fresh information about all the shards, replicas, etc. because of that, indexing operation may be pointed to incorrect shard (like not to the current leader), which can lead to data corruption. and because of that indexing (or cluster change) operation is jus not possible. it is generally worth remembering, that all operations that can lead to cluster state update or collections update won’t be possible when zookeeper quorum is not visible by solr (in our test case, it will be a lack of connectivity of a single zookeeper server). of course, we could leave you with what we wrote above, but let’s check if all that is true. running zookeeper a very simple step. for the purpose of the test we will only need a single zookeeper instance which is run using the following command from zookeeper installation directory: bin/zkserver.sh start we should see the following information on the console: jmx enabled by default using config: /users/gro/solry/zookeeper/bin/../conf/zoo.cfg starting zookeeper ... started and that means that we have a running zookeeper server. starting two solr instances to run the test we’ve used the newest available solr version – the 5.2.1 when this blog post was published. to run two solr instances we’ve used the following command: bin/solr start -e cloud -z localhost:2181 solr asked us a few questions when it was starting and the answers where the following: number of instances: 2 collection name: gettingstarted number of shards: 2 replication count: 1 configuration name: data_driven_schema_configs cluster topology after solr started was as follows: let’s index a few documents to see that solr is really running, we’ve indexed a few documents by running the following command: bin/post -c gettingstarted docs/ if everything went well, after running the following command: curl -xget 'localhost:8983/solr/gettingstarted/select?indent=true&q=*:*&rows=0' we should see solr responding with similar xml: 0 38 *:* true 0 we’ve indexed our documents, we have solr running. let’s stop zookeeper and index data to stop zookeeper server we will just run the following command in the zookeeper installation directory: bin/zkserver.sh stop and now, let’s again try to index our data: bin/post -c gettingstarted docs/ this time, instead of data being written into the collection we will get an error response similar to the following one: posting file index.html (text/html) to [base]/extract simpleposttool: warning: solr returned an error #503 (service unavailable) for url: http://localhost:8983/solr/gettingstarted/update/extract?resource.name=%2fusers%2fgro%2fsolry%2f5.2.1%2fdocs%2findex.html&literal.id=%2fusers%2fgro%2fsolry%2f5.2.1%2fdocs%2findex.html simpleposttool: warning: response: 5033cannot talk to zookeeper - updates are disabled.503 as we can see, the lack of zookeeper connectivity resulted in solr not being able to index data. of course querying still works. turning on zookeeper again and retrying indexing will be successful, because solr will automatically reconnect to zookeeper and will start working again. short summary of course this and the previous blog post related to zookeeper and solrcloud are only touching the surface of what is happening when zookeeper connection is not available. a very good test that shows us data consistency related information can be found at http://lucidworks.com/blog/call-maybe-solrcloud-jepsen-flaky-networks/ . i really recommend it if you would like to know what will happen with solrcloud in various emergency situations.
July 2, 2015
by Rafał Kuć
· 17,979 Views
article thumbnail
Azure Service Bus – As I Understand It: Part II (Queues & Messages)
continuing from my previous post about azure service bus, in this post i will share my learning about queues & messages. the focus of this post will be about some of the undocumented things i found as we implemented support for queues and messages in cloud portam . queues as mentioned in my previous post, queues is the simplest of the azure service bus service and kind of compares with azure storage queue service in the sense that it provides a unidirectional messaging infrastructure where a publisher publishes a message and the message is received by a receiver. there can be many receivers ready to receive the messages however one receiver can only receive a message. no two receivers can receive a single message simultaneously. now some learning about queues. queue name a queue name can be up to 260 characters in length and can contain letters, numbers, periods (.), hyphens (-), and underscores (_) . a queue name is case-insensitive. queue size when creating a queue, you must define the size of the queue. queue size could be one of the following values: 1 gb, 2 gb, 3 gb, 4 gb or 5 gb . a queue size can’t be changed once the queue is created. however if you create a “ partition enabled queue ” then service bus creates 16 partitions thus your queue size is automatically multiplied by 16 and your queue size becomes 16 gb, 32 gb, 48 gb, 64 gb or 80 gb depending on the size you selected (this confused me initially :)). queue properties a service bus queue has many properties. some of the properties can only be set during queue creation time while some of the properties can only be set if you are using “standard” tier of service bus. (above are the screenshots from cloud portam for creating a queue) status indicates the status of a queue – active or disabled . once a queue is disabled, it cannot send or receive messages. max delivery count (maxdeliverycount) indicates the maximum number of times a message can be delivered . once this count has exceeded, message will either be removed from the queue or dead-lettered. the way i understand it is this property is used to manage poison messages. if a message is not processed successfully by receivers for “x” number of times, just move it somewhere else for further inspection or remove it. message time to live (messagettl) indicates a time span for which a message will live inside a queue . if the message is not processed by that time, it will either be removed or dead-lettered. one interesting thing i noticed is that if you’re using “standard” tier, a message could live forever in a queue however in “basic” tier, a message can only live for a maximum of 14 days . lock duration (lockduration) indicates number of seconds for which a message will be locked by a receiver once it receives it so that no other receiver can receive that message . it essentially gives the receiver time to process the message. once this elapses, message will be available to be received by another receiver. maximum value for lock duration can be 5 minutes / 300 seconds . enable partitioning (enablepartitioning) indicates if the queue should be partitioned across multiple message brokers . as mentioned above, service bus automatically creates 16 partitions if this is enabled. this also results in maximum size of the queue increase by a factor of 16. this property can only be set during queue creation time . enable deadlettering (enabledeadlettering) indicates if the messages in the queue should be moved to dead-letter sub queue once they expire. if this property is not set, then the messages will be removed from the queue once they expire. enable batching (enablebatchedoperations) indicates if server-side batched operations are supported. this is used to improve the throughput of a queue as service bus holds the messages for up to 20ms before writing/deleting them in a batch. enable message ordering (supportordering) indicates if the queue supports ordering. requires duplicate detection (requiresduplicatedetection) indicates if the queue requires duplicate detection. this property can only be set during queue creation time and is only available for “standard” tier. enable express (enableexpress) indicates if the queue is an express queue. an express queue holds a message in memory temporarily before writing it to persistent storage. this property can only be set during queue creation time and is only available for “standard” tier. requires session (requiressession) indicates if the queue supports the concept of session. this property can only be set during queue creation time and is only available for “standard” tier. auto delete queue this property specifies a time period after which an idle queue should be deleted automatically by service bus . minimum period allowed is 5 minutes. this can only be set for “standard” tier . duplicate detection history time window (duplicatedetectionhistorytimewindow) defines the duration of the duplicate detection history. this can only be set for “standard” tier . forward messages to queue/topic (forwardto) you can use this property to automatically forward messages from a queue to another queue or topic. when setting this property, the queue/topic must exist in the account. this can only be set for “standard” tier . forward dead-lettered messages to queue/topic (forwarddeadletteredmessagesto) you can use this property to automatically forward dead-lettered message to another queue or topic. when setting this property, the queue/topic must exist in the account. user metadata (usermetadata) you can use this property to define any custom metadata for a queue. following table summarizes property applicability by tier and whether they are editable or not. property tier editable? size basic, standard no status basic, standard yes max delivery count basic, standard yes message time to live basic, standard yes lock duration basic, standard yes enable partitioning basic, standard no enable deadlettering basic, standard yes enable batching basic, standard yes enable message ordering basic, standard yes requires duplicate detection standard no enable express standard no require session standard no auto delete queue standard yes duplicate detection history time window standard yes forward messages to queue/topic standard yes forward dead-lettered messages to queue/topic basic, standard yes user metadata basic, standard yes to learn more about these properties, please see this link: https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.queuedescription.aspx . messages the way i see it, messages are the entities that contain information about the work a sender wants a receiver to do. as mentioned earlier, a sender sends a message to a queue and a receiver will receive the message. at any time, a message will be received by one and only one receiver. message processing there’re two ways by which a receiver will receive a message: peek and lock & receive and delete . peek and lock in peek and lock mode, the message is locked by the receiver for a duration specified by queue’s “ lock duration ” property or in other words under this mode a message is hidden from other receivers for a duration specified by lock duration. the receiver then would process the message and after that a receiver would mark the message as “ complete ” which essentially deletes the message from the queue. if the “lock duration” expires, other receivers will be able to fetch this message. receive and delete in receive and delete mode, once the message is received by a receiver it will be deleted from the queue automatically. if a receiver fails to process that message, then the message is lost forever. so unless you’re sure of receiver’s functionality that it will never fail or you don’t care if the message is processed successfully or not, use this mode cautiously. message composition a message in service bus consists of 3 things – message body, standard properties and custom properties. message body is the actual content of the message. there are some predefined properties of a message and those fall under standard properties. apart from that you can define custom properties on a message which are essentially a collection of name/value pairs. total size of a message is 256 kb. message properties now let’s take a look at some of the standard properties of a message that i found interesting. message id this is the identifier of a message. you can set it at the time of sending a message. because it is an identifier, one would assume that it needs to be unique but that’s not the case. different messages can have same message id. sequence number when a message is created, service bus assigns a number to a message. that number is stored in this property. please note that it is a read-only property. message time to live (message ttl) this is the time period for which a message will remain in the queue. if you recall, you can also define a default message time-to-live at queue level also. service bus actually picks the lower of the two values as message ttl. for example, if you have defined that a message will expire after 14 days at queue level but 5 minutes at the message level then the message will expire after 5 minutes. lock token whenever a message is received by a receiver in “ peek and lock ” mode, service bus returns a (lock) token that must be used to perform further operations (e.g. delete message or dead-letter message etc.) on that message. this token is valid for a duration specified by “ lock duration ” property. after the lock duration expires, the lock token becomes invalid and any attempt to use this token for performing any allowed operations will result in an error. once a lock token expires, a receiver must receive the message again. there are other properties as well which i have not included for the sake of brevity. for a complete list of properties, please see this link: https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.brokeredmessage_properties.aspx . summary that’s it for this post. in the next posts in this series, i will share my learning about topics and other service bus services. so stay tuned for that! again, if you think that i have provided some incorrect information, please let me know and i will fix them asap.
July 2, 2015
by Gaurav Mantri
· 8,625 Views
article thumbnail
Microservices = Death of the Enterprise Service Bus (ESB)? – Slide Deck and Video Recording
In 2015, the middleware world focuses on two buzzwords: Docker and Microservices. Software vendors still sell products such as an Enterprise Service Bus (ESB) or Complex Event Processing (CEP) engines. How is this related? Docker is a fascinating technology to deploy and distribute modules (middleware, applications, services) quickly and easily. Most people agree that Docker will change the future of software development in the next years. I will do another blog post about how Docker is related to TIBCO and how you can deploy and distribute Microservices with Docker and TIBCO products such as TIBCO EMS and BusinessWorks 6 easily. Microservices is NOT a technology, but a software architecture style. Many people say that Microservices kill the Enterprise Service Bus (ESB) because Microservices use smart endpoints and dumb pipes. I had a talk at the Microservices Meetup in Munich in June 2015. Most attendees were surprised, why TIBCO shall be relevant for Microservices. I heard that question in several customer meetings, too. This was the main motivation for this talk. I want to share the slide deck and video recording of the talk with you… Abstract: Why use TIBCO for Microservices? Microservices are the next step after SOA: Services implement a limited set of functions. Services are developed, deployed and scaled independently. Continuous Integration and Continuous Delivery control deployments. This way you get shorter time to results and increased flexibility. Microservices have to be independent regarding build, deployment, data management and business domains. A solid Microservices design requires single responsibility, loose coupling and a decentralized architecture. A Microservice can to be closed or open to partners and public via APIs. This session discusses the requirements, best practices and challenges for creating a good Microservices architecture, and if this spells the end of the Enterprise Service Bus (ESB). Key messages of the talk: Microservices = SOA done right Integration is key for success – the product name does not matter Real time event correlation is the game changer Slide Deck from Microservices Meetup in Munich, Germany Here is the slide deck: Microservices = Death of the Enterprise Service Bus (ESB)? from Kai Wähner Video Recording on Youtube The session was recorded (thanks to the guys from AutoScout24). Here is the Youtube upload: https://youtu.be/wMDHUTmUsKg Looking forward to your feedback… Is the ESB dead or not? If no, what kind of ESB (or better said in 2015: Service Delivery Platform) do you use? If yes, how to you implement “ESB features” in your projects? “Simple” REST services and server-code under the hood, or how else?
July 2, 2015
by Kai Wähner DZone Core CORE
· 6,061 Views · 3 Likes
article thumbnail
Microservice Container with Guzzle
This days I’m reading about Microservices. The idea is great. Instead of building a monolithic script using one language/framowork. We create isolated services and we build our application using those services (speaking HTTP between services and application). That’s means we’ll have several microservices and we need to use them, and maybe sometimes change one service with another one. In this post I want to build one small container to handle those microservices. Similar idea than Dependency Injection Containers. As we’re going to speak HTTP, we need a HTTP client. We can build one using curl, but in PHP world we have Guzzle, a great HTTP client library. In fact Guzzle has something similar than the idea of this post: Guzzle services, but I want something more siple. Imagine we have different services: One Silex service (PHP + Silex) use Silex\Application; $app = new Application(); $app->get('/hello/{username}', function($username) { return "Hello {$username} from silex service"; }); $app->run(); Another PHP service. This one using Slim framework use Slim\Slim; $app = new Slim(); $app->get('/hello/:username', function ($username) { echo "Hello {$username} from slim service"; }); $app->run(); And finally one Python service using Flask framework from flask import Flask, jsonify app = Flask(__name__) @app.route('/hello/') def show_user_profile(username): return "Hello %s from flask service" % username if __name__ == "__main__": app.run(debug=True, host='0.0.0.0', port=5000) Now, with our simple container we can use one service or another use Symfony\Component\Config\FileLocator; use MSIC\Loader\YamlFileLoader; use MSIC\Container; $container = new Container(); $ymlLoader = new YamlFileLoader($container, new FileLocator(__DIR__)); $ymlLoader->load('container.yml'); echo $container->getService('flaskServer')->get('/hello/Gonzalo')->getBody() . "\n"; echo $container->getService('silexServer')->get('/hello/Gonzalo')->getBody() . "\n"; echo $container->getService('slimServer')->get('/hello/Gonzalo')->getBody() . "\n"; And that’s all. You can see the project in my github account.
July 2, 2015
by Gonzalo Ayuso
· 3,485 Views
article thumbnail
JavaFX Table Cells: Interdependence and Dynamic Editability
The standard usage of JavaFX TableView with currently available set of cell controls, with all its indisputable merits, fails to meet a fairly important requirement, which usually arises when editable fields are mutually dependent. Generally it is easy enough to change values of all dependent fields when value of some field had been changed. The real problem emerges when a cell control, which is connected with a single data field (property), should become editable or not editable (having a fixed value) depending on values of some other fields (properties) of the same record (row data object). Besides, to spare user some useless clicks and confusion, it is important to make appearance of the cell reflect at least two conditions: editable/not editable and within/outside of the selected row. To make it work, there is no need to subclass anything but table-cell controls. While it is possible to subclass TableCell and recreate all the specific cell controls, it is much easier to subclass each particular cell control, although it does lead to some minor duplication of code. To insure that all our custom cells refer to the same style definitions in a CSS stylesheet, let's share style constants in an interface: public interface IDeCell { public static final String CLASS_DE_CELL = "de-cell"; public static final String PSEUDO_CLASS_NOT_EDITABLE = "not-editable"; public static final String PSEUDO_CLASS_ROW_SELECTED = "row-selected"; } Prefix "de" stands for Dynamic Editability. We are going to prefix our custom cell extensions with "De" and refer to such cells as "de-cells". Here is source code of custom TextFieldTableCell extension: public class DeTextFieldTableCell extends TextFieldTableCell implements IDeCell { private static final PseudoClass NOT_EDITABLE_PSEUDO_CLASS = PseudoClass.getPseudoClass(PSEUDO_CLASS_NOT_EDITABLE); private static final PseudoClass ROW_SELECTED_PSEUDO_CLASS = PseudoClass.getPseudoClass(PSEUDO_CLASS_ROW_SELECTED); public BooleanProperty notEditableProperty() { return notEditable; } public final boolean isNotEditable() { return notEditableProperty().get(); } private final BooleanProperty notEditable = new SimpleBooleanProperty(this, PSEUDO_CLASS_NOT_EDITABLE, false) { @Override protected void invalidated() { pseudoClassStateChanged(NOT_EDITABLE_PSEUDO_CLASS, get()); } }; public final BooleanProperty rowSelectedProperty() { return rowSelected; } public final boolean isRowSelected() { return rowSelectedProperty().get(); } private final BooleanProperty rowSelected = new SimpleBooleanProperty(this, PSEUDO_CLASS_ROW_SELECTED, false) { @Override protected void invalidated() { pseudoClassStateChanged(ROW_SELECTED_PSEUDO_CLASS, get()); } }; public SimpleObjectProperty recordProperty() { return record; } private SimpleObjectProperty record = new SimpleObjectProperty<>(); public DeTextFieldTableCell(StringConverter converter) { super(converter); getStyleClass().add(CLASS_DE_CELL); notEditable.bind(editableProperty().not()); tableRowProperty().addListener((ov, vOld, vNew)-> { record.unbind(); rowSelected.unbind(); if (vNew != null) { record.bind(vNew.itemProperty()); rowSelectedProperty().bind(vNew.selectedProperty()); } }); } } DeTextFieldTableCell and all other de-cell controls share identical lines of code which define Boolean properties "rowSelected" and "notEditable" and respective custom CSS pseudo-classes "row-selected" and "not-editable". Additionally, de-cells expose row object (record) with "record" property. Code, which makes editability and appearance of a cell depend on values of one or more properties of the "record" has to look like this: cell.recordProperty().addListener((ov, r0, r) -> { cell.editableProperty().unbind(); if (r != null) { cell.editableProperty().bind(r.someProperty().and(r.otherProperty())); } }); Here is sample CSS for de-cells: .de-cell:filled:not-editable { -fx-background-color: #cccccc; -fx-text-fill: #0000ff; -fx-border-width: 1px 0px 0px 1px; -fx-border-color: #eeeeee } .de-cell:filled:row-selected { -fx-background-color: skyblue; -fx-text-fill: black; -fx-border-width: 1px 0px 0px 1px; -fx-border-color: #eeeeee } .de-cell:filled:not-editable:row-selected { -fx-background-color: #999999; -fx-text-fill: #0000ff } The example below is a taken (with some simplification) from the existing working application. There is a system, which allows customers to buy online prints and file downloads. All general, non-specific files, which are accessible by general customers, have preassigned General Prices: price of 1 printed copy for printable and download price for not printable ones. Besides, there are special, tailor-made files, which are not accessible by general customers and don't have General Prices. A customer can fetch products himself (with General Price only), or some products can be pushed to him by a sales manager - with General Price or (relatively reduced) Special Price. In the first case number of the printed copies to be chosen by the customer, whereas in the last case number of printed copies has to be limited by the sales manager. In case of special, tailor-made files Special Price has to be assigned. Number of copies (quantity) for the file download is always equal 1. At some point a sales manager collects all products (both general and tailor-made) to be pushed to a customer and has to edit collected entries. He/She can change any general price to a special one, allow download of a printable product, edit Special prices and printed copies (quantities). There are following dependencies: 1. General/Special choice is enabled for the general products only. 2. Print/Download choice is enabled for printable products when Special Price is selected. 3. Price is editable when Special Price is selected, otherwise it has to be equal to preassigned General Price. 4. Quantity is editable when Special Price and Print Service are selected. When Download selected quantity is fixed and equals 1, when General and Print selected quantity is null (to be chosen by customer). For example, for a sample product "*A: General Print" the only editable dynamic field is "Price Type". If a user changes Price Type to Special then Service Type, Item Price and Quantity become editable. If a user changes Service Type to Download then Quantity become not editable (and set to 1). Here is source code for data object Entry: public class Entry { private final IntegerProperty entryId = new SimpleIntegerProperty(); private final StringProperty name = new SimpleStringProperty(); private final BooleanProperty printable = new SimpleBooleanProperty(); private final BooleanProperty useGeneralPrice = new SimpleBooleanProperty(); private final BooleanProperty usePrintService = new SimpleBooleanProperty(); private final ObjectProperty generalPrice = new SimpleObjectProperty<>(); private final ObjectProperty price = new SimpleObjectProperty<>(); private final ObjectProperty quantity = new SimpleObjectProperty<>(); private final ObjectProperty totalPrice = new SimpleObjectProperty<>(); public IntegerProperty entryIdProperty() { return entryId; } public StringProperty nameProperty() { return name; } public BooleanProperty printableProperty() { return printable; } public BooleanProperty useGeneralPriceProperty() { return useGeneralPrice; } public BooleanProperty usePrintServiceProperty() { return usePrintService; } public ObjectProperty generalPriceProperty() { return generalPrice; } public ObjectProperty priceProperty() { return price; } public ObjectProperty quantityProperty() { return quantity; } public ObjectProperty totalPriceProperty() { return totalPrice; } public Entry(int aEntryId, String aName, BigDecimal aGeneralPrice, boolean aPrintable) { entryId.set(aEntryId); name.set(aName); printable.set(aPrintable); useGeneralPrice.set(aGeneralPrice != null); usePrintService.set(aPrintable); generalPrice.set(aGeneralPrice); price.set(aGeneralPrice); quantity.set((aPrintable) ? null : 1); totalPrice.bind(new ObjectBinding() { { super.bind(price, quantity); } @Override protected Number computeValue() { return (price.get() == null || quantity.get() == null) ? null : price.get().doubleValue()*quantity.get(); } }); } } Here is an excerpt from the sample app - creation of table columns with de-cells: TableColumn priceTypeCol = new TableColumn<>("Price Type"); priceTypeCol.setPrefWidth(60); priceTypeCol.setCellValueFactory(new PropertyValueFactory<>("useGeneralPrice")); priceTypeCol.setCellFactory((tableColumn) -> { DeComboBoxTableCell cell = new DeComboBoxTableCell<>( new ABooleanConverter(PRICE_TYPES[0], PRICE_TYPES[1]), true, false); cell.recordProperty().addListener((ov, vOld, vNew) -> { cell.editableProperty().unbind(); if (vNew != null) cell.editableProperty().bind(vNew.generalPriceProperty().isNotNull()); }); return cell; }); priceTypeCol.setOnEditCommit((ev) -> { Entry entry = ev.getRowValue(); Boolean value = ev.getNewValue(); entry.useGeneralPriceProperty().set(value); if (value) { entry.priceProperty().set(entry.generalPriceProperty().get()); entry.usePrintServiceProperty().set(entry.printableProperty().get()); } entry.quantityProperty().set( (value && entry.usePrintServiceProperty().get()) ? null : 1); }); //============================================================== TableColumn serviceTypeCol = new TableColumn<>("Service Type"); serviceTypeCol.setPrefWidth(60); serviceTypeCol.setCellValueFactory(new PropertyValueFactory<>("usePrintService")); serviceTypeCol.setCellFactory((tableColumn) -> { DeComboBoxTableCell cell = new DeComboBoxTableCell<>( new ABooleanConverter(SERVICE_TYPES[0], SERVICE_TYPES[1]), true, false); cell.recordProperty().addListener((ov, vOld, vNew) -> { cell.editableProperty().unbind(); if (vNew != null) { cell.editableProperty().bind(vNew.useGeneralPriceProperty().not() .and(vNew.printableProperty())); } }); return cell; }); serviceTypeCol.setOnEditCommit((ev) -> { Entry entry = ev.getRowValue(); Boolean value = ev.getNewValue(); entry.usePrintServiceProperty().set(value); entry.quantityProperty().set(1); }); //============================================================== TableColumn priceCol = new TableColumn<>("Item Price"); priceCol.setPrefWidth(50); priceCol.setCellValueFactory(new PropertyValueFactory<>("price")); priceCol.setCellFactory((tableColumn) -> { DeTextFieldTableCell cell = new DeTextFieldTableCell<>( new AMoneyConverter()); cell.setAlignment(Pos.CENTER_RIGHT); cell.recordProperty().addListener((ov, vOld, vNew) -> { cell.editableProperty().unbind(); if (vNew != null) cell.editableProperty().bind(vNew.useGeneralPriceProperty().not()); }); return cell; }); priceCol.setOnEditCommit((ev) -> { ev.getRowValue().priceProperty().set(ev.getNewValue()); }); //============================================================== TableColumn quantityCol = new TableColumn<>("Quantity"); quantityCol.setPrefWidth(40); quantityCol.setCellValueFactory(new PropertyValueFactory<>("quantity")); quantityCol.setCellFactory((tableColumn) -> { DeTextFieldTableCell cell = new DeTextFieldTableCell<>( new IntegerStringConverter()); cell.setAlignment(Pos.CENTER); cell.recordProperty().addListener((ov, vOld, vNew) -> { cell.editableProperty().unbind(); if (vNew != null) { cell.editableProperty().bind(vNew.useGeneralPriceProperty().not() .and(vNew.usePrintServiceProperty())); } }); return cell; }); priceCol.setOnEditCommit((ev) -> { ev.getRowValue().priceProperty().set(ev.getNewValue()); }); //============================================================== TableColumn totalPriceCol = new TableColumn<>("Total Price"); totalPriceCol.setPrefWidth(50); totalPriceCol.setCellValueFactory(new PropertyValueFactory<>("totalPrice")); totalPriceCol.setCellFactory((tableColumn) -> { DeTextFieldTableCell cell = new DeTextFieldTableCell<>( new AMoneyConverter()); cell.setAlignment(Pos.CENTER_RIGHT); cell.setEditable(false); return cell; }); totalPriceCol.setEditable(false); --
July 2, 2015
by Felix Golubov
· 13,080 Views
article thumbnail
Using Liquibase Without a Database Connection
There are many, many different processes and requirements companies have for managing their database schemas. Some allow the application to directly manage them on startup, some require SQL scripts be executed by hand. Some have schemas that can differ across customers, some have only one database to deal with. For people who prefer to execute SQL themselves, Liquibase has always supported an “updateSQL” mode which does not update the database but instead outputs what would be run. This allows developers and DBAs to know exactly what will be ran and even make modifications as needed before actually executing the script. Before version 3.2, however, Liquibase required an active database connection for updateSQL. It used that connection to determine the SQL dialect to use and to query the DATABASECHANGELOG table to learn what changeSets have already been executed. Controlling updateSql SQL Syntax With version 3.2, Liquibase added a new “offline” mode. Instead of specifying a jdbc url such as “jdbc:mysql://localhost/lbcat” you can use “offline:mysql” or “offline:postgresql” which lets Liquibase know what dialect to use. For finer dialect control, you can specify parameters like “offline:mysql?version=3.4&caseSensitive=false Available dialect parameters: version: Standard X.Y.Z version of the database productName: String description of the database, like the JDBC driver would return catalog: String containing the name of the default top-level container ('database' in some databases 'schema' in others) caseSensitive: Boolean value specifying if the database is case sensitive or not Tracking History With CSV These parameters let Liquibase know what SQL to generate for each changeSet, but without an active database connection you cannot rely on the DATABASECHANGELOG table to track what changeSets have already been ran. Instead, offline mode uses a CSV file which mimics the structure of the DATABASECHANGELOG table. By default, Liquibase will use a file called “databasechangelog.csv” in the working directory, but it can be specified with a “changeLogFile” parameter such as “offline:mssql?changeLogFile=path/to/file.csv” It is up to you to ensure that the contents of the csv file match what is in the database. Running updateSQL automatically appends to the CSV file under the assumption that you will apply the SQL to the database. Since the csv file matches a particular database, it isn’t something you normally would store or share under version control because every database can (and probably will) be in a different state. If you do store the files in a central location, you will probably want to at least have a separate file for each database. By default, the SQL generated by updateSql in offline mode will still contain the standard DATABASECHANGELOG insert statements, so each database that you apply the SQL to will still have a correct DATABASECHANGELOG table. This means that you can switch between a direct-connection update and offline updateSQL as needed. It also means that you can also extract the current contents of the DATABASECHANGELOG table to a CSV file and use that as the file passed to the offline connection to ensure you have the right contents in the file. If you do not want the DATABASECHANGELOG table SQL included in updateSQL output, there is an “outputLiquibaseSql” parameter which can be passed in your offline url. Possible outputLiquibaseSql values: "none" will output no DATABASECHANGELOG statements "data_only" will output only INSERT INTO DATABASECHANGELOG statements "all" will output CREATE TABLE DATABASECHANGELOG if the csv file does not exist as well as INSERT statements (default value) Offline Snapshots The new 3.4.0 release of Liquibase expands offline support with a new “snapshot” parameter which can be passed to the offline url pointing to a saved database structure. Liquibase will use the snapshot anywhere it would have normally needed to read the current database state. This allows you to use preconditions and perform diff and diffChangeLog operations without an active connection and even between snapshots of the same database from different points in time. To create a snapshot of your live databases, use the “—snapshotFormat=json” parameter on the “snapshot” command. Command line example: $ liquibase --url=jdbc:mysql://localhost/lbcat snapshot --snapshotFormat=json > snapshot.json or $ liquibase --url=jdbc:mysql://localhost/lbcat –outputFile=path/to/output.json snapshot --snapshotFormat=json NOTE: currently only “json” is supported as a snapshotFormat. You can then use that file with your offline url and any snapshot operations will use it as the database state. liquibase –url=jdbc:mysql://localhost/lbcat –referenceUrl=offline:mysql?snapshot=path/to/snapshot.json diff will compare the stored snapshot with the current database state liquibase –url=offline:mysql?snapshot=path/to/snapshot.json diff –referenceUrl=offline:mysql?snapshot=path/to/older-snapshot.json diff will compare two snapshots liquibase –url=offline:mysql?snapshot=path/to/snapshot.json generateChangeLog will generate a changelog based on what is in the snapshot liquibase –url=jdbc:mysql://localhost/lbcat –referenceUrl=offline:mysql?snapshot=path/to/snapshot.json diffChangeLog will generate a changelog based on what is new in the real database compared to what is in the snapshot.
July 2, 2015
by Nathan Voxland
· 10,897 Views
article thumbnail
Emerging Niches and Technologies in Mobile App Development
If there have been wide array of successful consumer apps like Angry Bird or WhatsApp or DropBox. After years of reign in the publicity focus finally these consumer apps giants understood the importance of offering enterprise grade features. In last few years suddenly the focus shifted to enterprise mobile apps. Rapid development, tracking or monitoring apps, wearable apps, Internet of Things Apps, Geo-location technologies like iBeacon and Geofencing in business apps, the list of emerging app niches and technologies seem to be too long. Let us have a quick look at some of the most definitive app niches and technologies in recent times. Enterprise apps While smartphones and mobile devices continue to move off the shelves and millions of apps continue to make the app stores brimming with energy, activity and competition, most consumer app still fail to make a earning to survive beyond the year of their launch. This has been the sordid storyline for consumer apps for years. So, for some time the focus of developers is shifting towards enterprise apps. Moreover, now businesses are bent on going mobile and they are keen to develop apps that make their business process more productive. Although enterprise mobile apps have just started to take off this new and broad app niche already shown huge promise to take over consumer apps in just more than a year down the line. Rapid development As enterprises now focusing all out to embrace mobile apps in their business process, the new demand of enterprise grade apps made rapid development cycle obvious. When winning competition for businesses is boiling down to a fast and user focused mobile presence, fast paced development will naturally be the rule. This overwhelming demand of business apps and enterprise grade software made rapid development a criterion in the present scenario. Shortening the development lifecycle has now become the major focus for most mobile app development companies around the world. Mobile monitoring apps Wide adaptation of mobile devices and apps among all age groups and people in recent years gave rise to certain concerns. Child security concern, parental concern for negative influence on children, employer’s concern on employee productivity and information security, etc. are some of the major concerns centered on the mobile devices. IOS or Android monitoring software, child phone tracker apps, mobile spy software, text message tracking apps, are few of the app types getting increasingly popular these days to address the aforementioned concerns in family or workplace environments. Internet of Things (IOT) apps The world around us is becoming connected with the mobile devices and gadgets and devices around us are increasingly finding themselves equipped with mobile control interface. This new horizon of interconnected devices is referred as Internet of Things or IOT. Now an electric toaster can be controlled from its respective app on the mobile device. Similarly, the music system with the respective mobile app can be turned on and off, tuned in and given other commands. This new breed of apps is being called IOT apps. Wearable apps The smartphones or smart mobile devices are now playing the central role in connecting all types of wearable smart devices. Most smartwatch apps are still now in character only the extension of their mobile counterparts. But as smartwatch is slowly picking up to be the next big device platform as commonest wearable, a new breed of apps are being developed targeting smartwatch and wearable users besides offering their respective mobile apps as well. From smart jewelries to health trackers and fitness bands to optically mounted computers like Google Glass, these new wearable devices will be the target development platform for a vast majority of mobile app developers in the time to come. More user-optimized mobile UI design UI design is presently the most focus driven area for mobile app development around the world. Experiments and analysis on making UIs better and user optimized is continuing and a wide variety of new techniques and design approaches are giving birth to unprecedented level of excellence in user experience. From motivational design to flat design to and playful interfaces, we have come across quite a few dominating design trends and techniques. Geo-location technologies Contextual and user specific push notification is the new maneuver to engage users with a mobile app and to garner revenue from the process. This cannot be better done than by knowing the user location. When you know the location of a user close to your retail shop you can notify him with an offer to grab his attention and push him for a visit to your store. Thus knowing the user location translates to far better contextual and business driven messaging and notifications. Several mobile friendly Geo-location technologies like iBeacon, Geofencing, Geomagnetics, etc. are there to let you integrate location based user engagement features in your app.
July 2, 2015
by Juned Ghanchi
· 3,876 Views
article thumbnail
Captains with Benefits
When it comes to teaching or learning, video streaming is something that still frightens people away. As a matter of fact that video chats and webinars have been around for a relatively long time, however; its still hard to encourage an individual or business to take part as such. And yet the benefits of CaptainLive can be substantial in both, short as well as long term. As we have already seen the benefits of video marketing therefore, we want to encourage you to use CaptainLive in order to take advantage of your potential whether it’s hidden in you or you are well aware of it. CaptainLive was launched in early 2015 with a mission to connect people in need of knowledge and skills with Captains with Benefits that are willing to share and give their expertise and mentor skills. CaptainLive’s integrated service now allows for text, video and audio conferencing. It’s been used by a variety of individuals with different backgrounds. At CaptainLive you can schedule an online live video stream with the experts in number topics ranging from counseling up to entertainment. Captains/Experts on the site charges from $5 USD up to $150 USD, most of which offer free 5 minute sessions with no obligation to book their session thereafter. Who knows you might end up registering as Captain yourself and start a part time business of your own to help others with your skills while making a healthy stream of income for yourself, it’s surely well worth your effort.
July 1, 2015
by Peter Watson
· 888 Views
article thumbnail
Interoute Virtual Data Centre is the fastest transatlantic cloud service
Double the throughput and lower latency than the leading global cloud providers between the US and Europe in independent comparison research London & New York, 1 July, 2015. Interoute has today announced that its global cloud platform Interoute Virtual Data Centre (VDC), has been proven to deliver nearly double the throughput across the Atlantic than the next best cloud provider in comparison research conducted by Cloud Spectator. The research from March 2015 compared Interoute VDC with three leading cloud providers (Amazon AWS, Rackspace and Microsoft Azure), testing network throughput and latency between Europe and USA and between providers' European data centres. In all of the comparisons, Interoute VDC demonstrated the highest throughputs and lowest latencies. Cloud Spectator's full research report, and more information about Interoute VDC's performance and features, can be viewed here: http://bit.ly/1GHyzwJ Network performance is a significant factor in cloud computing for business services requiring the highest network capacity (throughput) and the shortest possible time from the server to the client (latency), to meet the needs of the businesses and their users. Innovating new applications and business services in the cloud needs network performance to match and this report shows the advantages of building the cloud into a huge global high performance network. Key research findings: Transatlantic: Interoute VDC delivered 1.1 Gbit/s throughput, which was 96% better than Amazon AWS, 141% better than Rackspace, and 195% better than Microsoft Azure. Interoute VDC had the lowest latency, between its London and New York data centres. Interoute was the only provider in the comparison with both of its transatlantic data centres located in key business cities, meaning that VDC users can access compute and storage resources, and deliver data to their customers, from two centres of European and US business activity. Within Europe: Interoute VDC achieved 1.3 Gbit/s throughput between its London and Amsterdam data centres. This was 52% better than Amazon AWS (Dublin - Frankfurt) and 73% better than Microsoft Azure (Dublin - Amsterdam) Interoute VDC achieved a latency of 6 milliseconds between London and Amsterdam, over three times better than the inter-data centre latency of the comparison providers. Matthew Finnie, CTO of Interoute, commented: "This independent report confirms and validates our networked cloud strategy. Building cloud into a world class network provides our customers with significantly better performance when compared with the traditional cloud models. Businesses looking to grow between Europe and US should definitely be looking at the importance of these network characteristics for their ability to shift workloads into the cloud. Interoute's fourteen global zones are all built into high performance network with over 300 interconnects in Europe alone. So wherever you choose to put your data and connect to us, your services are typically going to perform faster on Interoute than on many other global providers." Danny Gee, Senior Analyst, Cloud Spectator: "Users want to transfer large amounts of data between data centres quickly. Our study revealed that for a trans-Atlantic connection between cloud data centers, Interoute provided the highest throughput and lowest latency out of AWS, Rackspace and Azure. Interoute also had the higher network throughput and lowest latency in European testing compared to Azure and AWS (Rackspace was excluded, having only one location in Europe), making it a good option for users operating servers within this region. Interoute also provided the best latency, ideal for real-time communications. Users running geographically dispersed environments for such things as geo-redundancy would benefit from Interoute's high performance cloud connectivity."
July 1, 2015
by Fran Cator
· 1,215 Views
article thumbnail
Learning Spring-Cloud - Writing a Microservice
Continuing my Spring-Cloud learning journey, earlier I had covered how to write the infrastructure components of a typical Spring-Cloud and Netflix OSS based micro-services environment - in this specific instance two critical components, Eureka to register and discover services and Spring Cloud Configuration to maintain a centralized repository of configuration for a service. Here I will be showing how I developed two dummy micro-services, one a simple "pong" service and a "ping" service which uses the "pong" service. Sample-Pong microservice The endpoint handling the "ping" requests is a typical Spring MVC based endpoint: @RestController public class PongController { @Value("${reply.message}") private String message; @RequestMapping(value = "/message", method = RequestMethod.POST) public Resource pongMessage(@RequestBody Message input) { return new Resource<>( new MessageAcknowledgement(input.getId(), input.getPayload(), message)); } } It gets a message and responds with an acknowledgement. Here the service utilizes the Configuration server in sourcing the "reply.message" property. So how does the "pong" service find the configuration server, there are potentially two ways - directly by specifying the location of the configuration server, or by finding the Configuration server via Eureka. I am used to an approach where Eureka is considered a source of truth, so in this spirit I am using Eureka to find the Configuration server. Spring Cloud makes this entire flow very simple, all it requires is a "bootstrap.yml" property file with entries along these lines: --- spring: application: name: sample-pong cloud: config: discovery: enabled: true serviceId: SAMPLE-CONFIG eureka: instance: nonSecurePort: ${server.port:8082} client: serviceUrl: defaultZone: http://${eureka.host:localhost}:${eureka.port:8761}/eureka/ The location of Eureka is specified through the "eureka.client.serviceUrl" property and the "spring.cloud.config.discovery.enabled" is set to "true" to specify that the configuration server is discovered via the specified Eureka server. Just a note, this means that the Eureka and the Configuration server have to be completely up before trying to bring up the actual services, they are the pre-requisites and the underlying assumption is that the Infrastructure components are available at the application boot time. The Configuration server has the properties for the "sample-pong" service, this can be validated by using the Config-servers endpoint - http://localhost:8888/sample-pong/default, 8888 is the port where I had specified for the server endpoint, and should respond with a content along these lines: "name": "sample-pong", "profiles": [ "default" ], "label": "master", "propertySources": [ { "name": "classpath:/config/sample-pong.yml", "source": { "reply.message": "Pong" } } ] } As can be seen the "reply.message" property from this central configuration server will be used by the pong service as the acknowledgement message Now to set up this endpoint as a service, all that is required is a Spring-boot based entry point along these lines: @SpringBootApplication @EnableDiscoveryClient public class PongApplication { public static void main(String[] args) { SpringApplication.run(PongApplication.class, args); } } and that completes the code for the "pong" service. Sample-ping micro-service So now onto a consumer of the "pong" micro-service, very imaginatively named the "ping" micro-service. Spring-Cloud and Netflix OSS offer a lot of options to invoke endpoints on Eureka registered services, to summarize the options that I had: 1. Use raw Eureka DiscoveryClient to find the instances hosting a service and make calls using Spring's RestTemplate. 2. Use Ribbon, a client side load balancing solution which can use Eureka to find service instances 3. Use Feign, which provides a declarative way to invoke a service call. It internally uses Ribbon. I went with Feign. All that is required is an interface which shows the contract to invoke the service: package org.bk.consumer.feign; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.springframework.cloud.netflix.feign.FeignClient; import org.springframework.http.MediaType; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.ResponseBody; @FeignClient("samplepong") public interface PongClient { @RequestMapping(method = RequestMethod.POST, value = "/message", produces = MediaType.APPLICATION_JSON_VALUE, consumes = MediaType.APPLICATION_JSON_VALUE) @ResponseBody MessageAcknowledgement sendMessage(@RequestBody Message message); } The annotation @FeignClient("samplepong") internally points to a Ribbon "named" client called "samplepong". This means that there has to be an entry in the property files for this named client, in my case I have these entries in my application.yml file: samplepong: ribbon: DeploymentContextBasedVipAddresses: sample-pong NIWSServerListClassName: com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList ReadTimeout: 5000 MaxAutoRetries: 2 The most important entry here is the "samplepong.ribbon.DeploymentContextBasedVipAddresses" which points to the "pong" services Eureka registration address using which the service instance will be discovered by Ribbon. The rest of the application is a routine Spring Boot application. I have exposed this service call behind Hystrix which guards against service call failures and essentially wraps around this FeignClient: package org.bk.consumer.service; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.bk.consumer.feign.PongClient; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Service; @Service("hystrixPongClient") public class HystrixWrappedPongClient implements PongClient { @Autowired @Qualifier("pongClient") private PongClient feignPongClient; @Override @HystrixCommand(fallbackMethod = "fallBackCall") public MessageAcknowledgement sendMessage(Message message) { return this.feignPongClient.sendMessage(message); } public MessageAcknowledgement fallBackCall(Message message) { MessageAcknowledgement fallback = new MessageAcknowledgement(message.getId(), message.getPayload(), "FAILED SERVICE CALL! - FALLING BACK"); return fallback; } } Boot"ing up I have dockerized my entire set-up, so the simplest way to start up the set of applications is to first build the docker images for all of the artifacts this way: mvn clean package docker:build -DskipTests and bring all of them up using the following command, the assumption being that both docker and docker-compose are available locally: docker-compose up Assuming everything comes up cleanly, Eureka should show all the registered services, at http://dockerhost:8761 url - The UI of the ping application should be available at http://dockerhost:8080 url - Additionally a Hystrix dashboard should be available to monitor the requests to the "pong" app at this url http://dockerhost:8989/hystrix/monitor?stream=http%3A%2F%2Fsampleping%3A8080%2Fhystrix.stream: References 1. The code is available at my github location - https://github.com/bijukunjummen/spring-cloud-ping-pong-sample 2. Most of the code is heavily borrowed from the spring-cloud-samples repository - https://github.com/spring-cloud-samples
July 1, 2015
by Biju Kunjummen
· 13,727 Views · 4 Likes
article thumbnail
Gene Kim Explains ‘Why DevOps Matters’
Ever wonder why DevOps gets so much attention these days? The answer is simple: “DevOps solves the most important business problem of our generation, [which is] how organizations make the transition from good to great.” That’s according to Gene Kim, co-author of The Phoenix Project, founder of Tripwire, and a DevOps advocate. Gene headlined a New Relic DevOps roadshow with stops in Chicago, Dallas, and Houston last month, regaling attendees with the inside scoop of what DevOps really is, what it does, and how to make it work (more on that in upcoming blog posts). But perhaps his most important point was the overwhelming importance of the effort. Traditional IT leads to “hopelessness and despair” According to Gene, the opportunity cost of wasted IT spending is some $2.6 trillion. These days, he says, “every company is an IT company”—we like to say “every company is a software company,” but you get the message. Gene observes that 95% of all capital projects have an IT component and 50% of all capital spending is technology related. And every IT organization is pressured to simultaneously respond more quickly to urgent business needs while also providing stable, secure, and predictable IT service. That chronic conflict created what Gene described as “a horrible downward spiral that leads to horrendous outcomes. Every time we cut corners, or manually deploy code, or write code that doesn’t have automated testing, it all leads to the accumulation of technical debt.” And the ever-increasing amount of technical debt sets the stage for intertribal warfare that can exist between dev and ops. Those wars mean that “Devs submit code at 5 p.m. on Friday, and ops then works all weekend to deploy it by 9 a.m. Monday. Everyone becomes buried in unplanned work, and this deprives our ability to pay down the technical debt being created. This led to hopelessness and despair, with everyone doomed to repeat the same mistakes.” DevOps offers a better way Fortunately, Gene explained, “We know now there is a better way. The DevOps exemplars have shown us that we can have incredibly fast flow from dev to ops to deployment while preserving world-class quality and security.” According to Gene, the top predictors of IT performance are all associated with DevOps: Version control of all production artifacts Continuous integration and deployment Automated acceptance testing Peer-review of production changes (vs. external change approval) High-trust culture Proactive monitoring of the production environment Win-win relationship between dev and ops Lead time is the key metric Lead time from raw material to finished product is the key metric in manufacturing, “and that’s true for software, too,” Gene said. “How long does it take to go from code committed to code successfully running in production?” The standard 9-month software lead time common in waterfall development projects is “highly correlated with catastrophic deployment errors,” Gene warned. The key, he said, is to have smaller deployments, and to do them more frequently. That approach is already working for high-performing organizations, he added, who are accelerating away from the herd. “Ten deploys a day used to be startling,” Gene noted. “Now it’s probably considered merely average among high performers.” Amazon Web Services deploys every 11.6 seconds! That kind of speed is possible only by doing small deployments more frequently, Gene said. “The bigger the change, the bigger the crater when it hits.” DevOps correlates with business success! IT high-performers who incorporate DevOps are much more agile and more reliable, Gene said. Critically, he added, “They are more likely to win in the marketplace!” The common reaction to that statement is shock. Gene noted he often hears: “That’s absurd! How can IT ops practices be visible on the bottom line or in the stock price?” But the Puppet Labs 2014 State of DevOps report noted that IT high-performers are twice as likely to exceed profitability, market share, and productivity goals as well as enjoy 50% higher market capitalization growth over three years. Of course, that doesn’t mean all those good things will happen to your company just by moving to DevOps. But do you really want to risk the “horrendous outcomes” of staying with outmoded models that lead to excruciatingly long deployment cycles?
July 1, 2015
by Fredric Paul
· 1,929 Views
article thumbnail
Getting Started with D3.js
Originally authored by Elizabeth Engel You are thinking about including some nice charts and graphics into your current project? Maybe you heard about D3.js, as some people claim it the universal JavaScript visualization framework. Maybe you also heard about a steep learning curve. Let’s see if this is really true! First of all, what is D3.js? D3.js is an open source JavaScript framework written by Mike Bostock helping you to manipulate documents based on data. Okay, let’s first have a look at the syntax Therefore, lets look at the following hello world example. It will append an element saying ‘Hello World!’ to the content element. As you can see the syntax is very similar to frameworks like JQuery and obviously, it saves you a lot of code lines as it offers a nice fluent API. But let’s see how we can bind data to it: d3.select('#content') .selectAll('h1') .data(['Sarah', 'Robert', 'Maria', 'Marc']) .enter() .append('h1') .text(function(name) {return 'Hello ' + name + '!'}); What happens? The data function gets our names array as parameter and for each name we append an element with a personalized greeting message. For a second, we ignore the selectAll(‘h1′) and enter() method call, as we will explore them later. Looking into the browser we can see the following: Hello Sarah! Hello Robert! Hello Maria! Hello Marc! Not bad for a start! Inspecting the element in the browser, we see the following generated markup: [...] Hello Sarah! Hello Robert! Hello Maria! Hello Marc! [...] This already shows one enourmous advantage of D3.js: You acctually see the generated code and can spot errors easily. Now, let’s have a closer look at the data-document connection As mentioned in the beginning, D3.js helps you to manipulate documents based on data. Therefore, we only take care about handing the right data over to D3.js, so the framework can do the magic for us. To understand how D3.js handles data, we’ll first have a look at how data might change over time. Let’s take the document from our last example. Every name is one data entry. Easy. Now let’s assume new data comes in: As new data is coming in, the document needs to be updated. The entries of Robert and Maria need to be removed, Sarah and Marc can stay unchanged and Mike, Sam and Nora need a new entry each. Fortunately, using D3.js we don’t have to care about finding out which nodes need to be added and removed. D3.js will take care about it. It will also reuse old nodes to improve performance. This is one key benefit of D3.js. So how can we tell D3.js what to do when? To let D3.js update our data, we initially need a data join, so D3.js knows our data. Therefore, we select all existing nodes and connect them with our data. We can also hand over a function, so D3.js knows how to identify data nodes. As we initally don’t have nodes, the selectAll function will return an empty set. var textElements = svg.selectAll('h1').data(data, function(d) { return d; }); After the first iteration, the selectAll will hand over the existing nodes, in our case Sarah, Robert, Marc and Maria. So we can now update these existing nodes. For example, we can change their CSS class to grey: textElements.attr({'class': 'grey'}); Additionally, we can tell D3.js what to do with entering nodes, in our case Mike, Sam and Nora. For example, we can add an element for each of them and set the CSS class to green for each of them: textElements.enter().append('h1').attr({'class': 'green'}); As D3.js now updated the old nodes and added the new ones, we can define what will happen to both of them. In our cases this will affect the nodes of Mike, Sarah, Sam, Mark and Nora. For example, we can rotate them: textElements.attr({'transform', rotate(30 20,40)}); Furthermore, we can specify what D3.js will do to nodes like Robert and Maria, that are not contained in the data set any more. Let’s change their CSS class to red: textElements.exit().attr({'class': 'red'}); You can find the full example code to illustrate the data-document connection of D3.js as JSFiddle here: https://jsfiddle.net/q5sgh4rs/1/ But how to visualize data with D3.js? Now that we know about the basics of D3.js, let’s go to the most interesting part of D3.js: drawing graphics. To do so, we use SVG, which stands for scalable vector graphics. Maybe you already know it from other contexts. In a nutshell, it’s a XML-based vector image language supporting animation and interaction. Fortunately, we can just add SVG tags in our HTML and all common browsers will display it directly. This also facilitates debugging, as we can inspect generated elements in the browser. In the following, we see some basic SVG elements and their attributes: To get a better understanding of how SVG looks like, we’ll have a look at it as a basic example of SVG code, generating a rectangle, a line and a circle. To generate the same code using D3.js, we need to add an SVG to our content and then append the tree elements with their attributes like this: var svg = d3.select('#content').append('svg'); svg.append('rect').attr({x: 10, y: 15, width: 60, height: 20}); svg.append('line').attr({x1: 85, y1: 35, x2: 105, y2: 15}); svg.append('circle').attr({cx: 130, cy: 25, r: 6}); Of course, for static SVG code, we wouldn’t do this, but as we already saw, D3.js can fill attributes with our data. So we are now able to create charts! Let’s see how this works: This will draw our first bar chart for us! Have a look at it: https://jsfiddle.net/tLhomz11/2/ How to turn this basic bar chart into an amazing one? Now that we started drawing charts, we can make use of all the nice features D3.js offers. First of all, we will adjust the width of each bar to fill the available space by using a linear scale, so we don’t have to scale our values by hand. Therefore, we specify the range we want to get values in and the domain we have. In our case, the data is in between 0 and 200 and we would like to scale it to a range of 0 to 400, like this: var xScale = d3.scale.linear().range([0, 400]).domain([0,200]); If we now specify x values, we just use this function and get an eqivalent value in the right range. If we don’t know our maximum value for the domain, we can use the d3.max() function to calculate it based on the data set we want to display. To add an axis to our bar chart, we can use the following function and call it on our SVG. To get it in the right position, we need to transform it below the chart. [svg from above].call(d3.svg.axis().scale(xScale).orient("bottom")); Now, we can also add interaction and react to user input. For example, we can give an alert, if someone clicks one our chart: [svg from above].on("click", function () { alert("Houston, we get attention here!"); }) Adding a text node for each line, we get the following chart rendered in the browser: If you would like to play around with it, here is the code: https://jsfiddle.net/Loco5ddt/ If you would like to see even more D3.js code, using the same data to display a pie chart and adding an update button, look at the following one: https://jsfiddle.net/4eqzyquL/ Data import Finally, we can import our data in CSV, TSV or JSON format. To import a JSON file, for example, use the following code. Of course, you can also fetch your JSON via a server call instead of importing a static file. d3.json("data.json", function(data) { [access your data using the data variable] } What else does D3,js offer? Just to name a few, D3.js helps you with layouts, geometry, scales, ranges, data transformation, array and math functions, colors, time formating and scales, geography, as well as drag & drop. There are a lot of examples online: https://github.com/mbostock/d3/wiki/Gallery TL;DR + based on web standards + totally flexible + easy to debug + many, many examples online + Libaries build on D3.js (NVD3.js, C3.js or IVML) – a lot of code compared to other libraries – for standard charts too heavy Learning more As this blog post is based on a presentation held at the MunichJS Meetup, you can find the original slides here: http://slides.com/elisabethengel/d3js#/ The recording is available on youTube: https://www.youtube.com/watch?v=EYmJEsReewo For further information, have a look at: https://github.com/mbostock/d3/wiki https://github.com/mbostock/d3/wiki/API-Reference http://bost.ocks.org/mike/ Getting Started with D3 by Mike Dewar Interactive Data Visualization for the Web by Scott Murray (online for free) https://github.com/alignedleft/d3-book If you have any questions or suggestions, feel free to comment or write me: [email protected]
June 30, 2015
by Comsysto Gmbh
· 6,494 Views · 3 Likes
article thumbnail
DevOps Tools for Continuous Delivery: Workloads Distribution and Jenkins Installation
the vast majority of software development companies have to place a great emphasis on the process of continuous integration and rapid delivery of new versions of their product. obviously, when supplying enterprise-level projects, such processes need to be automated as much as possible. and this is when the cloud devops tools come in handy. thus, in today’s article we’d like to pay a special attention to the devops tools that automate the continuous integration and delivery within the jelastic paas that can be installed on any bare metal or cloud infrastructure as virtual private cloud or hybrid cloud. this is a pretty complex example of enterprise application life cycle with continuous integration and seamless migration throughout devops pipeline from development to several productions (you can use simplified process if you have less complex project ). the instruction below will be useful for jelastic cluster administrators such as systems integrators, hosting service providers, enterprises, and isv customers, who can easily implement it at their jelastic cloud installations. nevertheless, this guide contains plenty of features and continuous integration tips described, which can be interesting for different developers. so, let’s get started with the first part of the instruction! setting up dedicated user groups first of all, you need to allocate separate hardware sets for all your project teams (one per each development phase, i.e. development > testing > production ) and adjust the access permissions to make them completely isolated and not influenced by others. the multi-regions for a hybrid cloud option, that became available within the recently released jelastic 3.3 version , is optimally suited for this task. to start with, create three hardware node groups (within one region) and name them after the corresponding stages for more convenience (e.g. dev , test , production ). the next step is to prepare three user groups and attach them to the corresponding hardware – in our case the dev group has access to the dev hardware node group only, qa – to the test one, and ops should work specifically with the production set. in such a way, users from the appropriate groups can use the specified sets of hardware only, but at the same time – they have a possibility to transfer their environments throughout the whole platform, between different teams’ accounts. jenkins continuous integration server configuration now we need the integration tool, that will control and perform all of the required operations automatically, i.e. build the cloud devops pipeline. our choice fell on jenkins as one of the most popular solutions used for this goal – it can be easily installed from our marketplace either at the corresponding site page or directly via the dashboard . as a result, you’ll get the pure jenkins installed, which should be properly adjusted before you start organizing your application life cycle. thus, select the open in browser button and proceed with the following configurations steps: while at the home page, click on the manage jenkins option at the left-hand menu and select the manage plugins link within the appeared list. after you’ve been redirected to the plugin manager, switch to the available tab, find the following plugins using the search filter field above and tick them for installation: git plugin – is required for building our project’s source (stored at the github repository) envfile plugin – is used for storing system environment variables (its necessity is driven by security restrictions, implemented at jelastic, which forbid the direct exporting of environment variables from the tomcat server) click install without restart when ready. during the installation process, tick the restart jenkins when installation is complete and no jobs are running option to automatically restart jenkins for enabling the chosen plugins. then, you also need to install maven, which will be used for building the project. for that, navigate to the manage jenkins > configure system menu, scroll down to the maven section and click add maven. within the expanded section, type the desired name for your maven installation (e.g. maven ) and save the changes using the same-named button at the bottom of the page. in such a way, this tool will be also automatically installed when required (i.e. during the first app build). now your jenkins server is well-staffed for the further work. add deployment process scripts to the jenkins container the next step is to upload the scripts that you are going to use for automating different organizational actions, required to be applied to your application at the intermediate development life cycle phases (like deploying, placing it to the appropriate hardware according to the stage, running auto-test, etc). the easiest way to do this is to access your jenkins container via the jelastic ssh-gateway. in the case you haven’t performed similar operations before, you need to: generate an ssh keypair add your public ssh key to the dashboard access your account via ssh protocol once inside, create a new folder for your project (we’ll use demo ) and move in there: mkdir /opt/tomcat/demo cd /opt/tomcat/demo this location can be used for storing your scripts, variables, logs etc. here, you can upload the required scripts using the command of the following type: curl -fssl {link_to_script} -o {file_name} we also provide the set of script examples, which can be used as templates for your own ones: install.sh – gets a user session and creates a new environment via the jelastic api according to the specified manifest file. it also defines, that the name of this environment will be equal to its creation date and time (as a unique name is required for every script execution, but you won’t be able to set it manually as this operation would be run automatically). however, you can set your own dynamic name pattern to be used here transfer.sh – changes the environment ownership based on the jelastic environment transferring feature migrate.sh – physically moves an environment to another hardware set (hardnode group) note: that before the appliance, each of the script templates, presented above, have to be additionally adjusted to make them work properly within a particular jelastic installation. thus, the list of parameters below should be obligatory substituted according to your platform’s settings: /path/to/scripts/ – the full path to your scripts folder (created in the previous step) {cloud_domain} – your jelastic platform domain name {jca_dashboard_appid} – your dashboard id, that could be seen within the platform.dashboard_appid parameter at the jca > about section {jca_appstore_appid} – appstore id, listed within the same section (at the platform.appstore_appid parameter) {url_to_manifest} – link to the manifest file created according to our documentation (you may also use this one as an example – it sets up two tomcat application servers with the nginx load-balancer in front of them) note: above you can see one more runtest.sh script uploaded – it simulates the testing activities for demonstration purposes, thus we don’t provide its code in this tutorial. if required, create your own one according the specifics of your application and upload it alongside the rest of the scripts. in addition, you need to create a separate file for storing the variable with environment name (as it needs to be dynamically changed each time a new environment is created): echo env_name= > /opt/tomcat/demo/variables these are the main steps of preparation to achieve automatic continuous integration and delivery of your web application with a help of jenkins within jelastic cloud platform. in the second part of these blog series, we’ll configure the set of jobs at the jenkins server, which represents the core of our automation. each of them will be devoted for a particular operation, required to be run at the corresponding application life cycle phase: create environment > build and deploy > dev tests > migrate to qa > qa tests > migrate to production stay tuned to see the next steps. if you still don’t have jelastic installation, contact us to get access to our free demo for cloud platform evaluation or just start with trial registration at one of our hosting partners .
June 30, 2015
by Tetiana Markova
· 3,191 Views · 1 Like
article thumbnail
Enterprises Signing Up For More Than One New Cloud Service Every Day
No weekend downtime for cloud risk in Europe warns Skyhigh Networks in new report Report: info.skyhighnetworks.com/rs/274-AUP-214/images/WP-Cloud-Adoption-and-Risk-Report-Q2-EU.pdf LONDON - 30 JUNE 2015 - European enterprises are adding new cloud services at a rapid rate, finds a new report from Skyhigh Networks, the cloud security and enablement company. According to The Cloud Adoption and Risk Report for Q2 2015 - based on real data from over 2.5 million European employees across 12,000 cloud services - the average number of services has increased by more than 365 in a year. More than ten percent of cloud activity takes place outside the Monday-Friday working week, as overall cloud use continues to surge. Key findings of the report include: The average European enterprise now uses 897 cloud services (and a minimum of 507 services), up 61% from a year ago. Cloud usage never stops - with over 14% of traffic taking place at weekends 72.1% of European organisations have exposure to compromised credentials and 8.5% of employees have at least one compromised credential for sale on the darknet Security measure adoption: Only 15.4% of cloud services support multi-factor authentication, 9.4% encrypt data at rest and 2.8% support ISO 27001 64.9% of cloud services are not safe for EU data Enabled by cloud computing, flexible working has had a huge impact on the enterprise and the amount of work conducted out of office hours and on weekends. Skyhigh Networks analysed cloud usage by day of the week and found that weekend usage did not drop to zero, with Saturdays and Sundays representing 6.8% and 7.8% of weekly cloud usage respectively. IT departments therefore need to consider that enterprise exposure to cloud-related security risks is not tied to traditional office hours. The report also found that cloud adoption in Europe continues to grow, with the average European enterprise now using 897 services, 61% more than the same quarter in 2014. Indeed, from its database, Skyhigh found that the minimum number of services in use by a single organisation is 507 (for a company with less than 200 employees) and the highest is more than 3,000. "European business use of cloud is at an all-time high," said Nigel Hawthorn, European spokesperson for Skyhigh Networks. "Companies are adding a new cloud service to their network each day and it won't be long until the average organisation is using well over 1,000 distinct services. While cloud services offer clear agility gains for the enterprise, the situation is by no means perfect. Too few cloud services are suited to enterprise use. The vast majority fall at the first hurdle: failure to adopt the right IT security features or adhere to the EU Data Protection regulations." The report investigated current cloud service security capabilities. Just 7% of the 12,000 cloud services analysed meet enterprise security and compliance requirements, as rated by Skyhigh's CloudTrust Program. Only 15.4% support multi-factor authentication, 2.8% have ISO 27001 certification, and 9.4% encrypt data stored at rest. More worrying still, Skyhigh founds that 72.1% of European organisations have exposure to compromised credentials and 8.5% of employees at European companies have at least one compromised credential for sale on the darknet. "While security measures are increasingly being introduced by CSPs, our findings that no user is the same, combined with the fact that each enterprise is using an additional cloud service each day, demonstrates the sheer complexity of the issue. IT departments need to get their heads out of the clouds - or perhaps in it - and take an active role in ensuring their enterprise isn't weighed down by risks, poorly thought out cloud usage policies, and blanket bans. In a nutshell, the potential rewards of a cloud-enabled business are just too good for an apathetic approach to cloud," concluded Hawthorn.
June 30, 2015
by Fran Cator
· 858 Views
  • Previous
  • ...
  • 837
  • 838
  • 839
  • 840
  • 841
  • 842
  • 843
  • 844
  • 845
  • 846
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×