Understand the limitations of the Internet of Things and how the Web of Things can help build an application layer for the IoT.

We spoke with 20 executives across the IoT space about problems the Internet of Things are addressing.

An exploration of the difference between the concepts of microservices and microsegmentation.

Learn how to configure rtl_fm and Direwolf for Decoding Amateur Radio Packet on the Raspberry Pi

If there have been wide array of successful consumer apps like Angry Bird or WhatsApp or DropBox. After years of reign in the publicity focus finally these consumer apps giants understood the importance of offering enterprise grade features. In last few years suddenly the focus shifted to enterprise mobile apps. Rapid development, tracking or monitoring apps, wearable apps, Internet of Things Apps, Geo-location technologies like iBeacon and Geofencing in business apps, the list of emerging app niches and technologies seem to be too long. Let us have a quick look at some of the most definitive app niches and technologies in recent times. Enterprise apps While smartphones and mobile devices continue to move off the shelves and millions of apps continue to make the app stores brimming with energy, activity and competition, most consumer app still fail to make a earning to survive beyond the year of their launch. This has been the sordid storyline for consumer apps for years. So, for some time the focus of developers is shifting towards enterprise apps. Moreover, now businesses are bent on going mobile and they are keen to develop apps that make their business process more productive. Although enterprise mobile apps have just started to take off this new and broad app niche already shown huge promise to take over consumer apps in just more than a year down the line. Rapid development As enterprises now focusing all out to embrace mobile apps in their business process, the new demand of enterprise grade apps made rapid development cycle obvious. When winning competition for businesses is boiling down to a fast and user focused mobile presence, fast paced development will naturally be the rule. This overwhelming demand of business apps and enterprise grade software made rapid development a criterion in the present scenario. Shortening the development lifecycle has now become the major focus for most mobile app development companies around the world. Mobile monitoring apps Wide adaptation of mobile devices and apps among all age groups and people in recent years gave rise to certain concerns. Child security concern, parental concern for negative influence on children, employer’s concern on employee productivity and information security, etc. are some of the major concerns centered on the mobile devices. IOS or Android monitoring software, child phone tracker apps, mobile spy software, text message tracking apps, are few of the app types getting increasingly popular these days to address the aforementioned concerns in family or workplace environments. Internet of Things (IOT) apps The world around us is becoming connected with the mobile devices and gadgets and devices around us are increasingly finding themselves equipped with mobile control interface. This new horizon of interconnected devices is referred as Internet of Things or IOT. Now an electric toaster can be controlled from its respective app on the mobile device. Similarly, the music system with the respective mobile app can be turned on and off, tuned in and given other commands. This new breed of apps is being called IOT apps. Wearable apps The smartphones or smart mobile devices are now playing the central role in connecting all types of wearable smart devices. Most smartwatch apps are still now in character only the extension of their mobile counterparts. But as smartwatch is slowly picking up to be the next big device platform as commonest wearable, a new breed of apps are being developed targeting smartwatch and wearable users besides offering their respective mobile apps as well. From smart jewelries to health trackers and fitness bands to optically mounted computers like Google Glass, these new wearable devices will be the target development platform for a vast majority of mobile app developers in the time to come. More user-optimized mobile UI design UI design is presently the most focus driven area for mobile app development around the world. Experiments and analysis on making UIs better and user optimized is continuing and a wide variety of new techniques and design approaches are giving birth to unprecedented level of excellence in user experience. From motivational design to flat design to and playful interfaces, we have come across quite a few dominating design trends and techniques. Geo-location technologies Contextual and user specific push notification is the new maneuver to engage users with a mobile app and to garner revenue from the process. This cannot be better done than by knowing the user location. When you know the location of a user close to your retail shop you can notify him with an offer to grab his attention and push him for a visit to your store. Thus knowing the user location translates to far better contextual and business driven messaging and notifications. Several mobile friendly Geo-location technologies like iBeacon, Geofencing, Geomagnetics, etc. are there to let you integrate location based user engagement features in your app.

Double the throughput and lower latency than the leading global cloud providers between the US and Europe in independent comparison research London & New York, 1 July, 2015. Interoute has today announced that its global cloud platform Interoute Virtual Data Centre (VDC), has been proven to deliver nearly double the throughput across the Atlantic than the next best cloud provider in comparison research conducted by Cloud Spectator. The research from March 2015 compared Interoute VDC with three leading cloud providers (Amazon AWS, Rackspace and Microsoft Azure), testing network throughput and latency between Europe and USA and between providers' European data centres. In all of the comparisons, Interoute VDC demonstrated the highest throughputs and lowest latencies. Cloud Spectator's full research report, and more information about Interoute VDC's performance and features, can be viewed here: http://bit.ly/1GHyzwJ Network performance is a significant factor in cloud computing for business services requiring the highest network capacity (throughput) and the shortest possible time from the server to the client (latency), to meet the needs of the businesses and their users. Innovating new applications and business services in the cloud needs network performance to match and this report shows the advantages of building the cloud into a huge global high performance network. Key research findings: Transatlantic: Interoute VDC delivered 1.1 Gbit/s throughput, which was 96% better than Amazon AWS, 141% better than Rackspace, and 195% better than Microsoft Azure. Interoute VDC had the lowest latency, between its London and New York data centres. Interoute was the only provider in the comparison with both of its transatlantic data centres located in key business cities, meaning that VDC users can access compute and storage resources, and deliver data to their customers, from two centres of European and US business activity. Within Europe: Interoute VDC achieved 1.3 Gbit/s throughput between its London and Amsterdam data centres. This was 52% better than Amazon AWS (Dublin - Frankfurt) and 73% better than Microsoft Azure (Dublin - Amsterdam) Interoute VDC achieved a latency of 6 milliseconds between London and Amsterdam, over three times better than the inter-data centre latency of the comparison providers. Matthew Finnie, CTO of Interoute, commented: "This independent report confirms and validates our networked cloud strategy. Building cloud into a world class network provides our customers with significantly better performance when compared with the traditional cloud models. Businesses looking to grow between Europe and US should definitely be looking at the importance of these network characteristics for their ability to shift workloads into the cloud. Interoute's fourteen global zones are all built into high performance network with over 300 interconnects in Europe alone. So wherever you choose to put your data and connect to us, your services are typically going to perform faster on Interoute than on many other global providers." Danny Gee, Senior Analyst, Cloud Spectator: "Users want to transfer large amounts of data between data centres quickly. Our study revealed that for a trans-Atlantic connection between cloud data centers, Interoute provided the highest throughput and lowest latency out of AWS, Rackspace and Azure. Interoute also had the higher network throughput and lowest latency in European testing compared to Azure and AWS (Rackspace was excluded, having only one location in Europe), making it a good option for users operating servers within this region. Interoute also provided the best latency, ideal for real-time communications. Users running geographically dispersed environments for such things as geo-redundancy would benefit from Interoute's high performance cloud connectivity."

Value-Driven SON® provides the right mobile network experience for the right customers at the right time Cellwize, the innovative Self-Organizing Networks provider, has revealed how its Value-Driven SON® solution can provide mobile operators with the means to ensure better customer experience for selected audiences, thereby driving new business for operators. Cellwize’ Value-Driven SON shifts the value generated by the network to the end customer – moving from a Network-Centric SON to a Customer-Centric activity. The latest whitepaper from Cellwize “Value-Driven SON – Putting the Customer at the Network’s Center” is a must read for forward thinking operators who are introducing customer-focused metrics. As operators globally look to maximize user satisfaction, several are using metrics to measure quality of experience (QoE), net promoter scores (NPS) and data service revenues. In fact some are starting to use the term Key Quality Indicator (KQI) to describe these metrics and distinguish them from the more operational KPIs. And, unlike KPIs, KQI’s are generally at a customer level – either for selected customers or a segment of customers. “We believe that SON should be driven by the business objectives of the operator and not by the immediate need to reduce complexity and costs in the network itself,” said Ofir Zemer, CEO of Cellwize. “And since operator’s business objectives aim to deliver superior value to various groups of end-users; Value-Driven SON allows mobile operators to turn this value to enhanced revenue. Value-Driven SON connects customer insight generated in and around the network with network optimization technology, and is driven by the operators’ business goals. Ultimately, the network is there to serve a purpose and the shift to the user will enable a quicker path for operators to deliver better performance and better user experience.” The Whitepaper illustrates the benefits of a customer-centric approach with Value-Driven SON through two types of use cases: commuters and enterprise customers. The first depicts Value-Driven SON for Space & Time use cases - prioritizing network resources to meet the demand of commuters along a specific route experiencing poor service quality, such as dropped calls, session termination, latency and data throughput. The second, explains how Value-Driven SON can prioritize and maximize network performance for a defined group of customers e.g. enterprise customers. Download the whitepaper to find out more about “Value-Driven SON®: Putting the Customer at the Network’s Center”. Additional information Cellwize elastic-SON® and Value Driven-SON®

Create a motion-activated Pushbullet Notification to your smartphone.

Alpnames becomes the second largest registrar for new domain endings Wednesday 24th June 2015, Gibraltar: Alpnames smashed through the half million mark last night in terms of domain under management. As an ICANN accredited registrar specialising in new generic Top Level Domains, Alpnames manages nearly 8% of all new gTLD names registered to date making it the second most popular new gTLD registrar after the industry giant GoDaddy and recently overtaking the first and longest standing registrar Network Solutions (part of web.com). “We are delighted with the extraordinary growth of Alpnames” said Damon Barnard, Chief Operations Officer at Alpnames. “Our decision to focus on bringing competitively priced new domain extensions to the attention of Internet users, who might be frustrated at the lack of affordable domain names under .COM, is clearly the right one” There is currently a massive change happening in the Internet Name space meaning that domain names can now end in almost any series of letters such as .science, .party or .club. ICANN, the body that regulates names on the internet, relaxed the rules on internet domain extensions in 2013 in order to promote competition, innovation and consumer choice. Currently, over 5.75 million domain names have been registered in nearly 640 new domain extensions. These so called new generic Top Level Domains (gTLDs) will form the backbone of the next generation of the Internet, opening up the web to a whole host of new possibilities. “Alpnames is proud to be a leading player in the new era of domain names” continued Barnard. “We intend to continue bringing new domain names to Internet users at affordable prices; names that help define communities of interest online and assist Internet users in navigating to websites that provide the content or interaction that they are looking for.” With more than 500,000 web addresses under management, Alpnames is one of the fastest growing domain name registrars in history.

Business leaders today are becoming more aware of the many ways social knowledge networks can bring value to an organization. Communication is enhanced, knowledge is organized and accessible, products and customer service are improved, and business culture is strengthened through team collaboration. However, what many business leaders are failing to remember is that finding the right technology is only the first step in implementing a successful social knowledge network. The technology must be paired with smart planning and a well-thought-out structure. Elizabeth Lupfer, social collaboration and HR expert, claims, “A social knowledge network is one part technology and two parts process. The technology is only an enabler, and may only be worth 20 percent of the total value of the intranet.” The other 80 percent of value is created through effective structure and long-lasting employee engagement. We have researched and created a 10-step social knowledge network success plan that if implemented properly can maximize the value of any social knowledge network. Have a plan. Taking the time to create a social knowledge network implementation plan for your organization ensures managers and employees are motivated and committed to maximizing the value of the network. The network will only become as valuable as managers and employees plan to make it. Setting expectations for productivity and engagement ahead of time is important. Give employees a detailed explanation of how the social knowledge network connects to the organizational purpose. Communicate the value and purpose of the social knowledge network. Employees must clearly understand how the social knowledge network will contribute to the organizational purpose, long term and short term goals of the organization, and their individual role. Set up the organization and structure of the intranet at the beginning. Understand the priorities of each department, region, and team within your organization and how they connect. Who communicates with whom the most? What knowledge is of most value to each team? Your social knowledge network structure should enhance easy collaboration between individuals who value the information being shared. Thinking about the structure of your social knowledge network in advance will allow for easier search capabilities in the network and less time wasted sifting through information that is not relevant. Define roles and expectations around engagement for employees. Collaboration can be encouraged by asking employees to share projects and knowledge on the social knowledge network so that all team members are on the same page and no question is asked twice. Create and announce objectives with time-specific deadlines. It is important to create quantitative and time-specific objectives for your social knowledge network, such as “60 percent of users will have contributed some knowledge to our social knowledge network by June 30th.” Creating measurable engagement expectations will encourage employees to actively work towards the objective and increase network participation toward a common goal. Assign key social knowledge network ambassadors to start and maintain collaboration. Have people in your organization serve as the “ambassadors” of the social knowledge network. The ambassadors will be the network’s support, taking care of employee questions and concerns about the social knowledge network and asking the correct individuals for answers when they don’t know the answer themselves. Get the CEO and top level management involved. CEO and top-level involvement in a social knowledge network is very valuable, as employees pay special attention to what leaders have to share. Top level management should post openly about company activities to keep employees inspired about the broader picture of the organization. Management contributions should push brand, company culture, and core values. Industry and competitor news, trends, and expectations should be shared by management as well. Determine and reward achievement of Key Performance Indicators. Financial and non-financial benefits are important ways to encourage employee engagement in your social knowledge network. Gift cards, time off, or simple acknowledgment given to active users are good ways to inspire employees to participate more. Rewards given to the most active team or department will encourage group involvement and can be a great way to spark participation. Upload company documents and shared files. Ensure that your social knowledge network is the main place to access information, forms, presentations, and documents of all kinds. Keep all documents updated and organized so that they are accurate and searchable within the platform. Onboarding information, required customer forms, sales presentations, marketing tools, company calendars, and price lists should all be easily accessible on your social knowledge network. This will immediately spark network participation because these documents are needed by many people in your organization who will now access them through your social knowledge network. Be social! Although your social knowledge network should be a valuable business tool, there are many benefits of adding social, non-business related activity. Social activity on your network will create increased openness, allowing employees to get to know each other better and become more comfortable with sharing thoughts and opinions. Adding personal employee profile contributions, weekly peer-to-peer shout-outs for special accomplishments, employee group activities outside of work, or even posting something humorous once a week can help your social knowledge network become more open and fun, drawing employees back to check and use the network more often. Finding the right social knowledge network software for your organization is only the start. Research from the 2014 Social Business Global Executive Study shows only 17 percent of respondents see their organization as having mature social business practices. Following these steps will help you successfully integrate your social knowledge network into the flow of work. Once your organization becomes a true social business, you’ll see measurable value in improved innovation, talent management, and operations. If you are interested in learning more about how a knowledge base can break down silos within your organization, check out our white paper, Proving the Value: Getting Internal Buy-In for a Knowledge Base. Like this post? Click here to subscribe to our blog and receive the latest content on social learning, customer support, sales enablement, or all three.

IoT is really hot, but can also be a bit confusing. Read about these 8 development key findings.

COLOGNE, Germany – June 22, 2015 – ParStream, the IoT analytics company, today announced its participation at the TDWI Munich Conference 2015, one of the largest gatherings of expert Business Intelligence, Big Data and data warehousing leaders and educators in Europe. The conference will take place June 22-24, 2015 at the MOC Order and Event Center in Munich, Germany. Albert Aschauer, Sales Director DACH at ParStream, will present on requirements for an analytics platform for the Internet of Things (IoT) based on real-world use cases from the renewable energy and telecommunications industries. Big Data, fast data, edge analytics and real-time insights are driving new technology innovation to meet the demand for getting more value from IoT data. Additional details on the speaking session are below. What: “Requirements of an Analytics Platform for the Internet of Things” When: Monday, June 22, 2015 at 11:35 a.m. CEST Who: Albert Aschauer, Sales Director DACH at ParStream Where: MOC Munich, Germany – Room F112 To schedule a one-on-one meeting with Albert Aschauer and ParStream at TDWI Munich Conference 2015, send an email to events(at)parstream(dot)com.

The common scenario for the mobile IoT Gateways is to cache collected data locally on the device storage and synchronizing the data with the data center.

At Red Hat's Virtual Event, Building Data-driven Solutions for the Internet of Things, Kenneth Peeples spoke about connecting to the IoT with the MQTT protocol.

written by josh long on the spring blog applications generated more and more data than ever before and a huge part of the challenge - before it can even be analyzed - is accommodating the load in the first place. apache’s kafka meets this challenge. it was originally designed by linkedin and subsequently open-sourced in 2011. the project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. the design is heavily influenced by transaction logs. it is a messaging system, similar to traditional messaging systems like rabbitmq, activemq, mqseries, but it’s ideal for log aggregation, persistent messaging, fast (_hundreds_ of megabytes per second!) reads and writes, and can accommodate numerous clients. naturally, this makes it perfect for cloud-scale architectures! kafka powers many large production systems . linkedin uses it for activity data and operational metrics to power the linkedin news feed, and linkedin today, as well as offline analytics going into hadoop. twitter uses it as part of their stream-processing infrastructure. kafka powers online-to-online and online-to-offline messaging at foursquare. it is used to integrate foursquare monitoring and production systems with hadoop-based offline infrastructures. square uses kafka as a bus to move all system events through square’s various data centers. this includes metrics, logs, custom events, and so on. on the consumer side, it outputs into splunk, graphite, or esper-like real-time alerting. netflix uses it for 300-600bn messages per day. it’s also used by airbnb, mozilla, goldman sachs, tumblr, yahoo, paypal, coursera, urban airship, hotels.com, and a seemingly endless list of other big-web stars. clearly, it’s earning its keep in some powerful systems! installing apache kafka there are many different ways to get apache kafka installed. if you’re on osx, and you’re using homebrew, it can be as simple as brew install kafka . you can also download the latest distribution from apache . i downloaded kafka_2.10-0.8.2.1.tgz , unzipped it, and then within you’ll find there’s a distribution of apache zookeeper as well as kafka, so nothing else is required. i installed apache kafka in my $home directory, under another directory, bin , then i created an environment variable, kafka_home , that points to $home/bin/kafka . start apache zookeeper first, specifying where the configuration properties file it requires is: $kafka_home/bin/zookeeper-server-start.sh $kafka_home/config/zookeeper.properties the apache kafka distribution comes with default configuration files for both zookeeper and kafka, which makes getting started easy. you will in more advanced use cases need to customize these files. then start apache kafka. it too requires a configuration file, like this: $kafka_home/bin/kafka-server-start.sh $kafka_home/config/server.properties the server.properties file contains, among other things, default values for where to connect to apache zookeeper ( zookeeper.connect ), how much data should be sent across sockets, how many partitions there are by default, and the broker id ( broker.id - which must be unique across a cluster). there are other scripts in the same directory that can be used to send and receive dummy data, very handy in establishing that everything’s up and running! now that apache kafka is up and running, let’s look at working with apache kafka from our application. some high level concepts.. a kafka broker cluster consists of one or more servers where each may have one or more broker processes running. apache kafka is designed to be highly available; there are no master nodes. all nodes are interchangeable. data is replicated from one node to another to ensure that it is still available in the event of a failure. in kafka, a topic is a category, similar to a jms destination or both an amqp exchange and queue. topics are partitioned, and the choice of which of a topic’s partition a message should be sent to is made by the message producer. each message in the partition is assigned a unique sequenced id, its offset . more partitions allow greater parallelism for consumption, but this will also result in more files across the brokers. producers send messages to apache kafka broker topics and specify the partition to use for every message they produce. message production may be synchronous or asynchronous. producers also specify what sort of replication guarantees they want. consumers listen for messages on topics and process the feed of published messages. as you’d expect if you’ve used other messaging systems, this is usually (and usefully!) asynchronous. like spring xd and numerous other distributed system, apache kafka uses apache zookeeper to coordinate cluster information. apache zookeeper provides a shared hierarchical namespace (called znodes ) that nodes can share to understand cluster topology and availability (yet another reason that spring cloud has forthcoming support for it..). zookeeper is very present in your interactions with apache kafka. apache kafka has, for example, two different apis for acting as a consumer. the higher level api is simpler to get started with and it handles all the nuances of handling partitioning and so on. it will need a reference to a zookeeper instance to keep the coordination state. let’s turn now turn to using apache kafka with spring. using apache kafka with spring integration the recently released apache kafka 1.1 spring integration adapter is very powerful, and provides inbound adapters for working with both the lower level apache kafka api as well as the higher level api. the adapter, currently, is xml-configuration first, though work is already underway on a spring integration java configuration dsl for the adapter and milestones are available. we’ll look at both here, now. to make all these examples work, i added the libs-milestone-local maven repository and used the following dependencies: org.apache.kafka:kafka_2.10:0.8.1.1 org.springframework.boot:spring-boot-starter-integration:1.2.3.release org.springframework.boot:spring-boot-starter:1.2.3.release org.springframework.integration:spring-integration-kafka:1.1.1.release org.springframework.integration:spring-integration-java-dsl:1.1.0.m1 using the spring integration apache kafka with the spring integration xml dsl first, let’s look at how to use the spring integration outbound adapter to send message instances from a spring integration flow to an external apache kafka instance. the example is fairly straightforward: a spring integration channel named inputtokafka acts as a conduit that forwards message messages to the outbound adapter, kafkaoutboundchanneladapter . the adapter itself can take its configuration from the defaults specified in the kafka:producer-context element or it from the adapter-local configuration overrides. there may be one or many configurations in a given kafka:producer-context element. here’s the java code from a spring boot application to trigger message sends using the outbound adapter by sending messages into the incoming inputtokafka messagechannel . package xml; import org.apache.commons.logging.log; import org.apache.commons.logging.logfactory; import org.springframework.beans.factory.annotation.qualifier; import org.springframework.boot.commandlinerunner; import org.springframework.boot.springapplication; import org.springframework.boot.autoconfigure.springbootapplication; import org.springframework.context.annotation.bean; import org.springframework.context.annotation.dependson; import org.springframework.context.annotation.importresource; import org.springframework.integration.config.enableintegration; import org.springframework.messaging.messagechannel; import org.springframework.messaging.support.genericmessage; @springbootapplication @enableintegration @importresource("/xml/outbound-kafka-integration.xml") public class demoapplication { private log log = logfactory.getlog(getclass()); @bean @dependson("kafkaoutboundchanneladapter") commandlinerunner kickoff(@qualifier("inputtokafka") messagechannel in) { return args -> { for (int i = 0; i < 1000; i++) { in.send(new genericmessage<>("#" + i)); log.info("sending message #" + i); } }; } public static void main(string args[]) { springapplication.run(demoapplication.class, args); } } using the new apache kafka spring integration java configuration dsl shortly after the spring integration 1.1 release, spring integration rockstar artem bilan got to work on adding a spring integration java configuration dsl analog and the result is a thing of beauty! it’s not yet ga (you need to add the libs-milestone repository for now), but i encourage you to try it out and kick the tires. it’s working well for me and the spring integration team are always keen on getting early feedback whenever possible! here’s an example that demonstrates both sending messages and consuming them from two different integrationflow s. the producer is similar to the example xml above. new in this example is the polling consumer. it is batch-centric, and will pull down all the messages it sees at a fixed interval. in our code, the message received will be a map that contains as its keys the topic and as its value another map with the partition id and the batch (in this case, of 10 records), of records read. there is a messagelistenercontainer -based alternative that processes messages as they come. package jc; import org.apache.commons.logging.log; import org.apache.commons.logging.logfactory; import org.springframework.beans.factory.annotation.autowired; import org.springframework.beans.factory.annotation.qualifier; import org.springframework.beans.factory.annotation.value; import org.springframework.boot.commandlinerunner; import org.springframework.boot.springapplication; import org.springframework.boot.autoconfigure.springbootapplication; import org.springframework.context.annotation.bean; import org.springframework.context.annotation.configuration; import org.springframework.context.annotation.dependson; import org.springframework.integration.integrationmessageheaderaccessor; import org.springframework.integration.config.enableintegration; import org.springframework.integration.dsl.integrationflow; import org.springframework.integration.dsl.integrationflows; import org.springframework.integration.dsl.sourcepollingchanneladapterspec; import org.springframework.integration.dsl.kafka.kafka; import org.springframework.integration.dsl.kafka.kafkahighlevelconsumermessagesourcespec; import org.springframework.integration.dsl.kafka.kafkaproducermessagehandlerspec; import org.springframework.integration.dsl.support.consumer; import org.springframework.integration.kafka.support.zookeeperconnect; import org.springframework.messaging.messagechannel; import org.springframework.messaging.support.genericmessage; import org.springframework.stereotype.component; import java.util.list; import java.util.map; /** * demonstrates using the spring integration apache kafka java configuration dsl. * thanks to spring integration ninja artem bilan * for getting the java configuration dsl working so quickly! * * @author josh long */ @enableintegration @springbootapplication public class demoapplication { public static final string test_topic_id = "event-stream"; @component public static class kafkaconfig { @value("${kafka.topic:" + test_topic_id + "}") private string topic; @value("${kafka.address:localhost:9092}") private string brokeraddress; @value("${zookeeper.address:localhost:2181}") private string zookeeperaddress; kafkaconfig() { } public kafkaconfig(string t, string b, string zk) { this.topic = t; this.brokeraddress = b; this.zookeeperaddress = zk; } public string gettopic() { return topic; } public string getbrokeraddress() { return brokeraddress; } public string getzookeeperaddress() { return zookeeperaddress; } } @configuration public static class producerconfiguration { @autowired private kafkaconfig kafkaconfig; private static final string outbound_id = "outbound"; private log log = logfactory.getlog(getclass()); @bean @dependson(outbound_id) commandlinerunner kickoff( @qualifier(outbound_id + ".input") messagechannel in) { return args -> { for (int i = 0; i < 1000; i++) { in.send(new genericmessage<>("#" + i)); log.info("sending message #" + i); } }; } @bean(name = outbound_id) integrationflow producer() { log.info("starting producer flow.."); return flowdefinition -> { consumer spec = (kafkaproducermessagehandlerspec.producermetadataspec metadata)-> metadata.async(true) .batchnummessages(10) .valueclasstype(string.class) .valueencoder(string::getbytes); kafkaproducermessagehandlerspec messagehandlerspec = kafka.outboundchanneladapter( props -> props.put("queue.buffering.max.ms", "15000")) .messagekey(m -> m.getheaders().get(integrationmessageheaderaccessor.sequence_number)) .addproducer(this.kafkaconfig.gettopic(), this.kafkaconfig.getbrokeraddress(), spec); flowdefinition .handle(messagehandlerspec); }; } } @configuration public static class consumerconfiguration { @autowired private kafkaconfig kafkaconfig; private log log = logfactory.getlog(getclass()); @bean integrationflow consumer() { log.info("starting consumer.."); kafkahighlevelconsumermessagesourcespec messagesourcespec = kafka.inboundchanneladapter( new zookeeperconnect(this.kafkaconfig.getzookeeperaddress())) .consumerproperties(props -> props.put("auto.offset.reset", "smallest") .put("auto.commit.interval.ms", "100")) .addconsumer("mygroup", metadata -> metadata.consumertimeout(100) .topicstreammap(m -> m.put(this.kafkaconfig.gettopic(), 1)) .maxmessages(10) .valuedecoder(string::new)); consumer endpointconfigurer = e -> e.poller(p -> p.fixeddelay(100)); return integrationflows .from(messagesourcespec, endpointconfigurer) .>>handle((payload, headers) -> { payload.entryset().foreach(e -> log.info(e.getkey() + '=' + e.getvalue())); return null; }) .get(); } } public static void main(string[] args) { springapplication.run(demoapplication.class, args); } } the example makes heavy use of java 8 lambdas. the producer spends a bit of time establishing how many messages will be sent in a single send operation, how keys and values are encoded (kafka only knows about byte[] arrays, after all) and whether messages should be sent synchronously or asynchronously. in the next line, we configure the outbound adapter itself and then define an integrationflow such that all messages get sent out via the kafka outbound adapter. the consumer spends a bit of time establishing which zookeeper instance to connect to, how many messages to receive (10) in a batch, etc. once the message batches are recieved, they’re handed to the handle method where i’ve passed in a lambda that’ll enumerate the payload’s body and print it out. nothing fancy. using apache kafka with spring xd apache kafka is a message bus and it can be very powerful when used as an integration bus. however, it really comes into its own because it’s fast enough and scalable enough that it can be used to route big-data through processing pipelines. and if you’re doing data processing, you really want spring xd ! spring xd makes it dead simple to use apache kafka (as the support is built on the apache kafka spring integration adapter!) in complex stream-processing pipelines. apache kafka is exposed as a spring xd source - where data comes from - and a sink - where data goes to. spring xd exposes a super convenient dsl for creating bash -like pipes-and-filter flows. spring xd is a centralized runtime that manages, scales, and monitors data processing jobs. it builds on top of spring integration, spring batch, spring data and spring for hadoop to be a one-stop data-processing shop. spring xd jobs read data from sources , run them through processing components that may count, filter, enrich or transform the data, and then write them to sinks. spring integration and spring xd ninja marius bogoevici , who did a lot of the recent work in the spring integration and spring xd implementation of apache kafka, put together a really nice example demonstrating how to get a full working spring xd and kafka flow working . the readme walks you through getting apache kafka, spring xd and the requisite topics all setup. the essence, however, is when you use the spring xd shell and the shell dsl to compose a stream. spring xd components are named components that are pre-configured but have lots of parameters that you can override with --.. arguments via the xd shell and dsl. (that dsl, by the way, is written by the amazing andy clement of spring expression language fame!) here’s an example that configures a stream to read data from an apache kafka source and then write the message a component called log , which is a sink. log , in this case, could be syslogd, splunk, hdfs, etc. xd> stream create kafka-source-test --definition "kafka --zkconnect=localhost:2181 --topic=event-stream | log"--deploy and that’s it! naturally, this is just a tase of spring xd, but hopefully you’ll agree the possibilities are tantalizing. deploying a kafka server with lattice and docker it’s easy to get an example kafka installation all setup using lattice , a distributed runtime that supports, among other container formats, the very popular docker image format. there’s a docker image provided by spotify that sets up a collocated zookeeper and kafka image . you can easily deploy this to a lattice cluster, as follows: ltc create --run-as-root m-kafka spotify/kafka from there, you can easily scale the apache kafka instances and even more easily still consume apache kafka from your cloud-based services. next steps you can find the code for this blog on my github account . we’ve only scratched the surface! if you want to learn more (and why wouldn’t you?), then be sure to check out marius bogoevici and dr. mark pollack’s upcoming webinar on reactive data-pipelines using spring xd and apache kafka where they’ll demonstrate how easy it can be to use rxjava, spring xd and apache kafka!

In the traditional 3-tier architecture, data processing is performed by the application server where the data itself is stored in the database server. Application server and database server are typically two different machine. Therefore, the processing cycle proceeds as follows Application server send a query to the database server to retrieve the necessary data Application server perform processing on the received data Application server will save the changed data to the database server In the traditional data processing paradigm, we move data to the code. It can be depicted as follows ... Then big data phenomenon arrives. Because the data volume is huge, it cannot be hold by a single database server. Big data is typically partitioned and stored across many physical DB server machines. On the other hand, application servers need to be added to increase the processing power of big data. However, as we increase the number of App servers and DB servers for storing and processing the big data, more data need to be transfer back and forth across the network during the processing cycle, up to a point where the network becomes a major bottleneck. Moving Code to Data To overcome the network bottleneck, we need a new computing paradigm. Instead of moving data to the code, we move the code to the data and perform the processing at where the data is stored. Notice the change of the program structure The program execution starts at a driver, which orchestrate the execution happening remotely across many worker servers within a cluster. Data is no longer transferred to the driver program, the driver program holds a data reference in its variable rather than the data itself. The data reference is basically an id to locate the corresponding data residing in the database server Code is shipped from the program to the database server, where the execution is happening, and data is modified at the database server without leaving the server machine. Finally the program request a save of the modified data. Since the modified data resides in the database server, no data transfer happens over the network. By moving the code to the data, the volume of data transfer over network is significantly reduced. This is an important paradigm shift for big data processing. In the following session, I will use Apache Spark to illustrate how this big data processing paradigm is implemented. RDD Resilient Distributed Dataset (RDD) is how Spark implements the data reference concept. RDD is a logical reference of a dataset which is partitioned across many server machines in the cluster. To make a clear distinction between data reference and data itself, a Spark program is organized as a sequence of execution steps, which can either be a "transformation" or an "action". Programming Model A typical program is organized as follows From an environment variable "context", create some initial data reference RDD objects Transform initial RDD objects to create more RDD objects. Transformation is expressed in terms of functional programming where a code block is shipped from the driver program to multiple remote worker server, which hold a partition of the RDD. Variable appears inside the code block can either be an item of the RDD or a local variable inside the driver program which get serialized over to the worker machine. After the code (and the copy of the serialized variables) is received by the remote worker server, it will be executed there by feeding the items of RDD residing in that partition. Notice that the result of a transformation is a brand new RDD (the original RDD is not mutated) Finally, the RDD object (the data reference) will need to be materialized. This is achieved through an "action", which will dump the RDD into a storage, or return its value data to the driver program. Here is a word count example # Get initial RDD from the context file = spark.textFile("hdfs://...") # Three consecutive transformation of the RDD counts = file.flatMap(lambda line: line.split(" ")) .map(lambda word: (word, 1)) .reduceByKey(lambda a, b: a + b) # Materialize the RDD using an action counts.saveAsTextFile("hdfs://...") When the driver program starts its execution, it builds up a graph where nodes are RDD and edges are transformation steps. However, no execution is happening at the cluster until an action is encountered. At that point, the driver program will ship the execution graph as well as the code block to the cluster, where every worker server will get a copy. The execution graph is a DAG. Each DAG is a atomic unit of execution. Each source node (no incoming edge) is an external data source or driver memory Each intermediate node is a RDD Each sink node (no outgoing edge) is an external data source or driver memory Green edge connecting to RDD represents a transformation. Red edge connecting to a sink node represents an action Data Shuffling Although we ship the code to worker server where the data processing happens, data movement cannot be completely eliminated. For example, if the processing requires data residing in different partitions to be grouped first, then we need to shuffle data among worker server. Spark carefully distinguish "transformation" operation in two types. "Narrow transformation" refers to the processing where the processing logic depends only on data that is already residing in the partition and data shuffling is unnecessary. Examples of narrow transformation includes filter(), sample(), map(), flatMap() .... etc. "Wide transformation" refers to the processing where the processing logic depends on data residing in multiple partitions and therefore data shuffling is needed to bring them together in one place. Example of wide transformation includes groupByKey(), reduceByKey() ... etc. Joining two RDD can also affect the amount of data being shuffled. Spark provides two ways to join data. In a shuffle join implementation, data of two RDD with the same key will be redistributed to the same partition. In other words, each of the items in each RDD will be shuffled across worker servers. Beside shuffle join, Spark provides another alternative call broadcast join. In this case, one of the RDD will be broadcasted and copied over to every partition. Imagine the situation when one of the RDD is significantly smaller relative to the other, then broadcast join will reduce the network traffic because only the small RDD need to be copied to all worker servers while the large RDD doesn't need to be shuffled at all. In some cases, transformation can be re-ordered to reduce the amount of data shuffling. Below is an example of a JOIN between two huge RDDs followed by a filtering. Plan1 is a naive implementation which follows the given order. It first join the two huge RDD and then apply the filter on the join result. This ends up causing a big data shuffling because the two RDD is huge, even though the result after filtering is small. Plan2 offers a smarter way by using the "push-down-predicate" technique where we first apply the filtering in both RDDs before joining them. Since the filtering will reduce the number of items of each RDD significantly, the join processing will be much cheaper. Execution Planning As explain above, data shuffling incur the most significant cost in the overall data processing flow. Spark provides a mechanism that generate an execute plan from the DAG that minimize the amount of data shuffling. Analyze the DAG to determine the order of transformation. Notice that we starts from the action (terminal node) and trace back to all dependent RDDs. To minimize data shuffling, we group the narrow transformation together in a "stage" where all transformation tasks can be performed within the partition and no data shuffling is needed. The transformations becomes tasks that are chained together within a stage Wide transformation sits at the boundary of two stages, which requires data to be shuffled to a different worker server. When a stage finishes its execution, it persist the data into different files (one per partition) of the local disks. Worker nodes of the subsequent stage will come to pickup these files and this is where data shuffling happens Below is an example how the execution planning turns the DAG into an execution plan involving stages and tasks. Reliability and Fault Resiliency Since the DAG defines a deterministic transformation steps between different partitions of data within each RDD RDD, fault recovery is very straightforward. Whenever a worker server crashes during the execution of a stage, another worker server can simply re-execute the stage from the beginning by pulling the input data from its parent stage that has the output data stored in local files. In case the result of the parent stage is not accessible (e.g. the worker server lost the file), the parent stage need to be re-executed as well. Imagine this is a lineage of transformation steps, and any failure of a step will trigger a restart of execution from its last step. Since the DAG itself is an atomic unit of execution, all the RDD values will be forgotten after the DAG finishes its execution. Therefore, after the driver program finishes an action (which execute a DAG to its completion), all the RDD value will be forgotten and if the program access the RDD again in subsequent statement, the RDD needs to be recomputed again from its dependents. To reduce this repetitive processing, Spark provide a caching mechanism to remember RDDs in worker server memory (or local disk). Once the execution planner finds the RDD is already cache in memory, it will use the RDD right away without tracing back to its parent RDDs. This way, we prune the DAG once we reach an RDD that is in the cache. Overall speaking, Apache Spark provides a powerful framework for big data processing. By the caching mechanism that holds previous computation result in memory, Spark out-performs Hadoop significantly because it doesn't need to persist all the data into disk for each round of parallel processing. Although it is still very new, I think Spark will take off as the main stream approach to process big data.

Many organizations in which I've coached are concerned when the people on their teams aren't being "utilized" at 100% of their capacity. If someone is at work for 8 hours per day, minus an hour for lunch and breaks, that's 7 hours of potential capacity. Some organizations are progressive enough to see that the organization's overhead of administrative activities lowers that value to 6 hours per day. By extension, a team's capacity is simply a multiple of the number of team members and the number of hours available to be utilized, i.e. a team of 5 has 30 person-hours per day. So, anything less than 30 person-hours per day spent on tasks means that a team isn't being utilized 100%. Which is bad, apparently. A Thought Exercise If you're reading this, you have a computer and it's connected to a network. Your computer might be a small one that makes phone calls or a large one that can also handle the complex scenery of games at 60 frames per second without breaking a sweat. Regardless, it's a computer. Thanks to John von Neumann, almost all computers today have the same basic architecture, the heart of which is the Central Processing Unit or CPU. When the CPU on your computer, regardless of its size, reaches 100% utilization, what happens? Your computer slows down. Significantly. The CPU is saturated with instructions and does its best to process them, but it has a finite limit on how quickly it can do that. There's a finite limit to the speed at which the instructions and the data they use can be passed around among the internal components of the CPU. That processor just can't go any faster. To a person using the computer, it appears to be locked up. To see this in action, try running a modern game like Faster Than Light on a 1st generation iPad. The older processor can't keep up with the computing demand from a game targeted towards more a powerful CPU. The communications network to which your computer is connected might be wireless or have a "hard" connection, but behind the scenes the public internet is dominated by TCP/IP and Ethernet. What happens when the network to which your computer is connected is running at 100% capacity? The network slows down. Significantly. The equipment moving the packets of data about will experience numerous collisions and will have to send requests back to your computer to resend data. The equipment will also begin to simply "drop" packets because it can't process them quickly enough. As far as you're concerned the network has become either very slow or has simply stopped. To see this in real life, look at what happens to a web site when it's the subject of a Denial of Service (DoS) attack, which effectively saturates that site's network with data requests. The site either crashes completely or appears to be unresponsive because it can't handle the traffic. Knowledge work such as software development requires a huge amount of thinking (processing) and a huge amount of collaboration (communication). The human brain is a processor, and like the von Neumann architecture has subcomponents (lobes), working memory, etc. What happens when our processor hits 100% utilization? We simply can't process all the inputs fast enough and our ability to process information slows down dramatically. We even reach a state where the decision-making centre in our brain shuts itself down. Coupled with a decreasing ability to process information and make decisions, we experience increasing levels of anxiety. Now, let's look at teams. They represent a communications network, with the people being the nodes in the network. When the team reaches 100% utilization, the individual processors (the brains of the people!) begin to slow as the ability to process information diminishes and anxiety increases. This has the effect of slowing the team's ability to communicate and operate effectively. When one or multiple people reach the point of shutting down, the network collapses. Why does anyone think that 100% utilization is a good thing? In manufacturing, 100% utilization of the capacity of your workers is actually desirable. The thinking aspects have already been done in the design and engineering of the item being created, and the assembly process is infinitely repeatable until a new item comes along. This approach to the process was taken from the manufacturing world and applied to the development of software, with the thinking that assembly was done by the programmers after all the design and engineering had taken place. Well, that simply isn't true. The "assembly" of software isn't programming, it's the compilation and packing into something deployable. Writing software is a confluence of design and engineering that has creative and technical aspects. It's not an assembly line, and probably won't ever be within my lifetime. As long ago as the late 1950's, management guru Peter Drucker realized that there were people who "think for a living". He coined the term knowledge worker to describe that type of work. Developing software is a classic knowledge worker role and therefore has different rules for productivity. As Drucker said in his 1999 book Management Challenges of the 21st Century, Productivity of the knowledge worker is not — at least not primarily — a matter of the quantity of output. Quality is at least as important. If we're pushing individuals and teams to 100% capacity, the quality of work and therefore the productivity of the team will be reduced. In 2001, Tom deMarco released Slack: Getting Past Burnout, Busywork and the Myth of Total Efficiency, an entire book describing the problems created by trying to achieve 100% utilization. I highly recommend the book, and also recommend you give your manager a copy. :) Here are some selected quotes that are pertinent to reducing the load on our processor and communication network: Very successful companies have never struck me as particularly busy; in fact, they are, as a group, rather laid back. Energy is evident in the workplace, but it's not the energy tinged with fear that comes from being slightly behind on everything. The principal resource needed for invention is slack. When companies can't invent, it's usually because their people are too damn busy. People under time pressure don't think faster. - Tim Lister How Can We Deal With This? First and foremost, stop trying to do so damned much! This is truly a slow down to go faster type of solution. For a Team If you're an agile team using iterations or sprints, pull less work into a sprint even if your velocity says you should be able to pull more. The extra time can be used to increase the quality of the work that you do complete, which is in itself a motivating factor for knowledge workers. Increased quality means decreased rework, which means the team as a whole delivers faster over the long term. Similarly, the reduced anxiety and stress on the team members increases their ability to think about what they're doing, meaning better and more innovative solutions to problems. Also, ensure that teams are focusing on activities that contribute directly to their work. Meetings tend to be the worst offenders of this, including the meetings defined within agile processes. Some suggestions: No meeting can be scheduled unless there is a specific question to be answered or specific information to be passed along. Reduce the default meeting length from 1 hour to 30 minutes in whatever calendar tool you use. Even better, reduce it to 20 minutes. Choose blocks of time that are deemed meeting free zones. No one, regardless of their position in the org chart can schedule a meeting with the team or the team members during that time. Minimize meetings where people are on a phone. My observation is that people who call into meetings tend to speak longer than when they are in the room with everyone else. I know that I do this, and I see it often in others. For an Individual Learn to say, "No." At the very least, learn to say, "Not yet." It's really difficult to do this, and I speak from personal experience. We want to help others, we want to be team players. However, if we don't say no, we take on too much, our processing and communication ability suffers and we end up disappointing those we were trying to help! Also, learn to take breaks. Yes, take breaks. The same concepts from Slack apply here, with brief diversions clearing our mind and allowing us to work better afterwards. At the extreme, if you feel like you can't keep your eyes open, take a nap! I highly recommend the Pomodoro Technique as at least a starting point for giving yourself some slack. Step away from your work for just a few minutes and return recharged. Conclusion We don't want the CPU in our computer to be working at 100% and we don't want the communications network to which it's connected to be at 100% capacity either. Our brains are processors and teams are a communications network, so why would we want those to be 100% busy all the time? In fact, ensuring that teams and the people on them are always busy is a provably wrong approach for software development. It has it's roots in a manufacturing mindset and developing software is knowledge work. Software development requires substantially different ways to make teams and the individual people on them as productive as possible. Finally, there is no magic number for what percentage of utilization is best. People are variable and 90% utilization for a person might be fine on a given day while %30 is all that the same person can handle on the next. Don't aim for a number, aim for an environment where people know that they don't have to appear busy. That will leave plenty of capacity for each person's processor and their team's communications network to run smoothly and deliver real value more effectively.

Retry-After is a lesser known HTTP response header.

in this brief tutorial you will learn how to develop a camera viewer application in c# that allows you to display the image of your usb webcam and to stream the camera image to remote pcs and smartphones. instead of presenting a long article, i would rather show how to implement such application with a few lines of c# code by using the prewritten components of a c# camera library. prerequisites a visual c# wpf application created in visual studio the voipsdk.dll added to the references. (it can be found on the official website of this c# camera library .) a media player supporting rtsp streaming (e.g. vlc) installed on a remote pc first of all let’s build the gui. if you follow the content of the mainwindow.xaml file line-by-line, you will see how to create user all the necessary gui elements that allows the user to be able to connect to a usb camera and display its image, and to set the listen address (including 2 textboxes for the ip address and the port number) that makes rtsp streaming possible. (the following figure illustrates the gui that can be created by using this code snippet.) in the mainwindow.xaml.cs file you will see how to implement the camera viewer functionality and how to turn your application as a video server. to test your application run the program, click the connect button, then when the camera image is displayed, enter the ipv4 address of your pc as listening address, and specify ’554’ as a port number. thereafter open the vlc media player on an other pc or smartphone, and open the network media stream by entering the following network url: rtsp://192.168.115.1:554 (that is: rtsp://youripv4address/portnumber). the result can be seen below: i hope my code snippet was useful! happy programming! // mainwindow.xaml // mainwindow.xaml.cs using system; using system.collections.generic; using system.linq; using system.runtime.interopservices; using system.text; using system.threading.tasks; using system.windows; using system.windows.controls; using system.windows.data; using system.windows.documents; using system.windows.input; using system.windows.media; using system.windows.media.imaging; using system.windows.navigation; using system.windows.shapes; using ozeki.media.ipcamera; using ozeki.media.mediahandlers; using ozeki.media.mediahandlers.video; using ozeki.media.mjpegstreaming; using ozeki.media.video.controls; namespace basic_cameraviewer { /// /// interaction logic for mainwindow.xaml /// public partial class mainwindow : window { private videoviewerwpf _videoviewerwpf; private bitmapsourceprovider _provider; private iipcamera _ipcamera; private webcamera _webcamera; private mediaconnector _connector; private myserver _server; private ivideosender _videosender; public mainwindow() { initializecomponent(); _connector = new mediaconnector(); _provider = new bitmapsourceprovider(); _server = new myserver(); setvideoviewer(); } private void setvideoviewer() { _videoviewerwpf = new videoviewerwpf { horizontalalignment = horizontalalignment.stretch, verticalalignment = verticalalignment.stretch, background = brushes.black }; camerabox.children.add(_videoviewerwpf); _videoviewerwpf.setimageprovider(_provider); } #region usb camera connect/disconnect private void connectusbcamera_click(object sender, routedeventargs e) { _webcamera = webcamera.getdefaultdevice(); if (_webcamera == null) return; _connector.connect(_webcamera, _provider); _videosender = _webcamera; _webcamera.start(); _videoviewerwpf.start(); } private void disconnectusbcamera_click(object sender, routedeventargs e) { if (_webcamera == null) return; _videoviewerwpf.stop(); _webcamera.stop(); _webcamera.dispose(); _connector.disconnect(_webcamera, _provider); } #endregion private void guithread(action action) { dispatcher.begininvoke(action); } private void startserver_click(object sender, routedeventargs e) { var ipadress = ipaddresstext.text; var port = int.parse(porttext.text); _server.videosender = _videosender; _server.onclientcountchanged += server_onclientcountchanged; _server.start(); _server.setlistenaddress(ipadress, port); } void server_onclientcountchanged(object sender, eventargs e) { guithread(() => { connectedclientlist.items.clear(); foreach (var client in _server.connectedclients) connectedclientlist.items.add("end point: " + client.transportinfo.remoteendpoint); }); } private void stopserver_click(object sender, routedeventargs e) { _server.onclientcountchanged -= server_onclientcountchanged; _server.stop(); } } }

There are some instances when you want to store your passwords in files to be used by programs or scripts. But storing your passwords in plain text is not a good idea. Use the SecurePasswordVault to encrypt your passwords before storing and get it decrypted when you want to use it. You can use the SecurePasswordVault described here to store any number of encrypted passwords. Passwords are stored as key value pairs. Key - any name given by the user for the password Value - encrypted password SecurePasswordVault will create a file with the given name in the working directory if it doesn't exist. If a file exists then the information in that file will be read. Passwords are encrypted using the MAC address of the network card. SecurePasswordVault will use the first network card MAC which is not the loop back interface. So the encrypted file can only be decrypted with that particular MAC address. If you want to reset the pass word details, just delete the password file and run the SecurePasswordVault. You can download the sample code from the following GitHub repository https://github.com/jsdjayanga/secure_password com.wso2.devgov; import org.bouncycastle.util.encoders.Base64; import javax.crypto.*; import javax.crypto.spec.SecretKeySpec; import java.io.*; import java.net.NetworkInterface; import java.net.SocketException; import java.security.InvalidKeyException; import java.security.NoSuchAlgorithmException; import java.security.Security; import java.util.*; /** * Created by jayanga on 3/31/14. */ public class SecurePasswordVault { private static final int AES_KEY_LEN = 32; private static final int PASSWORD_LEN = 256; private static boolean initialized; private final String secureFile; private final byte[] networkHardwareHaddress; private Map secureDataMap; private List secureDataList; SecretKeySpec secretKey; public SecurePasswordVault(String filename, String[] secureData) throws IOException { Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider()); initialized = false; secureFile = filename; networkHardwareHaddress = SecurePasswordVault.readNetworkHardwareAddress(); secureDataMap = new HashMap(); this.secureDataList = new ArrayList(secureData.length); Collections.addAll(secureDataList, secureData); byte[] key = new byte[AES_KEY_LEN]; Arrays.fill(key, (byte)0); for(int index = 0; index < networkHardwareHaddress.length; index++){ key[index] = networkHardwareHaddress[index]; } secretKey = new SecretKeySpec(key, "AES"); if (!isInitialized()){ readSecureData(secureDataList); persistSecureData(); } readSecureDataFromFile(); } private boolean isInitialized(){ if (initialized == true){ return true; }else{ File file = new File(secureFile); if (file.exists()){ initialized = true; return initialized; } } return false; } private static byte[] readNetworkHardwareAddress() throws SocketException { Enumeration networkInterfaceEnumeration = NetworkInterface.getNetworkInterfaces(); if (networkInterfaceEnumeration != null){ NetworkInterface networkInterface = null; while (networkInterfaceEnumeration.hasMoreElements()){ networkInterface = networkInterfaceEnumeration.nextElement(); if (!networkInterface.isLoopback()){ break; } } if (networkInterface == null){ networkInterface = networkInterfaceEnumeration.nextElement(); } byte[] hwaddr = networkInterface.getHardwareAddress(); return hwaddr; }else{ throw new RuntimeException("Cannot initialize. Failed to generate unique id."); } } private byte[] encrypt(String word) { byte[] password = new byte[PASSWORD_LEN]; Arrays.fill(password, (byte)0); byte[] pw = new byte[0]; try { pw = word.getBytes("UTF-8"); for(int index = 0; index < pw.length; index++){ password[index] = pw[index]; } byte[] cipherText = new byte[password.length]; Cipher cipher = null; try { cipher = Cipher.getInstance("AES/ECB/NoPadding"); try { cipher.init(Cipher.ENCRYPT_MODE, secretKey); int ctLen = 0; try { ctLen = cipher.update(password, 0, password.length, cipherText, 0); ctLen += cipher.doFinal(cipherText, ctLen); return cipherText; } catch (ShortBufferException e) { e.printStackTrace(); } catch (BadPaddingException e) { e.printStackTrace(); } catch (IllegalBlockSizeException e) { e.printStackTrace(); } } catch (InvalidKeyException e) { e.printStackTrace(); } } catch (NoSuchAlgorithmException e) { e.printStackTrace(); } catch (NoSuchPaddingException e) { e.printStackTrace(); } } catch (UnsupportedEncodingException e) { e.printStackTrace(); } return null; } private String decrypt(byte[] cipherText) { byte[] plainText = new byte[PASSWORD_LEN]; Cipher cipher = null; try { cipher = Cipher.getInstance("AES/ECB/NoPadding"); try { cipher.init(Cipher.DECRYPT_MODE, secretKey); int plainTextLen = 0; try { plainTextLen = cipher.update(cipherText, 0, PASSWORD_LEN, plainText, 0); try { plainTextLen += cipher.doFinal(plainText, plainTextLen); String password = new String(plainText); return password.trim(); } catch (IllegalBlockSizeException e) { e.printStackTrace(); } catch (BadPaddingException e) { e.printStackTrace(); } } catch (ShortBufferException e) { e.printStackTrace(); } } catch (InvalidKeyException e) { e.printStackTrace(); } } catch (NoSuchAlgorithmException e) { e.printStackTrace(); } catch (NoSuchPaddingException e) { e.printStackTrace(); } return null; } public void readSecureData(List secureDataList) throws IOException { BufferedReader bufferRead = new BufferedReader(new InputStreamReader(System.in)); for(int index = 0; index < secureDataList.size(); index++){ System.out.println("Please enter the value for :" + secureDataList.get(index)); String value = new String(Base64.encode(encrypt(bufferRead.readLine()))); secureDataMap.put(secureDataList.get(index), value); } } public String getSecureData(String key) { String value = secureDataMap.get(key); if (value != null){ return decrypt(Base64.decode(value.getBytes())); } throw new RuntimeException("Given key is unknown. [key=" + key + "]"); } private void readSecureDataFromFile() throws IOException { BufferedReader br = new BufferedReader(new FileReader(secureFile)); String line; while ((line = br.readLine()) != null){ int dividerPoint = line.indexOf("="); if (dividerPoint > 0){ secureDataMap.put(line.substring(0, dividerPoint), line.substring(dividerPoint + 1)); } } } private void persistSecureData() throws IOException { FileWriter fileWriter = new FileWriter(secureFile); for(String key : secureDataMap.keySet()){ fileWriter.append(key + "=" + secureDataMap.get(key) + "\n"); } fileWriter.close(); } }