Sofien, one of GitGuardian's tech leads, describes how pre-commit hooks are used to save time and also secure commits company-wide.

SRE is a modern approach to managing the risks inherent in running complex, dynamic software deployments – risks like downtime, slowdowns, and the like.

Custom user authentication needs to be set up essentially, so we are going to define the major ways for the developer: from ready services to custom coding.

Learn how to build a geo-distributed API layer with Kong Gateway.

This article will show how to expose an application using the service type load balancer.

OWASP’s number one risk in the Top 10 has featured prominently in a high-profile attack this time resulting in the leak of over 40,000 records from Bell in Canada. It was pretty self-evident from the original info leaked by the attackers that SQL injection had played a prominent role in the breach, but now we have some pretty conclusive evidence of it as well: The usual fanfare quickly followed – announcements by the attackers, silence by the impacted company (at least for the first day), outrage by affected customers and the new normal for public breaches: I got the data loaded into Have I been pwned? and searchable as soon as I’d verified it. Now you would think – quite reasonably – that SQLi would be becoming a thing of the past what with all the awareness and Top 10 stuff let alone the emergence of tools like object relational mappers that make it almost impossible to screw this up, but here we are. Clearly we need a bit of a refresher on the risk and what better way to do it than to reconstruct the Bell system that was breached then, well, breach it again. Let’s get to it. A long time ago in a language far, far away The first thing that should hit savvy readers in the image above is that this is an ASP web site. No, not ASP.NET, go back further – classic ASP. This is not classic like, say, a Ferrari 250 GTO which has grown increasingly desirable with age, rather it’s “classic” like Citroën 2CV; it was kinda cool at the time but you’d be damned if you want a mate seeing you in one today. But I digress. Classic ASP was replaced almost 12 years ago to the day with the platform that remains Microsoft’s framework of choice for building web sites today – ASP.NET. You could forgive someone for persevering with classic ASP a decade ago, perhaps even 5 years ago, but today? I don’t think so. If you’re running this platform today to host anything of any value whatsoever on the web, you’ve got rocks in your head. (Yes, I know it’s still supported but seriously folks, it was built for another era and just isn’t resilient to today’s web risks) Anyway, to reproduce this risk I’m going to create a very simple classic ASP site that looks just like the one above. That’s one thing that was great about classic ASP – it was dead easy to create a simple site! For added realism I’ll create a local host entry for protectionmanagement.bell.ca and even add a self-signed cert so we can hit it via HTTPS. The affected site has been well and truly pulled by now, but of course nothing is ever really gone on the internet, it just goes to Google cache heaven: That shouldn’t be too hard to reproduce, how’s this look? Forgive me if I don’t go so far as to recreate the broken images! Let’s move on to the other thing we know about the attack, and that’s what the back end database looks like. Implementing the back end What we need to make this whole thing resemble a real attack is a little bit of classic ASP wiring and a database. The latter is quite easy to reconstruct because the entire schema was dumped along with the breach. Yes, yes, the breach got pulled very early by the powers that be, but per the earlier point, cache is your friend. Here’s what we’ve been told about the credentials table: Columns: tblCredentials tblCredentials.CredentialID, tblCredentials.OrderID, tblCredentials.CustomerID, tblCredentials.ServiceType, tblCredentials.UserName, tblCredentials.Password, tblCredentials.Level, tblCredentials.CustomerName, tblCredentials.PersonName, tblCredentials.GroupID, tblCredentials.SecretQuestion, tblCredentials.SecretAnswer, tblCredentials.UserEmail tblCredentials.UserLanguage By prefixing the table with the letters “tbl” you know that it’s a table and not a magic unicorn or a Chinese dissident (let us not digress into the insanity that is the tbl prefix). Anyway, I’ve recreated that table and another one called tblTransaction2010 in my local SQL instance that looks just like this. I’ve then whipped up a little VB Script in the .asp file (“Tonight we’re gonna code like it’s 1999”) which connects to the database and runs a SQL statement constructed like this: SQL = "SELECT * FROM tblCredentials WHERE UserName='" + Request.Form("UserName") + "'" Yeah, that looks about right! Let’s see what happens now… Employing HackBar The extension you see in the first image of this post is HackBar, a simple little add-on for testing things like SQLi and XSS. The premise is that it can monitor requests the browser makes and then make it dead easy to reconstruct them with manipulated parameters, you know, the kind of stuff that can exploit SQLi risks. It looks like this: What I’ve done is tried to perform a reset for the username “troy” (which performs a post request to the server) then I’ve just hit the “Load URL” button and checked “Enable Post data”. That then gives us the resource that was hit in the top text box and the form data with name value pairs in the bottom. Dead simple, now let’s break some stuff. Mounting the attack What we see in the first image above is what’s known as an error-based SQLi attack or in other words, the attacks are using exceptions thrown by the server and sent back in the response to discover the internal implementation of the system. I talk about this and other SQLi attacks patterns in my post on Everything you wanted to know about SQL injection (but were afraid to ask). Let’s reproduce what the attackers have in that first image – disclosure of the internal database version. This is a useful first step as it helps attackers understand what they’re playing with. Different database environments and even versions are exploited in different ways so discovering this early is important, question is, how do you get the database to cough this information up? In the post I mention above, I show how attempting to cast non-integer values to an integer will throw an internal exception which discloses the data. The first thing we need to establish is how to generate the data which in this case is the DB version. That’s dead simple, we just ask for @@VERSION then if we try to convert that to an int and the exception bubbles up to the browser, we’ve got ourselves some useful info. Does this look about right? And there we have it – the DB version data. I’ve all done with the post data is sent it over like this: UserName=' or 1=convert(int, @@version)-- Clearly the version won’t convert to an int so we get the error above. The %2x values you’re seeing in the HackBar window are simply URL encoded characters which can be achieved by selecting the string and then then choosing the correct encoding context from the menu (I’ll leave the unencoded values there in future grabs for the sake of legibility): So this is a start, but where’s the good stuff? How about we move onto discovering the schema because until we know what tables and columns are in there, it’s going to be a tough job pulling the data. Let’s start with table names: tblTransaction2010 is it? We know it’s a table because of the prefix… ok, I’ll let it go, point is we now know a table name and all it took was to select out of sysobjects. I go into detail about how this works in the aforementioned everything you want to know post so I won’t dwell on it here, let’s get another table name: Ah, so there’s our tblCredentials table and all it took was to adjust one number in the query so that the inner select statement took the top 2 records instead of the top 1 thus allowing the outer select to grab the next table in sysobjects. Let’s get some columns and there’s no one “right way” of doing this as there are multiple ways of pulling columns names from SQL Server (and for pulling table names too, for that matter). Let’s try this one: The exception discloses the presence of a column called UserName on the table tblCredentials. That’s handy, let’s move on and I’ll just keep incrementing the integer in the inner select statement: Ah, so there’s a password column as well, that’s handy, let’s see about pulling some data out of there: I’ve deliberately simplified this statement so it just pulls the first record in the default order but by now these nested, sorted selects should give you an idea of how easy it is to enumerate through the data. So there’s the username – “troy” – let’s grab the password too: This is unfortunate because clearly I’ve taken my personal security seriously and substituted not only the “a” for an “@”, but also the “o” for a “0”. But when you don’t have any cryptographic storage on the credentials which was the case with Bell, even my real passwords that are all randomly generated by 1Password have nowhere to hide when an SQLi attack hits pay dirt. In practice, you’re not going to go through and manually enumerate every single table, column and then row (column by column, I might add), instead you’re going to automate the process using a tool like Havij once you’ve discovered an at-risk target. If Havij is new to you, it’s child’s play – here’s my 3 year old learning how to use it, it really is that simple. There will be nuances between how I’ve replicated the attack here and how the guys behind it actually went about it. There might be other vectors through other pages or depending on how the original password recovery page responded, more streamlined ways of pulling the data. There may have even been SQL credential exposure at some point which would make the whole thing dead easy. Either way Bell (or whoever is copping the blame) will have more than enough data in their logs to reconstruct the attack and know exactly where it all went wrong. Hardening Bell’s environment Firstly, yes, I know that Bell has laid the blame on a partner providing services to some of their customers but it’s Bell in the headlines, it’s Bell sending out the apology emails and it’s Bell who now has to clean up this mess. I say this not to berate Bell but to draw attention to the responsibility that organisations have to ensure that their partners are employing appropriate security measures. The risk above could have been discovered in minutes by and automated tool and almost as quickly by even the most junior penetration tester. Nobody tested this system for security vulnerabilities – including Bell – and now they have a very unfortunate blight on their record that will be referenced for years to come. Anyway, let’s focus on the mitigations of this risk because as I said from the outset, this needs to be taken as an opportunity for others to learn some fundamentals that could save them from a similar fate. Let me summarise in point form: Using out-dated frameworks: Classic ASP guys – get rid of it. It has nowhere near the defences that modern web platforms have in place not just for SQLi, but for a whole range of attacks. You cannot afford to keep running VB script on the server. No white-listing of untrusted data: In the example above (and inevitably in the real system), SQLi attacks were thrown at the website and it… welcomed them with open arms. “Validate all untrusted data against a whitelist of allowable values” is the mantra I’ve repeated so many times and the username should only be allowing characters that it actually accepted when people signed up so that means no brackets, quotes, spaces, etc (none of these are in the breached data). Non-parameterised SQL: My example earlier on about how the SQL statement was likely constructed shows just a concatenated string with the potential to mix the query with untrusted data. This is what got them and I talk extensively about the right way to do this in part one of my series on the Top 10. Internal implementation leakage: This attack was made dead easy by the fact that internal exceptions bubbled up to the UI. Someone had to actually enable this – newer versions of IIS won’t allow to happen by default. The extent of this risk goes well beyond SQLi as well as there are some very, very juicy things that web sites sharing their internals can disclose. Plain text password storage: Shit happens. Sites get breached. We now all understand that, but what makes it a whole lot worse is when the data is usable by attackers, not just the ones who pulled it, but anyone in the general public who now has access to it. Passwords should always be stored with a strong cryptographic hashing algorithm designed for protecting credentials. Anything short of this leaves you naked in an attack. They’re just a few easy ones – SQLi 101 – and they should be painfully obvious. In conclusion… SQLi attacks remain rampant. They’re still in the number one spot in OWASP’s Top 10 (even the latest 2013 version) and it’s still rated as easy to exploit and as having a severe impact. They’re favoured by attackers because they’re just so easy to crack open which was the point of showing my 3 year old doing it earlier on. In this case the attackers actually showed a decent understanding of the mechanics behind SQLi, but the point is that the barrier to entry for this attack can be very, very low. Lastly, if you’re a dev or managing devs then get them into some in-depth security training whether that be via my Pluralsight courses or though any of the other excellent resources out there. You can’t wait until after things go wrong to do this.

Avalanche has emerged as one of the most formidable alternatives to Ethereum, becoming the fastest smart contract platform in the blockchain industry.

google cloud messaging (or gcm) sends two types of messages: collapsible, “send-to-sync” messages, where new messages replace older ones in the sending queue. (i.e. the older messages are “collapsed”). non-collapsible messages with payload, where every single message is delivere d. each payload in non-collapsible messages is a unique content that has to be delivered and can’t be just replaced with a more recent message in the server sending queue. on the other hand, a collapsible message can be a simple ping from the server to ask its mobile clients to sync their data. when to use messages with payload? instant messaging (im) applications come to mind. another use case is when we need to include data into our push notifications to save our clients a round-trip to the server. an example use case would be sending daily/monthy online game rankings of top players. instead of just notifying the android clients to go to the server to get the information ( send-to-sync ), the data is included in the multicast messages themselves so that they can be directly consumed by the clients. we can easily see why collapsible messaging wouldn’t make much sense here, since we want our users to receive every single ranking we send them at the end of every week/month. let’s now jump into coding. as suggested in the two previous article, these code examples are modifications of the gcm demo application available for download (client + server) and using the gcm helper library for java . server code creating a message with payload in server code is similar to the process for the collapsible type, except that we simply omit the collapse_key parameter. // in imports import com.google.android.gcm.server.message; // inside send method, construct a message with payload message message = new message.builder() .delaywhileidle(true) // wait for device to become active before sending. .adddata( "rankings", "top 5 high game scorers" ) .adddata( "1", "1. yankeedoodle (15 trophies)" ) .adddata( "2", "2. billy_the_kid (13 trophies)" ) .adddata( "3", "3. viper (10 trophies)" ) .adddata( "4", "4. silversurfer (9 trophies)" ) .adddata( "5", "5. gypsy (8 trophies)" ) .build(); client code the client just retrieves the data by its keys: // inside gcmintentservice @override protected void onmessage(context context, intent intent) { // message with payload string message = intent.getstringextra("rankings") + "\n" + intent.getstringextra("1")+ "\n" + intent.getstringextra("2")+ "\n" + intent.getstringextra("3")+ "\n" + intent.getstringextra("4")+ "\n" + intent.getstringextra("5"); displaymessage(context, message); // notifies user generatenotification(context, message); } this is how it looks like on an android device: these messages can contain a payload limit of 4k of data so they are not indicated for applications that need to send more than that, although it is theoretically possible to get around that limit with multiple messages and some assembly work on the receiving android client application. another aspect to consider would be performance and impact on the handset’s batteries, since messages with payload are not as lightweight as their collapsible cousins. regardless of the message type, all payload data must be string values . so if we need to send some other type, it is up to our application to properly encode/decode the content.

I have talked about human filters and my plan for digital curation. These items are the fruits of those ideas, the items I deemed worthy from my Google Reader feeds. These items are a combination of tech business news, development news and programming tools and techniques. Dew Drop – June 7, 2013 (#1,563) (Alvin Ashcraft's Morning Dew) On friction in software (Ayende @ Rahien) Caching, jQuery Ajax and Other IE Fun (HTML5 Zone) IndexedDB and Date Example (HTML5 Zone) DevOps Scares Me – Part 1 (Architects Zone – Architectural Design Patterns & Best Practices) Visualizing the News with Vivagraph.js (Architects Zone – Architectural Design Patterns & Best Practices) My First Clojure Workflow (Javalobby – The heart of the Java developer community) Helping an ISV Look at Their Cloud Options (Architects Zone – Architectural Design Patterns & Best Practices) Ignore Requirements to Gain Flexibility, Value, Insights! The Power of Why (Javalobby – The heart of the Java developer community) Estimating the Unknown: Dates or Budgets, Part 1 (Agile Zone – Software Methodologies for Development Managers) Team Decision Making Techniques – Fist to Five and others (Agile Zone – Software Methodologies for Development Managers) The Daily Six Pack: June 7, 2013 (Dirk Strauss) Pastime (xkcd.com) The Affect Heuristic (Mark Needham) Every great company has been built the same way: bit by bit (Hacker News) Under the Hood: The entities graph (Facebook Engineering's Facebook Notes) Entrepreneurship With a Family is for Crazy People (Stay N Alive) Thinking Together for Release Planning (Javalobby – The heart of the Java developer community) I hope you enjoy today’s items, and please participate in the discussions on those sites.

I have talked about human filters and my plan for digital curation. These items are the fruits of those ideas, the items I deemed worthy from my Google Reader feeds. These items are a combination of tech business news, development news and programming tools and techniques. Real-Time Ad Impression Bids Using DynamoDB (Amazon Web Services Blog) The mother of all M&A rumors: AT&T, Verizon to jointly buy Vodafone (GigaOM) Is this the future of memory? A Hybrid Memory Cube spec makes its debut. (GigaOM) Dew Drop – April 2, 2013 (#1,518) (Alvin Ashcraft's Morning Dew) Rosetta Stone acquires Livemocha for $8.5m to move its language learning platform into the cloud (The Next Web) Double Shot #1098 (A Fresh Cup) Extending git (Atlassian Blogs) A Thorough Introduction To Backbone.Marionette (Part 2) (Smashing Magazine Feed) 60 Problem Solving Strategies (Javalobby – The heart of the Java developer community) Why asm.js is a big deal for game developers (HTML5 Zone) Implementing DAL in Play 2.x (Scala), Slick, ScalaTest (Javalobby – The heart of the Java developer community) “It’s Open Source, So the Source is, You Know, Open.” (Javalobby – The heart of the Java developer community) How to Design a Good, Regular API (Javalobby – The heart of the Java developer community) Scalding: Finding K Nearest Neighbors for Fun and Profit (Javalobby – The heart of the Java developer community) The Daily Six Pack: April 2, 2013 (Dirk Strauss) Usually When Developers Are Mean, It Is About Power (Agile Zone – Software Methodologies for Development Managers) Do Predictive Modelers Need to Know Math? (Data Mining and Predictive Analytics) Heroku Forces Customer Upgrade To Fix Critical PostgreSQL Security Hole (TechCrunch) DYNAMO (Lambda the Ultimate – Programming Languages Weblog) FitNesse your ScalaTest with custom Scala DSL (Java Code Geeks) LinkBench: A database benchmark for the social graph (Facebook Engineering's Facebook Notes) Khan Academy Checkbook Scaling to 6 Million Users a Month on GAE (High Scalability) Famo.us, The Framework For Fast And Beautiful HTML5 Apps, Will Be Free Thanks To “Huge Hardware Vendor Interest” (TechCrunch) Why We Need Lambda Expressions in Java – Part 2 (Javalobby – The heart of the Java developer community) I hope you enjoy today’s items, and please participate in the discussions on those sites.

A few weeks ago I wrote a blog on synchronizing multithreaded integration tests, which was republished on DZone Javalobby from where it received a comment from Robert Saulnier who quite rightly pointed out that you can also use join() to synchronize a worker thread and its unit tests. This got me thinking, just how many ways can you synchronise multi-threaded integration tests? So, I started counting... and came up with: Using a random delay. Adding a CountDownLatch Thread.join() Acquiring a Semaphore With a Future and ExecutorService Now, I’m not going to explain all the following in great detail, I’ll let the code speak for itself, except to say that all the code samples do roughly the same thing: the unit test creates a ThreadWrapper instance and then calls its doWork() method (or call() in the case of the Future). The unit test’s main thread then waits for the worker thread to complete before asserting that the test has passed. For the sample code demonstrating points 1 and 2 take a look at my original blog on Synchronizing Multithreaded Integration Tests, though I wouldn’t recommend point 1: using a random delay. Thread.join() public class ThreadWrapper { private Thread thread; /** * Start the thread running so that it does some work. */ public void doWork() { thread = new Thread() { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); } private void addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } }; thread.start(); System.out.println("Off and running..."); } /** * Synchronization method. */ public void join() { try { thread.join(); } catch (InterruptedException ex) { Thread.currentThread().interrupt(); } } } public class ThreadWrapperTest { @Test public void testDoWork() throws InterruptedException { ThreadWrapper instance = new ThreadWrapper(); instance.doWork(); instance.join(); boolean result = getResultFromDatabase(); assertTrue(result); } /** * Dummy database method - just return true */ private boolean getResultFromDatabase() { return true; } } Acquiring a Semaphore public class ThreadWrapper { /** * Start the thread running so that it does some work. */ public void doWork() { doWork(null); } @VisibleForTesting void doWork(final Semaphore semaphore) { Thread thread = new Thread() { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); semaphore.release(); } private void addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } }; aquire(semaphore); thread.start(); System.out.println("Off and running..."); } private void aquire(Semaphore semaphore) { try { semaphore.acquire(); } catch (InterruptedException e) { e.printStackTrace(); } } } public class ThreadWrapperTest { @Test public void testDoWork() throws InterruptedException { ThreadWrapper instance = new ThreadWrapper(); Semaphore semaphore = new Semaphore(1); instance.doWork(semaphore); semaphore.acquire(); boolean result = getResultFromDatabase(); assertTrue(result); } /** * Dummy database method - just return true */ private boolean getResultFromDatabase() { return true; } } With a Future public class ThreadWrapper implements Callable { @Override public Boolean call() throws Exception { System.out.println("Start of the thread"); Boolean added = addDataToDB(); System.out.println("End of the thread method"); return added; } /** * Add to the DB and return true if added okay */ private Boolean addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } return Boolean.valueOf(true); } } public class ThreadWrapperTest { @Test public void testCall() throws ExecutionException, InterruptedException { ThreadWrapper instance = new ThreadWrapper(); ExecutorService executorService = Executors.newFixedThreadPool(1); Future future = executorService.submit(instance); Boolean result = future.get(); assertTrue(result); } } Having listed all these methods, the next thing to consider is which one is the best? In asking that question you have to define the word “best” in terms of best for what? Best for simplicity? Maintainability? Speed or code size? After all Programming the Art of Making the Right Decision. You may have guessed that I don’t like the random delay idea and prefer the use of a CountDownLatch. Thread.join() is a bit old school; remember that the likes of Semaphore and CountDownLatch were written to improve on the original Java threading techniques. ExecutorService seems a little heavy weight for what we need here. At the end of the day the choice of technique really comes down to personal preference. Finally, I’ve listed five ways of synchronizing multi-threaded integration tests; however, if you can think of any others please let me know... The source code for this blog is available on Github.

In my last post, I described a new feature in TokuMX 1.5—partitioned collections—that’s aimed at making it easier and faster to work with time series data. Feedback from that post made me realize that some users may not immediately understand the differences between partitioning a collection and sharding a collection. In this post, I hope to clear that up. On the surface, partitioning a collection and sharding a collection seem similar. Both actions take a collection and break it into smaller pieces for some performance benefit. Also, the terms are sometimes used interchangeably when discussing other technologies. But for TokuMX, the two features are very different in purpose and implementation. In describing each feature’s purpose and implementation, I hope to clarify the differences between the two features. Let’s address sharding first. The purpose of sharding is to to distribute a collection across several machines (i.e. “scale-out”) so that writes and queries on the collection will be distributed. The main idea is that for big data, a single machine can only do so much. No matter how powerful your one machine is, that machine will still be limited by some resource, be it IOPS, CPU, or disk space. So, to get better performance for a collection, one can use sharding to distribute the collection across several machines, and thereby improve performance by increasing the amount of hardware. To perform these tasks, a sharded collection ought to have a relatively even distribution across shards. Therefore, it should have the following properties: User’s writes ought to be distributed amongst machines (or shards). After all, if all writes are targeted at a single shard, then they are not distributed and we are not scaling To keep data distribution relatively even, background process migrate data between shards if a shard is found to have too much or too little data Because of these properties, each shard contains a random subset of the collection’s data. Now let’s address partitioning. The purpose of partitioning is to break the collection into smaller collections so that large chunks of data may be removed very efficiently. A typical example is keeping a rolling period of 6 months of log data for a website. Another example is keeping the last 14 days of oplog data, as we do via partitioning in TokuMX 1.4. In such examples, typically only one partition (the latest one) is getting new data. Periodically, but infrequently, we drop the oldest partition to reclaim space. For the log data example, once a month we may drop a month’s worth of data. For the oplog, once a day we drop a day’s worth of data. To perform these tasks, we are not concerned with load distribution, as nearly all writes are typically going to the last partition. We are not spreading partitions across machines. With partitioning, each partition holds a continuous range of the data (e.g. all data from the month of February), whereas with sharding, each shard holds small random chunks of data from across the key space. With all this being said, there are still similarities when thinking of schema design with a partitioned collection and a sharded collection. As I touched on in my last post, designing a partition key has similarities to designing a shard key as far as queries are concerned. Queries on a sharded collection perform better if they target single shards. Similarly, queries on a partitioned collection perform better if they target a single partition. Queries that don’t can be thought of as “scatter/gather” for both sharded and partitioned collections. Hopefully this illuminates the difference between a partitioned collection and a sharded collection.

a recent study by gartner predicts a very significant increase in cloud usage by consumers in a few years, due in great part to the ever growing use of smartphone cameras by the average household. in this context, it could be useful to have a smartphone application that is able to upload / download digital content from a cloud provider. in this article, we will construct a basic android prototype that will allow us to plug in the windows azure cloud provider, and use the windows azure toolkit for android ( available at github ) to do all of the basic cloud operations : upload content to cloud storage, browse the storage, download or delete files in cloud storage. once those operations are implemented, we will see how to enable our android application to receive server push notifications . first things first, we need to set up a storage account in the azure cloud: a storage account comes with several options as for data management : we can keep data in blob, table or queue storage. in this article, we will use the blob storage to work with images. the storage account has a primary and secondary access key , either one of the two can be used to execute operations on the storage account. any of those keys can be regenerated if compromised. 1. preliminaries first, the prerequisites: eclipse ide for java android plugin for eclipse ( adt ) windows azure toolkit for android windows azure subscription (you can get a 90-day free trial ) a getting-started document on windows azure toolkit’s github page covers the installation procedure of all the the required software in detail. this whole project ( cloid ) is freely available at github . so here we’ll limit ourselves to presenting the most relevant code sections along with the corresponding screens. the user interface is composed of a few basic activity screens, spawned from the main screen (top center): since we use a technology not for its own sake but according to our needs, let’s start by specifying what we want: public abstract class storage { /** all providers will have accesss to context*/ protected context context; /** all providers will have accesss to sharedpreferences */ protected cloudpreferences prefs; /** all downloads from providers will be saved on sd card */ protected string download_path = "/sdcard/dcim/camera/"; /** * @throws operationexception * */ public storage(context ctx) throws operationexception { context = ctx; prefs = new cloudpreferences(ctx); } /** * @throws operationexception * */ public abstract void uploadtostorage(string file_path) throws operationexception; /** * @throws operationexception * */ public abstract void downloadfromstorage(string file_name) throws operationexception; /** * @throws operationexception * */ public abstract void browsestorage() throws operationexception; /** * @throws operationexception * */ public abstract void deleteinstorage(string file_name) throws operationexception; } the above is the contract that our cloud storage provider will satisfy. we’ll provide a mockstorage implementation that will pretend to carry out a command in order to test our ui (i.e. our scrollable items list, progress bar, exception messages, etc.), so that we can later just plug in azure storage operations. note from our activities screen above, that we can switch anytime between azure storage and mock storage with the press of the toggle button “cloud on/off” in the settings screen, saving the preferences afterward. public class mockstorage extends storage { // code here... @override public void uploadtostorage(string file_path) throws operationexception { donothingbutsleep(); //throw new operationexception( "test error message", // new throwable("reason: upload test") ); } // other methods will also do nothing but sleep... /***/ private void donothingbutsleep(){ try{ thread.sleep(5000l); } catch (interruptedexception iex){ return; } } 2. the azure toolkit the toolkit comes with a sample application called “simple”, and two library jars: access control for android.jar in the wa-toolkit-android\library\accesscontrol\bin folder azure storage for android.jar in the wa-toolkit-android\library\storage\bin folder here we will only use the latter, since we will access directly azure’s blob storage. needless to say, this is not the recommended way , since our credentials will be stored on the handset. a better approach security-wise would be to access azure storage through web services hosted on either azure or other public/private clouds. once the toolkit is ready for use, we need to think a bit about settings . using an azure blob storage only requires 3 fields: an account name , an access key , and a container for our images. the access key is quite a long string (88 characters) and is kind of a pain to type, so one way to do the setup is to configure the android res/values/strings.xml file to set the default values: ... cloid insert-access-key-here pictures ... however, because we may want to overwrite the default values above (e.g. create another container), we will also save the values on the settings screen in android’s sharedpreferences . and now, let’s implement the azurestorage class. 3. azure blob storage operations 3.1. storage initialization the azurestorage constructor gets its data from android preferences (from its superclass), then constructs a connection string used to access the storage account, creates a blob client and retrieves a reference to the container of images. if the user changed the default container “pictures” in settings, then a new (empty) one will be created with that new name. a container is any grouping of blobs under a name. no blob exists outside of a container. // package here // other imports import com.windowsazure.samples.android.storageclient.blobproperties; import com.windowsazure.samples.android.storageclient.cloudblob; import com.windowsazure.samples.android.storageclient.cloudblobclient; import com.windowsazure.samples.android.storageclient.cloudblobcontainer; import com.windowsazure.samples.android.storageclient.cloudblockblob; import com.windowsazure.samples.android.storageclient.cloudstorageaccount; public class azurestorage extends storage { private cloudblobcontainer container; / * @throws operationexception * */ public azurestorage(context ctx) throws operationexception { super(ctx); // set from prefs string acct_name = prefs.getaccountname(); string access_key = prefs.getaccesskey(); // get connection string string storageconn = "defaultendpointsprotocol=http;" + "accountname=" + acct_name + ";accountkey=" + access_key; // get cloudblobcontainer try { // retrieve storage account from storageconn cloudstorageaccount storageaccount = cloudstorageaccount.parse(conn); // create the blob client // to get reference objects for containers and blobs cloudblobclient blobclient = storageaccount.createcloudblobclient(); // retrieve reference to a previously created container container = blobclient.getcontainerreference( prefs.getcontainer() ); container.createifnotexist(); } catch (exception e) { throw new operationexception("error from initblob: " + e.getmessage(), e); } } // code... we will use that container reference cloudblobcontainer throughout our upcoming cloud operations. 3.2. uploading images we will upload a file from android’s gallery to the cloud, keeping the same filename. “screener” is just a utilities class (see github repository) that does a number of useful things, e.g. extracting a file name from its path and setting the right mime type (“image/jpeg”, “image/png”, etc.). the two kinds of blobs are page blobs and block blobs . the (very) short story is that page blobs are optimized for read & write operations, while block blobs let us upload large files efficiently. in particular we can upload multiple blocks in parallel to decrease upload time. here we are uploading a blob (gallery image) as a set of blocks. /** * @throws operationexception */ @override public void uploadtostorage(string file_path) throws operationexception { try { // create or overwrite blob with contents from a local file // use same name than in local storage cloudblockblob blob = container.getblockblobreference( screener.getnamefrompath(file_path) ); file source = new file(file_path); blob.upload( new fileinputstream(source), source.length() ); blob.getproperties().contenttype = screener.getimagemimetype(file_path); blob.uploadproperties(); } catch (exception e) { throw new operationexception("error from uploadtostorage: " + e.getmessage(), e); } } bear in mind that we are not checking if the file already exists in cloud storage. therefore we will overwrite any existing file with the same name as the one we are uploading. that is usually not desirable in production code. here’s the screen flow of the upload operation: 3.3. browsing the cloud for browsing, we store all our blobs in our container into a list of items that we will display in android as a scrollable list of image names in a subclass of android.app.listactivity . once one item in the list is clicked (“touched”) by the user, we want to display some image properties such as the image size (important when deciding to download), its mime type, and the date it was last operated upon. /** * @throws operationexception * */ @override public void browsestorage() throws operationexception{ // reset uri list for refresh - no caching item.itemlist.clear(); // loop over blobs within the container try { for (cloudblob blob : container.listblobs()) { blob.downloadattributes(); blobproperties props = blob.getproperties(); long ksize = props.length/1024; string type = props.contenttype; date lastmodified = props.lastmodified; item item = new item(blob.geturi(), blob.getname(), ksize, type, lastmodified); item.itemlist.add(item); } // end loop } catch (exception e) { throw new operationexception("error from browsestorage: " + e.getmessage(), e); } } here’s the screen flow of the browse operation. pressing on an item on the list displays its details and operations on the image, which we will look at next: 3.4. downloading images our download method is pretty straightforward. note that we are downloading to the android handset’s sd card by using download_path from the superclass. /** * @throws operationexception * */ @override public void downloadfromstorage(string file_name) throws operationexception{ try { for (cloudblob blob : container.listblobs()) { // download the item and save it to a file with the same name as arg if(blob.getname().equals(file_name)){ blob.download( new fileoutputstream(download_path + blob.getname()) ); break; } } } catch (exception e) { throw new operationexception("error from downloadfromstorage: " + e.getmessage(), e); } } and the corresponding ui flow. instead of displaying the image right after the download, we chose to include a link to the gallery (bottom of the screen) where the freshly retrieved image appears on top of the gallery’s stack of pictures: 3.5. deleting images the delete operation performed on a blob up in the cloud is also rather simple: /** * @throws operationexception */ @override public void deleteinstorage(string file_name) throws operationexception{ try { // retrieve reference to a blob named file_name cloudblockblob blob = container.getblockblobreference(file_name); // delete the blob blob.delete(); } catch (exception e) { throw new operationexception("error from deleteinstorage: " + e.getmessage(), e); } } and its associated ui screens series. note that after confirming the operation, and when deletion completes, the browsing list of items is automatically refreshed, and we can see that the image is no longer on the list of blobs in our storage container. 3.6. wrapping up the azurestorage methods are called inside a basic work thread, which will take care of all cloud operations: // called inside a thread try { // get storage instance from factory storage store = storagefactory.getstorageinstance(this, storagefactory.provider.azure_storage); // for the progress bar incrementworkcount(); // do ops switch(operation){ case upload : store.uploadtostorage(path); break; case browse : store.browsestorage(); break; case download : store.downloadfromstorage(path); // refresh gallery sendbroadcast( new intent( intent.action_media_mounted, uri.parse("file://"+ environment.getexternalstoragedirectory()) ) ); break; case delete : store.deleteinstorage(path); break; } // end switch } catch (operationexception e) { recorderror(e); } notice how we are telling the android image gallery to refresh by issuing a broadcast once a new file is downloaded from the cloud to the sd card. there are different ways to do this, but without that call, the gallery won’t show the new image before the next system scheduled media scan. again, for the full code, refer to this project on github. we are done with the basic cloud operations. all we had to do was plug in our azurestorage implementation class and get an instance of it through a factory, with minimal impact on preexisting code. 4. push notifications up to this point we have demonstrated device-initiated communication with the cloud. for cloud-initiated or push communication, the android platform uses google cloud messaging (gcm). in a previous article , i wrote about how to integrate gcm into an existing android application. here we will add a second set of settings for server push. our client code will connect with any gcm server and it will set the status on our main activity (last screen shot on the right) once the information in push preferences is correctly set. 5. conclusions the toolkit documentation is kind of sparse (which is why the community needs more articles like this). also, the sample application doesn’t cover much (maybe the reason why it’s called “simple”), and it has room for improvement. however, the library itself is fully functional, and once we figure out the api, it all works quite nicely. of course, this application is itself pretty basic and doesn’t cover lots of other features, like access control, permissions, metadata, and snapshots. but it is a start.

A rundown of NodePorts, LoadBalancers, Ingresses, and more in Kubernetes!

This article highlights key dimensions that organizations should consider while preparing for open banking.

Using TLS everywhere is one of the Kubernetes team's recommendations for hardening cluster security and increasing resilience. In this tutorial, you'll learn how to automate TLS certificate issuance to Kubernetes deployments.

Here we will use the Mulesoft Anypoint platform to attach VPC to the AWS transit gateway to form a single network topology.

Creating a card program can be challenging. Find out how Marqeta’s RiskControl portfolio mitigates risk and satisfies compliance needs, so you can remain focused.

In this blog, I would like to try another way of installing Pulsar in the containerized environment by using K3s.

Cloud platforms provide greater flexibility. How do you choose to compute service in Google Cloud?