Microservices vs. Small Services vs. Just-Right Services

Many people are debating microservices and for good reason. I have written a lot about the positive side of microservices. When I was at Pivotal, we had an awesome engineering culture, super agility, corporate buy-in, best-of-breed PaaS tools for microservices, an awesome open source library Spring Boot, and millions of reasons (containers!) to deploy microservices.

Your company may not have all (some or any) of those things. They certainly don't have hundreds of agile-trained developers who live and breath Twelve Factor Applications and the Microservice philosophy. Some people want to be all or nothing and see every problem that requires using their favorite solution (i.e., everything is a nail when you just bought a new hammer).

A lot of things I do fit the model of microservices really well. They are isolated, are self-contained, do one small function, use a few libraries, and are read-only. Those are awesome use cases for a microservice. I have a few examples of these posted here using Spring Boot to access limited data sets from Hive and Phoenix.

We need more best practices around when you can write a microservice and when you can't.

If you don't have a great infrastructure for microservices like CloudFoundry, AWS Lamba, Heroku, and other container-based approaches, it can be a mistake to jump all in microservices. Read the Twelve Factors and see if your infrastructure supports them.

I like to try to use microservices the Unix Way with the smallest functionality that will work and let them be connected to increase functionality. You need some infrastructure to help you.

For me, I have been using small cloud services for hosting Spring Boot microservices that run as Java JARs in their own Jetties on their own web ports about 8080. Those can easily be pushed into Docker containers, Cloud Foundry apps, Heroku, YARN, or elsewhere. I orchestrate them with Apache NiFi via GetHTTP and other processors. People always seem to think of microservices as REST or JSON only, but I think that an autonomous small self-running code as a microservice. This can be triggered by Kafka, JMS, MQTT, Execute, TCP, UDP, and other means. Apache NiFi provides a lot of ways to trigger small bits of modular code. Is that code enough to be a microservice? I would like to know.   

Working with big data, we have a lot of small applications, scripts, utilities, and simple command line apps. Orchestrating these with Apache NiFi (or Oozie), storing them in version control, using a build tool, and following test-driven development and agile methodologies will let us put out some really good code.  

Is a one-class Spark application in Scala that has less than 50 lines but is a full self-contained application triggered by incoming Kafka streams a big data microservice? Why not?

I write a lot of smaller Java, Scala, and Python applications that are standalone services. Could all of these be called microservices? Maybe. Is it easier?

Do you give up and write 1,000-class monoliths? I hope not.