SOA Darwinism & Social BPM - A Fireside Chat with Miko Matsumura
Join the DZone community and get the full member experience.Join For Free
DZone recently sat down with Miko Matsumura, vice president and deputy CTO of SOA products at Software AG webMethods. In this interview, Matsumura describes how cloud computing, virtualization and the current economic climate are helping facilitate the 'evolution' of SOA and what this ultimately means for the developer. He also sheds some light on 'Social BPM', the driving force behind Software AG's recently announced AlignSpace platform.
DZone: Miko, it's nice to touch base with you again. There have been a lot of developments since we last spoke with you on the state of SOA, so I thought we'd jump right into it. In a widely publicized blog entry titled, "SOA is Dead; Long Live Services," Burton Group analyst Anne Thomas Manes proclaimed that:
"SOA met its demise on January 1, 2009, when it was wiped out by the catastrophic impact of the economic recession." She goes on to say that SOA is now survived by its offspring, technology such as mashups, business process management, software as a service, cloud computing, and all the other architectural approaches that depend on "services."
What are your thoughts on this?
Miko Matsumura: Yes, I've had a great opportunity to hang out with Anne at various conferences and I just got an email from her this morning. We've been sharing slides on a variety of topics. I'm going to send her my slides from this morning's Enterprise Architect virtual conference.
I got her slides on "SOA is Dead" and the eulogy of SOA. Obviously I read her blog, which is a pretty exciting. See, in her blog post, there's only one picture and it's a picture of a dinosaur that says "SOAsaurus" written on the side of it. There's this huge meteor coming in from outer space that's labeled, "The Economy."
I think that her hypothesis is that this huge meteoric impact kind of smashes this SOAsaurus and it's kind of extinction time. Really, what we're seeing across the landscape, is we're actually not seeing extinction time. We're seeing evolution time.
The thing that's really fascinating - it kind of fits in with the model of evolution that's espoused by Stephen J. Gould, the late and great evolutionary biologist from Harvard. He really cited evolution as being punctuated equilibrium, which is that if things are stable and steady for a long time and then all of a sudden, boom! things are changed and evolution kind of leaps forward.
If you actually look at what happened to the dinosaur, in evolutionary times, the dinosaur itself actually is not extinct, rather that the ornithischian dinosaurs evolved flight. If you actually look - I'm looking outside my window right now and I see a bird sitting on a lamp post. That's really where the dinosaurs went.
It is a direct linkage and so adaptation has occurred. I think that that's really where we are with respect to SOA. There's a lot of adaptation, a kind of glorious, universal message of unification of architecture is being revised in that the economic position is forcing much more of a project-oriented mentality.
The thing that I think is important to try to piece back together from these pieces is that if you actually look at the statement that Anne made in her blog, SOA is survived by its offspring - mashups, BPM, SaaS, and cloud computing and other architectural approaches that depend on services.
The thing that I think is interesting about that statement, though, is that if all of those notions, if you treat them as being little mini projects, little development projects, they're certainly interesting. I think that unless you can actually take an architectural approach and re-integrate in a new environment this kind of directionality of creating an engine, a platform of technology for your organization, then I think you're kind of sliding back downhill towards architectural degradation.
Architectural degradation, from a developer perspective, I think it's really important to think about, because architecture is really kind of an economic imperative. The reason why I think this is worthwhile to think about from a developer perspective is that, you may as a developer be poking yourself into the cloud. You may hop up on EC2 and start racking up a bunch of cool applications up there. Or you may be doing some kind of mashup or you may be consuming services or whatever. The thing that I think is the key point is that you're doing so within the context of your organization.
DZone: How is the current economic climate impacting the way we think about SOA?
Miko: As enterprise developers, it's really important to kind of understand the economic context, actually today. I think people are starting to think about issues like job security.
It's unbelievable what's happening out in the economy. A friend of mine who was supposed to be interviewed yesterday by a reporter, he was a little bit irritated that the reporter didn't show up until he figured out that the reporter had been laid off that day. That's kind of what's going on. That was yesterday.
Just the feeling is, what is the significance of these larger-scale enterprise movements. Having had the good fortune to talk with literally hundreds of SOA projects and SOA leaders across all these continents, I've really had a great opportunity to understand the trend lines. Frankly, SOA is evolving in that people are continuing these projects and the imperative is pretty clear.
Let me explain the economic context of this, just because it's important to feel out where your IT projects and proposals and applications fit within the context of the enterprise. There's really two major forces in the economy, one of which is the competition.
The competition drives organizations to what an evolutionist calls "speciate," which is to divide and separate and fill niches. So, you're constantly trying to fill a niche. "Well, we're the people's bank." Or, "We're the local bank." Or, "We're the bank that provides access to banking on your telephone."
Everyone's trying to fill every available economic niche. What this does is it creates fragmentation. It creates a lot of complexity and a lot of diversity, which is healthy in an economy. Then the economy wants to sort out, which one of these diverse directions actually produces an economic benefit and actually fulfills a role in the ecosystem.
That's one of the forces, which sort of splits up the world. The force that kind of pulls the world together is consolidation. Consolidation is not opportunity-driven; it's cost-driven, which basically means that things like mergers and acquisitions are really mostly justified by things like cost. They're mostly justified by, "Gee, if we only had one of these things, then we could save money by shutting down the other one."
This is, I think, what creates the environment for SOA. It's why the environment for SOA continues. Even if you declare it to be dead, the kind of need for it continues to perpetuate because not only does this happen to all of of the enterprises in the world, where they're constantly being mashed together and then split apart again, but it's also happening at the levels of software. The software ecosystem is constantly being split-up to fill all available niches and then it's being smashed together by massive consolidation by the Oracles of the world.
The thing that I think is fascinating is if you look at it on those levels, the net result is continuous opportunity, continuous complexity, and a desire, in this economic climate, to actually identify sources of complexity, identify sources of risk, identify sources of cost, and you eliminate them. I don't want to paint a picture of the Terminator of the last few human survivors running around with these flying Terminators trying to kill them off.
DZone: How can the application developer adapt their thinking and approach to avoid the 'flying corporate Terminators'?
Miko: People in the development community are certainly concerned about being seen as a producer of cost and a producer of complexity. The SOA context for it is, that there are two pieces that I think are the kind of way that developers can try to fit in to the corporate or the enterprise objective. There's two ways that they can fit in as they're shifting their mentality into what I call true service orientation.
So, what is true service orientation? True service orientation has nothing to do with what protocol you choose to use. True service orientation is, it's an orientation to serve. So, the notion is, my goal is to produce value that can be consumed externally. This is related to the ability to provide agility.
The way that you ought to think about it is, how does my component participate in things like business processes? How does my component fit into the bigger picture? That's the real key issue. It's not just I want to satisfy my local environment, but I want to be useful to a number of different services.
I have a platform mentality. So what is a platform mentality? It means that my imagination should not limit the utilization of what I'm providing. It's like the way Google thinks about the Google API. This is that only the developers at the application-level can envision the number of different ways to recombine my capabilities.
Now, the thing that's funny about this is that you may think, as a developer, that you are the application. You are the application layer. But, suffice it to say, trends like BPM are starting to orchestrate on the top of existing applications. So, there may actually even be a second level of application developer that emerges at an abstraction layer above where you sit.
Instead of thinking of yourself as designing an entire car, think of yourself as designing an engine of a car and putting someone else in the driver's seat. That's true service. The concept of true service is the notion that the business is in the driver's seat, not the developer. So, it's a way of giving up some power, but it's giving up power to the people who provide you with your justification, your reason for existing.
So, the second thing that I want to touch upon is how you fit in to the larger context. People are looking to eliminate cost and complexity. The thing that I think is important to understand is that there's a holistic perspective on cost and complexity.
One of the aspects of cost and complexity is how many bugs do you generate? How much does it cost to generate new code? I know that this is a really tired paradigm, but I want you to think about reuse. And I want you to think about the notion of how much the organization benefits when you choose to unify pieces of code that you can reuse or utilize from a network service.
The reason why I want to encourage this is that the more there is this kind of reusability of code, the less redundancy there is. The thing that happens when you start to create redundancy is that these kinds of roving, flying terminator pods come by, and they can identify you as a threat or as a target. They can actually start dropping cluster bombs onto your project, because they just think like, well, your project is redundant. It's providing the same thing as this other project somewhere else.
The way you protect yourself from that is you try to ensure that you appear as a good corporate citizen. The good corporate citizen is truly service-oriented, which means it's very cognizant that there may be orchestrations or there may be utilizations of their application that are outside of their imagination. Someone else is plugging into it from above and orchestrating it into something completely novel. So, that's the platform mental model.
The other mental model is the reuse mental model, which I really want to encourage. I know that developers hate reuse because it's the big lie. Right? Not only is it the big lie, but reuse takes twice as long to execute. Because you have to study what the other guy did, and 90 percent of the time, it doesn't fit. You don't get the value that people tout.
I still want to encourage you to do so, because you have to keep in mind that these pods are flying. These SOI pods are looking for redundancy, and they're looking for these little potholes that they can wipe out. Because, it's all about consolidating costs these days.
DZone: Miko, that's an interesting analogy. There's this movement towards becoming increasingly Agile and improving time to market while meeting ever evolving customer requirements -- all this, without thinking about reuse and interoperability. How do you reconcile the need to be Agile, and to some extent, 'nimble', while thinking more holistically about the lifespan and broader scope of your application?
Miko: I think it's really a fairly complicated issue. What people are really focused on is looking at these local projects, but making sure that they're keeping in mind the bigger frame. You know, it's one of those cliches that comes out of the environmental movement, which is this notion of thinking globally and acting locally.
The thing I think that's happened in development and one of the things that's been exposed in extreme programming is this notion that you're using pair programming as a way of injecting externality. There are some people who claim that pair programming is about reducing operational risks. Right? It's the notion that if Joe gets hit by a bus, then the other guy can take over. It's a kind of risk-reduction technique.
The other way of looking at it is that it's a way of injecting externality. This basically means that two people have exposure to more of the understandings of the environment than one person. The notion would be that you're writing your code, and then the second person sitting next to you would be, "Hey, would you consider... I heard something from this guy over in the security group who said that we weren't supposed to be using X509 certs anymore."
The first programmer would go, "Oh, I didn't hear that. I didn't realize that was a requirement." And it wasn't even in the requirements doc. Like how did that escape that doc. The answer of course is that we're agile. We didn't over-specify this thing. We're really just trying to build a social network.
That is really the essential crux of what's happening. Development is increasingly becoming part of the social network. One of the things that's really essential about the nature of development is that there is an effort to rationally scale development. The tension here is the concept of scale. How do you make things productive on a larger scale? But, at the same time, how do you keep rationality in it? Those are the counter-balancing forces.
The tension there is that ideally you'd want one developer hiding away from society. But, the thing that turns out is that that model doesn't actually function anymore. It's no longer viable. The viable models start to scale up. You start to get to the scale of seven developers, or what I call the seven samurai model, and that's a change the world scale.
If you get a good team of seven developers together - you really interoperate, cooperate and create architectural integrity - they can really blow things up. Infrastructure like the cloud really supports that kind of model.
As soon as you get a bigger scale than that, which is almost the typical enterprise scenario, you start to run into this kind of tribal problem. You start to have platform wars. You start to have these really complicated geographic silos and all of these different tribal behaviors. That becomes a whole other scale of challenge. I see this allover big organizations. There's even tribalism across things like that system integrator tribes, regional tribes, central IT tribes and business units.
All of that stuff gets very, very exceedingly complex. It's stuff that, unfortunately, is just part of being human.
DZone: How does cloud computing augment my SOA? Would it be overly simplistic to say that SOA is now just cloud _in_ the enterprise?
Miko: Yeah. Unfortunately, that's a little bit simplistic. The thing that I think cloud represents from a developer perspective is another platform. You look at something like what an rPath is doing. They have a deployment platform that allows you to deploy onto multiple cloud providers, deploy out to a client, deploy onto different platforms - Linuxes and Unixes and all these different kind of environments.
The notion of cloud, from a development standpoint, is merging the fields of personal computing with what I call universal computing. You deal with your box and how you treat your box. But, really, in the network, it's a kind of cloud box. It's not just a single box.
The simplicity is there. That's the magic. Through the magic of virtualization and hosted, shared infrastructure services, you gain these tremendous economies of scale and the benefits of scale-up and scale-down operational dynamics. It lowers the cost of entry and the cost of failure, which is wonderful.
The thing that I think is transformative about the cloud proposition from a developer perspective is that you can justify experimentation more easily. You can say, "Hey, we'll just put this in the cloud." Anybody can access it. If it takes off, then it becomes self-justifying - the cost of infrastructure. If it doesn't take off, we haven't over-provisioned it. What will happen is that the cost is disappear, because it will trickle down, scale down and shut down.
So, what it does is it enables a developer environment where it costs less to fail. The decreasing cost of failure increases the opportunity for innovations, speciation, whatever specification, competition and the blooming of the proverbial 1000 flowers. I think, cloud is very exciting from that perspective.
Now, the thing that cloud does - and when I say cloud, it is yet another platform - so the thing that's really interesting about cloud is from the perspective of the enterprise. The enterprise looks at cloud from the consolidation perspective. If you look at the consolidation and competition dichotomy, the enterprise in the central IT unit thinks of cloud as being another source of complexity.
It can be flying overhead with the terminators trying to look for ways to shut it out. The last thing they want to do is have some person with a credit card go into an infinite loop, spawn 600,000 VMs in Amazon cloud and come up with an expense bill of $1,000,000 or something.
DZone: [laughs] Certainly not the best time for that, right?
Miko: Yeah. That would be a shock. That would be a big shock to the system. And frankly, Amazon would have a perfectly legitimate, billable... Obviously, it costs 10 cents per CPU hour, it's hard to rack up a bill that high. But, on the other hand, there are documented cases.
For example, start-up company SlugMug created this service in the cloud for rendering. They have a web-based rendering platform for images. This thing ended up self-replicating and spawning a whole bunch of instances and kind of going out of control. Fortunately, they were able to contain it.
They renamed this service SkyNet after the terminator self-aware orbital network of computers. The thing that was funny about that is that the renamed in SkyNet because they thought well, this thing has a life of its own. It's trying to take over. You know.
The thing that, to set in the SOA context, I think it's important to understand that while cloud represents an extremely exciting platform, it is considered potentially a threat to reliability. It's a threat to cost. It's a threat to the policy of the company, because it's kind of an escape hatch.
In a way, if you look at the way the Visa card represents an entry point for cloud for the developer, you can look at that as a way of smuggling capabilities of the organization and is another way of potentially even bypassing corporate IT. It's like, oh, it takes too long for them to provision cluster databases. If I go to AC2, I can just check the box that says "cluster" and I'm up. So, why waste my time with internal IT? I can just expense out my Visa card. That's a weird political dynamic.
I guess to put the cloud thing in context, and to try to wind up that conversation, I want to say that the developer perspective is not the only one. You want to make sure that you understand what the bigger perspective is on it, in case you are victim to the flying terminators.
DZone: Versioning is seriously understated or dealt with too late whereas if it was addressed early, many other things will fall into place. But versioning is tough - business processes change, services used by business processes change, the API wrappers around existing systems change. In a cloud environment, it just can get worse. Your thoughts on this challenge?
Miko: Yeah, absolutely. The reality is the following, which is that versioning is an octopus. Really, what happens is that every time you want "reuse," everyone wants something slightly different, and they need to version that flying service. It's typical of an architectural challenge to manage all of the different versions.
The thing that I think is critical is that a lot of the service paradigms out there have a mediation concept. You can actually use the mediation concept to virtualize a service interface. This at least gives you the flexibility. It doesn't tightly couple the two endpoints or provider consumers to a hard IP address. Because once you do need to migrate, in terms of versions, you want to be able to redirect people. You want to be able to transform that, et cetera. That's notion of the best practices.
The reality is that you will always struggle with octopus of divergent versions. In fact if you look at, you're going to struggle with the octopus business processes that are all subtle variations of the central theme of process, much like you see speciation happening in Darwin's finches in the Galapagos. You see variations on a theme. You're going to see 1000 variants of interface design. You're going to see 1000 variants of business process design and 1000 variants of schema or data design. So, versioning is an ever-present problem, and it's never going away.
DZone: The whole notion of building business processes, I guess, has always been pitched as the work of the business-savvy person and not necessarily that of a technical architect. Yet, it would appear that the tools and technologies are still geared primarily towards the technologist. What do you think is missing? What will it take to make business process modeling easier for the business-savvy person?
Miko: Yeah, I think part of it is implicit in the tooling. One of the things that I'm going to mention a little bit is that our organization, Software AG, is developing a cloud-based collaboration model for business process. The thing that's great about that is that we really want to lower what they call the infant mortality rate of BPM projects.
Because, one of the things that happens is that as things cross the blood-brain barrier between IT and business is that business says to me, "Wouldn't it be great if things worked like this?" They have to be really highly-connected and discoursed in IT reality, right?
Miko: It's kind of the feeling of - yeah, that would be great for you. But for me, that has nothing to do with what we have and would be this complete one-off. You'd have to build an entire instance of our ERP system to support your crazy idea.
So, what we've done with AlignSpace, which is our BPM in the cloud, is we've created a collaborative environment where someone could take a process idea, and they could share it with other people, including developers. There's this instant feeling of - by the way, here's why that won't work.
Now, I realize the concept of "here's why that won't work" is a kind of stick in the mud way of thinking about the world. But for us, we want to do the same thing that the cloud platform does for developers. We want to accelerate the rate of failure. We want to decrease the cost of failure. And we want to enable someone to fail and adapt on a continuous basis.
Because if you really want to talk about evolution, that's what evolution is doing constantly. It's constantly evolving and adapting, and it's constantly failing. In the process of creating vast diversity of activity, you can actually generate some innovations that stick. That's really what we're trying to do. We're trying to create this very fast, churning failure machine. We're trying to do so in a way that enables the business people to actually touch base with IT, and vice versa, about innovations and things like business processes.
DZone: Collaborative process modeling is the first capability you’re exposing in the beta version. What are some of the additional features we can expect to see?
Miko: The thing that we really see as being a direction for the AlignSpace capability is any kind of silo-straddling optimization. So, what does that mean? What I mean to say about that is that the entire ecosystem, as a function of consolidation, is cluttered with silos. What we want is a silo-straddling way to manage things like interface versions, to manage things like process versions, to manage things like any of these kinds of version controls - business rules - or any kind of declarative that can be produced in a SOA context. It is then able to be collaboratively versioned and managed.
Ultimately, what happens when you change anything in IT is that you impact others. The notion is that one person's perfect is another person's nightmare. What we want to ensure is that these connections can be maintained across high-level organizations. This is why we've designed the AlignSpace in the way that we have. It's a way of gluing together communities of interest that straddle our organizational, geographic platform silos - all these different kinds of silos that we talked about earlier.
DZone: Is it possible to use this platform to generate and export consumable process artifacts that can be used in BPM engines?
Miko: Yeah, exactly. The notion is that process becomes the first passage to begin with. It can actually suck in formats from popular BPMS environments, and then it can pump out formats that cue popular BPMS engines. It's designed to be platform controlled and neutral to the popular modeling tools that you have.
DZone: From the developer's perspective, over the next three to five years, how do you see all of these technologies consolidating? How will BPM fit into this emerging landscape comprised of cloud computing, software provisioning services and application virtualization?
Miko: Yeah. The thing that I think is really critical, just looking at it from a pure apps standpoint, is that you need to be cognizant of the potential of your application for it to be a self-infrastructure. In terms of the evolution of it, what you have to think about is you have to think about the evolution of a new layer of abstraction. People look at these new forms of BPM and BPMN. They think of all of these XML-based formats. They think about "business environments" and all of these kinds of things - business rules and stuff.
Developers tend to get a little irritated, because, it tends to be like just code. It's just code. Why build code on top of my code? That's just silliness, right?
But, if you look at evolution, DNA sequences are code, and protein sequences are code. There's code layered on top of code. There's an evolutionary reason for that increased layer of abstraction. I think it's important to recognize that, indeed, all of these things are just code. But, there is a function for taking certain kinds of code and doing what we call externalization.
The reason why externalization becomes important is, when you put into a declarative form like an XML assertion, what happens is that the ability to govern and change and the impacts of change across different silos improves. Because, you're manipulating it at a model level.
The thing that's important about the boundaries that we're crossing here is the model boundary. What does that mean, model boundary? It means that the model often tends to communicate different implications of changes in a system across to different, other people.
For example, something like a Java program doesn't do well, is it doesn't communicate the implications across organizational boundaries. Frankly a lot people in the business domain, when they look at something like a Java program, even if you can suck into your L-diagram, you still look at it and say, "I don't understand it. It's not my job. I can't get my head around what you're trying to do." It's that limitation of understanding.
That's why you're starting to see this difference that involves this declarative XML layer. Visualizing at that layer provides a higher level of abstraction, potentially more visualizations and simplicity and also the ability to orchestrate, collaborate and manage its impact and tendency to change. That's the crux of why layers evolve.
To be service-oriented, truly service-oriented, I think developers need to adapt. They need to understand that there's a reason for all of this. The reuse model sounds really stupid and painful. And as a practical matter, it is stupid and painful. The whole concept of having someone else building code on top of your code sounds extremely stupid and painful. But, there's another rationale that exists for that as well. I want to have people be receptive to some of these ideas and to try to fit them within the context of the bigger picture.
DZone: I hate to use the word 'job security', but I think part of what it means is that businesses are now looking at developers to think more holistically and strategically about what they're building. Ultimately, it goes back to Anne Manes' analogy and the fact that we're in an economic recession right now...
Miko: Absolutely, absolutely. I'm not going to pick on any geography, at all, because every single geography person has built a geography that will then cheapen on you.
If you think you're like, "Hey, I'm in India. My job is safe," don't bet on it. There are cheaper geographies that are evolving every day.
What does that mean? It means that if it's possible to disconnect your developer role from the network, from the social network... If it is possible to disconnect you from the social network, your job will be exported to a cheaper location. Right?
So, the point is that you hit the nail on the head. It is job security. It's job security to have a social context for your development to make sure that your development fits within the enterprise social network, the intentions of these external entities and that you are a good citizen. I realize that it sounds very grim, but you have to have these negative requirements in order to be a good citizen and not have your job sacked. You know, it's certainly top of mind of a lot people.
DZone: If you were to recommend two important skillsets a developer should invest in, say, over the next 12 to 24 months, what would those be?
Miko: I think that one of the big skill sets that will probably have to evolve is really starting to repersonalize yourself. I would really get into blogging and being a part of the social network of developers. This doesn't require justifying to your manager on going to expensive trainings, conferences and things like. You can really become a part of the conversation very cheaply. The car of entry is that you can do it on dial-up, right? You get onto the network and write yourself into a grant.
I think that's more and more important. You want to be irreplaceable. You want to be a source and a flow of value. If you actually plug into the creators of the value in the platform, you are more savvy and sensitive to the changes that are coming up in the platform.
You are much more in lock-step even with the ways that people talk about the dot-net platform, or the way that they talk about evolution, OSGI, these emerging technologies or whatever trend. You just listen to them and know that you're making investments in your knowledge that are going to pay off in the future. I think that's a big piece. You have to get into the social networking. That's a big skill set.
The second thing that I think is important is, if you have an office, go there. [laughs] Because, the serendipity of what you find by meeting people in the hallways is pretty important these days, and just spending the face time. One of the principles of adjunct development is face-to-face. This is something that we need to continue to value highly. I would say that those are the two things; the social networking both in cyberspace and in physical space.
DZone: Miko, thanks very much for your time, today.
Miko: Fantastic, much appreciated.
Opinions expressed by DZone contributors are their own.