The Real Impact of IT Owning Your SOA Effort
I had a great lunch discussion today regarding the impact of ownership of SOA within the business. As I have discussed multiple times, SOA is about business services, not technical services, but the reality of this point is lost in semantics. The following illustrates the different perspectives that each have and how that impacts the overall SOA initiative.
Let's look at two business capabilities in real estate: renting the building space (sales) and drawing up rental agreement (legal). Each of these business capabilities are effectively services to the business. In order to operate effectively, they need to share some basic data, such as property listings and renter information (CRM). They also operate synchronously as part of a business process that results in the execution of a rental of a property. Otherwise, they operate fairly autonomously from each other.
From the business' perspective, at a minimum, they'd like one and only one of each of these capabilities, each delivering in a consistent manner. Sometimes, that is not possible. Perhaps, there needs to be two distinct sales teams due to regulatory or licensing issues. In this case, the business will still want the multiple sales teams all operating in a consistent manner, using the same forms, if possible, and interacting with the legal department to draw up the contracts in a consistent manner. The consistency is what enables the business to operate efficiently even in the face of redundancy.
In this particular case, I believe, while not optimal, it is perfectly okay for there to be redundancy of certain information systems in the delivery of each of these business capabilities as long as it allows the business function to deliver consistently and efficiently. For example, if each of these groups had selected a different portal product to provide access to the property listings, that would be acceptable as long as they were operating off the same listings service. Forcing these groups to consolidate on a single portal product is not going to provide any additional efficiencies for these groups or simplify their ability to deliver.
However, what we see happening in the industry, is that SOA efforts are being led by IT and not by the business leaders. Hence, IT views the problem from a systems perspective and sees the need to remove redundancy at the information systems level. So, instead of focusing on business capabilities and making sure that the business rules, processes and goals are being executed consistently and that redundant business capabilities are consolidated, they look at the redundant portals and identify them as a target for consolidation, as so they should. ITs job is to keep IT costs down and are rewarded by simplifying the work of the IT department. However, as I used to remind my developers, it's not about what's easier to code, it's about making the jobs of the users easier, that's why they pay us.
The result of IT forcing consolidation of portal products is some cost savings, but interruption and change for the business. The overall productivity losses within the business function due to the interruption is typically never identified, let alone measured. Users will need to be re-trained on the new portal, things won't work initially and users will become frustrated and be less productive. All of this for the possibility of reducing the costs of managing one additional piece of software.
I say let sleeping dogs lie. There are plenty of real information management costs in most businesses that eliminating will offer a real benefit to the business' bottom line. Clearly, both of the aforementioned business capabilities need to be operating off the same customer and listing data. There's plenty of work to be done by IT without focusing on needless change. Eliminating the additional software licensing costs are penny-wise, pound foolish when the real impact is analyzed and assessed.
With regard to SOA, IT will view SOA as an IT effort that results in consolidation of technical capabilities. That's their task and that's how they are rewarded. IT is not rewarded for improving productivity in payroll by improving the end of month process. And, therein lies the flaw in the system. We have become heavily reliant upon our data and applications to run our business, and therefore, IT has become a critical function of the business. Yet, the business world rewards IT for running with as low of an overhead that can be comfortably be achieved without putting the business at significant risk. If we really expect SOA to live up to its promise, it is going to have to become an effort that is undertaken by the line of business as they reorganize for streamlined operations, or, if we're going to let IT own the SOA, then we are going to have reward IT for improving productivity and usability of the systems in a manner that is best for the users and not necessarily IT. That is, we're going to have to let them focus on actually aligning the information systems with the needs of the business and not force the business to live with the limitations of the systems.
Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
Thanks to @jadkins for the colorful ending to that cliched saying, which really got me thinking. Sometimes we just make certain aspects of architecture too difficult for no reason. Good design is important and can have significant financial impact if done incorrectly, but chances are the negative impact comes into play during the engineering effort, not during the architecture. To clarify my point, one must first understand what architecture is really focused on and how that differentiates from engineering.
For example, if we're designing a solution for financial services products. I can choose to take a framework approach to architecture in which we identify the common financial services components, such as securities and instruments and expose the manipulation of these components through the framework. Then we can continually add new algorithms to manipulate these same components in a modular fashion. Yet, another approach might be to examine things from the perspective of the buyer and the seller and allow each financial product to implement it's own representation of securities and instruments and orchestrate the process of selling and buying.
Either of the aforementioned scenarios is a viable architecture. In the former scenario, we can develop new computational models very quickly as long as we don't change the definition of the semantic elements. However, it can be a bit rigid since all new products must be able to define the components of its product based on these definitions. Also, sharing these products across trading partners becomes more complex.
In the latter scenario, we have a more service-oriented approach, focusing on the consumers and less on the definition of the products themselves. Hence, each service is unique, and may produce redundancy in defining the elements used to create a product, but we can orchestrate buying and selling across products in a common fashion. This approach facilitates incorporation of a much wider community since agreement is not required for the representation of elements to create a new derivatives product.
Which architectural approach is better? Well, as any good consultant will tell you it depends on your goals. If your goal is to allow your big brains to develop new models quickly to identify market making opportunities, then the first approach sounds ideal. If your goal is to enable a new derivative product to be treated as a single instrument even though it's comprised of securities that are bought and sold across a variety of exchanges, then the latter approach will probably suit your needs better than the former.
Neither scenario is optimal or guaranteed, but they both represent a means of simplifying the system enough to extend it over time without considerable re-work efforts.
When complete a design goes through an engineering effort where a software engineer will take this design and "press" it into software. During this effort, decisions are made about platform, programming language, external libraries, software development methodology, etc. Any one of these decisions, or a combination of them, may lead to the undoing of the great abstract architecture designed. For example, if the system cannot scale to the volume of financial services transactions, then that would be one major flaw. Using an open source library that ends up having a major security flaw is another potential flaw.
These scenarios are not very different than design and engineering in any other industry. The architects can design the most awesome car or impressive building, but if the engineers select the wrong parts or use inferior materials, the resulting product will never allow the architecture's brilliance to shine through. Note, manufacturers put significant effort into ensuring that their designs will work by building prototypes and testing against known environments that the product will have to operate under. Thus, from this perspective, architecture is responsible for putting forth a design that is credible before engineering commences.
The Busy Executive’s Service Oriented Architecture Reference Guide
Based on feedback from my last blog entry, "The Busy Executive's Quick Cloud Computing Reference Guide" the consensus was that people found it very helpful as a means of gaining quick understanding of what Cloud Computing is without getting mired in heaps of technology jargon and hype. So, I decided that SOA needed something very similar. After all, there's so much confusion and misunderstanding of SOA in the market right now.
Let's start with a basic understanding of a service. There are lots of things that you can call services, which is where many of the problems arise for SOA. I believe that the use of the word service in context of "Service Oriented Architecture" connotes a particular meaning that filters out most of non-applicable definitions of services. To further illustrate this meaning, I will discuss services in the context of software services so as to clarify the differences between SOA and not-SOA.
First, SOA is an architectural approach designed to identify and delineate business functional boundaries within a particular domain. A domain can be a single organizational unit or an entire industry. Regardless of the domain, if there are interactions within the domain, then there is most likely more than one unique business functions used to complete those interactions.
Many times, and more often with larger domains, there is more than one way to accomplish a particular task or objective. When a domain is required to support multiple paths to a singular end, it has been identified that the redundancy has an associated cost. Removing that redundancy results in efficiencies and cost savings. Very simply, SOA is the initiative designed around identifying the domain goal, marking the necessary business functions boundaries required to reach that goal, consolidating these business functions so that there is no redundancy and then orchestrating the process to reach the goal against the resulting set of business functions. The identified business functions are your services. Note, without these business functions you cannot complete your mission and achieve your task.
Most people agree on the following regarding desirable attributes of a service. We use them to do work for us that we don't want to do, we want them to be consistent and reliable, we want to pay a fair price to use them and we don't want to own them, and we don't expect to still be responsible for part of the job after hiring the service. If you hired a landscape service for your house that was supposed to prune your shrubs and you still had to go out on the weekends and clip the hedges, chances are you'd fire that service provider.
All the "chatter" in the media and on the Internet would have you believe SOA is very complex and requires expensive software and expensive resources when all we're doing is looking at a particular domain and figuring out how we can chop it up into a group of services that work together. For example, a landscape service and a mulching service have particularly good synergy, would you agree? These are two distinct services, but they have what we call a loosely-coupled relationship. That is, they work together for the duration of a task and then go their separate ways.
Granted, the task of looking at a domain and breaking it down into a set of services is a strategic skill that requires an experienced practitioner, but formulating the boundaries and putting the governance around those boundaries in place can be done by current management.
Now that we have these critical elements defined, let's compare three different topics of software that have all been associated or labeled SOA. Of the three, only one is truly SOA. These three topics are: Software-as-a-Service (SaaS), component-based software development and real SOA.
Why Isn't Software-as-a-Service a Service? To answer this, let's return to our agreed upon attributes of a service. SaaS meets one criteria we defined for a service—we want to pay a fair price to use them and we don't want to own them. However, most SaaS is just rented software. You're still responsible for integrating your data with the SaaS application after you rent it. You're still responsible for configuration of the forms and workflows after you rent it. For me, SaaS fails to meet the other key attributes of a service.
When many technologists and IT specialists discuss SOA, what they most often really mean is component-based software development. Many people in the IT industry have used the label SOA to describe a way to build software systems. This is possible, if the software represents critical business functions as we will explore shortly in the discussion of real SOA. However, in these conversations you will also hear things like SOA platforms and utility services. These are sure-fire indicators that the discussion has degraded into software design methodology and has left the realm of SOA.
Here's why there's no such thing as a "utility" service and it's just a software component. If the business itself would be less scalable or cease operation because the service is not available, then you have a service. If the mulching service doesn't show up when the landscaper is there, the landscaper cannot complete their task. Whereas, the utility service concept is a technological artifact of a software or solution design. The software providing the business service could exist without the existence of the utility service. The resulting solution may not be as modular of a design and, perhaps, not offer the service provider, the preferred level of agility and flexibility, but the service consumer would be no worse off than if you decided not to build a utility service, but instead, developed that business logic directly in to the software itself.
The last topic is SOA and real services that are comprised of software. The best way to describe this is to look at a couple examples of services that I have developed over the years.
The Alert Service – This is a software-based service that was used in data center management to notify personnel that a problem had occurred. A user or another piece of software could create an alert and it would send the notification via a number of mediums, such as pager, fax, phone, email and printed ticket. This software took care of the entire task, didn't have to be built for each individual usage and delivered the one and only one way to perform the task goal.
Zero Bond Calculator Service – This is a software-based service that provided various zero-coupon traders with a consistent means of quoting a zero coupon price. The service happened to be accessible via Excel, but could have been offered via phone or other mediums if necessary. The trader provided the inputs and the calculator service provided the pricing. It consolidated the various approaches to pricing used by each different trader, thus minimizing the overhead of the trading desk. Again, it took care of the entire task, didn't have to be built for each individual usage and delivered the one and only one way to perform the task goal.
In each of the aforementioned examples, the software represents a business function that was implemented using software because it was easily automated and reduced the need for human labor. However, because it was implemented in software, does not, in and of itself, make these services. It's the role they represent. These services allowed the business to reduce complexity, consolidate around one way of achieving a goal and do so in a way that was agile and flexible. Moreover, while they are not dependent upon one another to provide value to the business, one could easily see how they could work together synergistically to ensure high-availability of the zero coupon pricing calculator for traders.
That's SOA in a nutshell! Identify, consolidate, orchestrate and govern a set of business functions or capabilities.
JP’s Topsy-Turvy 2009 IT Year in Review
This time of year is a great time to reflect upon the last 12 months and see what has occurred as well as discuss the impact of the events on this year and next.
The Big News
The big news in IT this year has to be Cloud Computing. Like the result of a Gremlin fed after midnight, Cloud Computing turned into the Stripe of the IT world. As pundits and vendors raced to gain mindshare as a means of priming the economy-dulled sales pipeline, IT buyers were caught trying to figure out, "is this the next great frontier, or am I going to look like an ass for recommending yet another useless acquisition?" Interestingly, cost is not the major driver behind these offerings, but instead access to compute resources on demand that don't require up-front capital expenditure. Now, they only have to figure out how to actually make it so their applications can work in the Enterprise, but shift to the Cloud for additional resources when needed. This should lead to some interesting Fail Blog stories next year since most companies barely have the right mix of resources to keep the lights on let alone actually architect (Webster actually lists this as a verb now, so if jiggy is a valid word in the dictionary, I guess I have to accept architect as a verb now) a distributed application capable of crossing compute boundaries. I will be watching a number of fronts, such as application virtualization and Open Virtualization Format (OVF) to see if these do anything to help with this issue.
iPhones were a big IT story this year. No, not the fact that businesses started using them as a platform, but that so many individuals actually opt to use their own iPhone paid for with their own dollars and often no reimbursement for their service plan instead of a nice corporate provided Blackberry. What's next, people paying for their own health plans?
Of course, we can't talk about big IT news without discussing the impact of social networking. It seems a day doesn't go by that I get an invite to join a new network site or share data about where I am or where I'm travelling. The election of 2008 illustrated how digitally connected we all are as multiple candidates leveraged social networking to connect with voters and disseminate their ideas. In 2009, we began to see that impact start to seriously hit at the doorstep of IT. Gone are the days where Facebook and Twitter are blocked by the firewall, it's now a requirement for businesses to extend their brand to these platforms. Employees are encouraged to Tweet and extend their networks via LinkedIn and collaboration has gone from buzzword to system requirement. I can't read Arabic, but I think Al-qaeda even has a fan page now on Facebook (yes, that is facetious). The point being that social networking has changed the way we think about communicating and doing business, however, we need to educate users of these applications of the potential risks. These applications are ripe grounds for delivery of malware and phishing attacks; even if you don't do anything other than update your profile and don't limit who has access to view. Think about what's in your Facebook profile: birthdate, pet's name, spouse's name, link to spouse, etc. I'm sure everyone reading this has used some aspect of this information as the means to access another computer system; and this is only one example of thousands. So, when social networking, remember, protection!
Finally, business intelligence has re-appeared on the scene. BI has been on and off the radar screens of business multiple times over the past twenty years. Perhaps it was the lack of technology to really deliver useful intelligence from the globs of growing and poorly-manicured data or the fact that when things go south business leaders look to the great Oracle (pun intended) to tell them how to run their business. Either way, BI is back in style along with men showing chest hair and platform shoes. It's like the 70's on acid. Unfortunately, I have some interesting news, unless you actually spend the millions necessary to get your data house in order, all the BI tools in the world are not going to provide you with an accurate picture of how your business is trending. Besides, you only need to look in your bank account to determine that. I do believe BI has some interesting applications when partnered with Complex Event Processing engines as a way to tell you that all hell has broken loose or is about to anyway.
The Ongoing Battles
Another year has passed and over that year the same 15-20 people have come out and debated the relative goodness/badness of SOA and BPM. Meanwhile, all the people actually developing solutions totally ignored what we had to say and many of them laughed heartily at us as they ran past us – straight into that brick wall. Hello? McFly??? Anyone in there??? We're not here debating among ourselves for the sheer joy of it (although, it is really fun to watch @aleksb6 get his knickers in twist as @bmichelson pushes his buttons). We're debating these issues because we realize there's hundreds of millions of dollars at stake for businesses who are laboring to keep their users satisfied and don't have the ability to fully understand all the nuances of developing large-scale distributed applications in a virtualized universe. So, why does some junior architect with three weeks of training from IBM or Zapthink believe they are capable of taking on a task that industry veterans cannot agree on a common approach for and be successful?
SOA is undefined, but it's not a technology and is not a platform for developing applications, yet you'd better figure a way to get it on your resume if you want a job in IT in the 21st century. BPM is a methodology and a science, but it is not a toolkit or application. Both of these initiatives require experienced IT veterans with an understanding of enterprise capabilities, mission-critical deployment, all the –ilities (high availability, scalability, reliability, transactionality, etc.) and a comprehensive understanding of the business in order to properly align technology choices and directions for a successful outcome. This is a big picture thinker with the ability to drive that picture back down into attainable goals. We call them Enterprise Architects, you call them …. only when you're already in trouble!
The Emerging Disruptors
And, now, the fun part of our little journey through IT-land, a look at what's going to be hot or bite you in the butt next year!
Numero uno on the list of emerging disruptors has to be cybersecurity. A quick search into salaries for IT security personnel quickly tells the story of the state of affairs for this major league IT pain-in-the-rear-end. Most of you reading this probably already have a bot or malware running on a computer in your organization. These insidious beasts are being produced at a rate far faster than the software to recognize and eliminate them. Some of these are harmless, but some can be exposing critical flaws in your infrastructure that can be exploited by an individual with the appropriate skills facilitating access to your most confidential and private data. And, yet, the industry salaries for individuals with the capability to protect your networks and systems is below that of many other roles, such as Database Administrator (DBA), programmer, architect, virtualization engineer, storage engineer, etc. Obviously, protecting your information assets must be of minimal concern to you or you would be out seeking the skills of individuals or businesses that can provide you, minimally, with the knowledge that you have data leakage and, best case, close the holes and eliminate the problem.
It looks like Healthcare IT is shaping up to be a major land grab for software and hardware vendors next year. It seems that rising health costs and the government's emphasis on lowering costs via electronic health records has moved the IT target to this vertical. I have seen a number of major software vendors establishing Healthcare IT groups within their sales and marketing organizations in an effort to ramp up for what is deemed to be the next great sales frontier. Well I hope this report doesn't put a damper on those visions of grandeur. The report concludes that "health information technology has a modest impact on process measures of quality, but no impact on administrative efficiency or overall costs."
Here's one that my sources say to keep an eye on, IPv6. IPv6 is the next generation Internet Protocol. Many service providers will be switching to this standard soon because it provides them much more control over the number of Internet devices they can directly address and fixes many of the issues of IPv4. However, many of the IPv6 protocol stacks are new and have not been tested under full stress of a production environment. If you remember, it took Microsoft something like ten years to get their IP stack to a point where it was robust and mature. The IP stack is at the heart of every device that connects to the Internet. Changing this protocol is going to be slow and painful, but not changing soon is going to cause major disruptions in service as service providers are forced to use IPv6->IPv4 conversion, which at gigabit speeds is tedious and adds latency, not to mention that it will add overhead to the router for maintenance of multiple tables and increases memory consumption.
Desperately Seeking SOA Business Cases
The follow question was recently passed along to me by a peer:
"…There seems to be anecdotal evidence that moving a system to SOA, or creating a new system using that architecture is the right way to go but she was looking for something with numbers of metrics that she could use for her boss…"
This is a really important question because I believe that the person seeking this information is not alone in attempting to identify real value of investing in Service Oriented Architecture (SOA). The problem is that a properly done SOA should be unique to the mission, goals and processes of the organization making it of limited relative use to another organization. That is, SOA offers a framework for identifying, isolating, delivering and servicing a consumer need, and, while all businesses have some common aspects, the resulting business services should be unique to the needs of that business' consumers.
So, what then is the real value proposition for SOA? The answer to this question is the age-old consulting answer, "it depends!"
For one thing, SOA is so poorly-defined that just about any IT project can be labeled SOA, which means that anything you want to place an SOA label on that enhances productivity, reduces costs, increases profitability or otherwise has delivered value to your business. One of the yardsticks I use to measure an SOA by is that the effort has produced "product"—by product I mean a tangible and achievable asset has been defined—that consolidates execution of a business function within the business in one of two ways; a) a consistent process that may be replicated but is executed identically, or b) the function is executed by a single unit within the business. Note, in some discussion of SOA you may see this represented as a requirement for reuse, but reuse is not our goal as reuse implies a choice to use or not use. We are seeking consistency and, what I call the need to have something built once-and-only-once, ergo, no choice; the service must be designed in a way to support its consumer audience completely.
In (a) above, it's important to remain flexible in design. Some businesses are designed around their customer need, which means that a business function may need to be replicated due to privacy or confidentiality concerns. However, that does not mean that, where replicated, the function should not follow a consistent approach. The benefits and value of this to the business are significant. It allows employees to more easily be moved around, it reduces the costs associated with downstream processing and it simplifies the operations of the business for planning purposes. With regard to (b) above, that's fairly self-explanatory. By consolidating multiple, different approaches to delivering the same business function within the organization, e.g. receivables handling, it will result in lower expenses and overhead.
I believe the key take away from this blog entry is that you can explore how others have succeeded in applying the steps I have outlined above, but you will need to seriously consider what it will take for your organization to leverage SOA design concepts to deliver value to your business. Moreover, the first decision you need to make is what improvements or value propositions matter most to your business, and NOT what SOA metrics exist to justify moving in that direction. Or, to rephrase that famous Kennedy quote, "ask not what SOA can do for you, but ask what you can do to improve your business!"
The Reason Enterprise Architects Should Study Economics
Economics is the social science that studies the production, distribution, and consumption of goods and services and aims to explain how economies work and how economic agents interact. The really interesting thing about economics is that getting the right answer requires that you pose the right question. That is, if you choose to make conclusions based on casual observations of explicit cause/effect relationships, then your conclusions will seem logical and sound to most who will review them. However, only by ensuring that the cause/effect relationship holds up in the case where casual variables are changed, can you be sure that you have truly observed a phenomena and that the cause is accurate.
I've been reading Freakonomics (looking forward to Super Freakonomics when I'm done), and I'm astounded at how well Steven Levitt's approach to Economics could fit the needs of Enterprise Architecture. For example, in Freakonomics, Levitt uncovers the truth regarding many long-held beliefs, such as most drug dealers live at the poverty line or how little parenting actually impacts the outcome of a child. What helped to uncover these facts was asking the right question and identifying which variables in the equation needed to remain static to ensure a valid outcome. In many of the cases, Levitt undermines what many other Economists, experts and pundits before him rolled out as proven facts and only due to his keen mind and his approach to formulating the problem domain was he able to uncover that which his peers could not. To keep us focused, I won't discuss here how he discovered why violent crime rates dropped in the 90's, but needless to say well-held beliefs as to why were shredded by Levitt's innate ability to target just the right variables and "run the numbers."
If we apply Freakonomics-type analysis to current information technology markets, such as Cloud Computing, based on experts and pundits one would readily equate that: a) public Cloud Computing models offer a more cost-effective way to acquire computing resources; b) it's better to rent software than to own it and; c) in the future, all computing will be paid for based on usage versus ownership. Similarly, if you read the headlines of many tech publications, you would believe that: a) everyone is doing SOA; b) there are many successful SOA deployments and; c) SOA is the only approach to developing systems that you should consider. None of these conclusions are true in all cases, maybe in no cases, and, in fact, the real numbers would probably tell a very different story [I'm leaving this as an exercise to the reader since I want to focus this piece on the approach to EA, not the application of analysis to these issues]. These statements are being made arbitrarily based on extremely small sample sizes and are as valid as looking around you at a large business gathering noting all the iPhones and Blackberries, and concluding that iPhone & Blackberry must outsell all other phones, when in fact, their combined market share is a mere 3.0-3.5% of the overall mobile phone market.
In actuality, I see a lot of IT decisions being made using what, seemingly, is no more exploration than was used to determine that iPhones & Blackberries have a leading market share. It is the Enterprise Architect's role to ensure that the selection and approaches toward development of systems are sound relative to their business, not just other businesses. Moreover, where decisions are based on the work of other businesses' success, those successes need to be properly vetted to ensure that there is enough commonality between efforts, such that you could make the leap that your business will see similar results. Finally, assumptions and theories need to be tested by properly identifying the variables that need to remain static and then comparing; in essence normalizing the question to be homogenous in all situations. For example, in Freakonomics, Levitt points out that in order to prove that the numbers behind why violent crimes dropped in the 90's he needed to review similar trends of violent crimes in areas that had not made the purported changes that led to the large unanticipated decrease. When the trends were normalized to remove noise and anomalies, it was shown that those factors seemed to have relatively little impact on violent crime decrease.
This brings us to the all important question, "how do we demonstrate the value of Enterprise Architecture?" Unfortunately, you cannot demonstrate the value of Enterprise Architecture if you cannot monetize or enumerate the value of all possible choices relative to the choices that are being recommended or those that have been made. Moreover, it's critical that these analyses are carried out over enough time that short-term wins don't supersede long-term potential gains. Thus, it is here that Enterprise Architects, especially those we call Chief Architects, truly show their mettle. It is their experience, coupled with the ability to focus on the right set of variables, understanding the impact of change of those variables and being able to communicate that in a way that allows the business to make effective business decisions, which sets top notch practioners apart from Sr. Software Engineers that the organization placated with a title to keep them happy so they wouldn't leave.
IT Business Edge’s Loraine Lawson Interviews Me On SOA & EA
A couple of weeks ago I did an interview with Loraine Lawson (@lowrain) who covers integration technology for IT Business Edge. There were two different aspects to the interview, one focused on my blog entry "Perhaps SOA is More Strategy Than Architecture" and the other my beliefs on Enterprise Architecture and why I still believe strongly that EA holds great value for business and that business should not be deterred by past missteps due to ineffective applications of EA.
Part I entitled, " Is Strategy the Key Characteristic of SOA?" is located here.
Part II entitled, "In Defense of the Enterprise Architect", is located here.
My Prediction of Cloud & SOA in 2005
Okay, maybe it's petty and I'm just tooting my own horn, but I found this old article I wrote for Upstream CIO's October issue (written in July '05). In rereading this article today, I surprised myself how aware I was of the forthcoming Cloud & SOA convergence.
The popularity of Service-Oriented Architecture (SOA) is gaining ground on the heels of the success of eXtensible Mark-up Language (XML) as a means for representing data in a technology-neutral manner and its ability to move data between applications. However, SOA plays a more important role than merely a moniker to capture the use of applications that communicate using XML. SOA is the software world's entry into the utility computing model.
For CIOs, the utility computing model has come to be synonymous with the ability to provision computing resources on demand, thus maximizing these resources based upon needs of the organization and its customers. In some cases, the utility computing model has begun to incorporate metering, which allows IT organizations to charge back usage to individual departments or projects, thus enabling the CIO to demonstrate the need for the current resources and to plan properly for capital expenditures of new resources.
Moving to a utility computing model helps organizations consolidate computing resources, which results in lower costs of ownership and management of those resources. However, due to today's inefficient software designs, most computing resources are dedicated to a single function, such as Web applications or enterprise resource planning. Such dedicated resource allocation means you cannot re-provision them to other tasks when the demand hits. Instead, IT departments are forced to acquire enough computing resources to satisfy each function running at 100%. This results in organizations overspending on computing resources to compensate for this limitation imposed by the software. Thus, today's application models are undermining consolidation efforts and limiting the cost savings to the organization.
Enter SOA, which offers a software architecture that maps more closely to the utility computing models that have already been adopted, allowing better allocation of resources and greater provisioning controls. SOA enables organizations to develop and, more importantly, deploy their software in a loosely coupled fashion. This means the software is designed as a set of black boxes that have well-defined inputs and outputs, such that they can be linked together in a process flow with ease.
SOA operates on the concept of service providers and consumers who have agreed upon service contracts. These contracts are based on well-defined messages that are passed between the provider and the consumer, but at an abstract level, they are no different than the agreements that you have with any other utility, such as the phone or electric company.Indeed, supporting these contracts requires the same service level agreements that would be expected of mission-critical utilities.
While there is no concrete definition of SOA today, the generally agreed-upon attributes of SOA include being:
- Based on technology-neutral standards, such as XML and Web Services;
- Self-describing through metadata files created using the Web Services Definition
Language (WSDL); - Discoverable through a URL mechanism, which means they use common Web
mechanics for implementation; - Stateless, which means they can easily be reused since they are not reliant upon
specific resources being dedicated to them while they are in use.
These attributes lead to the creation of software that can be provisioned and allocated to a resource on demand, keeping in line with the tenets of utility computing. Moreover, the concept of providing a service means that computing power can be made available like other utility-based services, such as telecommunications and electricity, which have the ability to direct more or less service based on need.
Hence, this change in architecture leads to a fundamental change in the way software is designed, but also in the way that software is deployed and managed. For example, once a service is developed, it needs to be deployed into an infrastructure that is operationally managed. This means the service may require access controls, redundancy, quality-of-service, service level agreements and root cause analysis when it is not available. These are the same services one would expect of any utility-based service.
In keeping with this model, then, SOA maps perfectly into existing utility models, such that IT can charge back infrastructure usage based on its existing utility metering, adding on charges for the number of times the service is invoked. Additionally, IT has more control over the resources used by these services, such that a single service will not require a significant allocation of a particular resource, such as today's application servers or SAP servers require.
Thus, the organization can deploy services where there are available resources that fit the demand for that service. Moreover, this change will not impact existing applications if they use the model of "find and bind" that has become a cornerstone of SOA. That is, users can look up the location of a service in a registry and dynamically bind to and use that service without any prior knowledge of its existence.
This movement toward SOA requires a fundamental shift in the way that IT operates within most organizations today regarding software. Today, most IT organizations follow a system integrator model of delivering software, which means they take existing off-the-shelf applications and/or programming tools and deliver an automated solution to a business problem, similar to the service provided by Accenture or BearingPoint.
The move to utility computing using SOA requires the IT organization to operate more like the electric or phone company for their organization. The IT department may still do development work, but it will mostly be responsible for creating new services. Eventually, business tools will create these services. Thus, IT will become the organization responsible for the infrastructure for provisioning resources for these services based on demand, with the goal of no interruption in service.
This is a major change for many IT organizations that are managing silos of applications today. If one application that supports fifty users is unavailable, IT can expect five to ten calls to the help desk. However, if a service that supports fifty applications is unresponsive, the help desk can expect to receive ten times that many calls. Needless to say, the impact of this paradigm change to the organization and the IT resources is significant from both a cost and resource perspective.
The utility computing model can save organizations millions of dollars each year
through hardware and human resource consolidation. By adopting SOA, organizations can incorporate a software model that easily maps onto the utility computing model, such that resources do not need to be dedicated to particular applications, but instead can be provisioned based on demand within the organization. This approach requires the organization to plan for changes in how the IT department operates and the types of skills required to build and support an effective utility computing model.
The Role of The Business Analyst in an SOA World
Those of us that are part of SOA-related projects where traditional business analysts (BA) are involved often find ourselves frustrated by the incongruence between the analyst’s approach to requirements gathering and the SOA design. The problem arises because SOA models functionality of a business across multiple boundaries, whereas the business analyst wants to focus on a user’s needs. One focus is tactical and the other is strategic. However, more than this key difference, certain aspects of the tactical requirements overlap with the strategic requirements; specifically with regard to service boundaries. Thus, important business rules that are relevant to the definition of a particular service are buried among hundreds of irrelevant (to the SOA goals) requirements and the SOA architect is forced to mine these requirements like a miner mining for gold.
In figuring that someone else must have written on this topic, I visited my favorite Web search engine and pulled up the following article: “The new business analyst talks SOA”. I was surprised to find this content on Network World given the nature of the topic, but quickly realized that the author thought of SOA as an application architecture instead of an aspect of Enterprise Architecture. So, it was not much more of a surprise when they recommended that business analysts needed to become more savvy in SOA by learning BPEL, SCA and SDO; perhaps the worst advice I have ever seen regarding this topic. The last thing we want are the business analysts talking about SOA, let alone SODA aspects of SOA.
Still, the article had one point correct, business analysts had better be reading the signs and acclimating themselves to a different type of conversation with their users. Moreover, I believe SOA is a game changer in the way that we collect requirements going forward. For businesses that really want to be successful with SOA, the first conversations that need to be had are with the key business stakeholders, and an Enterprise Architect with SOA experience must be present and facilitating these discussions. This will be the basis for formulating a service model, which will then guide all future system designs. I would also engage a business process engineer to start analysis of business processes that directly impact the effort.
Ultimately, if a BA is able to make the transition to SOA mindset, then they might be a good candidate to drill down and analyze the requirements for the service contract and vocabulary for each service. Otherwise, an SOA architect should really handle these efforts. Which begs the question, what is the role of the BA in an SOA world? I believe there is a need for really good subject matter experts that can work with the Enterprise and SOA architects to fully define the interactions between business functions and, perhaps, to capture the usability models. However, even in the case of usability, a usability expert will offer more insight into how to present information to the user than the BA.
Overall, my analysis is that in an SOA universe, the role of the BA is greatly diminished unless they can adapt to the change in role and gain an understanding of how to facilitate requirements gathering in a much more compartmentalized manner as outlined by the SOA architect and the business process engineer.
The SOA (& Cloud) Funding Dilemma
Lately, and primarily among government users, I've been hearing about a potential clash of bureaucracy meets technology when it comes to funding shared services. It seems that the current government procurement and funding processes do not favor strategic sharing of software services as an outgrowth of a single development effort. I imagine that this issue might also arise in some large organizations where the business units are funding development using a self-funded IT organization, but I have yet to hear any specific stories coming from the private sector regarding this issue.
While there seems to be a number of workarounds to this problem, the basic issue still remains. If one government customer funds the development of an application out of their budget, and in developing that application, it is decided that one or more services could be of use to other government customers, there may be budgetary concerns for actually developing, hosting and maintaining those services for those that are not the original funding customer. Moreover, it becomes questionable how to go about adding features to those services when the requests are not coming from the original funding customer.
With all these government agencies pushing SOA in their IT strategies, one has to wonder, what exactly are they asking for? If the goal is not to remove redundancy within a single department and within a single agency--we're not even focusing on cross-agency here--as a means of reducing costs and providing a single means of delivering a business function in a non-ambiguous manner, then what is the goal?
From my own personal experiences, I believe certain agencies do desire to achieve the benefits of SOA, while others are simply seeking the benefits of component-based architecture, which is flexibility and agility in application design to lower costs of maintenance. However, due to "management by magazine" approaches in IT coupled with the "slap SOA on everything" mandates, agencies that are capable of only benefiting from component-based architecture--because of the procurement and funding issues or just basic needs--are now requesting SOA and ending up spending a lot more to develop a simple application than they should have in the first place and getting a lot more complexity and reliance on middleware infrastructure that is, at best, using a sledgehammer to kill a flea.
Now, most of this entry discusses the issue with regard to SOA, since this is where the problem has really become apparent recently. However, it's not too far of a leap to envision that Cloud Computing, that bastion of shared compute power, is going to have similar and more complex hurdles to overcome with regard to maintenance and operations of applications running in the Cloud. At the end of the day, someone responsible for budgeting today is going to have to figure out how much is required to support and maintain one of their applications next year. That may not be too difficult a task when they are the only group consuming that application, but the budgeting process is far more difficult when that agency accidentally becomes a SaaS provider to other agencies and departments interested in using that application.
Got any private or public sector stories regarding how the budgeting and procurement processes are hindering the strategic value SOA? Please share them with me as I am really interested in understanding the full boundaries of this issue.
Disclaimer
My Most Recent Book
:Other Links
Web Profiles
Recent Posts
- Cloud Computing Pragmatics
- The Real Impact of IT Owning Your SOA Effort
- Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
- The Busy Executive’s Service Oriented Architecture Reference Guide
- The Busy Executive’s Quick Cloud Computing Reference Guide
Tags
Recent Comments
- Steve Hughes on Cloud Computing Pragmatics
- CloudSigma on Cloud Computing Pragmatics
- Is ‘Service-Oriented Approach’ a Better Way to Define SOA? | SOA Governance - Service Oriented Architecture - SOA Business - SOA Design - SOA Services - SOA Software - SOA Solutions - SOA Security - SOA Web Service on Perhaps SOA is More Strategy Than Architecture
- Jaime Ramirez on The Busy Executive’s Quick Cloud Computing Reference Guide
- gon zalez on The Busy Executive’s Quick Cloud Computing Reference Guide
Blogroll
Other Links
Archives
- July 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- March 2009
- February 2009
- January 2009
- September 2008
- August 2008
- May 2008
- December 2007
- November 2006
- April 2005
