Cloud Computing Pragmatics
In October of 2009 I was interviewed by GovIT Journal and in that article I presented my view that Cloud Computing is highly dependent upon the network. The actual quote given was, “Which just goes to show, the telco providers still hold all this stuff by the balls!” More than ever, based on my work over the past four months as Merlin International’s Chief Architect, I still believe this is a critical and pertinent factor regardless of your Cloud Computing architecture.
Indeed, I have relished these past few months because they have presented me with the opportunity to delve deep into the muscle tissue of Cloud Computing. One of Merlin’s key areas of success has been in providing networking and data center hardware and software. While many architects can talk a good game about Cloud Computing, few have actually walked the stack top to bottom and actually touched the underbelly of the beast. Shoot, I even became a Riverbed Certified Solution Professional, a wickedly-cool WAN optimization product and am now focusing on Network Appliance certifications next. Understanding these “organs” of the Cloud truly provide unmatched insight into what is achievable and what is hype.
Meanwhile, I’ve been deep in muck gaining real insight into what Federal government customers are dealing with in trying to provide agile infrastructures to support the growing and changing needs of their user base. It’s real easy for pundits to step up and present a vision for Cloud Computing as a configurable resource that’s capable of meeting all needs, but I really believe that is a misnomer. In fact, more than ever I believe that we need to specialize Clouds to support a specific purpose. For example, I advocate that users need separate Cloud Computing infrastructures to support their full-motion video needs and their back office applications and that these should not live on the same Cloud infrastructure; especially if utilizing multicast video capabilities.
Vendors spewing forth mumbo-jumbo about creating a group of virtual machines and deploying them in an automated fashion seem to be heavily focused on new and simplistic database and business applications. Anyone doing heavy lifting on their network, dealing with saturated WAN egress points and leveraging legacy applications know that this is a pipe dream designed for the R&D lab. Putting together a Cloud Computing architecture requires a solid Enterprise Architecture effort in which the AS-IS and TO-BE architectures are fully understood and documented and there a roadmap that describes how to move from one to the other inclusive of details, such as security, auditing, monitoring, utilization, tuning, etc.
Oh, and let’s talk about security. I’m going to be putting forth an entire blog entry shortly on the real issues with defending against cyber threats, but, needless to say, building out a Cloud Computing solution before you even implement a single sign-on solution and identity management program is a recipe for redundancy and increased overhead at a minimum and breach in the worst case.
In this article, Hord Tipton, executive director of (ISC)2, the International Information Systems Security Certification Consortium, and ex-CIO of the Interior Department and Bureau of Land Management, makes an important point that security must be baked in from the start. I couldn’t agree more. Putting a few Intrusion Prevention Systems (IPS) or Data Loss Prevention (DLP) tools as the egress/ingress points cannot stop one of the biggest security flaws we have because they are designed into the core of the applications that manage and control the data. The move to the Cloud only exacerbates these problems. However, I will admit the move to Cloud offers a tremendous opportunity to deliver security as a service and then port the applications to the new architecture to minimize the risk of potential breaches.
One thing is for sure, this Cloud puzzle is large and offers a great opportunity for efficiencies. However, we need to be pragmatic in our approach or we risk exacerbating current problems instead of solving them.
Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
Thanks to @jadkins for the colorful ending to that cliched saying, which really got me thinking. Sometimes we just make certain aspects of architecture too difficult for no reason. Good design is important and can have significant financial impact if done incorrectly, but chances are the negative impact comes into play during the engineering effort, not during the architecture. To clarify my point, one must first understand what architecture is really focused on and how that differentiates from engineering.
For example, if we're designing a solution for financial services products. I can choose to take a framework approach to architecture in which we identify the common financial services components, such as securities and instruments and expose the manipulation of these components through the framework. Then we can continually add new algorithms to manipulate these same components in a modular fashion. Yet, another approach might be to examine things from the perspective of the buyer and the seller and allow each financial product to implement it's own representation of securities and instruments and orchestrate the process of selling and buying.
Either of the aforementioned scenarios is a viable architecture. In the former scenario, we can develop new computational models very quickly as long as we don't change the definition of the semantic elements. However, it can be a bit rigid since all new products must be able to define the components of its product based on these definitions. Also, sharing these products across trading partners becomes more complex.
In the latter scenario, we have a more service-oriented approach, focusing on the consumers and less on the definition of the products themselves. Hence, each service is unique, and may produce redundancy in defining the elements used to create a product, but we can orchestrate buying and selling across products in a common fashion. This approach facilitates incorporation of a much wider community since agreement is not required for the representation of elements to create a new derivatives product.
Which architectural approach is better? Well, as any good consultant will tell you it depends on your goals. If your goal is to allow your big brains to develop new models quickly to identify market making opportunities, then the first approach sounds ideal. If your goal is to enable a new derivative product to be treated as a single instrument even though it's comprised of securities that are bought and sold across a variety of exchanges, then the latter approach will probably suit your needs better than the former.
Neither scenario is optimal or guaranteed, but they both represent a means of simplifying the system enough to extend it over time without considerable re-work efforts.
When complete a design goes through an engineering effort where a software engineer will take this design and "press" it into software. During this effort, decisions are made about platform, programming language, external libraries, software development methodology, etc. Any one of these decisions, or a combination of them, may lead to the undoing of the great abstract architecture designed. For example, if the system cannot scale to the volume of financial services transactions, then that would be one major flaw. Using an open source library that ends up having a major security flaw is another potential flaw.
These scenarios are not very different than design and engineering in any other industry. The architects can design the most awesome car or impressive building, but if the engineers select the wrong parts or use inferior materials, the resulting product will never allow the architecture's brilliance to shine through. Note, manufacturers put significant effort into ensuring that their designs will work by building prototypes and testing against known environments that the product will have to operate under. Thus, from this perspective, architecture is responsible for putting forth a design that is credible before engineering commences.
JP’s Topsy-Turvy 2009 IT Year in Review
This time of year is a great time to reflect upon the last 12 months and see what has occurred as well as discuss the impact of the events on this year and next.
The Big News
The big news in IT this year has to be Cloud Computing. Like the result of a Gremlin fed after midnight, Cloud Computing turned into the Stripe of the IT world. As pundits and vendors raced to gain mindshare as a means of priming the economy-dulled sales pipeline, IT buyers were caught trying to figure out, "is this the next great frontier, or am I going to look like an ass for recommending yet another useless acquisition?" Interestingly, cost is not the major driver behind these offerings, but instead access to compute resources on demand that don't require up-front capital expenditure. Now, they only have to figure out how to actually make it so their applications can work in the Enterprise, but shift to the Cloud for additional resources when needed. This should lead to some interesting Fail Blog stories next year since most companies barely have the right mix of resources to keep the lights on let alone actually architect (Webster actually lists this as a verb now, so if jiggy is a valid word in the dictionary, I guess I have to accept architect as a verb now) a distributed application capable of crossing compute boundaries. I will be watching a number of fronts, such as application virtualization and Open Virtualization Format (OVF) to see if these do anything to help with this issue.
iPhones were a big IT story this year. No, not the fact that businesses started using them as a platform, but that so many individuals actually opt to use their own iPhone paid for with their own dollars and often no reimbursement for their service plan instead of a nice corporate provided Blackberry. What's next, people paying for their own health plans?
Of course, we can't talk about big IT news without discussing the impact of social networking. It seems a day doesn't go by that I get an invite to join a new network site or share data about where I am or where I'm travelling. The election of 2008 illustrated how digitally connected we all are as multiple candidates leveraged social networking to connect with voters and disseminate their ideas. In 2009, we began to see that impact start to seriously hit at the doorstep of IT. Gone are the days where Facebook and Twitter are blocked by the firewall, it's now a requirement for businesses to extend their brand to these platforms. Employees are encouraged to Tweet and extend their networks via LinkedIn and collaboration has gone from buzzword to system requirement. I can't read Arabic, but I think Al-qaeda even has a fan page now on Facebook (yes, that is facetious). The point being that social networking has changed the way we think about communicating and doing business, however, we need to educate users of these applications of the potential risks. These applications are ripe grounds for delivery of malware and phishing attacks; even if you don't do anything other than update your profile and don't limit who has access to view. Think about what's in your Facebook profile: birthdate, pet's name, spouse's name, link to spouse, etc. I'm sure everyone reading this has used some aspect of this information as the means to access another computer system; and this is only one example of thousands. So, when social networking, remember, protection!
Finally, business intelligence has re-appeared on the scene. BI has been on and off the radar screens of business multiple times over the past twenty years. Perhaps it was the lack of technology to really deliver useful intelligence from the globs of growing and poorly-manicured data or the fact that when things go south business leaders look to the great Oracle (pun intended) to tell them how to run their business. Either way, BI is back in style along with men showing chest hair and platform shoes. It's like the 70's on acid. Unfortunately, I have some interesting news, unless you actually spend the millions necessary to get your data house in order, all the BI tools in the world are not going to provide you with an accurate picture of how your business is trending. Besides, you only need to look in your bank account to determine that. I do believe BI has some interesting applications when partnered with Complex Event Processing engines as a way to tell you that all hell has broken loose or is about to anyway.
The Ongoing Battles
Another year has passed and over that year the same 15-20 people have come out and debated the relative goodness/badness of SOA and BPM. Meanwhile, all the people actually developing solutions totally ignored what we had to say and many of them laughed heartily at us as they ran past us – straight into that brick wall. Hello? McFly??? Anyone in there??? We're not here debating among ourselves for the sheer joy of it (although, it is really fun to watch @aleksb6 get his knickers in twist as @bmichelson pushes his buttons). We're debating these issues because we realize there's hundreds of millions of dollars at stake for businesses who are laboring to keep their users satisfied and don't have the ability to fully understand all the nuances of developing large-scale distributed applications in a virtualized universe. So, why does some junior architect with three weeks of training from IBM or Zapthink believe they are capable of taking on a task that industry veterans cannot agree on a common approach for and be successful?
SOA is undefined, but it's not a technology and is not a platform for developing applications, yet you'd better figure a way to get it on your resume if you want a job in IT in the 21st century. BPM is a methodology and a science, but it is not a toolkit or application. Both of these initiatives require experienced IT veterans with an understanding of enterprise capabilities, mission-critical deployment, all the –ilities (high availability, scalability, reliability, transactionality, etc.) and a comprehensive understanding of the business in order to properly align technology choices and directions for a successful outcome. This is a big picture thinker with the ability to drive that picture back down into attainable goals. We call them Enterprise Architects, you call them …. only when you're already in trouble!
The Emerging Disruptors
And, now, the fun part of our little journey through IT-land, a look at what's going to be hot or bite you in the butt next year!
Numero uno on the list of emerging disruptors has to be cybersecurity. A quick search into salaries for IT security personnel quickly tells the story of the state of affairs for this major league IT pain-in-the-rear-end. Most of you reading this probably already have a bot or malware running on a computer in your organization. These insidious beasts are being produced at a rate far faster than the software to recognize and eliminate them. Some of these are harmless, but some can be exposing critical flaws in your infrastructure that can be exploited by an individual with the appropriate skills facilitating access to your most confidential and private data. And, yet, the industry salaries for individuals with the capability to protect your networks and systems is below that of many other roles, such as Database Administrator (DBA), programmer, architect, virtualization engineer, storage engineer, etc. Obviously, protecting your information assets must be of minimal concern to you or you would be out seeking the skills of individuals or businesses that can provide you, minimally, with the knowledge that you have data leakage and, best case, close the holes and eliminate the problem.
It looks like Healthcare IT is shaping up to be a major land grab for software and hardware vendors next year. It seems that rising health costs and the government's emphasis on lowering costs via electronic health records has moved the IT target to this vertical. I have seen a number of major software vendors establishing Healthcare IT groups within their sales and marketing organizations in an effort to ramp up for what is deemed to be the next great sales frontier. Well I hope this report doesn't put a damper on those visions of grandeur. The report concludes that "health information technology has a modest impact on process measures of quality, but no impact on administrative efficiency or overall costs."
Here's one that my sources say to keep an eye on, IPv6. IPv6 is the next generation Internet Protocol. Many service providers will be switching to this standard soon because it provides them much more control over the number of Internet devices they can directly address and fixes many of the issues of IPv4. However, many of the IPv6 protocol stacks are new and have not been tested under full stress of a production environment. If you remember, it took Microsoft something like ten years to get their IP stack to a point where it was robust and mature. The IP stack is at the heart of every device that connects to the Internet. Changing this protocol is going to be slow and painful, but not changing soon is going to cause major disruptions in service as service providers are forced to use IPv6->IPv4 conversion, which at gigabit speeds is tedious and adds latency, not to mention that it will add overhead to the router for maintenance of multiple tables and increases memory consumption.
Metering and Instrumentation: Two Critical and Oft Forgotten Features of Cloud Services
An interesting facet of advancement is that it tends to limit know-how over time. At one time the mechanic in your local garage could take your entire engine apart, fix any problem and put it back into running order. Today, they can put a computer on the end, read a code, and hope it provides some understanding of the problem. Similarly, the application server revolution has unfortunately led to a generation of software developers that believe that the container can do all your instrumentation and metering for you, thus removing an impetus to build this into the application. These developers lose sight that the container can only tell you part of the story; the part it sees as an observer. If you don't make more internals of your application observable, the most it can see is how often it hands off control to your application, what services your application uses from the container and when you application terminates control. Useful information, but not enough to develop large-scale real world services used by hundreds of thousands, or even millions of users.
I ran across a product the other day doing some research that I found very interesting. It was a service bus that had built in metering.
Putting the metering into the container is one way around the problem of every applications metering for itself, however, how many people are going to trust their services to this product company? Additionally, it would be very difficult to migrate this service in the future unless the metering interface was standardized or replicated in another hosting container. So, here's a place where standardization of service offerings in service containers would be a very helpful feature; and something that needs to be the focus of Cloud interoperability.
Additionally, proper inclusion of metering and instrumentation interfaces into you Cloud service means that you have greater control over the granularity of the data produced through these interfaces. Most metering solutions I've seen today are very coarse-grained and often focused on infrastructure usage, such as cost per CPU cycle or gigabyte. What if you wanted to produce a service that had tiered pricing? This would require that those service operations provide the details of each transaction to a metering service.
Instrumentation is even more complex to manage than metering as it has the potential to introduce processing latency. If you capture too much instrumentation information then the service will not be responsive, but if you don't capture enough, you may not be able to determine a pending outage until it occurs. I have seen first-hand the problems of poorly-designed instrumentation in a high-volume service.
In my first-hand experience, traditional system network management tools were used to watch the application performance including CPU, memory and storage, but were unable to determine that a process was out of control and eating up storage space quickly enough to shut it down before it completely brought down all services sharing that volume. In an elastic Cloud environment, where dynamic storage is configured to come online to minimize outages a process like that could eat up a large percentage of your storage array before you had the chance to unwind it. This was made even more complex by the fact that the service provider was unaware of the typical size of a transaction from each of its customers, so it could not assume a particular quota since that may limit a legal request.
These are real problems that could have been mitigated by an appropriate level of metering and instrumentation built into the services themselves. These sub-services could have inferred average transaction sizes and average storage usage and identified that a particular process was out of range in enough time for a human to get involved and stop the runaway process.
It's interesting to see so much attention being paid to security in Cloud Computing since this was one of the most often ignored components for those implementing SOA-based designs. It seems that the immaturity of a particular technological direction is easily identified by what aspects of a robust, mature solution are ignored in emerging implementations. To ignore metering and instrumentation in the design of your Cloud services, or believe the infrastructure will handle this for you, to me illustrates the immaturity of Cloud Computing and a nativity on the part of those implementing Cloud-based services.
The Reason Enterprise Architects Should Study Economics
Economics is the social science that studies the production, distribution, and consumption of goods and services and aims to explain how economies work and how economic agents interact. The really interesting thing about economics is that getting the right answer requires that you pose the right question. That is, if you choose to make conclusions based on casual observations of explicit cause/effect relationships, then your conclusions will seem logical and sound to most who will review them. However, only by ensuring that the cause/effect relationship holds up in the case where casual variables are changed, can you be sure that you have truly observed a phenomena and that the cause is accurate.
I've been reading Freakonomics (looking forward to Super Freakonomics when I'm done), and I'm astounded at how well Steven Levitt's approach to Economics could fit the needs of Enterprise Architecture. For example, in Freakonomics, Levitt uncovers the truth regarding many long-held beliefs, such as most drug dealers live at the poverty line or how little parenting actually impacts the outcome of a child. What helped to uncover these facts was asking the right question and identifying which variables in the equation needed to remain static to ensure a valid outcome. In many of the cases, Levitt undermines what many other Economists, experts and pundits before him rolled out as proven facts and only due to his keen mind and his approach to formulating the problem domain was he able to uncover that which his peers could not. To keep us focused, I won't discuss here how he discovered why violent crime rates dropped in the 90's, but needless to say well-held beliefs as to why were shredded by Levitt's innate ability to target just the right variables and "run the numbers."
If we apply Freakonomics-type analysis to current information technology markets, such as Cloud Computing, based on experts and pundits one would readily equate that: a) public Cloud Computing models offer a more cost-effective way to acquire computing resources; b) it's better to rent software than to own it and; c) in the future, all computing will be paid for based on usage versus ownership. Similarly, if you read the headlines of many tech publications, you would believe that: a) everyone is doing SOA; b) there are many successful SOA deployments and; c) SOA is the only approach to developing systems that you should consider. None of these conclusions are true in all cases, maybe in no cases, and, in fact, the real numbers would probably tell a very different story [I'm leaving this as an exercise to the reader since I want to focus this piece on the approach to EA, not the application of analysis to these issues]. These statements are being made arbitrarily based on extremely small sample sizes and are as valid as looking around you at a large business gathering noting all the iPhones and Blackberries, and concluding that iPhone & Blackberry must outsell all other phones, when in fact, their combined market share is a mere 3.0-3.5% of the overall mobile phone market.
In actuality, I see a lot of IT decisions being made using what, seemingly, is no more exploration than was used to determine that iPhones & Blackberries have a leading market share. It is the Enterprise Architect's role to ensure that the selection and approaches toward development of systems are sound relative to their business, not just other businesses. Moreover, where decisions are based on the work of other businesses' success, those successes need to be properly vetted to ensure that there is enough commonality between efforts, such that you could make the leap that your business will see similar results. Finally, assumptions and theories need to be tested by properly identifying the variables that need to remain static and then comparing; in essence normalizing the question to be homogenous in all situations. For example, in Freakonomics, Levitt points out that in order to prove that the numbers behind why violent crimes dropped in the 90's he needed to review similar trends of violent crimes in areas that had not made the purported changes that led to the large unanticipated decrease. When the trends were normalized to remove noise and anomalies, it was shown that those factors seemed to have relatively little impact on violent crime decrease.
This brings us to the all important question, "how do we demonstrate the value of Enterprise Architecture?" Unfortunately, you cannot demonstrate the value of Enterprise Architecture if you cannot monetize or enumerate the value of all possible choices relative to the choices that are being recommended or those that have been made. Moreover, it's critical that these analyses are carried out over enough time that short-term wins don't supersede long-term potential gains. Thus, it is here that Enterprise Architects, especially those we call Chief Architects, truly show their mettle. It is their experience, coupled with the ability to focus on the right set of variables, understanding the impact of change of those variables and being able to communicate that in a way that allows the business to make effective business decisions, which sets top notch practioners apart from Sr. Software Engineers that the organization placated with a title to keep them happy so they wouldn't leave.
IT Business Edge’s Loraine Lawson Interviews Me On SOA & EA
A couple of weeks ago I did an interview with Loraine Lawson (@lowrain) who covers integration technology for IT Business Edge. There were two different aspects to the interview, one focused on my blog entry "Perhaps SOA is More Strategy Than Architecture" and the other my beliefs on Enterprise Architecture and why I still believe strongly that EA holds great value for business and that business should not be deterred by past missteps due to ineffective applications of EA.
Part I entitled, " Is Strategy the Key Characteristic of SOA?" is located here.
Part II entitled, "In Defense of the Enterprise Architect", is located here.
The Role of The Business Analyst in an SOA World
Those of us that are part of SOA-related projects where traditional business analysts (BA) are involved often find ourselves frustrated by the incongruence between the analyst’s approach to requirements gathering and the SOA design. The problem arises because SOA models functionality of a business across multiple boundaries, whereas the business analyst wants to focus on a user’s needs. One focus is tactical and the other is strategic. However, more than this key difference, certain aspects of the tactical requirements overlap with the strategic requirements; specifically with regard to service boundaries. Thus, important business rules that are relevant to the definition of a particular service are buried among hundreds of irrelevant (to the SOA goals) requirements and the SOA architect is forced to mine these requirements like a miner mining for gold.
In figuring that someone else must have written on this topic, I visited my favorite Web search engine and pulled up the following article: “The new business analyst talks SOA”. I was surprised to find this content on Network World given the nature of the topic, but quickly realized that the author thought of SOA as an application architecture instead of an aspect of Enterprise Architecture. So, it was not much more of a surprise when they recommended that business analysts needed to become more savvy in SOA by learning BPEL, SCA and SDO; perhaps the worst advice I have ever seen regarding this topic. The last thing we want are the business analysts talking about SOA, let alone SODA aspects of SOA.
Still, the article had one point correct, business analysts had better be reading the signs and acclimating themselves to a different type of conversation with their users. Moreover, I believe SOA is a game changer in the way that we collect requirements going forward. For businesses that really want to be successful with SOA, the first conversations that need to be had are with the key business stakeholders, and an Enterprise Architect with SOA experience must be present and facilitating these discussions. This will be the basis for formulating a service model, which will then guide all future system designs. I would also engage a business process engineer to start analysis of business processes that directly impact the effort.
Ultimately, if a BA is able to make the transition to SOA mindset, then they might be a good candidate to drill down and analyze the requirements for the service contract and vocabulary for each service. Otherwise, an SOA architect should really handle these efforts. Which begs the question, what is the role of the BA in an SOA world? I believe there is a need for really good subject matter experts that can work with the Enterprise and SOA architects to fully define the interactions between business functions and, perhaps, to capture the usability models. However, even in the case of usability, a usability expert will offer more insight into how to present information to the user than the BA.
Overall, my analysis is that in an SOA universe, the role of the BA is greatly diminished unless they can adapt to the change in role and gain an understanding of how to facilitate requirements gathering in a much more compartmentalized manner as outlined by the SOA architect and the business process engineer.
Perhaps SOA is More Strategy Than Architecture
On Thursday, September 10th, 2009, I moderated a panel at the 1105 Group's Enterprise Architecture Conference in Washington, DC entitled, “SOA Goes Mainstream - An Industry and Government Roadmap.” On the panel we had two Federal government agency representatives and two industry representatives, with one of the industry representatives providing a FedEx case study as the basis for their SOA experiences.
As expected, each of the panelists' SOA experiences was varied, with no two taking an identical approach. However, the interesting tidbit of information I garnered from moderating this panel is that after a couple of years of effort, some approximation of a methodology that is termed “SOA”, which is specific to each organization, emerged and started delivering value to the organization.
Granted, I have been one of the louder proponents calling for agreement on the use of the term ‘SOA’ because I believe that without common agreement a term is relatively meaningless—I still believe this is true. However, in opening myself up to the idea that SOA may be more of a concept than an actual architectural approach, it became clear to me that that each organization may also organically formulate their own approximation of SOA that meets the needs of their business as functional components, or services. Moreover, each business’ approach toward SOA will most likely look completely different than any other business’ approach, their pain points will be different and their approach toward measuring success will be different.
For me, this is a startling revelation surrounding SOA. If one looks at the supporting industry infrastructure that has built up around the term SOA—software, training, methodologies, blueprints, etc.—the realization that each organization may ultimately need to tailor the general concepts into something that is different from organization to organization, minimizes these efforts to nothing more than templates or patterns to use as a starting point. Moreover, it’s quite possible that the only consistent aspect of SOA from organization to organization may be the road to adoption.
This also raises the need to re-evaluate earlier claims regarding the “Death of SOA” and SOA’s “trough of disillusionment.” If a consistent part of each organization’s early SOA efforts is represented by this tailoring effort, then perhaps some of these early failures are nothing more than part of the process analogous to the caterpillar spinning its cocoon and then working its way into becoming a butterfly. It does seem from my research that there is a consistent pattern among SOA adopters that successes seem to be more prevalent in the second and third years of SOA initiatives. Moreover, if we know this information going into an SOA effort, we can better manage expectations of the business with regard to Return-on-Investment (ROI) and time frames. Most of all, if the tailoring effort is a known part of the process, then unrealistic expectations can be removed from the practitioners, allowing them to focus on reaching successful ends.
Note, this realization does not mitigate the enterprise architecture efforts that must accompany, or more accurately, precede an SOA effort in order to ensure success. This realization is focused solely on the transformation of the business based on the EA artifacts and a building a service-oriented methodology to support that transformation. Thus, SOA is effectively the transition plan for the organization to maximize its resources, applying a ‘once-and-only-once’ mantra toward development efforts and align the technology imperatives with the business goals and functions.
So, based on this thesis, is SOA really architecture, or is it simply a strategy for business transformation?
SOA & Cybersecurity
I recently started my research into cybersecurity and I am working to become more prolific in this area. Naturally, given my inclination to Service Oriented Architecture (SOA), I am really interested in issues related to both SOA and cybersecurity.
One thing I noticed immediately regarding cybersecurity is that, in general, there are relatively few experts in this area given the total number of IT professionals in the world. This is a known fact. The US Government estimates that we will need on the order of a hundred times more cybersecurity experts in coming years than we currently have. Moreover, due to the nature of work of cybersecurity experts, they don’t readily publish what they know and, thus, it’s even more difficult to expand the pool of cybersecurity experts.
Seek long and hard and you can find some excellent research and literature on the topic of cybersecurity. However, attempting to locate research that covers cybersecurity and SOA was a fruitless endeavor. Sure, we can start with basic concepts like digital signatures, encryption, policy management and access control, however, the literature and examples in these areas often focus on corporate enterprises being operated on secure networks. But, I delve too deeply, too quickly.
In the past, I have been a harsh critic on the lack of consistency of definition and agreement of SOA. Up till now, this has been an academic discussion that isn’t going to greatly impact the universe. If a company wants to build JBOWS (just a bunch of web services), call it their SOA strategy and believe think their acting strategically, so be it. However, lack of agreement on SOA has significant real-world implications with regard to cybersecurity.
If you can’t define it, you cannot secure it!
SOA has become a catch-all for multiple application development and enterprise architecture initiatives. So, if you’re tasked with focusing on cybersecurity for your SOA, you could focus on locking down access to your Web Services, stopping SQL injection attacks, addressing DDoS attacks against the service, etc? Each of these areas requires considerable knowledge of the entire computing stack from telecom through the hardware through the operating system and into the application. Holy rotten fish Batman! That’s a tall order for even the most adept team, but it’s made even more difficult by the fact that there aren’t that many cybersecurity experts available that understands this entire domain.
Additionally, if SOA is the architecture, then shouldn’t security be a primary consideration across the entire architecture? That is, shouldn’t the resulting artifacts of an SOA deliverable address security top-to-bottom? I believe it should, but if you’re in the camp that believes SOA is driven by identifying your service boundaries by your business processes versus business function, then it’s going to be much more difficult to manage appropriate access since processes cross boundaries so often. Nothing screams louder for ensuring proper granularity in an SOA like cybersecurity. A black box is easier to protect than a set of discrete, interconnected nodes.
I’m clearly at the beginning of this exploration. However, what I have experience in with regard to the WS-* security mechanisms, security tools and technologies for securing Web-based and non-Web-based applications, still do not begin to address the real hard issues regarding cybersecurity in an SOA; especially as we expand the notion of service. For example, Twitter, for all intensive purposes, is a service that, according to Twitter staff, was recently unavailable for a considerable amount of time due to directed denial of service attacks.
What if this service was germane to you running your business? Do you still believe WS-* is going to help you protect your SOA-based services? Furthermore, if you take your services into the Cloud, what impact is that going to have on securing your critical business services? The Internet is a darker and grimmer place that the pleasant face we see in Google everyday.
Of note, if you’re a cybersecurity expert looking to mentor someone on the real esoteric issues of how systems are compromised, let me know!
When SOA Fails, Just SCA
There's a lot of points that will be made in this blog entry. To keep them straight, I will highlight them right up front:
- Vendors are getting behind SCA because it reinforces a need for tools
- The importance of architects in software development
- Untested technologies are being pushed on IT like bad drugs from the FDA
So, in this morning's email I received a notification from ActiveVOS that their CTO is a primary contributor to a new book recently released on Service Component Architecture (SCA). Having just recently completed a full investigation into SCA, two things jumped out at me: 1) SCA is heavily being driven by the vendor community and 2) SCA breaks many of the rules of SOA that have been touted by these same vendors for the past 6 years. For example, SCA rewards an implied contract versus a contract-first approach to service development. That is, the contract is derived from the programming model versus defined by the architects.
What's going on here?
My subjective opinion based on all the factors that I currently have at my disposal is that SCA is a step backwards in software engineering. It's an abandonment of the SOA principles that have failed because of lack of investment in proper architecture toward a pure programming model driven by software engineers. Thus, the goal here is once again to allow poorly-designed systems to be built by software engineers with very little architecture experience so they can claim to have some attributes of SOA. Moreover, SCA is heavily dependent upon tools for creation and management. This is a game-changer from applications that be developed with a good 'ole EMACS editor and a command-line compiler. Starting to get the picture?
I recently was sent an email by a software engineer on one of my projects claiming that based on their experiences over the past year with J2EE and BPEL that this is the way that they would build all their applications. Unfortunately, the requirements for the application they're on call for pure workflow management, not integration. Which leads me to my next point, all software needs to be properly designed by an architect. Software engineers build, architects design. Architects should have a much wider berth of experience implementing and supporting various solutions using various technologies in a multitude of environments. Having accomplished the latter, one learns the reality of taking any concept from paper to production. Unfortunately, in many organizations, the developers are not forced to participate in operational support and are never privy to the impact of their bad decisions.
Additionally, tools vendors are blurring the line between form and function of various technologies and standards. BPEL is a great example. First off, BPEL 1.0 barely supported a plausible B2B integration scenario, and now BPEL 2.0, barely out of the starting gate, is the uber platform for all BPM, SOI, Integration and workflow? Come on, let's use our heads here. At best, this standard is in its infancy and hardly appropriate for development and delivery of production systems. Still, we have pundits discussing how XPDL lost the battle to BPEL (http://itredux.com/2008/09/28/why-bpel/), which is a bold claim considering the WfMC is currently working on BPMN into XPDL compliance. Then we give this information to software engineers and ask them to make architecture decisions based on it and we wonder why the CEO thinks IT is a bunch of backwards yahoos and the Cloud and outsourcing looks like the answer out of dumbass alley.
Finally, and this distresses me the most, we have a growing number of software engineers, like the one I discussed earlier, that become entrenched in understanding how to work with these tools and technologies, become ideological about them and reject alternative methods and approaches to the detriment of the software and solutions they are building. What's more is that these software engineers work to undermine alternative opinions and slur engineers and architects who do not agree with this approach. For the past few years, we have seen this with Spring and Hibernate, and will begin to see this more and more as BPEL and SCA gain in momentum as pushed by the vendors. Perhaps the reason Microsoft has never been threatened by the Java community is that they can see these vendors and engineers are cannibalizing each other through layers of complexity to the point where Microsoft's .NET simplicity looks attractive.
If I Was An IT Manager I Would...
So, if I was an IT Manager, Chief Architect, CIO, CTO for a mid- to large-sized business today, here's the things that would be on my checklist to ensure that the systems that I was building today were maintainable, sustainable and reliable:
1. Enterprise architecture - I would ensure that all applications in my portfolio were being rationalized with the needs of the business as a whole and not a single group of users. The latter is a bottom-up tactical approach that eats at the IT budget and limits the ability to design for the needs of the business as a whole. While they are difficult to avoid building, they act like parasites sucking the IT budget from a support and maintenance perspective limiting the opportunity to build software that will propel the business forward.
2. Chief Architect - I would put a single solution architect in charge of all the software that is being built. This person would rationalize the product and tools being used based on reliable, tested technologies, not the latest fad. This person would also provide objective, not ideological direction with regard to technology selection. Many businesses today are still asking Java vs. .NET. The reason for this is that .NET offers some very powerful solutions and allows for rapid application development that can be deployed on Linux boxes using Mono if that is desirable. However, the Java bigots aren't really open to having their skills made invalid by selection of C# and .NET platform. It's like offering a natural, homeopathic cure in the face of a big pharmaceutical company or a cross in the face of a vampire; these things cause mortal wounds, hence, you cannot allow them to control the choice.
3. Get the vendors out your organization. A recent blog entry by the CEO of WSO2 listed the current Oracle suite in the gigabytes, this is not expressing the virtues of KISS. Vendors' product suites are bloated, costly and in many ways proprietary. Developing distributed systems is complex, but can be made simpler by reducing the number of moving parts. The number of artifacts required to develop a SCA application is double to triple compared to the number of artifacts to create the same application using SpringWS. A properly-designed SOA reduces complexity, while these supposed SOA tools increase complexity significantly. Vendors know you are desperate for resources and will take advantage of this fact today ultimately locking you into a product selection that will be extremely costly to back out of in the future.
In Conclusion
Ultimately, the rubber needs to meet the road, and work needs to get done. The recommendations I make above are subject to the biases of the individuals and their experiences, but this too can be mitigated (to a degree). The CAEAP has an oath of professional conduct that if adhered to by the professional will lead to the best decision, even if that decision is not the preferred one of the architect. IT resources--specifically engineers and programmers--make their living based on their skills; the more their skills are in demand the greater the value and the more money that can be earned. Businesses would do well to stop hiring resources based on individual technology skills (except if you're hiring a consultant, because that's what you're buying) and start hiring based on the individual's ability to think your business to success.
Years ago, mid-80's, when I was interviewing for positions, businesses used to take pride in testing how programmers thought about problem solving, today, they look for buzzwords. Which brings us back to SCA. It won't be long till these three letters start showing up on resumes. Will your organization succumb to the pressure to use a tools-oriented approach to developing inferior networked applications that will need to redesigned and redeveloped in a few years, or will you avoid the peer-pressure sand trap this time and continue working toward a properly designed solution that will carry your company forward?
Disclaimer
My Most Recent Book
:Other Links
Web Profiles
Recent Posts
- Cloud Computing Pragmatics
- The Real Impact of IT Owning Your SOA Effort
- Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
- The Busy Executive’s Service Oriented Architecture Reference Guide
- The Busy Executive’s Quick Cloud Computing Reference Guide
Tags
Recent Comments
- Steve Hughes on Cloud Computing Pragmatics
- CloudSigma on Cloud Computing Pragmatics
- Is ‘Service-Oriented Approach’ a Better Way to Define SOA? | SOA Governance - Service Oriented Architecture - SOA Business - SOA Design - SOA Services - SOA Software - SOA Solutions - SOA Security - SOA Web Service on Perhaps SOA is More Strategy Than Architecture
- Jaime Ramirez on The Busy Executive’s Quick Cloud Computing Reference Guide
- gon zalez on The Busy Executive’s Quick Cloud Computing Reference Guide
Blogroll
Other Links
Archives
- July 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- March 2009
- February 2009
- January 2009
- September 2008
- August 2008
- May 2008
- December 2007
- November 2006
- April 2005
