Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
Thanks to @jadkins for the colorful ending to that cliched saying, which really got me thinking. Sometimes we just make certain aspects of architecture too difficult for no reason. Good design is important and can have significant financial impact if done incorrectly, but chances are the negative impact comes into play during the engineering effort, not during the architecture. To clarify my point, one must first understand what architecture is really focused on and how that differentiates from engineering.
For example, if we're designing a solution for financial services products. I can choose to take a framework approach to architecture in which we identify the common financial services components, such as securities and instruments and expose the manipulation of these components through the framework. Then we can continually add new algorithms to manipulate these same components in a modular fashion. Yet, another approach might be to examine things from the perspective of the buyer and the seller and allow each financial product to implement it's own representation of securities and instruments and orchestrate the process of selling and buying.
Either of the aforementioned scenarios is a viable architecture. In the former scenario, we can develop new computational models very quickly as long as we don't change the definition of the semantic elements. However, it can be a bit rigid since all new products must be able to define the components of its product based on these definitions. Also, sharing these products across trading partners becomes more complex.
In the latter scenario, we have a more service-oriented approach, focusing on the consumers and less on the definition of the products themselves. Hence, each service is unique, and may produce redundancy in defining the elements used to create a product, but we can orchestrate buying and selling across products in a common fashion. This approach facilitates incorporation of a much wider community since agreement is not required for the representation of elements to create a new derivatives product.
Which architectural approach is better? Well, as any good consultant will tell you it depends on your goals. If your goal is to allow your big brains to develop new models quickly to identify market making opportunities, then the first approach sounds ideal. If your goal is to enable a new derivative product to be treated as a single instrument even though it's comprised of securities that are bought and sold across a variety of exchanges, then the latter approach will probably suit your needs better than the former.
Neither scenario is optimal or guaranteed, but they both represent a means of simplifying the system enough to extend it over time without considerable re-work efforts.
When complete a design goes through an engineering effort where a software engineer will take this design and "press" it into software. During this effort, decisions are made about platform, programming language, external libraries, software development methodology, etc. Any one of these decisions, or a combination of them, may lead to the undoing of the great abstract architecture designed. For example, if the system cannot scale to the volume of financial services transactions, then that would be one major flaw. Using an open source library that ends up having a major security flaw is another potential flaw.
These scenarios are not very different than design and engineering in any other industry. The architects can design the most awesome car or impressive building, but if the engineers select the wrong parts or use inferior materials, the resulting product will never allow the architecture's brilliance to shine through. Note, manufacturers put significant effort into ensuring that their designs will work by building prototypes and testing against known environments that the product will have to operate under. Thus, from this perspective, architecture is responsible for putting forth a design that is credible before engineering commences.
My Experience Developing A Google Wave Robot
This past weekend I set out explore some of the extension capabilities of Google Wave. One of the weaknesses that have been identified by many is the lack of integration with email. For me, in particular, because Wave is new, many Waves are being orphaned as those playing and testing out Wave don't come back to the conversation for long periods, if at all. My goal was to use email as a means to bring people back to the Wave and keep the collaboration/discussion going in a single environment. Some developers are exploring letting users contribute from email, but in my opinion, that undermines the goal of Wave.
First, Kudos to Google for making it easy and straightforward to get a development environment ramped up quickly for developing extensions. Robots are based on their Google App Engine architecture, which allows users to leverage their Eclipse plug-in for App Engine development. For anyone that has ever developed a servlet-based application, the plug-in for packaging and deployment in the App Engine environment made the entire process quick and easy. Of course for fun, I had to set up an Ubuntu instance in my virtual machine environment to do this development.
After getting my environment configured, I downloaded a Java-based Robot sample that is the equivalent of "Hello World!" for Google Wave Robots just to ensure that I could compile and deploy. There was very little effort to make this work and test. Of course, not having a sandbox account yet meant that I had to use production Wave environment, so there's now a lot of orphaned waves that I used for my tests.
With my application harness up and running, it was time to dive into the documentation and play. In my initial design I was planning on using the Wave itself as a storage mechanism for the subscription registrations, but as I learned, there is no guarantee that a Robot will have access to all Blips—the unit of data within a Wavelet—within a Wave. However, to learn that took a lot of review of the Wave data structures and data received from the Wave server based on the events I was processing. At first, I used the Wave itself to produce the artifacts for review since I was having issues getting my debug statements to appear in the application log. This was not a good idea as the Wave environment couldn't handle the mass number of Blips being added at the speed the Robot was adding them. So, I had to aggregate the data and then put it all into one Blip. Making use of this data was also interesting since my formatting commands were being ignored. I finally figured out \n works in a text blip for a newline, but never got the markup content to be produced properly.
All of the above could have been avoided if Google simply documented examples of what the input to the Robot would look like and described the structure of the data received. There's still a major structure of the Wave content I still haven't fully grok'd regarding annotations and elements. Since this is mostly for visual support, and I was focused on a non-visual tool, it got back-burnered for another day.
Once I understood that I could not rely on having access to all Blips in the Wave at all times, I realized I was going to have to establish some persistence of data. App Engine's Data Nucleus Java persistence is excellent for this type of requirement. With little effort, I was able to create a mechanism that could store and retrieve a list of Java objects without having to use JDBC or proprietary persistence APIs. The other service that App Engine offers that I needed was email support via JavaMail. In the future, I will also try their XMPP service for Google Talk subscriptions.
After 2.5 days of effort, the outcome was highly successful. The command-line Robot that I called Checkwave is now published and available for anyone to include in their Wave. It support subscribe, unsubscribe and list functions. When you update a Wave all subscribed participants receive and email with text content of the Blip that was added and it includes a hyperlink that brings you back into the Google Wave environment with that Wave on screen.
A more technical description of the Robot can be viewed at http://checkwave.appspot.com.
Having developed a secure instant messaging platform at Ikimbo back in 2001 that allowed for robots and in-context replies, I am a big fan of these features in Wave. IM is great, but a conversation among multiple participants can become unwieldy to follow. The ability to explicitly address a prior message in context is a huge advantage over traditional IM systems. Moreover, the ability to facilitate this communication asynchronously is another major plus. Getting people to use a new tool, however, is a very difficult task. For me Checkwave was a critical tool to assist me in bringing people back to the Wave.
All in all, working with Wave was a pleasant experience and I look forward to seeing where Google takes this platform. Some additional notes that people may find useful regarding Google Wave robots:
1. Multiple robots in the same Wave can have unintended consequences. When Cartoony and Checkwave existed in the same Wave, Cartoony would turn my text to graphics before Checkwave had an opportunity to read it, hence, it broke the command-line capabilities.
2. As the owner of information now entrusted to me by those using Checkwave, I realize that I must protect this data, but it also made me aware that any Robot you add has access to everything that occurs on that Wave. So, you need to be careful which Robots you add to your Waves as you are providing implicit trust of any data provided through that Wave.
When SOA Fails, Just SCA
There's a lot of points that will be made in this blog entry. To keep them straight, I will highlight them right up front:
- Vendors are getting behind SCA because it reinforces a need for tools
- The importance of architects in software development
- Untested technologies are being pushed on IT like bad drugs from the FDA
So, in this morning's email I received a notification from ActiveVOS that their CTO is a primary contributor to a new book recently released on Service Component Architecture (SCA). Having just recently completed a full investigation into SCA, two things jumped out at me: 1) SCA is heavily being driven by the vendor community and 2) SCA breaks many of the rules of SOA that have been touted by these same vendors for the past 6 years. For example, SCA rewards an implied contract versus a contract-first approach to service development. That is, the contract is derived from the programming model versus defined by the architects.
What's going on here?
My subjective opinion based on all the factors that I currently have at my disposal is that SCA is a step backwards in software engineering. It's an abandonment of the SOA principles that have failed because of lack of investment in proper architecture toward a pure programming model driven by software engineers. Thus, the goal here is once again to allow poorly-designed systems to be built by software engineers with very little architecture experience so they can claim to have some attributes of SOA. Moreover, SCA is heavily dependent upon tools for creation and management. This is a game-changer from applications that be developed with a good 'ole EMACS editor and a command-line compiler. Starting to get the picture?
I recently was sent an email by a software engineer on one of my projects claiming that based on their experiences over the past year with J2EE and BPEL that this is the way that they would build all their applications. Unfortunately, the requirements for the application they're on call for pure workflow management, not integration. Which leads me to my next point, all software needs to be properly designed by an architect. Software engineers build, architects design. Architects should have a much wider berth of experience implementing and supporting various solutions using various technologies in a multitude of environments. Having accomplished the latter, one learns the reality of taking any concept from paper to production. Unfortunately, in many organizations, the developers are not forced to participate in operational support and are never privy to the impact of their bad decisions.
Additionally, tools vendors are blurring the line between form and function of various technologies and standards. BPEL is a great example. First off, BPEL 1.0 barely supported a plausible B2B integration scenario, and now BPEL 2.0, barely out of the starting gate, is the uber platform for all BPM, SOI, Integration and workflow? Come on, let's use our heads here. At best, this standard is in its infancy and hardly appropriate for development and delivery of production systems. Still, we have pundits discussing how XPDL lost the battle to BPEL (http://itredux.com/2008/09/28/why-bpel/), which is a bold claim considering the WfMC is currently working on BPMN into XPDL compliance. Then we give this information to software engineers and ask them to make architecture decisions based on it and we wonder why the CEO thinks IT is a bunch of backwards yahoos and the Cloud and outsourcing looks like the answer out of dumbass alley.
Finally, and this distresses me the most, we have a growing number of software engineers, like the one I discussed earlier, that become entrenched in understanding how to work with these tools and technologies, become ideological about them and reject alternative methods and approaches to the detriment of the software and solutions they are building. What's more is that these software engineers work to undermine alternative opinions and slur engineers and architects who do not agree with this approach. For the past few years, we have seen this with Spring and Hibernate, and will begin to see this more and more as BPEL and SCA gain in momentum as pushed by the vendors. Perhaps the reason Microsoft has never been threatened by the Java community is that they can see these vendors and engineers are cannibalizing each other through layers of complexity to the point where Microsoft's .NET simplicity looks attractive.
If I Was An IT Manager I Would...
So, if I was an IT Manager, Chief Architect, CIO, CTO for a mid- to large-sized business today, here's the things that would be on my checklist to ensure that the systems that I was building today were maintainable, sustainable and reliable:
1. Enterprise architecture - I would ensure that all applications in my portfolio were being rationalized with the needs of the business as a whole and not a single group of users. The latter is a bottom-up tactical approach that eats at the IT budget and limits the ability to design for the needs of the business as a whole. While they are difficult to avoid building, they act like parasites sucking the IT budget from a support and maintenance perspective limiting the opportunity to build software that will propel the business forward.
2. Chief Architect - I would put a single solution architect in charge of all the software that is being built. This person would rationalize the product and tools being used based on reliable, tested technologies, not the latest fad. This person would also provide objective, not ideological direction with regard to technology selection. Many businesses today are still asking Java vs. .NET. The reason for this is that .NET offers some very powerful solutions and allows for rapid application development that can be deployed on Linux boxes using Mono if that is desirable. However, the Java bigots aren't really open to having their skills made invalid by selection of C# and .NET platform. It's like offering a natural, homeopathic cure in the face of a big pharmaceutical company or a cross in the face of a vampire; these things cause mortal wounds, hence, you cannot allow them to control the choice.
3. Get the vendors out your organization. A recent blog entry by the CEO of WSO2 listed the current Oracle suite in the gigabytes, this is not expressing the virtues of KISS. Vendors' product suites are bloated, costly and in many ways proprietary. Developing distributed systems is complex, but can be made simpler by reducing the number of moving parts. The number of artifacts required to develop a SCA application is double to triple compared to the number of artifacts to create the same application using SpringWS. A properly-designed SOA reduces complexity, while these supposed SOA tools increase complexity significantly. Vendors know you are desperate for resources and will take advantage of this fact today ultimately locking you into a product selection that will be extremely costly to back out of in the future.
In Conclusion
Ultimately, the rubber needs to meet the road, and work needs to get done. The recommendations I make above are subject to the biases of the individuals and their experiences, but this too can be mitigated (to a degree). The CAEAP has an oath of professional conduct that if adhered to by the professional will lead to the best decision, even if that decision is not the preferred one of the architect. IT resources--specifically engineers and programmers--make their living based on their skills; the more their skills are in demand the greater the value and the more money that can be earned. Businesses would do well to stop hiring resources based on individual technology skills (except if you're hiring a consultant, because that's what you're buying) and start hiring based on the individual's ability to think your business to success.
Years ago, mid-80's, when I was interviewing for positions, businesses used to take pride in testing how programmers thought about problem solving, today, they look for buzzwords. Which brings us back to SCA. It won't be long till these three letters start showing up on resumes. Will your organization succumb to the pressure to use a tools-oriented approach to developing inferior networked applications that will need to redesigned and redeveloped in a few years, or will you avoid the peer-pressure sand trap this time and continue working toward a properly designed solution that will carry your company forward?
What's the difference between a software component and a service?
It seems that I am not as flexible as I believed I could be on my thinking regarding SOA. I attempted to categorize various SOA engagements in my SYS-CON article entitled “A Classification Scheme for Defining SOA” . I believed that I could hide my dissatisfaction with the lack of clarity surrounding SOA by lumping SODA/application development into its own subcategory. I was wrong! When it comes down to it, there's still just too much ambiguity surrounding the term service.
So, you might ask, “What is the big deal if we call everything running on a computer a service?” The answer is that not all services are created equally and there's no way to determine the type or extent of services when a single term is used as a catch-all.
For me, SOA is defined precisely as follows:
Service-Oriented Architecture (SOA) is an archetype—an architectural pattern —that focuses on design of systems from the perspective of providers and consumers as defined by a contract. SOA-based designs introduce agility by enabling interchangeability of service providers without requiring process changes in the consumers. Hence, the SOA is applied at the system level, not just at a single component within a system.
Because I define SOA as an archetype, you can not have a direct instance of SOA, you can use SOA to define a new architecture, which can then be used to create instances of systems. For example, Service-Oriented Integration (SOI), Web 2.0 and Cloud Computing are all architectural types based on the SOA archetype. However, to put things in context, FedEx and UPS, as businesses, are also SOA architectures. Needless to say, if you follow the laws of object-orientation, it's not invalid to identify an object by it's topmost ancestor, but in doing so you lose the object's essence. This is a great technique for lumping things together in a collection, but horrible if you want the richness and value of the object to come through.
Of the three technology-related architectures based on SOA listed above, SOI and Web 2.0 clearly have a strong software connection. Some identify the the software component that has a SOAP or HTTP interface as a service. Well, just as SOA is an archetype, service is an archetype as well, and, indirectly, these software components are services given that they derive from the service archetype.
To better understand my point we need to explore the technical ramifications of this for a moment. With the growth of TCP/IP as a ubiquitous networking protocol, so grew the concept of client/server computing. In client/server computing, a user interface application consumes networked software services to provide data on demand versus having the application exist as a monolithic entity on a single computer. Client/Server computing enabled networked shareable resources.
If I didn't use the term Client/Server in the above paragraph, 9 out of 10 technical people today would say I was talking about SOA. So, is everyone who is developing systems using Web services today really just doing Client/Server? I believe so, but that wouldn't be popular, after all, there aren't hundreds of job openings for client/server architects right now.
[Sidebar Note: What are these people asking for SOA architects really looking for if it's not client/server experience? From what I've seen, it's typically experience with a particular vendor's software for building distributed applications. But, to those people I warn you “big mistake”. The underlying standards will not make it significantly less expensive to switch from that tool to another one.]
To summarize, no one who claims to be doing SOA would openly admit they're just really doing client/server. There is a subset of people doing SOA that are actually focusing on modeling the business as a set of functional service areas (these people are really doing SOA). Then, there's a bunch of people developing software components using a client/server design pattern claiming they're doing SOA.
So, I ask you, do you still think that a common definition of SOA is not needed in the industry?
What’s New Is Really What’s Old
I'm working on a project with a fairly young engineer (let's just say I have 12 years on him). He's so enthusiastic about Spring and Hibernate, and so bought into concepts, such as Inversion of Control and declarative programming. It makes me laugh a bit how guys like him act like this stuff is all new and that no one understands the benefit of using it.
Unfortunately, we do know the benefits, but also the pitfalls since we've been doing it since the 90's. Greg Lomow and I were on an advanced development team at Banker's Trust in the 90's developing an IoC container for Microsoft COM objects before Microsoft even released DCOM. It supported persistence and messaging of C++-based COM objects.
If you want to see a great example of declarative programming vis a vis Spring-like injection, my EAI with XML and Java book had a chapter provided by Bill la Forge describing his MDSAX tool that created in 1999 allows users to define Java Swing interfaces in a declarative manner.
But, technology is like surfing, if you take the first small wave back to shore, you get a fun trip and others will learn how it's done on your trip, but there's a good chance you'll miss the Kowabunga wave that everyone will be talking about for years to come.
The Most Important Lesson I've Learned About the Information Technology Industry
It's taken more than 10 years, but I finally got it through my thick skull:
A design is only as good as the potential for it to be put it into practice.
I tend to come up with extremely elegant, agile and extensible designs. What I've learned, however, is that not everyone sees the elegance of the design as quickly as I do. They either are unable to see the value because of lack of experience or because other pressures are forcing them to limit their attention span to produce less elegant solutions faster even if they might incur the cost of having to be re-designed at a future point in time.
While I am glad to finally be able to accept this truth, it is a bit disheartening, because it means that most businesses are truly limited in its ability to apply technology effectively to their business problems. However, accepting this postulate means that now I can focus my attention on how to help them injure themselves the least. Most importantly, I learned that sometimes, we have to change the requirements because we cannot change the people or the tools.
Hence my new postulate:
Faster, Cheaper, Better -- Pick any 2
People, Tools, Elegance -- Pick any 2
Disclaimer
My Most Recent Book
:Other Links
Web Profiles
Recent Posts
- Cloud Computing Pragmatics
- The Real Impact of IT Owning Your SOA Effort
- Architecture: 1000 ways to skin the cat, either way, the cat’s till dead
- The Busy Executive’s Service Oriented Architecture Reference Guide
- The Busy Executive’s Quick Cloud Computing Reference Guide
Tags
Recent Comments
- Steve Hughes on Cloud Computing Pragmatics
- CloudSigma on Cloud Computing Pragmatics
- Is ‘Service-Oriented Approach’ a Better Way to Define SOA? | SOA Governance - Service Oriented Architecture - SOA Business - SOA Design - SOA Services - SOA Software - SOA Solutions - SOA Security - SOA Web Service on Perhaps SOA is More Strategy Than Architecture
- Jaime Ramirez on The Busy Executive’s Quick Cloud Computing Reference Guide
- gon zalez on The Busy Executive’s Quick Cloud Computing Reference Guide
Blogroll
Other Links
Archives
- July 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- March 2009
- February 2009
- January 2009
- September 2008
- August 2008
- May 2008
- December 2007
- November 2006
- April 2005
