This white paper outlines the greatest challenges facing CIO’s as they attempt to establish secure information infrastructure, and how the Interoperability Clearinghouse (ICHnet.org) has been formed to addresses these critical needs. The ICH is a non-profit "honest broker" technology research and validation collaboratory that defines "what works" and "what works with what" in an effort to defeat the technology churn rate, market hype and inherent complexity of moving from "architecture to implementation reality".

The emergence of the Internet and Information Assurance computer paradigms (aka Secure E-Business) offers new hope to the IT community. We are beginning, for the first time, to see failure rates drop below the 50% mark in industry (April 01 CIO Magazine Report). CIO’s have found that we must synchronize our Information Assurance systems with our E-Business (or E-Government) systems. Our leading researchers have found that IT can successfully undergo major transformation leveraging these converging internet and information assurance technologies by building secure internet infrastructure. By doing do, industry can better transform itself and better serve the communities they serve, integrate disparate systems, and accomplish long unmet interoperability goals.

The opportunities to the IT community to leverage these converging Information Assurance and Internet paradigms include a "plug and play" architecture without the pain of customer software development. It requires that organizations invest in the establishment of a standards based, secure information infrastructure (aka Secure E-Business/Government). This way, every time you develop a new "appliance", you don’t have to rewire your entire "house". Lesson learned gathered from industries leading CIO’s (Boeing, GM, Ford, NSA, Citigroup, AT&T, US Navy, Wells Fargo, Federal Express, Office Depot), indicate that establishing a secure infrastructure is priority #1.

However, this new paradigm offer not only opportunities, but major pitfalls that threaten to undermine any gains we may achieve. As the internet opens our systems, it also creates new vulnerabilities that were otherwise not exposed. The rate of change, and associated marketing hype undermines our decision making process.

The ptifalls of moving into this new paradigm are great. The risk of choosing the wrong solutions set (does meet needs, won’t interoperate, insecurable), or not having the right resources to deal with both the rate of change and the complexity brought upon us by all these complex choices. The reality most of us face is that instead of "plug and play", we are really faced with "plug and pray". To help demystify and understand how to deal with the fast paced IT market, a group of the industry’s most insightful IT leaders have launched a much needed collaboratory called the Interoperability Clearinghouse consortium. Finally, an organization dedicated to the needs of the IT user.

To better cope with the complexity of this COTS centric world, IT needs new architecture methods, tools and market knowledge to better insert these new technologies into the operational continuum. Today, architectonics (the science of architectures) has taken on an increased interest, and is the key to coping in large IT organizations.

Based on a recent survey of CIOs, today’s number one priority facing IT is building the secure information infrastructures that support this new internet enabled, "plug and play" paradigm. This information infrastructure is the common "plumbing and wiring" necessary to support acquiring "solution appliances" without bearing the prohibitively expensive cost to "re-wire" the "house" for each new appliance. IT organizations simply cannot afford to rebuild the infrastructure for each new application. Based on numerous studies, 70% of the effort and expense to develop major systems is dedicated to the infrastructure and 30% on the business rules. If we can successful develop a secure information infrastructure, we can cut the cost of application development by at least 50%.

An analogy recently shared with us by one prominent CIO:

Add ongoing prototyping efforts and internally developed capabilities to the fray, and today’s CIO really does face an ulcer-prone conundrum. Some say, the Internet revolution has redefined CIO as "Career Is Over". But this need not be the case!

Today’s CIO has no choice but to find a way of coping with this e-Business dilemma. Every new technology, and every old one, has been re-engineered to support the ubiquitous "plug and play" Internet paradigm. Many have already discovered that for most cases, "plug and play" really means "plug and pray". But rather than sticking with old approaches until a complete meltdown, varying degrees of success have been reached by creative CIOs who are coping with the rate of change, increasing complexity and dearth of accurate information. How does one move into this new component technology paradigm where one gets away from the software development traps, and leverage the dearth of commercial off the shelf software that plugs into an internet based information infrastructure.

There are several technology research, validation and insertion paths a CIO can take to achieve his or her IT goals. These approaches fall into one of three primary strategies.

Approach #1: Internal Testing:

Many large organizations support internal pre-production, interoperability, or experimentation testing labs where emerging products and technologies can be installed, tested, evaluated and assessed in a low-risk fashion. This approach for vetting emerging technologies is very time consuming and expensive. These labs are maintained with the anticipated hope that the supporting organization can better keep a leg up on the challenges of the actual integration of emerging tools and technology into their existing IT infrastructure. There are several problems and some bad historical data with this model. First of all, where does the lab staff come from; and who pays their typically expensive bills? Additionally, how is the ROI for this investment realized, if even calculable? It is a tough sell, especially in light of the deluge of raw material brought on by the Internet economy. Finally, such a lab has a finite ability to address an unending stream of material. Any lab large enough to truly keep abreast of everything would truly be cost prohibitive as an overhead expense.

While some organizations have done admirable jobs with this approach, it is like the runner who stumbles and has to run ever faster to keep his balance until, eventually he cannot run any faster, and falls. If we focus our testing on what is generally not known, and can leverage what has already been tested in our community of interest, this approach can work. However, those individuals who benefit from these labs are often unwilling to usher in change or acknowledge the futility of their efforts.

Approach #2: Outsourced Testing and Research:

"If you want to know what’s available, just post an RFI and let everyone else do all the research for us!" said the executive. And supposing the marketplace recognizes a juicy potential future contract following the RFI, the information really will flow -- much like a Galveston storm drain during a hurricane!

This approach, while clever, and clearly effective in gathering information, is also impractical. In fact, this model gathers only a very large snap shot of the marketplace’s best marketing propaganda. This information still must be systematically read, analyzed, assessed, and evaluated – a process that, by the time it has been completed, is so dated as to be virtually useless.

This "clever" approach is a variation on the massive "beta" program or "Open Source" initiative where the burden of testing and debugging is passed on to the consumer. Unfortunately, the quality of the end product can only be as good as the contributed effort. With RFIs, one experiences high-quality marketeering, supported only by a minimum of technical expertise.

Based on recent failing of those organizations who have operated these testing labs, we now know futility of their efforts when not coordinated. Though more efficient than internal testing labs, the rate of change has made their efforts even more vulnerable. Organizations like the Corporation for Open Systems and other branding efforts have been hard to justify to corporate IT. We cannot test for all the possible factors and must leverage current knowledge if this market is going to survive.

Approach #3: Collaborative Validation;

An internal lab, which addresses all the issues necessary to the CIO for quality decision-making, is typically cost prohibitive, and the RFI process addresses only snapshot moments in time, and therefore cannot keep current. However, a shared, external lab, or collaboratory, can dynamically address all the key information in a timely fashion. The foundation for this potentiality is simple synergy. By sharing in the information-input process, using a win-win model, everyone involved gains more than they contribute. In this model everyone involved wins.

This sharing process, like any multi-human social activity, requires mutually accepted language and fair rules to preserve harmony and prevent abuse. In today’s high-stakes competitive IT climate, fair play may seem idealistic if not completely elusive.

Nevertheless, isolated examples abound where accords have been reached, with some of the industry’s most open results, such as the case with Linux.

Of the approaches described above, the "collaboratory" model for coping demonstrates the most promise and the greatest potential short and long-term ROI. Recognizing this, many of the most forward-thinking IT players are coming together to link the IT supply chain under a non-profit umbrella called the "Interoperability Clearinghouse" (ICH). Based on Commercial Best Practices found in Boeing, EDS, Ernst & Young, MITRE, GM, Citigroup, DoD, and other market leaders, the ICH has collectively established an architecture driven COTS research and validation methodology and associated a secure e-solutions knowledge base that effectively addresses the fast paced market, and provides in context research results to members in the IT value chain.

In this white paper, we will provide a high-level analysis of the factors driving the ICH’s Architecture Validation methods, collaboration model, and mechanisms for sharing results within the IT value chain. It will provide an understanding of the automated process for capturing the knowledge base and keeping it current is necessary in order to accept its credibility. Additionally, to leverage this collaboratory, one must have a clear understanding of where their "link" contributions fit into the overall value chain. Additionally, it is absolutely critical that the participants have a common language for communicating, an architecture lexicon. We are leveraging the internet to empower our community of interest. We are leverage the internet and open systems to further our common goals. We are eating our own dog food!

Unfortunately, many past attempts at mapping or standardizing IT taxonomies and lexicons have failed. One major reason why this is the case is because the language of IT continues to expand rapidly and unpredictably. This is another area that the methodology addresses very effectively by using a set of "living" common criteria categorized by technology domain. Thus, the common architecture taxonomy necessary to use and assimilate the information in the clearinghouse repository grows readily with the data repository.

Through many man-years of architectonics research and development, the collaboratory architecture validation methodology has evolved into a proven process that truly addresses the rate of technology change and high complexity of "e-architectonics". The methodology is based on the fundamentals of the Reference Model for Open Distributed Processing (RM ODP), and takes advantage of several primary sources of validation data:

The common criteria used in the ICH validation methodology represent both the implicit and explicit development needs of multiple projects collected and refined over time. The initial criteria were based upon an evolution of the generic taxonomies now in widespread use. For example, IEEE 1471 and ISO RM-ODP (reference model for open distributed processing) both establish a taxonomy for information technology standards that applies generically across all domains. Similarly, the National Institute of Standards and Technology (NIST) has established the Application Portability Profile (APP) which is a cross-domain assemblage and categorization of technology standards. We are seeing government and industry pay more attention to architectures as a means of model their business and developing meaningful implementation plans. After all, architectures provide the means of communicating business needs to the IT community. The ICH validation methods enables the conversion of these architecture blueprints into implementation reality. It allows us to make in-context technology decisions based on our business needs vs who has the best marketing hype!

These initial criteria evolve constantly in the collaboratory to keep abreast of new capabilities or features as they become common factors of a technology domain. The net criteria are representative of the needs of the development community, and can be selected according to an individual situation's needs.

The Impact of Common Criteria on Interoperability

Technical solutions are most usefully modeled and managed in terms of product technology categories that describe discrete features, functions and interfaces in an vendor neutral, unambiguous manner, and establishing common criteria that maps business needs to these solution sets. Obvious examples of categories include databases, firewalls, and operating systems. More recent category examples include distributed object transaction middleware, public key infrastructures, and portals.

Typical common criteria sets that have been developed and are maintained by the collaboratory include:

Categories are constantly being defined and redefined as the market evolves. The features and functions that define a category are defined by what is currently considered competitive in the market. Standards have an important role in defining new market categories where none existed previously. However, multiple and diverse standards may apply to any given product category.

As categories mature, they tend to merge previous functions into new categories or fade away completely. We certainly see this happening in the area of middleware, where basic object-request-broker functions are being absorbed into the operating system and are increasingly considered an integral part of related technologies, such as Web Application and Distributed Object Transaction Servers.

Using the Common Criteria to Clarify Real Contextual Need

Once identified, individual criteria elements for these domains are assigned a weighting factor representing the level of criticality to the success of the proposed e-architecture in a given specific context. The basic methodology weighting factors are:

By leveraging the common criteria for a specific category of product, you can be more certain of addressing the complete range of capabilities for any given technology domain. This provides a very useful mechanism for "warding off" high-power marketing pressure of the "we have more features" ilk. When faced with such a tactic, the weighted common criteria provide an anchor point of sensibility based on pre-considered, in-context functionality needs that effectively nullify market hype.

The following example is an extraction from the collaboratory’s common criteria for Relational Databases:

Extracted Common Criteria Elements

Maintenance and extension of the common criteria requires the ongoing participation of persons deeply familiar with the taxonomies behind the criteria. Such a person, possessing great intuition borne out of years of experience in specific technology domains, is critical to the correct classification and normalization of such a taxonomy. This is an additional benefit of the collaboratory approach, since participants include many renowned IT thinkers. Moreover, any isolated organization attempting such a feat faces great challenges in gaining the type of voluntary consensus that already exists within an established collaboratory.

The LEXICON Creation Process

The ICH method for creating each technology lexicon assures that the selection criteria established is based on known capabilities in today’s market. Each technology lexicon defines key features, functions and interfaces (including dependencies), in normalized and standards/product neutral terminology. It identifies those attributes that are based on standards, and those that are based on commercially available products. This prevents the development of COTS selection criteria that are not supported in commercial products (null set). It also enables software vendors and solution providers a mechanism for describing their offerings in a well understood, engineering oriented specification. Vendors are encouraged to "map" these normalized attributes to their own product documentation via an XML template. This also enables product testers, integrators and users an easy way of validating vendor representations and update the ICH e-solutions knowledge base.

A Lexicon is typically developed in concert with technology research organizations and publications who have a deep understanding of a specific technology segment, and are willing to develop these "profiles" in an unbiased format, and in business terms. Each lexicon has an architecture "view" that enables the user to "drill down" into one of the following;

The technical architecture (implementation blueprints) for any enterprise is typically in a constant state of flux as it continues to grow and adapt as needed to support specific technical needs. Functional and technical requirements are therefore in a state of continual evolution. Accordingly, it is necessary to make a number of assumptions related to the architectural direction in order to proceed with the any on-going enterprise-system programs. These assumptions must also be well documented and clearly understood as valid common criteria for future decision making.

Furthermore, assumptions made must be based, in part, upon a review of related architectural efforts to date. This usually requires at least an informal, targeted architectural baseline effort. Nevertheless, typical assumptions are:

For standards to be successful, they must be widely accepted and implemented. A standard that is perceived as useless, impossible, or worthless serves no useful purpose, even if it, in truth, has none of these negative characteristics. Thus, it is critical to obtain "buy-in" from key stakeholders, usually through collaborative authoring and review, such that these stakeholders commonly share a positive perception of a standard. This requires deft inter-organizational and inter-personal skills and an extremely open and technically excellent verifiable methodology.

When working with standards it is also critical to be adept at navigating the layered and complimentary nature of related standards. Thus, one who would propose new standards, or changes to existing standards, must carefully avoid the synchronization challenges that can result from redundancy across related standards and the confusion potential of conflicting or contradictory standards.

Standards go a long way toward interoperability, but as many who have tried to "buy CORBA" have learned, you can't just buy a standard off the shelf -- you can only buy products, which may implement (or partially implement) standards. Thus, to get the most out of standards, you really need a host of other implementation data as well - such as:

Regardless of how you source your architecture and technology research and validation efforts, you cannot succeed without valid data and an accurate requirements context in which to apply this knowledge to make sound IT decisions. Success today is multifaceted; it requires an understanding of your business (operational architecture), the implied technical requirements (systems architecture view), your technical constraints and standards (technical architecture view), and eventually, your implementation blueprints (implementation architecture view). Without these views, success is truly a crap shoot, and rarely repeatable.

IT Supply Chain Information Sources

To distil useful technical data from the deluge marketing literature and to minimize the time it takes to gather that data, it helps to understand the original information sources of the literature and their motivations. If we are to cope, we must engage the "owners" of these architectural artifacts and work in a singular context, or lexicon. So, lets look at what data we need, and how we can bring it all together.

Upon examination of the collaboratory model from the perspectives of the participant stakeholders necessary to realize it, we have critical need of certain kinds of up-to-date, unbiased, and accurate information. Some of these data types and typical sources are:

IT Supply Chain Perspectives & Motivations

Each of the supply chain players is driven by their own distinct marketplace forces. Let’s examine some of them:

Vendors know about their products. Typically, vendors spend a great deal on marketing attempting to educate potential customers and integrators about their products so that these players will recognize where the vendor’s products can meet their needs -- leading eventually to product or service sales. This highly generalized activity typically involves advertising, trade-shows, and direct marketing.

There are several problems with this (current) model:

The Integrator is typically closer to the user problem. This is because most integrators have developed expertise in both technology and industry domains. When making enterprise decisions, it is important to map out both the technology direction and problem domain. This requires experience in both of these areas – the experience typified by the integrator. The pressure to find very relevant integrator experience has created an entire new market of consultancy boutiques that are very specialized. Given the relevant success rate of these new boutiques, many large consultancies have organized into several small divisions which look and feel "boutique-like".

Monolithic testing organizations have appeared on the scene at various times claiming to be "the" source for validation or conformance testing. We have learned from the repeated failures of these attempts that they do not work. For example, the Corporation for Open Systems (COS) tried and failed to become the central test and validation point for pieces of the networking layers. They failed because they could not keep up the demand or rate of change while also sustaining viability in the marketplace. The same is true for the US National Institute of Technology (NIST) FIPS program, which provided government funds for testing standards and interoperability.

Non-monolithic testing labs are a different breed, however. These organizations focus on fulfilling a legitimate, marketable need in the marketplace for quality, certified testing. Where the monolithic organizations have failed, these focussed, contracted testing companies can and do thrive in a marketplace so fraught with risk of failure.

Syndicated research firms provide traditional research products. This kind of information is typically based on the opinions of consulted experts in specific domains, and can be highly valuable. However, IT decision-makers must use caution as this information is not a substitute for actual implementation data that can be provided by collaboratory consortia such as the Open Application Group (OAG) or the Interoperability Clearinghouse (ICH) which do provide in-context, implementation-based research.

Standards bodies survival depends on the utilization of the standard that they present. Typically, utilization of a standard can save an implementer from many of the pitfalls that the standards originators have already experienced and dealt with. This kind of pitfall prevention, as well as a desire to minimize brittle stove-pipe trends provide justification for users and implementers to financially support such bodies.

By participating in a collaboratory, standards bodies can gain exposure to more potential supporters as well as gain collaborative support for improving their actual standards.

It is the user’s needs that justify any expenditure on IT, and the user’s money that funds the entire IT food chain. As explained in the Vendor Perspective, it is the user’s response to the marketplace that eventually determines marketplace behavior. If user’s make decisions based on valid technical criteria verses "best marketed" criteria, the marketplace will respond with better technical information and solutions.

In the collaboratory model, the user provides the motive for all other participants to do their part. It is the user’s searching of the collaboratory database for products that motivates the vendor to keep that data up-to-date. It is the user’s search for integration assistance that motivates the integrator to document their integration success data, which in turn provides validation and strength-of-evidence data to support vendor-claimed functionality. It is the likewise, the user who seeks targeted testing information that provides motivation for testing organizations to document their testing results in the collaboratory.

An important consideration of a methodology-driven evaluation effort is to ensure assessed products not only make maximum use of existing investments and capabilities, but also allow for a smooth transition toward a targeted architecture. This glide path should account for evolving standards in the contextual domain for both "in use" and "preferred" components, products, tools, and methods.

Development environments are frequently viewed in terms of several broad objective requirements are:

The result of a methodology-driven effort is a set of detailed assessments of IT products not eliminated through the process (because they were not viable or not appropriate, based on context-weighted criteria matrixed across the entire spectrum of potential domain offerings). These assessments present the results of extensive collaboratory knowledge, technical literature reviews, tools market surveys, and product research based on a weighted evaluation criteria developed within the user’s real context.

It is these evaluation assessments, based on generalized criteria weightings that we will share in the following CIO Magazine articles. The preceding material provides the details necessary to understand the process used to obtain and maintain these assessments and to back up their credibility.