Complexity in State Government
Note: For privacy reasons, our client is referred to here as STATE (a state government in the United States) and the project is referred to here as PROJECT.
In 2007, STATE began contemplating a project that could easily have been doomed from the start. It was to replace its mission critical, aging and highly fragile PROJECT software system, a project whose cost would likely exceed $100M dollars.
Many states have gone through similar efforts. The traditional approach to this problem is to use a consolidated procurement model (CPM). In a CPM, a single huge Request for Proposal (RFP) is created and sent out to vendors who then respond. Eventually a vendor is chosen, a contract negotiated, and delivery commences.
STATE described this process accurately:
This traditional approach is fraught with problems at every stage of the process. First, the overwhelming scope of the problem virtually ensures that requirements will be poorly described or left out. Secondly, it is almost impossible for the vendor to accurately predict the cost of delivery. Third, once the system is delivered, it is almost certainly going to require a long period of contentious debate between the public sector organization and the vendor as to the success of the delivery.
Perhaps the biggest problem with the traditional approach is the excessive cost. There is a huge cost associated with creating the 500+ page RFP. The vendors who are most likely to respond are exactly those with the largest overhead. Because the vendor cannot accurately predict the overall cost, the vendor will almost certainly pad its figures. Because the RFP is almost certainly incomplete with respect to business requirements, the vendor will have multiple opportunities to add charges for changes.
But the worst problem has to do with the final delivery. Even if the project is delivered on time (unlikely) and on budget (inconceivable), the delivery will almost certainly suffer from massive, choking complexity that will plague the lives of those that must use, maintain, and upgrade the system for many years to come.
STATE was determined to find a better way.
A Better Way
STATE had heard about a new methodology called Simple Iterative Partitions (SIP). SIP is based on the mathematics of partitioning theory. It offers a guided approach to identifying atomic functionality and deciding the best possible way to partition that functionality into smaller sub-systems. SIP answered the very problem that was consuming the PROJECT planning team: how to partition the large system into manageable smaller pieces.
The team described why they chose SIP as the guiding methodology in their final report:
There are a number of advantages to the STATE approach [using SIP]. Some of the most important are these:
- The collective system cost of all of the ABCs [small partitioned projects] is likely to be much less than the cost of the single monolithic system.
- STATE will have a far greater choice in vendors, since many vendors that could not take on the monolithic project can take on one or two ABCs. In particular, this opens up the bidding process to historically underutilized businesses, such as minority and woman owned businesses.
- The overall partitioned project will be much easier to manage than would have been the monolithic project.
- The final deliverable will be much more maintainable, securable, scalable, and will be more likely to meet the needs of the business.
In addition to sharing an understanding with SIP as to deliverables, STATE found itself in close agreement with many SIP core philosophies. Again, from the PROJECT Final Report:
All systems go through a number of phases including pre-design, design, implementation, deployment, and maintenance. SIP introduces three guiding principles to this overall framework.
First, complexity is the enemy. Complexity is bad and not to be tolerated.
Second, only business and IT working in close collaboration can eliminate complexity. Business understands synergies. IT understands partitioning.
Third, the sooner complexity is addressed, the easier it is to eliminate. The ideal time to address complexity is in pre-design. Once design has started, complexity is much more entrenched, and by the time implementation has begun, complexity is cast in concrete.
As much as SIP seemed tailor-made for the STATE project, the planning team had a justifiable concern. SIP had never been used on a project of this magnitude. What if SIP was overwhelmed by the very complexity it promised to contain?
To address this concern, the planning team decided to adopt SIP in stages. At the end of each stage, the process would be re-evaluated by both the planning team and the management and a go/no-go decision would be made about moving forward. Only if it was clear to all of the stakeholders that the SIP methodology was delivering measurable value at every stage would SIP be permitted to run to completion.
ObjectWatch, the company that supplies the SIP patented methodology, encouraged this staged low risk approach.
Overview of SIP
SIP is not a replacement for other software design methodologies. SIP addresses a particular problem in the IT design process, that of partitioning. Partitioning, as related to software systems, is the problem of dividing up a large number of functions into an optimal arrangement of smaller, autonomous systems. For very large systems, such as the PROJECT system, the number of functions is so large that any ad hoc approach to partitioning is virtually guaranteed to fail.
Most design methodologies recognize the critical need to do partitioning. For example, TOGAF 9®, the most widely used IT design methodology, says
In order to develop architectures that have manageable levels of cost and complexity, it is necessary to partition the enterprise into specific architectures.
While most methodologies recognize the importance of doing partitioning, none (other than SIP) offer guidance as to how this crucial step is to be accomplished. TOGAF, for example, punts on the issue:
It is impractical to present a definitive partitioning model for architecture. Each enterprise needs to adopt a partitioning model that reflects its own operating model.
This is precisely the problem that SIP attacks: analyzing a large collection of functions and determining the optimal partitioning in a manner that is verifiable, reproducible, and mathematically sound. And it does so in a way that is not dependent on the operating model of a particular enterprise.
Defining the Stages
At a high level, STATE laid out the PROJECT replacement project in the following phases:
Preplanning: Partitioning the large PROJECT replacement project into sub-systems that could be treated autonomously.
- Planning: Filling in details on each of the sub-systems.
- Procurement: Going through the RFP process for the sub-systems.
- Delivery: Acceptance of the sub-systems.
- Deployment: Making the PROJECT replacement live.
In this scheme, SIP plays its primary role in the preplanning phase. Once partitioning is finalized, the remaining steps follow mostly standard planning methodologies.
As mentioned, the preplanning phase was further divided into stages so that the value of SIP could be continuously tested and re-evaluated. These stages were as follows:
- Introduction/Project Organization: Workshops were held to explain the importance of managing complexity and to introduce the SIP methodology and the SIP related teams were identified and trained.
- Functional Decomposition: The collection of business functions that collectively would make up the PROJECT replacement was determined.
- Partition Simplification: The business functions were partitioned into sub-projects called, in SIP terminology, autonomous business capabilities (ABCs).
- ABC definitions: Finalization of the ABC definitions included documenting the dependencies between ABCs.
- Prioritization: The collection of ABCs was prioritized to determine the order in which they would be procured.
- Prototype: A technical prototype was created to demonstrate the technical feasibility of the SIP approach.
These stages were specifically segmented so that no one stage would cost more than $50K, ensuring that risk was carefully controlled at all times.
The first stage was the Introduction and Project Organization phase. As described in the Final Report:
STATE created Team SIP, as advocated by Roger Sessions, with two business experts, two technical experts and a neutral party, as the managers of the SIP effort. The SIP Review Group worked on internal training by analyzing the SIP methodology and by working with Roger Sessions on how to implement the theory. The Review group stayed one step ahead of Team SIP and the Core Team until the workshop held on June 23, 24 and 25, where the Review Group and Team SIP together decided how to group the functions defined in the functional decomposition into autonomous business capabilities. The Core Team, whose members are the business experts, put the theory into practice.
The second stage was the partitioning identification. As described in the Final Report,
The Core Team completed the first iteration of phase two by defining the functional decomposition, which decomposes functions into lower-level functions and the ABCs, that logically groups the functions. Team SIP completed the first iteration of identifying the functionality that will be implemented.
By the end of the project, STATE had identified approximately 2500 business functions partitioned into 240 ABCs.
In order to manage the large number of ABCs STATE followed the SIP recommendation to separate the ABCs into groups by priority. Most ABCs could be prioritized easily. Those that were less obvious were subject to a formal priority analysis using a series of questions and a radar graph analysis in which high priority ABCs had large “radar” areas and low priority ABCs had low “radar” areas. An example of such a radar graph, one with relatively high priority, is shown in the following figure.
Figure: Sample ABC Radar Graph
Initially, prototyping was not planned to be part of the preplanning activity. However the technical team was anxious to see how the SIP defined ABCs would eventually be delivered from a technical perspective. The technical implementation of an ABC is called a software fortress. The technical team decided to build some software fortress prototypes despite the fact that no funding had been allocated for this effort. Because of their enthusiasm to get started, they completed the prototype as a skunk works project mostly on their own time. Here is the final report’s description of this effort:
At the September workshop with Roger Sessions, the SIP group made its first foray into the technical area of software fortresses.
Autonomous, defined units of functionality that can translate into software fortresses provide the technical designers with clear, useful information. The pre-design work on identifying ABCs pays huge dividends once starting technical design.
The technical members of the SIP group were able to take four ABCs as examples and quickly design four fortresses with guards and envoys to send and receive messages. The communications between the fortresses were clear and could be translated into messages sent and received by a service bus. Some interesting discussions arose around the desirability of synchronous versus asynchronous requests, but overall the SIP group agreed that ABCs-as-fortresses was a huge improvement on a monolithic design that normally would have had to be translated into technical specifications. Overall, the technical staff felt that the concept of software fortresses was natural and similar to their current thoughts on good software design.
The STATE team was pleased with the result of the SIP work. The partitioning yielded a project plan that was radically different from that used by other public sector organizations. Rather than create a single massive highly complex project that would be bid out to a single large consulting organization at a cost of at least $100M, STATE ended up with a collection of small projects averaging about $200K that could be bid out to small local consulting firms.
STATE identified 240 ABCs. While 240 small projects may seem like a daunting number of projects to track, the management task was greatly simplified by these factors:
- The ABCs were prioritized, so that project management could be phased in over time.
- The ABCs often had logical clusters, so that multiple ABCs could be bid out together.
- The dependencies between the ABCs were minimized by the SIP process, so that each project could be treated autonomously.
- ABCs could be managed either concurrently or sequentially, allowing great flexibility in deciding how ABCs would be delivered.
As the STATE team described it in their Final Report,
The value of the first four phases of SIP is now apparent: The technical experts can implement an ABC as an autonomous software fortress that will communicate with future fortresses in well-defined ways. This implementation will not be constrained by the complexity of all the PROJECT functionality. Instead it can be developed independently, provided that the technical infrastructure allows interoperability with future implementations.
At the end of the pre-design project, PROJECT has a description of the business function relationships with synergistic functions grouped into logical units. PROJECT also has an understanding of how these logical groups could develop into independent projects in the design phase. STATE is well positioned to embark on a replacement system project that is simpler because it is broken down into manageable pieces.
The difference between a single project exceeding $100M and a number (even a large number) of small $200K projects is important from a risk perspective. There is ample evidence that projects in the $100M range have very low success rates. Projects in this range have less than a 5% chance of being on budget, on time, and delivered with all important functionality properly implemented. Projects in the $200K range, on the other hand, have high success rates. These projects have a greater than 95% chance of being delivered on budget, on time, and with all important functionality intact and properly implement. So by using SIP partitioning, STATE increased its chances of success from less than 5% to more than 95%.
In addition to increased chances of success, there were a number of other less tangible benefits of the SIP methodology that soon became apparent. For example, many more vendors could now compete for contracts. As stated in the Final Report
Each ABC is a small unit of functionality that can be implemented autonomously. Vendors will be informed of the need for interoperability with other ABCs, as indicated by the partner relationships. This allows STATE to create smaller RFIs and/or RFPs and it allows more potential vendors to respond, because the vendors don’t need to be large corporations who have the resources to commit to writing responses that can take months.
One of the most unexpected intangible benefits of SIP turned out to be how much PROJECT learned about itself. As the team explained in the Final Report
After eight months of SIP, the facilitators had learned an enormous amount about the PROJECT business, much of the business was documented in a useful, meaningful but simple way, the business managers knew more about each other’s business and business and IT were still collaborating.
Requirements for Success
STATE cautioned others than there is no guarantee of success using SIP. The team identified a number of factors that they felt were critical to success with the SIP methodology. As the Final Report stated:
One of the biggest factors in STATE’s success was executive commitment. We had strong commitment from the IT Director, the PROJECT Director, and the CIO. This ensured that we could get the necessary commitments from those whose participation was critical.
Another important factor was in the willingness of the business and IT groups to work together. The SIP approach requires a close working relationship between business and IT groups. Although there was no history of these two groups working together here, there was also no history of strained relationships between these two groups.
Finally, there was a pre-existing understanding of the problems that complexity causes in IT systems. Because so many people had personal experience with these problems, there was a willingness to explore options to avoid these problems in the future.
The STATE team was transformed by their experience with SIP. They had a better understanding of their business, a closer collaborative relationship between the business and IT, and a new vision for how they could provide tangible benefits to the citizens of STATE both in reduced cost, better customer experiences, and directly support for the local economy. As stated in the PROJECT Final Report,
STATE is well positioned to embark on a replacement system project that is simpler because it is broken down into manageable pieces.
For the PROJECT team members in STATE, simple is no longer just a word. It is a core value. And SIP has helped them find it.
(Page last updated 6/6/2013)