Scaling Agile with SpiraPlan

Overview

The adoption of agile methodologies in the software industry has enabled teams to radically improve quality, reduce waste and deliver the right features on time. However, as agile is being used in larger organizations and to support more complex programs, beyond business units that create the software, its inherent limitations are requiring companies to think about how the scale agile for these new constantly emerging situations. In this whitepaper we outline how the SpiraPlan will enable organizations to meet the challenges of scaling agile, both now, and in the future.

Note that we use the term “scaling agile” rather than “Scaled Agile” since the latter has become associated with a specific framework (SAFe), and SpiraPlan has been designed to support many different frameworks allowing organizations to adopt them into methodologies that address their challenges, not just SAFe.

Why Do We Need to Scale Agile?

Before we go any further, the first thing we need to understand, is why scale agile in the first place? If agile methodologies are delivering better results (faster releases, higher quality and more flexibility) than traditional waterfall approaches, what reasons lead organizations to try and scale them.

What has happened is that agile has become in many ways a victim of its own success. As organizations use agile to manage increasingly large, complex and critical software development initiatives, the limitations and weaknesses are becoming more apparent. In this section we will discuss some of the limitations, and how you can “scale” agile to address these limitations.

Scaling to Deal with Size

The first dimension we’ll be looking at is size, scaling agile from a single product team, all the way to a large enterprise of many product teams, grouped into solutions, programs and portfolios, possibly working with other similarly configured organizations.

Single Product Team

When the original agile manifesto was published in 2001, inheriting different schools of thoughts, such as those from development of Scrum envisioned in 1986 and later eXtreme Programming (XP) in 1999, the focus was largely on small interdisciplinary teams, working in a physically collocated space, with continual access to the customer and a dedicated product owner competent in business and technology ready to give near real-time feedback on features/requirements called user stories.

In this initial configuration, the key tenets of the agile manifesto (early feedback, sustainable pace, continuous integration, seamless configuration, cross-discipline teams and an emphasis on continual testing) are delivered in a simple way. The product team gets its user stories from the product owner who develops the user stories from the features articulated by the. The team rewrites or splits these user stories and develops additional user stories either on physical story cards or uses an electronic story board (also known as a board or Scrum board in Scrum).

The team, guided by the Agile Coach or the Scrum Master, prioritizes the user stories from the product backlog, and delivers them in a series of small (often 1-2 weeks) increments, called either iterations or sprints. The code is integrated and (ideally) deployed continually, with a mixture of automated and manual tests performed to ensure that each release is of high quality and does not introduce quality regressions from the previous release. These processes are often called Development Operations (DevOps).

Large Solution

The first step up in terms of scaling agile is the ability to have multiple teams working on either different modules of a larger, integrated system (called a product or solution), or multiple different projects that have a common set of business objectives (called a program).

Regardless of the exact configuration or terminology, each of these teams is no longer independent of each other. That means there may not be a single customer or product owner for the entire system. It also means that the prioritization and scheduling of specific requirements in one team will affect the other. For example, one team may rely on an API that is being delivered by the other team.

In this situation, you will now have a new organization that is employed to coordinate and manage the different teams, the Program Management Office (PMO). The job of the PMO is to coordinate the different teams / products in the program and ensure they are working together to deliver an integrated solution that delivers the expected business value. The PMO is also tasked with making sure that any risks or impediments raised by one team that will affect other teams, are addressed.

With this larger organization, the same agile principles are followed, but the mechanisms and tools may be more formal. The teams need tools that let them communicate with the other teams and the PMO. The PMO needs processes and tools to keep track on the prioritized work, risks, and impediments. Furthermore, the teams may be self-organizing, but the dependencies of working with other teams means that more top-down decision making will be needed. Similarly, on the engineering side, more complex development “pipelines” will be needed to integrate the different products together, and more integrated automated testing will be needed to cover the business scenarios and use cases that span the different product teams.

Programs and Portfolios

The next level of size to deal with is the large program or portfolio (or both). In general, a program is used to describe a common set of high-level business features and use-cases that will be delivered to a customer. Generally, the products that comprise a program, will have technical interdependencies (e.g., system A needs to connect to system B for a specific transaction), and business dependencies (e.g., there is no point having a scheduling tool if there is no system that is able to be scheduled yet).

Conversely, a portfolio is considered to be a collection of products and/or solutions that may belong to a specific department, deal with a common customer, have a single budget line or business owner, but otherwise are independent of each other. For example, you might have a portfolio consisting of all consumer-facing web apps.

Project Portfolio Management (PPM) is the centralized management of the processes, methods, and technologies used by portfolio, program, product, and project managers. These techniques can also be used by business analysts, system analysts, scrum masters, and agile coaches. Both plan-driven and change-driven approaches benefit from the centralized management processes, methods, and technologies provided by PPM. The managers use PPM to analyze and collectively manage a portfolio of current or proposed projects based on numerous key characteristics such as risk, price, and business need.

When managing a portfolio of systems and products, there is a tendency to fall into the micromanagement trap of top-down decision making, which goes against the principles of the agile manifesto. They key to successfully using PPM techniques is to understand what decisions should be taken at each level (portfolio, program, product and team). In addition, the tools and systems used to communicate upwards and downwards (between these levels) should reflect the information needed by each stakeholder. It is worth noting here that the agile manifesto does not eliminate processes and tools.

For example, the Vice President (VP) in charge of a portfolio of systems should not care that one task, associated with one user story is late by X days, or that a specific team member has an impediment working on a particular feature. Similarly, there should not be an expectation of task dependency (the Finish-to-Start thinking) or lead/lag considerations in an agile product, as these considerations do not promote self-organization.

However, key information such as project risks and major changes in scope should not be isolated to the team level. Conversely, the VP needs to have the ability to reassess the priority of a specific product/system in the portfolio and reassign resources, change the scope and even cancel a product if it will not meet the needs of the organization.

Enterprise Agility

The next level of scaling by size, is to the entire enterprise, with the product teams working in solutions or programs, organized into portfolios. This enables the company to deliver on its strategy and business objectives, while supported by the infrastructure, culture, and core values of the organization.

In this context, the agile teams will be providing high-level metrics and key performance indicators (KPIs) to the executive team of the organization. Not only that, but they will provide equally important metrics to supporting shared services such as IT, HR, marketing, staffing, and customer support, so that the entire enterprise can deliver on its promises.

For example, knowing that a specific feature is on track for release by a certain date will impact the marketing of that feature, the training of customer support agents, upgrades to IT infrastructure, as well the ability to meet company financial targets.

Networked Agile Supply Chain

Many of the classic methodologies and frameworks for scaling agile will stop at the enterprise or organizational level, however that is not the full picture. Most companies that develop software will rely on some components and systems delivered externally by their suppliers, and many will supply other organizations (their customers) with systems that they in turn integrate and combine with other products.

In this regard, the development and delivery of software resembles that of a typical supply chain, where companies may even supply products, which can include consumable software components like access to application programming interface (API), to multiple competitors, or have multiple competing suppliers delivering them the same component.

Just like in the logistics world, where information replaced inventory to create Just In Time (JIT) supply chains, this concept of an integrated agile software supply chain is changing how software is developed and released.

Previously, each company would define its own requirements (e.g., user stories), develop its code, do its testing and release the product to its customers. Its customers could independently test the product, provide feedback on its functionality and log defects. However, we are now entering the era where a customer demands visibility into how the software is developed, how it was tested, and even visibility into what the requirements were in the first place.

What this means is that our methodology, processes and tools for managing the projects needs to account for a whole new set of stakeholders and stakeholder groups, like regulators, in both directions. A product team needs to be able to have visibility into the DevOps toolchain of its suppliers, and provide similar visibility of its DevOps toolchain to its customers. Since many of these customers and suppliers may be competitors, the flow of data may need to be segregated and partitioned.

Scaling to Deal with Complexity

The second area where agile methodologies can run into problems is related to the requirements themselves. Ideally, the requirements of a system can be described as a series of functional and non-functional user stories and/or Behavior Driven Development (BDD) scenarios that are simple to understand and communicate the functionality from an end user’s view. For example, an insurance claims system, might have a story: “As a customer I want to be able to file a claim for an auto collision on the website so that I start the claim processing immediately”.

However, most IT systems have to work in the world of laws, regulations and policies. So, in addition to this user-centric requirement, there is also probably a large body of rules and regulations for insurance claims processing (that will vary by country and state) that the system will need to follow. Disciplined Agile recognizes seven important scaling factors where the domain complexity, solution complexity, and regulatory compliance landscape are also critical in today’s digitally transformed society among other things. Therefore, lack of attention to these scaling factors leads us to the “requirements gap” dilemma.

The “Requirements Gap”

When we have interviewed customers about how they develop software, we have found that the product teams are working primarily with user stories and BDD scenarios, managing them in a Scrum board of some type. Conversely, there is a separate “business analysis” (BA) team, staffed with domain/industry experts that manage a much larger set of formal requirements in documents, spreadsheets or separate requirements management systems.

In almost all cases, there was not good coordination or visibility between the requirements in these two different groups. The BA team would refer to their documents as “their requirements” and the product team would refer to their user stories as “their requirements”. However, in the end, the system being developed needs to fulfill both sets or it will not be successful. So, one goal of scaling agile is to have a single set of “requirements” that the different teams can work from.

It is indicative that whereas traditional agile methods such as Scrum or Kanban tend to eschew the term "requirement”, calling them backlog items or user stories, SAFe has decided to bring back the term requirement.

The Minimum Viable Product (MVP) “Trap”

Another key agile method is the creation of the Minimum Viable Product (MVP). The idea is that you need to develop just enough functionality to prove out a new concept before spending lots of time and money building a production system. This is a great idea, since it avoids the waste of building a system that no one will use (a common problem with waterfall projects).

When developing an MVP, the idea is to prove out all three areas of possible risk to creating the full system:

However, one of the pitfalls is that instead of using the MVP to explore the technical areas of risk and validate the schedule assumptions, the MVP becomes exclusively focused on showing the possible features. This is great in that it will validate the concept makes sense, but the danger is that the customer/product owner likes what they see, but the technical architecture needed to release a production version is not well understood, and the schedule for releasing it is unrealistic. Eventually, the product released is either not secure, stable, scalable, serviceable, or sustainable!

This will lead to a delayed or canceled project and could be avoided with a better designed MVP. We call these badly-designed MVPs, minimal non-viable products MNVPs instead!

Risk Management

Finally, one area that is absent from most agile methodologies is an appreciation of risk management, and how one of the key information flows between the different stakeholders in a large organization (e.g., between the executive leadership, the PMOs, and the product teams) is around risks.

Key decision makers don’t necessarily need to know every single decision made on a product team, but they do need to understand the impact of key technical, security and business/financial risks on customers, the business, the team and non-human resources, such as the facilities, equipment, materials, infrastructure, and supplies.

Scaled Agile Methodologies

In this section, we give a quick overview of the different methodologies available for helping you scale agile. We have a more detailed article on the subject – Scaled Agile, learn the methods and tools.

Scaled Agile Framework (SAFe)

The most common implementation of scaled agile is the SAFe methodology. The Scaled Agile Framework (or SAFe) is a software development framework that takes the concepts of agile development and provides a “larger picture” methodology that allows you to use agile approaches across an entire enterprise.

(Image is Copyright Scaled Agile Inc).

SAFe comes in four main configurations, for use in different contexts:

• Essential SAFe is the most basic configuration. It describes the most critical elements needed and is intended to provide the majority of the framework's benefits. It includes the team and program levels.

• Large Solution SAFe allows for coordination and synchronization across multiple programs, but without the portfolio considerations.

• Portfolio SAFe includes concerns for strategic direction, investment funding, and lean governance.

• Full SAFe combines the other three levels into a single methodology.

For more information on these individual SAFe configurations, see our guide here.

Other Scaled Agile Methods

Although SAFe is the most familiar methodology for scaling agile, it is by no means the only one. Some other popular scaled agile methods not listed in any priority order include:

• Large Scale Scrum (LeSS)

• Scrum of Scrums (SoS)

• The Nexus Model

• Disciplined Agile Delivery (DAD)

• The Spotify Model

For more information on these different methodologies, see our guide here.

Scaling Agile with Spira

So now that we understand the challenges in using agile methodologies at increasing scale and complexity, in this section we will discuss how SpiraPlan has been designed to make this a seamless process. You will notice that we have not designed SpiraPlan to be limited to any one scaled agile methodology, and consequently we have decided to use terminology in SpiraPlan that is easier to understand and use. We will provide a mapping between SpiraPlan and the different frameworks later on.

Vertical Scaling with SpiraPlan

We will first be looking at how SpiraPlan lets you scale “vertically” from a single agile product team to an organization of portfolios, programs and products.

Products

SpiraPlan has been designed from day one to support a rich hierarchy of requirements. This flexible hierarchy lets you create a product backlog consisting of as many levels of requirement as you need. In a typical scaled agile product, you may have: Epic > Feature > Story as illustrated below:

Each product in SpiraPlan is designed to be a long-lived entity that contains multiple different parallel releases of the system, being developed in iterations or sprints and released in coordinated releases to the end customer. Each release, phase, or sprint acts like a project with defined start and end dates delivering a unique product increment, service or result, consistent with the definition of a project, within the larger product lifecycle.

You can assign user stories to each of the sprints, with the status and progress of each story will then update the status/progress of its parent feature and epic. Similarly, the status of each sprint/iteration will update the status of the release that sprint is a part of.

So, what happens when you have two products that each have their own backlog, and need to coordinate? SpiraPlan provides the cross-product association functionality. But what happens when these two product backlogs need to be managed as part of an overall program:

That’s where the new program-level artifacts come in.

Programs

As part of the planned work in 2023 on implementing the higher-levels of scaled agile, SpiraPlan is introducing new program-level artifacts called:

• Capabilities

• Program Milestones

The relationship of these is illustrated below:

Each program has a set of key business capabilities that need to be prioritized and delivered by the various products in the program, we call these capabilities. Similarly, the program manager needs to be able communicate at a high-level by what date that new business feature will be delivered, we call these program milestones.

Note that a capability should be much higher-level than a product feature, and a program milestone will be much less granular than a product release or sprint. In SpiraPlan, program managers can create a two-level hierarchy of capabilities, in the example above we have called them program epics and features, but you can choose the name that makes sense to your organization.

Now, each of those program capabilities will be delivered by one or more product requirement. For example, a capability for allowing banking customers to check their bank balance online, might map to an enabling requirement in the web banking product and another requirement in the mobile app banking product.

Similarly, the dates of the program milestones will be influenced by the dates of the associated product releases. If a product release gets delayed, it will necessarily impact the program, milestone it is associated with.

However, the progress against the program milestone will be measured by the completion of the contained capabilities, rather than directly from the product releases. This is important because the capabilities determine which specific features in a release are important to different capabilities in the program.

But what happens when we have many programs and we want to understand how they support the company’s overarching business? That’s where the new portfolio-level artifacts will come in.

Portfolios

As part of the planned work in 2024 on implementing the highest-levels of scaled agile, SpiraPlan is introducing new portfolio-level artifacts called:

• Strategic Outcomes

• Portfolio Milestones

The relationship of these is illustrated below:

Each portfolio has a set of key strategic outcomes that need to be achieved by the various programs. Similarly, the executive in charge of the portfolio needs to be able communicate at a high-level by what date the strategic outcome will be delivered, we call these portfolio milestones.

Note that a strategic outcome should be much higher-level than even a program capability, ideally written so that members of the C-suite can understand their importance. Unlike program capabilities and product requirements, there is no hierarchy, since the list should be much smaller.

Each of those strategic outcomes will be delivered by one or more program capability. For example, a strategic outcome to have the best online banking experience for customers might map to the different capabilities of being able to manage balances online, transfer funds, apply for a loan, etc.

Similarly, the dates of the portfolio milestones will be influenced by the dates of the associated program milestones, which is in turn affected by the dates of associated product releases. If a product release gets delayed, it will necessarily impact the program and portfolio milestones it is associated with.

However, the progress against the portfolio milestone will be measured by the completion of the contained strategic outcomes, rather than directly from the program milestones.

Mapping to SAFe

Although we have deliberately not used exclusively SAFe terminology in SpiraPlan because we believe that the artifact types and names should be methodology independent, it is useful to see how SpiraPlan’s key artifacts align with those described in SAFe:

(Image is Copyright Scaled Agile Inc).

SpiraPlan provides the following support for Full SAFe:

Full SAFe	SpiraPlan Artifact	Artifact Sub-Type
Value Stream	Portfolio
- Strategic Theme	- Strategic Outcome	Strategic Theme
- Portfolio Backlog	- Portfolio Backlog
- Epic	- Parent Requirement	Epic
- Feature	- Strategic Outcome	Feature
- Enabler	- Strategic Outcome	Enabler
Solution Train	Program
- Solution Backlog	- Program Backlog
- Capability	- Capability	Capability
- Enabler	- Capability	Enabler
- Program Increment	- Program Milestone
Agile Release Train	Product
- Product Backlog	- Product Backlog
- Epic	- Parent Requirement	Epic
- Feature	- Parent Requirement	Feature
- Enabler	- Parent Requirement	Enabler
Team	Release	Major Release
- Team Backlog	- Release Backlog
- Iteration	- Release	Sprint
- Story	- Requirement	User Story
- Enabler Story	- Requirement	Enabler
- Task	- Task
- Acceptance Test	- Test Case	Acceptance
- Unit Test	- Test Case	Unit
- Component Test	- Test Case	Component

Horizontal Scaling with Spira.Net

The one remaining area to consider is that of the networked software supply chain we described earlier. This is an example of ‘horizontal scaling’, where you want different organizations to be able to share information at a product/team level so that different teams (from different organizations) can collaborate on shared projects, or be able to direct the efforts of their suppliers.

SpiraPlan will be solving this challenge in 2023 with the introduction of the new Spira.NET platform:

This new platform will adopt a ‘loosely coupled’ distributed agile approach, where the different product teams can work in their own separate Spira instances and yet be able to share backlog items and other artifacts.

The beauty of this approach is that you can still scale vertically within each organization from a management and reporting structure, but can collaborate and synchronize horizontally at the same time.

One very common use case for this approach is that of managed service companies. The team that is providing the managed service is just one of many teams that the service provider has working at any one time, and the customer may have many professional service organizations working for them at the same time. Both organizations can use this approach to have a top-down view of their own businesses, but able to have the outsourced team work collaboratively with the client team.

Value Stream Management

Value Stream Management (VSM) is a method that allows you to optimize the steps necessary to envision, implement and deliver software to your customers. Value Stream Management is a management technique or practice - common in Scaled Agile frameworks such as SAFe and Disciplined Agile Delivery - that focuses on increasing the flow of business value from customer request to customer delivery. Its systematic approach to measuring and improving flow helps organizations shorten time-to-market, increase throughput, improve product quality, and optimize for business outcomes.

In traditional organizational structures, individuals are organized into functional teams, and then temporary teams are formed to meet the demands of specific projects, borrowing individuals (or percentages of individuals’ time) from those functional teams. The temporary nature of the teams and organizations means that no one is focused on the long-term creation of value, no one has the overarching business goal in mind, and the opportunity to retain and grows skills in the team is lost.

What is Value Stream Mapping

Value-stream mapping is a lean-management method for analyzing the current state and designing a future state for the series of events that take a product or service from the beginning of the specific process until it reaches the customer. The purpose of value-stream mapping is to identify and remove or reduce "waste" in value streams, thereby increasing the efficiency of a given value stream. Waste removal is intended to increase productivity by creating leaner operations which in turn make waste and quality problems easier to identify.

In order to evaluate the effectiveness of your software practices and tools, you need total visibility throughout the software delivery life cycle. Therefore, value stream mapping should be a part of the overall lean approach that allows you to analyze the current state of your processes and build a better future state based on the series of events that take your product/service from initial concept to the finished product that the customer receives. This eliminates waste and allows you to prioritize based on your customers' needs.

There are two types of value streams – operational value streams and development value streams – that should be considered when looking at value stream mapping.

Operational Value Streams

Operational value streams are a series of steps used to provide goods and services to a customer, either internal or external. This is primarily how a business makes its money

Development Value Streams

Development value streams, are a series of steps used to develop new offerings. So, relating them to the bigger picture, development value streams build the systems that operational value streams use to deliver value.