I often find a common set of challenges faced by teams that are in the process of implementing Agile adoption. I’m not surprised though. So many things can go wrong due to the fact that Agile adoption requires involvement from all levels of the organization. The changes touch people, processes, tools and organizational culture.
The common misconception is that adopting Agile practice is simply another methodology that you can pick up and implement. The challenge with Agile adoption, though, is that the culture must change at the same time as the Agile approach. This is because Agile practices are founded on lean principles. Lean principles espouse
- creating value for the customer quickly;
- taking a scientific approach to understand challenges and solutions;
- developing incrementally and iteratively to learn quickly through fast feedback;
- understanding the concepts of flow and the benefits of optimizing the entire value stream;
- understanding the benefits of pull over push;
- and the pursuit of continuous improvement.
That was a mouthful, wasn’t it? Well, don’t worry. The point is that culture change is just as important if you want to implement Agile successfully, and within the bounds of its true spirit.
Other principles regarding building a culture of trust are also intertwined and important to Agile. We will discuss Lean principles further in another post, but it is important to understand that you cannot execute in an Agile way without understanding the cultural aspects required to really implement Agile successfully.
So here are some common pitfalls that I’ve observed over the years:
Maintaining a command and control structure
Agile teams work best in a full trust environment so that team members are given the authority to make decisions where the information rests. This means that decisions are made by involving the subject matter experts first, and not escalated to management immediately. Escalation of decisions delay progress while the team is waiting for a decision and can often be avoided if the team is provided the intent and entrusted to make the right decisions in support of the project. Often, the right subject matter expertise lies with a team member anyway.
Not optimizing the entire value stream
The value stream is the collection of all of the activities that must take place to deliver value to the customer. Similarly, a development value stream identifies all of the steps that the team must perform to implement the capability or feature. In this scenario, if the development team is delivering every two weeks, but the testing team requires another 6 weeks to test, then the testing team is a bottleneck in the development value stream since the team can only move as quickly as the slowest link in the chain. This is also important when you have teams of Agile teams working on a large solution. All the teams must work together and at a synchronized pace, or the solution will suffer from timing challenges and bottlenecks. As a result, developing a synchronized cadence among teams of Agile teams is critical to developing a “heartbeat” or rhythm.
Not taking the time to understand the problem up-front
There is a misconception by some starting out in Agile that you don’t need to perform up-front planning, and that documentation is not required. This is simply not true. Not true. There is a preference for working code over thorough documentation as is stated in the Agile Manifesto, but you still need documentation. In fact, you need to really take time up-front to understand the problem using a scientific approach. Understanding the problem helps define a clear direction for solving the problem and building a solution. Iterating through design and is performed incrementally.
Locking into a design up-front
In the traditional waterfall methodology, teams are asked to lock down requirements and design before work starts. But Lean-Agile principles encourage remaining open to design and flexible with requirements. We assume that we cannot know everything in the beginning of a project. So let’s design and building incrementally, gain fast feedback and learn. Then let’s adjust and refine our design as we learn more.
Following a traditional waterfall or predictive execution and reporting structure
Some Agile teams still must follow a traditional waterfall-like (or predictive) life cycle due to their organizational policies. As a result, they are forced to develop requirements and design documents that become shelf ware, or worse, tie the team to a design that will inevitably change. Management often holds the team to the early designs and the team spends an inordinate amount of time trying to explain changes to senior management.
To avoid this situation, work with your PMO to understand what benefits or information they are trying to receive by forcing the project into a specific life cycle. Having those discussions might lead to more creative ways to provide visibility and transparency into the project’s progress without having to recreate the traditional documents and reports (e.g., EVM reporting). Sometimes avoiding the organizational life cycle requirements is not an option, but there might be opportunities to tailor it or provide functional equivalents that meet the intent.
Acquisition stuck in the traditional contracting approach
Similar to the early lockdown of requirements and design that you see in more predictive life cycle approaches, many acquisition teams also develop contracts that hold the vendors to the early requirements and design. But as the team determines that the design must change, the contract becomes one more challenge. Ultimately, by the end of the contract, the vendor and the organization end up haggling over the scope of work. In the end, nobody seems to be satisfied.
To avoid this situation, encourage the acquisition and contracting teams to be more flexible in how they issue work. Explain to the acquisition and contracting teams that design will not be locked down initially. There are options such as challenge-based acquisitions to refine down to a design, and other types of contracts that allow flexibility, which we will cover in a different post.
Testing, Security, Integration and Operations not integrated soon enough
Often times, the Agile team delivers something quickly only to find that it will take months to complete testing and deployment into the production environment. This is usually because of the often silo’ed nature of the development, testing, security and operations teams. The development team throws finished products over the wall to the testing, security and operations teams. This leads to finding quality, security and integration issues very late in the process, which costs much more in terms of time and money to fix.
There are lots of ways to avoid this, but they all start with including testing, engineering, environment, security and operations teams earlier in the life cycle as part of the team. They must be involved in the discussions as early as possible. Increasingly, Agile teams are finding that the coordination of development and operations activities is critical to optimize the entire value stream. Security is now being considered an integral part as well (a.k.a., DevSecOps). If the team wants to increase the number of deployments, then the tighter integration is a must. This allows for more opportunities to coordinate and practice. Automation of testing with test scripts and tools, and incorporation of code-based deployments will also help ensure that builds remain stable and deployments are uneventful. There will be more on DevSecOps practices in another post, as it is a big topic on its own.
Lack of top-down support or understanding of the scope of change
Agile teams often find that they are following Agile methods, but the culture has not adopted the Lean-Agile habits necessary to really be successful. Sometimes the lack or resistance to adoption is because of a lack of top-down support for the change. In order to be successful, Agile needs to be tacked from the bottom-up by the team, and encouraged from the Top-down by the senior leadership. The senior leadership needs to create the urgency for changes and encourage the organization to adopt lean-Agile practices.
Lack of training
I like to call it Agile by urban-legend. Team members think they understand Agile. But in reality, they understand enough to be dangerous, thinking that Agile requires no documentation or planning while still enforcing traditional waterfall or predictive life cycle approaches and subscribing to command and control, top-heavy decision-making policies. When the Agile project fails, they blame Agile.
In order for the team to be successful, everyone involved in the project needs to be trained in Agile so that there is a solid and minimum core knowledge base of Agile upon which to move forward. This includes Senior Management. In fact, I recommend that Senior Management be the first recipients of the training.
Lack of understanding of the Agile at scale
Agile for large solutions requires additional considerations and must be tailored to account for the size. Consider the size of the project and determine the impact to the Agile approach ahead of time, as this will impact the cost and structure of the team among other things. For example, if you are building a solution with multiple capabilities that impact multiples systems, then it is very likely that you would have more than one Agile team. In this case, you might want to ensure that you account for a team of Scrum teams that work together. You would need other roles to account for integration challenges. There are frameworks out there, such as the Scaled Agile Framework (SAFe), that provide a framework for larger implementations of Agile at scale. Spend some time to research the right fit for you and your team.
Do you agree? Do you have additional observations or comments? If so, let me know and I’ll update the post to include your comments.