The Dark Side of Agile Projects

These days I’m reading articles from the agile community attacking the figure of the project manager and the project management discipline. I get the message that project managers are no longer needed in software projects. We are on the dark side because we prevent good team performance, good quality work, value orientation, sustainable pace, team building, creativity, common sense, etc.

When software projects fail that way, should we always blame the project manager? When the project manager is the one to blame, is he or she a good project manager?

These readings have great impact on me because I consider myself an active member of the agile community, like many other colleagues, and we really do not feel we are on the dark side when it comes to leading an agile project.

When they talk about avoiding our role, we ask ourselves: Who is going to be responsible for finishing the project on time and on budget? Organization managers will ask themselves: Who is the one to blame when the project goes wrong? ;-D

Demonizing the project manager, or the project management practice, is an unfair and ill-advised simplification. Project management trending topics go just in the opposite direction:

• Project management frameworks and methods agree on the two main project development lifecycles: 1) Predictive or waterfall lifecycles and 2) Adaptive lifecycles (= agile = value oriented = change oriented).
• Project management certification bodies are offering specific training programs to manage agile projects and have modified requirements so that applicants have an agile project management base.
• Most public and private organizations are tailoring management and governance to include predictive and agile projects in their portfolios. They apply agile management not just for software projects, but also for business projects when creation and innovation is needed, or just when requirements are not clear.

Agile frameworks do not mention the role of the project manager because they were designed for product management, not project management. However, the PM role is required when the organization approves a project that must be completed within a certain period of time and below a certain budget. Professional PMs are responsible, among other things, for the agile project to end on time and on budget, meeting stakeholders’ requirements.

Project Managers are not complete professionals if they do not know how to manage the two types of projects:

Predictive projects try to close all requirements at the beginning, and then they go in sequential phases. This is the case for EPC projects—Engineering, Procurement and Construction. Predictive project managers try to close a project management plan, including a Gantt chart and many other components that can be changed if needed. The project plan is used as a “music sheet”. The orchestra leader conducts the musicians with the music sheet, the same way the project manager conducts the team members with the project plan. In the project review meetings, actual performance is compared to plan and corrective actions are taken to get back on track. Measuring and adjusting continuously we get the project on time, on budget, etc.

Some software projects follow the waterfall model: Progress flows in largely one direction—“downwards” like a waterfall—through the phases of conception, initiation, analysis, design, construction, testing, deployment and maintenance.

When solution requirements are closed, and it is clear what to do, then working in batches is very efficient. For example: if we deliver an information system in English language, it is quick to deliver also in Spanish… As water is not going up in a waterfall, we are not going back in a waterfall lifecycle. As everybody knows, waterfall lifecycles is pointless in most software projects, but it is still recommended in some software projects: mission critical—aeronautic systems—, embebbed systems—autonomous vehicles—, software with many non functional requirements, software with a fixed product roadmap, etc.

Why would the waterfall model turn useless at most user centric software projects? Because no one can imagine all the information system’s use cases and requirements precisely. If we deliver first release after 5 months, chances are that most features are rejected and lots of rework should be done.

When I managed software projects for customers, I usually criticized them when they did not make up their minds and wanted changes day in day out. Now I am a customer myself with the PMPeople software development outsourced to a software factory in India. Despite I consider myself an expert when modeling information systems, I was not able to make up my mind on requirements from the very beginning. I like chaning requirements progressively. Not a sinble mockup I developed 3 years ago has something to do with the real screens and logic customers are using right now.

There is a sad truth on software projects we have to live with:

“Software customers don’t know what they want. They only know what they don’t want when they see it.”

We have to confirm value frequently, typically every 2 weeks, making them see something working. When my colleagues in India show me a demo, then and only then I’m clear on what I want. Choosing a predictive lifecycle would have been a clear mistake in this project. We would have failed long time ago. The key factor of this project was to choose a value driven lifecycle:

How did we manage this agile project? We limited schedule to 12-18 moths, the project goal being a commercial version for clients (B2B). Beyond that milestone, corrective maintenance and small evolutions were no longer project management but operation management. Mobile version was a new project. We reused the software factory project team and the product owner. Freemium model for general users (B2C SaaS) was another project again.

Initial high level requirements have been more or less stable from the very beginning. Project budget calculation is simple math, since the team is stable for a given period of time. Following Scrum terminology, I can consider myself as a stakeholder— “chicken” —, along with other colleagues I invited for validations. Regarding the project, I am the project manager and the business analyst. I talk quite often with the Product Owner, and behind him, we had a development team and a Scrum Master.

We use Asana to manage backlogs and virtual boards and Slack for continuous communication. I share the product backlog with the product owner. I do not access the sprint backlogs, neither attend daily standups—too early in Spain. I can see the size in ideal hours for each story. They practice continuous integration: when a story is finished, they deploy the software on testing environment. I get an Asana notification when the card is moved from “in progress” to “testing”. I test is passed, then I move the card to “done”, otherwise I move it back to “in progress” and explain why in Asana comments.

Scrum Master can update the burndown chart every day using an Excel sheet. Click here to download the Excel template.

Most agile projects do not need Gantt charts. Agile projects are usually on time, since scope is changed to deliver the most value features progressively. Notice the example below, so simple and linear, one release after another, the only schedule deviation due to a spike.

We executed a plan with 5 releases:

• Release #1 produced a PDF prototype for the mobile application: over 100 Balsamiq mockups we were able to show to many colleagues to validate and correct many approaches before hiring the software factory. Click here to download the actual PDF file. You can click PDF links to check the user experience—watch a short video here.
• Release #2 delivered value through a real mobile application in iOS and Android, not quite functional but enough to make users validate high level features.
• Release #3 was to develop our first web application including all roles but Resource Manager. All projects, programs and portfolios structures were tested.
• Release #4 developed main features for projects and resources: resource manager role, resource pools, timesheets and expenses workflows, etc.
• Release #5 complete our first commercial release: multi-organization, user invitations, project reviews, EVM, Gantt, task management and work packages.

My practice as a project manager was aligned with the PMI Agile Certified Practitioner Examination Content Outline (pdf). This document describes 7 domain areas of responsibility for an agile project manager.

1. Agile Principles and Mindset: Explore, embrace, and apply agile principles and mindset within the context of the project team and organization.

2. Value-Driven Delivery: Deliver valuable results by producing high value increments for review, early and often, based on stakeholder priorities. Have the stakeholders provide feedback on these increments, and use this feedback to prioritize and improve future increments.

3. Stakeholder Engagement: Engage current and future interested parties by building a trusting environment that aligns their needs and expectations and balances their request with understanding of the cost/effort involved. Promote participation and collaboration throughout the project lifecycle and provide the tools for effective and informed decision-making.

4. Team Performance: Create an environment of trust, learning, collaboration, and conflict resolution that promotes team self-organization, enhances relationships amongst team members, and cultivates a culture of high performance.

5. Adaptive Planning: Produce and maintain an evolving plan, from initiation to closure, based on goals, values, risks, constraints, stakeholder feedback, and review findings.

6. Problem Detection and Resolution: Continuously identify problems, impediments, and risks; prioritize and resolve in a timely manner; monitor and communicate the problem resolution status, and implement process improvements to prevent them occurring again.

7. Continuous Improvement (Product, Process, People): Continuously improve the quality, effectiveness, and value of the product, the process, and the team.

How could anyone possibly think that agile project managers are expendable?

Let’s see domains in detail. These are the tasks an agile project manager is supposed to do every day:

Domain 1. Agile Principles:

1. Advocate for agile principles by modeling those principles and discussing agile values in order to develop a shared mindset across the team as well as between the customer and the team.
2. Help ensure that everyone has a common understanding of the values and principles of agile and a common knowledge around the agile practices and terminology being used in order to work effectively.
3. Support change at the system or organization level by educating the organization and influencing processes, behaviors, and people in order to make the organization more effective and efficient.
4. Practice visualization by maintaining highly visible information radiators showing real progress and real team performance in order to enhance transparency and trust.
5. Contribute to a safe and trustful team environment by allowing everyone to experiment and make mistakes so that each can learn and continuously improve the way they work.
6. Enhance creativity by experimenting with new techniques and process ideas in order to discover more efficient and effective ways of working.
7. Encourage team members to share knowledge by collaborating and working together in order to lower risks around knowledge silos and reduce bottlenecks.
8. Encourage emergent leadership within the team by establishing a safe and respectful environment in which new approaches can be tried in order to make improvements and foster self-organization and empowerment.
9. Practice servant leadership by supporting and encouraging others in their endeavors so that they can perform at their highest level and continue to improve.

Domain 2. Value-Driven Delivery:

2.1. Define Positive Value:

1. Define deliverables by identifying units that can be produced incrementally in order to maximize their value to stakeholders while minimizing non-value added work.
2. Refine requirements by gaining consensus on the acceptance criteria for features on a just-in-time basis in order to deliver value.
3. Select and tailor the team’s process based on project and organizational characteristics as well as team experience in order to optimize value delivery.

2.2. Avoid Potential Downsides:

1. Plan for small releasable increments by organizing requirements into minimally marketable features/minimally viable product in order to allow for the early recognition and delivery of value.
2. Limit increment size and increase review frequency with appropriate stakeholders in order to identify and respond to risks early on and at minimal cost.
3. Solicit customer and user feedback by reviewing increments often in order to confirm and enhance business value.

2.3. Prioritization:

1. Prioritize the units of work through collaboration with stakeholders in order to optimize the value of the deliverables.
2. Perform frequent review and maintenance of the work results by prioritizing and maintaining internal quality in order to reduce the overall cost of incremental development.
3. Continuously identify and prioritize the environmental, operational, and infrastructure factors in order to improve the quality and value of the deliverables.

2.4. Incremental Development:

1. Conduct operational reviews and/or periodic checkpoints with stakeholders in order to obtain feedback and corrections to the work in progress and planned work.
2. Balance development of deliverable units and risk reduction efforts by incorporating both value producing and risk reducing work into the backlog in order to maximize the total value proposition over time.
3. Re-prioritize requirements periodically in order to reflect changes in the environment and stakeholder needs or preferences in order to maximize the value.
4. Elicit and prioritize relevant non-functional requirements (such as operations and security) by considering the environment in which the solution will be used in order to minimize the probability of failure.
5. Conduct frequent reviews of work products by performing inspections, reviews, and/or testing in order to identify and incorporate improvements into the overall process and product/service.

Domain 3. Stakeholder Engagement:

3.1. Understand Stakeholder Needs:

1. Identify and engage effective and empowered business stakeholder(s) through periodic reviews in order to ensure that the team is knowledgeable about stakeholders’ interests, needs, and expectations.
2. Identify and engage all stakeholders (current and future) by promoting knowledge sharing early and throughout the project to ensure the unimpeded flow of information and value throughout the lifespan of the project.

3.2. Ensure Stakeholder Involvement:

1. Establish stakeholder relationships by forming a working agreement among key stakeholders in order to promote participation and effective collaboration.
2. Maintain proper stakeholder involvement by continually assessing changes in the project and organization in order to ensure new stakeholders are appropriately engaged.
3. Establish collaborative behaviors among the members of the organization by fostering group decision-making and conflict resolution in order to improve decision quality and reduce the time required to make decisions.

3.3. Manage Stakeholder Expectations:

1. Establish a shared vision of the various project increments (products, deliverables, releases, iterations) by developing a high level vision and supporting objectives in order to align stakeholders’ expectations and build trust.
2. Establish and maintain a shared understanding of success criteria, deliverables and acceptable trade-offs by facilitating awareness among stakeholders in order to align expectations and build trust.
3. Provide transparency regarding work status by communicating team progress, work quality, impediments, and risks in order to help the primary stakeholders make informed decisions.
4. Provide forecasts at a level of detail that balances the need for certainty and the benefits of adaptability in order to allow stakeholders to plan effectively.

Domain 4. Team Performance:

4.1. Team Formation:

1. Cooperate with the other team members to devise ground rules and internal processes in order to foster team coherence and strengthen team members’ commitment to shared outcomes.
2. Help create a team that has the interpersonal and technical skills needed to achieve all known project objectives in order to create business value with minimal delay.

4.2. Team Empowerment:

1. Encourage team members to become generalizing specialists in order to reduce team size and bottlenecks, and to create a high-performing cross-functional team.
2. Contribute to self-organizing the work by empowering others and encouraging emerging leadership in order to produce effective solutions and manage complexity.
3. Continuously discover team and personal motivators and de-motivators in order to ensure that team morale is high and team members are motivated and productive throughout the project.

4.3. Team Collaboration and Commitment:

1. Facilitate close communication within the team and with appropriate external stakeholders through co-location or the use of collaboration tools in order to reduce miscommunication and rework.
2. Reduce distractions in order to establish a predictable outcome and optimize the value delivered.
3. Participate in aligning project and team goals by sharing project vision in order to ensure the team understands how their objectives fit into the overall goals of the project.
4. Encourage the team to measure its velocity by tracking and measuring actual performance in previous iterations or releases in order for members to gain a better understanding of their capacity and create more accurate forecasts.

Domain 5. Adaptive Planning:

5.1. Levels of Planning:

1. Plan at multiple levels (strategic, release, iteration, daily) creating appropriate detail by using rolling wave planning and progressive elaboration to balance predictability of outcomes with ability to exploit opportunities.
2. Make planning activities visible and transparent by encouraging participation of key stakeholders and publishing planning results in order to increase commitment level and reduce uncertainty.
3. As the project unfolds, set and manage stakeholder expectations by making increasingly specific levels of commitments in order to ensure common understanding of the expected deliverables.

5.2. Adaptation:

1. Adapt the cadence and the planning process based on results of periodic retrospectives about characteristics and/or the size/complexity/criticality of the project deliverables in order to maximize the value.
2. Inspect and adapt the project plan to reflect changes in requirements, schedule, budget, and shifting priorities based on team learning, delivery experience, stakeholder feedback, and defects in order to maximize business value delivered.

5.3. Agile Sizing and Estimation:

1. Size items by using progressive elaboration techniques in order to determine likely project size independent of team velocity and external variables.
2. Adjust capacity by incorporating maintenance and operations demands and other factors in order to create or update the range estimate.
3. Create initial scope, schedule and cost range estimates that reflect current high level understanding of the effort necessary to deliver the project in order to develop a starting point for managing the project.
4. Refine scope, schedule, and cost range estimates that reflect the latest understanding of the effort necessary to deliver the project in order to manage the project.
5. Continuously use data from changes in resource capacity, project size, and velocity metrics in order to evaluate the estimate to complete.

Domain 6. Problem Detection and Resolution:

1. Create an open and safe environment by encouraging conversation and experimentation, in order to surface problems and impediments that are slowing the team down or preventing its ability to deliver value.
2. Identify threats and issues by educating and engaging the team at various points in the project in order to resolve them at the appropriate time, and improve processes that caused issues.
3. Ensure issues are resolved by appropriate team members and/or reset expectations in light of issues that cannot be resolved in order to maximize the value delivered.
4. Maintain a visible, monitored, and prioritized list of threats and issues in order to elevate accountability, encourage action, and track ownership and resolution status.
5. Communicate status of threats and issues by maintaining threat list and incorporating activities into backlog of work in order to provide transparency.

Domain 7. Continuous Improvement (Product, Process, People):

1. Tailor and adapt the project process by periodically reviewing and integrating team practices, organizational culture, and delivery goals in order to ensure team effectiveness within established organizational guidelines and norms.
2. Improve team processes by conducting frequent retrospectives and improvement experiments in order to continually enhance the effectiveness of the team, project, and organization.
3. Seek feedback on the product by incremental delivery and frequent demonstrations in order to improve the value of the product.
4. Create an environment of continued learning by providing opportunities for people to develop their skills in order to develop a more productive team of generalizing specialists.
5. Challenge existing process elements by performing a value stream analysis and removing waste in order to increase individual efficiency and team effectiveness.
6. Create systemic improvements by disseminating knowledge and practices across projects and organizational boundaries in order to avoid re-occurrence of identified problems and improve the effectiveness of the organization as a whole.