Why doesn’t the classic PMP process work with software?

The official Program Management Body of Knowledge provides a well-defined process workflow for the initiation and execution of projects.  At a very high level, the workflow is something like this:

Define Scope -> Create WBS (deliverables list) -> Create Activity List -> Estimate Activity Durations -> Create Schedule

There’s nothing wrong with this process.  It makes perfect common sense.  If I was doing something like remodeling my house this process would work pretty well.  So why do software and IT projects have so many schedule problems?  Why doesn’t this process work for them?

There are really two different root causes here.  In particular, two of the specific process activities have challenges in the software arena.

1) Create Deliverables List

Let’s take an example of why this is hard.  During the initiation of the project, the scope is defined as “I want a webapp.”  Maybe its more specific: “I want a web-based typical CRUD database application.”  OK, sounds good.  Let’s decompose to deliverables, play planning poker, and make a sprint schedule.

So things are going along just fine.  Hopefully, you’ve got enough experience and expertise that your decomposition to deliverables took all the little nuances into consideration.  For example, for the database?  Actually we’ll need to provision a new VM, install the OS, patch it, get the DB software installed and tuned and secured, etc.

However, because of the fast-moving and constantly changing nature of software and IT, its very very likely you missed something in the deliverables list, or it changed.

• A new version of the OS or database software was released and you need to roll forward
• Some new vulnerability was discovered and some configuration changes are needed to the platform, which breaks something in the software release and necessitates that module to be reworked.

Once that deliverables list is changed, then everything built on top of it (activity list, activity estimates, and schedule) is now invalid and has to be reworked.

2) Estimate Activity Durations

To be blunt, this is really hard to do with software.  Planning poker (which is really just a gamified way of doing dephi) is a great way to do estimates, and I support it wholeheartedly.  But there’s still a fairly large margin of error in estimation.

• The assigned developer has more of a learning curve than expected, either with the technology itself or with understanding all the interfaces and dependencies and integrations
• Unexpected issues appear, especially when using third-party libraries.  I can’t tell you how many times I’ve found some edge case in my implementation which isn’t supported or doesn’t quite work in the external library I’m using.

So this is where the Agile Manifesto came into play to try and address this.  Key from the Principles of Agile: “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”

In practice, what does this mean?  It means don’t plan (i.e. decompose, define, estimate) for everything at once.  Just do it for the next chunk of work in front of you.  Why bother trying to build the entire schedule if you pretty much know for certain that the Deliverables List and Activity List is going to change?

The idea is that the earlier in the process that problems are found, the cheaper they are to fix.  So why do the process on “everything” at once, thus ensuring that problems are found later?  Only do the process for smaller chunks that you are about to do, so that way when you find a problem you are earlier in the process for that chunk of work and thus its cheaper to fix.

As with everything, there are pros and cons:

Pro:

You won’t blow up a plan. You can at least have some control over the scope right in front of your face.  You are at least making sure that things are running as efficiently as possible at that moment in time.  You are increasing quality of the project by minimizing rework and finding bugs faster.

Con:

You can’t see estimated (accurate) total cost/schedule for your awesome software project.

From a business and strategic perspective, the shortcomings of Agile can be a challenge.  You know you want some set of features (scope) for a customer, or a time to market, or to beat a competitor to market.  But since in Agile you can’t define up front both schedule and scope, then you are limited to either “its done when its done” or “it doesn’t have all the features that we wanted”.

Now granted, the overall project timeline is probably going to be shorter because there are fewer bugs.  But if you’re trying to build a bigger picture strategy and roadmap for your company then this limitation can still be a frustration.

The key to strategic success is finding and supporting a team that can successfully execute Agile to develop and deliver software, and be astute enough to roll those new features and releases into your strategic roadmap in an effective way.  Look at services like Netflix or Amazon.  Are there really “new releases” that are in front of the customer?  Not really, there’s just continual gradual improvement.  Which is why to successfully use Agile you probably also need to use Continuous Integration/Continuous Delivery.  But that’s another discussion for another time.