Why do SharePoint Projects Fail? – Part 2

Hey there.

Welcome to Part 2 of a series that examines why SharePoint projects fail. If you have come straight from Part 1, you probably still have a hang over and would most likely not want to see tequila ever again! But if you missed the first article, there is a drinking game to be played first 🙂 .

Now as it happens, we haven’t even gotten to SharePoint yet. That’s because to examine why SharePoint projects fail, one has to examine why “projects” fail. Part 1 introduced the work of Horst Rittel and the term ‘Wicked Problem’. His work was not related to IT problems per se, but most of the ten characteristics he described 35 years ago are applicable to IT projects. Although subsequent academic works have refined (and in some cases criticised) his work, it still stands up pretty well in my opinion.

When you examine the various survey based studies undertaken on the success rate of IT projects, bad projects have a significant root cause at the initiating and planning phases. My own personal experience pretty much corroborates the studies and in hindsight, it’s plain that the worst of them were wicked problems.

In this post, I am going to play the role of the “ghost of wicked problems past” (think Dickens: A Christmas Carol). I am going to take you on an exciting, magical journey back in time, to examine some of the early work on wicked problems, and show that in the 25-35 years since they were written, not a heck of the wisdom imparted has found its way to prominence 🙂

CleverWorkarounds tequila shot rating with a redbull thrown in (on account of the history lessons):

I just realised that in all of the articles that I have written on this blog, I have never taken the piss out of project managers! So why should they miss out from my unfair stereotyping?

The strange world of the project manager

Project Managers are generally brown-nosers who have senior management ambitions. The project management phase of their career is basically a stepping stone to bigger and better things. For you readers who are technical/programming types, remember when you spent your first year or so on level 1 tech support? Project management is kind of like the helpdesk for management people 😛 (don’t snigger technical geeks, they still get paid more than you do!)

Also, given that PM’s have to spend a lot of time talking to management level stakeholders, trying to understand a problem that doesn’t necessarily affect them, PM’s do have a propensity for metrosexual tendencies and wear suits, even on Fridays. This creates a credibility problem with the lowly, poorly dressed tech geeks on the project team, as well as the amusement of senior management who see them as trying a little too hard. Talk about being caught between a rock and a hard place!

More inexperienced project managers would likely have a massive horn for Prince II or PMBOK, and have a very idealistic view of the project management process based on their chosen methodology. They will most likely get all flustered and angry when you, for example, raise new requirements for a phase that’s already been completed and throw lots of forms at you to fill in.

But let’s be fair here. I have a lot of respect for project managers, because they have a thankless job. They are the central point that has to deal with YOU lot – all of the various project team members, sponsors and stakeholders. Dogmatic tech geeks have little appreciation of how truly annoying they really are to everyone else, and on top of that, most stakeholders are even bigger ambitious brown-nosers than PM’s, and thus office politics routinely gets in the way. It is very common that stakeholders do not actually talk to each other except *through* the project manager.

So consider the PM dilemma. Someone has put a proposal to management to solve a business problem. Stakeholders all have different views of the problem, depending on their position, perspective and previous turf-wars played out against other stakeholders in the business. Opinionated geeks have even less understanding of the problem, but according to them, everybody else is stupid and it can of course be easily solved by an open source product that they googled the day before. The PM has to steer a course through all of these personality types and on top of that, ensure that tasks get done, and therefore has to incessantly nag people as well.

It is a tough job people… 

Joke disclaimer: If you are an offended PM and your blood is boiling as you read this, you have missed the point – I’m KIDDING!

Back to wicked problems – John S. Gero

Alrightie then! In Part 1, We looked at the ten characteristics of “Wicked Problems” as defined in 1973 by Rittel and Webber. As mentioned, I think that this was a work of seminal importance, although it was not written with IT projects in mind.

Our next trip down memory lane is to 1975, possibly the earliest article that referred to Rittels earlier work in an IT centric setting. Entitled, “Ethics in computer-aided design: a polemic” by John S. Gero, this paper discusses the issues that IT has on problem definition and project scope in an ethics context.

It is common to listen to learned papers at conferences that commence with statements limiting the scope of the problem being considered. And then, by the end of the paper, the author concludes that he has now solved the problem without putting it back into its original context

Later in this series of posts, I will get into techniques for dealing with wicked problems, but Gero has in fact highlighted the weakness of one of those common techniques, so I will mention it briefly here. One common approach is to have the project sponsor, or someone authorative to make some initial scope decisions and mandate them on the stakeholders. But Gero asserts that project scope and requirements are ethical questions in nature, because the decision on scope and requirements is subjective, and will vary from stakeholder to stakeholder.

So the inevitable question that arises is whether you really have solved a wicked problem by having the head honcho put his foot down, or simply delayed it for a while? (Let’s save that for a future post).

Next stop 1978…

“Relating software requirements and design” was a paper written by Lawrence Peters in 1978. In effect he was talking about what you would today call an Agile software development methodology. Peters questioned the validity of requirements and design phase being distinctly separate. He subscribed to Rittel’s assertion that a statement of a problem (the requirements) is also a form of a solution statement (the design). He named his requirements/design methodology SAMM (Systematic Activity Modelling Method). This article is interesting more for it being the first evidence of an SDLC lifecycle that directly cited Rittel’s work.

Our experience … seems to indicate that our view of the relationship between requirements and design must be modified from one of exact, firm, detailed problem statements which precede design to one of a controlled succession of proposed models and refinements culminating in the delivered system

Lysles and Mitroff

In 1980, an article was written for Administrative Science Quarterly, that focused not on problem solving, but instead, the formation of a problem statement in the first place. Entitled “Organizational Problem Formation – An Empirical Study”, the authors interviewed a sample group of senior level management and how they tackled a problem that was having a significant impact on the organisation. They were specifically asked when and how they became aware of the problem, what they did as a result of the awareness and how the problem was defined within the organsiation. They combined the interview with questionnaire data that measured attitudes and styles in problem solving.

90 percent of the problems reported by managers fell within the ill-defined category. Two-thirds of these arose from internal organisational factors relating to management style, organizational structure and long range planning … About 80 percent of managers said they were aware of a problem before formal indicators, such as financial figures, showed that it existed and before a superior or subordinate presented the problem to them

This article went on to identify “emergent themes” to investigate further and found that

commitment, turnover, political manoeuvring, denial, etc recurred throughout the cases … Themes such as problem avoidance, especially human problems, organisational rigidity, fear and political power are major frames of reference for further research

I always felt that many SharePoint installations would be train wrecks, but after reading through this study, now I am even more scared!

Enough of the history already!

For what its worth, if I had the time I’d pursue follow-up studies to the Lysles and Mitroff one, but are we nerds or psychologists? 🙂  The point is with all of this, that what I am telling you here isn’t exactly new. After all, I have just mentioned academic work that is over 30 years old. In all this time, you think some of this would have made it to some of the various methodologies implemented around problem solving like CMMI, PMBOK and the like.

I’m not criticising those methodologies at all – as I’m a methodology kind of guy. I personally was well aware of project management methodologies of PMBOK and Prince II for a number of years. But I have also been involved in a wicked problem where PMBOK was introduced to try and solve a wicked problem. They did not work of course. In fact the opposite happened – now on top of arguing over what it actually is we were trying to solve, we had a lot more paperwork to do 🙁

So that is it for the history lesson into wicked problem theory. I think enough has been said for readers to get a deeper understanding into the nature of problems that go awry. We are now two posts in and the tool hasn’t even cracked a mention yet. So in the next post, we will examine why Microsoft’s SharePoint has a propensity to become a wicked problem project.

Until then, bye for now

Paul