In any major project, you’re likely to be unsure about the output (which is why you are using a project management methodology in the first place); you’re even less certain about the outcome and when it comes to the benefits … don’t even get me started.

One solution has always been to spend far more time planning; exploring scenarios and positing options in an endeavour to remove as much of the uncertainty as possible and prepare for risks or change before it happens. The challenge here seems to be threefold: 1. We continually aspire for better, longer, more effective planning; 2. One of the great driving forces for projects is our need to be active … doing stuff and over-bureaucracy can very effectively stifle project momentum; 3. We struggle to manage both things at once ‘delivering and planning simultaneously’.

Another solution is represented in a variety of different forms by agile or Agile approaches to projects; short-cycles of activity interspersed with a return to the planning room before the next effort. But, as yet, no one has definitively demonstrated how to manage this approach to projects at scale or across a long period of time.

The LSE paper I found on planned and emergent change captures the tension and the challenges very effectively.

Read the paper

I took a lot from this piece of research, for example, the sense from the researchers’ interviewees that change is meaningless as a thing in its own right (I paraphrase) but rather a way of working:

‘my instinct is it should not be a change project but a project that delivers something else, hopefully embedded, you see something that does not work and you change it.’ … ‘For Paula change is about involving people, getting them to think, share and not knowing all the answers as the beginning.’

That instinctively senior managers are suspicious of this approach:

‘In general, decision makers describe experiences where they have learned to embrace change or even thrive on change. But they do not seem to trust their own employees possessing that capacity and therefore change programmes are usually presented as having to be ‘directed and imposed’.

Here then is perhaps the real conundrum about managing this tension between the emergent and the planned. It’s not about project management – whether you pursue waterfall, agile or even ‘wet agile’; although you certainly need to adopt the mix of methodologies that are most appropriate to each project. Nor is it about planned and defined programmes to transform the organization from command and control to agile.

It’s more about organizational design, management style and the platforms and opportunities that you create to encourage conversation; to provide everyone in the organization with the opportunity to reflect what’s going on and how you capture and reflect the insight they generate in your portfolio, your programmes and your approach to individual projects.

I was intrigued to see that this paper was originally published in 2010. The observations and conclusions it generates are absolutely relevant to 2020 and yet, at the same time, from the perspective of UK PLC, society and the global community, 2010 seems like a lifetime in the past.