As I wrote in Expecting the unexpected vs not expecting the expected/ No-one expects the net present value in the eNPV, there are substantial estimation flaws in the process of developing eNPVs. One of the biggest sources of error is the ‘Probability of Technical Success’ - the calculation of PTS (or PTRS) varies between companies, between divisions of companies and in the way it imports numbers from elsewhere.

But, even if it is always wrong, is it useful, as the saying goes? Well, there is another source of error, at the portfolio level, that is entirely preventable.

Consider this short story. A CEO stands in front of 100 of her Project Leads, and shows a range of pipeline opportunities.

One is shown on the big screen, a molecule that clearly doesn’t excite her. “This BHT24 has a 76.4% PTRS, but only has $700-800m peak sales.”

She carries on down the list, until she gets to another project, and her eyes light up. “We’ve run the numbers on this CN27, and we can only get to a 24.3% PTRS. But, but if it did hit, it would be huge.”

After running through all the programmes, she invites the group to vote on which they’d like to take on. Guess what? All 100 of the Project Leads vote for projects with PTRS above 75%. She steps off stage, and confides in her Chief of Staff, “Hmm. If the numbers are right, I’d still want 25 of those folks to want to take on the CN27, and I’d only want 75 of them to want the BHT24. What am I doing wrong?”

The answer, of course, is that the statistics work at her level, but don’t work at the level of the Project Lead. She understands the risk represented by a 24.3% PTRS, and at a portfolio level wants the company to take that risk, because of that huge upside opportunity. She also, at a portfolio level, wants to hedge the risk of the 76.4% asset. Her team, though, are making rational individual decisions, to opt for the ostensibly safer asset - the statistics don’t work at their level.

This story illustrates one of the issues we see across pharma: teams, who work on their own product plans, opt for the higher probability, if lower value, paths. The more ‘interesting’/ ‘swing for the fences’ options never even make it to portfolio review, because the teams themselves don’t want to work on them.

The opacity of PTRS, and the subjectivity of some of the estimates within its calculation, and its subsequent effect on eNPV, hides all kind of risk aversion at the team level. That is even if it made sense to believe that an asset has an eNPV, rather than 100 path-specific values, the system prefers the higher probability programmes.

That could be fine, on average, if the numbers were accurate or directionally accurate. Of course, we then come back to the first problem. Key to all of this is to remove the belief in the possibility of knowing/ predicting the probability of success. Because the number cannot be accurate, key is to understand the contribution of the next experiment to the choice of path. Teams should be showing exciting if uncertain options to portfolio heads - the math works different up there.