Visualising portfolio risk and performance reviews - presentation delivered at the AIPM 2017 National Conference

Written by Bryan Fenech, Director - PPM Intelligence.

Note: This article is a summary of a presentation that I gave at the recent AIPM 2017 Conference in Melbourne. A link to the full presentation is provided at the end.

Traditional approaches to portfolio risk and performance review

More often than not the implementations of portfolio management that I come across do not deliver on their promised benefits. The reason for this is almost always the same: portfolio governance boards and portfolio management offices (PMOs) do not make sufficient use of the plethora of data available to them to support decision making about portfolio selection, monitoring and controlling.

Organisations are particularly weak in the areas of portfolio performance, risk and benefits management. The most common approach followed when portfolio governance boards review their inflight portfolio is to run through a list of projects and programs with health ratings for a set of simple performance metrics along with some basic commentary. The health ratings are usually represented as traffic lights – red, amber, green. The performance metrics usually include overall, budget, schedule, resources, risk, benefits and overall health.If you are doing this you are wasting your time!

Firstly, it is duplication of the governance practices that occur at project and program Steering Committee level and adds little if any value. If something is going wrong with one of the projects or programs in the list the best that the portfolio governance board can do is authorise actions to be taken with respect to the project or program concerned. The portfolio governance board is doing little more than delegating back (with interest) what the Sponsor has escalated up to them and is already doing their best to deal with.

Secondly, and worse still, it is ineffective as it does not make it easy to identify and resolve systemic issues that impact across the portfolio, for eg resource bottlenecks in certain skill categories causing delays in multiple projects, or common risk types and estimation errors. This misses the opportunity for creating learning and facilitating evidence-based decision making that improves structural and systemic problems and inefficiencies regarding the organisation’s portfolio management capability and capacity. This situation is not a good use of anyone’s time and energy.

Portfolio data analytics

An alternative approach is to give your portfolio data to a data analyst with project management knowledge and ask them to turn it into valuable information that tells you something about portfolio performance, risk profile and benefits position as a whole, rather than project by project.

Here are a few examples of what you might ask them to provide you:

1. What types of risk are the most common across the portfolio? Is there a particular type of resource bottleneck that presents regularly? Is vendor management a recurring problem? Is there evidence that project managers are consistently demonstrating forms of cognitive bias in their estimation, such as being overly optimistic?
2. Are there particular “break points” in the portfolio (ie, projects or programs upon which there is a high degree of dependency by other projects and programs)? What is the health of the most significant break points? What is the combined value of 'at risk' investment dollars or benefits dependent on these break points?
3. How does the health of tier 1 projects and programs compare with less critical projects and programs? Is there a point of increasing complexity at which portfolio performance and throughput tends to 'fall off a cliff'? Is the organisation allowing sufficient schedule and budget contingency in its tier 1 projects for these realities?
4. What is the combined value of investment dollars or benefits by overall traffic light rating? Is the position improving over time?

The information derived can be a snapshot at a point in time or represent historical trends.

Portfolio data visualisation

However, it is critical that portfolio analytics are presented in a way that decision makers can make immediate sense of them. Data visualisation techniques can help to present an incredibly rich picture of your project portfolio. Bringing together graphical representations of portfolio information in a portfolio info-graphic facilitates immediate practical application by senior decision makers. This approach delivers vastly superior outcomes in terms of the quality of learning and decision making.

Charles Minard’s famous graphical depiction of Napoleon’s campaign against Russia helps us to think about how we might represent portfolio information graphically. The Minard diagram shows the losses suffered by Napoleon's army in the 1812–1813 period. Six variables are plotted: the size of the army, its location on a two-dimensional surface (x and y), time, direction of movement, and temperature. The line width illustrates a comparison (size of the army at points in time) while the temperature axis suggests a cause of the change in army size. This multivariate display on a two dimensional surface tells a story that can be grasped immediately while identifying the source data to build credibility.

Types of visualisation

Author Stephen Few described eight types of quantitative messages that users may attempt to understand or communicate from a set of data and the associated graphs used to help communicate the message. I’ve adapted his typology so that it applies to portfolio analytics:

1. Time-series: A single variable is captured over a period of time, such variances in planned portfolio spend or benefits over a 3-year period. A line chart may be used to demonstrate the trend.
2. Ranking: Categorical subdivisions are ranked in ascending or descending order, such as a ranking of cost/benefit or contribution to strategy (the measure) by projects in the portfolio (the category, with each sales person a categorical subdivision) during a single period. A bar chart may be used to show the comparison across the projects.
3. Part-to-whole: Categorical subdivisions are measured as a ratio to the whole (i.e., a percentage out of 100%). A pie chart or bar chart can show the comparison of ratios, such as the incidence of risk types across the portfolio, or the total portfolio spend or expected benefits by overall health indicator.
4. Deviation: Categorical subdivisions are compared against a reference, such as a comparison of actual vs. budget expenses for enterprise, divisional or department portfolios for a given time period. A bar chart can show comparison of the actual versus the baseline amount.
5. Frequency distribution: Shows the number of observations of a particular variable for given interval, such as the number of years in which the stock market return is between intervals such as 0-10%, 11-20%, etc. A histogram, a type of bar chart, may be used for this analysis. A boxplot helps visualize key statistics about the distribution, such as median, quartiles, outliers, etc.
6. Correlation: Comparison between observations represented by two variables (X,Y) to determine if they tend to move in the same or opposite directions. For example, plotting project size (X) and the incidence of cost and schedule overruns (Y) for a sample of months. A scatter plot is typically used for this message.
7. Nominal comparison: Comparing categorical subdivisions in no particular order, such as project performance by type, size or business division. A bar chart may be used for this comparison.
8. Geographic or geospatial: Comparison of a variable across a map or layout, such as the portfolio performance by office location. A cartogram is a typical graphic used.

Using portfolio infographics

Portfolio infographics can demonstrate critical insights in an instant that are otherwise buried in the data:

• What types of risk are the most common across the portfolio?
• Is there a particular type of resource bottleneck that presents regularly? 
• Is vendor management a recurring problem?
• Is there evidence that project managers are consistently demonstrating forms of cognitive bias in their estimation, such as being overly optimistic?
• Are there particular “break points” in the portfolio (ie, projects or programs upon which there is a high degree of dependency by other projects and programs)? What is the health of the most significant break points? What is the combined value of 'at risk' investment dollars or benefits dependent on these break points? 
• How does the health of tier 1 projects and programs compare with less critical projects and programs?
• Is there a point of increasing complexity at which portfolio performance and throughput tends to 'fall off a cliff'? Is the organisation allowing sufficient schedule and budget contingency in its tier 1 projects for these realities? 
• What is the combined value of investment dollars or benefits by overall traffic light rating? Is the position improving over time? 

Benefits of portfolio intelligence

This information is far more valuable than a list of projects and programs with health indicators. It facilitates evidence-based decision making for steering the portfolio and resolving problems relating to capability and capacity, and provides the business case for action to address systemic problems. This is a more appropriate focus for portfolio governance boards.


The power of this information in the hands of senior management is significant. It can be applied to everything from which risk categories to address first, to where to focus recruitment efforts, to how to tailor project management training needs to get the most value for the organisation.

Roles

This has implications for PMOs. In particular, the role of the portfolio analyst needs to become more analytical and versed in data visualisation techniques to be able to surface key messages and trends out of the detail of portfolio data. In most cases, PMO personnel come from a delivery background, and adopting portfolio analysis and visualisation techniques and approaches requires a significant degree of upskilling.


This in turn highlights the importance of the role of independent portfolio assurance. Assurance, through health checks and other reviews, of your most important projects and programs remains essential. However, a portfolio intelligence expert can give you immediate access to the relevant analytics and visualisation skills needed to bring your portfolio information to life. Such services are a highly cost effective use of your assurance investment, resulting in improved portfolio performance and knowledge transfer.

You can get the full presentation here.