Measuring success: How do we know if individuals are winning?

‘Measure your success’ is a theme that winds its way through this entire book. It’s simply something we haven’t tackled effectively as an industry or as fiduciaries. Over the years, we have acquired an impressive battery of dos and don’ts to help our funds and the individuals we serve achieve a better post-retirement income, design a more tailored employee benefits programme, determine a better medical plan, earn a better investment return, and produce more attentiongrabbing communication materials and more substantive financial education programmes. But have we paid enough attention to the most important question: are these efforts actually producing better results for individuals?

To address this, we need to tackle two important questions:

1. Is the structure or design capable of delivering the goods? Have we got at least that part of the problem correct?
2. Have we measured what happens at the individual’s level, and do we understand what dynamics inhibit the translation of good concepts into good results?

We can propose all sorts of intuitively appealing ideas to improve today’s conventional practices. But until we can demonstrate how these ideas actually change individual members' outcomes, they’re purely academic. We have used this book to propose a number of ideas that could be attacked on just such grounds: how do we know they will produce the results we would like? The frank answer is that for some of these ideas, we don’t. But it doesn’t mean we shouldn’t challenge convention and experiment. We want to start the conversations, and we welcome your thoughts in response. For a growing number of issues, though, we have at least established a base-case benchmark against which we can measure our progress each year as we learn more effective and efficient ways to deliver.

According to the Pension Funds Act, trustees have a fiduciary duty to look after the best interests of the members of the fund¹. Although it is not yet an explicit requirement that trustees monitor their members’ outcomes, it is only logical that fiduciaries assess whether the decisions they make really are improving outcomes. For financial service providers, the introduction of Treating Customers Fairly (TCF) legislation leaves us no option. We need to ensure that we are delivering on promises made to members. 

Achieving this outcome will demand a rethink of current conventions. Simply looking at the results of the activities of service providers is no longer enough.  It’s time we start monitoring the decision-makers and the aggregate value chain of delivery. Each year, we need to assess whether outcomes for members have improved or declined as a result of our collaborative efforts.

This means that technology will be key, and technology can now easily fulfil this role at little or no additional cost. In the case study that follows, we highlight the difference between monitoring the success of decisions at the hypothetical member level (what most trustees typically are exposed to) and monitoring success once we incorporate real activities and member choices. The differences are stark; the added value of the exercise is undeniable.

By doing this, we can also break down our assessment into its critical components:

1. Can the plan design deliver what’s required?
2. What changes should be made within the scope of the trustees that can improve outcomes?
3. Are there behavioural issues at the member level that might be affecting outcomes?

Applying this analysis allows us to gain some important insights into what’s working, what isn’t, and what would be the most effective strategy for improving outcomes. The fund in the case study that follows belongs to the public sector – the sector with perhaps the highest success rates for their members. But how successful have they really been?

STEP 1: Can the design deliver?

The graph on the next page captures the range of inputs and assumptions for the specific fund, including:

• Gross contributions average around 23% of pensionable pay per year.
• Costs, including for risk benefits, average around 2.5% per year.
• Investment costs average around 0.8% of assets under management per year.
• Expectations of performance of the fund’s strategic asset allocation, set out in the investment policy document, are for 4.5% real returns (in other words, after inflation) per year – for the next 40 years.
• We estimate that the alpha from the investment strategy is equal to the assumed average investment cost over 40 years. (Alpha is defined in the glossary).
• The fund’s normal retirement age is 63 years.
• Given most employees start work at 23, the investment term is about 40 years.
• We assume the pensionable pay represents 100% of employees’ take-home pay and when employees switch in or out of this fund, they preserve 100% of their fund credits. (While many boards of trustees start with these two basic assumptions, an important question that we will be asking is: are they correct?)

Using these assumptions, we prepare a scatterplot that projects what replacement ratio each member of the fund is likely to achieve when they retire. This projection is based on the strategy that the individual is invested in today, their contribution rates and costs, and how well we expect their broader investment strategy to perform over the time period they have left until retirement. Each dot on the graph represents an individual member – showing both their age and their projected replacement ratio. It thus accounts for their savings history to date.

In the top left of the picture we also see the projected replacement ratio for a new member entering into the fund at the age of 23 and continuing until retirement. Given the structure of all the inputs, costs and investment decisions, we would expect that new employee to be able to achieve a replacement ratio of 94% when they retire in 40 years’ time. It is this specific information that gives us an indication of whether the design of this fund is capable of delivering decent outcomes for individuals. This analysis suggests it can.

Each coloured dot on the graph represents how healthy a specific member’s projected outcome is. If the member has a green dot, they are on track to achieve a replacement ratio of 75% or more. But if a member has a pink dot then they are unlikely to be able to retire with a sustainable level of income in retirement unless trustees or the employer intervenes.

Projected replacement ratio of active fund members

STEP 2 : How do we stack up against other funds?

While those results are definitely in line with what the fiduciaries were targeting, what we don’t know is how well this stacks up against other funds in the same industry, or perhaps even in the country. The table below contrasts this fund with other comparable sector funds in the Alexander Forbes Member Watch^TM data set. This specific fund appears to stand head and shoulders above others in its industry – something an employer could really brag about.

For many trustees this level of comparison may not seem relevant, given their assumed range of responsibilities. But we believe that this is where trustees can potentially play their most valuable role in the future. If comparisons with other comparable funds show that this fund’s structure is inadequate for its members to reach a reasonable replacement ratio, then trustees and employers need to collectively identify why shortfalls are occurring. Invariably the problem will sit somewhere between these two bodies. So far so good. But let’s continue probing.

Benefits by the numbers

STEP 3: Getting to the crux of the problem

Up to this point, the data and outcomes we’ve been using employ the kinds of assumptions that most boards are shown: an idealised world where members behave and make all the right decisions. But what if we integrate the actual behaviour of members into our analysis? What happens to the outcomes then?

Pensionable pay and its impact
Let’s start with a point that often eludes boards of trustee or management committees. One of the first decisions a member may make (or the employer may make on their behalf) when they sign a contract of employment is what percentage of their total cost to company will be deemed pensionable. In other words, from what portion of the salary should the fund deduct their contributions and costs? The lower the pensionable pay percentage is, the higher the take-home pay. As such, there is often a strong short term incentive for employees to lower that pensionable pay.

But if trustees or management committee members are not aware of those choices, they may easily fail to appreciate that targeting a 75% replacement ratio on what constitutes only 50% of someone’s total cost to company is going to leave members severely short on retirement.

As our model on the next page illustrates, under such conditions the replacement ratio for a new member decreases from 94% to 48% when compared to their total cost to company. The grey dots show the projected outcomes for members using our previous assumptions. The orange dots show what the projections look like using the reduced pensionable pay percentage.

Projected replacement ratios of active fund members on a lower pensionable salary percentage.

STEP 4: How member decisions affect outcomes

Employee turnover and preservation

One way that member behaviour can seriously affect fund outcomes is by changing jobs and not preserving fund credits when they do.

To begin with, as the chart below highlights, the fund has an enviable record of low turnover with its employees.

But particularly problematic is the point that when employees do switch jobs, they typically cash out their accumulated savings instead of preserving it. In fact, the preservation behaviour of members of this fund has been significantly lower than members in either the sector or our member base as a whole. Just imagine what would happen to the fund if this behaviour persisted – that members continued to cash out their fund credits instead of preserving them. The net effect of these two elements is that the true outcomes for members might be significantly different from what was initially projected.

If we take the exit and preservation behaviour of members into account, and if we allow for the fact that pensionable salaries are 60% of an individual's total cost to company, the average replacement ratio across all fund members drops to 37.7%, as seen in the second graph on the next page.

This insight appears to be something of a show-stopper and would indeed persist as one – known only to the members when they actually retire – if we are unable to integrate both the actual member experience in the fund and the projected experience. But just as this monitoring framework changes our focus dramatically, so does it allow us to see solutions through a more effective lens.

Exit rate of members of different ages

The average preservation rate for the fund over the last year was 1.9% while the average preservation rate for the public sector was 7.8%.

Preservation rate of members of different ages

Average projected replacement ratio

STEP 5: So what can be done?

Given the effects of member behaviour, are there design features that trustees or employers can change to improve outcomes? For most boards, the following three options reflect their usual starting point:

• Could we change the investment managers and increase the alpha of the fund (return in excess of the strategic asset allocation benchmark)?
• Could we revisit our strategic asset allocation, or perhaps our estimates of expected returns on those asset classes?
•  Could we gain significant advantages by lowering our costs?

The question is whether any of them can take our member outcomes to a 75% replacement ratio.

The surprising answers are: no, no and no!
To move from a 37.7% replacement ratio to a 75% replacement ratio, the fund would have to generate more than 6.5% real return (in other words, after inflation and after costs) – consistently over the next 40 years. If we look back on the last 12 years, this seems relatively simple. We’ve easily achieved that average 6% real return. But in the wake of the global financial crisis, prospects going forward are grim at best.

The same story applies to upping the potential for alpha in the fund. The operative word here is ‘potential’. To shift the fund to a 75% replacement ratio would demand a consistent, 40-year outperformance of the strategic asset allocation benchmark of 1.7%. Again – this was do-able during our extraordinarily strong bull market of the last decade – but for this to persist for another 40 years is highly unlikely.

The other interesting part of this exercise, though, is that reducing costs also doesn’t have the impact we would expect. Because asset management fees are typically charged as a percentage of assets under management each year and administration fees are typically charged as once-off fees each time the funds flow in, reducing asset management fees has significantly greater impact than reducing risk benefit costs or administration fees. Over the course of 40 years for the fund, a reduction of 0.5% in fees for asset management can translate into an additional seven percentage points to the replacement ratio, whereas a reduction of 0.5% in administration and benefit fees adds a mere two percentage points to the final replacement values.

By playing with all the relevant inputs, we can now see how hard it is to use design features to compensate for poor behaviour. To some extent, changing member behaviour links inextricably to the quality of communications trustees and management committee members provide to their fund members. But helping members understand the impact of their decisions provides only a small part of the required correction. Trustees and management committee members may have to engage with the employer to see if changes in HR policies and communications can limit the potential damage that employees could inflict on themselves. But again, it takes time and careful planning to make sure employment policies and retirement benefits are properly aligned.

As we suggested at the outset, our ultimate level of assessment is to be able to measure the impact of every decision that we, the fiduciaries, make on the outcomes of each and every member. We know that the impact of each decision will likely be different on each member because their starting point, time frame and cost structures are all likely to differ. As such, monitoring at fund level for the average member simply isn’t going to cut it in the future.

In short

As of now, strategy dashboards can allow trustees to drill down to the individual member level and focus on an individual who has a particularly poor replacement ratio outcome (they have a pink dot). This informs the type of specific communications required to address the particular challenges that that member or group of members face.

By doing this analysis, we radically enhance our ability to have meaningful conversations with our members. Whether we talk to them through a broad distribution communication strategy or one-on-one through their benefit projection statements, we can now reach out with specific advice on how exactly members could get back on course. We are able to give individuals specific recommendations on what they could do and, as such, what expectations could be reasonable in light of any changes they might act upon. An analysis of this nature would need to be carried out regularly to ensure that any design changes or interventions implemented after the last analysis have truly improved outcomes. If changes like targeted communications or financial education programmes don’t result in the anticipated improvements, then we might well have to demand another approach.

What constitutes a successful strategy may change over time, as employees and the environment change. For instance, an SMS communication to members who are ‘in the pink’ may work well for a few years, and then fail as apathy sets in. As such, all of these action plans and strategies must be part of an ongoing dynamic process – one that needs to be monitored and measured each and every year. All of this brings us back to the core message: it’s about the individual’s financial well-being. Monitor it and if things aren’t working, change them and then monitor those changes. If this is done correctly, over time, it can make a big difference.

Measuring fund managers' performance against their mandated benchmarks tells us something about how that service provider performed. This is an essential step. But it tells us nothing about whether we got value for money given the costs of that solution (or those underlying fund managers). We also don’t know if we were adequately rewarded for the magnitude of risks assumed by those underlying managers. Measuring our aggregate manager performance against a strategic asset allocation tells us something about whether the solution was well designed to beat that specific benchmark. But it tells us nothing about whether members can retire with an adequate income replacement.

In effect, it’s time we rethink exactly what we are measuring in our investment strategies and why. Regulation 28 is quite explicit in what is required of us as fiduciaries. Members’ liabilities are central to the design and monitoring of a retirement fund’s investment strategy:

This then suggests that, of all the things we measure about our investment strategies, the most important one is knowing whether they are meeting our members’ liabilities.

What this then implies is that trustees need to be able to demonstrate that:

• The fund’s investment strategy is appropriately designed to address member profiles and retirement income requirements.
• This strategy remains appropriate over time.

Combine these two insights with conventional investment performance monitoring, and finally, trustees will have a proper framework for assessing whether any components of their investment strategy have become suboptimal. Liabilities, like assets, change all the time. As markets move, liabilities move. This means we need to give fiduciaries and key decision-makers a tool that can assess these changes and their implications. Think of this as a live liability monitoring tool.

Simply put, the tool provides regularly updated projections of returns and evaluates whether the existing strategy is still expected to meet a fund’s liability target. The tool’s purpose is to act as an early warning system for identifying inadequate expected returns, which may trigger a full ALM. What a full ALM can then add to the mix is whether the composition of the membership has changed in any way and if so, how this too can have an impact on whether the fund strategy is still appropriate.

Tackling the problem head-on 

Below we use the example of a typical member invested in a life stage investment range, targeting a replacement ratio of 75%.

What makes this graph different from performance graphs that are generally presented to most investment committees, is the orange and grey stack bar. Taken together, this orange and grey stack represents the rate of return that is required to obtain an expected replacement ratio equal to the target. For example, here members need a return of 11.1% per year (0.6% plus 10.5%) from the highgrowth portfolio in their strategy to reach a targeted replacement ratio of 75%. The tool deconstructs this into the expected return based on the asset allocation of the portfolio (10.5% per year shown in grey in our example) and the current shortfall (0.6% per year in our example). This 0.6% per year is effectively the alpha that would be required beyond that projected asset class returns if the solution is going to meet the member liabilities.

Trustees should monitor this shortfall to ensure that the objectives in the fund and the strategies to reach these are reasonable. A large shortfall of projected return against the required return may indicate a fund needs a change of asset allocation strategy. But there is only so much that an asset allocation strategy can be realistically tweaked without introducing unacceptable risks. Really large shortfalls may require trustees to consider alternative inputs into the projected outcomes, such as a change in contribution rates and retirement ages.

In the example used, new members could expect a replacement ratio of 66% (which assumes the pensionable salary is 100%). Here the tool calculates how much more members would need to contribute to achieve an expected replacement ratio equal to the target set. In our case study here, members would need to contribute an extra 2.6% of their salary to get an expected replacement ratio of 75%. Contributing 2.6% more would have the same impact as earning 0.6% per year alpha. But really large shortfalls may also require a revisit to a detailed ALM exercise. At that point, our process starts again.

In 'The heart of the matter' we discussed how members’ healthcare needs change over time. We also discussed the increasing levels of complexity in the healthcare system and how members struggle to understand exactly what they are covered for.

In their role as experts, healthcare consultants will give employers advice on the range of healthcare cover to offer to their employees after taking into consideration the specific industry and the profile of the membership base. Based on the choices on offer, the healthcare consultants will then advise members on which choice of healthcare cover would suit them best.

While the decision of benefit design is not in the control of the employer or the employee in the open medical scheme market, one advantage members have in this environment is that they can change their option within their scheme once every year. This enables them to choose appropriate cover each year and should therefore aid in avoiding over- or underinsurance (without taking affordability into consideration).

So how can we help members here? One way to promote a proactive response from members is to provide them with a measure of their utilisation rates in their plan’s benefits over the previous year. This gives us a retrospective insight into whether the member and their family are over- or underinsured. We can then send intervention letters to members to prompt them to contact their healthcare consultant and get advice on the best option to choose for the upcoming benefit year.

Because of confidentiality issues, the data available for this analysis is somewhat limited and only high level information on benefit utilisation is available. In addition, very few people record full information on out-of-pocket expenditure. That said, what data we do have is particularly valuable.

Even with limited data, we can send intervention letters for the following situations:

• Members who potentially face a large self-payment gap and hence out-of-pocket expenditure. This may be because they chose the wrong option or they used their day-to-day benefits inefficiently.
• Members who are overinsured. This is based on individuals’ medical savings account (MSA) balances, accumulation of day-to-day spend towards the threshold benefits, as well as whether they are registered for the chronic illness benefit. Where members are not registered for chronic conditions, have high MSA balances and are not expected to reach their above threshold benefits (ATBs) during the year, they are likely to be overinsured and may be able to benefit from downgrading to a lower cost option. The lower option would still give the member enough cover (based on the expected claims pattern), but they would save money they could use elsewhere.
• Members who are underinsured. This is based on the MSA balances. If the member’s full MSA is exhausted at a certain point during the year, it may indicate that they are underinsured and need more cover.

The intervention analysis is based on only a subset of information and upon further investigation, which would occur as part of a full needs analysis, the overall recommendation may change. The additional information could include considerations such as affordability constraints, expected medical procedures in the future, once-off events that occurred in the previous year that are not expected to be repeated, the level of subsidy, or just-in-case cover. Overall, however, the intervention process should still help to prompt the majority of members who are deemed to be potentially on the wrong option, to consider a change.

Measuring the number of interventions made each year, and whether members respond to these, would assist healthcare consultants to assess whether their advice is adding value to these individuals' lives. A lower proportion of interventions from year to year would indicate that a larger proportion of the membership base is on the right option, and we could assume that this is a result of the advice provided.

Additionally, measuring the number of option changes each year gives an indication of whether members are settling into their benefit options and understanding the benefits on offer. A high level of movements would indicate that members are on the wrong option, that their needs have changed, or that they are struggling to settle down. Putting these concepts into real terms, let’s consider a subset of our client base in 2013. During the year-end process, we performed the intervention analysis.

The table below shows the percentage of members who were identified for intervention and of those, the percentage of members who either upgraded, downgraded or did not change options. Note that not all members who received intervention letters would have sought advice or changed options.

Of the group of members assessed, 22.8% received intervention letters and of these, 8.4% changed options. This may or may not have been as a direct result of the intervention letter, but it is likely that the letter prompted a more measured approach to their option choice for the 2014 benefit year. Further to this, the changes made resulted in a savings to total contributions of 0.7%, showing that intervening in members’ option choices each year helps them to choose the most appropriate cover and to not overspend on medical scheme contributions.

Conclusion

Ultimately, the objective of everything we have discussed so far has been to improve outcomes to individuals in some meaningful way, whether they are a member of a pension fund or medical aid scheme, an employee or simply an interested investor. But what we have also highlighted throughout this discussion is that providing a comprehensive measure of these outcomes as they directly impact on individuals has been sorely lacking in our industry. We no longer have the luxury of simply stating that, as fiduciaries, we have confirmed that our funds are keeping pace with their benchmarks, or that we have adequately provided for health and risk needs, or that we have developed a good communications policy or financial education programme. We need to know that members are winning.

References

1 Section 7C of the Pension Funds Act