Lloyd's three year
targeted validation plan – how will you put yours together?
7 April 2017
Lloyd’s has released a market bulletin setting out internal model validation requirements for the 2018 YoA and requested that managing agents submit a three year targeted validation plan by 23 June 2017. The objective is to encourage firms to take a more targeted approach to their internal model validation. So with less than three months to put your plan together, how can you meet the new requirements and get some value out of the process?
Lloyd’s state that the approach is not intended to increase resource being devoted to validation. However, in our view, careful planning will be needed to achieve this.
The market bulletin sets out the requirements on:
- the information to be submitted in the three year plan
- validation to be carried out every year (“core validation”)
- validation that may be carried out on a three year cycle (“deep dives”)
- circumstances that should trigger further validation (“triggers”)
- Lloyd's do not explicitly define a deep dive in the note, but from the context it seems be any testing that is not done every year. We expect this to be clarified in the planned workshops, along with whether the three year clock starts ticking now or whether the requirements will look back to what you validated in previous years
What should your three year plan cover?
Managing agents need to submit a plan that:
- includes a full list of core validation tests to be performed annually
- sets out “deep dive” reviews planned over the next three years
- provides rationale for the approach
- This is likely to require additional work for many agents, but is an important step to ensure the three year cycle achieves targeted validation without just adding more work
A helpful starting point is to look at the testing that has been planned or performed over the last few years. This can be split into three areas:
1. Clearly core
The core tests will be run every year. These are the most significant tests, because they address:
- material parameters
- a required test type (such as Profit and Loss attribution)
- the overall results
There is a risk that too much testing is classified as core. The core tests should be focused on the most important risks and critical business uses. For the tests that are classified as core, the focus should be on efficiency. It is worthwhile investing in the design of these tests as they will be performed each year. This could be achieved through the reporting process so that the results are automatically generated, in a way that makes it easy to judge whether or not the test has passed.
2. Definitely deep dive
For material areas this is the time to think deeply about the methodology for the most important risks to your business. Deep dives will also include validation of the non‑material parameters that are not covered by the core testing.
This is a chance to think about what you want to achieve from the deep dive review. Perhaps the modelling of a less material area could be simplified, and by looking at both the methodology and the parameters together, possible alternatives may be clearer. For the more material areas it is an opportunity for you to research or try out alternative modelling techniques which may be more appropriate.
3. The minefield of the middle ground
Not everything will be clear cut. For the remaining tests it is helpful to identify the triggers that might make you want to run them in future. This will help justify why they needn’t be core tests and cut the volume of work committed to on an annual basis, whilst ensuring tests are run when needed.
By focusing on triggers listed in the guidance (such as external environment or model changes) you can ensure efficient and effective testing in this middle ground.
This is an opportunity to make your approach to internal model validation more efficient and focused. In putting your 3 year plan together, make sure you capture views from around the business to get everyone on the same page.
Discover how our independent validation advice can help you to use your models with greater confidence and to meet regulatory requirements efficiently.