ESMA’s Data Quality report (SFTR) – window dressing or reliable data?

ESMA’s EMIR and SFTR data quality (DQ) report 2021, published on Friday 1 April is the first to truly incorporate SFTR, after the very limited feedback provided in the 2020 report given the newness of the regime. We have covered the EMIR aspect of the report in a separate blog.

It is important to give careful consideration to the DQ report as it reflects some of the National Competent Authorities’ (NCA) and ESMA’s areas of focus. However we would caution firms against using this as the principle measure of data quality. This is because the methods used are flawed and unlikely to reflect the much broader scope of analysis used by NCAs (who are after all, your regulators) in order to detect market abuse, systemic risk and ensure complete, accurate and timely reporting.  

We continue to advocate the best, most pressing data quality consideration that ESMA should address is the need for regulatory certainty and tightly defined reporting field definitions where they do not exist today. On a more positive note, we have it explicitly in writing for the first time that,

“Reporting counterparties are expected to have processes, systems and controls in place to ensure completeness, accuracy and timeliness of the reported information.”

and

“Counterparties are strongly encouraged to use the regulatory data in their own internal risk and compliance management processes.” 

On a completely different note, one of the most interesting things to come out of the report centres on one reporting counterparty’s failure to report terminations of open SFTs, resulting in a significant exaggeration of the value of outstanding transactions across the market for most of 2021.  This is something firms should take note of and ensure that they are not putting themselves in the same position.

The 2021 DQ report focus

• ESMA launched the Data Quality Engagement Framework (DQEF) for SFTR in 2021 to coordinate efforts to supervise reporting counterparties and trade repositories (TRs) with the NCAs
• The DQEF was tasked with focusing on the timeliness of reporting, rejections and pairing. We would caution that while some (but not all) timeliness issues may be identified, neither rejections nor pairing statistics will give a material measure of data quality issues
• Even this round, not all NCAs were able to contribute to this analysis and in the meantime will focus on building systems and supervisory engagements
• The 2021 SFTR DQ exercises were performed on weekly datasets taken in June and November 2021, focusing on timeliness of reporting, reports rejected under the validation rules and unsuccessfully paired reported records.

ESMA’s conclusions

1. Reported volumes fell by approximately 50% following Brexit
2. Approximately 10% of SFTs are reported late (after T+1)
3. Rejections have been low (around 2%)
4. Duplicate reporting “does not pose major issues” although there is limited scope for identifying duplicates reported across two TRs, only intra-TR
5. Reconciliation pairing rates “have been only around 60%”
6. Reconciliation rates for loan and collateral data “have been low but increasing to around 40% and 30% respectively”
7. TRs do not agree on the number of records they reconcile against each other, indicating possible inter-TR reconciliation issues
8. ESMA believes that the use of ISO 20022 XML end-to-end reporting has brought quality and accessibility benefits and that it is already seeing positive trends in rejections and reconciliations
9. ESMA was able to identify one counterparty failing to terminate their open SFTs (presumably a systemically significant counterparty given the size of the apparent bias), a factor that continued to inflate the volume outstanding until November 2021 when the NCA ensured it sent back-loaded early terminations.

Kaizen conclusions

1. The vast majority of reporting issues that Kaizen’s ReportShield™ Accuracy Testing identifies relate to reports that have passed the TR validation rules. Our testing tells us that there remains widespread issues with ‘valid but wrong’ reports. These are reports that pass validation but still contain incorrect data. Likewise, having an SFTR report accepted by the TR is setting the bar very low.  Sadly, the revised validation rules will do little to alleviate this, with limited true “validation” (many totally implausible values will still pass as valid) and a significant lack of effective rules on field conditionality too.
2. To suggest that even a fully reconciled SFTR report is accurate is naïve in itself, given the prevalence of delegated reporting and data aggregators and enrichers in the reporting space.  Two wrongs may match each other but it does not make them right.
3. In order to illustrate how ineffective reliance on pairing & matching is with regard to data quality, it is important to estimate the proportion of reports that are subject to the reconciliation, paired and successfully reconciled. The honest answer is that these figures are extraordinarily low.

   For example, the public data* tells us that at most during any period, on average, only c.20% of reports are two sided.  Of these, ESMA states that around 60% of them pair (12% of total reports). Of this 12% of paired reports, 40% of loan reports match (less than 5% of all reports) and around 30% of collateral reports match (less than 4% of reports).

   Continued and persistent focus on ineffective DQ measures that encapsulate less than 5% of reports seems a very inefficient mechanism for ensuring “the usability of the data for monitoring of financial stability risks.”

4. If you wish to truly establish the data quality of your SFTR reporting, we recommend that reporting counterparties receive independent quality assurance through frequent, periodic regulatory testing.

To reflect on the most poignant observation from this data quality report, a reporting firm’s internal controls should bear in mind that transactions remain live in the database until they reach their contractual maturity date, a modification adds a contractual maturity date or they are early terminated. Reporting open transactions and forgetting is likely to be met with the wrath of the NCA.    

• For a conversation with Jonathan about your SFTR Reporting data quality or a no-obligation review of the quality of your reporting, please contact us.

*For a consolidated view of the public data, visit ICMA’s website.