Regulatory expectations around the data that firms capture, retain, and monitor are increasing across Europe. For many regulated organizations, it is no longer enough to evidence that they’ve tried their best to capture data. The expectation is now that all business communication data is captured and retained in full – enabling regulators, investigators, and organizations to conduct full audits and reconstruct events end-to-end.
What is data completeness?
Data completeness isa state in which a dataset is retained as a whole. This means that it is free from missing data, corruption, or gaps. Complete data allows organizations to present a reliable picture of what’s happened in any given conversation or process.
For compliance teams, data completeness has become a catch-all phrase for the ability to answer three key questions:
- Reconstructability – can you recreate a chain of events across all channels and systems?
- Defensibility – using controls and audit trails, can you evidence that the captured data is whole and reliable, and that it hasn’t been altered, lost, or selectively deleted?
- Responsiveness – are you able to retrieve a complete record of events quickly to meet both internal investigations and external regulatory enquiries?
In the context of business communications, data completeness refers to a state where all business communication data required to be captured to meet regulatory requirements, from every channel, is captured and stored in its entirety – from the content of a message, through to associated metadata including timestamps, participants, and other relevant context.
For financial services, data completeness is particularly important in meeting MiFID II requirements. Article 16 requires firms to ensure that all records (including recordings of telephone conversations or electronic communications) are retained for all services, activities, and transactions.
In the context of trade and order reconstruction, data completeness means aggregating the data needed to understand a transaction lifecycle and detect potential misconduct. Firms are increasingly leaning on communications data to build a complete picture when trades are suspicious. This is reinforced by the Market Abuse Regulation (MAR)’s framework on detecting and reporting suspicious orders and transactions, as well as the sharing of inside information.
Having access to complete datasets allows all functions within a business to work from the same trustworthy data source, and unlocks multiple business benefits, including more accurate eDiscovery and faster searches and audits. However, the other side of the data completeness coin is that if datasets are incomplete, organizations may fall foul of regulatory recordkeeping rules – alongside in weaker risk-detection, risk-management, and compromised decision making.
What are the regulatory obligations for data completeness?
European compliance teams operate under a dense framework of requirements that require complete, accurate records, and reports be kept. Key examples include:
- MiFID II – Requires firms to capture telephone and electronic communications, retain them, and make this data accessible on request. It also requires client-facing communication to be fair, clear, and not misleading – the evidencing of which requires a complete data set.
- MiFIR transaction reporting – Obliges firms to report complete and accurate transaction details, with a focus on structured data.
- MAR – Mandates that firms have effective arrangements in place to detect and report suspicious orders and/or transactions. Complete data is essential for creating comprehensive Suspicious Transaction and Order Reports (STORs).
- Digital Operational Resilience Act (DORA) – Requires that firms maintain a comprehensive, accurate, and up to date ‘Register of Information’. Also obliges firms to ensure that data is accessible and retrievable in the event of disruption.
What are the benefits of data completeness?
Meet regulatory obligations
Ensuring that all business communications data is complete ensures that firms automatically meet a myriad regulatory obligations, from MiFID II to MAR and DORA.
Effective risk assessments
When something appears to have gone wrong, whether it’s suspected market abuse, non-financial misconduct, or a customer complaint, complete data is critical to establishing context and reconstructing a timeline.
More accurate analysis and surveillance
Identifying misconduct often relies on the ability to understand patterns and trends. To do this, firms must have the full picture. The European Securities and Markets Authority (ESMA) has highlighted that transaction reporting data is one of the most heavily relied upon datasets for identifying behavioural risk. Having communications data to understand the full audit trail is essential.
Operational efficiency and audit readiness
Complete, structured data reduces the time required to respond to legal or regulatory requests, investigations, and audits – and firms can win valuable “credit” with these bodies by responding to information requests quickly.
Four steps to maximize data completeness
1. Ensure data from all relevant sources is captured legitimately and consistently
The foundation for data completeness is understanding where your firm’s communications data subsides. Do you have visibility of all the channels employees are using to conduct business conversations?
MiFID II recordkeeping and ESMA guidance require that firms capture communications that lead to transactions, even if that transaction is placed later or on a different channel. Firms should understand where all communication is taking place and employ a strategy to legitimately capture this business data – including Microsoft Teams, Zoom, ICE Chat, and LinkedIn Messages.
2. Treat third‑party and outsourcing risk as a completeness risk
Data capture and retention frequently rely on third-party vendors. The more vendors a firm relies on, the greater the risk of data leakage, gaps, or security concerns. To ensure r data completeness, firms should assess their third-party vendor reliance and, where possible, look to consolidate third party risk. European supervisors have long emphasized the importance of access and auditability, which is now amplified through DORA. The fewer links in your chain of data, the lower the risk of weakness.
3. Break down data siloes
While all functions within a business work towards the same common goal, many will work independently of one another day-to-day. This data siloing can lead to situations where a lack of holistic overview or clear communication between teams results in risk.
Recordkeeping functions are, for example, “data owners,” and responsible for data ingestion and archiving. But often surveillance teams – who are responsible for reviewing communications – are working from different data sets or don’t have full oversight of all venues.
To avoid data gaps, recordkeeping and surveillance teams should look to share:
- A common inventory of channels and venues
- Shared monitoring and alerting for capture failures
- A consistent method for evidencing completeness
Ensuring teams have access to the same consistent, comprehensive, and reliable pool of data is essential to minimizing the risks from miscommunication or unclear roles and priorities.
4. Reconcile, reconcile, reconcile
European data regimes increasingly ask firms to guarantee validation and reconciliation of their data. The Securities Financing Transactions Regulation (SFTR) e expressly highlights validation and reconciliation mechanisms as key to reporting function. Similarly, MiFIR transaction reporting is now subject to structured completeness and accuracy testing, as well as supervisory follow‑ups where data is missing.
As such, validating each captured version of a message against the version from source is crucial to detect potential gaps, loss, or failures in captured data that may cause data to be incomplete.
Leveraging a solution that provides Constant Integrity Check (CIC) capabilities means data is constantly and automatically scanned and validated against audit events of every message’s lifecycle to verify that it is present, viable, and accessible throughout a retention term. This verification allows you to confirm that messages have been captured and stored completely, with all relevant data and metadata in place, and reduces lengthy manual reconciliation reviews.
The benchmark for data completeness has been set across Europe. Regulators expect firms to be able to evidence that they have captured, retained, and can retrieve all business communication data. Global Relay has a suite of data Connectors that plug communication data, from any channel, into a compliant archive.
