What is it about?
Banks and other monetary financial institutions increasingly rely on very large datasets generated by core banking systems and payment platforms. These data can support fraud detection and risk monitoring, but they are difficult to analyse because genuine distress and fraud events are relatively rare. This paper examines the technical challenges created by highly imbalanced financial big data, especially when data distributions change over time. It discusses problems such as limited examples of rare events, small disjuncts, changing distributions, and the difficulty of adapting conventional imbalance-handling techniques to distributed MapReduce environments. The study also reviews the role of artificial or synthetic data as a possible way to adjust imbalanced financial datasets and make them more suitable for large-scale analysis.
Featured Image
Photo by Aditya Chinchure on Unsplash
Why is it important?
Financial institutions need to detect fraud, distress, and unusual behaviour quickly, yet these events are much less common than normal transactions. This imbalance can make standard data-mining methods less effective and may cause important risk events to be missed. The challenge becomes greater in big-data environments because information must often be partitioned and processed across distributed systems. This can further reduce the representation of rare cases. Understanding these problems is therefore essential for developing more reliable fraud-detection and risk-monitoring systems in banking, especially as regulatory reporting and supervisory data requirements continue to grow.
Perspectives
Big data does not automatically lead to better risk detection. In financial systems, the most important events—such as fraud or institutional distress—may also be the rarest ones. This creates a fundamental modelling problem: algorithms trained on overwhelmingly normal observations may perform well overall while still failing to identify the events that matter most. The paper therefore highlights the need to adapt imbalance-handling techniques to distributed big-data environments and to investigate carefully how artificial data can be used without distorting the underlying financial behaviour.
Prof. Afshin Ashofteh
Universidade Nova de Lisboa
Read the Original
This page is a summary of: Mining Big Data in statistical systems of the monetary financial institutions (MFIs), July 2018, Universitat Politecnica de Valencia,
DOI: 10.4995/carma2018.2018.8570.
You can read the full text:
Contributors
The following have contributed to this page







