Whitepaper

Customer Feedback as Governance Infrastructure.

An External Accountability Layer for Automated BFSI Systems

A practical framework for turning customer feedback into operational, regulatory and executive intelligence, independent of the institutional systems it observes.

Read Online Download PDF

Working Paper

Status

June 2026

Published

1.8M reviews

Dataset

30 institutions

Coverage

India + West Africa

Geography

At a Glance

The paper in six boxes.

Problem

Governance frameworks for automated BFSI systems depend on signals generated by the same systems they're meant to govern. When that infrastructure fails, the failure is often invisible to the mechanisms designed to detect it.

Approach

Customer feedback is structurally independent of internal reporting. A three-stage pipeline extracts, classifies, and synthesizes it into attributed operational diagnostics no institution can suppress or delay.

Architecture

A multi-model classification pipeline, a nine-stage analytical pre-processing layer, evidence-constrained LLM synthesis, and a decision engine producing prioritized, owner-attributed findings.

Dataset

1.8 million customer reviews across 30 BFSI entities in India and West Africa, spanning commercial banks, neobanks, and fintech payment platforms.

Validation

98.47% token-level extraction accuracy, 95.47% classification accuracy, macro F1 0.915, and roughly 97% owner attribution accuracy validated across four West African institutions.

Key Contributions

An independent external accountability layer, institution-aware taxonomy restriction, evidence-constrained root cause analysis, and decision intelligence with revenue exposure ranges.

Problem

The systemic observability gap.

Existing governance frameworks for automated financial systems share a structural flaw: they depend on signals generated by the same systems they are meant to govern. Internal audit logs, system dashboards, and incident reports are produced by institutional infrastructure. When that infrastructure fails, the failure is often invisible to the very mechanisms designed to detect it.

Customer feedback is structurally different. When an automated payment pipeline silently debits without crediting, or a KYC verification loop traps a customer in a broken authentication flow, the failure appears in customer complaints within minutes, before it surfaces as an attributable pattern in institutional reporting, and before a regulator receives a report.

This framework reduces failure detection latency from weeks to hours.

The Numbers

Validated at real scale.

Customer reviews processed

BFSI institutions covered

Taxonomy labels

Classification accuracy

Entity extraction accuracy

~97%

Owner attribution rate

Architecture

Two pipelines. Raw text to executive report.

The framework operates as a three-stage pipeline, expanded below into the nine and eight sub-stages that make it up.

FIG. 1 — CLASSIFICATION PIPELINE: RAW REVIEW TEXT TO ATTRIBUTED CLASSIFIED CORPUS

Raw Customer Review Text

App reviews, complaints, social feeds.

Named Entity & Aspect Extraction

Multiple spans extracted per review as (entity, aspect) pairs.

Keyword-Augmented Aspect Verification

21,711 keyword strings across 47 rule groups, 6 domains.

Canonical Label Resolution

Deterministic lookup from 13,576 manually tagged pairs.

Institution-Specific Label Restriction

Label space restricted to the institution's product profile.

Taxonomy Classification

107-class DeBERTa model; 116-label full taxonomy.

Sentiment & Polarity Scoring

Polarity label and confidence score per span.

Domain Ownership Stamping

Primary domain, secondary domain, scope flag per span.

Classified Review Corpus

Span-level attributed, domain-stamped, fully traceable.

FIG. 2 — INTELLIGENCE PIPELINE: CLASSIFIED CORPUS TO ATTRIBUTED OPERATIONAL REPORT

Classified Review Corpus

Domain-stamped, polarity-scored spans.

Nine-Stage Signal Computation

Trends, anomalies, journey health, co-occurrence, release correlation.

Impact Score Eligibility Filter

Labels scoring below impact threshold 30 excluded from RCA.

Operational RCA Packet Assembly

Pre-computed signals bundled per failure cluster per institution.

Evidence-Constrained LLM Synthesis

L1 observable findings, L2 operational hypothesis, guardrail enforced.

Decision Scoring and Prioritisation

BIS, VaRI, SBI, SRI, TEI indices; quadrant and revenue range.

Institutional Owner Attribution

13 functional owners; 4-tier resolution; ~97% attribution rate.

Executive and Operational Reports

CXO brief, operational report, competitive intelligence brief.

Core Contributions

What's actually new here.

Independent external accountability layerBuilt entirely on public data, so the signal cannot be suppressed, filtered, or delayed by the institution being monitored.
Institution-aware taxonomy restrictionThe label space is restricted at inference time to each institution's actual product profile, so a payments-only fintech is never evaluated against lending labels it doesn't operate.
Evidence-constrained root cause analysisA forbidden inference guardrail prohibits asserting backend or infrastructure causes without explicit review evidence. Findings are hypotheses, never confirmed diagnoses.
Regulatory obligation mappingEvery span carries a complete, inspectable attribution chain from raw text through to domain scope, supporting supervisory and algorithmic accountability requirements directly.
Decision intelligenceFive composite indices and four action quadrants turn raw findings into prioritized, revenue-ranged decisions rather than an undifferentiated list.
Executive reportingFindings resolve to a named institutional function through a four-tier attribution process, validated at roughly 97% accuracy across four institutions.

Early Evidence

Three institutions, two geographies.

Validation across three institutions demonstrates cross-regional generalisability. Full detail, including a fourth West African cohort, is in the paper.

Institution A · West Africa · Commercial Bank

Authentication dependency on a deprecated tool creating systemic access failure.

13 spans extracted · 8 unique taxonomies · owners across Security, Payments Ops, Engineering, Core Banking

Institution B · India · Fintech Payment Platform

Regulatory verification failure blocking customer access to funds.

9 spans extracted · 9 unique taxonomies · owners across Compliance, KYC Ops, Support, Service Ops

Institution C · India · Private Sector Bank

Sequential exit journey detectable two stages before account closure.

9 spans extracted · 9 unique taxonomies · owners across Rewards, Cards Ops, Retail Banking, Product

Scope and Limitations

Public review platforms exhibit selection bias, customers who complain publicly aren't representative of all affected users, and silent failures affecting digitally excluded populations may not surface. Coordinated review manipulation is a data integrity risk this framework must account for. The system detects customer-perceived failure, not confirmed system failure, operational hypotheses require institutional verification before remediation, and revenue estimates carry inherent uncertainty. This positions the framework as a complementary observability layer, not a replacement for internal monitoring.

Customer Feedback as Governance Infrastructure

Rohit Sarang
Finlytix AI · Working Paper
June 2026

Read the complete paper.

Full methodology, the complete nine-stage analytical pipeline, decision engine formulation, and institutional case evidence across 30 BFSI entities. Free to read online or download, no form required.

This paper describes the complete architecture behind the intelligence engine shown throughout this website.

View PDF Download PDF

Want to see this pipeline applied to a review from your own product?

Talk to us

Buried insights cost you.
Finlytix doesn't.

Share your details and we will take you through how Finlytix identifies emerging issues and failure drivers in your product space using current data and intelligence models.