For my work at PayPal, as a Second Line of Defense, our usual task is to validate models built by the ML modelers in the First Line of Defense. I have listed the models I have validated and have improved as follows:

Validation Areas

Credit

• Business Loan Underwriting Model
• Business Loan Fraud Application Detection Model
• Buy Now Pay Later Underwriting Model

Fraud

• Onboarding Fraud Detection Model

Customer Complaints

• Complaints Identification Model
• Complaints Categorization Models

Challenger Models

• Through comprehensive model reviews, I conduct in-depth performance analyses that occasionally reveal optimization opportunities not initially captured in the original model design. When such opportunities are identified, I develop challenger models to test whether alternative approaches can deliver superior predictive performance.
• I've specialized in validating and creating challenger models for PayPal Credit/BNPL.
• Most common issue with the existing models I have observed so far are the quality of the model inputs, especially the lack of useful features.

Learning Through Implementation Challenges

My approach to model development involves rigorous experimentation with advanced techniques, each providing valuable insights into the practical constraints of enterprise-scale financial modeling:

Challenge 1: User Embedding Integration for BNPL Models

I explored incorporating user representation embeddings into our existing LightGBM framework for PayPal's Buy Now Pay Later product. The concept involved learning dense representations from user transaction sequences through a decoder layer (32-128 dimensions) to capture behavioral patterns that engineered features might miss, particularly for edge cases and new user segments. While the approach showed technical merit and gained leadership support, implementation revealed critical enterprise constraints: extended development timelines (3-6 months), infrastructure complexity requiring PyTorch integration unfamiliar to banking modelers, significant interpretability challenges due to embedding abstraction, system stability risks, and ultimately modest performance gains (1-3 KS improvement) that didn't justify the operational overhead.

Challenge 2: Product-Specific Feature Engineering for BNPL

When tasked with addressing performance issues in a BNPL underwriting model that had been adapted from PayPal Credit architecture, I identified fundamental misalignment between the model features and BNPL user behavior. BNPL users—typically younger, credit-averse, impulse purchasers—required different signals than traditional credit users. Point-of-sale context, purchase amounts, and short-term behavioral patterns were more predictive than historical spending averages and payment histories. Additionally, the user base's relative newness limited the effectiveness of traditional user representation techniques, necessitating completely different feature engineering approaches tailored to this demographic's distinct financial behaviors and seasonal patterns.

Challenge 3: Graph Neural Networks for Fraud Detection

I investigated GraphML implementation to enhance fraud detection across Venmo and PayPal's transaction networks, proposing graph neural networks to model account relationships and identify fraudulent activity clusters beyond our current two-tower transformer approach. However, the scale requirements proved prohibitive: processing 400+ million daily transactions would require graphs with billions of edges, creating O(|V| × |E|) complexity where memory requirements reached 200-800GB per layer. Graph traversal algorithms' poor cache locality and irregular memory access patterns, combined with model explainability requirements and network contamination risks, ultimately made our existing LightGBM and transformer architecture more operationally viable for this scale.

What a great modeler/data scientist would be:

The primary insight from these experiences is that enterprise financial modeling requires strategic thinking that simultaneously considers advanced methodological approaches and pragmatic feature engineering. When models degrade in production, the most effective intervention combines systematic feature discovery with careful evaluation of when sophisticated architectures can reveal feature interactions that traditional approaches miss.
A strategic modeler maintains vigilance toward both dimensions: identifying new predictive signals through domain expertise while remaining alert to opportunities where advanced methods (embeddings, graph networks, transformers) can capture complex patterns that engineered features alone cannot. The key is developing judgment about when the additional complexity is justified by measurable improvements in business outcomes.
The optimal approach requires balancing innovation with operational constraints—development timelines, infrastructure compatibility, regulatory requirements, model maintainability, and cost-benefit analysis. Superior modeling means designing solutions that push the boundaries of predictive performance while remaining implementable within enterprise constraints.