The anatomy of a resilient phishing detector: an in-depth analysis of a transformer’s interpretability, causality, and robustness

The increasing sophistication of phishing attacks necessitates the development of detection models that are not only accurate but also interpretable and secure. While Transformer-based models have shown great promise in this domain, their “black box” nature often hinders trust and deployment in critical security environments. This paper presents a comprehensive, multi-faceted analysis of a DistilBERT-based model for phishing URL detection, moving beyond standard performance metrics to investigate its internal mechanisms, causal reasoning, and security posture. We first establish a performance benchmark by training a high-performance DistilBERT classifier that achieves a 95.85% F1-score after robust threshold optimization, significantly outperforming traditional machine learning baselines. To deconstruct the model’s reasoning, we introduce a domain-specific probing method to identify and analyze “specialist” attention heads and contextualize these findings with established XAI methods like SHAP and LIME. A subsequent intervention-based ablation study on these heads yields a surprising insight into the model’s resilience: performance does not degrade when specialist heads are removed, revealing a high degree of distributed knowledge and functional redundancy. We evaluate theoretical adversarial vulnerability across multiple perturbation budgets (ϵ∈{0.1, 0.2, 0.3, 0.5}), finding that the model exhibits strong resilience against small perturbations (F1 > 0.95 for ϵ ≤ 0.1) while showing graceful degradation under stronger attacks, consistent with security benefits arising from distributed representations. We further evaluate a battery of operationally realistic, non-gradient obfuscations—including character transposition, leet substitution, homoglyph, and delimiter manipulations (e.g., micorsoft.com , paypa1-login.com )—and find that evasion success never exceeds 7.2%, with the subword tokenizer rendering such manipulations more detectable rather than less. Our findings demonstrate that the model’s strength lies not in a few critical components but in a distributed, non-brittle reasoning process, making it a promising component within layered phishing defense systems.