Table-First Guarded Reasoning: Outperforming Larger MLLMs in Chart Question Answering

Multimodal Large Language Models (MLLMs) have revolutionized visual understanding but frequently suffer from “visual hallucination” when extracting precise numerical values from charts. Existing solutions often mitigate this by scaling parameters to massive sizes (e.g., 70B+ models), creating significant computational barriers for resource-constrained environments. To address this, we introduce Table-First Guarded Reasoning (TFGR), a resource-efficient pipeline that decouples visual data extraction from logical reasoning. By integrating a dedicated chart-to-table extraction module (Chart2Table) with Zero-Shot Chain-of-Thought (CoT) prompting, we provide the model with structured tabular context to “guard” against perception errors. We evaluate our approach on the ChartQA benchmark using the Qwen2.5-VL-7B-Instruct model. Our pipeline achieves 93.96% accuracy, significantly outperforming the 88.00% baseline. Notably, our 8.87B-parameter architecture (when accounting for Chart2Table) surpasses the State-of-the-Art Qwen2-VL-72B (88.20%) [1] on the OpenVLM Leaderboard, demonstrating that architectural innovation can substitute for raw parameter scale. We further validate our findings through an extensive analysis of explorative experiments in fine-tuning, architectural alternatives, and complementary tools.