SMBC powers data-driven decision-making with Azure Databricks

One of the world’s largest banks, SMBC had historically taken a regional approach to data management, which created a fragmented data estate that hindered efforts to unlock data-driven insights.

SMBC Americas collaborated closely with Microsoft to reduce information silos using Azure Databricks. The company migrated more than 170 data attributes, creating a unified data estate to power data-driven analytics.

Since adopting Azure Databricks, SMBC has seen a 67% reduction in shuffle operations and improved run time by 40%–60%. Additionally, the journey has helped to shape a more consistent approach to data management.

“It was clear that we needed to modernize to meet our business goals and better service our clients. At SMBC, ‘customer first‘ is one of our five values, guiding how we support our clients and the business. Azure Databricks helped us take a more strategic approach to managing data, with stronger governance alongside data movement.”

Gordon Wilson, Chief Information Officer, SMBC Americas

SMBC Americas is unlocking the power of a unified data estate with Microsoft Azure solution Azure Databricks. SMBC onboarded more than 170 data attributes into its SMBC lakehouse and is harnessing the visualization tools of Microsoft Power BI to transform executive decision-making with data-driven analytics.

Unlocking the full business value of data

SMBC is the primary operating bank of Sumitomo Mitsui Financial Group, one of Japan's three largest banking groups and one of only 29 designated Global Systemically Important Banks (G-SIBs) worldwide. As a G-SIB, SMBC operates under the highest standards, with total group assets exceeding $2 trillion USD, operations spanning 39 countries and territories, and a legacy rooted in over 400 years of financial stewardship.

With such an extensive global footprint, SMBC’s regional divisions have historically adopted differing approaches to data management, contributing to a fragmented data estate, which limited aggregation and insight. “We recognized that data offers business value, but we couldn’t unlock its full value because it was stored in isolated, fragmented areas,” says Thomas Sawyer, Executive Director, Platform Architecture, at SMBC.

SMBC Americas began considering how to consolidate its data estate to lay the foundation for AI tools and data analytics. “It was clear that we needed to modernize to meet our business goals and better service our clients,” says Gordon Wilson, Chief Information Officer at SMBC Americas. “At SMBC, ‘customer first’ is one of our five values, guiding how we support our clients and the business. Azure Databricks helped us take a more strategic approach to managing data, with stronger governance alongside data movement.”

Consolidating more than 170 data attributes into Azure Databricks

The company’s transition to Azure Databricks jump-started a transformation throughout the organization. SMBC Americas used Azure Data Factory, a data integration service, to bring its data into an open, governed lakehouse. SMBC has already migrated more than 170 data attributes to its new lakehouse.

SMBC incorporated Microsoft Entra ID to control data access for groups and individuals. This approach gives SMBC greater visibility into its data while still maintaining the necessary controls.

“Once you have data aggregated into the lakehouse, you start to see how the disparate data elements come together to build new data products and unlock opportunities.”

Thomas Sawyer, Executive Director, Platform Architecture, SMBC Americas

SMBC is also using the visualization tools in Power BI to present analytic insights derived from its newly unified data estate, empowering senior leadership to make more informed decisions. “Once you have data aggregated into the lakehouse, you start to see how the disparate data elements come together to build new data products and unlock opportunities,” says Sawyer.

Driving efficiency and cultural change with unified data

The new Azure Databricks infrastructure is powering backend efficiencies, including a 67% reduction in shuffle operations, where data is redistributed across partitions. SMBC also lowered memory usage by 50%–70% and improved run time by 40%–60%.

The transition to Azure Databricks has also played a role in advancing change across the organization. Instead of regional divisions creating their own technology solutions, SMBC is adopting a holistic approach supported by executive leadership. “Our COO has been a critical supporter of this program, and his push for a global agenda has made this journey faster and more successful,” says Ketan Thakker, Chief Architect at SMBC Americas. “It’s critical for us to share data across the regions.”

Embracing a global vision for the future

For the team at SMBC Americas, this transformation has laid the foundation for future innovation. They are excited to continue building out new capabilities and changing company culture. “Change is always a challenge and always risky,” says Wilson. “I’m proud of my team for creating the momentum for change.”

“We moved from being a Microsoft customer to working more closely as a partner. There’s a shared understanding that long-term success depends on mutual outcomes.”

Thomas Sawyer, Executive Director, Platform Architecture, SMBC Americas

SMBC’s evolving relationship with Microsoft, marked by close collaboration between teams, has supported this change. “We moved from being a Microsoft customer to working more closely as a partner. There’s a shared understanding that long-term success depends on mutual outcomes,” says Sawyer.

Discover more about SMBC on LinkedIn, X/Twitter, and YouTube.