ESG 411: The ESG Data ChallengeHeading


By Liam Varn, Director

Click here for the complete white paper – ESG 411: The ESG Data Challenge

For those fresh to the concept, ESG is an opportunity to quantify the performance or risk of companies along the lines of Environmental, Social, and Governance factors. Investors are increasingly applying these non-financial factors to their analysis process and in turn placing pressure on those underlying companies to align their behavior accordingly. ESG metrics are not part of mandatory financial reporting, but most corporations disclose ESG data through sustainability reports or even as part of their annual financial reporting.

Social & Political Pressure Around ESG

ESG has proven a controversial topic within the financial services industry. In 2020, the Trump administration moved to ban ESG investing for tax-advantaged savings vehicles such as 401(k)s, although that news was largely overshadowed by the ongoing Covid pandemic. The topic was addressed again in 2021 when the Biden administration’s Department of Labor reversed course on discouraging ESG analysis.

But it seems that even as of 2019, ESG as a discipline had already reached critical mass. The world’s financial services firms launched slews of internal ESG initiatives. Most of these are part of larger strategies to capture the rapidly growing Millennial and Gen Z market share, both generations which largely reject the traditional business school mantra of corporations only existing to benefit shareholders. 

There is also a tremendous amount of social pressure on corporations today, perhaps especially on financial institutions. For banks and asset managers, there is a risk that by not implementing ESG programs they will be deemed socially out of touch. Furthermore, as part of the Paris Climate Agreement, banks in the EU are already facing pressure from various regulatory bodies to direct capital toward environmentally friendly companies. This manifests itself in various ways, from incorporating scenario analysis for the physical risks of climate change, to accounting for the costs associated with a transition to an economy with a lower carbon footprint. Now that the US has rejoined the Paris Agreement, the Federal Reserve is calling for financial regulators in the US to direct banks to quickly adopt similar new steps to manage climate-related risks.

ESG is Here to Stay…

… And the specific challenge we are focused on at Paradigm is that of data. Financial firms are accustomed to objective data, e.g., prices, rates, and tenor. No one argues about what a company’s financial performance was, and all the market data providers more or less reflect the same data values. Whereas calculating ESG performance (or ESG risk) is by its nature a subjective task, and that presents a number of new challenges for asset managers and banks attempting to source, aggregate, and rationalize the multitude of data options.

The primary way ESG data is calculated originates with the corporations themselves, who self-report. Corporate sustainability reports are consumed and digested by a number of ESG data providers, who typically place numeric score values or grades on companies. There are over 150 ESG vendor data sources, with new entrants constantly emerging. The old guard like Bloomberg, Morningstar, MSCI, S&P, and Fitch all have ESG data offerings or have acquired companies that do. There is also a substantial subset of investment data specifically around climate, the environment, and green bonds, as well as a cottage industry of tech firms scraping news sources and using online sentiment algorithms to evaluate companies’ social capital within an ESG framework.

Most asset managers and banks today subscribe to several ESG vendor data sets, and they often produce some of their own data in-house. In 2020 MSCI informally reported that of the asset managers they polled, most had at least three ESG vendor data sources in addition to any ESG data they calculate in-house. And at Paradigm we’ve seen that number increase dramatically across the financial services firms we support. Given the subjective nature of ESG evaluations, and the often disparate range of scores for the same data point on the same company from different data sources, it’s critical that firms form a methodology for rationalizing and weighing all the available data into a single firm-wide view.

Paradigm is helping financial services clients navigate various ESG journeys, from the design and build of new ESG portfolio construction tools, to creating new ESG performance analytics, establishing new methods and technologies to profile their customer base, and even ways to predict their customers’ preferences using AI. On the sell-side, banks are similarly evaluating their clients (and themselves) along ESG factors. For instance, a bank may want to see the environmental impact of companies they provide loans to, adjust their product offering in respect to their clients’ ESG preferences, or stress test various portfolios’ ESG performance. 

Every large capital markets firm has multiple ESG initiatives ongoing, and the most successful programs have a unified ESG data methodology. At Paradigm, we believe it’s critical that applications across the firm pull from the same golden source ESG data, rather than use different ESG scores for different applications or even having to pay for multiple subscriptions to the same data set. 

At this point it’s important to note that E, S, and G each have nearly endless potential components, and to view this exercise as just an effort of producing scores for each component would be an oversimplification. For instance, one investor may be comfortable with investing in coal mining companies and nuclear power utilities, but not in companies with oil fracking operations. Another investor might not care about the diversity of a company’s employees, but they refuse to own stocks of gun manufacturers. To provide that level of flexibility for different users it’s necessary for financial firms to view ESG data across multiple dimensions.

The ESG Data Process

So, the first step in the process is sourcing ESG data. Which data sources provide coverage for the factors a financial services firm is interested in? Which data sources best match the firm’s definitions? Which vendors seem the most reliable, and what are their subscription models and associated costs? And how much data is a firm willing to produce itself? No ESG data vendor covers every conceivable ESG factor for every company, and their methodologies are all different, so how does a bank or asset manager piece together the optimal assemblage of data sources to cover all their needs? 

The second step is aggregating that data. It’s problematic if different groups within the same firm are pulling data from different sources for different applications. For instance, at Paradigm we’ve seen cases where a client has built out one application for users to construct portfolios with consideration to ESG factors and another application to report ESG performance on existing portfolios; each application was sourcing data from different ESG vendors. Paradigm’s data management experts have stepped in to drive data aggregation and rationalization, centralizing and managing their ESG data in order to both control costs and foster internal alignment. 

The third step is rationalizing the universe of data a firm has aggregated. Once all of a firm’s ESG data is in one place, what happens when two or more sources inevitably reflect different values for the same metric? Does it make sense to average those values, or should one source take precedence over the others? Furthermore, what if the data sources reflect those values in different formats, e.g., one source provides a numeric score and the other provides a letter grade? It’s a tedious process, but ultimately a financial services firm needs to create a golden source consensus that reflects their firm-wide view of necessary ESG data.

Finally, once an organization has sourced, aggregated, and rationalized their ESG data, they can begin to build capabilities to better understand and serve their clients. It’s also very likely that these efforts will result in reduced data costs.

Acting on ESG | The Tools

It’s increasingly apparent that the challenges associated with ESG are especially novel and complex for the financial services industry. The good news is there is a spectrum of data management platforms and software solutions on the market, and Paradigm has partnered with many of them to help create specific utility in this space. For one thing, it’s convenient that the birth of ESG coincides with wide scale cloud adoption. Paradigm often implements Snowflake, and it is an especially useful platform for ESG data. It inherently solves the problem of centralizing a firm’s data, and it makes any data source available to any authorized user or application. Snowflake also natively offers subscriptions to several dozen ESG data providers through Snowflake Data Marketplace. (Amazon Web Services similarly offers ESG data subscriptions through its AWS Data Exchange.)

For firms that are in the process of moving existing data to the cloud, it might be useful to consider a solution such as Informatica MDM to understand, rationalize, and catalog ESG data. Having ESG data on the cloud, or at least centralized in one location, makes it available for applications across the firm and provides a future-proof foundation for any conceivable set of custom ESG applications. In terms of analytics, Tableau and Power BI are the traditional visualization tools, but new cloud-native technologies like ThoughtSpot are gaining traction and proving their value as business users become accustomed to data being at their fingertips.

Summary

Given that we seem to be well past the tipping point for long-term ESG adoption, it would benefit a financial firm of any size to take stock of their current ESG-related projects and align those into a larger business and technology strategy. The industry’s commitment to ESG is only growing, and ESG-related projects are mounting into fully fledged programs of work. Paradigm partners with financial services firms to tackle a wide range of data-driven challenges. We know that ESG adoption can be daunting, but with the right thought leadership and technical expertise we helped our clients through every step of the journey, from strategy and greenfield ideation to platform implementation and integration.

Click here for the complete white paper – ESG 411: The ESG Data Challenge

Recent Posts

Enabling Digital Intelligence Through ThoughtSpot

Written by: Azmath Pasha, Chief Digital Officer, and Kireet Kokala, Cloud Delivery Leader Click here for the complete white paper: Enabling Digital Intelligence Though ThoughtSpot ThoughtSpot, a modern cloud analytics company makes it easy for business teams to ask questions while providing power and flexibility for data experts. While ThoughtSpot’s platform has been around for almost a decade, it has recently enjoyed a larger audience thanks in part to data intelligence solutions that are being pushed by Snowflake Data Cloud, AWS, Azure, Google Cloud Platform, and several others.

Paradigm Welcomes Industry Veteran Peter Ku, Bolstering Data & AI Innovation in Financial Services

SCOTTSDALE, Ariz. (March 31, 2025) – Paradigm proudly announces the strategic appointment of Peter Ku as SVP & Global Head of Industries. A respected voice in financial services data strategy, Ku joins Paradigm at a pivotal moment of growth as the company deepens its footprint in the industry and accelerates its momentum as a trusted partner in enterprise data, AI, and business transformation.

Paradigm Welcomes Industry Veteran Peter Ku, Bolstering Data & AI Innovation in Financial Services

SCOTTSDALE, Ariz. (March 31, 2025) – Paradigm proudly announces the strategic appointment of Peter Ku as SVP & Global Head of Industries. A respected voice in financial services data strategy, Ku joins Paradigm at a pivotal moment of growth as the company deepens its footprint in the industry and accelerates its momentum as a trusted partner in enterprise data, AI, and business transformation.

Program Management for Leader in Home & Security Solutions

By synergizing client teams across lines of business and consultants, organizations can align efforts to meet strategic goals while ensuring critical path management drives on-time project delivery. Standardizing and simplifying processes, supported by strong project governance, establishes consistency and accountability. Leveraging repeatable project planning tools, templates, standards, and methodologies enhances efficiency and scalability, while increased rigor around testing and QA execution against defined business success criteria ensures higher quality outcomes and successful releases.

Supply Chain 360 for Multinational Consumer Electronics Retailer

By consolidating, standardizing, and modernizing supply chain reports and visualizations, organizations can improve supply chain business decision-making with greater accuracy and speed. Leveraging near real-time data through integrated self-service dashboards empowers teams with timely insights, while clearly defined KPIs ensure alignment with strategic goals. At the same time, eliminating data fragmentation creates a unified, consistent view of operations, enabling smarter, more agile decisioning across the supply chain.

Finance Analytics for Global Leader in Home & Security Solutions

By implementing clean automation, organizations can increase efficiency and reliability in monthly profitability reporting processes while reducing the need for manual intervention. Accurate functional data sets enable deeper analysis and greater confidence in decision-making, while automation helps minimize financial exposure. At the same time, breaking down complex legacy workflows paves the way for a modern, streamlined solution that enhances accuracy, speed, and scalability in profitability reporting.

Cloud Migration of Integration/API COE for Global Leader in Home & Security Solutions

By moving from on-premise to the cloud, organizations can significantly improve agility, performance, and productivity while modernizing their architecture to support the seamless movement of large quantities of data. This transition allows businesses to retire legacy systems, reduce infrastructure costs, and minimize technical debt, ultimately creating a more efficient and scalable environment. At the same time, leveraging modern cloud solutions increases confidence in data integrations, ensuring greater reliability, security, and adaptability to evolving business needs.

Customer Visibility for Transportation & Supply Chain Leader

Single source of critical carrier and shipper data. Simplify data sources, structure, and hierarches of shippers and carriers. Readily available customer data for price effectiveness, order management, and fulfillment. Integration of data between transportation management software (TMS), financials, and CRM.

API Management for Global 500 Construction Manufacturer

Effective and reusable data sharing between internal business teams and ecosystem constituents (ex: dealers). Easily adoptable and maintainable framework for data sharing and integration. Eliminate manual processes and improve accuracy/quality. IT and business collaboration for more governed processes
Business self-sufficiency. Cost optimization via platform rationalization.