Rating Agency Accused of Using Original, Then Rewritten Data

Refinitiv ESG’s rewriting not a one-time event

Prof. Dr. Kornelia Fabisik, Assistant Professor of Finance, Frankfurt School, reports on some discoveries at one of the ESG rating agencies at the conference “ESG Ratings: The Good, the Bad, the Ugly” of the Corporate Governance Institute (Prof. Dr. Julia Redenius-Hövermann) at the Frankfurt School of Finance & Management.

Refinitiv ESG is a key Environmental, Social, and Corporate Governance rating provider offering “one of the most comprehensive ESG databases in the industry”, and its ESG scores have been used (or referenced) in more than 1,500 academic articles since 2003. The scores were initially constructed by ASSET4, a company acquired in 2009 by Thomson Reuters, which became Refinitiv in 2018. Refinitiv ESG data are employed by major asset managers, such as BlackRock, to manage ESG-related investment risks.

“We document large rewriting of ESG ratings of Refinitiv ESG.” The same set of firm-year observations downloaded September 2018 and September 2020 provides evidence. For the full sample Kornelia Fabisik observerd a rewriting of 21% on average, 18 % median.

“Rewriting is not a one-time event”, warns Kornelia Fabisik. “Refinitv ESG continues to adjust the data ex-post, unannounced.”

“In April 2020, Refnitiv ESG changed the methodology used to determine the ratings.” There were two key changes: Treatment of boolean metrics and introduction of a propietary materiality matrix.

The ESG score deviations are related to firm characteristics, especially past stock returns. Firms that performed better in the past experienced rating upgrades. The ESG score deviations strongly affect the classification of firms into different ESG quantities.

“Has data rewriting stopped post methodology change? No,” ist the answer of Kornelia Fabisik, “the database changes on a weekly basis.” She shows concrete examples.

“ESG ratings industry follows an investor-pay model, whereby the data vendors compete on how useful their ratings are for ESG investments.”

“44% of carbon emission obsesrvations (Scope 1 CO2 emissions) have in some way been altered.” Specifically, 23.6% firm-years were added (i.e., data were missing in the 11/2019 download, but filled in for the 02/2021 download), 1.6% were deleted, and 18.4% were modified. The data rewriting affects all years and not just those closest to the end of the sample period.”

Prof. Dr. Kornelia Fabisik’s conclusions, as presented at the Frankfurt School conference:

• The large differences in results that we document have implications for empirical test strategies using Refinitiv ESG data. Moving forward, researchers and investment professionals need to verify whether the original or rewritten ESG scores are needed to perform their tests.
• For example, if the practitioners are unaware of the changes, asset managers could erroneously be benchmarked against the rewritten data that were unavailable at the time of portfolio formation (look-ahead bias).
• She argues that the results reflect the incentive of the data provider to introduce a positive relationship between ESG scores and returns in the data, in order to demonstrate that their ESG scores are useful for data users developing ESG-related investing strategies.
• Given that ESG research and ESG-related investment strategies are likely to grow even furhter, this is an important caveat for adhering to the status-quo.

Finance Working Paper N° 708/2020, August 2021, Abstract:

The explosion in ESG research has led to a strong reliance on ESG rating providers. The article documents widespread changes to the historical ratings of a key rating provider, Refinitiv ESG (formerly ASSET4). Depending on whether the original or rewritten data are used, ESG-based classifications of firms into ESG quantiles and tests that relate ESG scores to returns change. While there is a positive link between ESG scores and firms’ stock market performance in the rewritten data, the authors fail to observe such a relationship in the initial data. The ESG data rewriting is an ongoing rather than a one-off phenomenon.