The difficulties of assigning ESG ratings

Utah’s governor, attorney general and other state and federal representatives recently wrote a letter admonishing S&P Global Ratings for including environmental, social, and corporate governance (ESG) indicators in its credit rating of the state. The letter from Utah’s political leaders says:

S&P acknowledges that “having a social mission and strong ESG characteristics does not necessarily correlate with strong creditworthiness and vice versa.” S&P’s ESG credit indicators politicize what should be a purely financial decision. This politicization has manifested itself in the capital markets where, for example, banks are pressured to cut off capital to the oil, gas, coal, and firearms industries. ESG is a political rating and should be characterized as such. This is clear when recognizing the two layers of indeterminacy that make ESG an exercise in servitude: 1) which “ESG factors” are chosen, and 2) the “correct” answer to any given factor. Whoever answers those questions has all the power in achieving a desired outcome.

These are not technocratic questions; they are normative questions. No financial firm should substitute its political judgments for objective financial analysis, especially on matters that are unrelated to the underlying businesses, assets, and cash flows it evaluates. This is especially true of a properly regulated independent entity like S&P that is charged with providing objective clarity and insight. The use of ESG-related quantitative metrics and analytical frameworks confounds the distinction between subjective normative judgments and objective financial assessments. It is therefore unconscionable for S&P to weigh in on indeterminate and normative questions. Moreover, the answers to the normative factors can and do change depending on circumstances. We believe this entire exercise in identifying, evaluating, and publishing ESG factors is highly intrusive and leads to manipulation, coercion, and misleading outcomes.

As the term environmental, social, and corporate governance suggests, ESG ratings attempt to assess the societal impact of a company across three broad dimensions: environmental impact, social impact, and corporate governance. These ratings and ESG-centered investing have gained significant traction among many major asset managers. BlackRock, Vanguard, and State Street (which, combined, are the largest shareholders in 88% of S&P listed companies, according to a 2017 study) all issued ESG commitments and guidance for their portfolio companies in 2022.

As evidenced by S&P’s ESG credit indicator for states, the reach of ESG is not limited to private companies and investors. State public pension systems in New York, California, Colorado, Maryland, and Maine have begun incorporating ESG principles in investment decisions. However, as the National Association of State Retirement Administrators (NASRA) notes, “ESG investing has been challenged by some who believe that this approach is contradictory to fiduciary duty.”

Pension trustees are required by state laws to exercise their fiduciary responsibilities to maximize public pension systems’ investment returns at acceptable levels of risk for retirees and future beneficiaries.

In response to fiduciary concerns, the environmental, social, and corporate governance model is commonly justified under the auspices that investors do well when companies do good. The evidence for this claim is not clear. A comprehensive analysis by Vanguard found:

This article sets out to empirically investigate the performance characteristics of investable ESG equity funds to assess whether support for a particular direction in performance impact can be found…After controlling for style factor exposures, the majority of funds in any of the tested ESG categories does not produce statistically significant positive or negative gross alpha. An industry-based performance contribution analysis reveals that systematic differences in allocations relative to the broad market exist. However, their median contribution to performance is close to zero over time. Overall, return and risk differences of ESG funds can be significant but appear to be mainly driven by fund-specific criteria rather than by a homogeneous ESG factor.

This suggests it would be prudent for both private and public investors to assess ESG scores with skepticism. Even beyond very real concerns about the politicization of investment decisions, there needs to be an assessment of the validity of ESG rating metrics. Is there any precision in ESG ratings? In other words, do they measure what they’re trying to measure?

One of the leading ESG rating agencies, Morningstar’s Sustainalytics, provides publicly-accessible ESG Risk Ratings for many publicly traded companies. Sustainalytics’ ESG Risk Rating is a measure of “the degree to which a company’s economic value (enterprise value) is at risk [given the] company’s unmanaged ESG risk.” Sustainalytics assigns a rating by determining the total exposure risk of a company. Sustainalytics says the risk rating given to a company is a combination of entirely unmanageable risk and manageable risk that is not being addressed.

Size

Looking at the available ESG scores from Morningstar’s Sustainalytics ESG Risk Ratings, we see a trend where large companies rate better than small companies. On a scale in which a lower score indicates a better ESG rating, the median value of S&P 500 (large capitalization index) companies is 21.1 and Russell 2000 (small capitalization index) companies is 29.3. The difference in ratings is not just for the overall capitalization index but results in similar scoring across every single sector (communication services, consumer staples, energy, health care, etc.).

Large companies may have some structural advantages that make their businesses more ESG friendly, but it is also clear that they have structural advantages that enable them to boost ESG scores. A report for Legg Mason, which found a similar preference for company size across three rating agencies, notes:

“Many companies have started documenting their policies in publicly available sustainability disclosures; however, producing such disclosures is resource-intensive and financially burdensome. As a result, larger companies rate better as they generally have increased transparency and resources to dedicate to such initiatives.”

In some respects, the interplay between market capitalization and ESG scores is analogous to how businesses of different scales respond to government regulations. While reporting requirements may be burdensome for the large players, their bigger accounting, legal, and human resources departments are better able to take on these burdens than their smaller competitors.

Sector

In addition to size, there is also a clear stratification by sector in Morningstar's Sustainalytics. To some extent, this is logical. We would not expect a communication services company to have the same environmental rating as an oil company. Still, businesses are not isolated entities operating independently of other sectors.

For example, looking at the materials and technology sectors, the median materials sector ESG score is 30.9 while information technology has a median score of 22.4. If a technology company sources critical inputs like cobalt and lithium for its products from the mining sector, where do the ESG boundary lines fall, and are these ratings objectively reflective of a company’s true ESG impact?

Coherence

Beyond sector and size, there are specific results in Sustainalytics’ ESG risk ratings that call into question the logical coherence of the ratings. Notable is the comparison of leading defense contractors with a pasture-raised egg and butter company, Vital Farms. Even putting aside concerns about the underbelly of defense contracting, all four defense contractors are heavily taxpayer-subsidized. Northrop Grumman, Raytheon, Lockheed Martin, and Boeing, with ESG risk ratings ranging from 28.4 to 35, all rate better than Vital Farms’ 42.8. Ostensibly, the food company checks a lot of ESG boxes. It partners with small family farms and the company website states:

“Our purpose is rooted in a commitment to Conscious Capitalism, which prioritizes the long-term benefits to each of our stakeholders (farmers and suppliers, customers and consumers, communities and the environment, crew members and stockholders)...We are mission-minded people working together to bring ethically produced food from family farms to families’ tables.”

There may be a lot more to this picture, but seeing results like defense companies ranking better than Vital Farms raises several questions about ESG ratings. No single score applied to a complex world is flawless. Still, with ESG ratings increasingly being considered by corporations, governments, and institutional investors, the potential problems with these metrics shouldn’t be ignored.

Broadly, environmental, social, and corporate governance policies should be examined under a precision and accuracy framework. Do the metrics measure what they purport to measure—and are the priorities baked into ESG ratings the proper priorities in the first place?