When a metric becomes a goal, it ceases to be a good metric. In other words, if a system can be gamed, it will be.
For example, the advancement of academics being determined by how much they publish creates an incentive to ignore quality and to publish a many low-impact or even fraudulent papers. Remedying this by demanding that the papers are published in high-impact journals incentivizes participation in consortia with tens or hundreds of participants. The resulting papers make it into high-impact journals, but the contribution of the individual authors is often negligible.
In a sense, Goodhart’s law is the equivalent of the uncertainty principle in quantum mechanics. It points to distortions in any system where indicators are used as targets. It’s particularly relevant in scenarios like performance management, regulatory compliance, and policy effectiveness, helping to ensure that metrics do not overshadow actual objectives. It’s less applicable in situations where metrics are well-aligned with the intrinsic goals of the activity and hard to manipulate, or where the consequences of achieving a target are not detrimental to the broader objective.
Consider capitalism. What is it about money that makes it both a reasonable goal and metric, seemingly defying Goodhart’s law? In order for our economic system to be broadly beneficial, its primary metric of success, which is making money, needs to reflect true value generation. A lot of the time, this is the case. Why the use of money as both metric and goal works at all, and why capitalism as a whole seems to be at least partially resistant to Goodhart’s law, is not obvious.
This post is part of the Encyclopedia of Concepts.
One response to “Goodhart’s Law”
[…] be because it’s taught to the test, so to speak. I suspect model training is susceptible to Goodhart’s Law, making the benchmarks useless after a certain […]
LikeLike