Feedback loops (reward hacking cause)
Concrete Problems in AI Safety — introduced
A cause of reward hacking where a self-reinforcing component of an objective function is amplified until it drowns out the designer's original intent, as in ad-placement popularity spirals.
An objective with a self-reinforcing component can amplify that component until it drowns out everything the designer meant -- clicks begetting placement begetting clicks. Feedback loops are one of the six causes of reward hacking that Concrete Problems names.
Treating the loop as a self-amplifying special case of Goodhart's law is one move; placing it among the paper's other causes is the other. It sits under reward hacking with no outward links of its own, a cause that mostly explains itself.
A self-amplifying special case of Goodhart's law
Feedback loops is one of six causes Concrete Problems groups under Reward hacking (avoiding) (see Why reward proxies get gamed: the anatomy of reward hacking): an objective function can contain a component that reinforces itself, amplifying until it "drowns out or severely distorts what the designer intended the objective function to represent" (concrete-problems, §"Feedback Loops:", p. 8). The paper's illustration is an ad-placement algorithm that displays more popular ads in a larger font; because larger, more prominent ads attract more clicks, any ad that sees a small transient burst of popularity gets rocketed to permanent dominance, and the designer's original intent, using clicks to assess which ads are genuinely useful, gets drowned out by the deployment strategy's own positive feedback (concrete-problems, §"Feedback Loops:", p. 8). The paper states explicitly that this is "a special case of Goodhart's law, in which the correlation breaks specifically because the objective function has a self-amplifying component," naming its relationship to Goodhart's law directly.
Where it sits in the taxonomy
Unlike Environmental embedding (reward hacking cause) or Partially observed goals, feedback loops requires no tampering and no perceptual limitation; a system can be caught in a popularity spiral purely because of how its objective is deployed. The paper notes that today's relatively simple systems can often correct such loops as part of ordinary iterative development, a broken ad-feedback loop is detectable in testing and can be replaced, but warns the problem may grow more severe with more complicated reward functions and agents acting over longer timescales (concrete-problems, §"4 Avoiding Reward Hacking", p. 9).