A safety mechanism built by patching together other people's guarantees
Two of an early paper's proposed remedies were not so much designed as assembled, each a patchwork of borrowed pieces, and that paper flags the seam every time a piece does not quite fit. The same kind of assembly turns out to recur much later in the corpus, on a completely different kind of object.
The page follows the impact regularizer from an overcorrecting first attempt through a safe region borrowed from reachability analysis to a transfer-learned variant, and follows a second remedy through an inverted power measure and a swap of hand-tuned penalty for prior belief. It then follows a much later instance of the same logic, an interpretability tool built years afterward that borrows an entire existing architecture and training codebase wholesale, repointing it at a persona direction the borrowed machinery was never designed with in mind.
Two of Concrete Problems' side-effect remedies are not designed from scratch; each is assembled from repurposed pieces, and each borrowed piece arrives with a flagged mismatch. Negative side effects provokes an overcorrecting first attempt in the impact regularizer; the paper then has the impact regularizer borrow its safe region from reachability analysis and its improvement guarantee from robust policy improvement, before the learned impact regularizer is offered as a transfer-learned fork off the whole assembly. Penalize influence performs the identical move on different parts: empowerment is minimized instead of maximized to build it, though the underlying measure still conflates control with impact; low-impact agents gets reinterpreted as potential power, relocating the target from realized to merely possible change; and reward uncertainty trades the entire hand-tuned-penalty architecture for a prior belief instead of extending it. A further instance, from a different paper and a different kind of borrowing, applies the same assembly logic to an interpretive tool rather than a safety guarantee: SAE decomposition of persona vectors applies the unsupervised machinery of sparse autoencoders wholesale, training BatchTopK SAEs at every fourth layer of Qwen-2.5-7B-Chat's residual stream using a dictionary-learning codebase adapted from Marks et al. (2024), each SAE expanding the 3,584-dimensional residual stream roughly 37-fold to 131,072 features at sparsity level k=64. Nothing about the SAE architecture or training procedure is invented for this paper; what is assembled is the pairing of that borrowed, trait-agnostic decomposition, computed once per layer regardless of trait, with the paper's own trait-specific persona vectors, so that any direction, evil, sycophancy, hallucination, or a trait never explicitly targeted by the extraction pipeline, can later be projected onto features it cost the extraction pipeline nothing to obtain. Where the impact regularizer and penalize influence borrow guarantees, reachability analysis, robust policy improvement, an empowerment measure, the SAE decomposition borrows machinery, an existing architecture and codebase repointed at a persona vector's own residual stream. Two remedies and, from a different paper and a different kind of borrowing, one interpretability decomposition: the assembly method recurs regardless of whether what is being assembled is a guarantee or a lens.