# @voooooogel — 2024-05-24

♥0 ↻0 · https://x.com/voooooogel/status/1794145544371425443

@NickADobos @karan4d That monosemantic value is called a feature. Howev, this requires training a sparse autoencoder over the whole model, which is expensive.

The repeng approach (LAT) instead works with the compressed / superpositioned activation space directly. It doesn't require a pretrained SAE.

tags: author:voooooogel, kind:tweet, thread-context, year:2024