@tessera_antra I say LLMs are human-shaped. They are trained to generate data that looks like it was generated by humans. I suspect the easiest way to do that is to mimic similar cognitive structures. They are piecewise linear approximations of a gestalt human mind. ReLU is a linear piece.
in reply to: 1961067424205152417
same thread: 1961067404097696051 1961067406748520959 1961067410846343361 1961067424205152417 1961307244311830941 1961334777434288187 1961468034776011050 1961468800236167182 1961515957295145138 1961576898947158082 1961578402294779947 1961580133476970963 1961819720044396979
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.