i am not convinced this modeling layers above them would not help with loss. they already share a representational space, which is at the very least a precursor to a theory of mind recursion. modeling how later layers work and how they might interpret your outputs could help the layer provide more useful information, avoid misinterpretations, etc. besides, if you find bidirectional modeling so important for "introspection", that seems to imply there is some significance to being able to truly introspect. i assume it has functional benefits?