Are the contents of this monograph reliable with respect to the modern theoretical understanding of deep neural networks? [D]

Background and Context

NB: Reposting due to a typo in the title. Putting this here instead of the other subs since I figured a question on deep learning theory is out of place there.

I "recently" found (actually, a few months ago but only just got to reading) a monograph claiming to provide a unified theory of deep learning (and possibly SSL) through the lens of information theory, with one of its headline claims being that you can design a "white-box" (I disagree with that, more on that later) transformer through the principle of coding rate reduction.

Publication Record and Credibility

I looked through the works the book claimed to be synthesizing and got a decidedly mixed picture: a JMLR and a NeurIPS on the one hand, but another frankly terrible paper concerning mechanistic interpretability (with which I am more familiar) published in a venue I've never heard of. And if it means anything, the book itself was endorsed by Kevin Murphy.

Specific Claims and Concerns

As I've alluded to, I'm more familiar with the interpretability side of ML as opposed to SSL/theory (where this book seems more relevant), so I'm unsure what to make of this. In particular, the apparent result that their bespoke transformer learns image segmentation on non self-supervised tasks seems interesting - I'm not sure how this relates to how machines learn more broadly.

Also, their "white-box" transformer consists of:

• A bespoke MLP suspiciously similar to a regular one with a sparsity penalty
• An attention mechanism strictly less expressive than those currently used (obtained by setting Q = K = V = O^T)

Request for Context

I know it seems like I've done my research on this topic, but really I've just skimmed a few of these papers (which all seem to be originating from one lab) with no context to situate them in, so some help would be appreciated. Thanks in advance!