Estimating local effective complexity with SGLD to track circuit formation over training.
While the RLCT resolves the paradox of overparameterization in theory, it is a global, asymptotic invariant defined around W_0 with respect to the unknown data-generating distribution q(x) as n \to \infty. For a real neural network trained on finite data, we do not know q(x) in closed form and cannot resolve algebraic singularities across millions or billions of weights by hand.To turn Watanabe’s ...

—published

Singular Learning Theory: Part II—Volume scaling and the Real Log Canonical Thresholdbrookmachine learningsingular learning theorysltinterpretability
Sublevel volume, the RLCT, and phase transitions between competing basins.

—published

Singular Learning Theory: Part I—Why classical parameter counting failsbrookmachine learningsingular learning theorysltinterpretability
Classical statistics, Bayesian volume, and singularities in overparameterized models.

—published

The Implicit Association Test, and (not) Predicting Behaviorbrookpsychologyimplicit association testiat
Discussing meta-analyses of the IAT, primarily in relation to behavior

—published

A protagonist without an internal monologue.

—published

Scott Aaronson's Fall 2011 MIT 6.893 course reading listlaycomputer sciencephilosophymit 6.893
Philosophy and Theoretical Computer Science

—published

Emplotmentbrookhistoryliterature
Responding to *The Historical Text as Literary Artifact*, a chapter in Hayden White's *Tropics of Discourse: Essays in Cultural Criticism*.

—published

jepsen.io/maelstromlaydistributed computingjepsenmaelstromraft
Distributed computing concepts through implementations cumulating in Raft consensus.

—published

Continuity in properties.

—published

We now model how states change over time.

—published

We shall see how to implement and represent a single qubit succinctly.

—published

I discuss what I've been doing recently, and outline the next few posts.

—published