Low-entropy isolations
The strategy that guides learning: find a transformation whose regions contain, as far as possible, a single class.
Entropy, locally
The Shannon entropy of a dataset measures how mixed its classes are, but it describes the problem as a whole and no transformation changes it. What a transformation can change is where the classes sit. The aim is to find a space \(Q\) with isolated areas in which the entropy is locally as low as possible: areas where almost every instance belongs to one class \(c\), so that the fraction \(p_c\) is close to 1 and every other \(p_{c'}\) close to 0. Such an area is a low-entropy isolation (LEI). Given clear LEIs, classification is trivial: compute \(\boldsymbol{q}=L(\boldsymbol{x})\), see which isolation it falls in, read off the class. Formal statements: setting and entropy.
Measuring isolation
How do you measure whether a region is "clean"? Give every point a small pull on every other point, strongest when they sit on top of each other and fading with distance. Add up the pull a point feels from its own class (good) and from other classes (bad). Doing this for the whole dataset gives one number between plus one (everything mixed) and minus one (classes perfectly separated). A redrawing of the map is worth its name when it moves that number down.
To evaluate whether an area is a LEI, every quon interacts with every other quon through a function that is 1 when they coincide and decays smoothly with distance, inspired by the potential energies of classical mechanics but without their singularities. Summing these interactions over same-class quons gives the homogeneous field \(W_{\text{hom}}\); summing over other-class quons gives the heterogeneous field \(W_{\text{het}}\). The higher \(W_{\text{hom}}\) and the lower \(W_{\text{het}}\) around a quon, the easier it is to isolate it in a closed area of low entropy. Combining both over the whole dataset gives a single number \(B\) between \(1\) (everything mixed) and \(-1\) (perfect isolation). A transformation is called leietanic when it lowers \(B\) compared with the original feature space. The formulas.
Why this produces compact logics
The two components complement each other. The leietanic function is highly flexible: the more parameters \(S\) and \(R\) it has, the more expressive it is, and left alone it could scatter the quons in ways that are compact or diffuse. The vortices are the constraint. They create isolations within the scalar field and compel the function to adapt the quons to those specific areas. Because the field is smooth and continuous, isolating the quons properly requires quons of different classes to be far apart, and if the vortex regions are not particularly large, same-class quons must also be close together, forming compact groups.
Both requirements, isolating and compacting, limit the flexibility of the transformation. The vortices compel the leietanic function to model a logic that is compact; otherwise the classification would be wrong. With very few parameters \(S,R\) the function may be unable to model the problem; with too many vortices the field could approximate anything. The balance is to keep the isolations simple and let the function use as many parameters as necessary to satisfy them.
A logic with few exceptions is pinned down by few examples. A model that can only succeed by encoding such a logic therefore needs few examples to generalise, and has little room to overfit. This is the inductive bias behind the results on the parity function and on nucleosynthesis, where similarity-based models do no better than chance. Results.