Results overview
The logic replicant obtained the highest test accuracy in all four problems. This page gives the full comparison table and the overall pattern; each experiment has its own page.
The logic replicant gave the most accurate answers on all four problems. Here is the short story.
Replicant versus the best other model
The short story
On the two problems governed by a strict rule, parity and nucleosynthesis, the other methods are close to guessing. The random forest actually scores worse than a coin toss on parity, because it learned the wrong lesson: that neighbours share an answer. The logic replicant, which looks for the rule, gets almost everything right.
On the two problems where look-alikes really do belong together, the mice and the digits, the other methods are already good. The replicant still comes first, and it removes roughly a third to a half of the remaining mistakes. So it is not a specialist for puzzles: it is a good general classifier that also handles the puzzles.
| Problem | Logic replicant | Best rival | Rival |
|---|---|---|---|
| Parity | 98.6% | 57.3% | small neural network |
| Down syndrome in mice | 99.3% | 98.9% | small neural network |
| Nucleosynthesis | 72.9% | 10.2% | small neural network |
| Handwritten digits | 98.2% | 97.5% | small neural network |
Share of hidden test examples classified correctly, averaged over ten shuffles. Each problem has its own page with the picture of what the replicant learned.
Table 1 of the paper
Configuration for the replicant lists \(S,R\) of the leietanic function (14), \(D=\dim(Q)\) and the total number of vortices \(|V|\). For the MLP it gives the hidden-layer width, for the SOM the grid, and for RF and XGBoost the parameters altered from the library defaults.
| Problem | Model | Accur. (train) | Accur. (test) | Precis. (test) | Recall (test) | Configuration |
|---|---|---|---|---|---|---|
| Parity function | Replicant | 99.80% | 98.63% | 98.67% | 98.60% | \(S=0, R=0, D=1, |V|=8\) |
| RF | 100.00% | 0.20% | 0.21% | 0.18% | 100 estim., all features | |
| MLP | 95.66% | 57.25% | 42.61% | 59.72% | 20 neurons, 1 hidden layer | |
| SOM | 64.68% | 24.71% | 24.71% | 18.39% | 256 neurons (16×16) | |
| XGBoost | 64.00% | 5.10% | 4.74% | 5.27% | 100 estim., max depth 6 | |
| Mouse model of Down syndrome | Replicant | 99.99% | 99.34% | 99.38% | 99.40% | \(S=0, R=0, D=5, |V|=40\) |
| RF | 100.00% | 94.31% | 95.54% | 94.56% | 100 estim., all features | |
| MLP | 99.70% | 98.89% | 98.89% | 98.90% | 7 neurons, 1 hidden layer | |
| SOM | 92.30% | 87.04% | 87.79% | 87.07% | 3600 neurons (60×60) | |
| XGBoost | 99.98% | 93.29% | 93.50% | 93.50% | 100 estim., max depth 7 | |
| Nucleosynthesis | Replicant | 99.99% | 72.87% | 73.56% | 72.72% | \(S=0, R=0, D=1, |V|=17\) |
| RF | 99.17% | 8.33% | 12.50% | 7.25% | 100 estim., all features | |
| MLP | 100.00% | 10.20% | 10.48% | 11.05% | 5 neurons, 1 hidden layer | |
| SOM | 100.00% | 8.33% | 9.65% | 8.75% | 121 neurons (11×11) | |
| XGBoost | 17.08% | 6.67% | 8.61% | 6.46% | 100 estim., max depth 7 | |
| Recognition of handwritten digits | Replicant | 99.99% | 98.21% | 98.21% | 98.21% | \(S=4, R=4, D=8, |V|=20\) |
| RF | 100.00% | 89.22% | 93.38% | 89.47% | 100 estim., all features | |
| MLP | 100.00% | 97.49% | 97.49% | 97.55% | 50 neurons, 1 hidden layer | |
| SOM | 93.83% | 93.23% | 92.28% | 93.36% | 10 000 neurons (100×100) | |
| XGBoost | 99.64% | 93.70% | 93.71% | 93.89% | 100 estim., max depth 11 |
The pattern
Accuracy. The replicant's advantage is largest where the logic is compact and free of exceptions (parity, nucleosynthesis): the other models are close to random guessing, while the replicant generalises. The discrete inputs make those problems DFSMs, and a DFSM-like model with few parameters leaves little room for overfitting. On problems with continuous inputs (mice, digits), similarity-based baselines already do well and the replicant still wins, reducing the remaining error by 40.54% and 28.69% relative to the MLP. Continuous values do not prevent learning; what matters is that the distribution of input values in \(Q\) aligns with the distribution of output classes.
Interpretability. In every problem the two-dimensional \(Q\)-space plot shows both the logic found and the quality of the learning: regularity and symmetry for parity, an irregular aggregation with subgroups for the mice, a large logic with nested sub-logics for the digits. Reading a Q-space plot.
Summary. The leietanic function combined with vortices is a DFSM that replicates the inherent logic of several different problems in practice. Some problems cannot be generalised by the other approaches at all, whereas the replicant understands and predicts them properly from a limited number of instances; nucleosynthesis, which responds only to deterministic natural laws, is a remarkable case of independent scientific interest.