seismic-descent

Seismic Descent

An optimization algorithm based on gradient descent over a dynamic landscape perturbed by spatially correlated noise.

Seismic Descent

Read this documentation in Spanish

The Concept

Instead of taking random jumps to escape local minima (as seen in Simulated Annealing), we trigger an earthquake on the ground. The particle simply does the only thing it knows how to do: roll downhill. But because the ground tremors in a coherent, multi-scale fashion, local minima temporarily morph into slopes, allowing the particle to escape naturally and effortlessly.

f_total(x, t) = f_original(x) + A(t) * noise(x, t)

The key advantage over Simulated Annealing (SA): The noise is spatially correlated — two nearby points share similar perturbations. The particle smoothly slides boundaries to another valley; it does not blindly teleport. Superimposed noise octaves provide both broad inter-valley exploration and ultra-fine refinement locally, all packed into a single mathematical mechanism.

N-Dimensional Noise: Random Fourier Features

In 2D, strict Perlin noise is highly effective. To scale efficiently across N-Dimensions without grid-bound computational explosion, we approximate a Gaussian Random Field (GRF) via Random Fourier Features (Rahimi & Recht, 2007):

noise(x) ≈ sqrt(2/R) * A * Σ_r cos(ω_r · x + t*drift_r + φ_r)

Where ω_r ~ N(0, 1/l²·I) are vectors in R^D. Being N-Dimensional vectors, they force geometric spatial correlation and feature overlap in any high-dimensional search space instantly.

Milestones & Recent Optimizations (v7 - v17)

The repository condenses 24 hours of intensive empirical hackathon progress where the algorithm transcended severe bottlenecks:

Key Property: Seismic Ergodicity

A fundamental discovery in the development of the algorithm (consolidated in v19) is that the exploration driven by correlated noise must be ergodic.

Instead of blindly shaking the particle with high-frequency “white noise”, Seismic Descent generates complete, coherent topological landscapes of low and high frequencies (via RFF octaves), and smoothly morphs (interpolates) from one random landscape to the next over time.

This continuous mutation mathematically guarantees that a particle, guided purely by the gradient of this “mutating ground”, will eventually explore and visit the entirety of the search space without getting trapped in infinite loops or plateaus. Ergodicity is what allows the swarm to flow through the terrain like a liquid, guaranteeing an escape from even the deepest local minima.

Empirical Thermodynamic Properties (Laplacian Ergodicity)

Laplacian Ergodic Histogram Notice how the green ergodicity histogram perfectly draws a sharp Laplacian distribution ($e^{-|x|}$) around each local minimum. The peak height directly correlates with the minimum’s depth, while the width correlates with the steepness of the basin walls.

Observations from the 1D visualizer reveal a profound statistical mechanics property: as t -> ∞, the particle’s spatial probability density function (the ergodic heatmap) converges into sharp Laplacian peaks centered at local minima.

  1. Boltzmann-Gibbs Emulation: The depth of a minimum determines the exact statistical amplitude of the peak. This means Seismic Descent naturally performs robust Monte Carlo sampling equivalent to a thermodynamic system.
  2. Heavy-Tailed Escapes: Unlike traditional Gaussian (Brownian) noise used in Langevin dynamics or SGD ($e^{-x^2}$), the Laplacian signature ($e^{- x }$) empirically proves that the seismic spatial field induces heavy-tailed jumps. The probability of the particle massively leaping out of a basin’s boundaries is orders of magnitude higher than in standard random walks. This mathematically explains the algorithm’s exceptional capability to escape sub-optimal valleys where standard optimizers get permanently trapped.

Interactive Visualizers

To truly understand how Seismic Descent works, you can explore the algorithm interactively in your browser without any installation:

1D Visualizer Ergodicity Snapshot of the 1D Visualizer optimizing the highly non-linear Rastrigin function. The green histogram at the bottom (Ergodic Heatmap) perfectly maps the continuous topological exploration of the particle across all local minima basins, tangibly proving the algorithm avoids infinite entrapment.

Installation

pip install numpy noise matplotlib cma

Usage

# Original 2D Benchmark (Perlin mapping)
python perlin_opt.py

# N-Dimensional benchmark with RFF (Base ND variant)
python perlin_opt_nd_grf.py

# CLI Benchmark testing (Seismic Swarm vs SA vs CMA-ES)
python benchmark_budgets.py --preset low
python benchmark_budgets.py --preset med
python benchmark_budgets.py --preset high

PyTorch Integration

The Seismic Descent algorithm is now available as a standard PyTorch optimizer. This allows for training neural networks with spatially correlated “earthquake” tremors to escape local minima.

from seismic_optimizer import SeismicOptimizer

model = MyModel()
optimizer = SeismicOptimizer(
    model.parameters(), 
    lr=0.01, 
    noise_amplitude=0.5, 
    n_cycles=10
)

See benchmark_mnist.py for a complete example and docs/pytorch_optimizer.md for technical details.

Latest Benchmark (MNIST - 20 Epochs)

Optimizer Accuracy Margin
SGD 98.28% Base
Adaptive Floored Seismic 97.90% ✅ Beats Adam
Adam 97.79% -

Structure

docs/
  findings_v1.md           — 2D findings and amplitude A(t) schedule genesis
  findings_v4_rff.md       — Random Fourier Features, Rastrigin ND results
  findings_v7_rastrigin_analytic.md  — O(1) Analytic Gradients benchmarks
  findings_v8_no_abs.md              — Gold discovery of negative amplitude polarity 
  findings_v11_adam.md               — Explaining why Adam Optimizer chokes earthquakes
  findings_v12_swarm.md              — Seismic Swarm: Parallel Analytic RFF Matrix
  findings_v14_cycles.md             — Strict cyclic parametrization
  findings_v15_reactive.md           — Bang-Bang Control (Stagnation triggers) Diagnosis
  findings_v16_momentum.md           — Heavy-Ball Momentum (Sloshing & Filter failures)
  findings_v17_temporal_octaves.md   — Temporal Fractal Earthquakes (Fourier super-positioning)
  findings_budgets_scale.md          — Massive Budget scale up against Simulated Annealing & CMA-ES
  summary_of_experiments.md          — Total deep-dive history of the initial 24h repository life
perlin_opt.py              — Base 2D prototype
perlin_opt_nd_grf.py       — N-Dimensional RFF integration
perlin_opt_nd_grf_analytic*.py — Incremental historic repository variants (v7 to v17)
benchmark_budgets.py       — Highly-parametrized CLI algorithm vs algorithm testing suite

Future Scope