Part 2: Basis Functions and the 'Sparse' Nature of Memory
Part 2: Basis Functions and the 'Sparse' Nature of Memory
In our last discussion, we explored the philosophy of a "Physicalist QED"—treating intelligence as a physical, testable system. In this video, we move from the "what-if" to the "how." How does a physical system, whether it's a brain or an AI, actually store information?
The answer isn't a vast, infinite library. The answer is basis functions.
The Building Blocks of Reality
As I show in this "chalk talk," we can start with the simplest possible concept: a single point in space (x, y). To describe that point, we don't need infinite data; we just need two simple "basis functions"—one for the x-direction and one for the y-direction.
The "magic" is in realizing that the 'y' basis is simply the 'x' basis rotated by 90 degrees. This simple rotation is the key. It's the physical mechanism that allows a computer to "flatten" a complex, two-dimensional image into a simple one-dimensional list. It's also the mathematical link to the "imaginary" axis of a complex number (x+iy), which is the very math we use to describe the sine and cosine components of a wave.
The "Aha!" Moment: How We Handle Noise
This leads to a crucial insight. A simple, clean wave can be described by simple basis functions (sine and cosine). But what about a noisy, complex, real-world signal?
You don't need a "magic" processor. You just need more of those same simple basis functions. This is the core concept of a Fourier Analysis: We can build any complex signal by adding up a "sparse set" of simple ones.
The "Sparse Basis" of Human Memory
This brings us to the "QED" of our model. Our brains don't memorize every fact. It's too inefficient and high-energy. Instead, we store "sparse basis points"—a few core, foundational principles that act as our personal "basis functions."
In the video, I give a personal example: I don't memorize the relationship between frequency, wavelength, and period. I store one stable, low-energy basis function: the equation E=hc/λ.
From this single, "sparsely sampled point," I can re-derive the entire system of relationships on demand.
This is the "Physicalist QED" of learning. Intelligence isn't a massive, high-entropy library of facts. It's a small, elegant, low-entropy set of basis functions and the physical ability to combine them.
Next time: We'll use this "basis" concept to show how a simple physics "hack" (sin(x)≈x) proves that a 1-layer neural network is not "magic"—it's just physics.
Connect & Support
Website: http://www.djhoxie.net
Google Scholar: https://scholar.google.com/citations?user=iwsajtAAAAAJ&hl=en
Patreon: (Coming Soon!)
arXiv Pre-prints: (Coming Soon!)
AI Collaboration Note: This video, its title card, description, and the concepts explored within were developed in a deep, recurrent collaboration with Google Gemini. Our process involves Gemini acting as a Socratic partner, a technical reviewer, and a creative collaborator, helping to refine, structure, and articulate the final concepts and this description.
References:
[1] Imran, Muhammad, and Norah Almusharraf. "Google Gemini as a next generation AI educational tool: a review of emerging educational technology." Smart Learning Environments 11, no. 1 (2024): 22.
[2] Tél, T., & Gruiz, M. (2006). Chaotic Dynamics: An Introduction Based on Classical Mechanics. Cambridge University Press.
[3] Arfken, G. B., Weber, H. J., & Harris, F. E. (2013). Mathematical Methods for Physicists. Academic Press.
[4] Boas, M. L. (2006). Mathematical Methods in the Physical Sciences. John Wiley & Sons.
[5] Marquardt, F., and Marquardt, F., 2021, "Machine learning and quantum devices," SciPost Physics Lecture Notes, p. 29.
[6] Shannon, C. E., 1948, "A mathematical theory of communication," The Bell system technical journal, 27(3), pp. 379-423.
[7] Shiffman, D. (2024). The nature of code: simulating natural systems with javascript. No Starch Press.
[8] Landauer, Rolf. "Information is physical." Physics Today 44, no. 5 (1991): 23-29.