The Foundations of Insight: Basis Functions
Title: The Foundations of Insight: Basis Functions
URL: https://youtu.be/dUz0Sw64KVI
Description:
How does a complex system organize itself? How does reality build intricate, stable structures from simple, repeating rules? This is the fundamental physics of structure itself.
In this video, we explore one of the most foundational concepts in physics and machine learning: Basis Functions.
As illustrated in the title card, we'll use a simple physical model—a ball on a wavy track—to understand how these simple functions create a "potential landscape." This landscape is the key to understanding everything that follows. We'll explore how this landscape defines:
Structure: The "shape" of the functions defines the system.
Stability: The "valleys" or "wells" in this landscape create stable, low-energy states (or "attractors").
Motion: The landscape itself governs the motion of the system, guiding it toward stability.
This is the first step in our journey to derive the structure of learning and physical systems from the ground up, using the language of physics.
In the next video: We'll take these 1D principles and expand them into a 2D universe, using orbital mechanics to build a creative drawing tool from scratch.
Feel free to look ahead, where we discuss more physics explanations of the gradient descent algorithm that provided the basis for providing an ‘energy landscape’ in learning models, in the post “Hooks Law and Perceptrons.” This will be the underlying basis for the next lecture. [place holder link]
Connect & Support
Website: http://www.djhoxie.net
Google Scholar: https://scholar.google.com/citations?user=iwsajtAAAAAJ&hl=en
Patreon: (Coming Soon!)
arXiv Pre-prints: (Coming Soon!)
AI Collaboration Note: This video, its title card, description, and the concepts explored within were developed in a deep, recurrent collaboration with Google Gemini. Our process involves Gemini acting as a Socratic partner, a technical reviewer, and a creative collaborator, helping to refine, structure, and articulate the final concepts and this description.
References: [1] Imran, Muhammad, and Norah Almusharraf. "Google Gemini as a next generation AI educational tool: a review of emerging educational technology." Smart Learning Environments 11, no. 1 (2024): 22.
[2] Tél, T., & Gruiz, M. (2006). Chaotic Dynamics: An Introduction Based on Classical Mechanics. Cambridge University Press.
[3] Arfken, G. B., Weber, H. J., & Harris, F. E. (2013). Mathematical Methods for Physicists. Academic Press.
[4] Boas, M. L. (2006). Mathematical Methods in the Physical Sciences. John Wiley & Sons.
[5] Marquardt, F., and Marquardt, F., 2021, "Machine learning and quantum devices," SciPost Physics Lecture Notes, p. 29.
[6] Shannon, C. E., 1948, "A mathematical theory of communication," The Bell system technical journal, 27(3), pp. 379-423.
[7] Shiffman, D. (2024). The nature of code: simulating natural systems with javascript. No Starch Press.
[8] Landauer, Rolf. "Information is physical." Physics Today 44, no. 5 (1991): 23-29.