Machine Learning Street Talk (MLST)

Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves. GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg) • Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search. • The *architecture* of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard. • Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks. • Are these systems actually thinking outside the box, or are they parasitic on their starting conditions? When LLMs run autonomously, "nothing interesting happens." Robert pushes back with the stepping-stone argument — evolution doesn't need to extrapolate, just recombine usefully. • The AI Scientist question: can automated research pipelines produce real science, or just workshop-level slop that passes surface-level review? Robert is honest that the current version is more co-pilot than autonomous researcher. • Where this lands in 5-20 years — Robert's prediction that scientific research will be fundamentally transformed, and Tim's thought experiment about alien mathematical artifacts that no human could have conceived. Robert Lange: https://roberttlange.com/ --- TIMESTAMPS: 00:00:00 Introduction: Robert Lange, Sakana AI and Shinka Evolve 00:04:15 AlphaEvolve's Blind Spot: Co-Evolving Problems with Solutions 00:09:05 Unknown Unknowns, POET, and Auto-Curricula for AI Science 00:14:20 MAP-Elites and Quality-Diversity: Shinka's Evolutionary Architecture 00:28:00 UCB Bandits, Mutations and the Vibe Research Vision 00:40:00 Scaling Shinka: Meta-Evolution, Democratisation and the Three-Axis Model 00:47:10 Applications, ARC-AGI and the Future of Work 00:57:00 The AI Scientist and the Human Co-Pilot: Who Steers the Search? 01:06:00 AI Scientist v2, Slop Critique and the Future of Scientific Publishing --- REFERENCES: paper: [00:03:30] ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution https://arxiv.org/abs/2509.19349 [00:04:15] AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery https://arxiv.org/abs/2506.13131 [00:06:30] Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents https://arxiv.org/abs/2505.22954 [00:09:05] Paired Open-Ended Trailblazer (POET) https://arxiv.org/abs/1901.01753 [00:10:00] PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem https://arxiv.org/abs/1112.5309 [00:10:40] Automated Capability Discovery via Foundation Model Self-Exploration https://arxiv.org/abs/2502.07577 [00:15:30] Illuminating Search Spaces by Mapping Elites (MAP-Elites) https://arxiv.org/abs/1504.04909 [00:47:10] Automated Design of Agentic Systems (ADAS) https://arxiv.org/abs/2408.08435 PDF : https://app.rescript.info/api/sessions/b8a9dcf60623657c/pdf/download Transcript: https://app.rescript.info/public/share/SDOD_3oXOcli3zTqcAtR8eibT5U3gam84oo4KRtI-Vk

Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be turning software engineering into a slot machine and how to maintain true technical intuition in the age of large language models. GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg) Jeremy Howard is a renowned data scientist, researcher, entrepreneur, and educator. As the co-founder of fast.ai, former President of Kaggle, and the creator of ULMFiT, Jeremy has spent decades democratizing deep learning. His pioneering work laid the foundation for modern transfer learning and the pre-training and fine-tuning paradigm that powers today's language models. Key Topics and Main Insights Discussed: - The Origins of ULMFiT and Fine-Tuning - The Vibe Coding Illusion and Software Engineering - Cognitive Science, Friction, and Learning - The Future of Developers RESCRIPT: https://app.rescript.info/public/share/BhX5zP3b0m63srLOQDKBTFTooSzEMh_ARwmDG_h_izk Jeremy Howard: https://x.com/jeremyphoward https://www.answer.ai/ --- TIMESTAMPS (fixed): 00:00:00 Introduction & GTC Sponsor 00:04:30 ULMFiT & The Birth of Fine-Tuning 00:12:00 Intuition & The Mechanics of Learning 00:18:30 Abstraction Hierarchies & AI Creativity 00:23:00 Claude Code & The Interpolation Illusion 00:27:30 Coding vs. Software Engineering 00:30:00 Cosplaying Intelligence: Dennett vs. Searle 00:36:30 Automation, Radiology & Desirable Difficulty 00:42:30 Organizational Knowledge & The Slope 00:48:00 Vibe Coding as a Slot Machine 00:54:00 The Erosion of Control in Software 01:01:00 Interactive Programming & REPL Environments 01:05:00 The Notebook Debate & Exploratory Science 01:17:30 AI Existential Risk & Power Centralization 01:24:20 Current Risks, Privacy & Enfeeblement --- REFERENCES: Blog Post: [00:03:00] fast.ai Blog: Self-Supervised Learning https://www.fast.ai/posts/2020-01-13-self_supervised.html [00:13:30] DeepMind Blog: Gemini Deep Think https://deepmind.google/blog/accelerating-mathematical-and-scientific-discovery-with-gemini-deep-think/ [00:19:30] Modular Blog: Claude C Compiler analysis https://www.modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software [00:19:45] Anthropic Engineering Blog: Building C Compiler https://www.anthropic.com/engineering/building-c-compiler [00:48:00] Cursor Blog: Scaling Agents https://cursor.com/blog/scaling-agents [01:05:15] fast.ai Blog: NB Dev Merged Driver https://www.fast.ai/posts/2022-08-25-jupyter-git.html [01:17:30] Jeremy Howard: Response to AI Risk Letter https://www.normaltech.ai/p/is-avoiding-extinction-from-ai-really Book: [00:08:30] M. Chirimuuta: The Brain Abstracted https://mitpress.mit.edu/9780262548045/the-brain-abstracted/ [00:30:00] Daniel Dennett: Consciousness Explained https://www.amazon.com/Consciousness-Explained-Daniel-C-Dennett/dp/0316180661 [00:42:30] Cesar Hidalgo: Infinite Alphabet / Laws of Knowledge https://www.amazon.com/Infinite-Alphabet-Laws-Knowledge/dp/0241655676 Archive Article: [00:13:45] MLST Archive: Why Creativity Cannot Be Interpolated https://archive.mlst.ai/read/why-creativity-cannot-be-interpolated Research Study: [00:24:30] METR Study: AI OS Development https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ Paper: [00:24:45] Fred Brooks: No Silver Bullet https://www.cs.unc.edu/techreports/86-020.pdf [00:30:15] John Searle: Minds, Brains, and Programs https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A

What if life itself is just a really sophisticated computer program that wrote itself into existence? Blaise Agüera y Arcas presenting at ALife 2025 — the most technically detailed public walkthrough of the ideas in his *What is Life?* and *What is Intelligence?* books that we've come across.He covers the BFF experiments (self-replicating programs emerging spontaneously from random noise), the mathematical framework connecting Lotka-Volterra population dynamics with Smoluchowski coagulation, eigenvalue analysis of cooperation matrices, and his central claim that symbiogenesis — not mutation — is the primary engine of evolutionary novelty.The experimental results are genuinely striking: complex self-replicating code arising from random byte strings with zero mutation, a sharp phase transition that looks like gelation, and a proof that blocking deep symbiogenetic ancestry trees prevents the transition entirely.A few things worth flagging for critical viewers:— The substrate is more carefully engineered than the framing sometimes suggests. The choice of language, tape length, interaction protocol, and step limits all shape what emerges. Their own SUBLEQ counterexample (where self-replicators *don't* arise despite being theoretically possible) highlights that these design choices matter substantially — and a general theory of which substrates support this transition is still missing.— The leap from "self-replicating programs on fixed-length tapes" to "life was computational and intelligent from the start" involves significant philosophical extrapolation beyond what the experiments directly demonstrate.— The Bedau et al. (2000) open problems paper he references at the start actually sets a higher bar for Challenge 3.2 than BFF currently meets: it asks that "the internal organization of these 'organisms' and the boundaries separating them from their environment arise and be sustained through the activities of lower-level primitives" — whereas BFF's tape boundaries are fixed by design, not emergent. --- TIMESTAMPS: 00:00:00 Introduction: From Noise to Programs & ALife History 00:03:15 Defining Life: Function as the "Spirit" 00:05:45 Von Neumann's Insight: Life is Embodied Computation 00:09:15 Physics of Computation: Irreversibility & Fallacies 00:15:00 The BFF Experiment: Spontaneous Generation of Code 00:23:45 The Mystery: Complexity Growth Without Mutation 00:27:00 Symbiogenesis: The Engine of Novelty 00:33:15 Mathematical Proof: Blocking Symbiosis Stops Life 00:40:15 Evolutionary Implications: It's Symbiogenesis All The Way Down 00:44:30 Intelligence as Modeling Others 00:46:49 Q&A: Levels of Abstraction & Definitions --- REFERENCES: Paper: [00:01:16] Open Problems in Artificial Life https://direct.mit.edu/artl/article/6/4/363/2354/Open-Problems-in-Artificial-Life [00:09:30] When does a physical system compute? https://arxiv.org/abs/1309.7979 [00:15:00] Computational Life https://arxiv.org/abs/2406.19108 [00:27:30] On the Origin of Mitosing Cells https://pubmed.ncbi.nlm.nih.gov/11541392/ [00:42:00] The Major Evolutionary Transitions https://www.nature.com/articles/374227a0 [00:44:00] The ARC gene https://www.nih.gov/news-events/news-releases/memory-gene-goes-viral Person: [00:05:45] Alan Turing https://plato.stanford.edu/entries/turing/ [00:07:30] John von Neumann https://en.wikipedia.org/wiki/John_von_Neumann [00:11:15] Hector Zenil https://hectorzenil.net/ [00:12:00] Robert Sapolsky https://profiles.stanford.edu/robert-sapolsky --- LINKS: RESCRIPT: https://app.rescript.info/public/share/ff7gb6HpezOR3DF-gr9-rCoMFzzEgUjLQK6voV5XVWY

What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the philosophical and technical foundations of agency, intelligence, and the future of AI. Jeff doesn't hold back on the big questions. He argues that from a purely mathematical perspective, there's no structural difference between an agent and a rock – both execute policies that map inputs to outputs. The real distinction lies in *sophistication* – how complex are the internal computations? Does the system engage in planning and counterfactual reasoning, or is it just a lookup table that happens to give the right answers? *Key topics explored in this conversation:* *The Black Box Problem of Agency* – How can we tell if something is truly planning versus just executing a pre-computed response? Jeff explains why this question is nearly impossible to answer from the outside, and why the best we can do is ask which model gives us the simplest explanation. *Energy-Based Models Explained* – A masterclass on how EBMs differ from standard neural networks. The key insight: traditional networks only optimize weights, while energy-based models optimize *both* weights and internal states – a subtle but profound distinction that connects to Bayesian inference. *Why Your Brain Might Have Evolved from Your Nose* – One of the most surprising moments in the conversation. Jeff proposes that the complex, non-smooth nature of olfactory space may have driven the evolution of our associative cortex and planning abilities. *The JEPA Revolution* – A deep dive into Yann LeCun's Joint Embedding Prediction Architecture and why learning in latent space (rather than predicting every pixel) might be the key to more robust AI representations. *AI Safety Without Skynet Fears* – Jeff takes a refreshingly grounded stance on AI risk. He's less worried about rogue superintelligences and more concerned about humans becoming "reward function selectors" – couch potatoes who just approve or reject AI outputs. His proposed solution? Use inverse reinforcement learning to derive AI goals from observed human behavior, then make *small* perturbations rather than naive commands like "end world hunger." Whether you're interested in the philosophy of mind, the technical details of modern machine learning, or just want to understand what makes intelligence *tick,* this conversation delivers insights you won't find anywhere else. --- TIMESTAMPS: 00:00:00 Geometric Deep Learning & Physical Symmetries 00:00:56 Defining Agency: From Rocks to Planning 00:05:25 The Black Box Problem & Counterfactuals 00:08:45 Simulated Agency vs. Physical Reality 00:12:55 Energy-Based Models & Test-Time Training 00:17:30 Bayesian Inference & Free Energy 00:20:07 JEPA, Latent Space, & Non-Contrastive Learning 00:27:07 Evolution of Intelligence & Modular Brains 00:34:00 Scientific Discovery & Automated Experimentation 00:38:04 AI Safety, Enfeeblement & The Future of Work --- REFERENCES: Concept: [00:00:58] Free Energy Principle (FEP) https://en.wikipedia.org/wiki/Free_energy_principle [00:06:00] Monte Carlo Tree Search https://en.wikipedia.org/wiki/Monte_Carlo_tree_search Book: [00:09:00] The Intentional Stance https://mitpress.mit.edu/9780262540537/the-intentional-stance/ Paper: [00:13:00] A Tutorial on Energy-Based Learning (LeCun 2006) http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf [00:15:00] Auto-Encoding Variational Bayes (VAE) https://arxiv.org/abs/1312.6114 [00:20:15] JEPA (Joint Embedding Prediction Architecture) https://openreview.net/forum?id=BZ5a1r-kVsf [00:22:30] The Wake-Sleep Algorithm https://www.cs.toronto.edu/~hinton/absps/ws.pdf --- RESCRIPT: https://app.rescript.info/public/share/DJlSbJ_Qx080q315tWaqMWn3PixCQsOcM4Kf1IW9_Eo PDF: https://app.rescript.info/api/public/sessions/0efec296b9b6e905/pdf

Professor Mazviita Chirimuuta joins us for a fascinating deep dive into the philosophy of neuroscience and what it really means to understand the mind.*What can neuroscience actually tell us about how the mind works?* In this thought-provoking conversation, we explore the hidden assumptions behind computational theories of the brain, the limits of scientific abstraction, and why the question of machine consciousness might be more complicated than AI researchers assume.Mazviita, author of *The Brain Abstracted,* brings a unique perspective shaped by her background in both neuroscience research and philosophy. She challenges us to think critically about the metaphors we use to understand cognition — from the reflex theory of the late 19th century to today's dominant view of the brain as a computer.*Key topics explored:**The problem of oversimplification* — Why scientific models necessarily leave things out, and how this can sometimes lead entire fields astray. The cautionary tale of reflex theory shows how elegant explanations can blind us to biological complexity.*Is the brain really a computer?* — Mazviita unpacks the philosophical assumptions behind computational neuroscience and asks: if we can model anything computationally, what makes brains special? The answer might challenge everything you thought you knew about AI.*Haptic realism* — A fresh way of thinking about scientific knowledge that emphasizes interaction over passive observation. Knowledge isn't about reading the "source code of the universe" — it's something we actively construct through engagement with the world.*Why embodiment matters for understanding* — Can a disembodied language model truly understand? Mazviita makes a compelling case that human cognition is deeply entangled with our sensory-motor engagement and biological existence in ways that can't simply be abstracted away.*Technology and human finitude* — Drawing on Heidegger, we discuss how the dream of transcending our physical limitations through technology might reflect a fundamental misunderstanding of what it means to be a knower.This conversation is essential viewing for anyone interested in AI, consciousness, philosophy of mind, or the future of cognitive science. Whether you're skeptical of strong AI claims or a true believer in machine consciousness, Mazviita's careful philosophical analysis will give you new tools for thinking through these profound questions.---TIMESTAMPS:00:00:00 The Problem of Generalizing Neuroscience00:02:51 Abstraction vs. Idealization: The "Kaleidoscope"00:05:39 Platonism in AI: Discovering or Inventing Patterns?00:09:42 When Simplification Fails: The Reflex Theory00:12:23 Behaviorism and the "Black Box" Trap00:14:20 Haptic Realism: Knowledge Through Interaction00:20:23 Is Nature Protean? The Myth of Converging Truth00:23:23 The Computational Theory of Mind: A Useful Fiction?00:27:25 Biological Constraints: Why Brains Aren't Just Neural Nets00:31:01 Agency, Distal Causes, and Dennett's Stances00:37:13 Searle's Challenge: Causal Powers and Understanding00:41:58 Heidegger's Warning & The Experiment on Children---REFERENCES:Book:[00:01:28] The Brain Abstractedhttps://mitpress.mit.edu/9780262548045/the-brain-abstracted/[00:11:05] The Integrated Action of the Nervous Systemhttps://www.amazon.sg/integrative-action-nervous-system/dp/9354179029[00:18:15] The Quest for Certainty (Dewey)https://www.amazon.com/Quest-Certainty-Relation-Knowledge-Lectures/dp/0399501916[00:19:45] Realism for Realistic People (Chang)https://www.cambridge.org/core/books/realism-for-realistic-people/ACC93A7F03B15AA4D6F3A466E3FC5AB7---RESCRIPT:https://app.rescript.info/public/share/A6cZ1TY35p8ORMmYCWNBI0no9ChU3-Kx7dPXGJURvZ0PDF Transcript:https://app.rescript.info/api/public/sessions/0fb7767e066cf712/pdf

What if everything we think we know about the brain is just a really good metaphor that we forgot was a metaphor?This episode takes you on a journey through the history of scientific simplification, from a young Karl Friston watching wood lice in his garden to the bold claims that your mind is literally software running on biological hardware.We bring together some of the most brilliant minds we've interviewed — Professor Mazviita Chirimuuta, Francois Chollet, Joscha Bach, Professor Luciano Floridi, Professor Noam Chomsky, Nobel laureate John Jumper, and more — to wrestle with a deceptively simple question: *When scientists simplify reality to study it, what gets captured and what gets lost?**Key ideas explored:**The Spherical Cow Problem* — Science requires simplification. We're limited creatures trying to understand systems far more complex than our working memory can hold. But when does a useful model become a dangerous illusion?*The Kaleidoscope Hypothesis* — Francois Chollet's beautiful idea that beneath all the apparent chaos of reality lies simple, repeating patterns — like bits of colored glass in a kaleidoscope creating infinite complexity. Is this profound truth or Platonic wishful thinking?*Is Software Really Spirit?* — Joscha Bach makes the provocative claim that software is literally spirit, not metaphorically. We push back on this, asking whether the "sameness" we see across different computers running the same program exists in nature or only in our descriptions.*The Cultural Illusion of AGI* — Why does artificial general intelligence seem so inevitable to people in Silicon Valley? Professor Chirimuuta suggests we might be caught in a "cultural historical illusion" — our mechanistic assumptions about minds making AI seem like destiny when it might just be a bet.*Prediction vs. Understanding* — Nobel Prize winner John Jumper: AI can predict and control, but understanding requires a human in the loop. Throughout history, we've described the brain as hydraulic pumps, telegraph networks, telephone switchboards, and now computers. Each metaphor felt obviously true at the time. This episode asks: what will we think was naive about our current assumptions in fifty years?Featuring insights from *The Brain Abstracted* by Mazviita Chirimuuta — possibly the most influential book on how we think about thinking in 2025.---TIMESTAMPS:00:00:00 The Wood Louse & The Spherical Cow00:02:04 The Necessity of Abstraction00:04:42 Simplicius vs. Ignorantio: The Boxing Match00:06:39 The Kaleidoscope Hypothesis00:08:40 Is the Mind Software?00:13:15 Critique of Causal Patterns00:14:40 Temperature is Not a Thing00:18:24 The Ship of Theseus & Ontology00:23:45 Metaphors Hardening into Reality00:25:41 The Illusion of AGI Inevitability00:27:45 Prediction vs. Understanding00:32:00 Climbing the Mountain vs. The Helicopter00:34:53 Haptic Realism & The Limits of Knowledge---REFERENCES:Person:[00:00:00] Karl Friston (UCL)https://profiles.ucl.ac.uk/1236-karl-friston[00:06:30] Francois Chollethttps://fchollet.com/[00:14:41] Cesar Hidalgo, MLST interview.https://www.youtube.com/watch?v=vzpFOJRteeI[00:30:30] Terence Tao's Bloghttps://terrytao.wordpress.com/Book:[00:02:25] The Brain Abstractedhttps://mitpress.mit.edu/9780262548045/the-brain-abstracted/[00:06:00] On Learned Ignorancehttps://www.amazon.com/Nicholas-Cusa-learned-ignorance-translation/dp/0938060236[00:24:15] Science and the Modern Worldhttps://amazon.com/dp/0684836394 RESCRIPT:https://app.rescript.info/public/share/CYy0ex2M2kvcVRdMnSUky5O7H7hB7v2u_nVhoUiuKD4PDF Transcript: https://app.rescript.info/api/public/sessions/6c44c41e1e0fa6dd/pdf Thank you to Dr. Maxwell Ramstead for early script work on this show (Ph.D student of Friston) and the woodlice story came from him!

Dr. Jeff Beck, mathematician turned computational neuroscientist, joins us for a fascinating deep dive into why the future of AI might look less like ChatGPT and more like your own brain. **SPONSOR MESSAGES START** — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — **END** *What if the key to building truly intelligent machines isn't bigger models, but smarter ones?* In this conversation, Jeff makes a compelling case that we've been building AI backwards. While the tech industry races to scale up transformers and language models, Jeff argues we're missing something fundamental: the brain doesn't work like a giant prediction engine. It works like a scientist, constantly testing hypotheses about a world made of *objects* that interact through *forces* — not pixels and tokens. *The Bayesian Brain* — Jeff explains how your brain is essentially running the scientific method on autopilot. When you combine what you see with what you hear, you're doing optimal Bayesian inference without even knowing it. This isn't just philosophy — it's backed by decades of behavioral experiments showing humans are surprisingly efficient at handling uncertainty. *AutoGrad Changed Everything* — Forget transformers for a moment. Jeff argues the real hero of the AI boom was automatic differentiation, which turned AI from a math problem into an engineering problem. But in the process, we lost sight of what actually makes intelligence work. *The Cat in the Warehouse Problem* — Here's where it gets practical. Imagine a warehouse robot that's never seen a cat. Current AI would either crash or make something up. Jeff's approach? Build models that *know what they don't know*, can phone a friend to download new object models on the fly, and keep learning continuously. It's like giving robots the ability to say "wait, what IS that?" instead of confidently being wrong. *Why Language is a Terrible Model for Thought* — In a provocative twist, Jeff argues that grounding AI in language (like we do with LLMs) is fundamentally misguided. Self-report is the least reliable data in psychology — people routinely explain their own behavior incorrectly. We should be grounding AI in physics, not words. *The Future is Lots of Little Models* — Instead of one massive neural network, Jeff envisions AI systems built like video game engines: thousands of small, modular object models that can be combined, swapped, and updated independently. It's more efficient, more flexible, and much closer to how we actually think. Rescript: https://app.rescript.info/public/share/D-b494t8DIV-KRGYONJghvg-aelMmxSDjKthjGdYqsE --- TIMESTAMPS: 00:00:00 Introduction & The Bayesian Brain 00:01:25 Bayesian Inference & Information Processing 00:05:17 The Brain Metaphor: From Levers to Computers 00:10:13 Micro vs. Macro Causation & Instrumentalism 00:16:59 The Active Inference Community & AutoGrad 00:22:54 Object-Centered Models & The Grounding Problem 00:35:50 Scaling Bayesian Inference & Architecture Design 00:48:05 The Cat in the Warehouse: Solving Generalization 00:58:17 Alignment via Belief Exchange 01:05:24 Deception, Emergence & Cellular Automata --- REFERENCES: Paper: [00:00:24] Zoubin Ghahramani (Google DeepMind) https://pmc.ncbi.nlm.nih.gov/articles/PMC3538441/pdf/rsta201 [00:19:20] Mamba: Linear-Time Sequence Modeling https://arxiv.org/abs/2312.00752 [00:27:36] xLSTM: Extended Long Short-Term Memory https://arxiv.org/abs/2405.04517 [00:41:12] 3D Gaussian Splatting https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/ [01:07:09] Lenia: Biology of Artificial Life https://arxiv.org/abs/1812.05433 [01:08:20] Growing Neural Cellular Automata https://distill.pub/2020/growing-ca/ [01:14:05] DreamCoder https://arxiv.org/abs/2006.08381 [01:14:58] The Genomic Bottleneck https://www.nature.com/articles/s41467-019-11786-6 Person: [00:16:42] Karl Friston (UCL) https://www.youtube.com/watch?v=PNYWi996Beg

Tim sits down with Max Bennett to explore how our brains evolved over 600 million years—and what that means for understanding both human intelligence and AI. Max isn't a neuroscientist by training. He's a tech entrepreneur who got curious, started reading, and ended up weaving together three fields that rarely talk to each other: comparative psychology (what different animals can actually do), evolutionary neuroscience (how brains changed over time), and AI (what actually works in practice). *Your Brain Is a Guessing Machine* You don't actually "see" the world. Your brain builds a simulation of what it *thinks* is out there and just uses your eyes to check if it's right. That's why optical illusions work—your brain is filling in a triangle that isn't there, or can't decide if it's looking at a duck or a rabbit. *Rats Have Regrets* *Chimps Are Machiavellian* *Language Is the Human Superpower* *Does ChatGPT Think?* (truncated description, more on rescript) Understanding how the brain evolved isn't just about the past. It gives us clues about: - What's actually different between human intelligence and AI - Why we're so easily fooled by status games and tribal thinking - What features we might want to build into—or leave out of—future AI systems Get Max's book: https://www.amazon.com/Brief-History-Intelligence-Humans-Breakthroughs/dp/0063286343 Rescript: https://app.rescript.info/public/share/R234b7AXyDXZusqQ_43KMGsUSvJ2TpSz2I3emnI6j9A --- TIMESTAMPS: 00:00:00 Introduction: Outsider's Advantage & Neocortex Theories 00:11:34 Perception as Inference: The Filling-In Machine 00:19:11 Understanding, Recognition & Generative Models 00:36:39 How Mice Plan: Vicarious Trial & Error 00:46:15 Evolution of Self: The Layer 4 Mystery 00:58:31 Ancient Minds & The Social Brain: Machiavellian Apes 01:19:36 AI Alignment, Instrumental Convergence & Status Games 01:33:07 Metacognition & The IQ Paradox 01:48:40 Does GPT Have Theory of Mind? 02:00:40 Memes, Language Singularity & Brain Size Myths 02:16:44 Communication, Language & The Cyborg Future 02:44:25 Shared Fictions, World Models & The Reality Gap --- REFERENCES:Person: [00:00:05] Karl Friston (UCL) https://www.youtube.com/watch?v=PNYWi996Beg [00:00:06] Jeff Hawkins https://www.youtube.com/watch?v=6VQILbDqaI4 [00:12:19] Hermann von Helmholtz https://plato.stanford.edu/entries/hermann-helmholtz/ [00:38:34] David Redish (U. Minnesota) https://redishlab.umn.edu/ [01:10:19] Robin Dunbar https://www.psy.ox.ac.uk/people/robin-dunbar [01:15:04] Emil Menzel https://www.sciencedirect.com/bookseries/behavior-of-nonhuman-primates/vol/5/suppl/C [01:19:49] Nick Bostrom https://nickbostrom.com/ [02:28:25] Noam Chomsky https://linguistics.mit.edu/user/chomsky/ [03:01:22] Judea Pearl https://samueli.ucla.edu/people/judea-pearl/ Concept/Framework: [00:05:04] Active Inference https://www.youtube.com/watch?v=KkR24ieh5Ow Paper: [00:35:59] Predictions not commands [Rick A Adams] https://pubmed.ncbi.nlm.nih.gov/23129312/ Book: [01:25:42] The Elephant in the Brain https://www.amazon.com/Elephant-Brain-Hidden-Motives-Everyday/dp/0190495995 [01:28:27] The Status Game https://www.goodreads.com/book/show/58642436-the-status-game [02:00:40] The Selfish Gene https://amazon.com/dp/0198788606 [02:14:25] The Language Game https://www.amazon.com/Language-Game-Improvisation-Created-Changed/dp/1541674987 [02:54:40] The Evolution of Language https://www.amazon.com/Evolution-Language-Approaches/dp/052167736X [03:09:37] The Three-Body Problem https://amazon.com/dp/0765377063

César Hidalgo has spent years trying to answer a deceptively simple question: What is knowledge, and why is it so hard to move around? We all have this intuition that knowledge is just... information. Write it down in a book, upload it to GitHub, train an AI on it—done. But César argues that's completely wrong. Knowledge isn't a thing you can copy and paste. It's more like a living organism that needs the right environment, the right people, and constant exercise to survive. Guest: César Hidalgo, Director of the Center for Collective Learning 1. Knowledge Follows Laws (Like Physics) 2. You Can't Download Expertise 3. Why Big Companies Fail to Adapt 4. The "Infinite Alphabet" of Economies If you think AI can just "copy" human knowledge, or that development is just about throwing money at poor countries, or that writing things down preserves them forever—this conversation will change your mind. Knowledge is fragile, specific, and collective. It decays fast if you don't use it. The Infinite Alphabet [César A. Hidalgo] https://www.penguin.co.uk/books/458054/the-infinite-alphabet-by-hidalgo-cesar-a/9780241655672 https://x.com/cesifoti Rescript link. https://app.rescript.info/public/share/eaBHbEo9xamwbwpxzcVVm4NQjMh7lsOQKeWwNxmw0JQ --- TIMESTAMPS: 00:00:00 The Three Laws of Knowledge 00:02:28 Rival vs. Non-Rival: The Economics of Ideas 00:05:43 Why You Can't Just 'Download' Knowledge 00:08:11 The Detective Novel Analogy 00:11:54 Collective Learning & Organizational Networks 00:16:27 Architectural Innovation: Amazon vs. Barnes & Noble 00:19:15 The First Law: Learning Curves 00:23:05 The Samuel Slater Story: Treason & Memory 00:28:31 Physics of Knowledge: Joule's Cannon 00:32:33 Extensive vs. Intensive Properties 00:35:45 Knowledge Decay: Ise Temple & Polaroid 00:41:20 Absorptive Capacity: Sony & Donetsk 00:47:08 Disruptive Innovation & S-Curves 00:51:23 Team Size & The Cost of Innovation 00:57:13 Geography of Knowledge: Vespa's Origin 01:04:34 Migration, Diversity & 'Planet China' 01:12:02 Institutions vs. Knowledge: The China Story 01:21:27 Economic Complexity & The Infinite Alphabet 01:32:27 Do LLMs Have Knowledge? --- REFERENCES: Book: [00:47:45] The Innovator's Dilemma (Christensen) https://www.amazon.com/Innovators-Dilemma-Revolutionary-Change-Business/dp/0062060244 [00:55:15] Why Greatness Cannot Be Planned https://amazon.com/dp/3319155237 [01:35:00] Why Information Grows https://amazon.com/dp/0465048994 Paper: [00:03:15] Endogenous Technological Change (Romer, 1990) https://web.stanford.edu/~klenow/Romer_1990.pdf [00:03:30] A Model of Growth Through Creative Destruction (Aghion & Howitt, 1992) https://dash.harvard.edu/server/api/core/bitstreams/7312037d-2b2d-6bd4-e053-0100007fdf3b/content [00:14:55] Organizational Learning: From Experience to Knowledge (Argote & Miron-Spektor, 2011) https://www.researchgate.net/publication/228754233_Organizational_Learning_From_Experience_to_Knowledge [00:17:05] Architectural Innovation (Henderson & Clark, 1990) https://www.researchgate.net/publication/200465578_Architectural_Innovation_The_Reconfiguration_of_Existing_Product_Technologies_and_the_Failure_of_Established_Firms [00:19:45] The Learning Curve Equation (Thurstone, 1916) https://dn790007.ca.archive.org/0/items/learningcurveequ00thurrich/learningcurveequ00thurrich.pdf [00:21:30] Factors Affecting the Cost of Airplanes (Wright, 1936) https://pdodds.w3.uvm.edu/research/papers/others/1936/wright1936a.pdf [00:52:45] Are Ideas Getting Harder to Find? (Bloom et al.) https://web.stanford.edu/~chadj/IdeaPF.pdf [01:33:00] LLMs/ Emergence https://arxiv.org/abs/2506.11135 Person: [00:25:30] Samuel Slater https://en.wikipedia.org/wiki/Samuel_Slater [00:42:05] Masaru Ibuka (Sony) https://www.sony.com/en/SonyInfo/CorporateInfo/History/SonyHistory/1-02.html

This is a lively, no-holds-barred debate about whether AI can truly be intelligent, conscious, or understand anything at all — and what happens when (or if) machines become smarter than us. Dr. Mike Israetel is a sports scientist, entrepreneur, and co-founder of RP Strength (a fitness company). He describes himself as a "dilettante" in AI but brings a fascinating outsider's perspective. Jared Feather (IFBB Pro bodybuilder and exercise physiologist) The Big Questions: 1. When is superintelligence coming? 2. Does AI actually understand anything? 3. The Simulation Debate (The Spiciest Part) 4. Will AI kill us all? (The Doomer Debate) 5. What happens to human jobs and purpose? 6. Do we need suffering? Mikes channel: https://www.youtube.com/channel/UCfQgsKhHjSyRLOp9mnffqVg RESCRIPT INTERACTIVE PLAYER: https://app.rescript.info/public/share/GVMUXHCqctPkXH8WcYtufFG7FQcdJew_RL_MLgMKU1U --- TIMESTAMPS: 00:00:00 Introduction & Workout Demo 00:04:15 ASI Timelines & Definitions 00:10:24 The Embodiment Debate 00:18:28 Neutrinos & Abstract Knowledge 00:25:56 Can AI Learn From YouTube? 00:31:25 Diversity of Intelligence 00:36:00 AI Slop & Understanding 00:45:18 The Simulation Argument: Fire & Water 00:58:36 Consciousness & Zombies 01:04:30 Do Reasoning Models Actually Reason? 01:12:00 The Live Learning Problem 01:19:15 Superintelligence & Benevolence 01:28:59 What is True Agency? 01:37:20 Game Theory & The "Kill All Humans" Fallacy 01:48:05 Regulation & The China Factor 01:55:52 Mind Uploading & The Future of Love 02:04:41 Economics of ASI: Will We Be Useless? 02:13:35 The Matrix & The Value of Suffering 02:17:30 Transhumanism & Inequality 02:21:28 Debrief: AI Medical Advice & Final Thoughts --- REFERENCES: Paper: [00:10:45] Alchemy and Artificial Intelligence (Dreyfus) https://www.rand.org/content/dam/rand/pubs/papers/2006/P3244.pdf [00:10:55] The Chinese Room Argument (John Searle) https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf [00:11:05] The Symbol Grounding Problem (Stephen Harnad) https://arxiv.org/html/cs/9906002 [00:23:00] Attention Is All You Need https://arxiv.org/abs/1706.03762 [00:45:00] GPT-4 Technical Report https://arxiv.org/abs/2303.08774 [01:45:00] Anthropic Agentic Misalignment Paper https://www.anthropic.com/research/agentic-misalignment [02:17:45] Retatrutide https://pubmed.ncbi.nlm.nih.gov/37366315/ Organization: [00:15:50] CERN https://home.cern/ [01:05:00] METR Long Horizon Evaluations https://evaluations.metr.org/ MLST Episode: [00:23:10] MLST: Llion Jones - Inventors' Remorse https://www.youtube.com/watch?v=DtePicx_kFY [00:50:30] MLST: Blaise Agüera y Arcas Interview https://www.youtube.com/watch?v=rMSEqJ_4EBk [01:10:00] MLST: David Krakauer https://www.youtube.com/watch?v=dY46YsGWMIc Event: [00:23:40] ARC Prize/Challenge https://arcprize.org/ Book: [00:24:45] The Brain Abstracted https://www.amazon.com/Brain-Abstracted-Simplification-Philosophy-Neuroscience/dp/0262548046 [00:47:55] Pamela McCorduck https://www.amazon.com/Machines-Who-Think-Artificial-Intelligence/dp/1568812051 [01:23:15] The Singularity Is Nearer (Ray Kurzweil) https://www.amazon.com/Singularity-Nearer-Ray-Kurzweil-ebook/dp/B08Y6FYJVY [01:27:35] A Fire Upon The Deep (Vernor Vinge) https://www.amazon.com/Fire-Upon-Deep-S-F-MASTERWORKS-ebook/dp/B00AVUMIZE/ [02:04:50] Deep Utopia (Nick Bostrom) https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642 [02:05:00] Technofeudalism (Yanis Varoufakis) https://www.amazon.com/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1685891241 Visual Context Needed: [00:29:40] AT-AT Walker (Star Wars) https://starwars.fandom.com/wiki/All_Terrain_Armored_Transport Person: [00:33:15] Andrej Karpathy https://karpathy.ai/ Video: [01:40:00] Mike Israetel vs Liron Shapira AI Doom Debate https://www.youtube.com/watch?v=RaDWSPMdM4o Company: [02:26:30] Examine.com https://examine.com/

We often think of Large Language Models (LLMs) as all-knowing, but as the team reveals, they still struggle with the logic of a second-grader. Why can’t ChatGPT reliably add large numbers? Why does it "hallucinate" the laws of physics? The answer lies in the architecture. This episode explores how *Category Theory* —an ultra-abstract branch of mathematics—could provide the "Periodic Table" for neural networks, turning the "alchemy" of modern AI into a rigorous science. In this deep-dive exploration, *Andrew Dudzik*, *Petar Velichkovich*, *Taco Cohen*, *Bruno Gavranović*, and *Paul Lessard* join host *Tim Scarfe* to discuss the fundamental limitations of today’s AI and the radical mathematical framework that might fix them. TRANSCRIPT: https://app.rescript.info/public/share/LMreunA-BUpgP-2AkuEvxA7BAFuA-VJNAp2Ut4MkMWk --- Key Insights in This Episode: * *The "Addition" Problem:* *Andrew Dudzik* explains why LLMs don't actually "know" math—they just recognize patterns. When you change a single digit in a long string of numbers, the pattern breaks because the model lacks the internal "machinery" to perform a simple carry operation. * *Beyond Alchemy:* deep learning is currently in its "alchemy" phase—we have powerful results, but we lack a unifying theory. Category Theory is proposed as the framework to move AI from trial-and-error to principled engineering. [00:13:49] * *Algebra with Colors:* To make Category Theory accessible, the guests use brilliant analogies—like thinking of matrices as *magnets with colors* that only snap together when the types match. This "partial compositionality" is the secret to building more complex internal reasoning. [00:09:17] * *Synthetic vs. Analytic Math:* *Paul Lessard* breaks down the philosophical shift needed in AI research: moving from "Analytic" math (what things are made of) to "Synthetic" math [00:23:41] --- Why This Matters for AGI If we want AI to solve the world's hardest scientific problems, it can't just be a "stochastic parrot." It needs to internalize the rules of logic and computation. By imbuing neural networks with categorical priors, researchers are attempting to build a future where AI doesn't just predict the next word—it understands the underlying structure of the universe. --- TIMESTAMPS: 00:00:00 The Failure of LLM Addition & Physics 00:01:26 Tool Use vs Intrinsic Model Quality 00:03:07 Efficiency Gains via Internalization 00:04:28 Geometric Deep Learning & Equivariance 00:07:05 Limitations of Group Theory 00:09:17 Category Theory: Algebra with Colors 00:11:25 The Systematic Guide of Lego-like Math 00:13:49 The Alchemy Analogy & Unifying Theory 00:15:33 Information Destruction & Reasoning 00:18:00 Pathfinding & Monoids in Computation 00:20:15 System 2 Reasoning & Error Awareness 00:23:31 Analytic vs Synthetic Mathematics 00:25:52 Morphisms & Weight Tying Basics 00:26:48 2-Categories & Weight Sharing Theory 00:28:55 Higher Categories & Emergence 00:31:41 Compositionality & Recursive Folds 00:34:05 Syntax vs Semantics in Network Design 00:36:14 Homomorphisms & Multi-Sorted Syntax 00:39:30 The Carrying Problem & Hopf Fibrations Petar Veličković (GDM) https://petar-v.com/ Paul Lessard https://www.linkedin.com/in/paul-roy-lessard/ Bruno Gavranović https://www.brunogavranovic.com/ Andrew Dudzik (GDM) https://www.linkedin.com/in/andrew-dudzik-222789142/ --- REFERENCES: Model: [00:01:05] Veo https://deepmind.google/models/veo/ [00:01:10] Genie https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/ Paper: [00:04:30] Geometric Deep Learning Blueprint https://arxiv.org/abs/2104.13478 https://www.youtube.com/watch?v=bIZB1hIJ4u8 [00:16:45] AlphaGeometry https://arxiv.org/abs/2401.08312 [00:16:55] AlphaCode https://arxiv.org/abs/2203.07814 [00:17:05] FunSearch https://www.nature.com/articles/s41586-023-06924-6 [00:37:00] Attention Is All You Need https://arxiv.org/abs/1706.03762 [00:43:00] Categorical Deep Learning https://arxiv.org/abs/2402.15332

Is a car that wins a Formula 1 race the best choice for your morning commute? Probably not. In this sponsored deep dive with Prolific, we explore why the same logic applies to Artificial Intelligence. While models are currently shattering records on technical exams, they often fail the most important test of all: **the human experience.** Why High Benchmark Scores Don’t Mean Better AI Joining us are **Andrew Gordon** (Staff Researcher in Behavioral Science) and **Nora Petrova** (AI Researcher) from **Prolific**. They reveal the hidden flaws in how we currently rank AI and introduce a more rigorous, "humane" way to measure whether these models are actually helpful, safe, and relatable for real people. --- Key Insights in This Episode: * *The F1 Car Analogy:* Andrew explains why a model that excels at the "Humanities Last Exam" might be a nightmare for daily use. Technical benchmarks often ignore the nuances of human communication and adaptability. * *The "Wild West" of AI Safety:* As users turn to AI for sensitive topics like mental health, Nora highlights the alarming lack of oversight and the "thin veneer" of safety training—citing recent controversial incidents like Grok-3’s "Mecha Hitler." * *Fixing the "Leaderboard Illusion":* The team critiques current popular rankings like Chatbot Arena, discussing how anonymous, unstratified voting can lead to biased results and how companies can "game" the system. * *The Xbox Secret to AI Ranking:* Discover how Prolific uses *TrueSkill*—the same algorithm Microsoft developed for Xbox Live matchmaking—to create a fairer, more statistically sound leaderboard for LLMs. * *The Personality Gap:* Early data from the **Humane Leaderboard** suggests that while AI is getting smarter, it is actually performing *worse* on metrics like personality, culture, and "sycophancy" (the tendency for models to become annoying "people-pleasers"). --- About the HUMAINE Leaderboard Moving beyond simple "A vs. B" testing, the researchers discuss their new framework that samples participants based on *census data* (Age, Ethnicity, Political Alignment). By using a representative sample of the general public rather than just tech enthusiasts, they are building a standard that reflects the values of the real world. *Are we building models for benchmarks, or are we building them for humans? It’s time to change the scoreboard.* Rescript link: https://app.rescript.info/public/share/IDqwjY9Q43S22qSgL5EkWGFymJwZ3SVxvrfpgHZLXQc --- TIMESTAMPS: 00:00:00 Introduction & The Benchmarking Problem 00:01:58 The Fractured State of AI Evaluation 00:03:54 AI Safety & Interpretability 00:05:45 Bias in Chatbot Arena 00:06:45 Prolific's Three Pillars Approach 00:09:01 TrueSkill Ranking & Efficient Sampling 00:12:04 Census-Based Representative Sampling 00:13:00 Key Findings: Culture, Personality & Sycophancy --- REFERENCES: Paper: [00:00:15] MMLU https://arxiv.org/abs/2009.03300 [00:05:10] Constitutional AI https://arxiv.org/abs/2212.08073 [00:06:45] The Leaderboard Illusion https://arxiv.org/abs/2504.20879 [00:09:41] HUMAINE Framework Paper https://huggingface.co/blog/ProlificAI/humaine-framework Company: [00:00:30] Prolific https://www.prolific.com [00:01:45] Chatbot Arena https://lmarena.ai/ Person: [00:00:35] Andrew Gordon https://www.linkedin.com/in/andrew-gordon-03879919a/ [00:00:45] Nora Petrova https://www.linkedin.com/in/nora-petrova/ Event: Algorithm: [00:09:01] Microsoft TrueSkill https://www.microsoft.com/en-us/research/project/trueskill-ranking-system/ Leaderboard: [00:09:21] Prolific HUMAINE Leaderboard https://www.prolific.com/humaine [00:09:31] HUMAINE HuggingFace Space https://huggingface.co/spaces/ProlificAI/humaine-leaderboard [00:10:21] Prolific AI Leaderboard Portal https://www.prolific.com/leaderboard Dataset: [00:09:51] Prolific Social Reasoning RLHF Dataset https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf Organization: [00:10:31] MLCommons https://mlcommons.org/

What if everything we think we know about AI understanding is wrong? Is compression the key to intelligence? Or is there something more—a leap from memorization to true abstraction? In this fascinating conversation, we sit down with **Professor Yi Ma**—world-renowned expert in deep learning, IEEE/ACM Fellow, and author of the groundbreaking new book *Learning Deep Representations of Data Distributions*. Professor Ma challenges our assumptions about what large language models actually do, reveals why 3D reconstruction isn't the same as understanding, and presents a unified mathematical theory of intelligence built on just two principles: **parsimony** and **self-consistency**. **SPONSOR MESSAGES START** — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — **END** Key Insights: **LLMs Don't Understand—They Memorize** Language models process text (*already* compressed human knowledge) using the same mechanism we use to learn from raw data. **The Illusion of 3D Vision** Sora and NeRFs etc that can reconstruct 3D scenes still fail miserably at basic spatial reasoning **"All Roads Lead to Rome"** Why adding noise is *necessary* for discovering structure. **Why Gradient Descent Actually Works** Natural optimization landscapes are surprisingly smooth—a "blessing of dimensionality" **Transformers from First Principles** Transformer architectures can be mathematically derived from compression principles — INTERACTIVE AI TRANSCRIPT PLAYER w/REFS (ReScript): https://app.rescript.info/public/share/Z-dMPiUhXaeMEcdeU6Bz84GOVsvdcfxU_8Ptu6CTKMQ About Professor Yi Ma Yi Ma is the inaugural director of the School of Computing and Data Science at Hong Kong University and a visiting professor at UC Berkeley. https://people.eecs.berkeley.edu/~yima/ https://scholar.google.com/citations?user=XqLiBQMAAAAJ&hl=en https://x.com/YiMaTweets **Slides from this conversation:** https://www.dropbox.com/scl/fi/sbhbyievw7idup8j06mlr/slides.pdf?rlkey=7ptovemezo8bj8tkhfi393fh9&dl=0 **Related Talks by Professor Ma:** - Pursuing the Nature of Intelligence (ICLR): https://www.youtube.com/watch?v=LT-F0xSNSjo - Earlier talk at Berkeley: https://www.youtube.com/watch?v=TihaCUjyRLM TIMESTAMPS: 00:00:00 Introduction 00:02:08 The First Principles Book & Research Vision 00:05:21 Two Pillars: Parsimony & Consistency 00:09:50 Evolution vs. Learning: The Compression Mechanism 00:14:36 LLMs: Memorization Masquerading as Understanding 00:19:55 The Leap to Abstraction: Empirical vs. Scientific 00:27:30 Platonism, Deduction & The ARC Challenge 00:35:57 Specialization & The Cybernetic Legacy 00:41:23 Deriving Maximum Rate Reduction 00:48:21 The Illusion of 3D Understanding: Sora & NeRF 00:54:26 All Roads Lead to Rome: The Role of Noise 00:59:56 All Roads Lead to Rome: The Role of Noise 01:00:14 Benign Non-Convexity: Why Optimization Works 01:06:35 Double Descent & The Myth of Overfitting 01:14:26 Self-Consistency: Closed-Loop Learning 01:21:03 Deriving Transformers from First Principles 01:30:11 Verification & The Kevin Murphy Question 01:34:11 CRATE vs. ViT: White-Box AI & Conclusion REFERENCES: Book: [00:03:04] Learning Deep Representations of Data Distributions https://ma-lab-berkeley.github.io/deep-representation-learning-book/ [00:18:38] A Brief History of Intelligence https://www.amazon.co.uk/BRIEF-HISTORY-INTELLIGEN-HB-Evolution/dp/0008560099 [00:38:14] Cybernetics https://mitpress.mit.edu/9780262730099/cybernetics/ Book (Yi Ma): [00:03:14] 3-D Vision book https://link.springer.com/book/10.1007/978-0-387-21779-6 refs on ReScript link/YT

Pedro Domingos, author of the bestselling book "The Master Algorithm," introduces his latest work: Tensor Logic - a new programming language he believes could become the fundamental language for artificial intelligence. Think of it like this: Physics found its language in calculus. Circuit design found its language in Boolean logic. Pedro argues that AI has been missing its language - until now. **SPONSOR MESSAGES START** — Build your ideas with AI Studio from Google - http://ai.studio/build — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — **END** Current AI is split between two worlds that don't play well together: Deep Learning (neural networks, transformers, ChatGPT) - great at learning from data, terrible at logical reasoning Symbolic AI (logic programming, expert systems) - great at logical reasoning, terrible at learning from messy real-world data Tensor Logic unifies both. It's a single language where you can: Write logical rules that the system can actually learn and modify Do transparent, verifiable reasoning (no hallucinations) Mix "fuzzy" analogical thinking with rock-solid deduction INTERACTIVE TRANSCRIPT: https://app.rescript.info/public/share/NP4vZQ-GTETeN_roB2vg64vbEcN7isjJtz4C86WSOhw TOC: 00:00:00 - Introduction 00:04:41 - What is Tensor Logic? 00:09:59 - Tensor Logic vs PyTorch & Einsum 00:17:50 - The Master Algorithm Connection 00:20:41 - Predicate Invention & Learning New Concepts 00:31:22 - Symmetries in AI & Physics 00:35:30 - Computational Reducibility & The Universe 00:43:34 - Technical Details: RNN Implementation 00:45:35 - Turing Completeness Debate 00:56:45 - Transformers vs Turing Machines 01:02:32 - Reasoning in Embedding Space 01:11:46 - Solving Hallucination with Deductive Modes 01:16:17 - Adoption Strategy & Migration Path 01:21:50 - AI Education & Abstraction 01:24:50 - The Trillion-Dollar Waste REFS Tensor Logic: The Language of AI [Pedro Domingos] https://arxiv.org/abs/2510.12269 The Master Algorithm [Pedro Domingos] https://www.amazon.co.uk/Master-Algorithm-Ultimate-Learning-Machine/dp/0241004543 Einsum is All you Need (TIM ROCKTÄSCHEL) https://rockt.ai/2018/04/30/einsum https://www.youtube.com/watch?v=6DrCq8Ry2cw Autoregressive Large Language Models are Computationally Universal (Dale Schuurmans et al - GDM) https://arxiv.org/abs/2410.03170 Memory Augmented Large Language Models are Computationally Universal [Dale Schuurmans] https://arxiv.org/pdf/2301.04589 On the computational power of NNs [95/Siegelmann] https://binds.cs.umass.edu/papers/1995_Siegelmann_JComSysSci.pdf Sebastian Bubeck https://www.reddit.com/r/OpenAI/comments/1oacp38/openai_researcher_sebastian_bubeck_falsely_claims/ I am a strange loop - Hofstadter https://www.amazon.co.uk/Am-Strange-Loop-Douglas-Hofstadter/dp/0465030793 Stephen Wolfram https://www.youtube.com/watch?v=dkpDjd2nHgo The Complex World: An Introduction to the Foundations of Complexity Science [David C. Krakauer] https://www.amazon.co.uk/Complex-World-Introduction-Foundations-Complexity/dp/1947864629 Geometric Deep Learning https://www.youtube.com/watch?v=bIZB1hIJ4u8 Andrew Wilson (NYU) https://www.youtube.com/watch?v=M-jTeBCEGHc Yi Ma https://www.patreon.com/posts/yi-ma-scientific-141953348 Roger Penrose - road to reality https://www.amazon.co.uk/Road-Reality-Complete-Guide-Universe/dp/0099440687 Artificial Intelligence: A Modern Approach [Russel and Norvig] https://www.amazon.co.uk/Artificial-Intelligence-Modern-Approach-Global/dp/1292153962

The Transformer architecture (which powers ChatGPT and nearly all modern AI) might be trapping the industry in a localized rut, preventing us from finding true intelligent reasoning, according to the person who co-invented it. Llion Jones and Luke Darlow, key figures at the research lab Sakana AI, join the show to make this provocative argument, and also introduce new research which might lead the way forwards. **SPONSOR MESSAGES START** — Build your ideas with AI Studio from Google - http://ai.studio/build — Tufa AI Labs is hiring ML Research Engineers https://tufalabs.ai/ — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — **END** The "Spiral" Problem – Llion uses a striking visual analogy to explain what current AI is missing. If you ask a standard neural network to understand a spiral shape, it solves it by drawing tiny straight lines that just happen to look like a spiral. It "fakes" the shape without understanding the concept of spiraling. Introducing the Continuous Thought Machine (CTM) Luke Darlow deep dives into their solution: a biology-inspired model that fundamentally changes how AI processes information. The Maze Analogy: Luke explains that standard AI tries to solve a maze by staring at the whole image and guessing the entire path instantly. Their new machine "walks" through the maze step-by-step. Thinking Time: This allows the AI to "ponder." If a problem is hard, the model can naturally spend more time thinking about it before answering, effectively allowing it to correct its own mistakes and backtrack—something current Language Models struggle to do genuinely. https://sakana.ai/ https://x.com/YesThisIsLion https://x.com/LearningLukeD TRANSCRIPT: https://app.rescript.info/public/share/crjzQ-Jo2FQsJc97xsBdfzfOIeMONpg0TFBuCgV2Fu8 TOC: 00:00:00 - Stepping Back from Transformers 00:00:43 - Introduction to Continuous Thought Machines (CTM) 00:01:09 - The Changing Atmosphere of AI Research 00:04:13 - Sakana’s Philosophy: Research Freedom 00:07:45 - The Local Minimum of Large Language Models 00:18:30 - Representation Problems: The Spiral Example 00:29:12 - Technical Deep Dive: CTM Architecture 00:36:00 - Adaptive Computation & Maze Solving 00:47:15 - Model Calibration & Uncertainty 01:00:43 - Sudoku Bench: Measuring True Reasoning REFS: Why Greatness Cannot be planned [Kenneth Stanley] https://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 https://www.youtube.com/watch?v=lhYGXYeMq_E The Hardware Lottery [Sara Hooker] https://arxiv.org/abs/2009.06489 https://www.youtube.com/watch?v=sQFxbQ7ade0 Continuous Thought Machines [Luke Darlow et al / Sakana] https://arxiv.org/abs/2505.05522 https://sakana.ai/ctm/ LSTM: The Comeback Story? [Prof. Sepp Hochreiter] https://www.youtube.com/watch?v=8u2pW2zZLCs Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis [Kumar/Stanley] https://arxiv.org/pdf/2505.11581 A Spline Theory of Deep Networks [Randall Balestriero] https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf https://www.youtube.com/watch?v=86ib0sfdFtw https://www.youtube.com/watch?v=l3O2J3LMxqI On the Biology of a Large Language Model [Anthropic, Jack Lindsey et al] https://transformer-circuits.pub/2025/attribution-graphs/biology.html The ARC Prize 2024 Winning Algorithm [Daniel Franzen and Jan Disselhoff] “The ARChitects” https://www.youtube.com/watch?v=mTX_sAq--zY Neural Turing Machine [Graves] https://arxiv.org/pdf/1410.5401 Adaptive Computation Time for Recurrent Neural Networks [Graves] https://arxiv.org/abs/1603.08983 Sudoko Bench [Sakana] https://pub.sakana.ai/sudoku/

Ever wonder where AI models actually get their "intelligence"? We reveal the dirty secret of Silicon Valley: behind every impressive AI system are thousands of real humans providing crucial data, feedback, and expertise.Guest: Phelim Bradley, CEO and Co-founder of ProlificPhelim Bradley runs Prolific, a platform that connects AI companies with verified human experts who help train and evaluate their models. Think of it as a sophisticated marketplace matching the right human expertise to the right AI task - whether that's doctors evaluating medical chatbots or coders reviewing AI-generated software.Prolific: https://prolific.com/?utm_source=mlsthttps://uk.linkedin.com/in/phelim-bradley-84300826The discussion dives into:**The human data pipeline**: How AI companies rely on human intelligence to train, refine, and validate their models - something rarely discussed openly**Quality over quantity**: Why paying humans well and treating them as partners (not commodities) produces better AI training data**The matching challenge**: How Prolific solves the complex problem of finding the right expert for each specific task, similar to matching Uber drivers to riders but with deep expertise requirements**Future of work**: What it means when human expertise becomes an on-demand service, and why this might actually create more opportunities rather than fewer**Geopolitical implications**: Why the centralization of AI development in US tech companies should concern Europe and the UK

"What is life?" - asks Chris Kempes, a professor at the Santa Fe Institute. Chris explains that scientists are moving beyond a purely Earth-based, biological view and are searching for a universal theory of life that could apply to anything, anywhere in the universe. He proposes that things we don't normally consider "alive"—like human culture, language, or even artificial intelligence; could be seen as life forms existing on different "substrates". To understand this, Chris presents a fascinating three-level framework: - Materials: The physical stuff life is made of. He argues this could be incredibly diverse across the universe, and we shouldn't expect alien life to share our biochemistry. - Constraints: The universal laws of physics (like gravity or diffusion) that all life must obey, regardless of what it's made of. This is where different life forms start to look more similar. - Principles: At the highest level are abstract principles like evolution and learning. Chris suggests these computational or "optimization" rules are what truly define a living system. A key idea is "convergence" – using the example of the eye. It's such a complex organ that you'd think it evolved only once. However, eyes evolved many separate times across different species. This is because the physics of light provides a clear "target", and evolution found similar solutions to the problem of seeing, even with different starting materials. **SPONSOR MESSAGES** — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — Check out NotebookLM from Google here - https://notebooklm.google.com/ - it’s really good for doing research directly from authoritative source material, minimising hallucinations. — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — Prof. Chris Kempes: https://www.santafe.edu/people/profile/chris-kempes TRANSCRIPT: https://app.rescript.info/public/share/Y2cI1i0nX_-iuZitvlguHvaVLQTwPX1Y_E1EHxV0i9I TOC: 00:00:00 - Introduction to Chris Kempes and the Santa Fe Institute 00:02:28 - The Three Cultures of Science 00:05:08 - What Makes a Good Scientific Theory? 00:06:50 - The Universal Theory of Life 00:09:40 - The Role of Material in Life 00:12:50 - A Hierarchy for Understanding Life 00:13:55 - How Life Diversifies and Converges 00:17:53 - Adaptive Processes and Defining Life 00:19:28 - Functionalism, Memes, and Phylogenies 00:22:58 - Convergence at Multiple Levels 00:25:45 - The Possibility of Simulating Life 00:28:16 - Intelligence, Parasitism, and Spectrums of Life 00:32:39 - Phase Changes in Evolution 00:36:16 - The Separation of Matter and Logic 00:37:21 - Assembly Theory and Quantifying Complexity REFS: Developing a predictive science of the biosphere requires the integration of scientific cultures [Kempes et al] https://www.pnas.org/doi/10.1073/pnas.2209196121 Seeing with an extra sense (“Dangerous prediction”) [Rob Phillips] https://www.sciencedirect.com/science/article/pii/S0960982224009035 The Multiple Paths to Multiple Life [Christopher P. Kempes & David C. Krakauer] https://link.springer.com/article/10.1007/s00239-021-10016-2 The Information Theory of Individuality [David Krakauer et al] https://arxiv.org/abs/1412.2447 Minds, Brains and Programs [Searle] https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf The error threshold https://www.sciencedirect.com/science/article/abs/pii/S0168170204003843 Assembly theory and its relationship with computational complexity [Kempes et al] https://arxiv.org/abs/2406.12176

Blaise Agüera y Arcas explores some mind-bending ideas about what intelligence and life really are—and why they might be more similar than we think (filmed at ALIFE conference, 2025 - https://2025.alife.org/). Life and intelligence are both fundamentally computational (he says). From the very beginning, living things have been running programs. Your DNA? It's literally a computer program, and the ribosomes in your cells are tiny universal computers building you according to those instructions. **SPONSOR MESSAGES** — Prolific - Quality data. From real people. For faster breakthroughs. https://www.prolific.com/?utm_source=mlst — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — Blaise argues that there is more to evolution than random mutations (like most people think). The secret to increasing complexity is *merging* i.e. when different organisms or systems come together and combine their histories and capabilities. Blaise describes his "BFF" experiment where random computer code spontaneously evolved into self-replicating programs, showing how purpose and complexity can emerge from pure randomness through computational processes. https://en.wikipedia.org/wiki/Blaise_Ag%C3%BCera_y_Arcas https://x.com/blaiseaguera?lang=en TRANSCRIPT: https://app.rescript.info/public/share/VX7Gktfr3_wIn4Bj7cl9StPBO1MN4R5lcJ11NE99hLg TOC: 00:00:00 Introduction - New book "What is Intelligence?" 00:01:45 Life as computation - Von Neumann's insights 00:12:00 BFF experiment - How purpose emerges 00:26:00 Symbiogenesis and evolutionary complexity 00:40:00 Functionalism and consciousness 00:49:45 AI as part of collective human intelligence 00:57:00 Comparing AI and human cognition REFS: What is intelligence [Blaise Agüera y Arcas] https://whatisintelligence.antikythera.org/ [Read free online, interactive rich media] https://mitpress.mit.edu/9780262049955/what-is-intelligence/ [MIT Press] Large Language Models and Emergence: A Complex Systems Perspective https://arxiv.org/abs/2506.11135 Our first Noam Chomsky MLST interview https://www.youtube.com/watch?v=axuGfh4UR9Q Chance and Necessity [Jacques Monod] https://monoskop.org/images/9/99/Monod_Jacques_Chance_and_Necessity.pdf Wonderful Life: The Burgess Shale and the History of Nature [Stephen Jay Gould] https://www.amazon.co.uk/Wonderful-Life-Burgess-Nature-History/dp/0099273454 The major evolutionary transitions [E Szathmáry, J M Smith] https://wiki.santafe.edu/images/0/0e/Szathmary.MaynardSmith_1995_Nature.pdf Don't Sleep, There Are Snakes: Life and Language in the Amazonian Jungle [Dan Everett] https://www.amazon.com/Dont-Sleep-There-Are-Snakes/dp/0307386120 The Nature of Technology: What It Is and How It Evolves [W. Brian Arthur] https://www.amazon.com/Nature-Technology-What-How-Evolves-ebook/dp/B002RI9W16/ The MANIAC [Benjamin Labatut] https://www.amazon.com/MANIAC-Benjam%C3%ADn-Labatut/dp/1782279814 When We Cease to Understand the World [Benjamin Labatut] https://www.amazon.com/When-We-Cease-Understand-World/dp/1681375664/ The Boys in the Boat [Dan Brown] https://www.amazon.com/Boys-Boat-Americans-Berlin-Olympics/dp/0143125478 [Petter Johansson] (Split brain) https://www.lucs.lu.se/fileadmin/user_upload/lucs/2011/01/Johansson-et-al.-2006-How-Something-Can-Be-Said-About-Telling-More-Than-We-Can-Know.pdf If Anyone Builds It, Everyone Dies [Eliezer Yudkowsky, Nate Soares] https://www.amazon.com/Anyone-Builds-Everyone-Dies-Superhuman/dp/0316595640 The science of cycology https://link.springer.com/content/pdf/10.3758/bf03195929.pdf

We sat down with Sara Saab (VP of Product at Prolific) and Enzo Blindow (VP of Data and AI at Prolific) to explore the critical role of human evaluation in AI development and the challenges of aligning AI systems with human values. Prolific is a human annotation and orchestration platform for AI used by many of the major AI labs. This is a sponsored show in partnership with Prolific. **SPONSOR MESSAGES** — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — While technologists want to remove humans from the loop for speed and efficiency, these non-deterministic AI systems actually require more human oversight than ever before. Prolific's approach is to put "well-treated, verified, diversely demographic humans behind an API" - making human feedback as accessible as any other infrastructure service. When AI models like Grok 4 achieve top scores on technical benchmarks but feel awkward or problematic to use in practice, it exposes the limitations of our current evaluation methods. The guests argue that optimizing for benchmarks may actually weaken model performance in other crucial areas, like cultural sensitivity or natural conversation. We also discuss Anthropic's research showing that frontier AI models, when given goals and access to information, independently arrived at solutions involving blackmail - without any prompting toward unethical behavior. Even more concerning, the more sophisticated the model, the more susceptible it was to this "agentic misalignment." Enzo and Sarah present Prolific's "Humane" leaderboard as an alternative to existing benchmarking systems. By stratifying evaluations across diverse demographic groups, they reveal that different populations have vastly different experiences with the same AI models. Looking ahead, the guests imagine a world where humans take on coaching and teaching roles for AI systems - similar to how we might correct a child or review code. This also raises important questions about working conditions and the evolution of labor in an AI-augmented world. Rather than replacing humans entirely, we may be moving toward more sophisticated forms of human-AI collaboration. As AI tech becomes more powerful and general-purpose, the quality of human evaluation becomes more critical, not less. We need more representative evaluation frameworks that capture the messy reality of human values and cultural diversity. Visit Prolific: https://www.prolific.com/ Sara Saab (VP Product): https://uk.linkedin.com/in/sarasaab Enzo Blindow (VP Data & AI): https://uk.linkedin.com/in/enzoblindow TRANSCRIPT: https://app.rescript.info/public/share/xZ31-0kJJ_xp4zFSC-bunC8-hJNkHpbm7Lg88RFcuLE TOC: [00:00:00] Intro & Background [00:03:16] Human-in-the-Loop Challenges [00:17:19] Can AIs Understand? [00:32:02] Benchmarking & Vibes [00:51:00] Agentic Misalignment Study [01:03:00] Data Quality vs Quantity [01:16:00] Future of AI Oversight REFS: Anthropic Agentic Misalignment https://www.anthropic.com/research/agentic-misalignment Value Compass https://arxiv.org/pdf/2409.09586 Reasoning Models Don’t Always Say What They Think (Anthropic) https://www.anthropic.com/research/reasoning-models-dont-say-think https://assets.anthropic.com/m/71876fabef0f0ed4/original/reasoning_models_paper.pdf Apollo research - science of evals blog post https://www.apolloresearch.ai/blog/we-need-a-science-of-evals Leaderboard Illusion https://www.youtube.com/watch?v=9W_OhS38rIE MLST video The Leaderboard Illusion [2025] Shivalika Singh et al https://arxiv.org/abs/2504.20879 (Truncated, full list on YT)

Dr. Ilia Shumailov - Former DeepMind AI Security Researcher, now building security tools for AI agents Ever wondered what happens when AI agents start talking to each other—or worse, when they start breaking things? Ilia Shumailov spent years at DeepMind thinking about exactly these problems, and he's here to explain why securing AI is way harder than you think. **SPONSOR MESSAGES** —Check out notebooklm for your research project, it's really powerfulhttps://notebooklm.google.com/ — Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — We're racing toward a world where AI agents will handle our emails, manage our finances, and interact with sensitive data 24/7. But there is a problem. These agents are nothing like human employees. They never sleep, they can touch every endpoint in your system simultaneously, and they can generate sophisticated hacking tools in seconds. Traditional security measures designed for humans simply won't work. Dr. Ilia Shumailov https://x.com/iliaishacked https://iliaishacked.github.io/ https://sequrity.ai/ TRANSCRIPT: https://app.rescript.info/public/share/dVGsk8dz9_V0J7xMlwguByBq1HXRD6i4uC5z5r7EVGM TOC: 00:00:00 - Introduction & Trusted Third Parties via ML 00:03:45 - Background & Career Journey 00:06:42 - Safety vs Security Distinction 00:09:45 - Prompt Injection & Model Capability 00:13:00 - Agents as Worst-Case Adversaries 00:15:45 - Personal AI & CAML System Defense 00:19:30 - Agents vs Humans: Threat Modeling 00:22:30 - Calculator Analogy & Agent Behavior 00:25:00 - IMO Math Solutions & Agent Thinking 00:28:15 - Diffusion of Responsibility & Insider Threats 00:31:00 - Open Source Security Concerns 00:34:45 - Supply Chain Attacks & Trust Issues 00:39:45 - Architectural Backdoors 00:44:00 - Academic Incentives & Defense Work 00:48:30 - Semantic Censorship & Halting Problem 00:52:00 - Model Collapse: Theory & Criticism 00:59:30 - Career Advice & Ross Anderson Tribute REFS: Lessons from Defending Gemini Against Indirect Prompt Injections https://arxiv.org/abs/2505.14534 Defeating Prompt Injections by Design. Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F. https://arxiv.org/pdf/2503.18813 Agentic Misalignment: How LLMs could be insider threats https://www.anthropic.com/research/agentic-misalignment STOP ANTHROPOMORPHIZING INTERMEDIATE TOKENS AS REASONING/THINKING TRACES! Subbarao Kambhampati et al https://arxiv.org/pdf/2504.09762 Meiklejohn, S., Blauzvern, H., Maruseac, M., Schrock, S., Simon, L., & Shumailov, I. (2025). Machine learning models have a supply chain problem. https://arxiv.org/abs/2505.22778 Gao, Y., Shumailov, I., & Fawaz, K. (2025). Supply-chain attacks in machine learning frameworks. https://openreview.net/pdf?id=EH5PZW6aCr Apache Log4j Vulnerability Guidance https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance Bober-Irizar, M., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2022). Architectural backdoors in neural networks. https://arxiv.org/pdf/2206.07840 Position: Fundamental Limitations of LLM Censorship Necessitate New Approaches David Glukhov, Ilia Shumailov, ... https://proceedings.mlr.press/v235/glukhov24a.html AlphaEvolve MLST interview [Matej Balog, Alexander Novikov] https://www.youtube.com/watch?v=vC9nAosXrJw

We need AI systems to synthesise new knowledge, not just compress the data they see. Jeremy Berman, is a research scientist at Reflection AI and recent winner of the ARC-AGI v2 public leaderboard.**SPONSOR MESSAGES**—Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community!—cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economyOct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlstSubmit investment deck: https://cyber.fund/contact?utm_source=mlst— Imagine trying to teach an AI to think like a human i.e. solving puzzles that are easy for us but stump even the smartest models. Jeremy's evolutionary approach—evolving natural language descriptions instead of python code like his last version—landed him at the top with about 30% accuracy on the ARCv2.We discuss why current AIs are like "stochastic parrots" that memorize but struggle to truly reason or innovate as well as big ideas like building "knowledge trees" for real understanding, the limits of neural networks versus symbolic systems, and whether we can train models to synthesize new ideas without forgetting everything else. Jeremy Berman:https://x.com/jerber888TRANSCRIPT:https://app.rescript.info/public/share/qvCioZeZJ4Q_NlR66m-hNUZnh-qWlUJcS15Wc2OGwD0TOC:Introduction and Overview [00:00:00]ARC v1 Solution [00:07:20]Evolutionary Python Approach [00:08:00]Trade-offs in Depth vs. Breadth [00:10:33]ARC v2 Improvements [00:11:45]Natural Language Shift [00:12:35]Model Thinking Enhancements [00:13:05]Neural Networks vs. Symbolism Debate [00:14:24]Turing Completeness Discussion [00:15:24]Continual Learning Challenges [00:19:12]Reasoning and Intelligence [00:29:33]Knowledge Trees and Synthesis [00:50:15]Creativity and Invention [00:56:41]Future Directions and Closing [01:02:30]REFS:Jeremy’s 2024 article on winning ARCAGI1-pubhttps://jeremyberman.substack.com/p/how-i-got-a-record-536-on-arc-agiGetting 50% (SoTA) on ARC-AGI with GPT-4o [Greenblatt]https://blog.redwoodresearch.org/p/getting-50-sota-on-arc-agi-with-gpt https://www.youtube.com/watch?v=z9j3wB1RRGA [his MLST interview]A Thousand Brains: A New Theory of Intelligence [Hawkins]https://www.amazon.com/Thousand-Brains-New-Theory-Intelligence/dp/1541675819https://www.youtube.com/watch?v=6VQILbDqaI4 [MLST interview]Francois Chollet + Mike Knoop’s labhttps://ndea.com/On the Measure of Intelligence [Chollet]https://arxiv.org/abs/1911.01547On the Biology of a Large Language Model [Anthropic]https://transformer-circuits.pub/2025/attribution-graphs/biology.html The ARChitects [won 2024 ARC-AGI-1-private]https://www.youtube.com/watch?v=mTX_sAq--zY Connectionism critique 1998 [Fodor/Pylshyn]https://uh.edu/~garson/F&P1.PDF Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis [Kumar/Stanley]https://arxiv.org/pdf/2505.11581 AlphaEvolve interview (also program synthesis)https://www.youtube.com/watch?v=vC9nAosXrJw ShinkaEvolve: Evolving New Algorithms with LLMs, Orders of Magnitude More Efficiently [Lange et al]https://sakana.ai/shinka-evolve/ Deep learning with Python Rev 3 [Chollet] - READ CHAPTER 19 NOW!https://deeplearningwithpython.io/

Professor Andrew Wilson from NYU explains why many common-sense ideas in artificial intelligence might be wrong. For decades, the rule of thumb in machine learning has been to fear complexity. The thinking goes: if your model has too many parameters (is "too complex") for the amount of data you have, it will "overfit" by essentially memorizing the data instead of learning the underlying patterns. This leads to poor performance on new, unseen data. This is known as the classic "bias-variance trade-off" i.e. a balancing act between a model that's too simple and one that's too complex. **SPONSOR MESSAGES** — Tufa AI Labs is an AI research lab based in Zurich. **They are hiring ML research engineers!** This is a once in a lifetime opportunity to work with one of the best labs in Europe Contact Benjamin Crouzier - https://tufalabs.ai/ — Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! — cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst — Description Continued: Professor Wilson challenges this fundamental belief (fearing complexity). He makes a few surprising points: **Bigger Can Be Better**: massive models don't just get more flexible; they also develop a stronger "simplicity bias". So, if your model is overfitting, the solution might paradoxically be to make it even bigger. **The "Bias-Variance Trade-off" is a Misnomer**: Wilson claims you don't actually have to trade one for the other. You can have a model that is incredibly expressive and flexible while also being strongly biased toward simple solutions. He points to the "double descent" phenomenon, where performance first gets worse as models get more complex, but then surprisingly starts getting better again. **Honest Beliefs and Bayesian Thinking**: His core philosophy is that we should build models that honestly represent our beliefs about the world. We believe the world is complex, so our models should be expressive. But we also believe in Occam's razor—that the simplest explanation is often the best. He champions Bayesian methods, which naturally balance these two ideas through a process called marginalization, which he describes as an automatic Occam's razor. TOC: [00:00:00] Introduction and Thesis [00:04:19] Challenging Conventional Wisdom [00:11:17] The Philosophy of a Scientist-Engineer [00:16:47] Expressiveness, Overfitting, and Bias [00:28:15] Understanding, Compression, and Kolmogorov Complexity [01:05:06] The Surprising Power of Generalization [01:13:21] The Elegance of Bayesian Inference [01:33:02] The Geometry of Learning [01:46:28] Practical Advice and The Future of AI Prof. Andrew Gordon Wilson: https://x.com/andrewgwils https://cims.nyu.edu/~andrewgw/ https://scholar.google.com/citations?user=twWX2LIAAAAJ&hl=en https://www.youtube.com/watch?v=Aja0kZeWRy4 https://www.youtube.com/watch?v=HEp4TOrkwV4 TRANSCRIPT: https://app.rescript.info/public/share/H4Io1Y7Rr54MM05FuZgAv4yphoukCfkqokyzSYJwCK8 Hosts: Dr. Tim Scarfe / Dr. Keith Duggar (MIT Ph.D) REFS: Deep Learning is Not So Mysterious or Different [Andrew Gordon Wilson] https://arxiv.org/abs/2503.02113 Bayesian Deep Learning and a Probabilistic Perspective of Generalization [Andrew Gordon Wilson, Pavel Izmailov] https://arxiv.org/abs/2002.08791 Compute-Optimal LLMs Provably Generalize Better With Scale [Marc Finzi, Sanyam Kapoor, Diego Granziol, Anming Gu, Christopher De Sa, J. Zico Kolter, Andrew Gordon Wilson] https://arxiv.org/abs/2504.15208

In this episode, hosts Tim and Keith finally realize their long-held dream of sitting down with their hero, the brilliant neuroscientist Professor Karl Friston. The conversation is a fascinating and mind-bending journey into Professor Friston's life's work, the Free Energy Principle, and what it reveals about life, intelligence, and consciousness itself. **SPONSORS** Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal - https://github.com/google-gemini/gemini-cli --- Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! --- cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst *** They kick things off by looking back on the 20-year journey of the Free Energy Principle. Professor Friston explains it as a fundamental rule for survival: all living things, from a single cell to a human being, are constantly trying to make sense of the world and reduce unpredictability. It’s this drive to minimize surprise that allows things to exist and maintain their structure. This leads to a bigger question: What does it truly mean to be "intelligent"? The group debates whether intelligence is everywhere, even in a virus or a plant, or if it requires a certain level of complexity. Professor Friston introduces the idea of different "kinds" of things, suggesting that creatures like us, who can model themselves and think about the future, possess a unique and "strange" kind of agency that sets us apart. From intelligence, the discussion naturally flows to the even trickier concept of consciousness. Is it the same as intelligence? Professor Friston argues they are different. He explains that consciousness might emerge from deep, layered self-awareness—not just acting, but understanding that you are the one causing your actions and thinking about your place in the world. They also explore intelligence at different sizes. Is a corporation intelligent? What about the entire planet? Professor Friston suggests there might be a "Goldilocks zone" for intelligence. It doesn't seem to exist at the super-tiny atomic level or at the massive scale of planets and solar systems, but thrives in the complex middle-ground where we live. Finally, they tackle one of the most pressing topics of our time: Can we build a truly conscious AI? Professor Friston shares his doubts about whether our current computers are capable of a feat like that. He suggests that genuine consciousness might require a different kind of "mortal" computation, where the machine's physical body and its "mind" are inseparable, much like in biological creatures. TRANSCRIPT: https://app.rescript.info/public/share/FZkF8BO7HMt9aFfu2_q69WGT_ZbYZ1VVkC6RtU3eeOI TOC: 00:00:00: Introduction & Retrospective on the Free Energy Principle 00:09:34: Strange Particles, Agency, and Consciousness 00:37:45: The Scale of Intelligence: From Viruses to the Biosphere 01:01:35: Modelling, Boundaries, and Practical Application 01:21:12: Conclusion

We are joined by Cristopher Moore, a professor at the Santa Fe Institute with a diverse background in physics, computer science, and machine learning. The conversation begins with Cristopher, who calls himself a "frog" explaining that he prefers to dive deep into specific, concrete problems rather than taking a high-level "bird's-eye view". They explore why current AI models, like transformers, are so surprisingly effective. Cristopher argues it's because the real world isn't random; it's full of rich structures, patterns, and hierarchies that these models can learn to exploit, even if we don't fully understand how. **SPONSORS** Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community! --- cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy. Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++ Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst Submit investment deck: https://cyber.fund/contact?utm_source=mlst *** Cristopher Moore: https://sites.santafe.edu/~moore/ TOC: 00:00:00 - Introduction 00:02:05 - Meet Christopher Moore: A Frog in the World of Science 00:05:14 - The Limits of Transformers and Real-World Data 00:11:19 - Intelligence as Creative Problem-Solving 00:23:30 - Grounding, Meaning, and Shared Reality 00:31:09 - The Nature of Creativity and Aesthetics 00:44:31 - Computational Irreducibility and Universality 00:53:06 - Turing Completeness, Recursion, and Intelligence 01:11:26 - The Universe Through a Computational Lens 01:26:45 - Algorithmic Justice and the Need for Transparency TRANSCRIPT: https://app.rescript.info/public/share/VRe2uQSvKZOm0oIBoDsrNwt46OMCqRnShVnUF3qyoFk Filmed at DISI (Diverse Intelligences Summer Institute) https://disi.org/ REFS: The Nature of computation [Chris Moore] https://nature-of-computation.org/ Birds and Frogs [Freeman Dyson] https://www.ams.org/notices/200902/rtx090200212p.pdf Replica Theory [Parisi et al] https://arxiv.org/pdf/1409.2722 Janossy pooling [Fabian Fuchs] https://fabianfuchsml.github.io/equilibriumaggregation/ Cracking the cryptic [YT channel] https://www.youtube.com/c/CrackingTheCryptic Sudoko Bench [Sakana] https://sakana.ai/sudoku-bench/ Fractured entangled representations “phylogenetic locking in comment” [Kumar/Stanley] https://arxiv.org/pdf/2505.11581 (see our shows on this) The War Against Cliché: [Martin Amis] https://www.amazon.com/War-Against-Cliche-Reviews-1971-2000/dp/0375727167 Rule 110 (CA) https://mathworld.wolfram.com/Rule150.html Universality in Elementary Cellular Automata [Matt Cooke] https://wpmedia.wolfram.com/sites/13/2018/02/15-1-1.pdf Small Semi-Weakly Universal Turing Machines [Damien Woods] https://tilde.ini.uzh.ch/users/tneary/public_html/WoodsNeary-FI09.pdf COMPUTING MACHINERY AND INTELLIGENCE [Turing, 1950] https://courses.cs.umbc.edu/471/papers/turing.pdf Comment on Space Time as a causal set [Moore, 88] https://sites.santafe.edu/~moore/comment.pdf Recursion Theory on the Reals and Continuous-time Computation [Moore, 96]

Dr. Michael Timothy Bennett is a computer scientist who's deeply interested in understanding artificial intelligence, consciousness, and what it means to be alive. He's known for his provocative paper "What the F*** is Artificial Intelligence" which challenges conventional thinking about AI and intelligence.**SPONSOR MESSAGES***Prolific: Quality data. From real people. For faster breakthroughs.https://prolific.com/mlst?utm_campaign=98404559-MLST&utm_source=youtube&utm_medium=podcast&utm_content=mb***Michael takes us on a journey through some of the biggest questions in AI and consciousness. He starts by exploring what intelligence actually is - settling on the idea that it's about "adaptation with limited resources" (a definition from researcher Pei Wang that he particularly likes).The discussion ranges from technical AI concepts to philosophical questions about consciousness, with Michael offering fresh perspectives that challenge Silicon Valley's "just scale it up" approach to AI. He argues that true intelligence isn't just about having more parameters or data - it's about being able to adapt efficiently, like biological systems do.TOC:1. Introduction & Paper Overview [00:01:34]2. Definitions of Intelligence [00:02:54]3. Formal Models (AIXI, Active Inference) [00:07:06]4. Causality, Abstraction & Embodiment [00:10:45]5. Computational Dualism & Mortal Computation [00:25:51]6. Modern AI, AGI Progress & Benchmarks [00:31:30]7. Hybrid AI Approaches [00:35:00]8. Consciousness & The Hard Problem [00:39:35]9. The Diverse Intelligences Summer Institute (DISI) [00:53:20]10. Living Systems & Self-Organization [00:54:17]11. Closing Thoughts [01:04:24]Michaels socials:https://michaeltimothybennett.com/https://x.com/MiTiBennettTranscript:https://app.rescript.info/public/share/4jSKbcM77Sf6Zn-Ms4hda7C4krRrMcQt0qwYqiqPTPIReferences:Bennett, M.T. "What the F*** is Artificial Intelligence"https://arxiv.org/abs/2503.23923Bennett, M.T. "Are Biological Systems More Intelligent Than Artificial Intelligence?" https://arxiv.org/abs/2405.02325Bennett, M.T. PhD Thesis "How To Build Conscious Machines"https://osf.io/preprints/thesiscommons/wehmg_v1Legg, S. & Hutter, M. (2007). "Universal Intelligence: A Definition of Machine Intelligence"Wang, P. "Defining Artificial Intelligence" - on non-axiomatic reasoning systems (NARS)Chollet, F. (2019). "On the Measure of Intelligence" - introduces the ARC benchmark and developer-aware generalizationHutter, M. (2005). "Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability"Chalmers, D. "The Hard Problem of Consciousness"Descartes, R. - Cartesian dualism and the pineal gland theory (historical context)Friston, K. - Free Energy Principle and Active Inference frameworkLevin, M. - Work on collective intelligence, cancer as information isolation, and "mind blindness"Hinton, G. (2022). "The Forward-Forward Algorithm" - introduces mortal computation conceptAlexander Ororbia & Friston - Formal treatment of mortal computationSutton, R. "The Bitter Lesson" - on search and learning in AIPearl, J. "The Book of Why" - causal inference and reasoningAlternative AGI ApproachesWang, P. - NARS (Non-Axiomatic Reasoning System)Goertzel, B. - Hyperon system and modular AGI architecturesBenchmarks & EvaluationHendrycks, D. - Humanities Last Exam benchmark (mentioned re: saturation)Filmed at:Diverse Intelligences Summer Institute (DISI) https://disi.org/

Deep dive with Dan Hendrycks, a leading AI safety researcher and co-author of the "Superintelligence Strategy" paper with former Google CEO Eric Schmidt and Scale AI CEO Alexandr Wang. *** SPONSOR MESSAGES Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal - https://github.com/google-gemini/gemini-cli Prolific: Quality data. From real people. For faster breakthroughs. https://prolific.com/mlst?utm_campaign=98404559-MLST&utm_source=youtube&utm_medium=podcast&utm_content=script-gen *** Hendrycks argues that society is making a fundamental mistake in how it views artificial intelligence. We often compare AI to transformative but ultimately manageable technologies like electricity or the internet. He contends a far better and more realistic analogy is nuclear technology. Like nuclear power, AI has the potential for immense good, but it is also a dual-use technology that carries the risk of unprecedented catastrophe. The Problem with an AI "Manhattan Project": A popular idea is for the U.S. to launch a "Manhattan Project" for AI—a secret, all-out government race to build a superintelligence before rivals like China. Hendrycks argues this strategy is deeply flawed and dangerous for several reasons: - It wouldn’t be secret. You cannot hide a massive, heat-generating data center from satellite surveillance. - It would be destabilizing. A public race would alarm rivals, causing them to start their own desperate, corner-cutting projects, dramatically increasing global risk. - It’s vulnerable to sabotage. An AI project can be crippled in many ways, from cyberattacks that poison its training data to physical attacks on its power plants. This is what the paper refers to as a "maiming attack." This vulnerability leads to the paper's central concept: Mutual Assured AI Malfunction (MAIM). This is the AI-era version of the nuclear-era's Mutual Assured Destruction (MAD). In this dynamic, any nation that makes an aggressive, destabilizing bid for a world-dominating AI must expect its rivals to sabotage the project to ensure their own survival. This deterrence, Hendrycks argues, is already the default reality we live in. A Better Strategy: The Three Pillars Instead of a reckless race, the paper proposes a more stable, three-part strategy modeled on Cold War principles: - Deterrence: Acknowledge the reality of MAIM. The goal should not be to "win" the race to superintelligence, but to deter anyone from starting such a race in the first place through the credible threat of sabotage. - Nonproliferation: Just as we work to keep fissile materials for nuclear bombs out of the hands of terrorists and rogue states, we must control the key inputs for catastrophic AI. The most critical input is advanced AI chips (GPUs). Hendrycks makes the powerful claim that building cutting-edge GPUs is now more difficult than enriching uranium, making this strategy viable. - Competitiveness: The race between nations like the U.S. and China should not be about who builds superintelligence first. Instead, it should be about who can best use existing AI to build a stronger economy, a more effective military, and more resilient supply chains (for example, by manufacturing more chips domestically). Dan says the stakes are high if we fail to manage this transition: - Erosion of Control - Intelligence Recursion - Worthless Labor Hendrycks maintains that while the risks are existential, the future is not set. TOC: 1 Measuring the Beast [00:00:00] 2 Defining the Beast [00:11:34] 3 The Core Strategy [00:38:20] 4 Ideological Battlegrounds [00:53:12] 5 Mechanisms of Control [01:34:45] TRANSCRIPT: https://app.rescript.info/public/share/cOKcz4pWRPjh7BTIgybd7PUr_vChUaY6VQW64No8XMs

This episode features Shlomi Fuchter and Jack Parker Holder from Google DeepMind, who are unveiling a new AI called Genie 3. The host, Tim Scarfe, describes it as the most mind-blowing technology he has ever seen. We were invited to their offices to conduct the interview (not sponsored).Imagine you could create a video game world just by describing it. That's what Genie 3 does. It's an AI "world model" that learns how the real world works by watching massive amounts of video. Unlike a normal video game engine (like Unreal or the one for Doom) that needs to be programmed manually, Genie generates a realistic, interactive, 3D world from a simple text prompt.**SPONSOR MESSAGES***Prolific: Quality data. From real people. For faster breakthroughs.https://prolific.com/mlst?utm_campaign=98404559-MLST&utm_source=youtube&utm_medium=podcast&utm_content=script-gen***Here’s a breakdown of what makes it so revolutionary:From Text to a Virtual World: You can type "a drone flying by a beautiful lake" or "a ski slope," and Genie 3 creates that world for you in about three seconds. You can then navigate and interact with it in real-time.It's Consistent: The worlds it creates have a reliable memory. If you look away from an object and then look back, it will still be there, just as it was. The guests explain that this consistency isn't explicitly programmed in; it's a surprising, "emergent" capability of the powerful AI model.A Huge Leap Forward: The previous version, Genie 2, was a major step, but it wasn't fast enough for real-time interaction and was much lower resolution. Genie 3 is 720p, interactive, and photorealistic, running smoothly for several minutes at a time.The Killer App - Training Robots: Beyond entertainment, the team sees Genie 3 as a game-changer for training AI. Instead of training a self-driving car or a robot in the real world (which is slow and dangerous), you can create infinite simulations. You can even prompt rare events to happen, like a deer running across the road, to teach an AI how to handle unexpected situations safely.The Future of Entertainment: this could lead to a "YouTube version 2" or a new form of VR, where users can create and explore endless, interconnected worlds together, like the experience machine from philosophy.While the technology is still a research prototype and not yet available to the public, it represents a monumental step towards creating true artificial worlds from the ground up.Jack Parker Holder [Research Scientist at Google DeepMind in the Open-Endedness Team]https://jparkerholder.github.io/Shlomi Fruchter [Research Director, Google DeepMind]https://shlomifruchter.github.io/TOC:[00:00:00] - Introduction: "The Most Mind-Blowing Technology I've Ever Seen"[00:02:30] - The Evolution from Genie 1 to Genie 2[00:04:30] - Enter Genie 3: Photorealistic, Interactive Worlds from Text[00:07:00] - Promptable World Events & Training Self-Driving Cars[00:14:21] - Guest Introductions: Shlomi Fuchter & Jack Parker Holder[00:15:08] - Core Concepts: What is a "World Model"?[00:19:30] - The Challenge of Consistency in a Generated World[00:21:15] - Context: The Neural Network Doom Simulation[00:25:25] - How Do You Measure the Quality of a World Model?[00:28:09] - The Vision: Using Genie to Train Advanced Robots[00:32:21] - Open-Endedness: Human Skill and Prompting Creativity[00:38:15] - The Future: Is This the Next YouTube or VR?[00:42:18] - The Next Step: Multi-Agent Simulations[00:52:51] - Limitations: Thinking, Computation, and the Sim-to-Real Gap[00:58:07] - Conclusion & The Future of Game EnginesREFS:World Models [David Ha, Jürgen Schmidhuber]https://arxiv.org/abs/1803.10122POEThttps://arxiv.org/abs/1901.01753[Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley]The Fractured Entangled Representation Hypothesishttps://arxiv.org/pdf/2505.11581TRANSCRIPT:https://app.rescript.info/public/share/Zk5tZXk6mb06yYOFh6nSja7Lg6_qZkgkuXQ-kl5AJqM

Prof. David Krakauer, President of the Santa Fe Institute argues that we are fundamentally confusing knowledge with intelligence, especially when it comes to AI. He defines true intelligence as the ability to do more with less—to solve novel problems with limited information. This is contrasted with current AI models, which he describes as doing less with more; they require astounding amounts of data to perform tasks that don't necessarily demonstrate true understanding or adaptation. He humorously calls this "really shit programming". David challenges the popular notion of "emergence" in Large Language Models (LLMs). He explains that the tech community's definition—seeing a sudden jump in a model's ability to perform a task like three-digit math—is superficial. True emergence, from a complex systems perspective, involves a fundamental change in the system's internal organization, allowing for a new, simpler, and more powerful level of description. He gives the example of moving from tracking individual water molecules to using the elegant laws of fluid dynamics. For LLMs to be truly emergent, we'd need to see them develop new, efficient internal representations, not just get better at memorizing patterns as they scale. Drawing on his background in evolutionary theory, David explains that systems like brains, and later, culture, evolved to process information that changes too quickly for genetic evolution to keep up. He calls culture "evolution at light speed" because it allows us to store our accumulated knowledge externally (in books, tools, etc.) and build upon it without corrupting the original. This leads to his concept of "exbodiment," where we outsource our cognitive load to the world through things like maps, abacuses, or even language itself. We create these external tools, internalize the skills they teach us, improve them, and create a feedback loop that enhances our collective intelligence. However, he ends with a warning. While technology has historically complemented our deficient abilities, modern AI presents a new danger. Because we have an evolutionary drive to conserve energy, we will inevitably outsource our thinking to AI if we can. He fears this is already leading to a "diminution and dilution" of human thought and creativity. Just as our muscles atrophy without use, he argues our brains will too, and we risk becoming mentally dependent on these systems. TOC: [00:00:00] Intelligence: Doing more with less [00:02:10] Why brains evolved: The limits of evolution [00:05:18] Culture as evolution at light speed [00:08:11] True meaning of emergence: "More is Different" [00:10:41] Why LLM capabilities are not true emergence [00:15:10] What real emergence would look like in AI [00:19:24] Symmetry breaking: Physics vs. Life [00:23:30] Two types of emergence: Knowledge In vs. Out [00:26:46] Causality, agency, and coarse-graining [00:32:24] "Exbodiment": Outsourcing thought to objects [00:35:05] Collective intelligence & the boundary of the mind [00:39:45] Mortal vs. Immortal forms of computation [00:42:13] The risk of AI: Atrophy of human thought David Krakauer President and William H. Miller Professor of Complex Systems https://www.santafe.edu/people/profile/david-krakauer REFS: Large Language Models and Emergence: A Complex Systems Perspective David C. Krakauer, John W. Krakauer, Melanie Mitchell https://arxiv.org/abs/2506.11135 Filmed at the Diverse Intelligences Summer Institute: https://disi.org/

Dr. Maxwell Ramstead grills Guillaume Verdon (AKA “Beff Jezos”) who's the founder of Thermodynamic computing startup Extropic. Guillaume shares his unique path – from dreaming about space travel as a kid to becoming a physicist, then working on quantum computing at Google, to developing a radically new form of computing hardware for machine learning. He explains how he hit roadblocks with traditional physics and computing, leading him to start his company – building "thermodynamic computers." These are based on a new design for super-efficient chips that use the natural chaos of electrons (think noise and heat) to power AI tasks, which promises to speed up AND lower the costs of modern probabilistic techniques like sampling. He is driven by the pursuit of building computers that work more like your brain, which (by the way) runs on a banana and a glass of water!  Guillaume talks about his alter ego, Beff Jezos, and the "Effective Accelerationism" (e/acc) movement that he initiated. Its objective is to speed up tech progress in order to “grow civilization” (as measured by energy use and innovation), rather than “slowing down out of fear”. Guillaume argues we need to embrace variance, exploration, and optimism to avoid getting stuck or outpaced by competitors like China. He and Maxwell discuss big ideas like merging humans with AI, decentralizing intelligence, and why boundless growth (with smart constraints) is “key to humanity's future”. REFS: 1. John Archibald Wheeler - "It From Bit" Concept 00:04:45 - Foundational work proposing that physical reality emerges from information at the quantum level Learn more: https://cqi.inf.usi.ch/qic/wheeler.pdf  2. AdS/CFT Correspondence (Holographic Principle) 00:05:15 - Theoretical physics duality connecting quantum gravity in Anti-de Sitter space with conformal field theory https://en.wikipedia.org/wiki/Holographic_principle  3. Renormalization Group Theory 00:06:15 - Mathematical framework for analyzing physical systems across different length scales https://www.damtp.cam.ac.uk/user/dbs26/AQFT/Wilsonchap.pdf  4. Maxwell's Demon and Information Theory 00:21:15 - Thought experiment linking information processing to thermodynamics and entropy https://plato.stanford.edu/entries/information-entropy/  5. Landauer's Principle 00:29:45 - Fundamental limit establishing minimum energy required for information erasure https://en.wikipedia.org/wiki/Landauer%27s_principle  6. Free Energy Principle and Active Inference 01:03:00 - Mathematical framework for understanding self-organizing systems and perception-action loops https://www.nature.com/articles/nrn2787  7. Max Tegmark - Information Bottleneck Principle 01:07:00 - Connections between information theory and renormalization in machine learning https://arxiv.org/abs/1907.07331  8. Fisher's Fundamental Theorem of Natural Selection 01:11:45 - Mathematical relationship between genetic variance and evolutionary fitness https://en.wikipedia.org/wiki/Fisher%27s_fundamental_theorem_of_natural_selection  9. Tensor Networks in Quantum Systems 00:06:45 - Computational framework for simulating many-body quantum systems https://arxiv.org/abs/1912.10049  10. Quantum Neural Networks 00:09:30 - Hybrid quantum-classical models for machine learning applications https://en.wikipedia.org/wiki/Quantum_neural_network  11. Energy-Based Models (EBMs) 00:40:00 - Probabilistic framework for unsupervised learning based on energy functions https://www.researchgate.net/publication/200744586_A_tutorial_on_energy-based_learning  12. Markov Chain Monte Carlo (MCMC) 00:20:00 - Sampling algorithm fundamental to modern AI and statistical physics https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo  13. Metropolis-Hastings Algorithm 00:23:00 - Core sampling method for probability distributions https://arxiv.org/abs/1504.01896 ***SPONSOR MESSAGE*** Google Gemini 2.5 Flash is a state-of-the-art language model in the Gemini app. Sign up at https://gemini.google.com

Are the AI models you use today imposters? Please watch the intro video we did before this: https://www.youtube.com/watch?v=o1q6Hhz0MAg In this episode, hosts Dr. Tim Scarfe and Dr. Duggar are joined by AI researcher Prof. Kenneth Stanley and MIT PhD student Akash Kumar to discuss their fascinating paper, "Questioning Representational Optimism in Deep Learning." Imagine you ask two people to draw a perfect skull. One is a brilliant artist who understands anatomy, the other is a machine that just traces the image. Both drawings look identical, but the artist understands what a skull is—they know where the mouth is, how the jaw works, and that it's symmetrical. The machine just has a tangled mess of lines that happens to form the right picture. An AI with an elegant representation, has the building blocks to generate truly new ideas. The Path Is the Goal: As Kenneth Stanley puts it, "it matters not just where you get, but how you got there". Two students can ace a math test, but the one who truly understands the concepts—instead of just memorizing formulas—is the one who will go on to make new discoveries. The show is a mixture of 3 separate recordings we have done, the original Patreon warmup with Tim/Kenneth, the Tim/Keith "Steakhouse" recorded after the main interview, then the main interview with Kenneth/Akarsh/Keith/Tim. Feel free to skip around. We had to edit this in a rush as we are travelling next week but it's reasonably cleaned up. TOC: 00:00:00 Intro: Garbage vs. Amazing Representations 00:05:42 How Good Representations Form 00:11:14 Challenging the "Bitter Lesson" 00:18:04 AI Creativity & Representation Types 00:22:13 Steakhouse: Critiques & Alternatives 00:28:30 Steakhouse: Key Concepts & Goldilocks Zone 00:39:42 Steakhouse: A Sober View on AI Risk 00:43:46 Steakhouse: The Paradox of Open-Ended Search 00:47:58 Main Interview: Paper Intro & Core Concepts 00:56:44 Main Interview: Deception and Evolvability 01:36:30 Main Interview: Reinterpreting Evolution 01:56:16 Main Interview: Impostor Intelligence 02:11:15 Main Interview: Recommendations for AI Research REFS: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley https://arxiv.org/pdf/2505.11581 Kenneth O. Stanley, Joel Lehman Why Greatness Cannot Be Planned: The Myth of the Objective https://amzn.to/44xLaXK Original show with Kenneth from 4 years ago: https://www.youtube.com/watch?v=lhYGXYeMq_E Kenneth Stanley is SVP Open Endedness at Lila Sciences https://x.com/kenneth0stanley Akarsh Kumar (MIT) https://akarshkumar.com/ AND... Kenneth is HIRING (this is an OPPORTUNITY OF A LIFETIME!) Research Engineer: https://job-boards.greenhouse.io/lila/jobs/7890007002 Research Scientist: https://job-boards.greenhouse.io/lila/jobs/8012245002 TRANSCRIPT: https://app.rescript.info/public/share/W_T7E1OC2Wj49ccqlIOOztg2MJWaaVbovTeyxcFEQdU

What if today's incredible AI is just a brilliant "impostor"? This episode features host Dr. Tim Scarfe in conversation with guests Prof. Kenneth Stanley (ex-OpenAI), Dr. Keith Duggar (MIT), and Arkash Kumar (MIT).While AI today produces amazing results on the surface, its internal understanding is a complete mess, described as "total spaghetti" [00:00:49]. This is because it's trained with a brute-force method (SGD) that’s like building a sandcastle: it looks right from a distance, but has no real structure holding it together [00:01:45].To explain the difference, Keith Duggar shares a great analogy about his high school physics classes [00:03:18]. One class was about memorizing lots of formulas for specific situations (like the "impostor" AI). The other used calculus to derive the answers from a deeper understanding, which was much easier and more powerful. This is the core difference: one method memorizes, the other truly understands.The episode then introduces a different, more powerful way to build AI, based on Kenneth Stanley's old experiment, "Picbreeder" [00:04:45]. This method creates AI with a shockingly clean and intuitive internal model of the world. For example, it might develop a model of a skull where it understands the "mouth" as a separate component it can open and close, without ever being explicitly trained on that action [00:06:15]. This deep understanding emerges bottom-up, without massive datasets.The secret is to abandon a fixed goal and embrace "deception" [00:08:42]—the idea that the stepping stones to a great discovery often don't look anything like the final result. Instead of optimizing for a target, the AI is built through an open-ended process of exploring what's "interesting" [00:09:15]. This creates a more flexible and adaptable foundation, a bit like how evolvability wins out in nature [00:10:30].The show concludes by arguing that this choice matters immensely. The "impostor" path may be hitting a wall, requiring insane amounts of money and energy for progress and failing to deliver true creativity or continual learning [00:13:00]. The ultimate message is a call to not put all our eggs in one basket [00:14:25]. We should explore these open-ended, creative paths to discover a more genuine form of intelligence, which may be found where we least expect it.REFS:Questioning Representational Optimism in Deep Learning:The Fractured Entangled Representation HypothesisAkarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanleyhttps://arxiv.org/pdf/2505.11581Kenneth O. Stanley, Joel LehmanWhy Greatness Cannot Be Planned: The Myth of the Objectivehttps://amzn.to/44xLaXKOriginal show with Kenneth from 4 years ago:https://www.youtube.com/watch?v=lhYGXYeMq_EKenneth Stanley is SVP Open Endedness at Lila Scienceshttps://x.com/kenneth0stanleyAkarsh Kumar (MIT)https://akarshkumar.com/AND... Kenneth is HIRING (this is an OPPORTUNITY OF A LIFETIME!)Research Engineer: https://job-boards.greenhouse.io/lila/jobs/7890007002Research Scientist: https://job-boards.greenhouse.io/lila/jobs/8012245002Tim's Code visualisation of FER based on Akarsh repo: https://github.com/ecsplendid/ferTRANSCRIPT: https://app.rescript.info/public/share/YKAZzZ6lwZkjTLRpVJreOOxGhLI8y4m3fAyU8NSavx0

What if the most powerful technology in human history is being built by people who openly admit they don't trust each other? In this explosive 2-hour debate, three AI experts pull back the curtain on the shocking psychology driving the race to Artificial General Intelligence—and why the people building it might be the biggest threat of all. Kokotajlo predicts AGI by 2028 based on compute scaling trends. Marcus argues we haven't solved basic cognitive problems from his 2001 research. The stakes? If Kokotajlo is right and Marcus is wrong about safety progress, humanity may have already lost control. Sponsor messages: ======== Google Gemini: Google Gemini features Veo3, a state-of-the-art AI video generation model in the Gemini app. Sign up at https://gemini.google.com Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard! https://tufalabs.ai/ ======== Guest Powerhouse Gary Marcus - Cognitive scientist, author of "Taming Silicon Valley," and AI's most prominent skeptic who's been warning about the same fundamental problems for 25 years (https://garymarcus.substack.com/) Daniel Kokotajlo - Former OpenAI insider turned whistleblower who reveals the disturbing rationalizations of AI lab leaders in his viral "AI 2027" scenario (https://ai-2027.com/) Dan Hendrycks - Director of the Center for AI Safety who created the benchmarks used to measure AI progress and argues we have only years, not decades, to prevent catastrophe (https://danhendrycks.com/) Transcript: http://app.rescript.info/public/share/tEcx4UkToi-2jwS1cN51CW70A4Eh6QulBRxDILoXOno TOC: Introduction: The AI Arms Race 00:00:04 - The Danger of Automated AI R&D 00:00:43 - The Rationalization: "If we don't, someone else will" 00:01:56 - Sponsor Reads (Tufa AI Labs & Google Gemini) 00:02:55 - Guest Introductions The Philosophical Stakes 00:04:13 - What is the Positive Vision for AGI? 00:07:00 - The Abundance Scenario: Superintelligent Economy 00:09:06 - Differentiating AGI and Superintelligence (ASI) 00:11:41 - Sam Altman: "A Decade in a Month" 00:14:47 - Economic Inequality & The UBI Problem Policy and Red Lines 00:17:13 - The Pause Letter: Stopping vs. Delaying AI 00:20:03 - Defining Three Concrete Red Lines for AI Development 00:25:24 - Racing Towards Red Lines & The Myth of "Durable Advantage" 00:31:15 - Transparency and Public Perception 00:35:16 - The Rationalization Cascade: Why AI Labs Race to "Win" Forecasting AGI: Timelines and Methodologies 00:42:29 - The Case for Short Timelines (Median 2028) 00:47:00 - Scaling Limits: Compute, Data, and Money 00:49:36 - Forecasting Models: Bio-Anchors and Agentic Coding 00:53:15 - The 10^45 FLOP Thought Experiment The Great Debate: Cognitive Gaps vs. Scaling 00:58:41 - Gary Marcus's Counterpoint: The Unsolved Problems of Cognition 01:00:46 - Current AI Can't Play Chess Reliably 01:08:23 - Can Tools and Neurosymbolic AI Fill the Gaps? 01:16:13 - The Multi-Dimensional Nature of Intelligence 01:24:26 - The Benchmark Debate: Data Contamination and Reliability 01:31:15 - The Superhuman Coder Milestone Debate 01:37:45 - The Driverless Car Analogy The Alignment Problem 01:39:45 - Has Any Progress Been Made on Alignment? 01:42:43 - "Fairly Reasonably Scares the Sh*t Out of Me" 01:46:30 - Distinguishing Model vs. Process Alignment Scenarios and Conclusions 01:49:26 - Gary's Alternative Scenario: The Neurosymbolic Shift 01:53:35 - Will AI Become Jeff Dean? 01:58:41 - Takeoff Speeds and Exceeding Human Intelligence 02:03:19 - Final Disagreements and Closing Remarks REFS: Gary Marcus (2001) - The Algebraic Mind https://mitpress.mit.edu/9780262632683/the-algebraic-mind/ 00:59:00 Gary Marcus & Ernest Davis (2019) - Rebooting AI https://www.penguinrandomhouse.com/books/566677/rebooting-ai-by-gary-marcus-and-ernest-davis/ 01:31:59 Gary Marcus (2024) - Taming SV https://www.hachettebookgroup.com/titles/gary-marcus/taming-silicon-valley/9781541704091/ 00:03:01

We interview Professor Christopher Summerfield from Oxford University about his new book "These Strange New Minds: How AI Learned to Talk and What It". AI learned to understand the world just by reading text - something scientists thought was impossible. You don't need to see a cat to know what one is; you can learn everything from words alone. This is "the most astonishing scientific discovery of the 21st century."People are split: some refuse to call what AI does "thinking" even when it outperforms humans, while others believe if it acts intelligent, it is intelligent. Summerfield takes the middle ground - AI does something genuinely like human reasoning, but that doesn't make it human.Sponsor messages:========Google Gemini: Google Gemini features Veo3, a state-of-the-art AI video generation model in the Gemini app. Sign up at https://gemini.google.comTufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard! https://tufalabs.ai/========Prof. Christopher Summerfieldhttps://www.psy.ox.ac.uk/people/christopher-summerfieldThese Strange New Minds: How AI Learned to Talk and What It Meanshttps://amzn.to/4e26BVaTable of Contents:Introduction & Setup00:00:00 Superman 3 Metaphor - Humans Absorbed by Machines00:02:01 Book Introduction & AI Debate Context00:03:45 Sponsor Segments (Google Gemini, Tufa Labs)Philosophical Foundations00:04:48 The Fractured AI Discourse00:08:21 Ancient Roots: Aristotle vs Plato (Empiricism vs Rationalism)00:10:14 Historical AI: Symbolic Logic and Its LimitsThe Language Revolution00:12:11 ChatGPT as the Rubicon Moment00:14:00 The Astonishing Discovery: Learning Reality from Words Alone00:15:47 Equivalentists vs Exceptionalists DebateCognitive Science Perspectives00:19:12 Functionalism and the Duck Test00:21:48 Brain-AI Similarities and Computational Principles00:24:53 Reconciling Chomsky: Evolution vs Learning00:28:15 Lamarckian AI vs Darwinian Human LearningThe Reality of AI Capabilities00:30:29 Anthropomorphism and the Clever Hans Effect00:32:56 The Intentional Stance and Nature of Thinking00:37:56 Three Major AI Worries: Agency, Personalization, DynamicsSocietal Risks and Complex Systems00:37:56 AI Agents and Flash Crash Scenarios00:42:50 Removing Frictions: The Lawfare Example00:46:15 Gradual Disempowerment Theory00:49:18 The Faustian Pact of TechnologyHuman Agency and Control00:51:18 The Crisis of Authenticity00:56:22 Psychology of Control vs Reward01:00:21 Dopamine Hacking and Variable ReinforcementFuture Directions01:02:27 Evolution as Goal-less Optimization01:03:31 Open-Endedness and Creative Evolution01:06:46 Writing, Creativity, and AI-Generated Content01:08:18 Closing RemarksREFS:Academic References (Abbreviated)Essential Books"These Strange New Minds" - C. Summerfield [00:02:01] - Main discussion topic"The Mind is Flat" - N. Chater [00:33:45] - Summerfield's favorite on cognitive illusions"AI: A Guide for Thinking Humans" - M. Mitchell [00:04:58] - Host's previous favorite"Principia Mathematica" - Russell & Whitehead [00:11:00] - Logic Theorist reference"Syntactic Structures" - N. Chomsky (1957) [00:13:30] - Generative grammar foundation"Why Greatness Cannot Be Planned" - Stanley & Lehman [01:04:00] - Open-ended evolutionKey Papers & Studies"Gradual Disempowerment" - D. Duvenaud [00:46:45] - AI threat model"Counterfeit People" - D. Dennett (Atlantic) [00:52:45] - AI societal risks"Open-Endedness is Essential..." - DeepMind/Rocktäschel/Hughes [01:03:42]Heider & Simmel (1944) [00:30:45] - Agency attribution to shapesWhitehall Studies - M. Marmot [00:59:32] - Control and health outcomes"Clever Hans" - O. Pfungst (1911) [00:31:47] - Animal intelligence illusionHistorical References

"Blurring Reality" - Chai's Social AI Platform - sponsored This episode of MLST explores the groundbreaking work of Chai, a social AI platform that quietly built one of the world's largest AI companion ecosystems before ChatGPT's mainstream adoption. With over 10 million active users and just 13 engineers serving 2 trillion tokens per day, Chai discovered the massive appetite for AI companionship through serendipity while searching for product-market fit. CHAI sponsored this show *because they want to hire amazing engineers* -- CAREER OPPORTUNITIES AT CHAI Chai is actively hiring in Palo Alto with competitive compensation ($300K-$800K+ equity) for roles including AI Infrastructure Engineers, Software Engineers, Applied AI Researchers, and more. Fast-track qualification available for candidates with significant product launches, open source contributions, or entrepreneurial success. https://www.chai-research.com/jobs/ The conversation with founder William Beauchamp and engineers Tom Lu and Nischay Dhankhar covers Chai's innovative technical approaches including reinforcement learning from human feedback (RLHF), model blending techniques that combine smaller models to outperform larger ones, and their unique infrastructure challenges running exaflop-class compute. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers in Zurich and SF. Goto https://tufalabs.ai/ *** Key themes explored include: - The ethics of AI engagement optimization and attention hacking - Content moderation at scale with a lean engineering team - The shift from AI as utility tool to AI as social companion - How users form deep emotional bonds with artificial intelligence - The broader implications of AI becoming a social medium We also examine OpenAI's recent pivot toward companion AI with April's new GPT-4o, suggesting a fundamental shift in how we interact with artificial intelligence - from utility-focused tools to companion-like experiences that blur the lines between human and artificial intimacy. The episode also covers Chai's unconventional approach to hiring only top-tier engineers, their bootstrap funding strategy focused on user revenue over VC funding, and their rapid experimentation culture where one in five experiments succeed. TOC: 00:00:00 - Introduction: Steve Jobs' AI Vision & Chai's Scale 00:04:02 - Chapter 1: Simulators - The Birth of Social AI 00:13:34 - Chapter 2: Engineering at Chai - RLHF & Model Blending 00:21:49 - Chapter 3: Social Impact of GenAI - Ethics & Safety 00:33:55 - Chapter 4: The Lean Machine - 13 Engineers, Millions of Users 00:42:38 - Chapter 5: GPT-4o Becoming a Companion - OpenAI's Pivot 00:50:10 - Chapter 6: What Comes Next - The Future of AI Intimacy TRANSCRIPT: https://www.dropbox.com/scl/fi/yz2ewkzmwz9rbbturfbap/CHAI.pdf?rlkey=uuyk2nfhjzezucwdgntg5ubqb&dl=0

Today GoogleDeepMind released AlphaEvolve: a Gemini coding agent for algorithm discovery. It beat the famous Strassen algorithm for matrix multiplication set 56 years ago. Google has been killing it recently. We had early access to the paper and interviewed the researchers behind the work. AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ Authors: Alexander Novikov*, Ngân Vũ*, Marvin Eisenberger*, Emilien Dupont*, Po-Sen Huang*, Adam Zsolt Wagner*, Sergey Shirobokov*, Borislav Kozlovskii*, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, Matej Balog* (* indicates equal contribution or special designation, if defined elsewhere) SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** AlphaEvolve works like a very smart, tireless programmer. It uses powerful AI language models (like Gemini) to generate ideas for computer code. Then, it uses an "evolutionary" process – like survival of the fittest for programs. It tries out many different program ideas, automatically tests how well they solve a problem, and then uses the best ones to inspire new, even better programs. Beyond this mathematical breakthrough, AlphaEvolve has already been used to improve real-world systems at Google, such as making their massive data centers run more efficiently and even speeding up the training of the AI models that power AlphaEvolve itself. The discussion also covers how humans work with AlphaEvolve, the challenges of making AI discover things, and the exciting future of AI helping scientists make new discoveries. In short, AlphaEvolve is a powerful new AI tool that can invent new algorithms and solve complex problems, showing how AI can be a creative partner in science and engineering. Guests: Matej Balog: https://x.com/matejbalog Alexander Novikov: https://x.com/SashaVNovikov REFS: MAP Elites [Jean-Baptiste Mouret, Jeff Clune] https://arxiv.org/abs/1504.04909 FunSearch [Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli & Alhussein Fawzi] https://www.nature.com/articles/s41586-023-06924-6 TOC: [00:00:00] Introduction: Alpha Evolve's Breakthroughs, DeepMind's Lineage, and Real-World Impact [00:12:06] Introducing AlphaEvolve: Concept, Evolutionary Algorithms, and Architecture [00:16:56] Search Challenges: The Halting Problem and Enabling Creative Leaps [00:23:20] Knowledge Augmentation: Self-Generated Data, Meta-Prompting, and Library Learning [00:29:08] Matrix Multiplication Breakthrough: From Strassen to AlphaEvolve's 48 Multiplications [00:39:11] Problem Representation: Direct Solutions, Constructors, and Search Algorithms [00:46:06] Developer Reflections: Surprising Outcomes and Superiority over Simple LLM Sampling [00:51:42] Algorithmic Improvement: Hill Climbing, Program Synthesis, and Intelligibility [01:00:24] Real-World Application: Complex Evaluations and Robotics [01:05:39] Role of LLMs & Future: Advanced Models, Recursive Self-Improvement, and Human-AI Collaboration [01:11:22] Resource Considerations: Compute Costs of AlphaEvolve This is a trial of posting videos on Spotify, thoughts? Email me or chat in our Discord

Randall Balestriero joins the show to discuss some counterintuitive findings in AI. He shares research showing that huge language models, even when started from scratch (randomly initialized) without massive pre-training, can learn specific tasks like sentiment analysis surprisingly well, train stably, and avoid severe overfitting, sometimes matching the performance of costly pre-trained models. This raises questions about when giant pre-training efforts are truly worth it. He also talks about how self-supervised learning (where models learn from data structure itself) and traditional supervised learning (using labeled data) are fundamentally similar, allowing researchers to apply decades of supervised learning theory to improve newer self-supervised methods. Finally, Randall touches on fairness in AI models used for Earth data (like climate prediction), revealing that these models can be biased, performing poorly in specific locations like islands or coastlines even if they seem accurate overall, which has important implications for policy decisions based on this data. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT + SHOWNOTES: https://www.dropbox.com/scl/fi/n7yev71nsjso71jyjz1fy/RANDALLNEURIPS.pdf?rlkey=0dn4injp1sc4ts8njwf3wfmxv&dl=0 TOC: 1. Model Training Efficiency and Scale [00:00:00] 1.1 Training Stability of Large Models on Small Datasets [00:04:09] 1.2 Pre-training vs Random Initialization Performance Comparison [00:07:58] 1.3 Task-Specific Models vs General LLMs Efficiency 2. Learning Paradigms and Data Distribution [00:10:35] 2.1 Fair Language Model Paradox and Token Frequency Issues [00:12:02] 2.2 Pre-training vs Single-task Learning Spectrum [00:16:04] 2.3 Theoretical Equivalence of Supervised and Self-supervised Learning [00:19:40] 2.4 Self-Supervised Learning and Supervised Learning Relationships [00:21:25] 2.5 SSL Objectives and Heavy-tailed Data Distribution Challenges 3. Geographic Representation in ML Systems [00:25:20] 3.1 Geographic Bias in Earth Data Models and Neural Representations [00:28:10] 3.2 Mathematical Limitations and Model Improvements [00:30:24] 3.3 Data Quality and Geographic Bias in ML Datasets REFS: [00:01:40] Research on training large language models from scratch on small datasets, Randall Balestriero et al. https://openreview.net/forum?id=wYGBWOjq1Q [00:10:35] The Fair Language Model Paradox (2024), Andrea Pinto, Tomer Galanti, Randall Balestriero https://arxiv.org/abs/2410.11985 [00:12:20] Muppet: Massive Multi-task Representations with Pre-Finetuning (2021), Armen Aghajanyan et al. https://arxiv.org/abs/2101.11038 [00:14:30] Dissociating language and thought in large language models (2023), Kyle Mahowald et al. https://arxiv.org/abs/2301.06627 [00:16:05] The Birth of Self-Supervised Learning: A Supervised Theory, Randall Balestriero et al. https://openreview.net/forum?id=NhYAjAAdQT [00:21:25] VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Adrien Bardes, Jean Ponce, Yann LeCun https://arxiv.org/abs/2105.04906 [00:25:20] No Location Left Behind: Measuring and Improving the Fairness of Implicit Representations for Earth Data (2025), Daniel Cai, Randall Balestriero, et al. https://arxiv.org/abs/2502.06831 [00:33:45] Mark Ibrahim et al.'s work on geographic bias in computer vision datasets, Mark Ibrahim https://arxiv.org/pdf/2304.12210

Prof. Kevin Ellis and Dr. Zenna Tavares talk about making AI smarter, like humans. They want AI to learn from just a little bit of information by actively trying things out, not just by looking at tons of data. They discuss two main ways AI can "think": one way is like following specific rules or steps (like a computer program), and the other is more intuitive, like guessing based on patterns (like modern AI often does). They found combining both methods works well for solving complex puzzles like ARC. A key idea is "compositionality" - building big ideas from small ones, like LEGOs. This is powerful but can also be overwhelming. Another important idea is "abstraction" - understanding things simply, without getting lost in details, and knowing there are different levels of understanding. Ultimately, they believe the best AI will need to explore, experiment, and build models of the world, much like humans do when learning something new. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT: https://www.dropbox.com/scl/fi/3ngggvhb3tnemw879er5y/BASIS.pdf?rlkey=lr2zbj3317mex1q5l0c2rsk0h&dl=0 Zenna Tavares: http://www.zenna.org/ Kevin Ellis: https://www.cs.cornell.edu/~ellisk/ TOC: 1. Compositionality and Learning Foundations [00:00:00] 1.1 Compositional Search and Learning Challenges [00:03:55] 1.2 Bayesian Learning and World Models [00:12:05] 1.3 Programming Languages and Compositionality Trade-offs [00:15:35] 1.4 Inductive vs Transductive Approaches in AI Systems 2. Neural-Symbolic Program Synthesis [00:27:20] 2.1 Integration of LLMs with Traditional Programming and Meta-Programming [00:30:43] 2.2 Wake-Sleep Learning and DreamCoder Architecture [00:38:26] 2.3 Program Synthesis from Interactions and Hidden State Inference [00:41:36] 2.4 Abstraction Mechanisms and Resource Rationality [00:48:38] 2.5 Inductive Biases and Causal Abstraction in AI Systems 3. Abstract Reasoning Systems [00:52:10] 3.1 Abstract Concepts and Grid-Based Transformations in ARC [00:56:08] 3.2 Induction vs Transduction Approaches in Abstract Reasoning [00:59:12] 3.3 ARC Limitations and Interactive Learning Extensions [01:06:30] 3.4 Wake-Sleep Program Learning and Hybrid Approaches [01:11:37] 3.5 Project MARA and Future Research Directions REFS: [00:00:25] DreamCoder, Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [00:01:10] Mind Your Step, Ryan Liu et al. https://arxiv.org/abs/2410.21333 [00:06:05] Bayesian inference, Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. https://psycnet.apa.org/record/2008-06911-003 [00:13:00] Induction and Transduction, Wen-Ding Li, Zenna Tavares, Yewen Pu, Kevin Ellis https://arxiv.org/abs/2411.02272 [00:23:15] Neurosymbolic AI, Garcez, Artur d'Avila et al. https://arxiv.org/abs/2012.05876 [00:33:50] Induction and Transduction (II), Wen-Ding Li, Kevin Ellis et al. https://arxiv.org/abs/2411.02272 [00:38:35] ARC, François Chollet https://arxiv.org/abs/1911.01547 [00:39:20] Causal Reactive Programs, Ria Das, Joshua B. Tenenbaum, Armando Solar-Lezama, Zenna Tavares http://www.zenna.org/publications/autumn2022.pdf [00:42:50] MuZero, Julian Schrittwieser et al. http://arxiv.org/pdf/1911.08265 [00:43:20] VisualPredicator, Yichao Liang https://arxiv.org/abs/2410.23156 [00:48:55] Bayesian models of cognition, Joshua B. Tenenbaum https://mitpress.mit.edu/9780262049412/bayesian-models-of-cognition/ [00:49:30] The Bitter Lesson, Rich Sutton http://www.incompleteideas.net/IncIdeas/BitterLesson.html [01:06:35] Program induction, Kevin Ellis, Wen-Ding Li https://arxiv.org/pdf/2411.02272 [01:06:50] DreamCoder (II), Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [01:11:55] Project MARA, Zenna Tavares, Kevin Ellis https://www.basis.ai/blog/mara/

Eiso Kant, CTO of poolside AI, discusses the company's approach to building frontier AI foundation models, particularly focused on software development. Their unique strategy is reinforcement learning from code execution feedback which is an important axis for scaling AI capabilities beyond just increasing model size or data volume. Kant predicts human-level AI in knowledge work could be achieved within 18-36 months, outlining poolside's vision to dramatically increase software development productivity and accessibility. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** Eiso Kant: https://x.com/eisokant https://poolside.ai/ TRANSCRIPT: https://www.dropbox.com/scl/fi/szepl6taqziyqie9wgmk9/poolside.pdf?rlkey=iqar7dcwshyrpeoz0xa76k422&dl=0 TOC: 1. Foundation Models and AI Strategy [00:00:00] 1.1 Foundation Models and Timeline Predictions for AI Development [00:02:55] 1.2 Poolside AI's Corporate History and Strategic Vision [00:06:48] 1.3 Foundation Models vs Enterprise Customization Trade-offs 2. Reinforcement Learning and Model Economics [00:15:42] 2.1 Reinforcement Learning and Code Execution Feedback Approaches [00:22:06] 2.2 Model Economics and Experimental Optimization 3. Enterprise AI Implementation [00:25:20] 3.1 Poolside's Enterprise Deployment Strategy and Infrastructure [00:26:00] 3.2 Enterprise-First Business Model and Market Focus [00:27:05] 3.3 Foundation Models and AGI Development Approach [00:29:24] 3.4 DeepSeek Case Study and Infrastructure Requirements 4. LLM Architecture and Performance [00:30:15] 4.1 Distributed Training and Hardware Architecture Optimization [00:33:01] 4.2 Model Scaling Strategies and Chinchilla Optimality Trade-offs [00:36:04] 4.3 Emergent Reasoning and Model Architecture Comparisons [00:43:26] 4.4 Balancing Creativity and Determinism in AI Models [00:50:01] 4.5 AI-Assisted Software Development Evolution 5. AI Systems Engineering and Scalability [00:58:31] 5.1 Enterprise AI Productivity and Implementation Challenges [00:58:40] 5.2 Low-Code Solutions and Enterprise Hiring Trends [01:01:25] 5.3 Distributed Systems and Engineering Complexity [01:01:50] 5.4 GenAI Architecture and Scalability Patterns [01:01:55] 5.5 Scaling Limitations and Architectural Patterns in AI Code Generation 6. AI Safety and Future Capabilities [01:06:23] 6.1 Semantic Understanding and Language Model Reasoning Approaches [01:12:42] 6.2 Model Interpretability and Safety Considerations in AI Systems [01:16:27] 6.3 AI vs Human Capabilities in Software Development [01:33:45] 6.4 Enterprise Deployment and Security Architecture CORE REFS (see shownotes for URLs/more refs): [00:15:45] Research demonstrating how training on model-generated content leads to distribution collapse in AI models, Ilia Shumailov et al. (Key finding on synthetic data risk) [00:20:05] Foundational paper introducing Word2Vec for computing word vector representations, Tomas Mikolov et al. (Seminal NLP technique) [00:22:15] OpenAI O3 model's breakthrough performance on ARC Prize Challenge, OpenAI (Significant AI reasoning benchmark achievement) [00:22:40] Seminal paper proposing a formal definition of intelligence as skill-acquisition efficiency, François Chollet (Influential AI definition/philosophy) [00:30:30] Technical documentation of DeepSeek's V3 model architecture and capabilities, DeepSeek AI (Details on a major new model) [00:34:30] Foundational paper establishing optimal scaling laws for LLM training, Jordan Hoffmann et al. (Key paper on LLM scaling) [00:45:45] Seminal essay arguing that scaling computation consistently trumps human-engineered solutions in AI, Richard S. Sutton (Influential "Bitter Lesson" perspective)

Connor Leahy and Gabriel Alfour, AI researchers from Conjecture and authors of "The Compendium," joinus for a critical discussion centered on Artificial Superintelligence (ASI) safety and governance. Drawing from their comprehensive analysis in "The Compendium," they articulate a stark warning about the existential risks inherent in uncontrolled AI development, framing it through the lens of "intelligence domination"—where a sufficiently advanced AI could subordinate humanity, much like humans dominate less intelligent species. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT + REFS + NOTES: https://www.dropbox.com/scl/fi/p86l75y4o2ii40df5t7no/Compendium.pdf?rlkey=tukczgf3flw133sr9rgss0pnj&dl=0 https://www.thecompendium.ai/ https://en.wikipedia.org/wiki/Connor_Leahy https://www.conjecture.dev/about https://substack.com/@gabecc​ TOC: 1. AI Intelligence and Safety Fundamentals [00:00:00] 1.1 Understanding Intelligence and AI Capabilities [00:06:20] 1.2 Emergence of Intelligence and Regulatory Challenges [00:10:18] 1.3 Human vs Animal Intelligence Debate [00:18:00] 1.4 AI Regulation and Risk Assessment Approaches [00:26:14] 1.5 Competing AI Development Ideologies 2. Economic and Social Impact [00:29:10] 2.1 Labor Market Disruption and Post-Scarcity Scenarios [00:32:40] 2.2 Institutional Frameworks and Tech Power Dynamics [00:37:40] 2.3 Ethical Frameworks and AI Governance Debates [00:40:52] 2.4 AI Alignment Evolution and Technical Challenges 3. Technical Governance Framework [00:55:07] 3.1 Three Levels of AI Safety: Alignment, Corrigibility, and Boundedness [00:55:30] 3.2 Challenges of AI System Corrigibility and Constitutional Models [00:57:35] 3.3 Limitations of Current Boundedness Approaches [00:59:11] 3.4 Abstract Governance Concepts and Policy Solutions 4. Democratic Implementation and Coordination [00:59:20] 4.1 Governance Design and Measurement Challenges [01:00:10] 4.2 Democratic Institutions and Experimental Governance [01:14:10] 4.3 Political Engagement and AI Safety Advocacy [01:25:30] 4.4 Practical AI Safety Measures and International Coordination CORE REFS: [00:01:45] The Compendium (2023), Leahy et al. https://pdf.thecompendium.ai/the_compendium.pdf [00:06:50] Geoffrey Hinton Leaves Google, BBC News https://www.bbc.com/news/world-us-canada-65452940 [00:10:00] ARC-AGI, Chollet https://arcprize.org/arc-agi [00:13:25] A Brief History of Intelligence, Bennett https://www.amazon.com/Brief-History-Intelligence-Humans-Breakthroughs/dp/0063286343 [00:25:35] Statement on AI Risk, Center for AI Safety https://www.safe.ai/work/statement-on-ai-risk [00:26:15] Machines of Love and Grace, Amodei https://darioamodei.com/machines-of-loving-grace [00:26:35] The Techno-Optimist Manifesto, Andreessen https://a16z.com/the-techno-optimist-manifesto/ [00:31:55] Techno-Feudalism, Varoufakis https://www.amazon.co.uk/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1847927270 [00:42:40] Introducing Superalignment, OpenAI https://openai.com/index/introducing-superalignment/ [00:47:20] Three Laws of Robotics, Asimov https://www.britannica.com/topic/Three-Laws-of-Robotics [00:50:00] Symbolic AI (GOFAI), Haugeland https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence [00:52:30] Intent Alignment, Christiano https://www.alignmentforum.org/posts/HEZgGBZTpT4Bov7nH/mapping-the-conceptual-territory-in-ai-existential-safety [00:55:10] Large Language Model Alignment: A Survey, Jiang et al. http://arxiv.org/pdf/2309.15025 [00:55:40] Constitutional Checks and Balances, Bok https://plato.stanford.edu/entries/montesquieu/

We are joined by Francois Chollet and Mike Knoop, to launch the new version of the ARC prize! In version 2, the challenges have been calibrated with humans such that at least 2 humans could solve each task in a reasonable task, but also adversarially selected so that frontier reasoning models can't solve them. The best LLMs today get negligible performance on this challenge. https://arcprize.org/ SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT: https://www.dropbox.com/scl/fi/0v9o8xcpppdwnkntj59oi/ARCv2.pdf?rlkey=luqb6f141976vra6zdtptv5uj&dl=0 TOC: 1. ARC v2 Core Design & Objectives [00:00:00] 1.1 ARC v2 Launch and Benchmark Architecture [00:03:16] 1.2 Test-Time Optimization and AGI Assessment [00:06:24] 1.3 Human-AI Capability Analysis [00:13:02] 1.4 OpenAI o3 Initial Performance Results 2. ARC Technical Evolution [00:17:20] 2.1 ARC-v1 to ARC-v2 Design Improvements [00:21:12] 2.2 Human Validation Methodology [00:26:05] 2.3 Task Design and Gaming Prevention [00:29:11] 2.4 Intelligence Measurement Framework 3. O3 Performance & Future Challenges [00:38:50] 3.1 O3 Comprehensive Performance Analysis [00:43:40] 3.2 System Limitations and Failure Modes [00:49:30] 3.3 Program Synthesis Applications [00:53:00] 3.4 Future Development Roadmap REFS: [00:00:15] On the Measure of Intelligence, François Chollet https://arxiv.org/abs/1911.01547 [00:06:45] ARC Prize Foundation, François Chollet, Mike Knoop https://arcprize.org/ [00:12:50] OpenAI o3 model performance on ARC v1, ARC Prize Team https://arcprize.org/blog/oai-o3-pub-breakthrough [00:18:30] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Jason Wei et al. https://arxiv.org/abs/2201.11903 [00:21:45] ARC-v2 benchmark tasks, Mike Knoop https://arcprize.org/blog/introducing-arc-agi-public-leaderboard [00:26:05] ARC Prize 2024: Technical Report, Francois Chollet et al. https://arxiv.org/html/2412.04604v2 [00:32:45] ARC Prize 2024 Technical Report, Francois Chollet, Mike Knoop, Gregory Kamradt https://arxiv.org/abs/2412.04604 [00:48:55] The Bitter Lesson, Rich Sutton http://www.incompleteideas.net/IncIdeas/BitterLesson.html [00:53:30] Decoding strategies in neural text generation, Sina Zarrieß https://www.mdpi.com/2078-2489/12/9/355/pdf

Mohamed Osman joins to discuss MindsAI's highest scoring entry to the ARC challenge 2024 and the paradigm of test-time fine-tuning. They explore how the team, now part of Tufa Labs in Zurich, achieved state-of-the-art results using a combination of pre-training techniques, a unique meta-learning strategy, and an ensemble voting mechanism. Mohamed emphasizes the importance of raw data input and flexibility of the network. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT + REFS: https://www.dropbox.com/scl/fi/jeavyqidsjzjgjgd7ns7h/MoFInal.pdf?rlkey=cjjmo7rgtenxrr3b46nk6yq2e&dl=0 Mohamed Osman (Tufa Labs) https://x.com/MohamedOsmanML Jack Cole (Tufa Labs) https://x.com/MindsAI_Jack How and why deep learning for ARC paper: https://github.com/MohamedOsman1998/deep-learning-for-arc/blob/main/deep_learning_for_arc.pdf TOC: 1. Abstract Reasoning Foundations [00:00:00] 1.1 Test-Time Fine-Tuning and ARC Challenge Overview [00:10:20] 1.2 Neural Networks vs Programmatic Approaches to Reasoning [00:13:23] 1.3 Code-Based Learning and Meta-Model Architecture [00:20:26] 1.4 Technical Implementation with Long T5 Model 2. ARC Solution Architectures [00:24:10] 2.1 Test-Time Tuning and Voting Methods for ARC Solutions [00:27:54] 2.2 Model Generalization and Function Generation Challenges [00:32:53] 2.3 Input Representation and VLM Limitations [00:36:21] 2.4 Architecture Innovation and Cross-Modal Integration [00:40:05] 2.5 Future of ARC Challenge and Program Synthesis Approaches 3. Advanced Systems Integration [00:43:00] 3.1 DreamCoder Evolution and LLM Integration [00:50:07] 3.2 MindsAI Team Progress and Acquisition by Tufa Labs [00:54:15] 3.3 ARC v2 Development and Performance Scaling [00:58:22] 3.4 Intelligence Benchmarks and Transformer Limitations [01:01:50] 3.5 Neural Architecture Optimization and Processing Distribution REFS: [00:01:32] Original ARC challenge paper, François Chollet https://arxiv.org/abs/1911.01547 [00:06:55] DreamCoder, Kevin Ellis et al. https://arxiv.org/abs/2006.08381 [00:12:50] Deep Learning with Python, François Chollet https://www.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438 [00:13:35] Deep Learning with Python, François Chollet https://www.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438 [00:13:35] Influence of pretraining data for reasoning, Laura Ruis https://arxiv.org/abs/2411.12580 [00:17:50] Latent Program Networks, Clement Bonnet https://arxiv.org/html/2411.08706v1 [00:20:50] T5, Colin Raffel et al. https://arxiv.org/abs/1910.10683 [00:30:30] Combining Induction and Transduction for Abstract Reasoning, Wen-Ding Li, Kevin Ellis et al. https://arxiv.org/abs/2411.02272 [00:34:15] Six finger problem, Chen et al. https://openaccess.thecvf.com/content/CVPR2024/papers/Chen_SpatialVLM_Endowing_Vision-Language_Models_with_Spatial_Reasoning_Capabilities_CVPR_2024_paper.pdf [00:38:15] DeepSeek-R1-Distill-Llama, DeepSeek AI https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B [00:40:10] ARC Prize 2024 Technical Report, François Chollet et al. https://arxiv.org/html/2412.04604v2 [00:45:20] LLM-Guided Compositional Program Synthesis, Wen-Ding Li and Kevin Ellis https://arxiv.org/html/2503.15540 [00:54:25] Abstraction and Reasoning Corpus, François Chollet https://github.com/fchollet/ARC-AGI [00:57:10] O3 breakthrough on ARC-AGI, OpenAI https://arcprize.org/ [00:59:35] ConceptARC Benchmark, Arseny Moskvichev, Melanie Mitchell https://arxiv.org/abs/2305.07141 [01:02:05] Mixtape: Breaking the Softmax Bottleneck Efficiently, Yang, Zhilin and Dai, Zihang and Salakhutdinov, Ruslan and Cohen, William W. http://papers.neurips.cc/paper/9723-mixtape-breaking-the-softmax-bottleneck-efficiently.pdf

Iman Mirzadeh from Apple, who recently published the GSM-Symbolic paper discusses the crucial distinction between intelligence and achievement in AI systems. He critiques current AI research methodologies, highlighting the limitations of Large Language Models (LLMs) in reasoning and knowledge representation. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT + RESEARCH: https://www.dropbox.com/scl/fi/mlcjl9cd5p1kem4l0vqd3/IMAN.pdf?rlkey=dqfqb74zr81a5gqr8r6c8isg3&dl=0 TOC: 1. Intelligence vs Achievement in AI Systems [00:00:00] 1.1 Intelligence vs Achievement Metrics in AI Systems [00:03:27] 1.2 AlphaZero and Abstract Understanding in Chess [00:10:10] 1.3 Language Models and Distribution Learning Limitations [00:14:47] 1.4 Research Methodology and Theoretical Frameworks 2. Intelligence Measurement and Learning [00:24:24] 2.1 LLM Capabilities: Interpolation vs True Reasoning [00:29:00] 2.2 Intelligence Definition and Measurement Approaches [00:34:35] 2.3 Learning Capabilities and Agency in AI Systems [00:39:26] 2.4 Abstract Reasoning and Symbol Understanding 3. LLM Performance and Evaluation [00:47:15] 3.1 Scaling Laws and Fundamental Limitations [00:54:33] 3.2 Connectionism vs Symbolism Debate in Neural Networks [00:58:09] 3.3 GSM-Symbolic: Testing Mathematical Reasoning in LLMs [01:08:38] 3.4 Benchmark Evaluation and Model Performance Assessment REFS: [00:01:00] AlphaZero chess AI system, Silver et al. https://arxiv.org/abs/1712.01815 [00:07:10] Game Changer: AlphaZero's Groundbreaking Chess Strategies, Sadler & Regan https://www.amazon.com/Game-Changer-AlphaZeros-Groundbreaking-Strategies/dp/9056918184 [00:11:35] Cross-entropy loss in language modeling, Voita http://lena-voita.github.io/nlp_course/language_modeling.html [00:17:20] GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in LLMs, Mirzadeh et al. https://arxiv.org/abs/2410.05229 [00:21:25] Connectionism and Cognitive Architecture: A Critical Analysis, Fodor & Pylyshyn https://www.sciencedirect.com/science/article/pii/001002779090014B [00:28:55] Brain-to-body mass ratio scaling laws, Sutskever https://www.theverge.com/2024/12/13/24320811/what-ilya-sutskever-sees-openai-model-data-training [00:29:40] On the Measure of Intelligence, Chollet https://arxiv.org/abs/1911.01547 [00:33:30] On definition of intelligence, Gignac et al. https://www.sciencedirect.com/science/article/pii/S0160289624000266 [00:35:30] Defining intelligence, Wang https://cis.temple.edu/~wangp/papers.html [00:37:40] How We Learn: Why Brains Learn Better Than Any Machine... for Now, Dehaene https://www.amazon.com/How-We-Learn-Brains-Machine/dp/0525559884 [00:39:35] Surfaces and Essences: Analogy as the Fuel and Fire of Thinking, Hofstadter and Sander https://www.amazon.com/Surfaces-Essences-Analogy-Fuel-Thinking/dp/0465018475 [00:43:15] Chain-of-thought prompting, Wei et al. https://arxiv.org/abs/2201.11903 [00:47:20] Test-time scaling laws in machine learning, Brown https://podcasts.apple.com/mv/podcast/openais-noam-brown-ilge-akkaya-and-hunter-lightman-on/id1750736528?i=1000671532058 [00:47:50] Scaling Laws for Neural Language Models, Kaplan et al. https://arxiv.org/abs/2001.08361 [00:55:15] Tensor product variable binding, Smolensky https://www.sciencedirect.com/science/article/abs/pii/000437029090007M [01:08:45] GSM-8K dataset, OpenAI https://huggingface.co/datasets/openai/gsm8k

Dr. Max Bartolo from Cohere discusses machine learning model development, evaluation, and robustness. Key topics include model reasoning, the DynaBench platform for dynamic benchmarking, data-centric AI development, model training challenges, and the limitations of human feedback mechanisms. The conversation also covers technical aspects like influence functions, model quantization, and the PRISM project. Max Bartolo (Cohere): https://www.maxbartolo.com/ https://cohere.com/command TRANSCRIPT: https://www.dropbox.com/scl/fi/vujxscaffw37pqgb6hpie/MAXB.pdf?rlkey=0oqjxs5u49eqa2m7uaol64lbw&dl=0 TOC: 1. Model Reasoning and Verification [00:00:00] 1.1 Model Consistency and Reasoning Verification [00:03:25] 1.2 Influence Functions and Distributed Knowledge Analysis [00:10:28] 1.3 AI Application Development and Model Deployment [00:14:24] 1.4 AI Alignment and Human Feedback Limitations 2. Evaluation and Bias Assessment [00:20:15] 2.1 Human Evaluation Challenges and Factuality Assessment [00:27:15] 2.2 Cultural and Demographic Influences on Model Behavior [00:32:43] 2.3 Adversarial Examples and Model Robustness 3. Benchmarking Systems and Methods [00:41:54] 3.1 DynaBench and Dynamic Benchmarking Approaches [00:50:02] 3.2 Benchmarking Challenges and Alternative Metrics [00:50:33] 3.3 Evolution of Model Benchmarking Methods [00:51:15] 3.4 Hierarchical Capability Testing Framework [00:52:35] 3.5 Benchmark Platforms and Tools 4. Model Architecture and Performance [00:55:15] 4.1 Cohere's Model Development Process [01:00:26] 4.2 Model Quantization and Performance Evaluation [01:05:18] 4.3 Reasoning Capabilities and Benchmark Standards [01:08:27] 4.4 Training Progression and Technical Challenges 5. Future Directions and Challenges [01:13:48] 5.1 Context Window Evolution and Trade-offs [01:22:47] 5.2 Enterprise Applications and Future Challenges REFS: [00:03:10] Research at Cohere with Laura Ruis et al., Max Bartolo, Laura Ruis et al. https://cohere.com/research/papers/procedural-knowledge-in-pretraining-drives-reasoning-in-large-language-models-2024-11-20 [00:04:15] Influence functions in machine learning, Koh & Liang https://arxiv.org/abs/1703.04730 [00:08:05] Studying Large Language Model Generalization with Influence Functions, Roger Grosse et al. https://storage.prod.researchhub.com/uploads/papers/2023/08/08/2308.03296.pdf [00:11:10] The LLM ARChitect: Solving ARC-AGI Is A Matter of Perspective, Daniel Franzen, Jan Disselhoff, and David Hartmann https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf [00:12:10] Hugging Face model repo for C4AI Command A, Cohere and Cohere For AI https://huggingface.co/CohereForAI/c4ai-command-a-03-2025 [00:13:30] OpenInterpreter https://github.com/KillianLucas/open-interpreter [00:16:15] Human Feedback is not Gold Standard, Tom Hosking, Max Bartolo, Phil Blunsom https://arxiv.org/abs/2309.16349 [00:27:15] The PRISM Alignment Dataset, Hannah Kirk et al. https://arxiv.org/abs/2404.16019 [00:32:50] How adversarial examples arise, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, Aleksander Madry https://arxiv.org/abs/1905.02175 [00:43:00] DynaBench platform paper, Douwe Kiela et al. https://aclanthology.org/2021.naacl-main.324.pdf [00:50:15] Sara Hooker's work on compute limitations, Sara Hooker https://arxiv.org/html/2407.05694v1 [00:53:25] DataPerf: Community-led benchmark suite, Mazumder et al. https://arxiv.org/abs/2207.10062 [01:04:35] DROP, Dheeru Dua et al. https://arxiv.org/abs/1903.00161 [01:07:05] GSM8k, Cobbe et al. https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k [01:09:30] ARC, François Chollet https://github.com/fchollet/ARC-AGI [01:15:50] Command A, Cohere https://cohere.com/blog/command-a [01:22:55] Enterprise search using LLMs, Cohere https://cohere.com/blog/commonly-asked-questions-about-search-from-coheres-enterprise-customers

This sponsored episode features mathematician Ohad Asor discussing logical approaches to AI, focusing on the limitations of machine learning and introducing the Tau language for software development and blockchain tech. Asor argues that machine learning cannot guarantee correctness. Tau allows logical specification of software requirements, automatically creating provably correct implementations with potential to revolutionize distributed systems. The discussion highlights program synthesis, software updates, and applications in finance and governance.SPONSOR MESSAGES:***Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/***TRANSCRIPT + RESEARCH:https://www.dropbox.com/scl/fi/t849j6v1juk3gc15g4rsy/TAU.pdf?rlkey=hh11h2mhog3ncdbeapbzpzctc&dl=0Tau:https://tau.net/Tau Language:https://tau.ai/tau-language/Research:https://tau.net/Theories-and-Applications-of-Boolean-Algebras-0.29.pdfTOC:1. Machine Learning Foundations and Limitations [00:00:00] 1.1 Fundamental Limitations of Machine Learning and PAC Learning Theory [00:04:50] 1.2 Transductive Learning and the Three Curses of Machine Learning [00:08:57] 1.3 Language, Reality, and AI System Design [00:12:58] 1.4 Program Synthesis and Formal Verification Approaches2. Logical Programming Architecture [00:31:55] 2.1 Safe AI Development Requirements [00:32:05] 2.2 Self-Referential Language Architecture [00:32:50] 2.3 Boolean Algebra and Logical Foundations [00:37:52] 2.4 SAT Solvers and Complexity Challenges [00:44:30] 2.5 Program Synthesis and Specification [00:47:39] 2.6 Overcoming Tarski's Undefinability with Boolean Algebra [00:56:05] 2.7 Tau Language Implementation and User Control3. Blockchain-Based Software Governance [01:09:10] 3.1 User Control and Software Governance Mechanisms [01:18:27] 3.2 Tau's Blockchain Architecture and Meta-Programming Capabilities [01:21:43] 3.3 Development Status and Token Implementation [01:24:52] 3.4 Consensus Building and Opinion Mapping System [01:35:29] 3.5 Automation and Financial ApplicationsCORE REFS (more in pinned comment):[00:03:45] PAC (Probably Approximately Correct) Learning framework, Leslie Valianthttps://en.wikipedia.org/wiki/Probably_approximately_correct_learning[00:06:10] Boolean Satisfiability Problem (SAT), Varioushttps://en.wikipedia.org/wiki/Boolean_satisfiability_problem[00:13:55] Knowledge as Justified True Belief (JTB), Matthias Steuphttps://plato.stanford.edu/entries/epistemology/[00:17:50] Wittgenstein's concept of the limits of language, Ludwig Wittgensteinhttps://plato.stanford.edu/entries/wittgenstein/[00:21:25] Boolean algebras, Ohad Osorhttps://tau.net/tau-language-research/[00:26:10] The Halting Problemhttps://plato.stanford.edu/entries/turing-machine/#HaltProb[00:30:25] Alfred Tarski (1901-1983), Mario Gómez-Torrentehttps://plato.stanford.edu/entries/tarski/[00:41:50] DPLLhttps://www.cs.princeton.edu/~zkincaid/courses/fall18/readings/SATHandbook-CDCL.pdf[00:49:50] Tarski's undefinability theorem (1936), Alfred Tarskihttps://plato.stanford.edu/entries/tarski-truth/[00:51:45] Boolean Algebra mathematical foundations, J. Donald Monkhttps://plato.stanford.edu/entries/boolalg-math/[01:02:35] Belief Revision Theory and AGM Postulates, Sven Ove Hanssonhttps://plato.stanford.edu/entries/logic-belief-revision/[01:05:35] Quantifier elimination in atomless boolean algebra, H. Jerome Keislerhttps://people.math.wisc.edu/~hkeisler/random.pdf[01:08:35] Quantifier elimination in Tau language specification, Ohad Asorhttps://tau.ai/Theories-and-Applications-of-Boolean-Algebras-0.29.pdf[01:11:50] Tau Net blockchain platformhttps://tau.net/[01:19:20] Tau blockchain's innovative approach treating blockchain code itself as a contracthttps://tau.net/Whitepaper.pdf

John Palazza from CentML joins us in this sponsored interview to discuss the critical importance of infrastructure optimization in the age of Large Language Models and Generative AI. We explore how enterprises can transition from the innovation phase to production and scale, highlighting the significance of efficient GPU utilization and cost management. The conversation covers the open-source versus proprietary model debate, the rise of AI agents, and the need for platform independence to avoid vendor lock-in, as well as emerging trends in AI infrastructure and the pivotal role of strategic partnerships. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT: https://www.dropbox.com/scl/fi/dnjsygrgdgq5ng5fdlfjg/JOHNPALAZZA.pdf?rlkey=hl9wyydi9mj077rbg5acdmo3a&dl=0 John Palazza: Vice President of Global Sales @ CentML https://www.linkedin.com/in/john-p-b34655/ TOC: 1. Enterprise AI Organization and Strategy [00:00:00] 1.1 Organizational Structure and ML Ownership [00:02:59] 1.2 Infrastructure Efficiency and GPU Utilization [00:07:59] 1.3 Platform Centralization vs Team Autonomy [00:11:32] 1.4 Enterprise AI Adoption Strategy and Leadership 2. MLOps Infrastructure and Resource Management [00:15:08] 2.1 Technology Evolution and Enterprise Integration [00:19:10] 2.2 Enterprise MLOps Platform Development [00:22:15] 2.3 AI Interface Evolution and Agent-Based Solutions [00:25:47] 2.4 CentML's Infrastructure Solutions [00:30:00] 2.5 Workload Abstraction and Resource Allocation 3. LLM Infrastructure Optimization and Independence [00:33:10] 3.1 GPU Optimization and Cost Efficiency [00:36:47] 3.2 AI Efficiency and Innovation Challenges [00:41:40] 3.3 Cloud Provider Strategy and Infrastructure Control [00:46:52] 3.4 Platform Independence and Vendor Lock-in [00:50:53] 3.5 Technical Innovation and Growth Strategy REFS: [00:01:25] Apple Acquires GraphLab, Apple Inc. https://techcrunch.com/2016/08/05/apple-acquires-turi-a-machine-learning-company/ [00:03:50] Bain Tech Report 2024, Gartner https://www.bain.com/insights/topics/technology-report/ [00:04:50] PaaS vs IaaS Efficiency, Gartner https://www.gartner.com/en/newsroom/press-releases/2024-11-19-gartner-forecasts-worldwide-public-cloud-end-user-spending-to-total-723-billion-dollars-in-2025 [00:14:55] Fashion Quote, Oscar Wilde https://www.amazon.com/Complete-Works-Oscar-Wilde-Collins/dp/0007144369 [00:15:30] PointCast Network, PointCast Inc. https://en.wikipedia.org/wiki/Push_technology [00:18:05] AI Bain Report, Bain & Company https://www.bain.com/insights/how-generative-ai-changes-the-game-in-tech-services-tech-report-2024/ [00:20:40] Uber Michelangelo, Uber Engineering Team https://www.uber.com/en-SE/blog/michelangelo-machine-learning-platform/ [00:20:50] Algorithmia Acquisition, DataRobot https://www.datarobot.com/newsroom/press/datarobot-is-acquiring-algorithmia-enhancing-leading-mlops-architecture-for-the-enterprise/ [00:22:55] Fine Tuning vs RAG, Heydar Soudani, Evangelos Kanoulas & Faegheh Hasibi. https://arxiv.org/html/2403.01432v2 [00:24:40] LLM Agent Survey, Lei Wang et al. https://arxiv.org/abs/2308.11432 [00:26:30] CentML CServe, CentML https://docs.centml.ai/apps/llm [00:29:15] CentML Snowflake, Snowflake https://www.snowflake.com/en/engineering-blog/optimize-llms-with-llama-snowflake-ai-stack/ [00:30:15] NVIDIA H100 GPU, NVIDIA https://www.nvidia.com/en-us/data-center/h100/ [00:33:25] CentML\'s 60% savings, CentML https://centml.ai/platform/

Federico Barbero (DeepMind/Oxford) is the lead author of "Transformers Need Glasses!". Have you ever wondered why LLMs struggle with seemingly simple tasks like counting or copying long strings of text? We break down the theoretical reasons behind these failures, revealing architectural bottlenecks and the challenges of maintaining information fidelity across extended contexts. Federico explains how these issues are rooted in the transformer's design, drawing parallels to over-squashing in graph neural networks and detailing how the softmax function limits sharp decision-making. But it's not all bad news! Discover practical "glasses" that can help transformers see more clearly, from simple input modifications to architectural tweaks. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** https://federicobarbero.com/ TRANSCRIPT + RESEARCH: https://www.dropbox.com/s/h7ys83ztwktqjje/Federico.pdf?dl=0 TOC: 1. Transformer Limitations: Token Detection & Representation [00:00:00] 1.1 Transformers fail at single token detection [00:02:45] 1.2 Representation collapse in transformers [00:03:21] 1.3 Experiment: LLMs fail at copying last tokens [00:18:00] 1.4 Attention sharpness limitations in transformers 2. Transformer Limitations: Information Flow & Quantization [00:18:50] 2.1 Unidirectional information mixing [00:18:50] 2.2 Unidirectional information flow towards sequence beginning in transformers [00:21:50] 2.3 Diagonal attention heads as expensive no-ops in LAMA/Gemma [00:27:14] 2.4 Sequence entropy affects transformer model distinguishability [00:30:36] 2.5 Quantization limitations lead to information loss & representational collapse [00:38:34] 2.6 LLMs use subitizing as opposed to counting algorithms 3. Transformers and the Nature of Reasoning [00:40:30] 3.1 Turing completeness conditions in transformers [00:43:23] 3.2 Transformers struggle with sequential tasks [00:45:50] 3.3 Windowed attention as solution to information compression [00:51:04] 3.4 Chess engines: mechanical computation vs creative reasoning [01:00:35] 3.5 Epistemic foraging introduced REFS: [00:01:05] Transformers Need Glasses!, Barbero et al. https://proceedings.neurips.cc/paper_files/paper/2024/file/b1d35561c4a4a0e0b6012b2af531e149-Paper-Conference.pdf [00:05:30] Softmax is Not Enough, Veličković et al. https://arxiv.org/abs/2410.01104 [00:11:30] Adv Alg Lecture 15, Chawla https://pages.cs.wisc.edu/~shuchi/courses/787-F09/scribe-notes/lec15.pdf [00:15:05] Graph Attention Networks, Veličković https://arxiv.org/abs/1710.10903 [00:19:15] Extract Training Data, Carlini et al. https://arxiv.org/pdf/2311.17035 [00:31:30] 1-bit LLMs, Ma et al. https://arxiv.org/abs/2402.17764 [00:38:35] LLMs Solve Math, Nikankin et al. https://arxiv.org/html/2410.21272v1 [00:38:45] Subitizing, Railo https://link.springer.com/10.1007/978-1-4419-1428-6_578 [00:43:25] NN & Chomsky Hierarchy, Delétang et al. https://arxiv.org/abs/2207.02098 [00:51:05] Measure of Intelligence, Chollet https://arxiv.org/abs/1911.01547 [00:52:10] AlphaZero, Silver et al. https://pubmed.ncbi.nlm.nih.gov/30523106/ [00:55:10] Golden Gate Claude, Anthropic https://www.anthropic.com/news/golden-gate-claude [00:56:40] Chess Positions, Chase & Simon https://www.sciencedirect.com/science/article/abs/pii/0010028573900042 [01:00:35] Epistemic Foraging, Friston https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2016.00056/full

We speak with Sakana AI, who are building nature-inspired methods that could fundamentally transform how we develop AI systems. The guests include Chris Lu, a researcher who recently completed his DPhil at Oxford University under Prof. Jakob Foerster's supervision, where he focused on meta-learning and multi-agent systems. Chris is the first author of the DiscoPOP paper, which demonstrates how language models can discover and design better training algorithms. Also joining is Robert Tjarko Lange, a founding member of Sakana AI who specializes in evolutionary algorithms and large language models. Robert leads research at the intersection of evolutionary computation and foundation models, and is completing his PhD at TU Berlin on evolutionary meta-learning. The discussion also features Cong Lu, currently a Research Scientist at Google DeepMind's Open-Endedness team, who previously helped develop The AI Scientist and Intelligent Go-Explore. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** * DiscoPOP - A framework where language models discover their own optimization algorithms * EvoLLM - Using language models as evolution strategies for optimization The AI Scientist - A fully automated system that conducts scientific research end-to-end * Neural Attention Memory Models (NAMMs) - Evolved memory systems that make transformers both faster and more accurate TRANSCRIPT + REFS: https://www.dropbox.com/scl/fi/gflcyvnujp8cl7zlv3v9d/Sakana.pdf?rlkey=woaoo82943170jd4yyi2he71c&dl=0 Robert Tjarko Lange https://roberttlange.com/ Chris Lu https://chrislu.page/ Cong Lu https://www.conglu.co.uk/ Sakana https://sakana.ai/blog/ TOC: 1. LLMs for Algorithm Generation and Optimization [00:00:00] 1.1 LLMs generating algorithms for training other LLMs [00:04:00] 1.2 Evolutionary black-box optim using neural network loss parameterization [00:11:50] 1.3 DiscoPOP: Non-convex loss function for noisy data [00:20:45] 1.4 External entropy Injection for preventing Model collapse [00:26:25] 1.5 LLMs for black-box optimization using abstract numerical sequences 2. Model Learning and Generalization [00:31:05] 2.1 Fine-tuning on teacher algorithm trajectories [00:31:30] 2.2 Transformers learning gradient descent [00:33:00] 2.3 LLM tokenization biases towards specific numbers [00:34:50] 2.4 LLMs as evolution strategies for black box optimization [00:38:05] 2.5 DiscoPOP: LLMs discovering novel optimization algorithms 3. AI Agents and System Architectures [00:51:30] 3.1 ARC challenge: Induction vs. transformer approaches [00:54:35] 3.2 LangChain / modular agent components [00:57:50] 3.3 Debate improves LLM truthfulness [01:00:55] 3.4 Time limits controlling AI agent systems [01:03:00] 3.5 Gemini: Million-token context enables flatter hierarchies [01:04:05] 3.6 Agents follow own interest gradients [01:09:50] 3.7 Go-Explore algorithm: archive-based exploration [01:11:05] 3.8 Foundation models for interesting state discovery [01:13:00] 3.9 LLMs leverage prior game knowledge 4. AI for Scientific Discovery and Human Alignment [01:17:45] 4.1 Encoding Alignment & Aesthetics via Reward Functions [01:20:00] 4.2 AI Scientist: Automated Open-Ended Scientific Discovery [01:24:15] 4.3 DiscoPOP: LLM for Preference Optimization Algorithms [01:28:30] 4.4 Balancing AI Knowledge with Human Understanding [01:33:55] 4.5 AI-Driven Conferences and Paper Review

Clement Bonnet discusses his novel approach to the ARC (Abstraction and Reasoning Corpus) challenge. Unlike approaches that rely on fine-tuning LLMs or generating samples at inference time, Clement's method encodes input-output pairs into a latent space, optimizes this representation with a search algorithm, and decodes outputs for new inputs. This end-to-end architecture uses a VAE loss, including reconstruction and prior losses. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT + RESEARCH OVERVIEW: https://www.dropbox.com/scl/fi/j7m0gaz1126y594gswtma/CLEMMLST.pdf?rlkey=y5qvwq2er5nchbcibm07rcfpq&dl=0 Clem and Matthew- https://www.linkedin.com/in/clement-bonnet16/ https://github.com/clement-bonnet https://mvmacfarlane.github.io/ TOC 1. LPN Fundamentals [00:00:00] 1.1 Introduction to ARC Benchmark and LPN Overview [00:05:05] 1.2 Neural Networks' Challenges with ARC and Program Synthesis [00:06:55] 1.3 Induction vs Transduction in Machine Learning 2. LPN Architecture and Latent Space [00:11:50] 2.1 LPN Architecture and Latent Space Implementation [00:16:25] 2.2 LPN Latent Space Encoding and VAE Architecture [00:20:25] 2.3 Gradient-Based Search Training Strategy [00:23:39] 2.4 LPN Model Architecture and Implementation Details 3. Implementation and Scaling [00:27:34] 3.1 Training Data Generation and re-ARC Framework [00:31:28] 3.2 Limitations of Latent Space and Multi-Thread Search [00:34:43] 3.3 Program Composition and Computational Graph Architecture 4. Advanced Concepts and Future Directions [00:45:09] 4.1 AI Creativity and Program Synthesis Approaches [00:49:47] 4.2 Scaling and Interpretability in Latent Space Models REFS [00:00:05] ARC benchmark, Chollet https://arxiv.org/abs/2412.04604 [00:02:10] Latent Program Spaces, Bonnet, Macfarlane https://arxiv.org/abs/2411.08706 [00:07:45] Kevin Ellis work on program generation https://www.cs.cornell.edu/~ellisk/ [00:08:45] Induction vs transduction in abstract reasoning, Li et al. https://arxiv.org/abs/2411.02272 [00:17:40] VAEs, Kingma, Welling https://arxiv.org/abs/1312.6114 [00:27:50] re-ARC, Hodel https://github.com/michaelhodel/re-arc [00:29:40] Grid size in ARC tasks, Chollet https://github.com/fchollet/ARC-AGI [00:33:00] Critique of deep learning, Marcus https://arxiv.org/vc/arxiv/papers/2002/2002.06177v1.pdf

Prof. Jakob Foerster, a leading AI researcher at Oxford University and Meta, and Chris Lu, a researcher at OpenAI -- they explain how AI is moving beyond just mimicking human behaviour to creating truly intelligent agents that can learn and solve problems on their own. Foerster champions open-source AI for responsible, decentralised development. He addresses AI scaling, goal misalignment (Goodhart's Law), and the need for holistic alignment, offering a quick look at the future of AI and how to guide it. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT/REFS: https://www.dropbox.com/scl/fi/yqjszhntfr00bhjh6t565/JAKOB.pdf?rlkey=scvny4bnwj8th42fjv8zsfu2y&dl=0 Prof. Jakob Foerster https://x.com/j_foerst https://www.jakobfoerster.com/ University of Oxford Profile: https://eng.ox.ac.uk/people/jakob-foerster/ Chris Lu: https://chrislu.page/ TOC 1. GPU Acceleration and Training Infrastructure [00:00:00] 1.1 ARC Challenge Criticism and FLAIR Lab Overview [00:01:25] 1.2 GPU Acceleration and Hardware Lottery in RL [00:05:50] 1.3 Data Wall Challenges and Simulation-Based Solutions [00:08:40] 1.4 JAX Implementation and Technical Acceleration 2. Learning Frameworks and Policy Optimization [00:14:18] 2.1 Evolution of RL Algorithms and Mirror Learning Framework [00:15:25] 2.2 Meta-Learning and Policy Optimization Algorithms [00:21:47] 2.3 Language Models and Benchmark Challenges [00:28:15] 2.4 Creativity and Meta-Learning in AI Systems 3. Multi-Agent Systems and Decentralization [00:31:24] 3.1 Multi-Agent Systems and Emergent Intelligence [00:38:35] 3.2 Swarm Intelligence vs Monolithic AGI Systems [00:42:44] 3.3 Democratic Control and Decentralization of AI Development [00:46:14] 3.4 Open Source AI and Alignment Challenges [00:49:31] 3.5 Collaborative Models for AI Development REFS [[00:00:05] ARC Benchmark, Chollet https://github.com/fchollet/ARC-AGI [00:03:05] DRL Doesn't Work, Irpan https://www.alexirpan.com/2018/02/14/rl-hard.html [00:05:55] AI Training Data, Data Provenance Initiative https://www.nytimes.com/2024/07/19/technology/ai-data-restrictions.html [00:06:10] JaxMARL, Foerster et al. https://arxiv.org/html/2311.10090v5 [00:08:50] M-FOS, Lu et al. https://arxiv.org/abs/2205.01447 [00:09:45] JAX Library, Google Research https://github.com/jax-ml/jax [00:12:10] Kinetix, Mike and Michael https://arxiv.org/abs/2410.23208 [00:12:45] Genie 2, DeepMind https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ [00:14:42] Mirror Learning, Grudzien, Kuba et al. https://arxiv.org/abs/2208.01682 [00:16:30] Discovered Policy Optimisation, Lu et al. https://arxiv.org/abs/2210.05639 [00:24:10] Goodhart's Law, Goodhart https://en.wikipedia.org/wiki/Goodhart%27s_law [00:25:15] LLM ARChitect, Franzen et al. https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf [00:28:55] AlphaGo, Silver et al. https://arxiv.org/pdf/1712.01815.pdf [00:30:10] Meta-learning, Lu, Towers, Foerster https://direct.mit.edu/isal/proceedings-pdf/isal2023/35/67/2354943/isal_a_00674.pdf [00:31:30] Emergence of Pragmatics, Yuan et al. https://arxiv.org/abs/2001.07752 [00:34:30] AI Safety, Amodei et al. https://arxiv.org/abs/1606.06565 [00:35:45] Intentional Stance, Dennett https://plato.stanford.edu/entries/ethics-ai/ [00:39:25] Multi-Agent RL, Zhou et al. https://arxiv.org/pdf/2305.10091 [00:41:00] Open Source Generative AI, Foerster et al. https://arxiv.org/abs/2405.08597

Daniel Franzen and Jan Disselhoff, the "ARChitects" are the official winners of the ARC Prize 2024. Filmed at Tufa Labs in Zurich - they revealed how they achieved a remarkable 53.5% accuracy by creatively utilising large language models (LLMs) in new ways. Discover their innovative techniques, including depth-first search for token selection, test-time training, and a novel augmentation-based validation system. Their results were extremely surprising. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** Jan Disselhoff https://www.linkedin.com/in/jan-disselhoff-1423a2240/ Daniel Franzen https://github.com/da-fr ARC Prize: http://arcprize.org/ TRANSCRIPT AND BACKGROUND READING: https://www.dropbox.com/scl/fi/utkn2i1ma79fn6an4yvjw/ARCHitects.pdf?rlkey=67pe38mtss7oyhjk2ad0d2aza&dl=0 TOC 1. Solution Architecture and Strategy Overview [00:00:00] 1.1 Initial Solution Overview and Model Architecture [00:04:25] 1.2 LLM Capabilities and Dataset Approach [00:10:51] 1.3 Test-Time Training and Data Augmentation Strategies [00:14:08] 1.4 Sampling Methods and Search Implementation [00:17:52] 1.5 ARC vs Language Model Context Comparison 2. LLM Search and Model Implementation [00:21:53] 2.1 LLM-Guided Search Approaches and Solution Validation [00:27:04] 2.2 Symmetry Augmentation and Model Architecture [00:30:11] 2.3 Model Intelligence Characteristics and Performance [00:37:23] 2.4 Tokenization and Numerical Processing Challenges 3. Advanced Training and Optimization [00:45:15] 3.1 DFS Token Selection and Probability Thresholds [00:49:41] 3.2 Model Size and Fine-tuning Performance Trade-offs [00:53:07] 3.3 LoRA Implementation and Catastrophic Forgetting Prevention [00:56:10] 3.4 Training Infrastructure and Optimization Experiments [01:02:34] 3.5 Search Tree Analysis and Entropy Distribution Patterns REFS [00:01:05] Winning ARC 2024 solution using 12B param model, Franzen, Disselhoff, Hartmann https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf [00:03:40] Robustness of analogical reasoning in LLMs, Melanie Mitchell https://arxiv.org/html/2411.14215 [00:07:50] Re-ARC dataset generator for ARC task variations, Michael Hodel https://github.com/michaelhodel/re-arc [00:15:00] Analysis of search methods in LLMs (greedy, beam, DFS), Chen et al. https://arxiv.org/html/2408.00724v2 [00:16:55] Language model reachability space exploration, University of Toronto https://www.youtube.com/watch?v=Bpgloy1dDn0 [00:22:30] GPT-4 guided code solutions for ARC tasks, Ryan Greenblatt https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt [00:41:20] GPT tokenization approach for numbers, OpenAI https://platform.openai.com/docs/guides/text-generation/tokenizer-examples [00:46:25] DFS in AI search strategies, Russell & Norvig https://www.amazon.com/Artificial-Intelligence-Modern-Approach-4th/dp/0134610997 [00:53:10] Paper on catastrophic forgetting in neural networks, Kirkpatrick et al. https://www.pnas.org/doi/10.1073/pnas.1611835114 [00:54:00] LoRA for efficient fine-tuning of LLMs, Hu et al. https://arxiv.org/abs/2106.09685 [00:57:20] NVIDIA H100 Tensor Core GPU specs, NVIDIA https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/ [01:04:55] Original MCTS in computer Go, Yifan Jin https://stanford.edu/~rezab/classes/cme323/S15/projects/montecarlo_search_tree_report.pdf

Sepp Hochreiter, the inventor of LSTM (Long Short-Term Memory) networks – a foundational technology in AI. Sepp discusses his journey, the origins of LSTM, and why he believes his latest work, XLSTM, could be the next big thing in AI, particularly for applications like robotics and industrial simulation. He also shares his controversial perspective on Large Language Models (LLMs) and why reasoning is a critical missing piece in current AI systems. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT AND BACKGROUND READING: https://www.dropbox.com/scl/fi/n1vzm79t3uuss8xyinxzo/SEPPH.pdf?rlkey=fp7gwaopjk17uyvgjxekxrh5v&dl=0 Prof. Sepp Hochreiter https://www.nx-ai.com/ https://x.com/hochreitersepp https://scholar.google.at/citations?user=tvUH3WMAAAAJ&hl=en TOC: 1. LLM Evolution and Reasoning Capabilities [00:00:00] 1.1 LLM Capabilities and Limitations Debate [00:03:16] 1.2 Program Generation and Reasoning in AI Systems [00:06:30] 1.3 Human vs AI Reasoning Comparison [00:09:59] 1.4 New Research Initiatives and Hybrid Approaches 2. LSTM Technical Architecture [00:13:18] 2.1 LSTM Development History and Technical Background [00:20:38] 2.2 LSTM vs RNN Architecture and Computational Complexity [00:25:10] 2.3 xLSTM Architecture and Flash Attention Comparison [00:30:51] 2.4 Evolution of Gating Mechanisms from Sigmoid to Exponential 3. Industrial Applications and Neuro-Symbolic AI [00:40:35] 3.1 Industrial Applications and Fixed Memory Advantages [00:42:31] 3.2 Neuro-Symbolic Integration and Pi AI Project [00:46:00] 3.3 Integration of Symbolic and Neural AI Approaches [00:51:29] 3.4 Evolution of AI Paradigms and System Thinking [00:54:55] 3.5 AI Reasoning and Human Intelligence Comparison [00:58:12] 3.6 NXAI Company and Industrial AI Applications REFS: [00:00:15] Seminal LSTM paper establishing Hochreiter's expertise (Hochreiter & Schmidhuber) https://direct.mit.edu/neco/article-abstract/9/8/1735/6109/Long-Short-Term-Memory [00:04:20] Kolmogorov complexity and program composition limitations (Kolmogorov) https://link.springer.com/article/10.1007/BF02478259 [00:07:10] Limitations of LLM mathematical reasoning and symbolic integration (Various Authors) https://www.arxiv.org/pdf/2502.03671 [00:09:05] AlphaGo’s Move 37 demonstrating creative AI (Google DeepMind) https://deepmind.google/research/breakthroughs/alphago/ [00:10:15] New AI research lab in Zurich for fundamental LLM research (Benjamin Crouzier) https://tufalabs.ai [00:19:40] Introduction of xLSTM with exponential gating (Beck, Hochreiter, et al.) https://arxiv.org/abs/2405.04517 [00:22:55] FlashAttention: fast & memory-efficient attention (Tri Dao et al.) https://arxiv.org/abs/2205.14135 [00:31:00] Historical use of sigmoid/tanh activation in 1990s (James A. McCaffrey) https://visualstudiomagazine.com/articles/2015/06/01/alternative-activation-functions.aspx [00:36:10] Mamba 2 state space model architecture (Albert Gu et al.) https://arxiv.org/abs/2312.00752 [00:46:00] Austria’s Pi AI project integrating symbolic & neural AI (Hochreiter et al.) https://www.jku.at/en/institute-of-machine-learning/research/projects/ [00:48:10] Neuro-symbolic integration challenges in language models (Diego Calanzone et al.) https://openreview.net/forum?id=7PGluppo4k [00:49:30] JKU Linz’s historical and neuro-symbolic research (Sepp Hochreiter) https://www.jku.at/en/news-events/news/detail/news/bilaterale-ki-projekt-unter-leitung-der-jku-erhaelt-fwf-cluster-of-excellence/ YT: https://www.youtube.com/watch?v=8u2pW2zZLCs

Professor Randall Balestriero joins us to discuss neural network geometry, spline theory, and emerging phenomena in deep learning, based on research presented at ICML. Topics include the delayed emergence of adversarial robustness in neural networks ("grokking"), geometric interpretations of neural networks via spline theory, and challenges in reconstruction learning. We also cover geometric analysis of Large Language Models (LLMs) for toxicity detection and the relationship between intrinsic dimensionality and model control in RLHF. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? Goto https://tufalabs.ai/ *** Randall Balestriero https://x.com/randall_balestr https://randallbalestriero.github.io/ Show notes and transcript: https://www.dropbox.com/scl/fi/3lufge4upq5gy0ug75j4a/RANDALLSHOW.pdf?rlkey=nbemgpa0jhawt1e86rx7372e4&dl=0 TOC: - Introduction - 00:00:00: Introduction - Neural Network Geometry and Spline Theory - 00:01:41: Neural Network Geometry and Spline Theory - 00:07:41: Deep Networks Always Grok - 00:11:39: Grokking and Adversarial Robustness - 00:16:09: Double Descent and Catastrophic Forgetting - Reconstruction Learning - 00:18:49: Reconstruction Learning - 00:24:15: Frequency Bias in Neural Networks - Geometric Analysis of Neural Networks - 00:29:02: Geometric Analysis of Neural Networks - 00:34:41: Adversarial Examples and Region Concentration - LLM Safety and Geometric Analysis - 00:40:05: LLM Safety and Geometric Analysis - 00:46:11: Toxicity Detection in LLMs - 00:52:24: Intrinsic Dimensionality and Model Control - 00:58:07: RLHF and High-Dimensional Spaces - Conclusion - 01:02:13: Neural Tangent Kernel - 01:08:07: Conclusion REFS: [00:01:35] Humayun – Deep network geometry & input space partitioning https://arxiv.org/html/2408.04809v1 [00:03:55] Balestriero & Paris – Linking deep networks to adaptive spline operators https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf [00:13:55] Song et al. – Gradient-based white-box adversarial attacks https://arxiv.org/abs/2012.14965 [00:16:05] Humayun, Balestriero & Baraniuk – Grokking phenomenon & emergent robustness https://arxiv.org/abs/2402.15555 [00:18:25] Humayun – Training dynamics & double descent via linear region evolution https://arxiv.org/abs/2310.12977 [00:20:15] Balestriero – Power diagram partitions in DNN decision boundaries https://arxiv.org/abs/1905.08443 [00:23:00] Frankle & Carbin – Lottery Ticket Hypothesis for network pruning https://arxiv.org/abs/1803.03635 [00:24:00] Belkin et al. – Double descent phenomenon in modern ML https://arxiv.org/abs/1812.11118 [00:25:55] Balestriero et al. – Batch normalization’s regularization effects https://arxiv.org/pdf/2209.14778 [00:29:35] EU – EU AI Act 2024 with compute restrictions https://www.lw.com/admin/upload/SiteAttachments/EU-AI-Act-Navigating-a-Brave-New-World.pdf [00:39:30] Humayun, Balestriero & Baraniuk – SplineCam: Visualizing deep network geometry https://openaccess.thecvf.com/content/CVPR2023/papers/Humayun_SplineCam_Exact_Visualization_and_Characterization_of_Deep_Network_Geometry_and_CVPR_2023_paper.pdf [00:40:40] Carlini – Trade-offs between adversarial robustness and accuracy https://arxiv.org/pdf/2407.20099 [00:44:55] Balestriero & LeCun – Limitations of reconstruction-based learning methods https://openreview.net/forum?id=ez7w0Ss4g9 (truncated, see shownotes PDF)

Nicholas Carlini from Google DeepMind offers his view of AI security, emergent LLM capabilities, and his groundbreaking model-stealing research. He reveals how LLMs can unexpectedly excel at tasks like chess and discusses the security pitfalls of LLM-generated code. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? Goto https://tufalabs.ai/ *** Transcript: https://www.dropbox.com/scl/fi/lat7sfyd4k3g5k9crjpbf/CARLINI.pdf?rlkey=b7kcqbvau17uw6rksbr8ccd8v&dl=0 TOC: 1. ML Security Fundamentals [00:00:00] 1.1 ML Model Reasoning and Security Fundamentals [00:03:04] 1.2 ML Security Vulnerabilities and System Design [00:08:22] 1.3 LLM Chess Capabilities and Emergent Behavior [00:13:20] 1.4 Model Training, RLHF, and Calibration Effects 2. Model Evaluation and Research Methods [00:19:40] 2.1 Model Reasoning and Evaluation Metrics [00:24:37] 2.2 Security Research Philosophy and Methodology [00:27:50] 2.3 Security Disclosure Norms and Community Differences 3. LLM Applications and Best Practices [00:44:29] 3.1 Practical LLM Applications and Productivity Gains [00:49:51] 3.2 Effective LLM Usage and Prompting Strategies [00:53:03] 3.3 Security Vulnerabilities in LLM-Generated Code 4. Advanced LLM Research and Architecture [00:59:13] 4.1 LLM Code Generation Performance and O(1) Labs Experience [01:03:31] 4.2 Adaptation Patterns and Benchmarking Challenges [01:10:10] 4.3 Model Stealing Research and Production LLM Architecture Extraction REFS: [00:01:15] Nicholas Carlini’s personal website & research profile (Google DeepMind, ML security) - https://nicholas.carlini.com/ [00:01:50] CentML AI compute platform for language model workloads - https://centml.ai/ [00:04:30] Seminal paper on neural network robustness against adversarial examples (Carlini & Wagner, 2016) - https://arxiv.org/abs/1608.04644 [00:05:20] Computer Fraud and Abuse Act (CFAA) – primary U.S. federal law on computer hacking liability - https://www.justice.gov/jm/jm-9-48000-computer-fraud [00:08:30] Blog post: Emergent chess capabilities in GPT-3.5-turbo-instruct (Nicholas Carlini, Sept 2023) - https://nicholas.carlini.com/writing/2023/chess-llm.html [00:16:10] Paper: “Self-Play Preference Optimization for Language Model Alignment” (Yue Wu et al., 2024) - https://arxiv.org/abs/2405.00675 [00:18:00] GPT-4 Technical Report: development, capabilities, and calibration analysis - https://arxiv.org/abs/2303.08774 [00:22:40] Historical shift from descriptive to algebraic chess notation (FIDE) - https://en.wikipedia.org/wiki/Descriptive_notation [00:23:55] Analysis of distribution shift in ML (Hendrycks et al.) - https://arxiv.org/abs/2006.16241 [00:27:40] Nicholas Carlini’s essay “Why I Attack” (June 2024) – motivations for security research - https://nicholas.carlini.com/writing/2024/why-i-attack.html [00:34:05] Google Project Zero’s 90-day vulnerability disclosure policy - https://googleprojectzero.blogspot.com/p/vulnerability-disclosure-policy.html [00:51:15] Evolution of Google search syntax & user behavior (Daniel M. Russell) - https://www.amazon.com/Joy-Search-Google-Master-Information/dp/0262042878 [01:04:05] Rust’s ownership & borrowing system for memory safety - https://doc.rust-lang.org/book/ch04-00-understanding-ownership.html [01:10:05] Paper: “Stealing Part of a Production Language Model” (Carlini et al., March 2024) – extraction attacks on ChatGPT, PaLM-2 - https://arxiv.org/abs/2403.06634 [01:10:55] First model stealing paper (Tramèr et al., 2016) – attacking ML APIs via prediction - https://arxiv.org/abs/1609.02943

Join Prof. Subbarao Kambhampati and host Tim Scarfe for a deep dive into OpenAI's O1 model and the future of AI reasoning systems. * How O1 likely uses reinforcement learning similar to AlphaGo, with hidden reasoning tokens that users pay for but never see * The evolution from traditional Large Language Models to more sophisticated reasoning systems * The concept of "fractal intelligence" in AI - where models work brilliantly sometimes but fail unpredictably * Why O1's improved performance comes with substantial computational costs * The ongoing debate between single-model approaches (OpenAI) vs hybrid systems (Google) * The critical distinction between AI as an intelligence amplifier vs autonomous decision-maker SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? Goto https://tufalabs.ai/ *** TOC: 1. **O1 Architecture and Reasoning Foundations** [00:00:00] 1.1 Fractal Intelligence and Reasoning Model Limitations [00:04:28] 1.2 LLM Evolution: From Simple Prompting to Advanced Reasoning [00:14:28] 1.3 O1's Architecture and AlphaGo-like Reasoning Approach [00:23:18] 1.4 Empirical Evaluation of O1's Planning Capabilities 2. **Monte Carlo Methods and Model Deep-Dive** [00:29:30] 2.1 Monte Carlo Methods and MARCO-O1 Implementation [00:31:30] 2.2 Reasoning vs. Retrieval in LLM Systems [00:40:40] 2.3 Fractal Intelligence Capabilities and Limitations [00:45:59] 2.4 Mechanistic Interpretability of Model Behavior [00:51:41] 2.5 O1 Response Patterns and Performance Analysis 3. **System Design and Real-World Applications** [00:59:30] 3.1 Evolution from LLMs to Language Reasoning Models [01:06:48] 3.2 Cost-Efficiency Analysis: LLMs vs O1 [01:11:28] 3.3 Autonomous vs Human-in-the-Loop Systems [01:16:01] 3.4 Program Generation and Fine-Tuning Approaches [01:26:08] 3.5 Hybrid Architecture Implementation Strategies Transcript: https://www.dropbox.com/scl/fi/d0ef4ovnfxi0lknirkvft/Subbarao.pdf?rlkey=l3rp29gs4hkut7he8u04mm1df&dl=0 REFS: [00:02:00] Monty Python (1975) Witch trial scene: flawed logical reasoning. https://www.youtube.com/watch?v=zrzMhU_4m-g [00:04:00] Cade Metz (2024) Microsoft–OpenAI partnership evolution and control dynamics. https://www.nytimes.com/2024/10/17/technology/microsoft-openai-partnership-deal.html [00:07:25] Kojima et al. (2022) Zero-shot chain-of-thought prompting ('Let's think step by step'). https://arxiv.org/pdf/2205.11916 [00:12:50] DeepMind Research Team (2023) Multi-bot game solving with external and internal planning. https://deepmind.google/research/publications/139455/ [00:15:10] Silver et al. (2016) AlphaGo's Monte Carlo Tree Search and Q-learning. https://www.nature.com/articles/nature16961 [00:16:30] Kambhampati, S. et al. (2023) Evaluates O1's planning in "Strawberry Fields" benchmarks. https://arxiv.org/pdf/2410.02162 [00:29:30] Alibaba AIDC-AI Team (2023) MARCO-O1: Chain-of-Thought + MCTS for improved reasoning. https://arxiv.org/html/2411.14405 [00:31:30] Kambhampati, S. (2024) Explores LLM "reasoning vs retrieval" debate. https://arxiv.org/html/2403.04121v2 [00:37:35] Wei, J. et al. (2022) Chain-of-thought prompting (introduces last-letter concatenation). https://arxiv.org/pdf/2201.11903 [00:42:35] Barbero, F. et al. (2024) Transformer attention and "information over-squashing." https://arxiv.org/html/2406.04267v2 [00:46:05] Ruis, L. et al. (2023) Influence functions to understand procedural knowledge in LLMs. https://arxiv.org/html/2411.12580v1 (truncated - continued in shownotes/transcript doc)

Laura Ruis, a PhD student at University College London and researcher at Cohere, explains her groundbreaking research into how large language models (LLMs) perform reasoning tasks, the fundamental mechanisms underlying LLM reasoning capabilities, and whether these models primarily rely on retrieval or develop procedural knowledge. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? Goto https://tufalabs.ai/ *** TOC 1. LLM Foundations and Learning 1.1 Scale and Learning in Language Models [00:00:00] 1.2 Procedural Knowledge vs Fact Retrieval [00:03:40] 1.3 Influence Functions and Model Analysis [00:07:40] 1.4 Role of Code in LLM Reasoning [00:11:10] 1.5 Semantic Understanding and Physical Grounding [00:19:30] 2. Reasoning Architectures and Measurement 2.1 Measuring Understanding and Reasoning in Language Models [00:23:10] 2.2 Formal vs Approximate Reasoning and Model Creativity [00:26:40] 2.3 Symbolic vs Subsymbolic Computation Debate [00:34:10] 2.4 Neural Network Architectures and Tensor Product Representations [00:40:50] 3. AI Agency and Risk Assessment 3.1 Agency and Goal-Directed Behavior in Language Models [00:45:10] 3.2 Defining and Measuring Agency in AI Systems [00:49:50] 3.3 Core Knowledge Systems and Agency Detection [00:54:40] 3.4 Language Models as Agent Models and Simulator Theory [01:03:20] 3.5 AI Safety and Societal Control Mechanisms [01:07:10] 3.6 Evolution of AI Capabilities and Emergent Risks [01:14:20] REFS: [00:01:10] Procedural Knowledge in Pretraining & LLM Reasoning Ruis et al., 2024 https://arxiv.org/abs/2411.12580 [00:03:50] EK-FAC Influence Functions in Large LMs Grosse et al., 2023 https://arxiv.org/abs/2308.03296 [00:13:05] Surfaces and Essences: Analogy as the Core of Cognition Hofstadter & Sander https://www.amazon.com/Surfaces-Essences-Analogy-Fuel-Thinking/dp/0465018475 [00:13:45] Wittgenstein on Language Games https://plato.stanford.edu/entries/wittgenstein/ [00:14:30] Montague Semantics for Natural Language https://plato.stanford.edu/entries/montague-semantics/ [00:19:35] The Chinese Room Argument David Cole https://plato.stanford.edu/entries/chinese-room/ [00:19:55] ARC: Abstraction and Reasoning Corpus François Chollet https://arxiv.org/abs/1911.01547 [00:24:20] Systematic Generalization in Neural Nets Lake & Baroni, 2023 https://www.nature.com/articles/s41586-023-06668-3 [00:27:40] Open-Endedness & Creativity in AI Tim Rocktäschel https://arxiv.org/html/2406.04268v1 [00:30:50] Fodor & Pylyshyn on Connectionism https://www.sciencedirect.com/science/article/abs/pii/0010027788900315 [00:31:30] Tensor Product Representations Smolensky, 1990 https://www.sciencedirect.com/science/article/abs/pii/000437029090007M [00:35:50] DreamCoder: Wake-Sleep Program Synthesis Kevin Ellis et al. https://courses.cs.washington.edu/courses/cse599j1/22sp/papers/dreamcoder.pdf [00:36:30] Compositional Generalization Benchmarks Ruis, Lake et al., 2022 https://arxiv.org/pdf/2202.10745 [00:40:30] RNNs & Tensor Products McCoy et al., 2018 https://arxiv.org/abs/1812.08718 [00:46:10] Formal Causal Definition of Agency Kenton et al. https://arxiv.org/pdf/2208.08345v2 [00:48:40] Agency in Language Models Sumers et al. https://arxiv.org/abs/2309.02427 [00:55:20] Heider & Simmel’s Moving Shapes Experiment https://www.nature.com/articles/s41598-024-65532-0 [01:00:40] Language Models as Agent Models Jacob Andreas, 2022 https://arxiv.org/abs/2212.01681 [01:13:35] Pragmatic Understanding in LLMs Ruis et al. https://arxiv.org/abs/2210.14986

Jürgen Schmidhuber, the father of generative AI, challenges current AI narratives, revealing that early deep learning work is in his opinion misattributed, where it actually originated in Ukraine and Japan. He discusses his early work on linear transformers and artificial curiosity which preceded modern developments, shares his expansive vision of AI colonising space, and explains his groundbreaking 1991 consciousness model. Schmidhuber dismisses fears of human-AI conflict, arguing that superintelligent AI scientists will be fascinated by their own origins and motivated to protect life rather than harm it, while being more interested in other superintelligent AI and in cosmic expansion than earthly matters. He offers unique insights into how humans and AI might coexist. This was the long-awaited second, unreleased part of our interview we filmed last time. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? Goto https://tufalabs.ai/ *** Interviewer: Tim Scarfe TOC [00:00:00] The Nature and Motivations of AI [00:02:08] Influential Inventions: 20th vs. 21st Century [00:05:28] Transformer and GPT: A Reflection The revolutionary impact of modern language models, the 1991 linear transformer, linear vs. quadratic scaling, the fast weight controller, and fast weight matrix memory. [00:11:03] Pioneering Contributions to AI and Deep Learning The invention of the transformer, pre-trained networks, the first GANs, the role of predictive coding, and the emergence of artificial curiosity. [00:13:58] AI's Evolution and Achievements The role of compute, breakthroughs in handwriting recognition and computer vision, the rise of GPU-based CNNs, achieving superhuman results, and Japanese contributions to CNN development. [00:15:40] The Hardware Lottery and GPUs GPUs as a serendipitous advantage for AI, the gaming-AI parallel, and Nvidia's strategic shift towards AI. [00:19:58] AI Applications and Societal Impact AI-powered translation breaking communication barriers, AI in medicine for imaging and disease prediction, and AI's potential for human enhancement and sustainable development. [00:23:26] The Path to AGI and Current Limitations Distinguishing large language models from AGI, challenges in replacing physical world workers, and AI's difficulty in real-world versus board games. [00:25:56] AI and Consciousness Simulating consciousness through unsupervised learning, chunking and automatizing neural networks, data compression, and self-symbols in predictive world models. [00:30:50] The Future of AI and Humanity Transition from AGIs as tools to AGIs with their own goals, the role of humans in an AGI-dominated world, and the concept of Homo Ludens. [00:38:05] The AI Race: Europe, China, and the US Europe's historical contributions, current dominance of the US and East Asia, and the role of venture capital and industrial policy. [00:50:32] Addressing AI Existential Risk The obsession with AI existential risk, commercial pressure for friendly AIs, AI vs. hydrogen bombs, and the long-term future of AI. [00:58:00] The Fermi Paradox and Extraterrestrial Intelligence Expanding AI bubbles as an explanation for the Fermi paradox, dark matter and encrypted civilizations, and Earth as the first to spawn an AI bubble. [01:02:08] The Diversity of AI and AI Ecologies The unrealism of a monolithic super intelligence, diverse AIs with varying goals, and intense competition and collaboration in AI ecologies. [01:12:21] Final Thoughts and Closing Remarks REFERENCES: See pinned comment on YT: https://youtu.be/fZYUqICYCAk

Professor Yoshua Bengio is a pioneer in deep learning and Turing Award winner. Bengio talks about AI safety, why goal-seeking “agentic” AIs might be dangerous, and his vision for building powerful AI tools without giving them agency. Topics include reward tampering risks, instrumental convergence, global AI governance, and how non-agent AIs could revolutionize science and medicine while reducing existential threats. Perfect for anyone curious about advanced AI risks and how to manage them responsibly. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? They are hosting an event in Zurich on January 9th with the ARChitects, join if you can. Goto https://tufalabs.ai/ *** Interviewer: Tim Scarfe Yoshua Bengio: https://x.com/Yoshua_Bengio https://scholar.google.com/citations?user=kukA0LcAAAAJ&hl=en https://yoshuabengio.org/ https://en.wikipedia.org/wiki/Yoshua_Bengio TOC: 1. AI Safety Fundamentals [00:00:00] 1.1 AI Safety Risks and International Cooperation [00:03:20] 1.2 Fundamental Principles vs Scaling in AI Development [00:11:25] 1.3 System 1/2 Thinking and AI Reasoning Capabilities [00:15:15] 1.4 Reward Tampering and AI Agency Risks [00:25:17] 1.5 Alignment Challenges and Instrumental Convergence 2. AI Architecture and Safety Design [00:33:10] 2.1 Instrumental Goals and AI Safety Fundamentals [00:35:02] 2.2 Separating Intelligence from Goals in AI Systems [00:40:40] 2.3 Non-Agent AI as Scientific Tools [00:44:25] 2.4 Oracle AI Systems and Mathematical Safety Frameworks 3. Global Governance and Security [00:49:50] 3.1 International AI Competition and Hardware Governance [00:51:58] 3.2 Military and Security Implications of AI Development [00:56:07] 3.3 Personal Evolution of AI Safety Perspectives [01:00:25] 3.4 AI Development Scaling and Global Governance Challenges [01:12:10] 3.5 AI Regulation and Corporate Oversight 4. Technical Innovations [01:23:00] 4.1 Evolution of Neural Architectures: From RNNs to Transformers [01:26:02] 4.2 GFlowNets and Symbolic Computation [01:30:47] 4.3 Neural Dynamics and Consciousness [01:34:38] 4.4 AI Creativity and Scientific Discovery SHOWNOTES (Transcript, references, best clips etc): https://www.dropbox.com/scl/fi/ajucigli8n90fbxv9h94x/BENGIO_SHOW.pdf?rlkey=38hi2m19sylnr8orb76b85wkw&dl=0 CORE REFS (full list in shownotes and pinned comment): [00:00:15] Bengio et al.: "AI Risk" Statement https://www.safe.ai/work/statement-on-ai-risk [00:23:10] Bengio on reward tampering & AI safety (Harvard Data Science Review) https://hdsr.mitpress.mit.edu/pub/w974bwb0 [00:40:45] Munk Debate on AI existential risk, featuring Bengio https://munkdebates.com/debates/artificial-intelligence [00:44:30] "Can a Bayesian Oracle Prevent Harm from an Agent?" (Bengio et al.) on oracle-to-agent safety https://arxiv.org/abs/2408.05284 [00:51:20] Bengio (2024) memo on hardware-based AI governance verification https://yoshuabengio.org/wp-content/uploads/2024/08/FlexHEG-Memo_August-2024.pdf [01:12:55] Bengio’s involvement in EU AI Act code of practice https://digital-strategy.ec.europa.eu/en/news/meet-chairs-leading-development-first-general-purpose-ai-code-practice [01:27:05] Complexity-based compositionality theory (Elmoznino, Jiralerspong, Bengio, Lajoie) https://arxiv.org/abs/2410.14817 [01:29:00] GFlowNet Foundations (Bengio et al.) for probabilistic inference https://arxiv.org/pdf/2111.09266 [01:32:10] Discrete attractor states in neural systems (Nam, Elmoznino, Bengio, Lajoie) https://arxiv.org/pdf/2302.06403

François Chollet discusses the outcomes of the ARC-AGI (Abstraction and Reasoning Corpus) Prize competition in 2024, where accuracy rose from 33% to 55.5% on a private evaluation set. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? They are hosting an event in Zurich on January 9th with the ARChitects, join if you can. Goto https://tufalabs.ai/ *** Read about the recent result on o3 with ARC here (Chollet knew about it at the time of the interview but wasn't allowed to say): https://arcprize.org/blog/oai-o3-pub-breakthrough TOC: 1. Introduction and Opening [00:00:00] 1.1 Deep Learning vs. Symbolic Reasoning: François’s Long-Standing Hybrid View [00:00:48] 1.2 “Why Do They Call You a Symbolist?” – Addressing Misconceptions [00:01:31] 1.3 Defining Reasoning 3. ARC Competition 2024 Results and Evolution [00:07:26] 3.1 ARC Prize 2024: Reflecting on the Narrative Shift Toward System 2 [00:10:29] 3.2 Comparing Private Leaderboard vs. Public Leaderboard Solutions [00:13:17] 3.3 Two Winning Approaches: Deep Learning–Guided Program Synthesis and Test-Time Training 4. Transduction vs. Induction in ARC [00:16:04] 4.1 Test-Time Training, Overfitting Concerns, and Developer-Aware Generalization [00:19:35] 4.2 Gradient Descent Adaptation vs. Discrete Program Search 5. ARC-2 Development and Future Directions [00:23:51] 5.1 Ensemble Methods, Benchmark Flaws, and the Need for ARC-2 [00:25:35] 5.2 Human-Level Performance Metrics and Private Test Sets [00:29:44] 5.3 Task Diversity, Redundancy Issues, and Expanded Evaluation Methodology 6. Program Synthesis Approaches [00:30:18] 6.1 Induction vs. Transduction [00:32:11] 6.2 Challenges of Writing Algorithms for Perceptual vs. Algorithmic Tasks [00:34:23] 6.3 Combining Induction and Transduction [00:37:05] 6.4 Multi-View Insight and Overfitting Regulation 7. Latent Space and Graph-Based Synthesis [00:38:17] 7.1 Clément Bonnet’s Latent Program Search Approach [00:40:10] 7.2 Decoding to Symbolic Form and Local Discrete Search [00:41:15] 7.3 Graph of Operators vs. Token-by-Token Code Generation [00:45:50] 7.4 Iterative Program Graph Modifications and Reusable Functions 8. Compute Efficiency and Lifelong Learning [00:48:05] 8.1 Symbolic Process for Architecture Generation [00:50:33] 8.2 Logarithmic Relationship of Compute and Accuracy [00:52:20] 8.3 Learning New Building Blocks for Future Tasks 9. AI Reasoning and Future Development [00:53:15] 9.1 Consciousness as a Self-Consistency Mechanism in Iterative Reasoning [00:56:30] 9.2 Reconciling Symbolic and Connectionist Views [01:00:13] 9.3 System 2 Reasoning - Awareness and Consistency [01:03:05] 9.4 Novel Problem Solving, Abstraction, and Reusability 10. Program Synthesis and Research Lab [01:05:53] 10.1 François Leaving Google to Focus on Program Synthesis [01:09:55] 10.2 Democratizing Programming and Natural Language Instruction 11. Frontier Models and O1 Architecture [01:14:38] 11.1 Search-Based Chain of Thought vs. Standard Forward Pass [01:16:55] 11.2 o1’s Natural Language Program Generation and Test-Time Compute Scaling [01:19:35] 11.3 Logarithmic Gains with Deeper Search 12. ARC Evaluation and Human Intelligence [01:22:55] 12.1 LLMs as Guessing Machines and Agent Reliability Issues [01:25:02] 12.2 ARC-2 Human Testing and Correlation with g-Factor [01:26:16] 12.3 Closing Remarks and Future Directions SHOWNOTES PDF: https://www.dropbox.com/scl/fi/ujaai0ewpdnsosc5mc30k/CholletNeurips.pdf?rlkey=s68dp432vefpj2z0dp5wmzqz6&st=hazphyx5&dl=0

AI professor Jeff Clune ruminates on open-ended evolutionary algorithms—systems designed to generate novel and interesting outcomes forever. Drawing inspiration from nature’s boundless creativity, Clune and his collaborators aim to build “Darwin Complete” search spaces, where any computable environment can be simulated. By harnessing the power of large language models and reinforcement learning, these AI agents continuously develop new skills, explore uncharted domains, and even cooperate with one another in complex tasks. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? They are hosting an event in Zurich on January 9th with the ARChitects, join if you can. Goto https://tufalabs.ai/ *** A central theme throughout Clune’s work is “interestingness”: an elusive quality that nudges AI agents toward genuinely original discoveries. Rather than rely on narrowly defined metrics—which often fail due to Goodhart’s Law—Clune employs language models to serve as proxies for human judgment. In doing so, he ensures that “interesting” always reflects authentic novelty, opening the door to unending innovation. Yet with these extraordinary possibilities come equally significant risks. Clune says we need AI safety measures—particularly as the technology matures into powerful, open-ended forms. Potential pitfalls include agents inadvertently causing harm or malicious actors subverting AI’s capabilities for destructive ends. To mitigate this, Clune advocates for prudent governance involving democratic coalitions, regulation of cutting-edge models, and global alignment protocols. Jeff Clune: https://x.com/jeffclune http://jeffclune.com/ (Interviewer: Tim Scarfe) TOC: 1. Introduction [00:00:00] 1.1 Overview and Opening Thoughts 2. Sponsorship [00:03:00] 2.1 TufaAI Labs and CentML 3. Evolutionary AI Foundations [00:04:12] 3.1 Open-Ended Algorithm Development and Abstraction Approaches [00:07:56] 3.2 Novel Intelligence Forms and Serendipitous Discovery [00:11:46] 3.3 Frontier Models and the 'Interestingness' Problem [00:30:36] 3.4 Darwin Complete Systems and Evolutionary Search Spaces 4. System Architecture and Learning [00:37:35] 4.1 Code Generation vs Neural Networks Comparison [00:41:04] 4.2 Thought Cloning and Behavioral Learning Systems [00:47:00] 4.3 Language Emergence in AI Systems [00:50:23] 4.4 AI Interpretability and Safety Monitoring Techniques 5. AI Safety and Governance [00:53:56] 5.1 Language Model Consistency and Belief Systems [00:57:00] 5.2 AI Safety Challenges and Alignment Limitations [01:02:07] 5.3 Open Source AI Development and Value Alignment [01:08:19] 5.4 Global AI Governance and Development Control 6. Advanced AI Systems and Evolution [01:16:55] 6.1 Agent Systems and Performance Evaluation [01:22:45] 6.2 Continuous Learning Challenges and In-Context Solutions [01:26:46] 6.3 Evolution Algorithms and Environment Generation [01:35:36] 6.4 Evolutionary Biology Insights and Experiments [01:48:08] 6.5 Personal Journey from Philosophy to AI Research Shownotes: We craft detailed show notes for each episode with high quality transcript and references and best parts bolded. https://www.dropbox.com/scl/fi/fz43pdoc5wq5jh7vsnujl/JEFFCLUNE.pdf?rlkey=uu0e70ix9zo6g5xn6amykffpm&st=k2scxteu&dl=0

Neel Nanda, a senior research scientist at Google DeepMind, leads their mechanistic interpretability team. In this extensive interview, he discusses his work trying to understand how neural networks function internally. At just 25 years old, Nanda has quickly become a prominent voice in AI research after completing his pure mathematics degree at Cambridge in 2020. Nanda reckons that machine learning is unique because we create neural networks that can perform impressive tasks (like complex reasoning and software engineering) without understanding how they work internally. He compares this to having computer programs that can do things no human programmer knows how to write. His work focuses on "mechanistic interpretability" - attempting to uncover and understand the internal structures and algorithms that emerge within these networks. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ *** SHOWNOTES, TRANSCRIPT, ALL REFERENCES (DONT MISS!): https://www.dropbox.com/scl/fi/36dvtfl3v3p56hbi30im7/NeelShow.pdf?rlkey=pq8t7lyv2z60knlifyy17jdtx&st=kiutudhc&dl=0 We riff on: * How neural networks develop meaningful internal representations beyond simple pattern matching * The effectiveness of chain-of-thought prompting and why it improves model performance * The importance of hands-on coding over extensive paper reading for new researchers * His journey from Cambridge to working with Chris Olah at Anthropic and eventually Google DeepMind * The role of mechanistic interpretability in AI safety NEEL NANDA: https://www.neelnanda.io/ https://scholar.google.com/citations?user=GLnX3MkAAAAJ&hl=en https://x.com/NeelNanda5 Interviewer - Tim Scarfe TOC: 1. Part 1: Introduction [00:00:00] 1.1 Introduction and Core Concepts Overview 2. Part 2: Outside Interview [00:06:45] 2.1 Mechanistic Interpretability Foundations 3. Part 3: Main Interview [00:32:52] 3.1 Mechanistic Interpretability 4. Neural Architecture and Circuits [01:00:31] 4.1 Biological Evolution Parallels [01:04:03] 4.2 Universal Circuit Patterns and Induction Heads [01:11:07] 4.3 Entity Detection and Knowledge Boundaries [01:14:26] 4.4 Mechanistic Interpretability and Activation Patching 5. Model Behavior Analysis [01:30:00] 5.1 Golden Gate Claude Experiment and Feature Amplification [01:33:27] 5.2 Model Personas and RLHF Behavior Modification [01:36:28] 5.3 Steering Vectors and Linear Representations [01:40:00] 5.4 Hallucinations and Model Uncertainty 6. Sparse Autoencoder Architecture [01:44:54] 6.1 Architecture and Mathematical Foundations [02:22:03] 6.2 Core Challenges and Solutions [02:32:04] 6.3 Advanced Activation Functions and Top-k Implementations [02:34:41] 6.4 Research Applications in Transformer Circuit Analysis 7. Feature Learning and Scaling [02:48:02] 7.1 Autoencoder Feature Learning and Width Parameters [03:02:46] 7.2 Scaling Laws and Training Stability [03:11:00] 7.3 Feature Identification and Bias Correction [03:19:52] 7.4 Training Dynamics Analysis Methods 8. Engineering Implementation [03:23:48] 8.1 Scale and Infrastructure Requirements [03:25:20] 8.2 Computational Requirements and Storage [03:35:22] 8.3 Chain-of-Thought Reasoning Implementation [03:37:15] 8.4 Latent Structure Inference in Language Models

Jonas Hübotter, PhD student at ETH Zurich's Institute for Machine Learning, discusses his groundbreaking research on test-time computation and local learning. He demonstrates how smaller models can outperform larger ones by 30x through strategic test-time computation and introduces a novel paradigm combining inductive and transductive learning approaches. Using Bayesian linear regression as a surrogate model for uncertainty estimation, Jonas explains how models can efficiently adapt to specific tasks without massive pre-training. He draws an analogy to Google Earth's variable resolution system to illustrate dynamic resource allocation based on task complexity. The conversation explores the future of AI architecture, envisioning systems that continuously learn and adapt beyond current monolithic models. Jonas concludes by proposing hybrid deployment strategies combining local and cloud computation, suggesting a future where compute resources are allocated based on task complexity rather than fixed model size. This research represents a significant shift in machine learning, prioritizing intelligent resource allocation and adaptive learning over traditional scaling approaches. SPONSOR MESSAGES: CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ Transcription, references and show notes PDF download: https://www.dropbox.com/scl/fi/cxg80p388snwt6qbp4m52/JonasFinal.pdf?rlkey=glk9mhpzjvesanlc14rtpvk4r&st=6qwi8n3x&dl=0 Jonas Hübotter https://jonhue.github.io/ https://scholar.google.com/citations?user=pxi_RkwAAAAJ Transductive Active Learning: Theory and Applications (NeurIPS 2024) https://arxiv.org/pdf/2402.15898 EFFICIENTLY LEARNING AT TEST-TIME: ACTIVE FINE-TUNING OF LLMS (SIFT) https://arxiv.org/pdf/2410.08020 TOC: 1. Test-Time Computation Fundamentals [00:00:00] Intro [00:03:10] 1.1 Test-Time Computation and Model Performance Comparison [00:05:52] 1.2 Retrieval Augmentation and Machine Teaching Strategies [00:09:40] 1.3 In-Context Learning vs Fine-Tuning Trade-offs 2. System Architecture and Intelligence [00:15:58] 2.1 System Architecture and Intelligence Emergence [00:23:22] 2.2 Active Inference and Constrained Agency in AI [00:29:52] 2.3 Evolution of Local Learning Methods [00:32:05] 2.4 Vapnik's Contributions to Transductive Learning 3. Resource Optimization and Local Learning [00:34:35] 3.1 Computational Resource Allocation in ML Models [00:35:30] 3.2 Historical Context and Traditional ML Optimization [00:37:55] 3.3 Variable Resolution Processing and Active Inference in ML [00:43:01] 3.4 Local Learning and Base Model Capacity Trade-offs [00:48:04] 3.5 Active Learning vs Local Learning Approaches 4. Information Retrieval and Model Interpretability [00:51:08] 4.1 Information Retrieval and Nearest Neighbor Limitations [01:03:07] 4.2 Model Interpretability and Surrogate Models [01:15:03] 4.3 Bayesian Uncertainty Estimation and Surrogate Models 5. Distributed Systems and Deployment [01:23:56] 5.1 Memory Architecture and Controller Systems [01:28:14] 5.2 Evolution from Static to Distributed Learning Systems [01:38:03] 5.3 Transductive Learning and Model Specialization [01:41:58] 5.4 Hybrid Local-Cloud Deployment Strategies

Professor Swarat Chaudhuri from the University of Texas at Austin and visiting researcher at Google DeepMind discusses breakthroughs in AI reasoning, theorem proving, and mathematical discovery. Chaudhuri explains his groundbreaking work on COPRA (a GPT-based prover agent), shares insights on neurosymbolic approaches to AI. Professor Swarat Chaudhuri: https://www.cs.utexas.edu/~swarat/ SPONSOR MESSAGES: CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ TOC: [00:00:00] 0. Introduction / CentML ad, Tufa ad 1. AI Reasoning: From Language Models to Neurosymbolic Approaches [00:02:27] 1.1 Defining Reasoning in AI [00:09:51] 1.2 Limitations of Current Language Models [00:17:22] 1.3 Neuro-symbolic Approaches and Program Synthesis [00:24:59] 1.4 COPRA and In-Context Learning for Theorem Proving [00:34:39] 1.5 Symbolic Regression and LLM-Guided Abstraction 2. AI in Mathematics: Theorem Proving and Concept Discovery [00:43:37] 2.1 AI-Assisted Theorem Proving and Proof Verification [01:01:37] 2.2 Symbolic Regression and Concept Discovery in Mathematics [01:11:57] 2.3 Scaling and Modularizing Mathematical Proofs [01:21:53] 2.4 COPRA: In-Context Learning for Formal Theorem-Proving [01:28:22] 2.5 AI-driven theorem proving and mathematical discovery 3. Formal Methods and Challenges in AI Mathematics [01:30:42] 3.1 Formal proofs, empirical predicates, and uncertainty in AI mathematics [01:34:01] 3.2 Characteristics of good theoretical computer science research [01:39:16] 3.3 LLMs in theorem generation and proving [01:42:21] 3.4 Addressing contamination and concept learning in AI systems REFS: 00:04:58 The Chinese Room Argument, https://plato.stanford.edu/entries/chinese-room/ 00:11:42 Software 2.0, https://medium.com/@karpathy/software-2-0-a64152b37c35 00:11:57 Solving Olympiad Geometry Without Human Demonstrations, https://www.nature.com/articles/s41586-023-06747-5 00:13:26 Lean, https://lean-lang.org/ 00:15:43 A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-Play, https://www.science.org/doi/10.1126/science.aar6404 00:19:24 DreamCoder (Ellis et al., PLDI 2021), https://arxiv.org/abs/2006.08381 00:24:37 The Lambda Calculus, https://plato.stanford.edu/entries/lambda-calculus/ 00:26:43 Neural Sketch Learning for Conditional Program Generation, https://arxiv.org/pdf/1703.05698 00:28:08 Learning Differentiable Programs With Admissible Neural Heuristics, https://arxiv.org/abs/2007.12101 00:31:03 Symbolic Regression With a Learned Concept Library (Grayeli et al., NeurIPS 2024), https://arxiv.org/abs/2409.09359 00:41:30 Formal Verification of Parallel Programs, https://dl.acm.org/doi/10.1145/360248.360251 01:00:37 Training Compute-Optimal Large Language Models, https://arxiv.org/abs/2203.15556 01:18:19 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, https://arxiv.org/abs/2201.11903 01:18:42 Draft, Sketch, and Prove: Guiding Formal Theorem Provers With Informal Proofs, https://arxiv.org/abs/2210.12283 01:19:49 Learning Formal Mathematics From Intrinsic Motivation, https://arxiv.org/pdf/2407.00695 01:20:19 An In-Context Learning Agent for Formal Theorem-Proving (Thakur et al., CoLM 2024), https://arxiv.org/pdf/2310.04353 01:23:58 Learning to Prove Theorems via Interacting With Proof Assistants, https://arxiv.org/abs/1905.09381 01:39:58 An In-Context Learning Agent for Formal Theorem-Proving (Thakur et al., CoLM 2024), https://arxiv.org/pdf/2310.04353 01:42:24 Programmatically Interpretable Reinforcement Learning (Verma et al., ICML 2018), https://arxiv.org/abs/1804.02477

Nora Belrose, Head of Interpretability Research at EleutherAI, discusses critical challenges in AI safety and development. The conversation begins with her technical work on concept erasure in neural networks through LEACE (LEAst-squares Concept Erasure), while highlighting how neural networks' progression from simple to complex learning patterns could have important implications for AI safety. Many fear that advanced AI will pose an existential threat -- pursuing its own dangerous goals once it's powerful enough. But Belrose challenges this popular doomsday scenario with a fascinating breakdown of why it doesn't add up. Belrose also provides a detailed critique of current AI alignment approaches, particularly examining "counting arguments" and their limitations when applied to AI safety. She argues that the Principle of Indifference may be insufficient for addressing existential risks from advanced AI systems. The discussion explores how emergent properties in complex AI systems could lead to unpredictable and potentially dangerous behaviors that simple reductionist approaches fail to capture. The conversation concludes by exploring broader philosophical territory, where Belrose discusses her growing interest in Buddhism's potential relevance to a post-automation future. She connects concepts of moral anti-realism with Buddhist ideas about emptiness and non-attachment, suggesting these frameworks might help humans find meaning in a world where AI handles most practical tasks. Rather than viewing this automated future with alarm, she proposes that Zen Buddhism's emphasis on spontaneity and presence might complement a society freed from traditional labor. SPONSOR MESSAGES: CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ Nora Belrose: https://norabelrose.com/ https://scholar.google.com/citations?user=p_oBc64AAAAJ&hl=en https://x.com/norabelrose SHOWNOTES: https://www.dropbox.com/scl/fi/38fhsv2zh8gnubtjaoq4a/NORA_FINAL.pdf?rlkey=0e5r8rd261821g1em4dgv0k70&st=t5c9ckfb&dl=0 TOC: 1. Neural Network Foundations [00:00:00] 1.1 Philosophical Foundations and Neural Network Simplicity Bias [00:02:20] 1.2 LEACE and Concept Erasure Fundamentals [00:13:16] 1.3 LISA Technical Implementation and Applications [00:18:50] 1.4 Practical Implementation Challenges and Data Requirements [00:22:13] 1.5 Performance Impact and Limitations of Concept Erasure 2. Machine Learning Theory [00:32:23] 2.1 Neural Network Learning Progression and Simplicity Bias [00:37:10] 2.2 Optimal Transport Theory and Image Statistics Manipulation [00:43:05] 2.3 Grokking Phenomena and Training Dynamics [00:44:50] 2.4 Texture vs Shape Bias in Computer Vision Models [00:45:15] 2.5 CNN Architecture and Shape Recognition Limitations 3. AI Systems and Value Learning [00:47:10] 3.1 Meaning, Value, and Consciousness in AI Systems [00:53:06] 3.2 Global Connectivity vs Local Culture Preservation [00:58:18] 3.3 AI Capabilities and Future Development Trajectory 4. Consciousness Theory [01:03:03] 4.1 4E Cognition and Extended Mind Theory [01:09:40] 4.2 Thompson's Views on Consciousness and Simulation [01:12:46] 4.3 Phenomenology and Consciousness Theory [01:15:43] 4.4 Critique of Illusionism and Embodied Experience [01:23:16] 4.5 AI Alignment and Counting Arguments Debate (TRUNCATED, TOC embedded in MP3 file with more information)

Prof. Gennady Pekhimenko (CEO of CentML, UofT) joins us in this *sponsored episode* to dive deep into AI system optimization and enterprise implementation. From NVIDIA's technical leadership model to the rise of open-source AI, Pekhimenko shares insights on bridging the gap between academic research and industrial applications. Learn about "dark silicon," GPU utilization challenges in ML workloads, and how modern enterprises can optimize their AI infrastructure. The conversation explores why some companies achieve only 10% GPU efficiency and practical solutions for improving AI system performance. A must-watch for anyone interested in the technical foundations of enterprise AI and hardware optimization. CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Cheaper, faster, no commitments, pay as you go, scale massively, simple to setup. Check it out! https://centml.ai/pricing/ SPONSOR MESSAGES: MLST is also sponsored by Tufa AI Labs - https://tufalabs.ai/ They are hiring cracked ML engineers/researchers to work on ARC and build AGI! SHOWNOTES (diarised transcript, TOC, references, summary, best quotes etc) https://www.dropbox.com/scl/fi/w9kbpso7fawtm286kkp6j/Gennady.pdf?rlkey=aqjqmncx3kjnatk2il1gbgknk&st=2a9mccj8&dl=0 TOC: 1. AI Strategy and Leadership [00:00:00] 1.1 Technical Leadership and Corporate Structure [00:09:55] 1.2 Open Source vs Proprietary AI Models [00:16:04] 1.3 Hardware and System Architecture Challenges [00:23:37] 1.4 Enterprise AI Implementation and Optimization [00:35:30] 1.5 AI Reasoning Capabilities and Limitations 2. AI System Development [00:38:45] 2.1 Computational and Cognitive Limitations of AI Systems [00:42:40] 2.2 Human-LLM Communication Adaptation and Patterns [00:46:18] 2.3 AI-Assisted Software Development Challenges [00:47:55] 2.4 Future of Software Engineering Careers in AI Era [00:49:49] 2.5 Enterprise AI Adoption Challenges and Implementation 3. ML Infrastructure Optimization [00:54:41] 3.1 MLOps Evolution and Platform Centralization [00:55:43] 3.2 Hardware Optimization and Performance Constraints [01:05:24] 3.3 ML Compiler Optimization and Python Performance [01:15:57] 3.4 Enterprise ML Deployment and Cloud Provider Partnerships 4. Distributed AI Architecture [01:27:05] 4.1 Multi-Cloud ML Infrastructure and Optimization [01:29:45] 4.2 AI Agent Systems and Production Readiness [01:32:00] 4.3 RAG Implementation and Fine-Tuning Considerations [01:33:45] 4.4 Distributed AI Systems Architecture and Ray Framework 5. AI Industry Standards and Research [01:37:55] 5.1 Origins and Evolution of MLPerf Benchmarking [01:43:15] 5.2 MLPerf Methodology and Industry Impact [01:50:17] 5.3 Academic Research vs Industry Implementation in AI [01:58:59] 5.4 AI Research History and Safety Concerns

Eliezer Yudkowsky and Stephen Wolfram discuss artificial intelligence and its potential existen‑ tial risks. They traversed fundamental questions about AI safety, consciousness, computational irreducibility, and the nature of intelligence. The discourse centered on Yudkowsky’s argument that advanced AI systems pose an existential threat to humanity, primarily due to the challenge of alignment and the potential for emergent goals that diverge from human values. Wolfram, while acknowledging potential risks, approached the topic from a his signature measured perspective, emphasizing the importance of understanding computational systems’ fundamental nature and questioning whether AI systems would necessarily develop the kind of goal‑directed behavior Yudkowsky fears. *** MLST IS SPONSORED BY TUFA AI LABS! The current winners of the ARC challenge, MindsAI are part of Tufa AI Labs. They are hiring ML engineers. Are you interested?! Please goto https://tufalabs.ai/ *** TOC: 1. Foundational AI Concepts and Risks [00:00:01] 1.1 AI Optimization and System Capabilities Debate [00:06:46] 1.2 Computational Irreducibility and Intelligence Limitations [00:20:09] 1.3 Existential Risk and Species Succession [00:23:28] 1.4 Consciousness and Value Preservation in AI Systems 2. Ethics and Philosophy in AI [00:33:24] 2.1 Moral Value of Human Consciousness vs. Computation [00:36:30] 2.2 Ethics and Moral Philosophy Debate [00:39:58] 2.3 Existential Risks and Digital Immortality [00:43:30] 2.4 Consciousness and Personal Identity in Brain Emulation 3. Truth and Logic in AI Systems [00:54:39] 3.1 AI Persuasion Ethics and Truth [01:01:48] 3.2 Mathematical Truth and Logic in AI Systems [01:11:29] 3.3 Universal Truth vs Personal Interpretation in Ethics and Mathematics [01:14:43] 3.4 Quantum Mechanics and Fundamental Reality Debate 4. AI Capabilities and Constraints [01:21:21] 4.1 AI Perception and Physical Laws [01:28:33] 4.2 AI Capabilities and Computational Constraints [01:34:59] 4.3 AI Motivation and Anthropomorphization Debate [01:38:09] 4.4 Prediction vs Agency in AI Systems 5. AI System Architecture and Behavior [01:44:47] 5.1 Computational Irreducibility and Probabilistic Prediction [01:48:10] 5.2 Teleological vs Mechanistic Explanations of AI Behavior [02:09:41] 5.3 Machine Learning as Assembly of Computational Components [02:29:52] 5.4 AI Safety and Predictability in Complex Systems 6. Goal Optimization and Alignment [02:50:30] 6.1 Goal Specification and Optimization Challenges in AI Systems [02:58:31] 6.2 Intelligence, Computation, and Goal-Directed Behavior [03:02:18] 6.3 Optimization Goals and Human Existential Risk [03:08:49] 6.4 Emergent Goals and AI Alignment Challenges 7. AI Evolution and Risk Assessment [03:19:44] 7.1 Inner Optimization and Mesa-Optimization Theory [03:34:00] 7.2 Dynamic AI Goals and Extinction Risk Debate [03:56:05] 7.3 AI Risk and Biological System Analogies [04:09:37] 7.4 Expert Risk Assessments and Optimism vs Reality 8. Future Implications and Economics [04:13:01] 8.1 Economic and Proliferation Considerations SHOWNOTES (transcription, references, summary, best quotes etc): https://www.dropbox.com/scl/fi/3st8dts2ba7yob161dchd/EliezerWolfram.pdf?rlkey=b6va5j8upgqwl9s2muc924vtt&st=vemwqx7a&dl=0

Francois Chollet, a prominent AI expert and creator of ARC-AGI, discusses intelligence, consciousness, and artificial intelligence. Chollet explains that real intelligence isn't about memorizing information or having lots of knowledge - it's about being able to handle new situations effectively. This is why he believes current large language models (LLMs) have "near-zero intelligence" despite their impressive abilities. They're more like sophisticated memory and pattern-matching systems than truly intelligent beings. *** MLST IS SPONSORED BY TUFA AI LABS! The current winners of the ARC challenge, MindsAI are part of Tufa AI Labs. They are hiring ML engineers. Are you interested?! Please goto https://tufalabs.ai/ *** He introduced his "Kaleidoscope Hypothesis," which suggests that while the world seems infinitely complex, it's actually made up of simpler patterns that repeat and combine in different ways. True intelligence, he argues, involves identifying these basic patterns and using them to understand new situations. Chollet also talked about consciousness, suggesting it develops gradually in children rather than appearing all at once. He believes consciousness exists in degrees - animals have it to some extent, and even human consciousness varies with age and circumstances (like being more conscious when learning something new versus doing routine tasks). On AI safety, Chollet takes a notably different stance from many in Silicon Valley. He views AGI development as a scientific challenge rather than a religious quest, and doesn't share the apocalyptic concerns of some AI researchers. He argues that intelligence itself isn't dangerous - it's just a tool for turning information into useful models. What matters is how we choose to use it. ARC-AGI Prize: https://arcprize.org/ Francois Chollet: https://x.com/fchollet Shownotes: https://www.dropbox.com/scl/fi/j2068j3hlj8br96pfa7bi/CHOLLET_FINAL.pdf?rlkey=xkbr7tbnrjdl66m246w26uc8k&st=0a4ec4na&dl=0 TOC: 1. Intelligence and Model Building [00:00:00] 1.1 Intelligence Definition and ARC Benchmark [00:05:40] 1.2 LLMs as Program Memorization Systems [00:09:36] 1.3 Kaleidoscope Hypothesis and Abstract Building Blocks [00:13:39] 1.4 Deep Learning Limitations and System 2 Reasoning [00:29:38] 1.5 Intelligence vs. Skill in LLMs and Model Building 2. ARC Benchmark and Program Synthesis [00:37:36] 2.1 Intelligence Definition and LLM Limitations [00:41:33] 2.2 Meta-Learning System Architecture [00:56:21] 2.3 Program Search and Occam's Razor [00:59:42] 2.4 Developer-Aware Generalization [01:06:49] 2.5 Task Generation and Benchmark Design 3. Cognitive Systems and Program Generation [01:14:38] 3.1 System 1/2 Thinking Fundamentals [01:22:17] 3.2 Program Synthesis and Combinatorial Challenges [01:31:18] 3.3 Test-Time Fine-Tuning Strategies [01:36:10] 3.4 Evaluation and Leakage Problems [01:43:22] 3.5 ARC Implementation Approaches 4. Intelligence and Language Systems [01:50:06] 4.1 Intelligence as Tool vs Agent [01:53:53] 4.2 Cultural Knowledge Integration [01:58:42] 4.3 Language and Abstraction Generation [02:02:41] 4.4 Embodiment in Cognitive Systems [02:09:02] 4.5 Language as Cognitive Operating System 5. Consciousness and AI Safety [02:14:05] 5.1 Consciousness and Intelligence Relationship [02:20:25] 5.2 Development of Machine Consciousness [02:28:40] 5.3 Consciousness Prerequisites and Indicators [02:36:36] 5.4 AGI Safety Considerations [02:40:29] 5.5 AI Regulation Framework

Anil Ananthaswamy is an award-winning science writer and former staff writer and deputy news editor for the London-based New Scientist magazine. Machine learning systems are making life-altering decisions for us: approving mortgage loans, determining whether a tumor is cancerous, or deciding if someone gets bail. They now influence developments and discoveries in chemistry, biology, and physics—the study of genomes, extrasolar planets, even the intricacies of quantum systems. And all this before large language models such as ChatGPT came on the scene. We are living through a revolution in machine learning-powered AI that shows no signs of slowing down. This technology is based on relatively simple mathematical ideas, some of which go back centuries, including linear algebra and calculus, the stuff of seventeenth- and eighteenth-century mathematics. It took the birth and advancement of computer science and the kindling of 1990s computer chips designed for video games to ignite the explosion of AI that we see today. In this enlightening book, Anil Ananthaswamy explains the fundamental math behind machine learning, while suggesting intriguing links between artificial and natural intelligence. Might the same math underpin them both? As Ananthaswamy resonantly concludes, to make safe and effective use of artificial intelligence, we need to understand its profound capabilities and limitations, the clues to which lie in the math that makes machine learning possible. Why Machines Learn: The Elegant Math Behind Modern AI: https://amzn.to/3UAWX3D https://anilananthaswamy.com/ Sponsor message: DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? Interested? Apply for an ML research position: [email protected] Shownotes: https://www.dropbox.com/scl/fi/wpv22m5jxyiqr6pqfkzwz/anil.pdf?rlkey=9c233jo5armr548ctwo419n6p&st=xzhahtje&dl=0 Chapters: 1. ML Fundamentals and Prerequisites [00:00:00] 1.1 Differences Between Human and Machine Learning [00:00:35] 1.2 Mathematical Prerequisites and Societal Impact of ML [00:02:20] 1.3 Author's Journey and Book Background [00:11:30] 1.4 Mathematical Foundations and Core ML Concepts [00:21:45] 1.5 Bias-Variance Tradeoff and Modern Deep Learning 2. Deep Learning Architecture [00:29:05] 2.1 Double Descent and Overparameterization in Deep Learning [00:32:40] 2.2 Mathematical Foundations and Self-Supervised Learning [00:40:05] 2.3 High-Dimensional Spaces and Model Architecture [00:52:55] 2.4 Historical Development of Backpropagation 3. AI Understanding and Limitations [00:59:13] 3.1 Pattern Matching vs Human Reasoning in ML Models [01:00:20] 3.2 Mathematical Foundations and Pattern Recognition in AI [01:04:08] 3.3 LLM Reliability and Machine Understanding Debate [01:12:50] 3.4 Historical Development of Deep Learning Technologies [01:15:21] 3.5 Alternative AI Approaches and Bio-inspired Methods 4. Ethical and Neurological Perspectives [01:24:32] 4.1 Neural Network Scaling and Mathematical Limitations [01:31:12] 4.2 AI Ethics and Societal Impact [01:38:30] 4.3 Consciousness and Neurological Conditions [01:46:17] 4.4 Body Ownership and Agency in Neuroscience

Professor Michael Levin explores the revolutionary concept of diverse intelligence, demonstrating how cognitive capabilities extend far beyond traditional brain-based intelligence. Drawing from his groundbreaking research, he explains how even simple biological systems like gene regulatory networks exhibit learning, memory, and problem-solving abilities. Levin introduces key concepts like "cognitive light cones" - the scope of goals a system can pursue - and shows how these ideas are transforming our approach to cancer treatment and biological engineering. His insights challenge conventional views of intelligence and agency, with profound implications for both medicine and artificial intelligence development. This deep discussion reveals how understanding intelligence as a spectrum, from molecular networks to human minds, could be crucial for humanity's future technological development. Contains technical discussion of biological systems, cybernetics, and theoretical frameworks for understanding emergent cognition. Prof. Michael Levin https://as.tufts.edu/biology/people/faculty/michael-levin https://x.com/drmichaellevin Sponsor message: DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? Interested? Apply for an ML research position: [email protected] TOC 1. Intelligence Fundamentals and Evolution [00:00:00] 1.1 Future Evolution of Human Intelligence and Consciousness [00:03:00] 1.2 Science Fiction's Role in Exploring Intelligence Possibilities [00:08:15] 1.3 Essential Characteristics of Human-Level Intelligence and Relationships [00:14:20] 1.4 Biological Systems Architecture and Intelligence 2. Biological Computing and Cognition [00:24:00] 2.1 Agency and Intelligence in Biological Systems [00:30:30] 2.2 Learning Capabilities in Gene Regulatory Networks [00:35:37] 2.3 Biological Control Systems and Competency Architecture [00:39:58] 2.4 Scientific Metaphors and Polycomputing Paradigm 3. Systems and Collective Intelligence [00:43:26] 3.1 Embodiment and Problem-Solving Spaces [00:44:50] 3.2 Perception-Action Loops and Biological Intelligence [00:46:55] 3.3 Intelligence, Wisdom and Collective Systems [00:53:07] 3.4 Cancer and Cognitive Light Cones [00:57:09] 3.5 Emergent Intelligence and AI Agency Shownotes: https://www.dropbox.com/scl/fi/i2vl1vs009thg54lxx5wc/LEVIN.pdf?rlkey=dtk8okhbsejryiu2vrht19qp6&st=uzi0vo45&dl=0 REFS: [0:05:30] A Fire Upon the Deep - Vernor Vinge sci-fi novel on AI and consciousness [0:05:35] Maria Chudnovsky - MacArthur Fellow, Princeton mathematician, graph theory expert [0:14:20] Bow-tie architecture in biological systems - Network structure research by Csete & Doyle [0:15:40] Richard Watson - Southampton Professor, evolution and learning systems expert [0:17:00] Levin paper on human issues in AI and evolution [0:19:00] Bow-tie architecture in Darwin's agential materialism - Levin [0:22:55] Philip Goff - Work on panpsychism and consciousness in Galileo's Error [0:23:30] Strange Loop - Hofstadter's work on self-reference and consciousness [0:25:00] The Hard Problem of Consciousness - Van Gulick [0:26:15] Daniel Dennett - Theories on consciousness and intentional systems [0:29:35] Principle of Least Action - Light path selection in physics [0:29:50] Free Energy Principle - Friston's unified behavioral framework [0:30:35] Gene regulatory networks - Learning capabilities in biological systems [0:36:55] Minimal networks with learning capacity - Levin [0:38:50] Multi-scale competency in biological systems - Levin [0:41:40] Polycomputing paradigm - Biological computation by Bongard & Levin [0:45:40] Collective intelligence in biology - Levin et al. [0:46:55] Niche construction and stigmergy - Torday [0:53:50] Tasmanian Devil Facial Tumor Disease - Transmissible cancer research [0:55:05] Cognitive light cone - Computational boundaries of self - Levin [0:58:05] Cognitive properties in sorting algorithms - Zhang, Goldstein & Levin

Will Williams is CTO of Speechmatics in Cambridge. In this sponsored episode - he shares deep technical insights into modern speech recognition technology and system architecture. The episode covers several key technical areas: * Speechmatics' hybrid approach to ASR, which focusses on unsupervised learning methods, achieving comparable results with 100x less data than fully supervised approaches. Williams explains why this is more efficient and generalizable than end-to-end models like Whisper. * Their production architecture implementing multiple operating points for different latency-accuracy trade-offs, with careful latency padding (up to 1.8 seconds) to ensure consistent user experience. The system uses lattice-based decoding with language model integration for improved accuracy. * The challenges and solutions in real-time ASR, including their approach to diarization (speaker identification), handling cross-talk, and implicit source separation. Williams explains why these problems remain difficult even with modern deep learning approaches. * Their testing and deployment infrastructure, including the use of mirrored environments for catching edge cases in production, and their strategy of maintaining global models rather than allowing customer-specific fine-tuning. * Technical evolution in ASR, from early days of custom CUDA kernels and manual memory management to modern frameworks, with Williams offering interesting critiques of current PyTorch memory management approaches and arguing for more efficient direct memory allocation in production systems. Get coding with their API! This is their URL: https://www.speechmatics.com/ DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? MLST is sponsored by Tufa Labs: Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more. Interested? Apply for an ML research position: [email protected] TOC 1. ASR Core Technology & Real-time Architecture [00:00:00] 1.1 ASR and Diarization Fundamentals [00:05:25] 1.2 Real-time Conversational AI Architecture [00:09:21] 1.3 Neural Network Streaming Implementation [00:12:49] 1.4 Multi-modal System Integration 2. Production System Optimization [00:29:38] 2.1 Production Deployment and Testing Infrastructure [00:35:40] 2.2 Model Architecture and Deployment Strategy [00:37:12] 2.3 Latency-Accuracy Trade-offs [00:39:15] 2.4 Language Model Integration [00:40:32] 2.5 Lattice-based Decoding Architecture 3. Performance Evaluation & Ethical Considerations [00:44:00] 3.1 ASR Performance Metrics and Capabilities [00:46:35] 3.2 AI Regulation and Evaluation Methods [00:51:09] 3.3 Benchmark and Testing Challenges [00:54:30] 3.4 Real-world Implementation Metrics [01:00:51] 3.5 Ethics and Privacy Considerations 4. ASR Technical Evolution [01:09:00] 4.1 WER Calculation and Evaluation Methodologies [01:10:21] 4.2 Supervised vs Self-Supervised Learning Approaches [01:21:02] 4.3 Temporal Learning and Feature Processing [01:24:45] 4.4 Feature Engineering to Automated ML 5. Enterprise Implementation & Scale [01:27:55] 5.1 Future AI Systems and Adaptation [01:31:52] 5.2 Technical Foundations and History [01:34:53] 5.3 Infrastructure and Team Scaling [01:38:05] 5.4 Research and Talent Strategy [01:41:11] 5.5 Engineering Practice Evolution Shownotes: https://www.dropbox.com/scl/fi/d94b1jcgph9o8au8shdym/Speechmatics.pdf?rlkey=bi55wvktzomzx0y5sic6jz99y&st=6qwofv8t&dl=0

Dr. Sanjeev Namjoshi, a machine learning engineer who recently submitted a book on Active Inference to MIT Press, discusses the theoretical foundations and practical applications of Active Inference, the Free Energy Principle (FEP), and Bayesian mechanics. He explains how these frameworks describe how biological and artificial systems maintain stability by minimizing uncertainty about their environment. DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? MLST is sponsored by Tufa Labs: Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more. Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2. Interested? Apply for an ML research position: [email protected] Namjoshi traces the evolution of these fields from early 2000s neuroscience research to current developments, highlighting how Active Inference provides a unified framework for perception and action through variational free energy minimization. He contrasts this with traditional machine learning approaches, emphasizing Active Inference's natural capacity for exploration and curiosity through epistemic value. He sees Active Inference as being at a similar stage to deep learning in the early 2000s - poised for significant breakthroughs but requiring better tools and wider adoption. While acknowledging current computational challenges, he emphasizes Active Inference's potential advantages over reinforcement learning, particularly its principled approach to exploration and planning. Dr. Sanjeev Namjoshi https://snamjoshi.github.io/ TOC: 1. Theoretical Foundations: AI Agency and Sentience [00:00:00] 1.1 Intro [00:02:45] 1.2 Free Energy Principle and Active Inference Theory [00:11:16] 1.3 Emergence and Self-Organization in Complex Systems [00:19:11] 1.4 Agency and Representation in AI Systems [00:29:59] 1.5 Bayesian Mechanics and Systems Modeling 2. Technical Framework: Active Inference and Free Energy [00:38:37] 2.1 Generative Processes and Agent-Environment Modeling [00:42:27] 2.2 Markov Blankets and System Boundaries [00:44:30] 2.3 Bayesian Inference and Prior Distributions [00:52:41] 2.4 Variational Free Energy Minimization Framework [00:55:07] 2.5 VFE Optimization Techniques: Generalized Filtering vs DEM 3. Implementation and Optimization Methods [00:58:25] 3.1 Information Theory and Free Energy Concepts [01:05:25] 3.2 Surprise Minimization and Action in Active Inference [01:15:58] 3.3 Evolution of Active Inference Models: Continuous to Discrete Approaches [01:26:00] 3.4 Uncertainty Reduction and Control Systems in Active Inference 4. Safety and Regulatory Frameworks [01:32:40] 4.1 Historical Evolution of Risk Management and Predictive Systems [01:36:12] 4.2 Agency and Reality: Philosophical Perspectives on Models [01:39:20] 4.3 Limitations of Symbolic AI and Current System Design [01:46:40] 4.4 AI Safety Regulation and Corporate Governance 5. Socioeconomic Integration and Modeling [01:52:55] 5.1 Economic Policy and Public Sentiment Modeling [01:55:21] 5.2 Free Energy Principle: Libertarian vs Collectivist Perspectives [01:58:53] 5.3 Regulation of Complex Socio-Technical Systems [02:03:04] 5.4 Evolution and Current State of Active Inference Research 6. Future Directions and Applications [02:14:26] 6.1 Active Inference Applications and Future Development [02:22:58] 6.2 Cultural Learning and Active Inference [02:29:19] 6.3 Hierarchical Relationship Between FEP, Active Inference, and Bayesian Mechanics [02:33:22] 6.4 Historical Evolution of Free Energy Principle [02:38:52] 6.5 Active Inference vs Traditional Machine Learning Approaches Transcript and shownotes with refs and URLs: https://www.dropbox.com/scl/fi/qj22a660cob1795ej0gbw/SanjeevShow.pdf?rlkey=w323r3e8zfsnve22caayzb17k&st=el1fdgfr&dl=0

Dr. Joscha Bach discusses advanced AI, consciousness, and cognitive modeling. He presents consciousness as a virtual property emerging from self-organizing software patterns, challenging panpsychism and materialism. Bach introduces "Cyberanima," reinterpreting animism through information processing, viewing spirits as self-organizing software agents. He addresses limitations of current large language models and advocates for smaller, more efficient AI models capable of reasoning from first principles. Bach describes his work with Liquid AI on novel neural network architectures for improved expressiveness and efficiency. The interview covers AI's societal implications, including regulation challenges and impact on innovation. Bach argues for balancing oversight with technological progress, warning against overly restrictive regulations. Throughout, Bach frames consciousness, intelligence, and agency as emergent properties of complex information processing systems, proposing a computational framework for cognitive phenomena and reality. SPONSOR MESSAGE: DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? MLST is sponsored by Tufa Labs: Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more. Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2. Interested? Apply for an ML research position: [email protected] TOC [00:00:00] 1.1 Consciousness and Intelligence in AI Development [00:07:44] 1.2 Agency, Intelligence, and Their Relationship to Physical Reality [00:13:36] 1.3 Virtual Patterns and Causal Structures in Consciousness [00:25:49] 1.4 Reinterpreting Concepts of God and Animism in Information Processing Terms [00:32:50] 1.5 Animism and Evolution as Competition Between Software Agents 2. Self-Organizing Systems and Cognitive Models in AI [00:37:59] 2.1 Consciousness as self-organizing software [00:45:49] 2.2 Critique of panpsychism and alternative views on consciousness [00:50:48] 2.3 Emergence of consciousness in complex systems [00:52:50] 2.4 Neuronal motivation and the origins of consciousness [00:56:47] 2.5 Coherence and Self-Organization in AI Systems 3. Advanced AI Architectures and Cognitive Processes [00:57:50] 3.1 Second-Order Software and Complex Mental Processes [01:01:05] 3.2 Collective Agency and Shared Values in AI [01:05:40] 3.3 Limitations of Current AI Agents and LLMs [01:06:40] 3.4 Liquid AI and Novel Neural Network Architectures [01:10:06] 3.5 AI Model Efficiency and Future Directions [01:19:00] 3.6 LLM Limitations and Internal State Representation 4. AI Regulation and Societal Impact [01:31:23] 4.1 AI Regulation and Societal Impact [01:49:50] 4.2 Open-Source AI and Industry Challenges Refs in shownotes and MP3 metadata Shownotes: https://www.dropbox.com/scl/fi/g28dosz19bzcfs5imrvbu/JoschaInterview.pdf?rlkey=s3y18jy192ktz6ogd7qtvry3d&st=10z7q7w9&dl=0

Alessandro Palmarini is a post-baccalaureate researcher at the Santa Fe Institute working under the supervision of Melanie Mitchell. He completed his undergraduate degree in Artificial Intelligence and Computer Science at the University of Edinburgh. Palmarini's current research focuses on developing AI systems that can efficiently acquire new skills from limited data, inspired by François Chollet's work on measuring intelligence. His work builds upon the DreamCoder program synthesis system, introducing a novel approach called "dream decompiling" to improve library learning in inductive program synthesis. Palmarini is particularly interested in addressing the Abstraction and Reasoning Corpus (ARC) challenge, aiming to create AI systems that can perform abstract reasoning tasks more efficiently than current approaches. His research explores the balance between computational efficiency and data efficiency in AI learning processes. DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)? MLST is sponsored by Tufa Labs: Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more. Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2. Interested? Apply for an ML research position: [email protected] TOC: 1. Intelligence Measurement in AI Systems [00:00:00] 1.1 Defining Intelligence in AI Systems [00:02:00] 1.2 Research at Santa Fe Institute [00:04:35] 1.3 Impact of Gaming on AI Development [00:05:10] 1.4 Comparing AI and Human Learning Efficiency 2. Efficient Skill Acquisition in AI [00:06:40] 2.1 Intelligence as Skill Acquisition Efficiency [00:08:25] 2.2 Limitations of Current AI Systems in Generalization [00:09:45] 2.3 Human vs. AI Cognitive Processes [00:10:40] 2.4 Measuring AI Intelligence: Chollet's ARC Challenge 3. Program Synthesis and ARC Challenge [00:12:55] 3.1 Philosophical Foundations of Program Synthesis [00:17:14] 3.2 Introduction to Program Induction and ARC Tasks [00:18:49] 3.3 DreamCoder: Principles and Techniques [00:27:55] 3.4 Trade-offs in Program Synthesis Search Strategies [00:31:52] 3.5 Neural Networks and Bayesian Program Learning 4. Advanced Program Synthesis Techniques [00:32:30] 4.1 DreamCoder and Dream Decompiling Approach [00:39:00] 4.2 Beta Distribution and Caching in Program Synthesis [00:45:10] 4.3 Performance and Limitations of Dream Decompiling [00:47:45] 4.4 Alessandro's Approach to ARC Challenge [00:51:12] 4.5 Conclusion and Future Discussions Refs: Full reflist on YT VD, Show Notes and MP3 metadata Show Notes: https://www.dropbox.com/scl/fi/x50201tgqucj5ba2q4typ/Ale.pdf?rlkey=0ubvk7p5gtyx1gpownpdadim8&st=5pniu3nq&dl=0

François Chollet discusses the limitations of Large Language Models (LLMs) and proposes a new approach to advancing artificial intelligence. He argues that current AI systems excel at pattern recognition but struggle with logical reasoning and true generalization. This was Chollet's keynote talk at AGI-24, filmed in high-quality. We will be releasing a full interview with him shortly. A teaser clip from that is played in the intro! Chollet introduces the Abstraction and Reasoning Corpus (ARC) as a benchmark for measuring AI progress towards human-like intelligence. He explains the concept of abstraction in AI systems and proposes combining deep learning with program synthesis to overcome current limitations. Chollet suggests that breakthroughs in AI might come from outside major tech labs and encourages researchers to explore new ideas in the pursuit of artificial general intelligence. TOC 1. LLM Limitations and Intelligence Concepts [00:00:00] 1.1 LLM Limitations and Composition [00:12:05] 1.2 Intelligence as Process vs. Skill [00:17:15] 1.3 Generalization as Key to AI Progress 2. ARC-AGI Benchmark and LLM Performance [00:19:59] 2.1 Introduction to ARC-AGI Benchmark [00:20:05] 2.2 Introduction to ARC-AGI and the ARC Prize [00:23:35] 2.3 Performance of LLMs and Humans on ARC-AGI 3. Abstraction in AI Systems [00:26:10] 3.1 The Kaleidoscope Hypothesis and Abstraction Spectrum [00:30:05] 3.2 LLM Capabilities and Limitations in Abstraction [00:32:10] 3.3 Value-Centric vs Program-Centric Abstraction [00:33:25] 3.4 Types of Abstraction in AI Systems 4. Advancing AI: Combining Deep Learning and Program Synthesis [00:34:05] 4.1 Limitations of Transformers and Need for Program Synthesis [00:36:45] 4.2 Combining Deep Learning and Program Synthesis [00:39:59] 4.3 Applying Combined Approaches to ARC Tasks [00:44:20] 4.4 State-of-the-Art Solutions for ARC Shownotes (new!): https://www.dropbox.com/scl/fi/i7nsyoahuei6np95lbjxw/CholletKeynote.pdf?rlkey=t3502kbov5exsdxhderq70b9i&st=1ca91ewz&dl=0 [0:01:15] Abstraction and Reasoning Corpus (ARC): AI benchmark (François Chollet) https://arxiv.org/abs/1911.01547 [0:05:30] Monty Hall problem: Probability puzzle (Steve Selvin) https://www.tandfonline.com/doi/abs/10.1080/00031305.1975.10479121 [0:06:20] LLM training dynamics analysis (Tirumala et al.) https://arxiv.org/abs/2205.10770 [0:10:20] Transformer limitations on compositionality (Dziri et al.) https://arxiv.org/abs/2305.18654 [0:10:25] Reversal Curse in LLMs (Berglund et al.) https://arxiv.org/abs/2309.12288 [0:19:25] Measure of intelligence using algorithmic information theory (François Chollet) https://arxiv.org/abs/1911.01547 [0:20:10] ARC-AGI: GitHub repository (François Chollet) https://github.com/fchollet/ARC-AGI [0:22:15] ARC Prize: $1,000,000+ competition (François Chollet) https://arcprize.org/ [0:33:30] System 1 and System 2 thinking (Daniel Kahneman) https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555 [0:34:00] Core knowledge in infants (Elizabeth Spelke) https://www.harvardlds.org/wp-content/uploads/2017/01/SpelkeKinzler07-1.pdf [0:34:30] Embedding interpretive spaces in ML (Tennenholtz et al.) https://arxiv.org/abs/2310.04475 [0:44:20] Hypothesis Search with LLMs for ARC (Wang et al.) https://arxiv.org/abs/2309.05660 [0:44:50] Ryan Greenblatt's high score on ARC public leaderboard https://arcprize.org/

Ivan Zhang, co-founder of Cohere, discusses the company's enterprise-focused AI solutions. He explains Cohere's early emphasis on embedding technology and training models for secure environments. Zhang highlights their implementation of Retrieval-Augmented Generation in healthcare, significantly reducing doctor preparation time. He explores the shift from monolithic AI models to heterogeneous systems and the importance of improving various AI system components. Zhang shares insights on using synthetic data to teach models reasoning, the democratization of software development through AI, and how his gaming skills transfer to running an AI company. He advises young developers to fully embrace AI technologies and offers perspectives on AI reliability, potential risks, and future model architectures. https://cohere.com/ https://ivanzhang.ca/ https://x.com/1vnzh TOC: 00:00:00 Intro 00:03:20 AI & Language Model Evolution 00:06:09 Future AI Apps & Development 00:09:29 Impact on Software Dev Practices 00:13:03 Philosophical & Societal Implications 00:16:30 Compute Efficiency & RAG 00:20:39 Adoption Challenges & Solutions 00:22:30 GPU Optimization & Kubernetes Limits 00:24:16 Cohere's Implementation Approach 00:28:13 Gaming's Professional Influence 00:34:45 Transformer Optimizations 00:36:45 Future Models & System-Level Focus 00:39:20 Inference-Time Computation & Reasoning 00:42:05 Capturing Human Thought in AI 00:43:15 Research, Hiring & Developer Advice REFS: 00:02:31 Cohere, https://cohere.com/ 00:02:40 The Transformer architecture, https://arxiv.org/abs/1706.03762 00:03:22 The Innovator's Dilemma, https://www.amazon.com/Innovators-Dilemma-Technologies-Management-Innovation/dp/1633691780 00:09:15 The actor model, https://en.wikipedia.org/wiki/Actor_model 00:14:35 John Searle's Chinese Room Argument, https://plato.stanford.edu/entries/chinese-room/ 00:18:00 Retrieval-Augmented Generation, https://arxiv.org/abs/2005.11401 00:18:40 Retrieval-Augmented Generation, https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag 00:35:39 Let’s Verify Step by Step, https://arxiv.org/pdf/2305.20050 00:39:20 Adaptive Inference-Time Compute, https://arxiv.org/abs/2410.02725 00:43:20 Ryan Greenblatt ARC entry, https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt Disclaimer: This show is part of our Cohere partnership series

Prof. Tim Rocktäschel, AI researcher at UCL and Google DeepMind, talks about open-ended AI systems. These systems aim to keep learning and improving on their own, like evolution does in nature. Ad: Are you a hardcore ML engineer who wants to work for Daniel Cahn at SlingshotAI building AI for mental health? Give him an email! - [email protected] TOC: 00:00:00 Introduction to Open-Ended AI and Key Concepts 00:01:37 Tim Rocktäschel's Background and Research Focus 00:06:25 Defining Open-Endedness in AI Systems 00:10:39 Subjective Nature of Interestingness and Learnability 00:16:22 Open-Endedness in Practice: Examples and Limitations 00:17:50 Assessing Novelty in Open-ended AI Systems 00:20:05 Adversarial Attacks and AI Robustness 00:24:05 Rainbow Teaming and LLM Safety 00:25:48 Open-ended Research Approaches in AI 00:29:05 Balancing Long-term Vision and Exploration in AI Research 00:37:25 LLMs in Program Synthesis and Open-Ended Learning 00:37:55 Transition from Human-Based to Novel AI Strategies 00:39:00 Expanding Context Windows and Prompt Evolution 00:40:17 AI Intelligibility and Human-AI Interfaces 00:46:04 Self-Improvement and Evolution in AI Systems Show notes (New!) https://www.dropbox.com/scl/fi/5avpsyz8jbn4j1az7kevs/TimR.pdf?rlkey=pqjlcqbtm3undp4udtgfmie8n&st=x50u1d1m&dl=0 REFS: 00:01:47 - UCL DARK Lab (Rocktäschel) - AI research lab focusing on RL and open-ended learning - https://ucldark.com/ 00:02:31 - GENIE (Bruce) - Generative interactive environment from unlabelled videos - https://arxiv.org/abs/2402.15391 00:02:42 - Promptbreeder (Fernando) - Self-referential LLM prompt evolution - https://arxiv.org/abs/2309.16797 00:03:05 - Picbreeder (Secretan) - Collaborative online image evolution - https://dl.acm.org/doi/10.1145/1357054.1357328 00:03:14 - Why Greatness Cannot Be Planned (Stanley) - Book on open-ended exploration - https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 00:04:36 - NetHack Learning Environment (Küttler) - RL research in procedurally generated game - https://arxiv.org/abs/2006.13760 00:07:35 - Open-ended learning (Clune) - AI systems for continual learning and adaptation - https://arxiv.org/abs/1905.10985 00:07:35 - OMNI (Zhang) - LLMs modeling human interestingness for exploration - https://arxiv.org/abs/2306.01711 00:10:42 - Observer theory (Wolfram) - Computationally bounded observers in complex systems - https://writings.stephenwolfram.com/2023/12/observer-theory/ 00:15:25 - Human-Timescale Adaptation (Rocktäschel) - RL agent adapting to novel 3D tasks - https://arxiv.org/abs/2301.07608 00:16:15 - Open-Endedness for AGI (Hughes) - Importance of open-ended learning for AGI - https://arxiv.org/abs/2406.04268 00:16:35 - POET algorithm (Wang) - Open-ended approach to generate and solve challenges - https://arxiv.org/abs/1901.01753 00:17:20 - AlphaGo (Silver) - AI mastering the game of Go - https://deepmind.google/technologies/alphago/ 00:20:35 - Adversarial Go attacks (Dennis) - Exploiting weaknesses in Go AI systems - https://www.ifaamas.org/Proceedings/aamas2024/pdfs/p1630.pdf 00:22:00 - Levels of AGI (Morris) - Framework for categorizing AGI progress - https://arxiv.org/abs/2311.02462 00:24:30 - Rainbow Teaming (Samvelyan) - LLM-based adversarial prompt generation - https://arxiv.org/abs/2402.16822 00:25:50 - Why Greatness Cannot Be Planned (Stanley) - 'False compass' and 'stepping stone collection' concepts - https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 00:27:45 - AI Debate (Khan) - Improving LLM truthfulness through debate - https://proceedings.mlr.press/v235/khan24a.html 00:29:40 - Gemini (Google DeepMind) - Advanced multimodal AI model - https://deepmind.google/technologies/gemini/ 00:30:15 - How to Take Smart Notes (Ahrens) - Effective note-taking methodology - https://www.amazon.com/How-Take-Smart-Notes-Nonfiction/dp/1542866502 (truncated, see shownotes)

Ben Goertzel discusses AGI development, transhumanism, and the potential societal impacts of superintelligent AI. He predicts human-level AGI by 2029 and argues that the transition to superintelligence could happen within a few years after. Goertzel explores the challenges of AI regulation, the limitations of current language models, and the need for neuro-symbolic approaches in AGI research. He also addresses concerns about resource allocation and cultural perspectives on transhumanism. TOC: [00:00:00] AGI Timeline Predictions and Development Speed [00:00:45] Limitations of Language Models in AGI Development [00:02:18] Current State and Trends in AI Research and Development [00:09:02] Emergent Reasoning Capabilities and Limitations of LLMs [00:18:15] Neuro-Symbolic Approaches and the Future of AI Systems [00:20:00] Evolutionary Algorithms and LLMs in Creative Tasks [00:21:25] Symbolic vs. Sub-Symbolic Approaches in AI [00:28:05] Language as Internal Thought and External Communication [00:30:20] AGI Development and Goal-Directed Behavior [00:35:51] Consciousness and AI: Expanding States of Experience [00:48:50] AI Regulation: Challenges and Approaches [00:55:35] Challenges in AI Regulation [00:59:20] AI Alignment and Ethical Considerations [01:09:15] AGI Development Timeline Predictions [01:12:40] OpenCog Hyperon and AGI Progress [01:17:48] Transhumanism and Resource Allocation Debate [01:20:12] Cultural Perspectives on Transhumanism [01:23:54] AGI and Post-Scarcity Society [01:31:35] Challenges and Implications of AGI Development New! PDF Show notes: https://www.dropbox.com/scl/fi/fyetzwgoaf70gpovyfc4x/BenGoertzel.pdf?rlkey=pze5dt9vgf01tf2wip32p5hk5&st=svbcofm3&dl=0 Refs: 00:00:15 Ray Kurzweil's AGI timeline prediction, Ray Kurzweil, https://en.wikipedia.org/wiki/Technological_singularity 00:01:45 Ben Goertzel: SingularityNET founder, Ben Goertzel, https://singularitynet.io/ 00:02:35 AGI Conference series, AGI Conference Organizers, https://agi-conf.org/2024/ 00:03:55 Ben Goertzel's contributions to AGI, Wikipedia contributors, https://en.wikipedia.org/wiki/Ben_Goertzel 00:11:05 Chain-of-Thought prompting, Subbarao Kambhampati, https://arxiv.org/abs/2405.04776 00:11:35 Algorithmic information content, Pieter Adriaans, https://plato.stanford.edu/entries/information-entropy/ 00:12:10 Turing completeness in neural networks, Various contributors, https://plato.stanford.edu/entries/turing-machine/ 00:16:15 AlphaGeometry: AI for geometry problems, Trieu, Li, et al., https://www.nature.com/articles/s41586-023-06747-5 00:18:25 Shane Legg and Ben Goertzel's collaboration, Shane Legg, https://en.wikipedia.org/wiki/Shane_Legg 00:20:00 Evolutionary algorithms in music generation, Yanxu Chen, https://arxiv.org/html/2409.03715v1 00:22:00 Peirce's theory of semiotics, Charles Sanders Peirce, https://plato.stanford.edu/entries/peirce-semiotics/ 00:28:10 Chomsky's view on language, Noam Chomsky, https://chomsky.info/1983____/ 00:34:05 Greg Egan's 'Diaspora', Greg Egan, https://www.amazon.co.uk/Diaspora-post-apocalyptic-thriller-perfect-MIRROR/dp/0575082097 00:40:35 'The Consciousness Explosion', Ben Goertzel & Gabriel Axel Montes, https://www.amazon.com/Consciousness-Explosion-Technological-Experiential-Singularity/dp/B0D8C7QYZD 00:41:55 Ray Kurzweil's books on singularity, Ray Kurzweil, https://www.amazon.com/Singularity-Near-Humans-Transcend-Biology/dp/0143037889 00:50:50 California AI regulation bills, California State Senate, https://sd18.senate.ca.gov/news/senate-unanimously-approves-senator-padillas-artificial-intelligence-package 00:56:40 Limitations of Compute Thresholds, Sara Hooker, https://arxiv.org/abs/2407.05694 00:56:55 'Taming Silicon Valley', Gary F. Marcus, https://www.penguinrandomhouse.com/books/768076/taming-silicon-valley-by-gary-f-marcus/ 01:09:15 Kurzweil's AGI prediction update, Ray Kurzweil, https://www.theguardian.com/technology/article/2024/jun/29/ray-kurzweil-google-ai-the-singularity-is-nearer

AI expert Prof. Gary Marcus doesn't mince words about today's artificial intelligence. He argues that despite the buzz, chatbots like ChatGPT aren't as smart as they seem and could cause real problems if we're not careful. Marcus is worried about tech companies putting profits before people. He thinks AI could make fake news and privacy issues even worse. He's also concerned that a few big tech companies have too much power. Looking ahead, Marcus believes the AI hype will die down as reality sets in. He wants to see AI developed in smarter, more responsible ways. His message to the public? We need to speak up and demand better AI before it's too late. Buy Taming Silicon Valley: https://amzn.to/3XTlC5s Gary Marcus: https://garymarcus.substack.com/ https://x.com/GaryMarcus Interviewer: Dr. Tim Scarfe (Refs in top comment) TOC [00:00:00] AI Flaws, Improvements & Industry Critique [00:16:29] AI Safety Theater & Image Generation Issues [00:23:49] AI's Lack of World Models & Human-like Understanding [00:31:09] LLMs: Superficial Intelligence vs. True Reasoning [00:34:45] AI in Specialized Domains: Chess, Coding & Limitations [00:42:10] AI-Generated Code: Capabilities & Human-AI Interaction [00:48:10] AI Regulation: Industry Resistance & Oversight Challenges [00:54:55] Copyright Issues in AI & Tech Business Models [00:57:26] AI's Societal Impact: Risks, Misinformation & Ethics [01:23:14] AI X-risk, Alignment & Moral Principles Implementation [01:37:10] Persistent AI Flaws: System Limitations & Architecture Challenges [01:44:33] AI Future: Surveillance Concerns, Economic Challenges & Neuro-Symbolic AI YT version with refs: https://youtu.be/o9MfuUoGlSw

Prof. Mark Solms, a neuroscientist and psychoanalyst, discusses his groundbreaking work on consciousness, challenging conventional cortex-centric views and emphasizing the role of brainstem structures in generating consciousness and affect. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Key points discussed: The limitations of vision-centric approaches to consciousness studies. Evidence from decorticated animals and hydranencephalic children supporting the brainstem's role in consciousness. The relationship between homeostasis, the free energy principle, and consciousness. Critiques of behaviorism and modern theories of consciousness. The importance of subjective experience in understanding brain function. The discussion also explored broader topics: The potential impact of affect-based theories on AI development. The role of the SEEKING system in exploration and learning. Connections between neuroscience, psychoanalysis, and philosophy of mind. Challenges in studying consciousness and the limitations of current theories. Mark Solms: https://neuroscience.uct.ac.za/contacts/mark-solms Show notes and transcript: https://www.dropbox.com/scl/fo/roipwmnlfmwk2e7kivzms/ACjZF-VIGC2-Suo30KcwVV0?rlkey=53y8v2cajfcgrf17p1h7v3suz&st=z8vu81hn&dl=0 TOC (*) are best bits 00:00:00 1. Intro: Challenging vision-centric approaches to consciousness * 00:02:20 2. Evidence from decorticated animals and hydranencephalic children * 00:07:40 3. Emotional responses in hydranencephalic children 00:10:40 4. Brainstem stimulation and affective states 00:15:00 5. Brainstem's role in generating affective consciousness * 00:21:50 6. Dual-aspect monism and the mind-brain relationship 00:29:37 7. Information, affect, and the hard problem of consciousness * 00:37:25 8. Wheeler's participatory universe and Chalmers' theories 00:48:51 9. Homeostasis, free energy principle, and consciousness * 00:59:25 10. Affect, voluntary behavior, and decision-making 01:05:45 11. Psychoactive substances, REM sleep, and consciousness research 01:12:14 12. Critiquing behaviorism and modern consciousness theories * 01:24:25 13. The SEEKING system and exploration in neuroscience Refs: 1. Mark Solms' book "The Hidden Spring" [00:20:34] (MUST READ!) https://amzn.to/3XyETb3 2. Karl Friston's free energy principle [00:03:50] https://www.nature.com/articles/nrn2787 3. Hydranencephaly condition [00:07:10] https://en.wikipedia.org/wiki/Hydranencephaly 4. Periaqueductal gray (PAG) [00:08:57] https://en.wikipedia.org/wiki/Periaqueductal_gray 5. Positron Emission Tomography (PET) [00:13:52] https://en.wikipedia.org/wiki/Positron_emission_tomography 6. Paul MacLean's triune brain theory [00:03:30] https://en.wikipedia.org/wiki/Triune_brain 7. Baruch Spinoza's philosophy of mind [00:23:48] https://plato.stanford.edu/entries/spinoza-epistemology-mind 8. Claude Shannon's "A Mathematical Theory of Communication" [00:32:15] https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf 9. Francis Crick's "The Astonishing Hypothesis" [00:39:57] https://en.wikipedia.org/wiki/The_Astonishing_Hypothesis 10. Frank Jackson's Knowledge Argument [00:40:54] https://plato.stanford.edu/entries/qualia-knowledge/ 11. Mesolimbic dopamine system [01:11:51] https://en.wikipedia.org/wiki/Mesolimbic_pathway 12. Jaak Panksepp's SEEKING system [01:25:23] https://en.wikipedia.org/wiki/Jaak_Panksepp#Affective_neuroscience

Dr. Patrick Lewis, who coined the term RAG (Retrieval Augmented Generation) and now works at Cohere, discusses the evolution of language models, RAG systems, and challenges in AI evaluation. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmented generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Key topics covered: - Origins and evolution of Retrieval Augmented Generation (RAG) - Challenges in evaluating RAG systems and language models - Human-AI collaboration in research and knowledge work - Word embeddings and the progression to modern language models - Dense vs sparse retrieval methods in information retrieval The discussion also explored broader implications and applications: - Balancing faithfulness and fluency in RAG systems - User interface design for AI-augmented research tools - The journey from chemistry to AI research - Challenges in enterprise search compared to web search - The importance of data quality in training AI models Patrick Lewis: https://www.patricklewis.io/ Cohere Command Models, check them out - they are amazing for RAG! https://cohere.com/command TOC 00:00:00 1. Intro to RAG 00:05:30 2. RAG Evaluation: Poll framework & model performance 00:12:55 3. Data Quality: Cleanliness vs scale in AI training 00:15:13 4. Human-AI Collaboration: Research agents & UI design 00:22:57 5. RAG Origins: Open-domain QA to generative models 00:30:18 6. RAG Challenges: Info retrieval, tool use, faithfulness 00:42:01 7. Dense vs Sparse Retrieval: Techniques & trade-offs 00:47:02 8. RAG Applications: Grounding, attribution, hallucination prevention 00:54:04 9. UI for RAG: Human-computer interaction & model optimization 00:59:01 10. Word Embeddings: Word2Vec, GloVe, and semantic spaces 01:06:43 11. Language Model Evolution: BERT, GPT, and beyond 01:11:38 12. AI & Human Cognition: Sequential processing & chain-of-thought Refs: 1. Retrieval Augmented Generation (RAG) paper / Patrick Lewis et al. [00:27:45] https://arxiv.org/abs/2005.11401 2. LAMA (LAnguage Model Analysis) probe / Petroni et al. [00:26:35] https://arxiv.org/abs/1909.01066 3. KILT (Knowledge Intensive Language Tasks) benchmark / Petroni et al. [00:27:05] https://arxiv.org/abs/2009.02252 4. Word2Vec algorithm / Tomas Mikolov et al. [01:00:25] https://arxiv.org/abs/1301.3781 5. GloVe (Global Vectors for Word Representation) / Pennington et al. [01:04:35] https://nlp.stanford.edu/projects/glove/ 6. BERT (Bidirectional Encoder Representations from Transformers) / Devlin et al. [01:08:00] https://arxiv.org/abs/1810.04805 7. 'The Language Game' book / Nick Chater and Morten H. Christiansen [01:11:40] https://amzn.to/4grEUpG Disclaimer: This is the sixth video from our Cohere partnership. We were not told what to say in the interview. Filmed in Seattle in June 2024.

Ashley Edwards, who was working at DeepMind when she co-authored the Genie paper and is now at Runway, covered several key aspects of the Genie AI system and its applications in video generation, robotics, and game creation. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Genie's approach to learning interactive environments, balancing compression and fidelity. The use of latent action models and VQE models for video processing and tokenization. Challenges in maintaining action consistency across frames and integrating text-to-image models. Evaluation metrics for AI-generated content, such as FID and PS&R diff metrics. The discussion also explored broader implications and applications: The potential impact of AI video generation on content creation jobs. Applications of Genie in game generation and robotics. The use of foundation models in robotics and the differences between internet video data and specialized robotics data. Challenges in mapping AI-generated actions to real-world robotic actions. Ashley Edwards: https://ashedwards.github.io/ TOC (*) are best bits 00:00:00 1. Intro to Genie & Brave Search API: Trade-offs & limitations * 00:02:26 2. Genie's Architecture: Latent action, VQE, video processing * 00:05:06 3. Genie's Constraints: Frame consistency & image model integration 00:07:26 4. Evaluation: FID, PS&R diff metrics & latent induction methods 00:09:44 5. AI Video Gen: Content creation impact, depth & parallax effects 00:11:39 6. Model Scaling: Training data impact & computational trade-offs 00:13:50 7. Game & Robotics Apps: Gamification & action mapping challenges * 00:16:16 8. Robotics Foundation Models: Action space & data considerations * 00:19:18 9. Mask-GPT & Video Frames: Real-time optimization, RL from videos 00:20:34 10. Research Challenges: AI value, efficiency vs. quality, safety 00:24:20 11. Future Dev: Efficiency improvements & fine-tuning strategies Refs: 1. Genie (learning interactive environments from videos) / Ashley and DM collegues [00:01] https://arxiv.org/abs/2402.15391 2. VQ-VAE (Vector Quantized Variational Autoencoder) / Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu [02:43] https://arxiv.org/abs/1711.00937 3. FID (Fréchet Inception Distance) metric / Martin Heusel et al. [07:37] https://arxiv.org/abs/1706.08500 4. PS&R (Precision and Recall) metric / Mehdi S. M. Sajjadi et al. [08:02] https://arxiv.org/abs/1806.00035 5. Vision Transformer (ViT) architecture / Alexey Dosovitskiy et al. [12:14] https://arxiv.org/abs/2010.11929 6. Genie (robotics foundation models) / Google DeepMind [17:34] https://deepmind.google/research/publications/60474/ 7. Chelsea Finn's lab work on robotics datasets / Chelsea Finn [17:38] https://ai.stanford.edu/~cbfinn/ 8. Imitation from observation in reinforcement learning / YuXuan Liu [20:58] https://arxiv.org/abs/1707.03374 9. Waymo's autonomous driving technology / Waymo [22:38] https://waymo.com/ 10. Gen3 model release by Runway / Runway [23:48] https://runwayml.com/ 11. Classifier-free guidance technique / Jonathan Ho and Tim Salimans [24:43] https://arxiv.org/abs/2207.12598

Saurabh Baji discusses Cohere's approach to developing and deploying large language models (LLMs) for enterprise use. * Cohere focuses on pragmatic, efficient models tailored for business applications rather than pursuing the largest possible models. * They offer flexible deployment options, from cloud services to on-premises installations, to meet diverse enterprise needs. * Retrieval-augmented generation (RAG) is highlighted as a critical capability, allowing models to leverage enterprise data securely. * Cohere emphasizes model customization, fine-tuning, and tools like reranking to optimize performance for specific use cases. * The company has seen significant growth, transitioning from developer-focused to enterprise-oriented services. * Major customers like Oracle, Fujitsu, and TD Bank are using Cohere's models across various applications, from HR to finance. * Baji predicts a surge in enterprise AI adoption over the next 12-18 months as more companies move from experimentation to production. * He emphasizes the importance of trust, security, and verifiability in enterprise AI applications. The interview provides insights into Cohere's strategy, technology, and vision for the future of enterprise AI adoption. https://www.linkedin.com/in/saurabhbaji/ https://x.com/sbaji https://cohere.com/ https://cohere.com/business MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. TOC (*) are best bits 00:00:00 1. Introduction and Background 00:04:24 2. Cloud Infrastructure and LLM Optimization 00:06:43 2.1 Model deployment and fine-tuning strategies * 00:09:37 3. Enterprise AI Deployment Strategies 00:11:10 3.1 Retrieval-augmented generation in enterprise environments * 00:13:40 3.2 Standardization vs. customization in cloud services * 00:18:20 4. AI Model Evaluation and Deployment 00:18:20 4.1 Comprehensive evaluation frameworks * 00:21:20 4.2 Key components of AI model stacks * 00:25:50 5. Retrieval Augmented Generation (RAG) in Enterprise 00:32:10 5.1 Pragmatic approach to RAG implementation * 00:33:45 6. AI Agents and Tool Integration 00:33:45 6.1 Leveraging tools for AI insights * 00:35:30 6.2 Agent-based AI systems and diagnostics * 00:42:55 7. AI Transparency and Reasoning Capabilities 00:49:10 8. AI Model Training and Customization 00:57:10 9. Enterprise AI Model Management 01:02:10 9.1 Managing AI model versions for enterprise customers * 01:04:30 9.2 Future of language model programming * 01:06:10 10. AI-Driven Software Development 01:06:10 10.1 AI bridging human expression and task achievement * 01:08:00 10.2 AI-driven virtual app fabrics in enterprise * 01:13:33 11. Future of AI and Enterprise Applications 01:21:55 12. Cohere's Customers and Use Cases 01:21:55 12.1 Cohere's growth and enterprise partnerships * 01:27:14 12.2 Diverse customers using generative AI * 01:27:50 12.3 Industry adaptation to generative AI * 01:29:00 13. Technical Advantages of Cohere Models 01:29:00 13.1 Handling large context windows * 01:29:40 13.2 Low latency impact on developer productivity * Disclaimer: This is the fifth video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview. Filmed in Seattle in Aug 2024.

David Hanson, CEO of Hanson Robotics and creator of the humanoid robot Sofia, explores the intersection of artificial intelligence, ethics, and human potential. In this thought-provoking interview, Hanson discusses his vision for developing AI systems that embody the best aspects of humanity while pushing beyond our current limitations, aiming to achieve what he calls "super wisdom." YT version: https://youtu.be/LFCIEhlsozU MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. The interview with David Hanson covers: The importance of incorporating biological drives and compassion into AI systems Hanson's concept of "existential pattern ethics" as a basis for AI morality The potential for AI to enhance human intelligence and wisdom Challenges in developing artificial general intelligence (AGI) The need to democratize AI technologies globally Potential future advancements in human-AI integration and their societal impacts Concerns about technological augmentation exacerbating inequality The role of ethics in guiding AI development and deployment Hanson advocates for creating AI systems that embody the best aspects of humanity while surpassing current human limitations, aiming for "super wisdom" rather than just artificial super intelligence. David Hanson: https://www.hansonrobotics.com/david-hanson/ https://www.youtube.com/watch?v=9u1O954cMmE TOC 1. Introduction and Background [00:00:00] 1.1. David Hanson's interdisciplinary background [0:01:49] 1.2. Introduction to Sofia, the realistic robot [0:03:27] 2. Human Cognition and AI [0:03:50] 2.1. Importance of social interaction in cognition [0:03:50] 2.2. Compassion as distinguishing factor [0:05:55] 2.3. AI augmenting human intelligence [0:09:54] 3. Developing Human-like AI [0:13:17] 3.1. Incorporating biological drives in AI [0:13:17] 3.2. Creating AI with agency [0:20:34] 3.3. Implementing flexible desires in AI [0:23:23] 4. Ethics and Morality in AI [0:27:53] 4.1. Enhancing humanity through AI [0:27:53] 4.2. Existential pattern ethics [0:30:14] 4.3. Expanding morality beyond restrictions [0:35:35] 5. Societal Impact of AI [0:38:07] 5.1. AI adoption and integration [0:38:07] 5.2. Democratizing AI technologies [0:38:32] 5.3. Human-AI integration and identity [0:43:37] 6. Future Considerations [0:50:03] 6.1. Technological augmentation and inequality [0:50:03] 6.2. Emerging technologies for mental health [0:50:32] 6.3. Corporate ethics in AI development [0:52:26] This was filmed at AGI-24

David Spivak, a mathematician known for his work in category theory, discusses a wide range of topics related to intelligence, creativity, and the nature of knowledge. He explains category theory in simple terms and explores how it relates to understanding complex systems and relationships. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. We discuss abstract concepts like collective intelligence, the importance of embodiment in understanding the world, and how we acquire and process knowledge. Spivak shares his thoughts on creativity, discussing where it comes from and how it might be modeled mathematically. A significant portion of the discussion focuses on the impact of artificial intelligence on human thinking and its potential role in the evolution of intelligence. Spivak also touches on the importance of language, particularly written language, in transmitting knowledge and shaping our understanding of the world. David Spivak http://www.dspivak.net/ TOC: 00:00:00 Introduction to category theory and functors 00:04:40 Collective intelligence and sense-making 00:09:54 Embodiment and physical concepts in knowledge acquisition 00:16:23 Creativity, open-endedness, and AI's impact on thinking 00:25:46 Modeling creativity and the evolution of intelligence 00:36:04 Evolution, optimization, and the significance of AI 00:44:14 Written language and its impact on knowledge transmission REFS: Mike Levin's work https://scholar.google.com/citations?user=luouyakAAAAJ&hl=en Eric Smith's videos on complexity and early life https://www.youtube.com/watch?v=SpJZw-68QyE Richard Dawkins' book "The Selfish Gene" https://amzn.to/3X73X8w Carl Sagan's statement about the cosmos knowing itself https://amzn.to/3XhPruK Herbert Simon's concept of "satisficing" https://plato.stanford.edu/entries/bounded-rationality/ DeepMind paper on open-ended systems https://arxiv.org/abs/2406.04268 Karl Friston's work on active inference https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind MIT category theory lectures by David Spivak (available on the Topos Institute channel) https://www.youtube.com/watch?v=UusLtx9fIjs

Jürgen Schmidhuber, the father of generative AI shares his groundbreaking work in deep learning and artificial intelligence. In this exclusive interview, he discusses the history of AI, some of his contributions to the field, and his vision for the future of intelligent machines. Schmidhuber offers unique insights into the exponential growth of technology and the potential impact of AI on humanity and the universe. YT version: https://youtu.be/DP454c1K_vQ MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. TOC 00:00:00 Intro 00:03:38 Reasoning 00:13:09 Potential AI Breakthroughs Reducing Computation Needs 00:20:39 Memorization vs. Generalization in AI 00:25:19 Approach to the ARC Challenge 00:29:10 Perceptions of Chat GPT and AGI 00:58:45 Abstract Principles of Jurgen's Approach 01:04:17 Analogical Reasoning and Compression 01:05:48 Breakthroughs in 1991: the P, the G, and the T in ChatGPT and Generative AI 01:15:50 Use of LSTM in Language Models by Tech Giants 01:21:08 Neural Network Aspect Ratio Theory 01:26:53 Reinforcement Learning Without Explicit Teachers Refs: ★ "Annotated History of Modern AI and Deep Learning" (2022 survey by Schmidhuber): ★ Chain Rule For Backward Credit Assignment (Leibniz, 1676) ★ First Neural Net / Linear Regression / Shallow Learning (Gauss & Legendre, circa 1800) ★ First 20th Century Pioneer of Practical AI (Quevedo, 1914) ★ First Recurrent NN (RNN) Architecture (Lenz, Ising, 1920-1925) ★ AI Theory: Fundamental Limitations of Computation and Computation-Based AI (Gödel, 1931-34) ★ Unpublished ideas about evolving RNNs (Turing, 1948) ★ Multilayer Feedforward NN Without Deep Learning (Rosenblatt, 1958) ★ First Published Learning RNNs (Amari and others, ~1972) ★ First Deep Learning (Ivakhnenko & Lapa, 1965) ★ Deep Learning by Stochastic Gradient Descent (Amari, 1967-68) ★ ReLUs (Fukushima, 1969) ★ Backpropagation (Linnainmaa, 1970); precursor (Kelley, 1960) ★ Backpropagation for NNs (Werbos, 1982) ★ First Deep Convolutional NN (Fukushima, 1979); later combined with Backprop (Waibel 1987, Zhang 1988). ★ Metalearning or Learning to Learn (Schmidhuber, 1987) ★ Generative Adversarial Networks / Artificial Curiosity / NN Online Planners (Schmidhuber, Feb 1990; see the G in Generative AI and ChatGPT) ★ NNs Learn to Generate Subgoals and Work on Command (Schmidhuber, April 1990) ★ NNs Learn to Program NNs: Unnormalized Linear Transformer (Schmidhuber, March 1991; see the T in ChatGPT) ★ Deep Learning by Self-Supervised Pre-Training. Distilling NNs (Schmidhuber, April 1991; see the P in ChatGPT) ★ Experiments with Pre-Training; Analysis of Vanishing/Exploding Gradients, Roots of Long Short-Term Memory / Highway Nets / ResNets (Hochreiter, June 1991, further developed 1999-2015 with other students of Schmidhuber) ★ LSTM journal paper (1997, most cited AI paper of the 20th century) ★ xLSTM (Hochreiter, 2024) ★ Reinforcement Learning Prompt Engineer for Abstract Reasoning and Planning (Schmidhuber 2015) ★ Mindstorms in Natural Language-Based Societies of Mind (2023 paper by Schmidhuber's team) https://arxiv.org/abs/2305.17066 ★ Bremermann's physical limit of computation (1982) EXTERNAL LINKS CogX 2018 - Professor Juergen Schmidhuber https://www.youtube.com/watch?v=17shdT9-wuA Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability (Neural Networks, 1997) https://sferics.idsia.ch/pub/juergen/loconet.pdf The paradox at the heart of mathematics: Gödel's Incompleteness Theorem - Marcus du Sautoy https://www.youtube.com/watch?v=I4pQbo5MQOs (Refs truncated, full version on YT VD)

Professor Pedro Domingos, is an AI researcher and professor of computer science. He expresses skepticism about current AI regulation efforts and argues for faster AI development rather than slowing it down. He also discusses the need for new innovations to fulfil the promises of current AI techniques. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmented generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Show notes: * Domingos' views on AI regulation and why he believes it's misguided * His thoughts on the current state of AI technology and its limitations * Discussion of his novel "2040", a satirical take on AI and tech culture * Explanation of his work on "tensor logic", which aims to unify neural networks and symbolic AI * Critiques of other approaches in AI, including those of OpenAI and Gary Marcus * Thoughts on the AI "bubble" and potential future developments in the field Prof. Pedro Domingos: https://x.com/pmddomingos 2040: A Silicon Valley Satire [Pedro's new book] https://amzn.to/3T51ISd TOC: 00:00:00 Intro 00:06:31 Bio 00:08:40 Filmmaking skit 00:10:35 AI and the wisdom of crowds 00:19:49 Social Media 00:27:48 Master algorithm 00:30:48 Neurosymbolic AI / abstraction 00:39:01 Language 00:45:38 Chomsky 01:00:49 2040 Book 01:18:03 Satire as a shield for criticism? 01:29:12 AI Regulation 01:35:15 Gary Marcus 01:52:37 Copyright 01:56:11 Stochastic parrots come home to roost 02:00:03 Privacy 02:01:55 LLM ecosystem 02:05:06 Tensor logic Refs: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World [Pedro Domingos] https://amzn.to/3MiWs9B Rebooting AI: Building Artificial Intelligence We Can Trust [Gary Marcus] https://amzn.to/3AAywvL Flash Boys [Michael Lewis] https://amzn.to/4dUGm1M

Andrew Ilyas, a PhD student at MIT who is about to start as a professor at CMU. We discuss Data modeling and understanding how datasets influence model predictions, Adversarial examples in machine learning and why they occur, Robustness in machine learning models, Black box attacks on machine learning systems, Biases in data collection and dataset creation, particularly in ImageNet and Self-selection bias in data and methods to address it. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api Andrew's site: https://andrewilyas.com/ https://x.com/andrew_ilyas TOC: 00:00:00 - Introduction and Andrew's background 00:03:52 - Overview of the machine learning pipeline 00:06:31 - Data modeling paper discussion 00:26:28 - TRAK: Evolution of data modeling work 00:43:58 - Discussion on abstraction, reasoning, and neural networks 00:53:16 - "Adversarial Examples Are Not Bugs, They Are Features" paper 01:03:24 - Types of features learned by neural networks 01:10:51 - Black box attacks paper 01:15:39 - Work on data collection and bias 01:25:48 - Future research plans and closing thoughts References: Adversarial Examples Are Not Bugs, They Are Features https://arxiv.org/pdf/1905.02175 TRAK: Attributing Model Behavior at Scale https://arxiv.org/pdf/2303.14186 Datamodels: Predicting Predictions from Training Data https://arxiv.org/pdf/2202.00622 Adversarial Examples Are Not Bugs, They Are Features https://arxiv.org/pdf/1905.02175 IMAGENET-TRAINED CNNS https://arxiv.org/pdf/1811.12231 ZOO: Zeroth Order Optimization Based Black-box https://arxiv.org/pdf/1708.03999 A Spline Theory of Deep Networks https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf Scaling Monosemanticity https://transformer-circuits.pub/2024/scaling-monosemanticity/ Adversarial Examples Are Not Bugs, They Are Features https://gradientscience.org/adv/ Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies https://proceedings.mlr.press/v235/bartoldson24a.html Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors https://arxiv.org/abs/1807.07978 Estimation of Standard Auction Models https://arxiv.org/abs/2205.02060 From ImageNet to Image Classification: Contextualizing Progress on Benchmarks https://arxiv.org/abs/2005.11295 Estimation of Standard Auction Models https://arxiv.org/abs/2205.02060 What Makes A Good Fisherman? Linear Regression under Self-Selection Bias https://arxiv.org/abs/2205.03246 Towards Tracing Factual Knowledge in Language Models Back to the Training Data [Akyürek] https://arxiv.org/pdf/2205.11482

Dr. Joscha Bach introduces a surprising idea called "cyber animism" in his AGI-24 talk - the notion that nature might be full of self-organizing software agents, similar to the spirits in ancient belief systems. Bach suggests that consciousness could be a kind of software running on our brains, and wonders if similar "programs" might exist in plants or even entire ecosystems. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Joscha takes us on a tour de force through history, philosophy, and cutting-edge computer science, teasing us to rethink what we know about minds, machines, and the world around us. Joscha believes we should blur the lines between human, artificial, and natural intelligence, and argues that consciousness might be more widespread and interconnected than we ever thought possible. Dr. Joscha Bach https://x.com/Plinz This is video 2/9 from our coverage of AGI-24 in Seattle https://agi-conf.org/2024/ Watch the official MLST interview with Joscha which we did right after this talk on our Patreon now on early access - https://www.patreon.com/posts/joscha-bach-110199676 (you also get access to our private discord and biweekly calls) TOC: 00:00:00 Introduction: AGI and Cyberanimism 00:03:57 The Nature of Consciousness 00:08:46 Aristotle's Concepts of Mind and Consciousness 00:13:23 The Hard Problem of Consciousness 00:16:17 Functional Definition of Consciousness 00:20:24 Comparing LLMs and Human Consciousness 00:26:52 Testing for Consciousness in AI Systems 00:30:00 Animism and Software Agents in Nature 00:37:02 Plant Consciousness and Ecosystem Intelligence 00:40:36 The California Institute for Machine Consciousness 00:44:52 Ethics of Conscious AI and Suffering 00:46:29 Philosophical Perspectives on Consciousness 00:49:55 Q&A: Formalisms for Conscious Systems 00:53:27 Coherence, Self-Organization, and Compute Resources YT version (very high quality, filmed by us live) https://youtu.be/34VOI_oo-qM Refs: Aristotle's work on the soul and consciousness Richard Dawkins' work on genes and evolution Gerald Edelman's concept of Neural Darwinism Thomas Metzinger's book "Being No One" Yoshua Bengio's concept of the "consciousness prior" Stuart Hameroff's theories on microtubules and consciousness Christof Koch's work on consciousness Daniel Dennett's "Cartesian Theater" concept Giulio Tononi's Integrated Information Theory Mike Levin's work on organismal intelligence The concept of animism in various cultures Freud's model of the mind Buddhist perspectives on consciousness and meditation The Genesis creation narrative (for its metaphorical interpretation) California Institute for Machine Consciousness

Prof Gary Marcus revisited his keynote from AGI-21, noting that many of the issues he highlighted then are still relevant today despite significant advances in AI. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Gary Marcus criticized current large language models (LLMs) and generative AI for their unreliability, tendency to hallucinate, and inability to truly understand concepts. Marcus argued that the AI field is experiencing diminishing returns with current approaches, particularly the "scaling hypothesis" that simply adding more data and compute will lead to AGI. He advocated for a hybrid approach to AI that combines deep learning with symbolic AI, emphasizing the need for systems with deeper conceptual understanding. Marcus highlighted the importance of developing AI with innate understanding of concepts like space, time, and causality. He expressed concern about the moral decline in Silicon Valley and the rush to deploy potentially harmful AI technologies without adequate safeguards. Marcus predicted a possible upcoming "AI winter" due to inflated valuations, lack of profitability, and overhyped promises in the industry. He stressed the need for better regulation of AI, including transparency in training data, full disclosure of testing, and independent auditing of AI systems. Marcus proposed the creation of national and global AI agencies to oversee the development and deployment of AI technologies. He concluded by emphasizing the importance of interdisciplinary collaboration, focusing on robust AI with deep understanding, and implementing smart, agile governance for AI and AGI. YT Version (very high quality filmed) https://youtu.be/91SK90SahHc Pre-order Gary's new book here: Taming Silicon Valley: How We Can Ensure That AI Works for Us https://amzn.to/4fO46pY Filmed at the AGI-24 conference: https://agi-conf.org/2024/ TOC: 00:00:00 Introduction 00:02:34 Introduction by Ben G 00:05:17 Gary Marcus begins talk 00:07:38 Critiquing current state of AI 00:12:21 Lack of progress on key AI challenges 00:16:05 Continued reliability issues with AI 00:19:54 Economic challenges for AI industry 00:25:11 Need for hybrid AI approaches 00:29:58 Moral decline in Silicon Valley 00:34:59 Risks of current generative AI 00:40:43 Need for AI regulation and governance 00:49:21 Concluding thoughts 00:54:38 Q&A: Cycles of AI hype and winters 01:00:10 Predicting a potential AI winter 01:02:46 Discussion on interdisciplinary approach 01:05:46 Question on regulating AI 01:07:27 Ben G's perspective on AI winter

DeepMind Research Scientist / MIT scholar Dr. Timothy Nguyen discusses his recent paper on understanding transformers through n-gram statistics. Nguyen explains his approach to analyzing transformer behavior using a kind of "template matching" (N-grams), providing insights into how these models process and predict language. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Key points covered include: A method for describing transformer predictions using n-gram statistics without relying on internal mechanisms. The discovery of a technique to detect overfitting in large language models without using holdout sets. Observations on curriculum learning, showing how transformers progress from simpler to more complex rules during training. Discussion of distance measures used in the analysis, particularly the variational distance. Exploration of model sizes, training dynamics, and their impact on the results. We also touch on philosophical aspects of describing versus explaining AI behavior, and the challenges in understanding the abstractions formed by neural networks. Nguyen concludes by discussing potential future research directions, including attempts to convert descriptions of transformer behavior into explanations of internal mechanisms. Timothy Nguyen's earned his B.S. and Ph.D. in mathematics from Caltech and MIT, respectively. He held positions as Research Assistant Professor at the Simons Center for Geometry and Physics (2011-2014) and Visiting Assistant Professor at Michigan State University (2014-2017). During this time, his research expanded into high-energy physics, focusing on mathematical problems in quantum field theory. His work notably provided a simplified and corrected formulation of perturbative path integrals. Since 2017, Nguyen has been working in industry, applying his expertise to machine learning. He is currently at DeepMind, where he contributes to both fundamental research and practical applications of deep learning to solve real-world problems. Refs: The Cartesian Cafe https://www.youtube.com/@TimothyNguyen Understanding Transformers via N-Gram Statistics https://www.researchgate.net/publication/382204056_Understanding_Transformers_via_N-Gram_Statistics TOC 00:00:00 Timothy Nguyen's background 00:02:50 Paper overview: transformers and n-gram statistics 00:04:55 Template matching and hash table approach 00:08:55 Comparing templates to transformer predictions 00:12:01 Describing vs explaining transformer behavior 00:15:36 Detecting overfitting without holdout sets 00:22:47 Curriculum learning in training 00:26:32 Distance measures in analysis 00:28:58 Model sizes and training dynamics 00:30:39 Future research directions 00:32:06 Conclusion and future topics

Jay Alammar, renowned AI educator and researcher at Cohere, discusses the latest developments in large language models (LLMs) and their applications in industry. Jay shares his expertise on retrieval augmented generation (RAG), semantic search, and the future of AI architectures. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Cohere Command R model series: https://cohere.com/command Jay Alamaar: https://x.com/jayalammar Buy Jay's new book here! Hands-On Large Language Models: Language Understanding and Generation https://amzn.to/4fzOUgh TOC: 00:00:00 Introduction to Jay Alammar and AI Education 00:01:47 Cohere's Approach to RAG and AI Re-ranking 00:07:15 Implementing AI in Enterprise: Challenges and Solutions 00:09:26 Jay's Role at Cohere and the Importance of Learning in Public 00:15:16 The Evolution of AI in Industry: From Deep Learning to LLMs 00:26:12 Expert Advice for Newcomers in Machine Learning 00:32:39 The Power of Semantic Search and Embeddings in AI Systems 00:37:59 Jay Alammar's Journey as an AI Educator and Visualizer 00:43:36 Visual Learning in AI: Making Complex Concepts Accessible 00:47:38 Strategies for Keeping Up with Rapid AI Advancements 00:49:12 The Future of Transformer Models and AI Architectures 00:51:40 Evolution of the Transformer: From 2017 to Present 00:54:19 Preview of Jay's Upcoming Book on Large Language Models Disclaimer: This is the fourth video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview. Note also that this combines several previously unpublished interviews from Jay into one, the earlier one at Tim's house was shot in Aug 2023, and the more recent one in Toronto in May 2024. Refs: The Illustrated Transformer https://jalammar.github.io/illustrated-transformer/ Attention Is All You Need https://arxiv.org/abs/1706.03762 The Unreasonable Effectiveness of Recurrent Neural Networks http://karpathy.github.io/2015/05/21/rnn-effectiveness/ Neural Networks in 11 Lines of Code https://iamtrask.github.io/2015/07/12/basic-python-network/ Understanding LSTM Networks (Chris Olah's blog post) http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Luis Serrano's YouTube Channel https://www.youtube.com/channel/UCgBncpylJ1kiVaPyP-PZauQ Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks https://arxiv.org/abs/1908.10084 GPT (Generative Pre-trained Transformer) models https://jalammar.github.io/illustrated-gpt2/ https://openai.com/research/gpt-4 BERT (Bidirectional Encoder Representations from Transformers) https://jalammar.github.io/illustrated-bert/ https://arxiv.org/abs/1810.04805 RoPE (Rotary Positional Encoding) https://arxiv.org/abs/2104.09864 (Linked paper discussing rotary embeddings) Grouped Query Attention https://arxiv.org/pdf/2305.13245 RLHF (Reinforcement Learning from Human Feedback) https://openai.com/research/learning-from-human-preferences https://arxiv.org/abs/1706.03741 DPO (Direct Preference Optimization) https://arxiv.org/abs/2305.18290

Daniel Cahn, co-founder of Slingshot AI, on the potential of AI in therapy. Why is anxiety and depression affecting a large population? To what extent are these real categories? Why is the mental health getting worse? How often do you want an AI to agree with you? What are the ethics of persuasive AI? You will discover all in this conversation. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. Daniel Cahn (who is also hiring ML engineers by the way!) https://x.com/thecahnartist?lang=en / cahnd https://thinkingmachinespodcast.com/ TOC: 00:00:00 Intro 00:01:56 Therapy effectiveness vs drugs and societal implications 00:04:02 Mental health categories: Iatrogenesis and social constructs 00:10:19 Psychiatric treatment models and cognitive perspectives 00:13:30 AI design and human-like interactions: Intentionality debates 00:20:04 AI in therapy: Ethics, anthropomorphism, and loneliness mitigation 00:28:13 Therapy efficacy: Neuroplasticity, suffering, and AI placebos 00:33:29 AI's impact on human agency and cognitive modeling 00:41:17 Social media's effects on brain structure and behavior 00:50:46 AI ethics: Altering values and free will considerations 01:00:00 Work value perception and personal identity formation 01:13:37 Free will, agency, and mutable personal identity in therapy 01:24:27 AI in healthcare: Challenges, ethics, and therapy improvements 01:53:25 AI development: Societal impacts and cultural implications Full references on YT VD: https://www.youtube.com/watch?v=7hwX6OZyNC0 (and baked into mp3 metadata)

Prof. Subbarao Kambhampati argues that while LLMs are impressive and useful tools, especially for creative tasks, they have fundamental limitations in logical reasoning and cannot provide guarantees about the correctness of their outputs. He advocates for hybrid approaches that combine LLMs with external verification systems. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. TOC (sorry the ones baked into the MP3 were wrong apropos due to LLM hallucination!) [00:00:00] Intro [00:02:06] Bio [00:03:02] LLMs are n-gram models on steroids [00:07:26] Is natural language a formal language? [00:08:34] Natural language is formal? [00:11:01] Do LLMs reason? [00:19:13] Definition of reasoning [00:31:40] Creativity in reasoning [00:50:27] Chollet's ARC challenge [01:01:31] Can we reason without verification? [01:10:00] LLMs cant solve some tasks [01:19:07] LLM Modulo framework [01:29:26] Future trends of architecture [01:34:48] Future research directions Youtube version: https://www.youtube.com/watch?v=y1WnHpedi2A Refs: (we didn't have space for URLs here, check YT video description instead) Can LLMs Really Reason and Plan? On the Planning Abilities of Large Language Models : A Critical Investigation Chain of Thoughtlessness? An Analysis of CoT in Planning On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve "Task Success" is not Enough Partition function (number theory) (Srinivasa Ramanujan and G.H. Hardy's work) Poincaré conjecture Gödel's incompleteness theorems ROT13 (Rotate13, "rotate by 13 places") A Mathematical Theory of Communication (C. E. SHANNON) Sparks of AGI Kambhampati thesis on speech recognition (1983) PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change Explainable human-AI interaction Tree of Thoughts On the Measure of Intelligence (ARC Challenge) Getting 50% (SoTA) on ARC-AGI with GPT-4o (Ryan Greenblatt ARC solution) PROGRAMS WITH COMMON SENSE (John McCarthy) - "AI should be an advice taker program" Original chain of thought paper ICAPS 2024 Keynote: Dale Schuurmans on "Computing and Planning with Large Generative Models" (COT) The Hardware Lottery (Hooker) A Path Towards Autonomous Machine Intelligence (JEPA/LeCun) AlphaGeometry FunSearch Emergent Abilities of Large Language Models Language models are not naysayers (Negation in LLMs) The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" Embracing negative results

How seriously should governments take the threat of existential risk from AI, given the lack of consensus among researchers? On the one hand, existential risks (x-risks) are necessarily somewhat speculative: by the time there is concrete evidence, it may be too late. On the other hand, governments must prioritize — after all, they don’t worry too much about x-risk from alien invasions. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at brave.com/api. Sayash Kapoor is a computer science Ph.D. candidate at Princeton University's Center for Information Technology Policy. His research focuses on the societal impact of AI. Kapoor has previously worked on AI in both industry and academia, with experience at Facebook, Columbia University, and EPFL Switzerland. He is a recipient of a best paper award at ACM FAccT and an impact recognition award at ACM CSCW. Notably, Kapoor was included in TIME's inaugural list of the 100 most influential people in AI. Sayash Kapoor https://x.com/sayashk https://www.cs.princeton.edu/~sayashk/ Arvind Narayanan (other half of the AI Snake Oil duo) https://x.com/random_walker AI existential risk probabilities are too unreliable to inform policy https://www.aisnakeoil.com/p/ai-existential-risk-probabilities Pre-order AI Snake Oil Book https://amzn.to/4fq2HGb AI Snake Oil blog https://www.aisnakeoil.com/ AI Agents That Matter https://arxiv.org/abs/2407.01502 Shortcut learning in deep neural networks https://www.semanticscholar.org/paper/Shortcut-learning-in-deep-neural-networks-Geirhos-Jacobsen/1b04936c2599e59b120f743fbb30df2eed3fd782 77% Of Employees Report AI Has Increased Workloads And Hampered Productivity, Study Finds https://www.forbes.com/sites/bryanrobinson/2024/07/23/employees-report-ai-increased-workload/ TOC: 00:00:00 Intro 00:01:57 How seriously should we take Xrisk threat? 00:02:55 Risk too unrealiable to inform policy 00:10:20 Overinflated risks 00:12:05 Perils of utility maximisation 00:13:55 Scaling vs airplane speeds 00:17:31 Shift to smaller models? 00:19:08 Commercial LLM ecosystem 00:22:10 Synthetic data 00:24:09 Is AI complexifying our jobs? 00:25:50 Does ChatGPT make us dumber or smarter? 00:26:55 Are AI Agents overhyped? 00:28:12 Simple vs complex baselines 00:30:00 Cost tradeoff in agent design 00:32:30 Model eval vs downastream perf 00:36:49 Shortcuts in metrics 00:40:09 Standardisation of agent evals 00:41:21 Humans in the loop 00:43:54 Levels of agent generality 00:47:25 ARC challenge

Sara Hooker is VP of Research at Cohere and leader of Cohere for AI. We discuss her recent paper critiquing the use of compute thresholds, measured in FLOPs (floating point operations), as an AI governance strategy. We explore why this approach, recently adopted in both US and EU AI policies, may be problematic and oversimplified. Sara explains the limitations of using raw computational power as a measure of AI capability or risk, and discusses the complex relationship between compute, data, and model architecture. Equally important, we go into Sara's work on "The AI Language Gap." This research highlights the challenges and inequalities in developing AI systems that work across multiple languages. Sara discusses how current AI models, predominantly trained on English and a handful of high-resource languages, fail to serve the linguistic diversity of our global population. We explore the technical, ethical, and societal implications of this gap, and discuss potential solutions for creating more inclusive and representative AI systems. We broadly discuss the relationship between language, culture, and AI capabilities, as well as the ethical considerations in AI development and deployment. YT Version: https://youtu.be/dBZp47999Ko TOC: [00:00:00] Intro [00:02:12] FLOPS paper [00:26:42] Hardware lottery [00:30:22] The Language gap [00:33:25] Safety [00:38:31] Emergent [00:41:23] Creativity [00:43:40] Long tail [00:44:26] LLMs and society [00:45:36] Model bias [00:48:51] Language and capabilities [00:52:27] Ethical frameworks and RLHF Sara Hooker https://www.sarahooker.me/ https://www.linkedin.com/in/sararosehooker/ https://scholar.google.com/citations?user=2xy6h3sAAAAJ&hl=en https://x.com/sarahookr Interviewer: Tim Scarfe Refs The AI Language gap https://cohere.com/research/papers/the-AI-language-gap.pdf On the Limitations of Compute Thresholds as a Governance Strategy. https://arxiv.org/pdf/2407.05694v1 The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm https://arxiv.org/pdf/2406.18682 Cohere Aya https://cohere.com/research/aya RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs https://arxiv.org/pdf/2407.02552 Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs https://arxiv.org/pdf/2402.14740 Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ EU AI Act https://www.europarl.europa.eu/doceo/document/TA-9-2024-0138_EN.pdf The bitter lesson http://www.incompleteideas.net/IncIdeas/BitterLesson.html Neel Nanda interview https://www.youtube.com/watch?v=_Ygf0GnlwmY Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet https://transformer-circuits.pub/2024/scaling-monosemanticity/ Chollet's ARC challenge https://github.com/fchollet/ARC-AGI Ryan Greenblatt on ARC https://www.youtube.com/watch?v=z9j3wB1RRGA Disclaimer: This is the third video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview.

Murray Shanahan is a professor of Cognitive Robotics at Imperial College London and a senior research scientist at DeepMind. He challenges our assumptions about AI consciousness and urges us to rethink how we talk about machine intelligence. We explore the dangers of anthropomorphizing AI, the limitations of current language in describing AI capabilities, and the fascinating intersection of philosophy and artificial intelligence. Show notes and full references: https://docs.google.com/document/d/1ICtBI574W-xGi8Z2ZtUNeKWiOiGZ_DRsp9EnyYAISws/edit?usp=sharing Prof Murray Shanahan: https://www.doc.ic.ac.uk/~mpsha/ (look at his selected publications) https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en https://en.wikipedia.org/wiki/Murray_Shanahan https://x.com/mpshanahan Interviewer: Dr. Tim Scarfe Refs (links in the Google doc linked above): Role play with large language models Waluigi effect "Conscious Exotica" - Paper by Murray Shanahan (2016) "Simulators" - Article by Janis from LessWrong "Embodiment and the Inner Life" - Book by Murray Shanahan (2010) "The Technological Singularity" - Book by Murray Shanahan (2015) "Simulacra as Conscious Exotica" - Paper by Murray Shanahan (newer paper of the original focussed on LLMs) A recent paper by Anthropic on using autoencoders to find features in language models (referring to the "Scaling Monosemanticity" paper) Work by Peter Godfrey-Smith on octopus consciousness "Metaphors We Live By" - Book by George Lakoff (1980s) Work by Aaron Sloman on the concept of "space of possible minds" (1984 article mentioned) Wittgenstein's "Philosophical Investigations" (posthumously published) Daniel Dennett's work on the "intentional stance" Alan Turing's original paper on the Turing Test (1950) Thomas Nagel's paper "What is it like to be a bat?" (1974) John Searle's Chinese Room Argument (mentioned but not detailed) Work by Richard Evans on tackling reasoning problems Claude Shannon's quote on knowledge and control "Are We Bodies or Souls?" - Book by Richard Swinburne Reference to work by Ethan Perez and others at Anthropic on potential deceptive behavior in language models Reference to a paper by Murray Shanahan and Antonia Creswell on the "selection inference framework" Mention of work by Francois Chollet, particularly the ARC (Abstraction and Reasoning Corpus) challenge Reference to Elizabeth Spelke's work on core knowledge in infants Mention of Karl Friston's work on planning as inference (active inference) The film "Ex Machina" - Murray Shanahan was the scientific advisor "The Waluigi Effect" Anthropic's constitutional AI approach Loom system by Lara Reynolds and Kyle McDonald for visualizing conversation trees DeepMind's AlphaGo (mentioned multiple times as an example) Mention of the "Golden Gate Claude" experiment Reference to an interview Tim Scarfe conducted with University of Toronto students about self-attention controllability theorem Mention of an interview with Irina Rish Reference to an interview Tim Scarfe conducted with Daniel Dennett Reference to an interview with Maria Santa Caterina Mention of an interview with Philip Goff Nick Chater and Martin Christianson's book ("The Language Game: How Improvisation Created Language and Changed the World") Peter Singer's work from 1975 on ascribing moral status to conscious beings Demis Hassabis' discussion on the "ladder of creativity" Reference to B.F. Skinner and behaviorism

In the coming decades, the technology that enables virtual and augmented reality will improve beyond recognition. Within a century, world-renowned philosopher David J. Chalmers predicts, we will have virtual worlds that are impossible to distinguish from non-virtual worlds. But is virtual reality just escapism? In a highly original work of 'technophilosophy', Chalmers argues categorically, no: virtual reality is genuine reality. Virtual worlds are not second-class worlds. We can live a meaningful life in virtual reality - and increasingly, we will. What is reality, anyway? How can we lead a good life? Is there a god? How do we know there's an external world - and how do we know we're not living in a computer simulation? In Reality+, Chalmers conducts a grand tour of philosophy, using cutting-edge technology to provide invigorating new answers to age-old questions. David J. Chalmers is an Australian philosopher and cognitive scientist specializing in the areas of philosophy of mind and philosophy of language. He is Professor of Philosophy and Neural Science at New York University, as well as co-director of NYU's Center for Mind, Brain, and Consciousness. Chalmers is best known for his work on consciousness, including his formulation of the "hard problem of consciousness." Reality+: Virtual Worlds and the Problems of Philosophy https://amzn.to/3RYyGD2 https://consc.net/ https://x.com/davidchalmers42 00:00:00 Reality+ Intro 00:12:02 GPT conscious? 10/10 00:14:19 The consciousness processor thought experiment (11/10) 00:20:34 Intelligence and Consciousness entangled? 10/10 00:22:44 Karl Friston / Meta Problem 10/10 00:29:05 Knowledge argument / subjective experience (6/10) 00:32:34 Emergence 11/10 (best chapter) 00:42:45 Working with Douglas Hofstadter 10/10 00:46:14 Intelligence is analogy making? 10/10 00:50:47 Intelligence explosion 8/10 00:58:44 Hypercomputation 10/10 01:09:44 Who designed the designer? (7/10) 01:13:57 Experience machine (7/10)

Ryan Greenblatt from Redwood Research recently published "Getting 50% on ARC-AGI with GPT-4.0," where he used GPT4o to reach a state-of-the-art accuracy on Francois Chollet's ARC Challenge by generating many Python programs. Sponsor: Sign up to Kalshi here https://kalshi.onelink.me/1r91/mlst -- the first 500 traders who deposit $100 will get a free $20 credit! Important disclaimer - In case it's not obvious - this is basically gambling and a *high risk* activity - only trade what you can afford to lose. We discuss: - Ryan's unique approach to solving the ARC Challenge and achieving impressive results. - The strengths and weaknesses of current AI models. - How AI and humans differ in learning and reasoning. - Combining various techniques to create smarter AI systems. - The potential risks and future advancements in AI, including the idea of agentic AI. https://x.com/RyanPGreenblatt https://www.redwoodresearch.org/ Refs: Getting 50% (SoTA) on ARC-AGI with GPT-4o [Ryan Greenblatt] https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt On the Measure of Intelligence [Chollet] https://arxiv.org/abs/1911.01547 Connectionism and Cognitive Architecture: A Critical Analysis [Jerry A. Fodor and Zenon W. Pylyshyn] https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf Software 2.0 [Andrej Karpathy] https://karpathy.medium.com/software-2-0-a64152b37c35 Why Greatness Cannot Be Planned: The Myth of the Objective [Kenneth Stanley] https://amzn.to/3Wfy2E0 Biographical account of Terence Tao’s mathematical development. [M.A.(KEN) CLEMENTS] https://gwern.net/doc/iq/high/smpy/1984-clements.pdf Model Evaluation and Threat Research (METR) https://metr.org/ Why Tool AIs Want to Be Agent AIs https://gwern.net/tool-ai Simulators - Janus https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators AI Control: Improving Safety Despite Intentional Subversion https://www.lesswrong.com/posts/d9FJHawgkiMSPjagR/ai-control-improving-safety-despite-intentional-subversion https://arxiv.org/abs/2312.06942 What a Compute-Centric Framework Says About Takeoff Speeds https://www.openphilanthropy.org/research/what-a-compute-centric-framework-says-about-takeoff-speeds/ Global GDP over the long run https://ourworldindata.org/grapher/global-gdp-over-the-long-run?yScale=log Safety Cases: How to Justify the Safety of Advanced AI Systems https://arxiv.org/abs/2403.10462 The Danger of a “Safety Case" http://sunnyday.mit.edu/The-Danger-of-a-Safety-Case.pdf The Future Of Work Looks Like A UPS Truck (~02:15:50) https://www.npr.org/sections/money/2014/05/02/308640135/episode-536-the-future-of-work-looks-like-a-ups-truck SWE-bench https://www.swebench.com/ Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model https://arxiv.org/pdf/2201.11990 Algorithmic Progress in Language Models https://epochai.org/blog/algorithmic-progress-in-language-models

Aidan Gomez, CEO of Cohere, reveals how they're tackling AI hallucinations and improving reasoning abilities. He also explains why Cohere doesn't use any output from GPT-4 for training their models. Aidan shares his personal insights into the world of AI and LLMs and Cohere's unique approach to solving real-world business problems, and how their models are set apart from the competition. Aidan reveals how they are making major strides in AI technology, discussing everything from last mile customer engineering to the robustness of prompts and future architectures. He also touches on the broader implications of AI for society, including potential risks and the role of regulation. He discusses Cohere's guiding principles and the health the of startup scene. With a particular focus on enterprise applications. Aidan provides a rare look into the internal workings of Cohere and their vision for driving productivity and innovation. https://cohere.com/ https://x.com/aidangomez Check out Cohere's amazing new Command R* models here https://cohere.com/command Disclaimer: This is the second video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview.

The ARC Challenge, created by Francois Chollet, tests how well AI systems can generalize from a few examples in a grid-based intelligence test. We interview the current winners of the ARC Challenge—Jack Cole, Mohammed Osman and their collaborator Michael Hodel. They discuss how they tackled ARC (Abstraction and Reasoning Corpus) using language models. We also discuss the new "50%" public set approach announced today from Redwood Research (Ryan Greenblatt). Jack and Mohammed explain their winning approach, which involves fine-tuning a language model on a large, specifically-generated dataset and then doing additional fine-tuning at test-time, a technique known in this context as "active inference". They use various strategies to represent the data for the language model and believe that with further improvements, the accuracy could reach above 50%. Michael talks about his work on generating new ARC-like tasks to help train the models. They also debate whether their methods stay true to the "spirit" of Chollet's measure of intelligence. Despite some concerns, they agree that their solutions are promising and adaptable for other similar problems. Note: Jack's team is still the current official winner at 33% on the private set. Ryan's entry is not on the private leaderboard or eligible. Chollet invented ARC in 2019 (not 2017 as stated) "Ryan's entry is not a new state of the art. We don't know exactly how well it does since it was only evaluated on 100 tasks from the evaluation set and does 50% on those, reportedly. Meanwhile Jacks team i.e. MindsAI's solution does 54% on the entire eval set and it is seemingly possible to do 60-70% with an ensemble" Jack Cole: https://x.com/Jcole75Cole https://lab42.global/community-interview-jack-cole/ Mohamed Osman: Mohamed is looking to do a PhD in AI/ML, can you help him? Email: [email protected] https://www.linkedin.com/in/mohamedosman1905/ Michael Hodel: https://arxiv.org/pdf/2404.07353v1 https://www.linkedin.com/in/michael-hodel/ https://x.com/bayesilicon https://github.com/michaelhodel Getting 50% (SoTA) on ARC-AGI with GPT-4o - Ryan Greenblatt https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt Neural networks for abstraction and reasoning: Towards broad generalization in machines [Mikel Bober-Irizar, Soumya Banerjee] https://arxiv.org/pdf/2402.03507 Measure of intelligence: https://arxiv.org/abs/1911.01547 YT version: https://youtu.be/jSAT_RuJ_Cg

Nick Frosst, co-founder of Cohere, on the future of LLMs, and AGI. Learn how Cohere is solving real problems for business with their new AI models. This is the first podcast from our new Cohere partnership! Nick talks about his journey at Google Brain, working with AI legends like Geoff Hinton, and the amazing things his company, Cohere, is doing. From creating the must useful language models for businesses to making tools for developers, Nick shares a lot of interesting insights. He even talks about his band, Good Kid! Nick said that RAG is one of the best features of Cohere's new Command R* models. We are about to release a deep-dive on RAG with Patrick Lewis from Cohere, keep an eye out for that - he explains why their models are specifically optimised for RAG use cases. Learn more about Cohere Command R* models here: https://cohere.com/commandhttps://github.com/cohere-ai/cohere-toolkit Nick's band Good Kid: https://goodkidofficial.com/ Nick on Twitter: https://x.com/nickfrosst Disclaimer: We are in a partnership with Cohere to release content for them. We were not told what to say in the interview, and didn't edit anything out from the interview. We are currently planning to release 2 shows per month under the partnership about their AI platform, research and strategy.

These two scientists have mapped out the insides or “reachable space” of a language model using control theory, what they discovered was extremely surprising. Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening. https://patreon.com/mlst YT version: https://youtu.be/Bpgloy1dDn0 Aman Bhargava from Caltech and Cameron Witkowski from the University of Toronto to discuss their groundbreaking paper, “What’s the Magic Word? A Control Theory of LLM Prompting.” (the main theorem on self-attention controllability was developed in collaboration with Dr. Shi-Zhuo Looi from Caltech). They frame LLM systems as discrete stochastic dynamical systems. This means they look at LLMs in a structured way, similar to how we analyze control systems in engineering. They explore the “reachable set” of outputs for an LLM. Essentially, this is the range of possible outputs the model can generate from a given starting point when influenced by different prompts. The research highlights that prompt engineering, or optimizing the input tokens, can significantly influence LLM outputs. They show that even short prompts can drastically alter the likelihood of specific outputs. Aman and Cameron’s work might be a boon for understanding and improving LLMs. They suggest that a deeper exploration of control theory concepts could lead to more reliable and capable language models. We dropped an additional, more technical video on the research on our Twitter account here: https://x.com/MLStreetTalk/status/1795093759471890606 Additional 20 minutes of unreleased footage on our Patreon here: https://www.patreon.com/posts/whats-magic-word-104922629 What's the Magic Word? A Control Theory of LLM Prompting (Aman Bhargava, Cameron Witkowski, Manav Shah, Matt Thomson) https://arxiv.org/abs/2310.04444 LLM Control Theory Seminar (April 2024) https://www.youtube.com/watch?v=9QtS9sVBFM0 Society for the pursuit of AGI (Cameron founded it) https://agisociety.mydurable.com/ Roger Federer demo http://conway.languagegame.io/inference Neural Cellular Automata, Active Inference, and the Mystery of Biological Computation (Aman) https://aman-bhargava.com/ai/neuro/neuromorphic/2024/03/25/nca-do-active-inference.html Aman and Cameron also want to thank Dr. Shi-Zhuo Looi and Prof. Matt Thomson from from Caltech for help and advice on their research. (https://thomsonlab.caltech.edu/ and https://pma.caltech.edu/people/looi-shi-zhuo) https://x.com/ABhargava2000 https://x.com/witkowski_cam

Maria Santacaterina, with her background in the humanities, brings a critical perspective on the current state and future implications of AI technology, its impact on society, and the nature of human intelligence and creativity. She emphasizes that despite technological advancements, AI lacks fundamental human traits such as consciousness, empathy, intuition, and the ability to engage in genuine creative processes. Maria argues that AI, at its core, processes data but does not have the capability to understand or generate new, intrinsic meaning or ideas as humans do. Throughout the conversation, Maria highlights her concern about the overreliance on AI in critical sectors such as healthcare, the justice system, and business. She stresses that while AI can serve as a tool, it should not replace human judgment and decision-making. Maria points out that AI systems often operate on past data, which may lead to outdated or incorrect decisions if not carefully managed. The discussion also touches upon the concept of "adaptive resilience", which Maria describes in her book. She explains adaptive resilience as the capacity for individuals and enterprises to evolve and thrive amidst challenges by leveraging technology responsibly, without undermining human values and capabilities. A significant portion of the conversation focussed on ethical considerations surrounding AI. Tim and Maria agree that there's a pressing need for strong governance and ethical frameworks to guide AI development and deployment. They discuss how AI, without proper ethical considerations, risks exacerbating issues like privacy invasion, misinformation, and unintended discrimination. Maria is skeptical about claims of achieving Artificial General Intelligence (AGI) or a technological singularity where machines surpass human intelligence in all aspects. She argues that such scenarios neglect the complex, dynamic nature of human intelligence and consciousness, which cannot be fully replicated or replaced by machines. Tim and Maria discuss the importance of keeping human agency and creativity at the forefront of technology development. Maria asserts that efforts to automate or standardize complex human actions and decisions are misguided and could lead to dehumanizing outcomes. They both advocate for using AI as an aid to enhance human capabilities rather than a substitute. In closing, Maria encourages a balanced approach to AI adoption, urging stakeholders to prioritize human well-being, ethical standards, and societal benefit above mere technological advancement. The conversation ends with Maria pointing people to her book for more in-depth analysis and thoughts on the future interaction between humans and technology. Buy Maria's book here: https://amzn.to/4avF6kq https://www.linkedin.com/in/mariasantacaterina TOC 00:00:00 - Intro to Book 00:03:23 - What Life Is 00:10:10 - Agency 00:18:04 - Tech and Society 00:21:51 - System 1 and 2 00:22:59 - We Are Being Pigeonholed 00:30:22 - Agency vs Autonomy 00:36:37 - Explanations 00:40:24 - AI Reductionism 00:49:50 - How Are Humans Intelligent 01:00:22 - Semantics 01:01:53 - Emotive AI and Pavlovian Dogs 01:04:05 - Technology, Social Media and Organisation 01:18:34 - Systems Are Not That Automated 01:19:33 - Hiring 01:22:34 - Subjectivity in Orgs 01:32:28 - The AGI Delusion 01:45:37 - GPT-laziness Syndrome 01:54:58 - Diversity Preservation 01:58:24 - Ethics 02:11:43 - Moral Realism 02:16:17 - Utopia 02:18:02 - Reciprocity 02:20:52 - Tyranny of Categorisation

Thomas Parr and his collaborators wrote a book titled "Active Inference: The Free Energy Principle in Mind, Brain and Behavior" which introduces Active Inference from both a high-level conceptual perspective and a low-level mechanistic, mathematical perspective. Active inference, developed by the legendary neuroscientist Prof. Karl Friston - is a unifying mathematical framework which frames living systems as agents which minimize surprise and free energy in order to resist entropy and persist over time. It unifies various perspectives from physics, biology, statistics, and psychology - and allows us to explore deep questions about agency, biology, causality, modelling, and consciousness. Buy Active Inference: The Free Energy Principle in Mind, Brain, and Behavior https://amzn.to/4dj0iMj YT version: https://youtu.be/lbb-Si5wa_o Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening. https://patreon.com/mlst Chapters should be embedded in the mp3, let me me know if issues

Connor is the CEO of Conjecture and one of the most famous names in the AI alignment movement. This is the "behind the scenes footage" and bonus Patreon interviews from the day of the Beff Jezos debate, including an interview with Daniel Clothiaux. It's a great insight into Connor's philosophy. At the end there is an unreleased additional interview with Beff. Support MLST: Please support us on Patreon. We are entirely funded from Patreon donations right now. Patreon supports get private discord access, biweekly calls, very early-access + exclusive content and lots more. https://patreon.com/mlst Donate: https://www.paypal.com/donate/?hosted_button_id=K2TYRVPBGXVNA If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail Topics: Externalized cognition and the role of society and culture in human intelligence The potential for AI systems to develop agency and autonomy The future of AGI as a complex mixture of various components The concept of agency and its relationship to power The importance of coherence in AI systems The balance between coherence and variance in exploring potential upsides The role of dynamic, competent, and incorruptible institutions in handling risks and developing technology Concerns about AI widening the gap between the haves and have-nots The concept of equal access to opportunity and maintaining dynamism in the system Leahy's perspective on life as a process that "rides entropy" The importance of distinguishing between epistemological, decision-theoretic, and aesthetic aspects of morality (inc ref to Hume's Guillotine) The concept of continuous agency and the idea that the first AGI will be a messy admixture of various components The potential for AI systems to become more physically embedded in the future The challenges of aligning AI systems and the societal impacts of AI technologies like ChatGPT and Bing The importance of humility in the face of complexity when considering the future of AI and its societal implications Disclaimer: this video is not an endorsement of e/acc or AGI agential existential risk from us - the hosts of MLST consider both of these views to be quite extreme. We seek diverse views on the channel. 00:00:00 Intro 00:00:56 Connor's Philosophy 00:03:53 Office Skit 00:05:08 Connor on e/acc and Beff 00:07:28 Intro to Daniel's Philosophy 00:08:35 Connor on Entropy, Life, and Morality 00:19:10 Connor on London 00:20:21 Connor Office Interview 00:20:46 Friston Patreon Preview 00:21:48 Why Are We So Dumb? 00:23:52 The Voice of the People, the Voice of God / Populism 00:26:35 Mimetics 00:30:03 Governance 00:33:19 Agency 00:40:25 Daniel Interview - Externalised Cognition, Bing GPT, AGI 00:56:29 Beff + Connor Bonus Patreons Interview

Professor Chris Bishop is a Technical Fellow and Director at Microsoft Research AI4Science, in Cambridge. He is also Honorary Professor of Computer Science at the University of Edinburgh, and a Fellow of Darwin College, Cambridge. In 2004, he was elected Fellow of the Royal Academy of Engineering, in 2007 he was elected Fellow of the Royal Society of Edinburgh, and in 2017 he was elected Fellow of the Royal Society. Chris was a founding member of the UK AI Council, and in 2019 he was appointed to the Prime Minister’s Council for Science and Technology. At Microsoft Research, Chris oversees a global portfolio of industrial research and development, with a strong focus on machine learning and the natural sciences. Chris obtained a BA in Physics from Oxford, and a PhD in Theoretical Physics from the University of Edinburgh, with a thesis on quantum field theory. Chris's contributions to the field of machine learning have been truly remarkable. He has authored (what is arguably) the original textbook in the field - 'Pattern Recognition and Machine Learning' (PRML) which has served as an essential reference for countless students and researchers around the world, and that was his second textbook after his highly acclaimed first textbook Neural Networks for Pattern Recognition. Recently, Chris has co-authored a new book with his son, Hugh, titled 'Deep Learning: Foundations and Concepts.' This book aims to provide a comprehensive understanding of the key ideas and techniques underpinning the rapidly evolving field of deep learning. It covers both the foundational concepts and the latest advances, making it an invaluable resource for newcomers and experienced practitioners alike. Buy Chris' textbook here: https://amzn.to/3vvLcCh More about Prof. Chris Bishop: https://en.wikipedia.org/wiki/Christopher_Bishop https://www.microsoft.com/en-us/research/people/cmbishop/ Support MLST: Please support us on Patreon. We are entirely funded from Patreon donations right now. Patreon supports get private discord access, biweekly calls, early-access + exclusive content and lots more. https://patreon.com/mlst Donate: https://www.paypal.com/donate/?hosted_button_id=K2TYRVPBGXVNA If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail TOC: 00:00:00 - Intro to Chris 00:06:54 - Changing Landscape of AI 00:08:16 - Symbolism 00:09:32 - PRML 00:11:02 - Bayesian Approach 00:14:49 - Are NNs One Model or Many, Special vs General 00:20:04 - Can Language Models Be Creative 00:22:35 - Sparks of AGI 00:25:52 - Creativity Gap in LLMs 00:35:40 - New Deep Learning Book 00:39:01 - Favourite Chapters 00:44:11 - Probability Theory 00:45:42 - AI4Science 00:48:31 - Inductive Priors 00:58:52 - Drug Discovery 01:05:19 - Foundational Bias Models 01:07:46 - How Fundamental Is Our Physics Knowledge? 01:12:05 - Transformers 01:12:59 - Why Does Deep Learning Work? 01:16:59 - Inscrutability of NNs 01:18:01 - Example of Simulator 01:21:09 - Control

Dr. Philip Ball is a freelance science writer. He just wrote a book called "How Life Works", discussing the how the science of Biology has advanced in the last 20 years. We focus on the concept of Agency in particular. He trained as a chemist at the University of Oxford, and as a physicist at the University of Bristol. He worked previously at Nature for over 20 years, first as an editor for physical sciences and then as a consultant editor. His writings on science for the popular press have covered topical issues ranging from cosmology to the future of molecular biology. YT: https://www.youtube.com/watch?v=n6nxUiqiz9I Transcript link on YT description Philip is the author of many popular books on science, including H2O: A Biography of Water, Bright Earth: The Invention of Colour, The Music Instinct and Curiosity: How Science Became Interested in Everything. His book Critical Mass won the 2005 Aventis Prize for Science Books, while Serving the Reich was shortlisted for the Royal Society Winton Science Book Prize in 2014. This is one of Tim's personal favourite MLST shows, so we have designated it a special edition. Enjoy! Buy Philip's book "How Life Works" here: https://amzn.to/3vSmNqp Support MLST: Please support us on Patreon. We are entirely funded from Patreon donations right now. Patreon supports get private discord access, biweekly calls, early-access + exclusive content and lots more. https://patreon.com/mlst Donate: https://www.paypal.com/donate/?hosted... If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail

Dr. Paul Lessard and his collaborators have written a paper on "Categorical Deep Learning and Algebraic Theory of Architectures". They aim to make neural networks more interpretable, composable and amenable to formal reasoning. The key is mathematical abstraction, as exemplified by category theory - using monads to develop a more principled, algebraic approach to structuring neural networks. We also discussed the limitations of current neural network architectures in terms of their ability to generalise and reason in a human-like way. In particular, the inability of neural networks to do unbounded computation equivalent to a Turing machine. Paul expressed optimism that this is not a fundamental limitation, but an artefact of current architectures and training procedures. The power of abstraction - allowing us to focus on the essential structure while ignoring extraneous details. This can make certain problems more tractable to reason about. Paul sees category theory as providing a powerful "Lego set" for productively thinking about many practical problems. Towards the end, Paul gave an accessible introduction to some core concepts in category theory like categories, morphisms, functors, monads etc. We explained how these abstract constructs can capture essential patterns that arise across different domains of mathematics. Paul is optimistic about the potential of category theory and related mathematical abstractions to put AI and neural networks on a more robust conceptual foundation to enable interpretability and reasoning. However, significant theoretical and engineering challenges remain in realising this vision. Please support us on Patreon. We are entirely funded from Patreon donations right now. https://patreon.com/mlst If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail Links: Categorical Deep Learning: An Algebraic Theory of Architectures Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković Paper: https://categoricaldeeplearning.com/ Symbolica: https://twitter.com/symbolica https://www.symbolica.ai/ Dr. Paul Lessard (Principal Scientist - Symbolica) https://www.linkedin.com/in/paul-roy-lessard/ Interviewer: Dr. Tim Scarfe TOC: 00:00:00 - Intro 00:05:07 - What is the category paper all about 00:07:19 - Composition 00:10:42 - Abstract Algebra 00:23:01 - DSLs for machine learning 00:24:10 - Inscrutibility 00:29:04 - Limitations with current NNs 00:30:41 - Generative code / NNs don't recurse 00:34:34 - NNs are not Turing machines (special edition) 00:53:09 - Abstraction 00:55:11 - Category theory objects 00:58:06 - Cat theory vs number theory 00:59:43 - Data and Code are one in the same 01:08:05 - Syntax and semantics 01:14:32 - Category DL elevator pitch 01:17:05 - Abstraction again 01:20:25 - Lego set for the universe 01:23:04 - Reasoning 01:28:05 - Category theory 101 01:37:42 - Monads 01:45:59 - Where to learn more cat theory

Dr. Minqi Jiang and Dr. Marc Rigter explain an innovative new method to make the intelligence of agents more general-purpose by training them to learn many worlds before their usual goal-directed training, which we call "reinforcement learning". Their new paper is called "Reward-free curricula for training robust world models" https://arxiv.org/pdf/2306.09205.pdf https://twitter.com/MinqiJiang https://twitter.com/MarcRigter Interviewer: Dr. Tim Scarfe Please support us on Patreon, Tim is now doing MLST full-time and taking a massive financial hit. If you love MLST and want this to continue, please show your support! In return you get access to shows very early and private discord and networking. https://patreon.com/mlst We are also looking for show sponsors, please get in touch if interested mlstreettalk at gmail. MLST Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778

Nick Chater is Professor of Behavioural Science at Warwick Business School, who works on rationality and language using a range of theoretical and experimental approaches. We discuss his books The Mind is Flat, and the Language Game. Please support me on Patreon (this is now my main job!) - https://patreon.com/mlst - Access the private Discord, networking, and early access to content. MLST Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778 https://twitter.com/MLStreetTalk Buy The Language Game: https://amzn.to/3SRHjPm Buy The Mind is Flat: https://amzn.to/3P3BUUC YT version: https://youtu.be/5cBS6COzLN4 https://www.wbs.ac.uk/about/person/nick-chater/ https://twitter.com/nickjchater?lang=en

See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network". The idea is that you follow interests, NOT people. It's a social network without the popularity contest. We discuss the phgilosophy and technology behind the venture in great detail. The main ideas of which came from Kenneth's famous book "Why greatness cannot be planned". See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network".The idea is that you follow interests, NOT people. It's a social network without the popularity contest. YT version: https://www.youtube.com/watch?v=pWIrXN-yy8g Chapters should be baked into the MP3 file now MLST public Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778 Please support our work on Patreon - get access to interviews months early, private Patreon, networking, exclusive content and regular calls with Tim and Keith. https://patreon.com/mlst Get Maven here: https://www.heymaven.com/ Kenneth: https://twitter.com/kenneth0stanley https://www.kenstanley.net/home Host - Tim Scarfe: https://www.linkedin.com/in/ecsquizor/ https://www.mlst.ai/ Original MLST show with Kenneth: https://www.youtube.com/watch?v=lhYGXYeMq_E Tim explains the book more here: https://www.youtube.com/watch?v=wNhaz81OOqw

Brandon Rohrer who obtained his Ph.D from MIT is driven by understanding algorithms ALL the way down to their nuts and bolts, so he can make them accessible to everyone by first explaining them in the way HE himself would have wanted to learn! Please support us on Patreon for loads of exclusive content and private Discord: https://patreon.com/mlst (public discord) https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk Brandon Rohrer is a seasoned data science leader and educator with a rich background in creating robust, efficient machine learning algorithms and tools. With a Ph.D. in Mechanical Engineering from MIT, his expertise encompasses a broad spectrum of AI applications — from computer vision and natural language processing to reinforcement learning and robotics. Brandon's career has seen him in Principle-level roles at Microsoft and Facebook. An educator at heart, he also shares his knowledge through detailed tutorials, courses, and his forthcoming book, "How to Train Your Robot." YT version: https://www.youtube.com/watch?v=4Ps7ahonRCY Brandon's links: https://github.com/brohrer https://www.youtube.com/channel/UCsBKTrp45lTfHa_p49I2AEQ https://www.linkedin.com/in/brohrer/ How transformers work: https://e2eml.school/transformers Brandon's End-to-End Machine Learning school courses, posts, and tutorials https://e2eml.school Free course: https://end-to-end-machine-learning.teachable.com/p/complete-course-library-full-end-to-end-machine-learning-catalog Blog: https://e2eml.school/blog.html Ziptie: Learning Useful Features [Brandon Rohrer] https://www.brandonrohrer.com/ziptie TOC should be baked into the MP3 file now 00:00:00 - Intro to Brandon 00:00:36 - RLHF 00:01:09 - Limitations of transformers 00:07:23 - Agency - we are all GPTs 00:09:07 - BPE / representation bias 00:12:00 - LLM true believers 00:16:42 - Brandon's style of teaching 00:19:50 - ML vs real world = Robotics 00:29:59 - Reward shaping 00:37:08 - No true Scotsman - when do we accept capabilities as real 00:38:50 - Externalism 00:43:03 - Building flexible robots 00:45:37 - Is reward enough 00:54:30 - Optimization curse 00:58:15 - Collective intelligence 01:01:51 - Intelligence + creativity 01:13:35 - ChatGPT + Creativity 01:25:19 - Transformers Tutorial

The world's second-most famous AI doomer Connor Leahy sits down with Beff Jezos, the founder of the e/acc movement debating technology, AI policy, and human values. As the two discuss technology, AI safety, civilization advancement, and the future of institutions, they clash on their opposing perspectives on how we steer humanity towards a more optimal path. Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon. We have some amazing content going up there with Max Bennett and Kenneth Stanley this week! https://patreon.com/mlst (public discord) https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk Post-interview with Beff and Connor: https://www.patreon.com/posts/97905213 Pre-interview with Connor and his colleague Dan Clothiaux: https://www.patreon.com/posts/connor-leahy-and-97631416 Leahy, known for his critical perspectives on AI and technology, challenges Jezos on a variety of assertions related to the accelerationist movement, market dynamics, and the need for regulation in the face of rapid technological advancements. Jezos, on the other hand, provides insights into the e/acc movement's core philosophies, emphasizing growth, adaptability, and the dangers of over-legislation and centralized control in current institutions. Throughout the discussion, both speakers explore the concept of entropy, the role of competition in fostering innovation, and the balance needed to mediate order and chaos to ensure the prosperity and survival of civilization. They weigh up the risks and rewards of AI, the importance of maintaining a power equilibrium in society, and the significance of cultural and institutional dynamism. Beff Jezos (Guillaume Verdon): https://twitter.com/BasedBeffJezos https://twitter.com/GillVerd Connor Leahy: https://twitter.com/npcollapse YT: https://www.youtube.com/watch?v=0zxi0xSBOaQ TOC: 00:00:00 - Intro 00:03:05 - Society library reference 00:03:35 - Debate starts 00:05:08 - Should any tech be banned? 00:20:39 - Leaded Gasoline 00:28:57 - False vacuum collapse method? 00:34:56 - What if there are dangerous aliens? 00:36:56 - Risk tolerances 00:39:26 - Optimizing for growth vs value 00:52:38 - Is vs ought 01:02:29 - AI discussion 01:07:38 - War / global competition 01:11:02 - Open source F16 designs 01:20:37 - Offense vs defense 01:28:49 - Morality / value 01:43:34 - What would Conor do 01:50:36 - Institutions/regulation 02:26:41 - Competition vs. Regulation Dilemma 02:32:50 - Existential Risks and Future Planning 02:41:46 - Conclusion and Reflection Note from Tim: I baked the chapter metadata into the mp3 file this time, does that help the chapters show up in your app? Let me know. Also I accidentally exported a few minutes of dead audio at the end of the file - sorry about that just skip on when the episode finishes.

Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon: https://patreon.com/mlst (public discord) https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk YT version: https://youtu.be/n8G50ynU0Vg In this interview on MLST, Dr. Tim Scarfe interviews Mahault Albarracin, who is the director of product for R&D at VERSES and also a PhD student in cognitive computing at the University of Quebec in Montreal. They discuss a range of topics related to consciousness, cognition, and machine learning. Throughout the conversation, they touch upon various philosophical and computational concepts such as panpsychism, computationalism, and materiality. They consider the "hard problem" of consciousness, which is the question of how and why we have subjective experiences. Albarracin shares her views on the controversial Integrated Information Theory and the open letter of opposition it received from the scientific community. She reflects on the nature of scientific critique and rivalry, advising caution in declaring entire fields of study as pseudoscientific. A substantial part of the discussion is dedicated to the topic of science itself, where Albarracin talks about thresholds between legitimate science and pseudoscience, the role of evidence, and the importance of validating scientific methods and claims. They touch upon language models, discussing whether they can be considered as having a "theory of mind" and the implications of assigning such properties to AI systems. Albarracin challenges the idea that there is a pure form of intelligence independent of material constraints and emphasizes the role of sociality in the development of our cognitive abilities. Albarracin offers her thoughts on scientific endeavors, the predictability of systems, the nature of intelligence, and the processes of learning and adaptation. She gives insights into the concept of using degeneracy as a way to increase resilience within systems and the role of maintaining a degree of redundancy or extra capacity as a buffer against unforeseen events. The conversation concludes with her discussing the potential benefits of collective intelligence, likening the adaptability and resilience of interconnected agent systems to those found in natural ecosystems. https://www.linkedin.com/in/mahault-albarracin-1742bb153/ 00:00:00 - Intro / IIT scandal 00:05:54 - Gaydar paper / What makes good science 00:10:51 - Language 00:18:16 - Intelligence 00:29:06 - X-risk 00:40:49 - Self modelling 00:43:56 - Anthropomorphisation 00:46:41 - Mediation and subjectivity 00:51:03 - Understanding 00:56:33 - Resiliency Technical topics: 1. Integrated Information Theory (IIT) - Giulio Tononi 2. The "hard problem" of consciousness - David Chalmers 3. Panpsychism and Computationalism in philosophy of mind 4. Active Inference Framework - Karl Friston 5. Theory of Mind and its computation in AI systems 6. Noam Chomsky's views on language models and linguistics 7. Daniel Dennett's Intentional Stance theory 8. Collective intelligence and system resilience 9. Redundancy and degeneracy in complex systems 10. Michael Levin's research on bioelectricity and pattern formation 11. The role of phenomenology in cognitive science

Chai AI is the leading platform for conversational chat artificial intelligence. Note: this is a sponsored episode of MLST. William Beauchamp is the founder of two $100M+ companies - Chai Research, an AI startup, and Seamless Capital, a hedge fund based in Cambridge, UK. Chaiverse is the Chai AI developer platform, where developers can train, submit and evaluate on millions of real users to win their share of $1,000,000. https://www.chai-research.com https://www.chaiverse.com https://twitter.com/chai_research https://facebook.com/chairesearch/ https://www.instagram.com/chairesearch/ Download the app on iOS and Android (https://onelink.to/kqzhy9 ) #chai #chai_ai #chai_research #chaiverse #generative_ai #LLMs

Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon: https://patreon.com/mlst (public discord) https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk DOES AI HAVE AGENCY? With Professor. Karl Friston and Riddhi J. Pitliya Agency in the context of cognitive science, particularly when considering the free energy principle, extends beyond just human decision-making and autonomy. It encompasses a broader understanding of how all living systems, including non-human entities, interact with their environment to maintain their existence by minimising sensory surprise. According to the free energy principle, living organisms strive to minimize the difference between their predicted states and the actual sensory inputs they receive. This principle suggests that agency arises as a natural consequence of this process, particularly when organisms appear to plan ahead many steps in the future. Riddhi J. Pitliya is based in the computational psychopathology lab doing her Ph.D at the University of Oxford and works with Professor Karl Friston at VERSES. https://twitter.com/RiddhiJP References: THE FREE ENERGY PRINCIPLE—A PRECIS [Ramstead] https://www.dialecticalsystems.eu/contributions/the-free-energy-principle-a-precis/ Active Inference: The Free Energy Principle in Mind, Brain, and Behavior [Thomas Parr, Giovanni Pezzulo, Karl J. Friston] https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind The beauty of collective intelligence, explained by a developmental biologist | Michael Levin https://www.youtube.com/watch?v=U93x9AWeuOA Growing Neural Cellular Automata https://distill.pub/2020/growing-ca Carcinisation https://en.wikipedia.org/wiki/Carcinisation Prof. KENNETH STANLEY - Why Greatness Cannot Be Planned https://www.youtube.com/watch?v=lhYGXYeMq_E On Defining Artificial Intelligence [Pei Wang] https://sciendo.com/article/10.2478/jagi-2019-0002 Why? The Purpose of the Universe [Goff] https://amzn.to/4aEqpfm Umwelt https://en.wikipedia.org/wiki/Umwelt An Immense World: How Animal Senses Reveal the Hidden Realms [Yong] https://amzn.to/3tzzTb7 What's it like to be a bat [Nagal] https://www.sas.upenn.edu/~cavitch/pdf-library/Nagel_Bat.pdf COUNTERFEIT PEOPLE. DANIEL DENNETT. (SPECIAL EDITION) https://www.youtube.com/watch?v=axJtywd9Tbo We live in the infosphere [FLORIDI] https://www.youtube.com/watch?v=YLNGvvgq3eg Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398 https://www.youtube.com/watch?v=MVYrJJNdrEg Black Mirror: Rachel, Jack and Ashley Too | Official Trailer | Netflix https://www.youtube.com/watch?v=-qIlCo9yqpY

Watch behind the scenes, get early access and join private Discord by supporting us on Patreon: https://patreon.com/mlst https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk In this comprehensive exploration of the field of deep learning with Professor Simon Prince who has just authored an entire text book on Deep Learning, we investigate the technical underpinnings that contribute to the field's unexpected success and confront the enduring conundrums that still perplex AI researchers. Key points discussed include the surprising efficiency of deep learning models, where high-dimensional loss functions are optimized in ways which defy traditional statistical expectations. Professor Prince provides an exposition on the choice of activation functions, architecture design considerations, and overparameterization. We scrutinize the generalization capabilities of neural networks, addressing the seeming paradox of well-performing overparameterized models. Professor Prince challenges popular misconceptions, shedding light on the manifold hypothesis and the role of data geometry in informing the training process. Professor Prince speaks about how layers within neural networks collaborate, recursively reconfiguring instance representations that contribute to both the stability of learning and the emergence of hierarchical feature representations. In addition to the primary discussion on technical elements and learning dynamics, the conversation briefly diverts to audit the implications of AI advancements with ethical concerns. Follow Prof. Prince: https://twitter.com/SimonPrinceAI https://www.linkedin.com/in/simon-prince-615bb9165/ Get the book now! https://mitpress.mit.edu/9780262048644/understanding-deep-learning/ https://udlbook.github.io/udlbook/ Panel: Dr. Tim Scarfe - https://www.linkedin.com/in/ecsquizor/ https://twitter.com/ecsquendor TOC: [00:00:00] Introduction [00:11:03] General Book Discussion [00:15:30] The Neural Metaphor [00:17:56] Back to Book Discussion [00:18:33] Emergence and the Mind [00:29:10] Computation in Transformers [00:31:12] Studio Interview with Prof. Simon Prince [00:31:46] Why Deep Neural Networks Work: Spline Theory [00:40:29] Overparameterization in Deep Learning [00:43:42] Inductive Priors and the Manifold Hypothesis [00:49:31] Universal Function Approximation and Deep Networks [00:59:25] Training vs Inference: Model Bias [01:03:43] Model Generalization Challenges [01:11:47] Purple Segment: Unknown Topic [01:12:45] Visualizations in Deep Learning [01:18:03] Deep Learning Theories Overview [01:24:29] Tricks in Neural Networks [01:30:37] Critiques of ChatGPT [01:42:45] Ethical Considerations in AI References on YT version VD: https://youtu.be/sJXn4Cl4oww

Watch behind the scenes with Bert on Patreon: https://www.patreon.com/posts/bert-de-vries-93230722 https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk Note, there is some mild background music on chapter 1 (Least Action), 3 (Friston) and 5 (Variational Methods) - please skip ahead if annoying. It's a tiny fraction of the overall podcast. YT version: https://youtu.be/2wnJ6E6rQsU Bert de Vries is Professor in the Signal Processing Systems group at Eindhoven University. His research focuses on the development of intelligent autonomous agents that learn from in-situ interactions with their environment. His research draws inspiration from diverse fields including computational neuroscience, Bayesian machine learning, Active Inference and signal processing. Bert believes that development of signal processing systems will in the future be largely automated by autonomously operating agents that learn purposeful from situated environmental interactions. Bert received nis M.Sc. (1986) and Ph.D. (1991) degrees in Electrical Engineering from Eindhoven University of Technology (TU/e) and the University of Florida, respectively. From 1992 to 1999, he worked as a research scientist at Sarnoff Research Center in Princeton (NJ, USA). Since 1999, he has been employed in the hearing aids industry, both in engineering and managerial positions. De Vries was appointed part-time professor in the Signal Processing Systems Group at TU/e in 2012. Contact: https://twitter.com/bertdv0 https://www.tue.nl/en/research/researchers/bert-de-vries https://www.verses.ai/about-us Panel: Dr. Tim Scarfe / Dr. Keith Duggar TOC: [00:00:00] Principle of Least Action [00:05:10] Patreon Teaser [00:05:46] On Friston [00:07:34] Capm Peterson (VERSES) [00:08:20] Variational Methods [00:16:13] Dan Mapes (VERSES) [00:17:12] Engineering with Active Inference [00:20:23] Jason Fox (VERSES) [00:20:51] Riddhi Jain Pitliya [00:21:49] Hearing Aids as Adaptive Agents [00:33:38] Steven Swanson (VERSES) [00:35:46] Main Interview Kick Off, Engineering and Active Inference [00:43:35] Actor / Streaming / Message Passing [00:56:21] Do Agents Lose Flexibility with Maturity? [01:00:50] Language Compression [01:04:37] Marginalisation to Abstraction [01:12:45] Online Structural Learning [01:18:40] Efficiency in Active Inference [01:26:25] SEs become Neuroscientists [01:35:11] Building an Automated Engineer [01:38:58] Robustness and Design vs Grow [01:42:38] RXInfer [01:51:12] Resistance to Active Inference? [01:57:39] Diffusion of Responsibility in a System [02:10:33] Chauvinism in "Understanding" [02:20:08] On Becoming a Bayesian Refs: RXInfer https://biaslab.github.io/rxinfer-website/ Prof. Ariel Caticha https://www.albany.edu/physics/faculty/ariel-caticha Pattern recognition and machine learning (Bishop) https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf Data Analysis: A Bayesian Tutorial (Sivia) https://www.amazon.co.uk/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320 Probability Theory: The Logic of Science (E. T. Jaynes) https://www.amazon.co.uk/Probability-Theory-Principles-Elementary-Applications/dp/0521592712/ #activeinference #artificialintelligence

Please support us https://www.patreon.com/mlst https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk Lance Da Costa aims to advance our understanding of intelligent systems by modelling cognitive systems and improving artificial systems. He's a PhD candidate with Greg Pavliotis and Karl Friston jointly at Imperial College London and UCL, and a student in the Mathematics of Random Systems CDT run by Imperial College London and the University of Oxford. He completed an MRes in Brain Sciences at UCL with Karl Friston and Biswa Sengupta, an MASt in Pure Mathematics at the University of Cambridge with Oscar Randal-Williams, and a BSc in Mathematics at EPFL and the University of Toronto. Summary: Lance did pure math originally but became interested in the brain and AI. He started working with Karl Friston on the free energy principle, which claims all intelligent agents minimize free energy for perception, action, and decision-making. Lance has worked to provide mathematical foundations and proofs for why the free energy principle is true, starting from basic assumptions about agents interacting with their environment. This aims to justify the principle from first physics principles. Dr. Scarfe and Da Costa discuss different approaches to AI - the free energy/active inference approach focused on mimicking human intelligence vs approaches focused on maximizing capability like deep reinforcement learning. Lance argues active inference provides advantages for explainability and safety compared to black box AI systems. It provides a simple, sparse description of intelligence based on a generative model and free energy minimization. They discuss the need for structured learning and acquiring core knowledge to achieve more human-like intelligence. Lance highlights work from Josh Tenenbaum's lab that shows similar learning trajectories to humans in a simple Atari-like environment. Incorporating core knowledge constraints the space of possible generative models the agent can use to represent the world, making learning more sample efficient. Lance argues active inference agents with core knowledge can match human learning capabilities. They discuss how to make generative models interpretable, such as through factor graphs. The goal is to be able to understand the representations and message passing in the model that leads to decisions. In summary, Lance argues active inference provides a principled approach to AI with advantages for explainability, safety, and human-like learning. Combining it with core knowledge and structural learning aims to achieve more human-like artificial intelligence. https://www.lancelotdacosta.com/ https://twitter.com/lancelotdacosta Interviewer: Dr. Tim Scarfe TOC 00:00:00 - Start 00:09:27 - Intelligence 00:12:37 - Priors / structure learning 00:17:21 - Core knowledge 00:29:05 - Intelligence is specialised 00:33:21 - The magic of agents 00:39:30 - Intelligibility of structure learning #artificialintelligence #activeinference

Please support us! https://www.patreon.com/mlst https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk YT version (with intro not found here) https://youtu.be/6iaT-0Dvhnc This is the epic special edition show you have been waiting for! With two of the most brilliant scientists alive today. Atoms, things, agents, ... observers. What even defines an "observer" and what properties must all observers share? How do objects persist in our universe given that their material composition changes over time? What does it mean for a thing to be a thing? And do things supervene on our lower-level physical reality? What does it mean for a thing to have agency? What's the difference between a complex dynamical system with and without agency? Could a rock or an AI catflap have agency? Can the universe be factorised into distinct agents, or is agency diffused? Have you ever pondered about these deep questions about reality? Prof. Friston and Dr. Wolfram have spent their entire careers, some 40+ years each thinking long and hard about these very questions and have developed significant frameworks of reference on their respective journeys (the Wolfram Physics project and the Free Energy principle). Panel: MIT Ph.D Keith Duggar Production: Dr. Tim Scarfe Refs: TED Talk with Stephen: https://www.ted.com/talks/stephen_wolfram_how_to_think_computationally_about_ai_the_universe_and_everything https://writings.stephenwolfram.com/2023/10/how-to-think-computationally-about-ai-the-universe-and-everything/ TOC 00:00:00 - Show kickoff 00:02:38 - Wolfram gets to grips with FEP 00:27:08 - How much control does an agent/observer have 00:34:52 - Observer persistence, what universe seems like to us 00:40:31 - Black holes 00:45:07 - Inside vs outside 00:52:20 - Moving away from the predictable path 00:55:26 - What can observers do 01:06:50 - Self modelling gives agency 01:11:26 - How do you know a thing has agency? 01:22:48 - Deep link between dynamics, ruliad and AI 01:25:52 - Does agency entail free will? Defining Agency 01:32:57 - Where do I probe for agency? 01:39:13 - Why is the universe the way we see it? 01:42:50 - Alien intelligence 01:43:40 - The hard problem of Observers 01:46:20 - Summary thoughts from Wolfram 01:49:35 - Factorisability of FEP 01:57:05 - Patreon interview teaser

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 YT version: https://www.youtube.com/watch?v=c4praCiy9qU Dr. Jeff Beck is a computational neuroscientist studying probabilistic reasoning (decision making under uncertainty) in humans and animals with emphasis on neural representations of uncertainty and cortical implementations of probabilistic inference and learning. His line of research incorporates information theoretic and hierarchical statistical analysis of neural and behavioural data as well as reinforcement learning and active inference. https://www.linkedin.com/in/jeff-beck... https://scholar.google.com/citations?... Interviewer: Dr. Tim Scarfe TOC 00:00:00 Intro 00:00:51 Bayesian / Knowledge 00:14:57 Active inference 00:18:58 Mediation 00:23:44 Philosophy of mind / science 00:29:25 Optimisation 00:42:54 Emergence 00:56:38 Steering emergent systems 01:04:31 Work plan 01:06:06 Representations/Core knowledge #activeinference

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Prof. Melanie Mitchell argues that the concept of "understanding" in AI is ill-defined and multidimensional - we can't simply say an AI system does or doesn't understand. She advocates for rigorously testing AI systems' capabilities using proper experimental methods from cognitive science. Popular benchmarks for intelligence often rely on the assumption that if a human can perform a task, an AI that performs the task must have human-like general intelligence. But benchmarks should evolve as capabilities improve. Large language models show surprising skill on many human tasks but lack common sense and fail at simple things young children can do. Their knowledge comes from statistical relationships in text, not grounded concepts about the world. We don't know if their internal representations actually align with human-like concepts. More granular testing focused on generalization is needed. There are open questions around whether large models' abilities constitute a fundamentally different non-human form of intelligence based on vast statistical correlations across text. Mitchell argues intelligence is situated, domain-specific and grounded in physical experience and evolution. The brain computes but in a specialized way honed by evolution for controlling the body. Extracting "pure" intelligence may not work. Other key points: - Need more focus on proper experimental method in AI research. Developmental psychology offers examples for rigorous testing of cognition. - Reporting instance-level failures rather than just aggregate accuracy can provide insights. - Scaling laws and complex systems science are an interesting area of complexity theory, with applications to understanding cities. - Concepts like "understanding" and "intelligence" in AI force refinement of fuzzy definitions. - Human intelligence may be more collective and social than we realize. AI forces us to rethink concepts we apply anthropomorphically. The overall emphasis is on rigorously building the science of machine cognition through proper experimentation and benchmarking as we assess emerging capabilities. TOC: [00:00:00] Introduction and Munk AI Risk Debate Highlights [05:00:00] Douglas Hofstadter on AI Risk [00:06:56] The Complexity of Defining Intelligence [00:11:20] Examining Understanding in AI Models [00:16:48] Melanie's Insights on AI Understanding Debate [00:22:23] Unveiling the Concept Arc [00:27:57] AI Goals: A Human vs Machine Perspective [00:31:10] Addressing the Extrapolation Challenge in AI [00:36:05] Brain Computation: The Human-AI Parallel [00:38:20] The Arc Challenge: Implications and Insights [00:43:20] The Need for Detailed AI Performance Reporting [00:44:31] Exploring Scaling in Complexity Theory Eratta: Note Tim said around 39 mins that a recent Stanford/DM paper modelling ARC “on GPT-4 got around 60%”. This is not correct and he misremembered. It was actually davinci3, and around 10%, which is still extremely good for a blank slate approach with an LLM and no ARC specific knowledge. Folks on our forum couldn’t reproduce the result. See paper linked below. Books (MUST READ): Artificial Intelligence: A Guide for Thinking Humans (Melanie Mitchell) https://www.amazon.co.uk/Artificial-Intelligence-Guide-Thinking-Humans/dp/B07YBHNM1C/?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=44ccac78973f47e59d745e94967c0f30&camp=1634&creative=6738 Complexity: A Guided Tour (Melanie Mitchell) https://www.amazon.co.uk/Audible-Complexity-A-Guided-Tour?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=3f8bd505d86865c50c02dd7f10b27c05&camp=1634&creative=6738 Show notes (transcript, full references etc) https://atlantic-papyrus-d68.notion.site/Melanie-Mitchell-2-0-15e212560e8e445d8b0131712bad3000?pvs=25 YT version: https://youtu.be/29gkDpR2orc

We explore connections between FEP and enactivism, including tensions raised in a paper critiquing FEP from an enactivist perspective. Dr. Maxwell Ramstead provides background on enactivism emerging from autopoiesis, with a focus on embodied cognition and rejecting information processing/computational views of mind. Chris shares his journey from robotics into FEP, starting as a skeptic but becoming convinced it's the right framework. He notes there are both "high road" and "low road" versions, ranging from embodied to more radically anti-representational stances. He doesn't see a definitive fork between dynamical systems and information theory as the source of conflict. Rather, the notion of operational closure in enactivism seems to be the main sticking point. The group explores definitional issues around structure/organization, boundaries, and operational closure. Maxwell argues the generative model in FEP captures organizational dependencies akin to operational closure. The Markov blanket formalism models structural interfaces. We discuss the concept of goals in cognitive systems - Chris advocates an intentional stance perspective - using notions of goals/intentions if they help explain system dynamics. Goals emerge from beliefs about dynamical trajectories. Prof Friston provides an elegant explanation of how goal-directed behavior naturally falls out of the FEP mathematics in a particular "goldilocks" regime of system scale/dynamics. The conversation explores the idea that many systems simply act "as if" they have goals or models, without necessarily possessing explicit representations. This helps resolve tensions between enactivist and computational perspectives. Throughout the dialogue, Maxwell presses philosophical points about the FEP abolishing what he perceives as false dichotomies in cognitive science such as internalism/externalism. He is critical of enactivists' commitment to bright line divides between subject areas. Prof. Karl Friston - Inventor of the free energy principle https://scholar.google.com/citations?user=q_4u0aoAAAAJ Prof. Chris Buckley - Professor of Neural Computation at Sussex University https://scholar.google.co.uk/citations?user=nWuZ0XcAAAAJ&hl=en Dr. Maxwell Ramstead - Director of Research at VERSES https://scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=fr We address critique in this paper: Laying down a forking path: Tensions between enaction and the free energy principle (Ezequiel A. Di Paolo, Evan Thompson, Randall D. Beere) https://philosophymindscience.org/index.php/phimisci/article/download/9187/8975 Other refs: Multiscale integration: beyond internalism and externalism (Maxwell J D Ramstead) https://pubmed.ncbi.nlm.nih.gov/33627890/ MLST panel: Dr. Tim Scarfe and Dr. Keith Duggar TOC (auto generated): 0:00 - Introduction 0:41 - Defining enactivism and its variants 6:58 - The source of the conflict between dynamical systems and information theory 8:56 - Operational closure in enactivism 10:03 - Goals and intentions 12:35 - The link between dynamical systems and information theory 15:02 - Path integrals and non-equilibrium dynamics 18:38 - Operational closure defined 21:52 - Structure vs. organization in enactivism 24:24 - Markov blankets as interfaces 28:48 - Operational closure in FEP 30:28 - Structure and organization again 31:08 - Dynamics vs. information theory 33:55 - Goals and intentions emerge in the FEP mathematics 36:58 - The Good Regulator Theorem 49:30 - enactivism and its relation to ecological psychology 52:00 - Goals, intentions and beliefs 55:21 - Boundaries and meaning 58:55 - Enactivism's rejection of information theory 1:02:08 - Beliefs vs goals 1:05:06 - Ecological psychology and FEP 1:08:41 - The Good Regulator Theorem 1:18:38 - How goal-directed behavior emerges 1:23:13 - Ontological vs metaphysical boundaries 1:25:20 - Boundaries as maps 1:31:08 - Connections to the maximum entropy principle 1:33:45 - Relations to quantum and relational physics

Please check out Numerai - our sponsor @ http://numer.ai/mlst Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB The Second Law: Resolving the Mystery of the Second Law of Thermodynamics Buy Stephen's book here - https://tinyurl.com/2jj2t9wa The Language Game: How Improvisation Created Language and Changed the World by Morten H. Christiansen and Nick Chater Buy here: https://tinyurl.com/35bvs8be Stephen Wolfram starts by discussing the second law of thermodynamics - the idea that entropy, or disorder, tends to increase over time. He talks about how this law seems intuitively true, but has been difficult to prove. Wolfram outlines his decades-long quest to fully understand the second law, including failed early attempts to simulate particles mixing as a 12-year-old. He explains how irreversibility arises from the computational irreducibility of underlying physical processes coupled with our limited ability as observers to do the computations needed to "decrypt" the microscopic details. The conversation then shifts to discussing language and how concepts allow us to communicate shared ideas between minds positioned in different parts of "rule space." Wolfram talks about the successes and limitations of using large language models to generate Wolfram Language code from natural language prompts. He sees it as a useful tool for getting started programming, but one still needs human refinement. The final part of the conversation focuses on AI safety and governance. Wolfram notes uncontrolled actuation is where things can go wrong with AI systems. He discusses whether AI agents could have intrinsic experiences and goals, how we might build trust networks between AIs, and that managing a system of many AIs may be easier than a single AI. Wolfram emphasizes the need for more philosophical depth in thinking about AI aims, and draws connections between potential solutions and his work on computational irreducibility and physics. Show notes: https://docs.google.com/document/d/1hXNHtvv8KDR7PxCfMh9xOiDFhU3SVDW8ijyxeTq9LHo/edit?usp=sharing Pod version: TBA https://twitter.com/stephen_wolfram TOC: 00:00:00 - Introduction 00:02:34 - Second law book 00:14:01 - Reversibility / entropy / observers / equivalence 00:34:22 - Concepts/language in the ruliad 00:49:04 - Comparison to free energy principle 00:53:58 - ChatGPT / Wolfram / Language 01:00:17 - AI risk Panel: Dr. Tim Scarfe @ecsquendor / Dr. Keith Duggar @DoctorDuggar

Please check out Numerai - our sponsor @ http://numer.ai/mlst Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Professor Jürgen Schmidhuber, the father of artificial intelligence, joins us today. Schmidhuber discussed the history of machine learning, the current state of AI, and his career researching recursive self-improvement, artificial general intelligence and its risks. Schmidhuber pointed out the importance of studying the history of machine learning to properly assign credit for key breakthroughs. He discussed some of the earliest machine learning algorithms. He also highlighted the foundational work of Leibniz, who discovered the chain rule that enables training of deep neural networks, and the ancient Antikythera mechanism, the first known gear-based computer. Schmidhuber discussed limits to recursive self-improvement and artificial general intelligence, including physical constraints like the speed of light and what can be computed. He noted we have no evidence the human brain can do more than traditional computing. Schmidhuber sees humankind as a potential stepping stone to more advanced, spacefaring machine life which may have little interest in humanity. However, he believes commercial incentives point AGI development towards being beneficial and that open-source innovation can help to achieve "AI for all" symbolised by his company's motto "AI∀". Schmidhuber discussed approaches he believes will lead to more general AI, including meta-learning, reinforcement learning, building predictive world models, and curiosity-driven learning. His "fast weight programming" approach from the 1990s involved one network altering another network's connections. This was actually the first Transformer variant, now called an unnormalised linear Transformer. He also described the first GANs in 1990, to implement artificial curiosity. Schmidhuber reflected on his career researching AI. He said his fondest memories were gaining insights that seemed to solve longstanding problems, though new challenges always arose: "then for a brief moment it looks like the greatest thing since sliced bread and and then you get excited ... but then suddenly you realize, oh, it's still not finished. Something important is missing.” Since 1985 he has worked on systems that can recursively improve themselves, constrained only by the limits of physics and computability. He believes continual progress, shaped by both competition and collaboration, will lead to increasingly advanced AI. On AI Risk: Schmidhuber: "To me it's indeed weird. Now there are all these letters coming out warning of the dangers of AI. And I think some of the guys who are writing these letters, they are just seeking attention because they know that AI dystopia are attracting more attention than documentaries about the benefits of AI in healthcare." Schmidhuber believes we should be more concerned with existing threats like nuclear weapons than speculative risks from advanced AI. He said: "As far as I can judge, all of this cannot be stopped but it can be channeled in a very natural way that is good for humankind...there is a tremendous bias towards good AI, meaning AI that is good for humans...I am much more worried about 60 year old technology that can wipe out civilization within two hours, without any AI.” [this is truncated, read show notes] YT: https://youtu.be/q27XMPm5wg8 Show notes: https://docs.google.com/document/d/13-vIetOvhceZq5XZnELRbaazpQbxLbf5Yi7M25CixEE/edit?usp=sharing Note: Interview was recorded 15th June 2023. https://twitter.com/SchmidhuberAI Panel: Dr. Tim Scarfe @ecsquendor / Dr. Keith Duggar @DoctorDuggar Pod version: TBA TOC: [00:00:00] Intro / Numerai [00:00:51] Show Kick Off [00:02:24] Credit Assignment in ML [00:12:51] XRisk [00:20:45] First Transformer variant of 1991 [00:47:20] Which Current Approaches are Good [00:52:42] Autonomy / Curiosity [00:58:42] GANs of 1990 [01:11:29] OpenAI, Moats, Legislation

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB George Hotz and Connor Leahy discuss the crucial challenge of developing beneficial AI that is aligned with human values. Hotz believes truly aligned AI is impossible, while Leahy argues it's a solvable technical challenge.Hotz contends that AI will inevitably pursue power, but distributing AI widely would prevent any single AI from dominating. He advocates open-sourcing AI developments to democratize access. Leahy counters that alignment is necessary to ensure AIs respect human values. Without solving alignment, general AI could ignore or harm humans.They discuss whether AI's tendency to seek power stems from optimization pressure or human-instilled goals. Leahy argues goal-seeking behavior naturally emerges while Hotz believes it reflects human values. Though agreeing on AI's potential dangers, they differ on solutions. Hotz favors accelerating AI progress and distributing capabilities while Leahy wants safeguards put in place.While acknowledging risks like AI-enabled weapons, they debate whether broad access or restrictions better manage threats. Leahy suggests limiting dangerous knowledge, but Hotz insists openness checks government overreach. They concur that coordination and balance of power are key to navigating the AI revolution. Both eagerly anticipate seeing whose ideas prevail as AI progresses. Transcript and notes: https://docs.google.com/document/d/1smkmBY7YqcrhejdbqJOoZHq-59LZVwu-DNdM57IgFcU/edit?usp=sharing Note: this is not a normal episode i.e. the hosts are not part of the debate (and for the record don't agree with Connor or George). TOC: [00:00:00] Introduction to George Hotz and Connor Leahy [00:03:10] George Hotz's Opening Statement: Intelligence and Power [00:08:50] Connor Leahy's Opening Statement: Technical Problem of Alignment and Coordination [00:15:18] George Hotz's Response: Nature of Cooperation and Individual Sovereignty [00:17:32] Discussion on individual sovereignty and defense [00:18:45] Debate on living conditions in America versus Somalia [00:21:57] Talk on the nature of freedom and the aesthetics of life [00:24:02] Discussion on the implications of coordination and conflict in politics [00:33:41] Views on the speed of AI development / hard takeoff [00:35:17] Discussion on potential dangers of AI [00:36:44] Discussion on the effectiveness of current AI [00:40:59] Exploration of potential risks in technology [00:45:01] Discussion on memetic mutation risk [00:52:36] AI alignment and exploitability [00:53:13] Superintelligent AIs and the assumption of good intentions [00:54:52] Humanity’s inconsistency and AI alignment [00:57:57] Stability of the world and the impact of superintelligent AIs [01:02:30] Personal utopia and the limitations of AI alignment [01:05:10] Proposed regulation on limiting the total number of flops [01:06:20] Having access to a powerful AI system [01:18:00] Power dynamics and coordination issues with AI [01:25:44] Humans vs AI in Optimization [01:27:05] The Impact of AI's Power Seeking Behavior [01:29:32] A Debate on the Future of AI

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Join us for a fascinating discussion of the free energy principle with Dr. Maxwell Ramsted, a leading thinker exploring the intersection of math, physics, and philosophy and Director of Research at VERSES. The FEP was proposed by renowned neuroscientist Karl Friston, this principle offers a unifying theory explaining how systems maintain order and their identity. The free energy principle inverts traditional survival logic. Rather than asking what behaviors promote survival, it queries - given things exist, what must they do? The answer: minimizing free energy, or "surprise." Systems persist by constantly ensuring their internal states match anticipated states based on a model of the world. Failure to minimize surprise leads to chaos as systems dissolve into disorder. Thus, the free energy principle elucidates why lifeforms relentlessly model and predict their surroundings. It is an existential imperative counterbalancing entropy. Essentially, this principle describes the mind's pursuit of harmony between expectations and reality. Its relevance spans from cells to societies, underlying order wherever longevity is found. Our discussion explores the technical details and philosophical implications of this paradigm-shifting theory. How does it further our understanding of cognition and intelligence? What insights does it offer about the fundamental patterns and properties of existence? Can it precipitate breakthroughs in disciplines like neuroscience and artificial intelligence? Dr. Ramstead completed his Ph.D. at McGill University in Montreal, Canada in 2019, with frequent research visits to UCL in London, under the supervision of the world’s most cited neuroscientist, Professor Karl Friston (UCL). YT version: https://youtu.be/8qb28P7ksyE https://scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=frhttps://spatialwebfoundation.org/team/maxwell-ramstead/https://www.linkedin.com/in/maxwell-ramstead-43a1991b7/https://twitter.com/mjdramstead VERSES AI: https://www.verses.ai/ Intro: Tim Scarfe (Ph.D) Interviewer: Keith Duggar (Ph.D MIT) TOC: 0:00:00 - Tim Intro 0:08:10 - Intro and philosophy 0:14:26 - Intro to Maxwell 0:18:00 - FEP 0:29:08 - Markov Blankets 0:51:15 - Verses AI / Applications of FEP 1:05:55 - Potential issues with deploying FEP 1:10:50 - Shared knowledge graphs 1:14:29 - XRisk / Ethics 1:24:57 - Strength of Verses 1:28:30 - Misconceptions about FEP, Physics vs philosophy/criticism 1:44:41 - Emergence / consciousness References: Principia Mathematica https://www.abebooks.co.uk/servlet/BookDetailsPL?bi=30567249049 Andy Clark's paper "Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science" (Behavioral and Brain Sciences, 2013) https://pubmed.ncbi.nlm.nih.gov/23663408/ "Math Does Not Represent" by Erik Curiel https://www.youtube.com/watch?v=aA_T20HAzyY A free energy principle for generic quantum systems (Chris Fields et al) https://arxiv.org/pdf/2112.15242.pdf Designing explainable artificial intelligence with active inference https://arxiv.org/abs/2306.04025 Am I Self-Conscious? (Friston) https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full The Meta-Problem of Consciousness https://philarchive.org/archive/CHATMO-32v1 The Map-Territory Fallacy Fallacy https://arxiv.org/abs/2208.06924 A Technical Critique of Some Parts of the Free Energy Principle - Martin Biehl et al https://arxiv.org/abs/2001.06408 WEAK MARKOV BLANKETS IN HIGH-DIMENSIONAL, SPARSELY-COUPLED RANDOM DYNAMICAL SYSTEMS - DALTON A R SAKTHIVADIVEL https://arxiv.org/pdf/2207.07620.pdf

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB The discussion between Tim Scarfe and David Foster provided an in-depth critique of the arguments made by panelists at the Munk AI Debate on whether artificial intelligence poses an existential threat to humanity. While the panelists made thought-provoking points, Scarfe and Foster found their arguments largely speculative, lacking crucial details and evidence to support claims of an impending existential threat. Scarfe and Foster strongly disagreed with Max Tegmark’s position that AI has an unparalleled “blast radius” that could lead to human extinction. Tegmark failed to provide a credible mechanism for how this scenario would unfold in reality. His arguments relied more on speculation about advanced future technologies than on present capabilities and trends. As Foster argued, we cannot conclude AI poses a threat based on speculation alone. Evidence is needed to ground discussions of existential risks in science rather than science fiction fantasies or doomsday scenarios. They found Yann LeCun’s statements too broad and high-level, critiquing him for not providing sufficiently strong arguments or specifics to back his position. While LeCun aptly noted AI remains narrow in scope and far from achieving human-level intelligence, his arguments lacked crucial details on current limitations and why we should not fear superintelligence emerging in the near future. As Scarfe argued, without these details the discussion descended into “philosophy” rather than focusing on evidence and data. Scarfe and Foster also took issue with Yoshua Bengio’s unsubstantiated speculation that machines would necessarily develop a desire for self-preservation that threatens humanity. There is no evidence today’s AI systems are developing human-like general intelligence or desires, let alone that these attributes would manifest in ways dangerous to humans. The question is not whether machines will eventually surpass human intelligence, but how and when this might realistically unfold based on present technological capabilities. Bengio’s arguments relied more on speculation about advanced future technologies than on evidence from current systems and research. In contrast, they strongly agreed with Melanie Mitchell’s view that scenarios of malevolent or misguided superintelligence are speculation, not backed by evidence from AI as it exists today. Claims of an impending “existential threat” from AI are overblown, harmful to progress, and inspire undue fear of technology rather than consideration of its benefits. Mitchell sensibly argued discussions of risks from emerging technologies must be grounded in science and data, not speculation, if we are to make balanced policy and development decisions. Overall, while the debate raised thought-provoking questions about advanced technologies that could eventually transform our world, none of the speakers made a credible evidence-based case that today’s AI poses an existential threat. Scarfe and Foster argued the debate failed to discuss concrete details about current capabilities and limitations of technologies like language models, which remain narrow in scope. General human-level AI is still missing many components, including physical embodiment, emotions, and the "common sense" reasoning that underlies human thinking. Claims of existential threats require extraordinary evidence to justify policy or research restrictions, not speculation. By discussing possibilities rather than probabilities grounded in evidence, the debate failed to substantively advance our thinking on risks from AI and its plausible development in the coming decades. David's new podcast: https://podcasts.apple.com/us/podcast/the-ai-canvas/id1692538973 Generative AI book: https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/

Sponsored Episode - YouAi What if an AI truly knew you—your thoughts, values, aptitudes, and dreams? An AI that could enhance your life in profound ways by amplifying your strengths, augmenting your weaknesses, and connecting you with like-minded souls. That is the vision of YouAi. YouAi founder Dmitri Shapiro believes digitizing our inner lives could unlock tremendous benefits. But mapping the human psyche also poses deep questions. As technology mediates our self-understanding, what risks rendering our minds in bits and algorithms? Could we gain a new means of flourishing or lose something intangible? There are no easy answers, but YouAi offers a vision balanced by hard thinking. Shapiro discussed YouAi's app, which builds personalized AI assistants by learning how individuals think through interactive questions. As people share, YouAi develops a multidimensional model of their mind. Users get a tailored feed of prompts to continue engaging and teaching their AI. YouAi's vision provides a glimpse into a future that could unsettle or fulfill our hopes. As technology mediates understanding ourselves and others, will we risk losing what makes us human or find new means of flourishing? YouAI believes that together, we can build a future where our minds contain infinite potential—and their technology helps unlock it. But we must proceed thoughtfully, upholding human dignity above all else. Our minds shape who we are. And who we can become.Digitise your mind today: YouAi - https://YouAi.aiMIndStudio – https://YouAi.ai/mindstudioYouAi Mind Indexer - https://YouAi.ai/trainJoin the MLST discord and register for the YouAi event on July 13th: https://discord.gg/ESrGqhf5CB TOC: 0:00:00 - Introduction to Mind Digitization 0:09:31 - The YouAi Platform and Personal Applications 0:27:54 - The Potential of Group Alignment 0:30:28 - Applications in Human-to-Human Communication 0:35:43 - Applications in Interfacing with Digital Technology 0:43:41 - Introduction to the Project 0:44:51 - Brain digitization and mind vs. brain 0:49:55 - The Extended Mind and Neurofeedback 0:54:16 - Personalized Learning and the Future of Education 1:02:19 - Privacy and Data Security 1:14:20 - Ethical Considerations of Digitizing the Mind 1:19:49 - The Metaverse and the Future of Digital Identity 1:25:17 - Digital Immortality and Legacy 1:29:09 - The Nature of Consciousness 1:34:11 - Digitization of the Mind 1:35:06 - Potential Inequality in a Digital World 1:38:00 - The Role of Technology in Equalizing or Democratizing Society 1:40:51 - The Future of the Startup and Community Involvement

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk The first 10 mins of audio from Joscha isn't great, it improves after. Transcript and longer summary: https://docs.google.com/document/d/1TUJhlSVbrHf2vWoe6p7xL5tlTK_BGZ140QqqTudF8UI/edit?usp=sharing Dr. Joscha Bach argued that general intelligence emerges from civilization, not individuals. Given our biological constraints, humans cannot achieve a high level of general intelligence on our own. Bach believes AGI may become integrated into all parts of the world, including human minds and bodies. He thinks a future where humans and AGI harmoniously coexist is possible if we develop a shared purpose and incentive to align. However, Bach is uncertain about how AI progress will unfold or which scenarios are most likely. Bach argued that global control and regulation of AI is unrealistic. While regulation may address some concerns, it cannot stop continued progress in AI. He believes individuals determine their own values, so "human values" cannot be formally specified and aligned across humanity. For Bach, the possibility of building beneficial AGI is exciting but much work is still needed to ensure a positive outcome. Connor Leahy believes we have more control over the future than the default outcome might suggest. With sufficient time and effort, humanity could develop the technology and coordination to build a beneficial AGI. However, the default outcome likely leads to an undesirable scenario if we do not actively work to build a better future. Leahy thinks finding values and priorities most humans endorse could help align AI, even if individuals disagree on some values. Leahy argued a future where humans and AGI harmoniously coexist is ideal but will require substantial work to achieve. While regulation faces challenges, it remains worth exploring. Leahy believes limits to progress in AI exist but we are unlikely to reach them before humanity is at risk. He worries even modestly superhuman intelligence could disrupt the status quo if misaligned with human values and priorities. Overall, Bach and Leahy expressed optimism about the possibility of building beneficial AGI but believe we must address risks and challenges proactively. They agreed substantial uncertainty remains around how AI will progress and what scenarios are most plausible. But developing a shared purpose between humans and AI, improving coordination and control, and finding human values to help guide progress could all improve the odds of a beneficial outcome. With openness to new ideas and willingness to consider multiple perspectives, continued discussions like this one could help ensure the future of AI is one that benefits and inspires humanity. TOC: 00:00:00 - Introduction and Background 00:02:54 - Different Perspectives on AGI 00:13:59 - The Importance of AGI 00:23:24 - Existential Risks and the Future of Humanity 00:36:21 - Coherence and Coordination in Society 00:40:53 - Possibilities and Future of AGI 00:44:08 - Coherence and alignment 01:08:32 - The role of values in AI alignment 01:18:33 - The future of AGI and merging with AI 01:22:14 - The limits of AI alignment 01:23:06 - The scalability of intelligence 01:26:15 - Closing statements and future prospects

In this wide-ranging conversation, Tim Scarfe interviews Neel Nanda, a researcher at DeepMind working on mechanistic interpretability, which aims to understand the algorithms and representations learned by machine learning models. Neel discusses how models can represent their thoughts using motifs, circuits, and linear directional features which are often communicated via a "residual stream", an information highway models use to pass information between layers. Neel argues that "superposition", the ability for models to represent more features than they have neurons, is one of the biggest open problems in interpretability. This is because superposition thwarts our ability to understand models by decomposing them into individual units of analysis. Despite this, Neel remains optimistic that ambitious interpretability is possible, citing examples like his work reverse engineering how models do modular addition. However, Neel notes we must start small, build rigorous foundations, and not assume our theoretical frameworks perfectly match reality. The conversation turns to whether models can have goals or agency, with Neel arguing they likely can based on heuristics like models executing long term plans towards some objective. However, we currently lack techniques to build models with specific goals, meaning any goals would likely be learned or emergent. Neel highlights how induction heads, circuits models use to track long range dependencies, seem crucial for phenomena like in-context learning to emerge. On the existential risks from AI, Neel believes we should avoid overly confident claims that models will or will not be dangerous, as we do not understand them enough to make confident theoretical assertions. However, models could pose risks through being misused, having undesirable emergent properties, or being imperfectly aligned. Neel argues we must pursue rigorous empirical work to better understand and ensure model safety, avoid "philosophizing" about definitions of intelligence, and focus on ensuring researchers have standards for what it means to decide a system is "safe" before deploying it. Overall, a thoughtful conversation on one of the most important issues of our time. Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk Neel Nanda: https://www.neelnanda.io/ TOC [00:00:00] Introduction and Neel Nanda's Interests (walk and talk) [00:03:15] Mechanistic Interpretability: Reverse Engineering Neural Networks [00:13:23] Discord questions [00:21:16] Main interview kick-off in studio [00:49:26] Grokking and Sudden Generalization [00:53:18] The Debate on Systematicity and Compositionality [01:19:16] How do ML models represent their thoughts [01:25:51] Do Large Language Models Learn World Models? [01:53:36] Superposition and Interference in Language Models [02:43:15] Transformers discussion [02:49:49] Emergence and In-Context Learning [03:20:02] Superintelligence/XRisk discussion Transcript: https://docs.google.com/document/d/1FK1OepdJMrqpFK-_1Q3LQN6QLyLBvBwWW_5z8WrS1RI/edit?usp=sharing Refs: https://docs.google.com/document/d/115dAroX0PzSduKr5F1V4CWggYcqIoSXYBhcxYktCnqY/edit?usp=sharing

Please check out Numerai - our sponsor using our link @ http://numer.ai/mlst Numerai is a groundbreaking platform which is taking the data science world by storm. Tim has been using Numerai to build state-of-the-art models which predict the stock market, all while being a part of an inspiring community of data scientists from around the globe. They host the Numerai Data Science Tournament, where data scientists like us use their financial dataset to predict future stock market performance. Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk YT version: https://youtu.be/axJtywd9Tbo In this fascinating interview, Dr. Tim Scarfe speaks with renowned philosopher Daniel Dennett about the potential dangers of AI and the concept of "Counterfeit People." Dennett raises concerns about AI being used to create artificial colleagues, and argues that preventing counterfeit AI individuals is crucial for societal trust and security. They delve into Dennett's "Two Black Boxes" thought experiment, the Chinese Room Argument by John Searle, and discuss the implications of AI in terms of reversibility, reontologisation, and realism. Dr. Scarfe and Dennett also examine adversarial LLMs, mental trajectories, and the emergence of consciousness and semanticity in AI systems. Throughout the conversation, they touch upon various philosophical perspectives, including Gilbert Ryle's Ghost in the Machine, Chomsky's work, and the importance of competition in academia. Dennett concludes by highlighting the need for legal and technological barriers to protect against the dangers of counterfeit AI creations. Join Dr. Tim Scarfe and Daniel Dennett in this thought-provoking discussion about the future of AI and the potential challenges we face in preserving our civilization. Don't miss this insightful conversation! TOC: 00:00:00 Intro 00:09:56 Main show kick off 00:12:04 Counterfeit People 00:16:03 Reversibility 00:20:55 Reontologisation 00:24:43 Realism 00:27:48 Adversarial LLMs are out to get us 00:32:34 Exploring mental trajectories and Chomsky 00:38:53 Gilbert Ryle and Ghost in machine and competition in academia 00:44:32 2 Black boxes thought experiment / intentional stance 01:00:11 Chinese room 01:04:49 Singularitarianism 01:07:22 Emergence of consciousness and semanticity References: Tree of Thoughts: Deliberate Problem Solving with Large Language Models https://arxiv.org/abs/2305.10601 The Problem With Counterfeit People (Daniel Dennett) https://www.theatlantic.com/technology/archive/2023/05/problem-counterfeit-people/674075/ The knowledge argument https://en.wikipedia.org/wiki/Knowledge_argument The Intentional Stance https://www.researchgate.net/publication/271180035_The_Intentional_Stance Two Black Boxes: a Fable (Daniel Dennett) https://www.researchgate.net/publication/28762339_Two_Black_Boxes_a_Fable The Chinese Room Argument (John Searle) https://plato.stanford.edu/entries/chinese-room/ https://web-archive.southampton.ac.uk/cogprints.org/7150/1/10.1.1.83.5248.pdf From Bacteria to Bach and Back: The Evolution of Minds (Daniel Dennett) https://www.amazon.co.uk/Bacteria-Bach-Back-Evolution-Minds/dp/014197804X Consciousness Explained (Daniel Dennett) https://www.amazon.co.uk/Consciousness-Explained-Penguin-Science-Dennett/dp/0140128670/ The Mind's I: Fantasies and Reflections on Self and Soul (Hofstadter, Douglas R; Dennett, Daniel C.) https://www.abebooks.co.uk/servlet/BookDetailsPL?bi=31494476184 #DanielDennett #ArtificialIntelligence #CounterfeitPeople

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk In this eye-opening discussion between Tim Scarfe and Prof. Jim Hughes, a professor of gene regulation at Oxford University, they explore the intersection of creativity, genomics, and artificial intelligence. Prof. Hughes brings his expertise in genomics and insights from his interdisciplinary research group, which includes machine learning experts, mathematicians, and molecular biologists. The conversation begins with an overview of Prof. Hughes' background and the importance of creativity in scientific research. They delve into the challenges of unlocking the secrets of the human genome and how machine learning, specifically convolutional neural networks, can assist in decoding genome function. As they discuss validation and interpretability concerns in machine learning, they acknowledge the need for experimental tests and ponder the complex nature of understanding the basic code of life. They touch upon the fascinating world of morphogenesis and emergence, considering the potential crossovers into AI and their implications for self-repairing systems in medicine. Examining the ethical and regulatory aspects of genomics and AI, the duo explores the implications of having access to someone's genome, the potential to predict traits or diseases, and the role of AI in understanding complex genetic signals. They also consider the challenges of keeping up with the rapidly expanding body of scientific research and the pressures faced by researchers in academia. To wrap up the discussion, Tim and Prof. Hughes shed light on the significance of creativity and diversity in scientific research, emphasizing the need for divergent processes and diverse perspectives to foster innovation and avoid consensus-driven convergence. Filmed at https://www.creativemachine.io/Prof. Jim Hughes: https://www.rdm.ox.ac.uk/people/jim-hughesDr. Tim Scarfe: https://xrai.glass/ Table of Contents: 1. [0:00:00] Introduction and Prof. Jim Hughes' background 2. [0:02:48] Creativity and its role in science 3. [0:07:13] Challenges in understanding the human genome 4. [0:13:20] Using convolutional neural networks to decode genome function 5. [0:15:32] Validation and interpretability concerns in machine learning 6. [0:17:56] Challenges in understanding the basic code of life 7. [0:19:36] Morphogenesis, emergence, and potential crossovers into AI 8. [0:21:38] Ethics and regulation in genomics and AI 9. [0:23:30] The role of AI in understanding and managing genetic risks 10. [0:32:37] Creativity and diversity in scientific research

Please check out Numerai - our sponsor @ https://numerai.com/mlst Numerai is a groundbreaking platform which is taking the data science world by storm. Tim has been using Numerai to build state-of-the-art models which predict the stock market, all while being a part of an inspiring community of data scientists from around the globe. They host the Numerai Data Science Tournament, where data scientists like us use their financial dataset to predict future stock market performance. Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk Welcome to an exciting episode featuring an outstanding guest, Robert Miles! Renowned for his extraordinary contributions to understanding AI and its potential impacts on our lives, Robert is an artificial intelligence advocate, researcher, and YouTube sensation. He combines engaging discussions with entertaining content, captivating millions of viewers from around the world. With a strong computer science background, Robert has been actively involved in AI safety projects, focusing on raising awareness about potential risks and benefits of advanced AI systems. His YouTube channel is celebrated for making AI safety discussions accessible to a diverse audience through breaking down complex topics into easy-to-understand nuggets of knowledge, and you might also recognise him from his appearances on Computerphile. In this episode, join us as we dive deep into Robert's journey in the world of AI, exploring his insights on AI alignment, superintelligence, and the role of AI shaping our society and future. We'll discuss topics such as the limits of AI capabilities and physics, AI progress and timelines, human-machine hybrid intelligence, AI in conflict and cooperation with humans, and the convergence of AI communities. Robert Miles: @RobertMilesAI https://twitter.com/robertskmiles https://aisafety.info/ YT version: https://www.youtube.com/watch?v=kMLKbhY0ji0 Panel: Dr. Tim Scarfe Dr. Keith Duggar Joint CTOs - https://xrai.glass/ Refs: Are Emergent Abilities of Large Language Models a Mirage? (Rylan Schaeffer) https://arxiv.org/abs/2304.15004 TOC: Intro [00:00:00] Numerai Sponsor Messsage [00:02:17] AI Alignment [00:04:27] Limits of AI Capabilities and Physics [00:18:00] AI Progress and Timelines [00:23:52] AI Arms Race and Innovation [00:31:11] Human-Machine Hybrid Intelligence [00:38:30] Understanding and Defining Intelligence [00:42:48] AI in Conflict and Cooperation with Humans [00:50:13] Interpretability and Mind Reading in AI [01:03:46] Mechanistic Interpretability and Deconfusion Research [01:05:53] Understanding the core concepts of AI [01:07:40] Moon landing analogy and AI alignment [01:09:42] Cognitive horizon and limits of human intelligence [01:11:42] Funding and focus on AI alignment [01:16:18] Regulating AI technology and potential risks [01:19:17] Aligning AI with human values and its dynamic nature [01:27:04] Cooperation and Allyship [01:29:33] Orthogonality Thesis and Goal Preservation [01:33:15] Anthropomorphic Language and Intelligent Agents [01:35:31] Maintaining Variety and Open-ended Existence [01:36:27] Emergent Abilities of Large Language Models [01:39:22] Convergence vs Emergence [01:44:04] Criticism of X-risk and Alignment Communities [01:49:40] Fusion of AI communities and addressing biases [01:52:51] AI systems integration into society and understanding them [01:53:29] Changing opinions on AI topics and learning from past videos [01:54:23] Utility functions and von Neumann-Morgenstern theorems [01:54:47] AI Safety FAQ project [01:58:06] Building a conversation agent using AI safety dataset [02:00:36]

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk In a historic and candid Senate hearing, OpenAI CEO Sam Altman, Professor Gary Marcus, and IBM's Christina Montgomery discussed the regulatory landscape of AI in the US. The discussion was particularly interesting due to its timing, as it followed the recent release of the EU's proposed AI Act, which could potentially ban American companies like OpenAI and Google from providing API access to generative AI models and impose massive fines for non-compliance. The speakers openly addressed potential risks of AI technology and emphasized the need for precision regulation. This was a unique approach, as historically, US companies have tried their hardest to avoid regulation. The hearing not only showcased the willingness of industry leaders to engage in discussions on regulation but also demonstrated the need for a balanced approach to avoid stifling innovation. The EU AI Act, scheduled to come into power in 2026, is still just a proposal, but it has already raised concerns about its impact on the American tech ecosystem and potential conflicts between US and EU laws. With extraterritorial jurisdiction and provisions targeting open-source developers and software distributors like GitHub, the Act could create more problems than it solves by encouraging unsafe AI practices and limiting access to advanced AI technologies. One core issue with the Act is the designation of foundation models in the highest risk category, primarily due to their open-ended nature. A significant risk theme revolves around users creating harmful content and determining who should be held accountable – the users or the platforms. The Senate hearing served as an essential platform to discuss these pressing concerns and work towards a regulatory framework that promotes both safety and innovation in AI. 00:00 Show 01:35 Legals 03:44 Intro 10:33 Altman intro 14:16 Christina Montgomery 18:20 Gary Marcus 23:15 Jobs 26:01 Scorecards 28:08 Harmful content 29:47 Startups 31:35 What meets the definition of harmful? 32:08 Moratorium 36:11 Social Media 46:17 Gary's take on BingGPT and pivot into policy 48:05 Democratisation

Generative Deep Learning, 2nd Edition [David Foster] https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/ Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk In this conversation, Tim Scarfe and David Foster, the author of 'Generative Deep Learning,' dive deep into the world of generative AI, discussing topics ranging from model families and auto regressive models to the democratization of AI technology and its potential impact on various industries. They explore the connection between language and true intelligence, as well as the limitations of GPT and other large language models. The discussion also covers the importance of task-independent world models, the concept of active inference, and the potential of combining these ideas with transformer and GPT-style models. Ethics and regulation in AI development are also discussed, including the need for transparency in data used to train AI models and the responsibility of developers to ensure their creations are not destructive. The conversation touches on the challenges posed by AI-generated content on copyright laws and the diminishing role of effort and skill in copyright due to generative models. The impact of AI on education and creativity is another key area of discussion, with Tim and David exploring the potential benefits and drawbacks of using AI in the classroom, the need for a balance between traditional learning methods and AI-assisted learning, and the importance of teaching students to use AI tools critically and responsibly. Generative AI in music is also explored, with David and Tim discussing the potential for AI-generated music to change the way we create and consume art, as well as the challenges in training AI models to generate music that captures human emotions and experiences. Throughout the conversation, Tim and David touch on the potential risks and consequences of AI becoming too powerful, the importance of maintaining control over the technology, and the possibility of government intervention and regulation. The discussion concludes with a thought experiment about AI predicting human actions and creating transient capabilities that could lead to doom. TOC: Introducing Generative Deep Learning [00:00:00] Model Families in Generative Modeling [00:02:25] Auto Regressive Models and Recurrence [00:06:26] Language and True Intelligence [00:15:07] Language, Reality, and World Models [00:19:10] AI, Human Experience, and Understanding [00:23:09] GPTs Limitations and World Modeling [00:27:52] Task-Independent Modeling and Cybernetic Loop [00:33:55] Collective Intelligence and Emergence [00:36:01] Active Inference vs. Reinforcement Learning [00:38:02] Combining Active Inference with Transformers [00:41:55] Decentralized AI and Collective Intelligence [00:47:46] Regulation and Ethics in AI Development [00:53:59] AI-Generated Content and Copyright Laws [00:57:06] Effort, Skill, and AI Models in Copyright [00:57:59] AI Alignment and Scale of AI Models [00:59:51] Democratization of AI: GPT-3 and GPT-4 [01:03:20] Context Window Size and Vector Databases [01:10:31] Attention Mechanisms and Hierarchies [01:15:04] Benefits and Limitations of Language Models [01:16:04] AI in Education: Risks and Benefits [01:19:41] AI Tools and Critical Thinking in the Classroom [01:29:26] Impact of Language Models on Assessment and Creativity [01:35:09] Generative AI in Music and Creative Arts [01:47:55] Challenges and Opportunities in Generative Music [01:52:11] AI-Generated Music and Human Emotions [01:54:31] Language Modeling vs. Music Modeling [02:01:58] Democratization of AI and Industry Impact [02:07:38] Recursive Self-Improving Superintelligence [02:12:48] AI Technologies: Positive and Negative Impacts [02:14:44] Runaway AGI and Control Over AI [02:20:35] AI Dangers, Cybercrime, and Ethics [02:23:42]

https://www.perplexity.ai/ https://www.perplexity.ai/iphone https://www.perplexity.ai/android Interview with Aravind Srinivas, CEO and Co-Founder of Perplexity AI – Revolutionizing Learning with Conversational Search Engines Dr. Tim Scarfe talks with Dr. Aravind Srinivas, CEO and Co-Founder of Perplexity AI, about his journey from studying AI and reinforcement learning at UC Berkeley to launching Perplexity – a startup that aims to revolutionize learning through the power of conversational search engines. By combining the strengths of large language models like GPT-* with search engines, Perplexity provides users with direct answers to their questions in a decluttered user interface, making the learning process not only more efficient but also enjoyable. Aravind shares his insights on how advertising can be made more relevant and less intrusive with the help of large language models, emphasizing the importance of transparency in relevance ranking to improve user experience. He also discusses the challenge of balancing the interests of users and advertisers for long-term success. The interview delves into the challenges of maintaining truthfulness and balancing opinions and facts in a world where algorithmic truth is difficult to achieve. Aravind believes that opinionated models can be useful as long as they don't spread misinformation and are transparent about being opinions. He also emphasizes the importance of allowing users to correct or update information, making the platform more adaptable and dynamic. Lastly, Aravind shares his thoughts on embracing a digital society with large language models, stressing the need for frequent and iterative deployments of these models to reduce fear of AI and misinformation. He envisions a future where using AI tools effectively requires clear thinking and first-principle reasoning, ultimately benefiting society as a whole. Education and transparency are crucial to counter potential misuse of AI for political or malicious purposes. YT version: https://youtu.be/_vMOWw3uYvk Aravind Srinivas: https://www.linkedin.com/in/aravind-srinivas-16051987/ https://scholar.google.com/citations?user=GhrKC1gAAAAJ&hl=en https://twitter.com/aravsrinivas?lang=en Interviewer: Dr. Tim Scarfe (CTO XRAI Glass) Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB TOC: Introduction and Background of Perplexity AI [00:00:00] The Importance of a Decluttered UI and User Experience [00:04:19] Advertising in Search Engines and Potential Improvements [00:09:02] Challenges and Opportunities in this new Search Modality [00:18:17] Benefits of Perplexity and Personalized Learning [00:21:27] Objective Truth and Personalized Wikipedia [00:26:34] Opinions and Truth in Answer Engines [00:30:53] Embracing the Digital Society with Language Models [00:37:30] Impact on Jobs and Future of Learning [00:40:13] Educating users on when perplexity works and doesn't work [00:43:13] Improving user experience and the possibilities of voice-to-voice interaction [00:45:04] The future of language models and auto-regressive models [00:49:51] Performance of GPT-4 and potential improvements [00:52:31] Building the ultimate research and knowledge assistant [00:55:33] Revolutionizing note-taking and personal knowledge stores [00:58:16] References: Evaluating Verifiability in Generative Search Engines (Nelson F. Liu et al, Stanford University) https://arxiv.org/pdf/2304.09848.pdf Note: this was a sponsored interview.

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Twitter: https://twitter.com/MLStreetTalk In this exclusive interview, Dr. Tim Scarfe sits down with Minqi Jiang, a leading PhD student at University College London and Meta AI, as they delve into the fascinating world of deep reinforcement learning (RL) and its impact on technology, startups, and research. Discover how Minqi made the crucial decision to pursue a PhD in this exciting field, and learn from his valuable startup experiences and lessons. Minqi shares his insights into balancing serendipity and planning in life and research, and explains the role of objectives and Goodhart's Law in decision-making. Get ready to explore the depths of robustness in RL, two-player zero-sum games, and the differences between RL and supervised learning. As they discuss the role of environment in intelligence, emergence, and abstraction, prepare to be blown away by the possibilities of open-endedness and the intelligence explosion. Learn how language models generate their own training data, the limitations of RL, and the future of software 2.0 with interpretability concerns. From robotics and open-ended learning applications to learning potential metrics and MDPs, this interview is a goldmine of information for anyone interested in AI, RL, and the cutting edge of technology. Don't miss out on this incredible opportunity to learn from a rising star in the AI world! TOC Tech & Startup Background [00:00:00] Pursuing PhD in Deep RL [00:03:59] Startup Lessons [00:11:33] Serendipity vs Planning [00:12:30] Objectives & Decision Making [00:19:19] Minimax Regret & Uncertainty [00:22:57] Robustness in RL & Zero-Sum Games [00:26:14] RL vs Supervised Learning [00:34:04] Exploration & Intelligence [00:41:27] Environment, Emergence, Abstraction [00:46:31] Open-endedness & Intelligence Explosion [00:54:28] Language Models & Training Data [01:04:59] RLHF & Language Models [01:16:37] Creativity in Language Models [01:27:25] Limitations of RL [01:40:58] Software 2.0 & Interpretability [01:45:11] Language Models & Code Reliability [01:48:23] Robust Prioritized Level Replay [01:51:42] Open-ended Learning [01:55:57] Auto-curriculum & Deep RL [02:08:48] Robotics & Open-ended Learning [02:31:05] Learning Potential & MDPs [02:36:20] Universal Function Space [02:42:02] Goal-Directed Learning & Auto-Curricula [02:42:48] Advice & Closing Thoughts [02:44:47] References: - Why Greatness Cannot Be Planned: The Myth of the Objective by Kenneth O. Stanley and Joel Lehman https://www.springer.com/gp/book/9783319155234 - Rethinking Exploration: General Intelligence Requires Rethinking Exploration https://arxiv.org/abs/2106.06860 - The Case for Strong Emergence (Sabine Hossenfelder) https://arxiv.org/abs/2102.07740 - The Game of Life (Conway) https://www.conwaylife.com/ - Toolformer: Teaching Language Models to Generate APIs (Meta AI) https://arxiv.org/abs/2302.04761 - OpenAI's POET: Paired Open-Ended Trailblazer https://arxiv.org/abs/1901.01753 - Schmidhuber's Artificial Curiosity https://people.idsia.ch/~juergen/interest.html - Gödel Machines https://people.idsia.ch/~juergen/goedelmachine.html - PowerPlay https://arxiv.org/abs/1112.5309 - Robust Prioritized Level Replay: https://openreview.net/forum?id=NfZ6g2OmXEk - Unsupervised Environment Design: https://arxiv.org/abs/2012.02096 - Excel: Evolving Curriculum Learning for Deep Reinforcement Learning https://arxiv.org/abs/1901.05431 - Go-Explore: A New Approach for Hard-Exploration Problems https://arxiv.org/abs/1901.10995 - Learning with AMIGo: Adversarially Motivated Intrinsic Goals https://www.researchgate.net/publication/342377312_Learning_with_AMIGo_Adversarially_Motivated_Intrinsic_Goals PRML https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf Sutton and Barto https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Twitter: https://twitter.com/MLStreetTalk Chris Eliasmith is a renowned interdisciplinary researcher, author, and professor at the University of Waterloo, where he holds the prestigious Canada Research Chair in Theoretical Neuroscience. As the Founding Director of the Centre for Theoretical Neuroscience, Eliasmith leads the Computational Neuroscience Research Group in exploring the mysteries of the brain and its complex functions. His groundbreaking work, including the Neural Engineering Framework, Neural Engineering Objects software environment, and the Semantic Pointer Architecture, has led to the development of Spaun, the most advanced functional brain simulation to date. Among his numerous achievements, Eliasmith has received the 2015 NSERC "Polany-ee" Award and authored two influential books, "How to Build a Brain" and "Neural Engineering." Chris' homepage: http://arts.uwaterloo.ca/~celiasmi/ Interviewers: Dr. Tim Scarfe and Dr. Keith Duggar TOC: Intro to Chris [00:00:00] Continuous Representation in Biologically Plausible Neural Networks [00:06:49] Legendre Memory Unit and Spatial Semantic Pointer [00:14:36] Large Contexts and Data in Language Models [00:20:30] Spatial Semantic Pointers and Continuous Representations [00:24:38] Auto Convolution [00:30:12] Abstractions and the Continuity [00:36:33] Compression, Sparsity, and Brain Representations [00:42:52] Continual Learning and Real-World Interactions [00:48:05] Robust Generalization in LLMs and Priors [00:56:11] Chip design [01:00:41] Chomsky + Computational Power of NNs and Recursion [01:04:02] Spiking Neural Networks and Applications [01:13:07] Limits of Empirical Learning [01:22:43] Philosophy of Mind, Consciousness etc [01:25:35] Future of human machine interaction [01:41:28] Future research and advice to young researchers [01:45:06] Refs: http://compneuro.uwaterloo.ca/publications/dumont2023.html  http://compneuro.uwaterloo.ca/publications/voelker2019lmu.html  http://compneuro.uwaterloo.ca/publications/voelker2018.html http://compneuro.uwaterloo.ca/publications/lu2019.html  https://www.youtube.com/watch?v=I5h-xjddzlY

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 In this podcast with the legendary Connor Leahy (CEO Conjecture) recorded in Dec 2022, we discuss various topics related to artificial intelligence (AI), including AI alignment, the success of ChatGPT, the potential threats of artificial general intelligence (AGI), and the challenges of balancing research and product development at his company, Conjecture. He emphasizes the importance of empathy, dehumanizing our thinking to avoid anthropomorphic biases, and the value of real-world experiences in learning and personal growth. The conversation also covers the Orthogonality Thesis, AI preferences, the mystery of mode collapse, and the paradox of AI alignment. Connor Leahy expresses concern about the rapid development of AI and the potential dangers it poses, especially as AI systems become more powerful and integrated into society. He argues that we need a better understanding of AI systems to ensure their safe and beneficial development. The discussion also touches on the concept of "futuristic whack-a-mole," where futurists predict potential AGI threats, and others try to come up with solutions for those specific scenarios. However, the problem lies in the fact that there could be many more scenarios that neither party can think of, especially when dealing with a system that's smarter than humans. https://www.linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse Interviewer: Dr. Tim Scarfe (Innovation CTO @ XRAI Glass https://xrai.glass/) TOC: The success of ChatGPT and its impact on the AI field [00:00:00] Subjective experience [00:15:12] AI Architectural discussion including RLHF [00:18:04] The paradox of AI alignment and the future of AI in society [00:31:44] The impact of AI on society and politics [00:36:11] Future shock levels and the challenges of predicting the future [00:45:58] Long termism and existential risk [00:48:23] Consequentialism vs. deontology in rationalism [00:53:39] The Rationalist Community and its Challenges [01:07:37] AI Alignment and Conjecture [01:14:15] Orthogonality Thesis and AI Preferences [01:17:01] Challenges in AI Alignment [01:20:28] Mechanistic Interpretability in Neural Networks [01:24:54] Building Cleaner Neural Networks [01:31:36] Cognitive horizons / The problem with rapid AI development [01:34:52] Founding Conjecture and raising funds [01:39:36] Inefficiencies in the market and seizing opportunities [01:45:38] Charisma, authenticity, and leadership in startups [01:52:13] Autistic culture and empathy [01:55:26] Learning from real-world experiences [02:01:57] Technical empathy and transhumanism [02:07:18] Moral status and the limits of empathy [02:15:33] Anthropomorphic Thinking and Consequentialism [02:17:42] Conjecture: Balancing Research and Product Development [02:20:37] Epistemology Team at Conjecture [02:31:07] Interpretability and Deception in AGI [02:36:23] Futuristic whack-a-mole and predicting AGI threats [02:38:27] Refs: 1. OpenAI's ChatGPT: https://chat.openai.com/ 2. The Mystery of Mode Collapse (Article): https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse 3. The Rationalist Guide to the Galaxy https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795 5. Alfred Korzybski: https://en.wikipedia.org/wiki/Alfred_Korzybski 6. Instrumental Convergence: https://en.wikipedia.org/wiki/Instrumental_convergence 7. Orthogonality Thesis: https://en.wikipedia.org/wiki/Orthogonality_thesis 8. Brian Tomasik's Essays on Reducing Suffering: https://reducing-suffering.org/ 9. Epistemological Framing for AI Alignment Research: https://www.lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research 10. How to Defeat Mind readers: https://www.alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers 11. Society of mind: https://www.amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Send us a voice message which you want us to publish: https://podcasters.spotify.com/pod/show/machinelearningstreettalk/message In a recent open letter, over 1500 individuals called for a six-month pause on the development of advanced AI systems, expressing concerns over the potential risks AI poses to society and humanity. However, there are issues with this approach, including global competition, unstoppable progress, potential benefits, and the need to manage risks instead of avoiding them. Decision theorist Eliezer Yudkowsky took it a step further in a Time magazine article, calling for an indefinite and worldwide moratorium on Artificial General Intelligence (AGI) development, warning of potential catastrophe if AGI exceeds human intelligence. Yudkowsky urged for an immediate halt to all large AI training runs and the shutdown of major GPU clusters, calling for international cooperation to enforce these measures. However, several counterarguments question the validity of Yudkowsky's concerns: 1. Hard limits on AGI 2. Dismissing AI extinction risk 3. Collective action problem 4. Misplaced focus on AI threats While the potential risks of AGI cannot be ignored, it is essential to consider various arguments and potential solutions before making drastic decisions. As AI continues to advance, it is crucial for researchers, policymakers, and society as a whole to engage in open and honest discussions about the potential consequences and the best path forward. With a balanced approach to AGI development, we may be able to harness its power for the betterment of humanity while mitigating its risks. Eliezer Yudkowsky: https://en.wikipedia.org/wiki/Eliezer_Yudkowsky Connor Leahy: https://twitter.com/NPCollapse (we will release that interview soon) Gary Marcus: http://garymarcus.com/index.html Tim Scarfe is the innovation CTO of XRAI Glass: https://xrai.glass/ Gary clip filmed at AIUK https://ai-uk.turing.ac.uk/programme/ and our appreciation to them for giving us a press pass. Check out their conference next year! WIRED clip from Gary came from here: https://www.youtube.com/watch?v=Puo3VkPkNZ4 Refs: Statement from the listed authors of Stochastic Parrots on the “AI pause” letterTimnit Gebru, Emily M. Bender, Angelina McMillan-Major, Margaret Mitchell https://www.dair-institute.org/blog/letter-statement-March2023 Eliezer Yudkowsky on Lex: https://www.youtube.com/watch?v=AaTRHFaaPG8 Pause Giant AI Experiments: An Open Letter https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Pausing AI Developments Isn't Enough. We Need to Shut it All Down (Eliezer Yudkowsky) https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

HUGE ANNOUNCEMENT, CHATGPT+WOLFRAM! You saw it HERE first! YT version: https://youtu.be/z5WZhCBRDpU Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Stephen's announcement post: https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ OpenAI's announcement post: https://openai.com/blog/chatgpt-plugins In an era of technology and innovation, few individuals have left as indelible a mark on the fabric of modern science as our esteemed guest, Dr. Steven Wolfram. Dr. Wolfram is a renowned polymath who has made significant contributions to the fields of physics, computer science, and mathematics. A prodigious young man too, Wolfram earned a Ph.D. in theoretical physics from the California Institute of Technology by the age of 20. He became the youngest recipient of the prestigious MacArthur Fellowship at the age of 21. Wolfram's groundbreaking computational tool, Mathematica, was launched in 1988 and has become a cornerstone for researchers and innovators worldwide. In 2002, he published "A New Kind of Science," a paradigm-shifting work that explores the foundations of science through the lens of computational systems. In 2009, Wolfram created Wolfram Alpha, a computational knowledge engine utilized by millions of users worldwide. His current focus is on the Wolfram Language, a powerful programming language designed to democratize access to cutting-edge technology. Wolfram's numerous accolades include honorary doctorates and fellowships from prestigious institutions. As an influential thinker, Dr. Wolfram has dedicated his life to unraveling the mysteries of the universe and making computation accessible to all. First of all... we have an announcement to make, you heard it FIRST here on MLST! .... Intro [00:00:00] Big announcement! Wolfram + ChatGPT! [00:02:57] What does it mean to understand? [00:05:33] Feeding information back into the model [00:13:48] Semantics and cognitive categories [00:20:09] Navigating the ruliad [00:23:50] Computational irreducibility [00:31:39] Conceivability and interestingness [00:38:43] Human intelligible sciences [00:43:43]

YT version: https://youtu.be/P1j3VoKBxbc (references in pinned comment) Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Dan McQuillan, a visionary in digital culture and social innovation, emphasizes the importance of understanding technology's complex relationship with society. As an academic at Goldsmiths, University of London, he fosters interdisciplinary collaboration and champions data-driven equity and ethical technology. Dan's career includes roles at Amnesty International and Social Innovation Camp, showcasing technology's potential to empower and bring about positive change. In this conversation, we discuss the challenges and opportunities at the intersection of technology and society, exploring the profound impact of our digital world. Interviewer: Dr. Tim Scarfe [00:00:00] Dan's background and journey to academia [00:03:30] Dan's background and journey to academia [00:04:10] Writing the book "Resisting AI" [00:08:30] Necropolitics and its relation to AI [00:10:06] AI as a new form of colonization [00:12:57] LLMs as a new form of neo-techno-imperialism [00:15:47] Technology for good and AGI's skewed worldview [00:17:49] Transhumanism, eugenics, and intelligence [00:20:45] Valuing differences (disability) and challenging societal norms [00:26:08] Re-ontologizing and the philosophy of information [00:28:19] New materialism and the impact of technology on society [00:30:32] Intelligence, meaning, and materiality [00:31:43] The constraints of physical laws and the importance of science [00:32:44] Exploring possibilities to reduce suffering and increase well-being [00:33:29] The division between meaning and material in our experiences [00:35:36] Machine learning, data science, and neoplatonic approach to understanding reality [00:37:56] Different understandings of cognition, thought, and consciousness [00:39:15] Enactivism and its variants in cognitive science [00:40:58] Jordan Peterson [00:44:47] Relationism, relativism, and finding the correct relational framework [00:47:42] Recognizing privilege and its impact on social interactions [00:49:10] Intersectionality / Feminist thinking and the concept of care in social structures [00:51:46] Intersectionality and its role in understanding social inequalities [00:54:26] The entanglement of history, technology, and politics [00:57:39] ChatGPT article - we come to bury ChatGPT [00:59:41] Statistical pattern learning and convincing patterns in AI [01:01:27] Anthropomorphization and understanding in AI [01:03:26] AI in education and critical thinking [01:06:09] European Union policies and trustable AI [01:07:52] AI reliability and the halo effect [01:09:26] AI as a tool enmeshed in society [01:13:49] Luddites [01:15:16] AI is a scam [01:15:31] AI and Social Relations [01:16:49] Invisible Labor in AI and Machine Learning [01:21:09] Exploititative AI / alignment [01:23:50] Science fiction AI / moral frameworks [01:27:22] Discussing Stochastic Parrots and Nihilism [01:30:36] Human Intelligence vs. Language Models [01:32:22] Image Recognition and Emulation vs. Experience [01:34:32] Thought Experiments and Philosophy in AI Ethics (mimicry) [01:41:23] Abstraction, reduction, and grounding in reality [01:43:13] Process philosophy and the possibility of change [01:49:55] Mental health, AI, and epistemic injustice [01:50:30] Hermeneutic injustice and gendered techniques [01:53:57] AI and politics [01:59:24] Epistemic injustice and testimonial injustice [02:11:46] Fascism and AI discussion [02:13:24] Violence in various systems [02:16:52] Recognizing systemic violence [02:22:35] Fascism in Today's Society [02:33:33] Pace and Scale of Technological Change [02:37:38] Alternative approaches to AI and society [02:44:09] Self-Organization at Successive Scales / cybernetics

Support us! https://www.patreon.com/mlst   MLST Discord: https://discord.gg/aNPkGUQtc5 We are honoured to welcome Dr. Joel Lehman, an eminent machine learning research scientist, whose work in AI safety, reinforcement learning, creative open-ended search algorithms, and indeed the philosophy of open-endedness and abandoning objectives has paved the way for innovative ideas that challenge our preconceptions and inspire new visions for the future. Dr. Lehman's thought-provoking book, "Why Greatness Cannot Be Planned" penned with with our MLST favourite Professor Kenneth Stanley has left an indelible mark on the field and profoundly impacted the way we view innovation and the serendipitous nature of discovery. Those of you who haven't watched our special edition show on that, should do so at your earliest convenience! Building upon this foundation, Dr. Lehman has ventured into the domain of AI systems that embody principles of love, care, responsibility, respect, and knowledge, drawing from the works of Maslow, Erich Fromm, and positive psychology. YT version: https://youtu.be/23-TXgJEv-Q http://joellehman.com/ https://twitter.com/joelbot3000 Interviewer: Dr. Tim Scarfe TOC: Intro [00:00:00] Model [00:04:26] Intro and Paper Intro [00:08:52] Subjectivity [00:16:07] Reflections on Greatness Book [00:19:30] Representing Subjectivity [00:29:24] Nagal's Bat [00:31:49] Abstraction [00:38:58] Love as Action Rather Than Feeling [00:42:58] Reontologisation [00:57:38] Self Help [01:04:15] Meditation [01:09:02] The Human Reward Function / Effective... [01:16:52] Machine Hate [01:28:32] Societal Harms [01:31:41] Lenses We Use Obscuring Reality [01:56:36] Meta Optimisation and Evolution [02:03:14] Conclusion [02:07:06] References: What Is It Like to Be a Bat? (Thomas Nagel) https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/humananimalstudies/lectures/32/nagel_bat.pdf Why Greatness Cannot Be Planned: The Myth of the Objective (Kenneth O. Stanley and Joel Lehman) https://link.springer.com/book/10.1007/978-3-319-15524-1  Machine Love (Joel Lehman) https://arxiv.org/abs/2302.09248  How effective altruists ignored risk (Carla Cremer) https://www.vox.com/future-perfect/23569519/effective-altrusim-sam-bankman-fried-will-macaskill-ea-risk-decentralization-philanthropy Philosophy tube - The Rich Have Their Own Ethics: Effective Altruism https://www.youtube.com/watch?v=Lm0vHQYKI-Y Abandoning Objectives: Evolution through the Search for Novelty Alone (Joel Lehman and Kenneth O. Stanley) https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Dr. Raphaël Millière is the 2020 Robert A. Burt Presidential Scholar in Society and Neuroscience in the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. His research draws from his expertise in philosophy and cognitive science to explore the implications of recent progress in deep learning for models of human cognition, as well as various issues in ethics and aesthetics. He is also investigating what underlies the capacity to represent oneself as oneself at a fundamental level, in humans and non-human animals; as well as the role that self-representation plays in perception, action, and memory. In a world where technology is rapidly advancing, Dr. Millière is striving to gain a better understanding of how artificial neural networks work, and to establish fair and meaningful comparisons between humans and machines in various domains in order to shed light on the implications of artificial intelligence for our lives. https://www.raphaelmilliere.com/ https://twitter.com/raphaelmilliere Here is a version with hesitation sounds like "um" removed if you prefer (I didn't notice them personally): https://share.descript.com/view/aGelyTl2xpN YT: https://www.youtube.com/watch?v=fhn6ZtD6XeE TOC: Intro to Raphael [00:00:00] Intro: Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière) [00:01:18] Show Kick off [00:07:10] LLMs [00:08:37] Semantic Competence/Understanding [00:18:28] Forming Analogies/JPG Compression Article [00:30:17] Compositional Generalisation [00:37:28] Systematicity [00:47:08] Language of Thought [00:51:28] Bigbench (Conceptual Combinations) [00:57:37] Symbol Grounding [01:11:13] World Models [01:26:43] Theory of Mind [01:30:57] Refs (this is truncated, full list on YT video description): Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière) https://nautil.us/moving-beyond-mimicry-in-artificial-intelligence-238504/ On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (Bender et al) https://dl.acm.org/doi/10.1145/3442188.3445922 ChatGPT Is a Blurry JPEG of the Web (Ted Chiang) https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web The Debate Over Understanding in AI's Large Language Models (Melanie Mitchell) https://arxiv.org/abs/2210.13966 Talking About Large Language Models (Murray Shanahan) https://arxiv.org/abs/2212.03551 Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (Bender) https://aclanthology.org/2020.acl-main.463/ The symbol grounding problem (Stevan Harnad) https://arxiv.org/html/cs/9906002 Why the Abstraction and Reasoning Corpus is interesting and important for AI (Mitchell) https://aiguide.substack.com/p/why-the-abstraction-and-reasoning Linguistic relativity (Sapir–Whorf hypothesis) https://en.wikipedia.org/wiki/Linguistic_relativity Cooperative principle (Grice's four maxims of conversation - quantity, quality, relation, and manner) https://en.wikipedia.org/wiki/Cooperative_principle

This show is sponsored by Numerai, please visit them here with our sponsor link (we would really appreciate it) http://numer.ai/mlst  Prof. Karl Friston recently proposed a vision of artificial intelligence that goes beyond machines and algorithms, and embraces humans and nature as part of a cyber-physical ecosystem of intelligence. This vision is based on the principle of active inference, which states that intelligent systems can learn from their observations and act on their environment to reduce uncertainty and achieve their goals. This leads to a formal account of collective intelligence that rests on shared narratives and goals.  To realize this vision, Friston suggests developing a shared hyper-spatial modelling language and transaction protocol, as well as novel methods for measuring and optimizing collective intelligence. This could harness the power of artificial intelligence for the common good, without compromising human dignity or autonomy. It also challenges us to rethink our relationship with technology, nature, and each other, and invites us to join a global community of sense-makers who are curious about the world and eager to improve it. YT version: https://www.youtube.com/watch?v=V_VXOdf1NMw Support us! https://www.patreon.com/mlst  MLST Discord: https://discord.gg/aNPkGUQtc5 TOC:  Intro [00:00:00] Numerai (Sponsor segment) [00:07:10] Designing Ecosystems of Intelligence from First Principles (Friston et al) [00:09:48] Information / Infosphere and human agency [00:18:30] Intelligence [00:31:38] Reductionism [00:39:36] Universalism [00:44:46] Emergence [00:54:23] Markov blankets [01:02:11] Whole part relationships / structure learning [01:22:33] Enactivism [01:29:23] Knowledge and Language [01:43:53] ChatGPT [01:50:56] Ethics (is-ought) [02:07:55] Can people be evil? [02:35:06] Ethics in Al, subjectiveness [02:39:05] Final thoughts [02:57:00] References: Designing Ecosystems of Intelligence from First Principles (Friston et al) https://arxiv.org/abs/2212.01354 GLOM - How to represent part-whole hierarchies in a neural network (Hinton) https://arxiv.org/pdf/2102.12627.pdf Seven Brief Lessons on Physics (Carlo Rovelli) https://www.amazon.co.uk/Seven-Brief-Lessons-Physics-Rovelli/dp/0141981725 How Emotions Are Made: The Secret Life of the Brain (Lisa Feldman Barrett) https://www.amazon.co.uk/How-Emotions-Are-Made-Secret/dp/B01N3D4OON Am I Self-Conscious? (Or Does Self-Organization Entail Self-Consciousness?) (Karl Friston) https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full Integrated information theory (Giulio Tononi) https://en.wikipedia.org/wiki/Integrated_information_theory

Access Numerai here: http://numer.ai/mlst Michael Oliver is the Chief Scientist at Numerai, a hedge fund that crowdsources machine learning models from data scientists. He has a PhD in Computational Neuroscience from UC Berkeley and was a postdoctoral researcher at the Allen Institute for Brain Science before joining Numerai in 2020. He is also the host of Numerai Quant Club, a YouTube series where he discusses Numerai’s research, data and challenges. YT version: https://youtu.be/61s8lLU7sFg TOC: [00:00:00] Introduction to Michael and Numerai [00:02:03] Understanding / new Bing [00:22:47] Quant vs Neuroscience [00:36:43] Role of language in cognition and planning, and subjective...  [00:45:47] Boundaries in finance modelling [00:48:00] Numerai [00:57:37] Aggregation systems [01:00:52] Getting started on Numeral [01:03:21] What models are people using [01:04:23] Numerai Problem Setup [01:05:49] Regimes in financial data and quant talk [01:11:18] Esoteric approaches used on Numeral? [01:13:59]  Curse of dimensionality [01:16:32] Metrics [01:19:10] Outro References: Growing Neural Cellular Automata (Alexander Mordvintsev) https://distill.pub/2020/growing-ca/ A Thousand Brains: A New Theory of Intelligence (Jeff Hawkins) https://www.amazon.fr/Thousand-Brains-New-Theory-Intelligence/dp/1541675819 Perceptual Neuroscience: The Cerebral Cortex (Vernon B. Mountcastle) https://www.amazon.ca/Perceptual-Neuroscience-Cerebral-Vernon-Mountcastle/dp/0674661885 Numerai Quant Club with Michael Oliver https://www.youtube.com/watch?v=eLIxarbDXuQ&list=PLz3D6SeXhT3tTu8rhZmjwDZpkKi-UPO1F Numerai YT channel https://www.youtube.com/@Numerai/featured Support us! https://www.patreon.com/mlst  MLST Discord: https://discord.gg/aNPkGUQtc5

Support us! https://www.patreon.com/mlst   MLST Discord: https://discord.gg/aNPkGUQtc5 Christopher Summerfield, Department of Experimental Psychology, University of Oxford is a Professor of Cognitive Neuroscience at the University of Oxford and a Research Scientist at Deepmind UK. His work focusses on the neural and computational mechanisms by which humans make decisions. Chris has just released an incredible new book on AI called "Natural General Intelligence". It's my favourite book on AI I have read so so far.  The book explores the algorithms and architectures that are driving progress in AI research, and discusses intelligence in the language of psychology and biology, using examples and analogies to be comprehensible to a wide audience. It also tackles longstanding theoretical questions about the nature of thought and knowledge. With Chris' permission, I read out a summarised version of Chapter 2 from his book on which was on Intelligence during the 30 minute MLST introduction.   Buy his book here: https://global.oup.com/academic/product/natural-general-intelligence-9780192843883?cc=gb&lang=en& YT version: https://youtu.be/31VRbxAl3t0 Interviewer: Dr. Tim Scarfe TOC: [00:00:00] Walk and talk with Chris on Knowledge and Abstractions [00:04:08] Intro to Chris and his book [00:05:55] (Intro) Tim reads Chapter 2: Intelligence  [00:09:28] Intro continued: Goodhart's law [00:15:37] Intro continued: The "swiss cheese" situation   [00:20:23] Intro continued: On Human Knowledge [00:23:37] Intro continued: Neats and Scruffies [00:30:22] Interview kick off  [00:31:59] What does it mean to understand? [00:36:18] Aligning our language models [00:40:17] Creativity  [00:41:40] "Meta" AI and basins of attraction  [00:51:23] What can Neuroscience impart to AI [00:54:43] Sutton, neats and scruffies and human alignment [01:02:05] Reward is enough [01:19:46] Jon Von Neumann and Intelligence [01:23:56] Compositionality References: The Language Game (Morten H. Christiansen, Nick Chater https://www.penguin.co.uk/books/441689/the-language-game-by-morten-h-christiansen-and--nick-chater/9781787633483 Theory of general factor (Spearman) https://www.proquest.com/openview/7c2c7dd23910c89e1fc401e8bb37c3d0/1?pq-origsite=gscholar&cbl=1818401 Intelligence Reframed (Howard Gardner) https://books.google.co.uk/books?hl=en&lr=&id=Qkw4DgAAQBAJ&oi=fnd&pg=PT6&dq=howard+gardner+multiple+intelligences&ots=ERUU0u5Usq&sig=XqiDgNUIkb3K9XBq0vNbFmXWKFs#v=onepage&q=howard%20gardner%20multiple%20intelligences&f=false The master algorithm (Pedro Domingos) https://www.amazon.co.uk/Master-Algorithm-Ultimate-Learning-Machine/dp/0241004543 A Thousand Brains: A New Theory of Intelligence (Jeff Hawkins) https://www.amazon.co.uk/Thousand-Brains-New-Theory-Intelligence/dp/1541675819 The bitter lesson (Rich Sutton) http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Support us! https://www.patreon.com/mlst  MLST Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/i9VPPmQn9HQ Edward Grefenstette is a Franco-American computer scientist who currently serves as Head of Machine Learning at Cohere and Honorary Professor at UCL. He has previously been a research scientist at Facebook AI Research and staff research scientist at DeepMind, and was also the CTO of Dark Blue Labs. Prior to his move to industry, Edward was a Fulford Junior Research Fellow at Somerville College, University of Oxford, and was lecturing at Hertford College. He obtained his BSc in Physics and Philosophy from the University of Sheffield and did graduate work in the philosophy departments at the University of St Andrews. His research draws on topics and methods from Machine Learning, Computational Linguistics and Quantum Information Theory, and has done work implementing and evaluating compositional vector-based models of natural language semantics and empirical semantic knowledge discovery. https://www.egrefen.com/ https://cohere.ai/ TOC: [00:00:00] Introduction [00:02:52] Differential Semantics [00:06:56] Concepts [00:10:20] Ontology [00:14:02] Pragmatics [00:16:55] Code helps with language [00:19:02] Montague [00:22:13] RLHF [00:31:54] Swiss cheese problem / retrieval augmented [00:37:06] Intelligence / Agency [00:43:33] Creativity [00:46:41] Common sense [00:53:46] Thinking vs knowing References: Large language models are not zero-shot communicators (Laura Ruis) https://arxiv.org/abs/2210.14986 Some remarks on Large Language Models (Yoav Goldberg) https://gist.github.com/yoavg/59d174608e92e845c8994ac2e234c8a9 Quantum Natural Language Processing (Bob Coecke) https://www.cs.ox.ac.uk/people/bob.coecke/QNLP-ACT.pdf Constitutional AI: Harmlessness from AI Feedback https://www.anthropic.com/constitutional.pdf Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Patrick Lewis) https://www.patricklewis.io/publication/rag/ Natural General Intelligence (Prof. Christopher Summerfield) https://global.oup.com/academic/product/natural-general-intelligence-9780192843883 ChatGPT with Rob Miles - Computerphile https://www.youtube.com/watch?v=viJt_DXTfwA

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/Vbi288CKgis Michael Levin is a Distinguished Professor in the Biology department at Tufts University, and the holder of the Vannevar Bush endowed Chair. He is the Director of the Allen Discovery Center at Tufts and the Tufts Center for Regenerative and Developmental Biology. His research focuses on understanding the biophysical mechanisms of pattern regulation and harnessing endogenous bioelectric dynamics for rational control of growth and form. The capacity to generate a complex, behaving organism from the single cell of a fertilized egg is one of the most amazing aspects of biology. Levin' lab integrates approaches from developmental biology, computer science, and cognitive science to investigate the emergence of form and function. Using biophysical and computational modeling approaches, they seek to understand the collective intelligence of cells, as they navigate physiological, transcriptional, morphognetic, and behavioral spaces. They develop conceptual frameworks for basal cognition and diverse intelligence, including synthetic organisms and AI. Also joining us this evening is Irina Rish. Irina is a Full Professor at the Université de Montréal's Computer Science and Operations Research department, a core member of Mila - Quebec AI Institute, as well as the holder of the Canada CIFAR AI Chair and the Canadian Excellence Research Chair in Autonomous AI. She has a PhD in AI from UC Irvine. Her research focuses on machine learning, neural data analysis, neuroscience-inspired AI, continual lifelong learning, optimization algorithms, sparse modelling, probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis.  Interviewer: Dr. Tim Scarfe TOC: [00:00:00] Introduction [00:02:09] Emergence [00:13:16] Scaling Laws [00:23:12] Intelligence [00:44:36] Transhumanism Prof. Michael Levin https://en.wikipedia.org/wiki/Michael_Levin_(biologist) https://www.drmichaellevin.org/ https://twitter.com/drmichaellevin Prof. Irina Rish https://twitter.com/irinarish https://irina-rish.com/

Dr. Patrick Lewis is a London-based AI and Natural Language Processing Research Scientist, working at co:here. Prior to this, Patrick worked as a research scientist at the Fundamental AI Research Lab (FAIR) at Meta AI. During his PhD, Patrick split his time between FAIR and University College London, working with Sebastian Riedel and Pontus Stenetorp.  Patrick’s research focuses on the intersection of information retrieval techniques (IR) and large language models (LLMs). He has done extensive work on Retrieval-Augmented Language Models. His current focus is on building more powerful, efficient, robust, and update-able models that can perform well on a wide range of NLP tasks, but also excel on knowledge-intensive NLP tasks such as Question Answering and Fact Checking. YT version: https://youtu.be/Dm5sfALoL1Y MLST Discord: https://discord.gg/aNPkGUQtc5 Support us! https://www.patreon.com/mlst References: Patrick Lewis (Natural Language Processing Research Scientist @ co:here) https://www.patricklewis.io/ Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Patrick Lewis et al) https://arxiv.org/abs/2005.11401 Atlas: Few-shot Learning with Retrieval Augmented Language Models (Gautier Izacard, Patrick Lewis, et al) https://arxiv.org/abs/2208.03299 Improving language models by retrieving from trillions of tokens (RETRO) (Sebastian Borgeaud et al) https://arxiv.org/abs/2112.04426

YT version (with references): https://www.youtube.com/watch?v=lxaTinmKxs0 Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Carla Cremer and Igor Krawczuk argue that AI risk should be understood as an old problem of politics, power and control with known solutions, and that threat models should be driven by empirical work. The interaction between FTX and the Effective Altruism community has sparked a lot of discussion about the dangers of optimization, and Carla's Vox article highlights the need for an institutional turn when taking on a responsibility like risk management for humanity. Carla's “Democratizing Risk” paper found that certain types of risks fall through the cracks if they are just categorized into climate change or biological risks. Deliberative democracy has been found to be a better way to make decisions, and AI tools can be used to scale this type of democracy and be used for good, but the transparency of these algorithms to the citizens using the platform must be taken into consideration. Aggregating people’s diverse ways of thinking about a problem and creating a risk-averse procedure gives a likely, highly probable outcome for having converged on the best policy. There needs to be a good reason to trust one organization with the risk management of humanity and all the different ways of thinking about risk must be taken into account. AI tools can help to scale this type of deliberative democracy, but the transparency of these algorithms must be taken into consideration. The ambition of the EA community and Altruism Inc. is to protect and do risk management for the whole of humanity and this requires an institutional turn in order to do it effectively. The dangers of optimization are real, and it is essential to ensure that the risk management of humanity is done properly and ethically. By understanding the importance of aggregating people’s diverse ways of thinking about a problem, and creating a risk-averse procedure, it is possible to create a likely, highly probable outcome for having converged on the best policy. Carla Zoe Cremer https://carlacremer.github.io/ Igor Krawczuk https://krawczuk.eu/ Interviewer: Dr. Tim Scarfe TOC: [00:00:00] Introduction: Vox article and effective altruism / FTX [00:11:12] Luciano Floridi on Governance and Risk [00:15:50] Connor Leahy on alignment [00:21:08] Ethan Caballero on scaling [00:23:23] Alignment, Values and politics [00:30:50] Singularitarians vs AI-thiests [00:41:56] Consequentialism [00:46:44] Does scale make a difference? [00:51:53] Carla's Democratising risk paper [01:04:03] Vox article - How effective altruists ignored risk [01:20:18] Does diversity breed complexity? [01:29:50] Collective rationality [01:35:16] Closing statements

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 YT version: https://youtu.be/YLNGvvgq3eg We are living in an age of rapid technological advancement, and with this growth comes a digital divide. Professor Luciano Floridi of the Oxford Internet Institute / Oxford University believes that this divide not only affects our understanding of the implications of this new age, but also the organization of a fair society.  The Information Revolution has been transforming the global economy, with the majority of global GDP now relying on intangible goods, such as information-related services. This in turn has led to the generation of immense amounts of data, more than humanity has ever seen in its history. With 95% of this data being generated by the current generation, Professor Floridi believes that we are becoming overwhelmed by this data, and that our agency as humans is being eroded as a result.  According to Professor Floridi, the digital divide has caused a lack of balance between technological growth and our understanding of this growth. He believes that the infosphere is becoming polluted and the manifold of the infosphere is increasingly determined by technology and AI. Identifying, anticipating and resolving these problems has become essential, and Professor Floridi has dedicated his research to the Philosophy of Information, Philosophy of Technology and Digital Ethics.  We must equip ourselves with a viable philosophy of information to help us better understand and address the risks of this new information age. Professor Floridi is leading the charge, and his research on Digital Ethics, the Philosophy of Information and the Philosophy of Technology is helping us to better anticipate, identify and resolve problems caused by the digital divide. TOC: [00:00:00] Introduction to Luciano and his ideas [00:14:00] Chat GPT / language models [00:28:45] AI risk / "Singularitarians"  [00:37:15] Forms of governance [00:43:56] Re-ontologising the world [00:55:56] It from bit and Computationalism and philosophy without purpose [01:03:05] Getting into Digital Ethics Interviewer: Dr. Tim Scarfe References: GPT‐3: Its Nature, Scope, Limits, and Consequences [Floridi] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3827044 Ultraintelligent Machines, Singularity, and Other Sci-fi Distractions about AI [Floridi] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4222347 The Philosophy of Information [Floridi] https://www.amazon.co.uk/Philosophy-Information-Luciano-Floridi/dp/0199232393 Information: A Very Short Introduction [Floridi] https://www.amazon.co.uk/Information-Very-Short-Introduction-Introductions/dp/0199551375 https://en.wikipedia.org/wiki/Luciano_Floridi https://www.philosophyofinformation.net/

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 YT version: https://youtu.be/YLNGvvgq3eg (If music annoying, skip to main interview @ 14:14) We are living in an age of rapid technological advancement, and with this growth comes a digital divide. Professor Luciano Floridi of the Oxford Internet Institute / Oxford University believes that this divide not only affects our understanding of the implications of this new age, but also the organization of a fair society.  The Information Revolution has been transforming the global economy, with the majority of global GDP now relying on intangible goods, such as information-related services. This in turn has led to the generation of immense amounts of data, more than humanity has ever seen in its history. With 95% of this data being generated by the current generation, Professor Floridi believes that we are becoming overwhelmed by this data, and that our agency as humans is being eroded as a result.  According to Professor Floridi, the digital divide has caused a lack of balance between technological growth and our understanding of this growth. He believes that the infosphere is becoming polluted and the manifold of the infosphere is increasingly determined by technology and AI. Identifying, anticipating and resolving these problems has become essential, and Professor Floridi has dedicated his research to the Philosophy of Information, Philosophy of Technology and Digital Ethics.  We must equip ourselves with a viable philosophy of information to help us better understand and address the risks of this new information age. Professor Floridi is leading the charge, and his research on Digital Ethics, the Philosophy of Information and the Philosophy of Technology is helping us to better anticipate, identify and resolve problems caused by the digital divide. TOC: [00:00:00] Introduction to Luciano and his ideas [00:14:40] Chat GPT / language models [00:29:24] AI risk / "Singularitarians"  [00:30:34] Re-ontologising the world [00:56:35] It from bit and Computationalism and philosophy without purpose [01:03:43] Getting into Digital Ethics References: GPT‐3: Its Nature, Scope, Limits, and Consequences [Floridi] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3827044 Ultraintelligent Machines, Singularity, and Other Sci-fi Distractions about AI [Floridi] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4222347 The Philosophy of Information [Floridi] https://www.amazon.co.uk/Philosophy-Information-Luciano-Floridi/dp/0199232393 Information: A Very Short Introduction [Floridi] https://www.amazon.co.uk/Information-Very-Short-Introduction-Introductions/dp/0199551375 https://en.wikipedia.org/wiki/Luciano_Floridi https://www.philosophyofinformation.net/

Research has shown that humans possess strong inductive biases which enable them to quickly learn and generalize. In order to instill these same useful human inductive biases into machines, a paper was presented by Sreejan Kumar at the NeurIPS conference which won the Outstanding Paper of the Year award. The paper is called Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines. This paper focuses on using a controlled stimulus space of two-dimensional binary grids to define the space of abstract concepts that humans have and a feedback loop of collaboration between humans and machines to understand the differences in human and machine inductive biases.  It is important to make machines more human-like to collaborate with them and understand their behavior. Synthesised discrete programs running on a turing machine computational model instead of a neural network substrate offers promise for the future of artificial intelligence. Neural networks and program induction should both be explored to get a well-rounded view of intelligence which works in multiple domains, computational substrates and which can acquire a diverse set of capabilities. Natural language understanding in models can also be improved by instilling human language biases and programs into AI models. Sreejan used an experimental framework consisting of two dual task distributions, one generated from human priors and one from machine priors, to understand the differences in human and machine inductive biases. Furthermore, he demonstrated that compressive abstractions can be used to capture the essential structure of the environment for more human-like behavior. This means that emergent language-based inductive priors can be distilled into artificial neural networks, and AI  models can be aligned to the us, world and indeed, our values. Humans possess strong inductive biases which enable them to quickly learn to perform various tasks. This is in contrast to neural networks, which lack the same inductive biases and struggle to learn them empirically from observational data, thus, they have difficulty generalizing to novel environments due to their lack of prior knowledge.  Sreejan's results showed that when guided with representations from language and programs, the meta-learning agent not only improved performance on task distributions humans are adept at, but also decreased performa on control task distributions where humans perform poorly. This indicates that the abstraction supported by these representations, in the substrate of language or indeed, a program, is key in the development of aligned artificial agents with human-like generalization, capabilities, aligned values and behaviour. References Using natural language and program abstractions to instill human inductive biases in machines [Kumar et al/NEURIPS] https://openreview.net/pdf?id=buXZ7nIqiwE Core Knowledge [Elizabeth S. Spelke / Harvard] https://www.harvardlds.org/wp-content/uploads/2017/01/SpelkeKinzler07-1.pdf The Debate Over Understanding in AI's Large Language Models [Melanie Mitchell] https://arxiv.org/abs/2210.13966 On the Measure of Intelligence [Francois Chollet] https://arxiv.org/abs/1911.01547 ARC challenge [Chollet] https://github.com/fchollet/ARC

Pedro Domingos, Professor Emeritus of Computer Science and Engineering at the University of Washington, is renowned for his research in machine learning, particularly for his work on Markov logic networks that allow for uncertain inference. He is also the author of the acclaimed book "The Master Algorithm". Panel: Dr. Tim Scarfe TOC: [00:00:00] Introduction [00:01:34] Galaxtica / misinformation / gatekeeping [00:12:31] Is there a master algorithm? [00:16:29] Limits of our understanding  [00:21:57] Intentionality, Agency, Creativity [00:27:56] Compositionality  [00:29:30] Digital Physics / It from bit / Wolfram  [00:35:17] Alignment / Utility functions [00:43:36] Meritocracy   [00:45:53] Game theory  [01:00:00] EA/consequentialism/Utility [01:11:09] Emergence / relationalism  [01:19:26] Markov logic  [01:25:38] Moving away from anthropocentrism  [01:28:57] Neurosymbolic / infinity / tensor algerbra [01:53:45] Abstraction [01:57:26] Symmetries / Geometric DL [02:02:46] Bias variance trade off  [02:05:49] What seen at neurips [02:12:58] Chalmers talk on LLMs [02:28:32] Definition of intelligence [02:32:40] LLMs  [02:35:14] On experts in different fields [02:40:15] Back to intelligence [02:41:37] Spline theory / extrapolation YT version:  https://www.youtube.com/watch?v=C9BH3F2c0vQ References; The Master Algorithm [Domingos] https://www.amazon.co.uk/s?k=master+algorithm&i=stripbooks&crid=3CJ67DCY96DE8&sprefix=master+algorith%2Cstripbooks%2C82&ref=nb_sb_noss_2 INFORMATION, PHYSICS, QUANTUM: THE SEARCH FOR LINKS [John Wheeler/It from Bit] https://philpapers.org/archive/WHEIPQ.pdf A New Kind Of Science [Wolfram] https://www.amazon.co.uk/New-Kind-Science-Stephen-Wolfram/dp/1579550088 The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tom Chivers] https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795 The Status Game: On Social Position and How We Use It [Will Storr] https://www.goodreads.com/book/show/60598238-the-status-game Newcomb's paradox https://en.wikipedia.org/wiki/Newcomb%27s_paradox The Case for Strong Emergence [Sabine Hossenfelder] https://philpapers.org/rec/HOSTCF-3 Markov Logic: An Interface Layer for Artificial Intelligence [Domingos] https://www.morganclaypool.com/doi/abs/10.2200/S00206ED1V01Y200907AIM007 Note; Pedro discussed “Tensor Logic” - I was not able to find a reference Neural Networks and the Chomsky Hierarchy [Grégoire Delétang/DeepMind] https://arxiv.org/abs/2207.02098 Connectionism and Cognitive Architecture: A Critical Analysis [Jerry A. Fodor and Zenon W. Pylyshyn] https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf Every Model Learned by Gradient Descent Is Approximately a Kernel Machine [Pedro Domingos] https://arxiv.org/abs/2012.00152 A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27 [LeCun] https://openreview.net/pdf?id=BZ5a1r-kVsf Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković] https://arxiv.org/abs/2104.13478 The Algebraic Mind: Integrating Connectionism and Cognitive Science [Gary Marcus] https://www.amazon.co.uk/Algebraic-Mind-Integrating-Connectionism-D

Canadian Excellence Research Chair in Autonomous AI. Irina holds an MSc and PhD in AI from the University of California, Irvine as well as an MSc in Applied Mathematics from the Moscow Gubkin Institute. Her research focuses on machine learning, neural data analysis, and neuroscience-inspired AI. In particular, she is exploring continual lifelong learning, optimization algorithms for deep neural networks, sparse modelling and probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis. Prof. Rish holds 64 patents, has published over 80 research papers, several book chapters, three edited books, and a monograph on Sparse Modelling. She has served as a Senior Area Chair for NeurIPS and ICML.   Irina's research is focussed on taking us closer to the holy grail of Artificial General Intelligence.  She continues to push the boundaries of machine learning, continually striving to make advancements in neuroscience-inspired AI. In a conversation about artificial intelligence (AI), Irina and Tim discussed the idea of transhumanism and the potential for AI to improve human flourishing. Irina suggested that instead of looking at AI as something to be controlled and regulated, people should view it as a tool to augment human capabilities. She argued that attempting to create an AI that is smarter than humans is not the best approach, and that a hybrid of human and AI intelligence is much more beneficial. As an example, she mentioned how technology can be used as an extension of the human mind, to track mental states and improve self-understanding. Ultimately, Irina concluded that transhumanism is about having a symbiotic relationship with technology, which can have a positive effect on both parties. Tim then discussed the contrasting types of intelligence and how this could lead to something interesting emerging from the combination. He brought up the Trolley Problem and how difficult moral quandaries could be programmed into an AI. Irina then referenced The Garden of Forking Paths, a story which explores the idea of how different paths in life can be taken and how decisions from the past can have an effect on the present. To better understand AI and intelligence, Irina suggested looking at it from multiple perspectives and understanding the importance of complex systems science in programming and understanding dynamical systems. She discussed the work of Michael Levin, who is looking into reprogramming biological computers with chemical interventions, and Tim mentioned Alex Mordvinsev, who is looking into the self-healing and repair of these systems. Ultimately, Irina argued that the key to understanding AI and intelligence is to recognize the complexity of the systems and to create hybrid models of human and AI intelligence. Find Irina; https://mila.quebec/en/person/irina-rish/ https://twitter.com/irinarish YT version: https://youtu.be/8-ilcF0R7mI  MLST Discord: https://discord.gg/aNPkGUQtc5 References; The Garden of Forking Paths: Jorge Luis Borges [Jorge Luis Borges] https://www.amazon.co.uk/Garden-Forking-Paths-Penguin-Modern/dp/0241339057 The Brain from Inside Out [György Buzsáki] https://www.amazon.co.uk/Brain-Inside-Out-Gy%C3%B6rgy-Buzs%C3%A1ki/dp/0190905387 Growing Isotropic Neural Cellular Automata [Alexander Mordvintsev] https://arxiv.org/abs/2205.01681 The Extended Mind [Andy Clark and David Chalmers] https://www.jstor.org/stable/3328150 The Gentle Seduction [Marc Stiegler] https://www.amazon.co.uk/Gentle-Seduction-Marc-Stiegler/dp/0671698877

Support us! https://www.patreon.com/mlst Alan Chan is a PhD student at Mila, the Montreal Institute for Learning Algorithms, supervised by Nicolas Le Roux. Before joining Mila, Alan was a Masters student at the Alberta Machine Intelligence Institute and the University of Alberta, where he worked with Martha White. Alan's expertise and research interests encompass value alignment and AI governance. He is currently exploring the measurement of harms from language models and the incentives that agents have to impact the world. Alan's research focuses on understanding and controlling the values expressed by machine learning models. His projects have examined the regulation of explainability in algorithmic systems, scoring rules for performative binary prediction, the effects of global exclusion in AI development, and the role of a graduate student in approaching ethical impacts in AI research. In addition, Alan has conducted research into inverse policy evaluation for value-based sequential decision-making, and the concept of "normal accidents" and AI systems. Alan's research is motivated by the need to align AI systems with human values, and his passion for scientific and governance work in this field. Alan's energy and enthusiasm for his field is infectious.  This was a discussion at NeurIPS. It was in quite a loud environment so the audio quality could have been better.  References: The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tim Chivers] https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795 The implausibility of intelligence explosion [Chollet] https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec Superintelligence: Paths, Dangers, Strategies [Bostrom] https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111 A Theory of Universal Artificial Intelligence based on Algorithmic Complexity [Hutter] https://arxiv.org/abs/cs/0004001 YT version: https://youtu.be/XBMnOsv9_pk  MLST Discord: https://discord.gg/aNPkGUQtc5 

Support us! https://www.patreon.com/mlst Professor Murray Shanahan is a renowned researcher on sophisticated cognition and its implications for artificial intelligence. His 2016 article ‘Conscious Exotica’ explores the Space of Possible Minds, a concept first proposed by philosopher Aaron Sloman in 1984, which includes all the different forms of minds from those of other animals to those of artificial intelligence. Shanahan rejects the idea of an impenetrable realm of subjective experience and argues that the majority of the space of possible minds may be occupied by non-natural variants, such as the ‘conscious exotica’ of which he speaks.  In his paper ‘Talking About Large Language Models’, Shanahan discusses the capabilities and limitations of large language models (LLMs). He argues that prompt engineering is a key element for advanced AI systems, as it involves exploiting prompt prefixes to adjust LLMs to various tasks. However, Shanahan cautions against ascribing human-like characteristics to these systems, as they are fundamentally different and lack a shared comprehension with humans. Even though LLMs can be integrated into embodied systems, it does not mean that they possess human-like language abilities. Ultimately, Shanahan concludes that although LLMs are formidable and versatile, we must be wary of over-simplifying their capacities and limitations. YT version: https://youtu.be/BqkWpP3uMMU Full references on the YT description.  [00:00:00] Introduction [00:08:51] Consciousness and  Consciousness Exotica [00:34:59] Slightly Consciousness LLMs [00:38:05] Embodiment [00:51:32] Symbol Grounding  [00:54:13] Emergence [00:57:09] Reasoning [01:03:16] Intentional Stance [01:07:06] Digression on Chomsky show and Andrew Lampinen [01:10:31] Prompt Engineering Find Murray online: https://www.doc.ic.ac.uk/~mpsha/ https://twitter.com/mpshanahan?lang=en https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en MLST Discord: https://discord.gg/aNPkGUQtc5

Support us! https://www.patreon.com/mlst Sara Hooker is an exceptionally talented and accomplished leader and research scientist in the field of machine learning. She is the founder of Cohere For AI, a non-profit research lab that seeks to solve complex machine learning problems. She is passionate about creating more points of entry into machine learning research and has dedicated her efforts to understanding how progress in this field can be translated into reliable and accessible machine learning in the real-world. Sara is also the co-founder of the Trustworthy ML Initiative, a forum and seminar series related to Trustworthy ML. She is on the advisory board of Patterns and is an active member of the MLC research group, which has a focus on making participation in machine learning research more accessible. Before starting Cohere For AI, Sara worked as a research scientist at Google Brain. She has written several influential research papers, including "The Hardware Lottery", "The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation", "Moving Beyond “Algorithmic Bias is a Data Problem”" and "Characterizing and Mitigating Bias in Compact Models".  In addition to her research work, Sara is also the founder of the local Bay Area non-profit Delta Analytics, which works with non-profits and communities all over the world to build technical capacity and empower others to use data. She regularly gives tutorials on machine learning fundamentals, interpretability, model compression and deep neural networks and is dedicated to collaborating with independent researchers around the world. Sara Hooker is famous for writing a paper introducing the concept of the 'hardware lottery', in which the success of a research idea is determined not by its inherent superiority, but by its compatibility with available software and hardware. She argued that choices about software and hardware have had a substantial impact in deciding the outcomes of early computer science history, and that with the increasing heterogeneity of the hardware landscape, gains from advances in computing may become increasingly disparate. Sara proposed that an interim goal should be to create better feedback mechanisms for researchers to understand how their algorithms interact with the hardware they use. She suggested that domain-specific languages, auto-tuning of algorithmic parameters, and better profiling tools may help to alleviate this issue, as well as provide researchers with more informed opinions about how hardware and software should progress. Ultimately, Sara encouraged researchers to be mindful of the implications of the hardware lottery, as it could mean that progress on some research directions is further obstructed. If you want to learn more about that paper, watch our previous interview with Sara. YT version: https://youtu.be/7oJui4eSCoY MLST Discord: https://discord.gg/aNPkGUQtc5 TOC: [00:00:00] Intro [00:02:53] Interpretability / Fairness [00:35:29] LLMs Find Sara: https://www.sarahooker.me/ https://twitter.com/sarahookr

Support us! https://www.patreon.com/mlst Hattie Zhou, a PhD student at Université de Montréal and Mila, has set out to understand and explain the performance of modern neural networks, believing it a key factor in building better, more trusted models. Having previously worked as a data scientist at Uber, a private equity analyst at Radar Capital, and an economic consultant at Cornerstone Research, she has recently released a paper in collaboration with the Google Brain team, titled ‘Teaching Algorithmic Reasoning via In-context Learning’. In this work, Hattie identifies and examines four key stages for successfully teaching algorithmic reasoning to large language models (LLMs): formulating algorithms as skills, teaching multiple skills simultaneously, teaching how to combine skills, and teaching how to use skills as tools. Through the application of algorithmic prompting, Hattie has achieved remarkable results, with an order of magnitude error reduction on some tasks compared to the best available baselines. This breakthrough demonstrates algorithmic prompting’s viability as an approach for teaching algorithmic reasoning to LLMs, and may have implications for other tasks requiring similar reasoning capabilities. TOC [00:00:00] Hattie Zhou [00:19:49] Markus Rabe [Google Brain] Hattie's Twitter - https://twitter.com/oh_that_hat Website - http://hattiezhou.com/ Teaching Algorithmic Reasoning via In-context Learning [Hattie Zhou, Azade Nova, Hugo Larochelle, Aaron Courville, Behnam Neyshabur, and Hanie Sedghi] https://arxiv.org/pdf/2211.09066.pdf Markus Rabe [Google Brain]: https://twitter.com/markusnrabe https://research.google/people/106335/ https://www.linkedin.com/in/markusnrabe Autoformalization with Large Language Models [Albert Jiang Charles Edgar Staats Christian Szegedy Markus Rabe Mateja Jamnik Wenda Li Yuhuai Tony Wu] https://research.google/pubs/pub51691/ Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/80i6D2TJdQ4

Support us! https://www.patreon.com/mlst (On the main version we released; the music was a tiny bit too loud in places, and some pieces had percussion which was a bit distracting -- here is a version with all music removed so you have the option! ) David Chalmers is a professor of philosophy and neural science at New York University, and an honorary professor of philosophy at the Australian National University. He is the co-director of the Center for Mind, Brain, and Consciousness, as well as the PhilPapers Foundation. His research focuses on the philosophy of mind, especially consciousness, and its connection to fields such as cognitive science, physics, and technology. He also investigates areas such as the philosophy of language, metaphysics, and epistemology. With his impressive breadth of knowledge and experience, David Chalmers is a leader in the philosophical community. The central challenge for consciousness studies is to explain how something immaterial, subjective, and personal can arise out of something material, objective, and impersonal. This is illustrated by the example of a bat, whose sensory experience is much different from ours, making it difficult to imagine what it's like to be one. Thomas Nagel's "inconceivability argument" has its advantages and disadvantages, but ultimately it is impossible to solve the mind-body problem due to the subjective nature of experience. This is further explored by examining the concept of philosophical zombies, which are physically and behaviorally indistinguishable from conscious humans yet lack conscious experience. This has implications for the Hard Problem of Consciousness, which is the attempt to explain how mental states are linked to neurophysiological activity. The Chinese Room Argument is used as a thought experiment to explain why physicality may be insufficient to be the source of the subjective, coherent experience we call consciousness. Despite much debate, the Hard Problem of Consciousness remains unsolved. Chalmers has been working on a functional approach to decide whether large language models are, or could be conscious.  Filmed at #neurips22 Discord: https://discord.gg/aNPkGUQtc5 Pod: https://anchor.fm/machinelearningstreettalk/episodes/90---Prof--DAVID-CHALMERS---Slightly-Conscious-LLMs-e1sej50 TOC; [00:00:00] Introduction [00:00:40] LLMs consciousness pitch [00:06:33] Philosophical Zombies [00:09:26] The hard problem of consciousness [00:11:40] Nagal's bat and intelligibility  [00:21:04] LLM intro clip from NeurIPS [00:22:55] Connor Leahy on self-awareness in LLMs [00:23:30] Sneak peek from unreleased show - could consciousness be a submodule? [00:33:44] SeppH [00:36:15] Tim interviews David at NeurIPS (functionalism / panpsychism / Searle) [00:45:20] Peter Hase interviews Chalmers (focus on interpretability/safety) Panel: Dr. Tim Scarfe Dr. Keith Duggar Contact David; https://mobile.twitter.com/davidchalmers42 https://consc.net/ References; Could a Large Language Model Be Conscious? [Chalmers NeurIPS22 talk] https://nips.cc/media/neurips-2022/Slides/55867.pdf What Is It Like to Be a Bat? [Nagel] https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/humananimalstudies/lectures/32/nagel_bat.pdf Zombies https://plato.stanford.edu/entries/zombies/ zombies on the web [Chalmers] https://consc.net/zombies-on-the-web/ The hard problem of consciousness [Chalmers] https://psycnet.apa.org/record/2007-00485-017 David Chalmers, "Are Large Language Models Sentient?" [NYU talk, same as at NeurIPS] https://www.youtube.com/watch?v=-BcuCmf00_Y

Support us! https://www.patreon.com/mlst David Chalmers is a professor of philosophy and neural science at New York University, and an honorary professor of philosophy at the Australian National University. He is the co-director of the Center for Mind, Brain, and Consciousness, as well as the PhilPapers Foundation. His research focuses on the philosophy of mind, especially consciousness, and its connection to fields such as cognitive science, physics, and technology. He also investigates areas such as the philosophy of language, metaphysics, and epistemology. With his impressive breadth of knowledge and experience, David Chalmers is a leader in the philosophical community. The central challenge for consciousness studies is to explain how something immaterial, subjective, and personal can arise out of something material, objective, and impersonal. This is illustrated by the example of a bat, whose sensory experience is much different from ours, making it difficult to imagine what it's like to be one. Thomas Nagel's "inconceivability argument" has its advantages and disadvantages, but ultimately it is impossible to solve the mind-body problem due to the subjective nature of experience. This is further explored by examining the concept of philosophical zombies, which are physically and behaviorally indistinguishable from conscious humans yet lack conscious experience. This has implications for the Hard Problem of Consciousness, which is the attempt to explain how mental states are linked to neurophysiological activity. The Chinese Room Argument is used as a thought experiment to explain why physicality may be insufficient to be the source of the subjective, coherent experience we call consciousness. Despite much debate, the Hard Problem of Consciousness remains unsolved. Chalmers has been working on a functional approach to decide whether large language models are, or could be conscious.  Filmed at #neurips22 Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/T7aIxncLuWk TOC; [00:00:00] Introduction [00:00:40] LLMs consciousness pitch [00:06:33] Philosophical Zombies [00:09:26] The hard problem of consciousness [00:11:40] Nagal's bat and intelligibility [00:21:04] LLM intro clip from NeurIPS [00:22:55] Connor Leahy on self-awareness in LLMs [00:23:30] Sneak peek from unreleased show - could consciousness be a submodule? [00:33:44] SeppH [00:36:15] Tim interviews David at NeurIPS (functionalism / panpsychism / Searle) [00:45:20] Peter Hase interviews Chalmers (focus on interpretability/safety) Panel: Dr. Tim Scarfe Dr. Keith Duggar Contact David; https://mobile.twitter.com/davidchalmers42 https://consc.net/ References; Could a Large Language Model Be Conscious? [Chalmers NeurIPS22 talk]  https://nips.cc/media/neurips-2022/Slides/55867.pdf What Is It Like to Be a Bat? [Nagel] https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/humananimalstudies/lectures/32/nagel_bat.pdf Zombies https://plato.stanford.edu/entries/zombies/ zombies on the web [Chalmers] https://consc.net/zombies-on-the-web/ The hard problem of consciousness [Chalmers] https://psycnet.apa.org/record/2007-00485-017 David Chalmers, "Are Large Language Models Sentient?" [NYU talk, same as at NeurIPS] https://www.youtube.com/watch?v=-BcuCmf00_Y

Support us! https://www.patreon.com/mlst Dr. Walid Saba recently reviewed the book Machines Will Never Rule The World, which argues that strong AI is impossible. He acknowledges the complexity of modeling mental processes and language, as well as interactive dialogues, and questions the authors' use of "never." Despite his skepticism, he is impressed with recent developments in large language models, though he questions the extent of their success. We then discussed the successes of cognitive science. Walid believes that something has been achieved which many cognitive scientists would never accept, namely the ability to learn from data empirically. Keith agrees that this is a huge step, but notes that there is still much work to be done to get to the "other 5%" of accuracy. They both agree that the current models are too brittle and require much more data and parameters to get to the desired level of accuracy. Walid then expresses admiration for deep learning systems' ability to learn non-trivial aspects of language from ingesting text only. He argues that this is an "existential proof" of language competency and that it would be impossible for a group of luminaries such as Montague, Marvin Minsky, John McCarthy, and a thousand other bright engineers to replicate the same level of competency as we have now with LLMs. He then discusses the problem of semantics and pragmatics, as well as symbol grounding, and expresses skepticism about grounded meaning and embodiment. He believes that artificial intelligence should be used to solve real-world problems which require human intelligence but not believe that robots should be built to understand love or other subjective feelings. We discussed the unique properties of natural human language. Walid believes that the core unique property is the ability to do abductive reasoning, which is the process of reasoning to the best explanation or understanding. Keith adds that there are two types of abduction - one for generating hypotheses and one for justifying them. In both cases, abductive reasoning involves choosing from a set of plausible possibilities. Finally, we discussed the book "Machines Will Never Rule The World" and its argument that the current mathematics and technology is not enough to model complex systems. Walid agrees with the book's argument but is still optimistic that a new mathematics can be discovered. Keith suggests the possibility of an AGI discovering the mathematics to create itself. They also discussed how the book could serve as a reminder to temper the hype surrounding AI and to focus on exploration, creativity, and daring ideas. Walid ended by stressing the importance of science, noting that engineers should play within the Venn diagrams drawn by scientists, rather than trying to hack their way through it. Transcript: https://share.descript.com/view/BFQb5iaegJC Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/IMnWAuoucjo TOC: [00:00:00] Intro [00:06:52] Walid's change of heart on DL/LLMs and on the skeptics like Gary Marcus [00:22:52] Symbol Grounding [00:32:26] On Montague [00:40:41] On Abduction [00:50:54] Language of thought [00:56:08] Why machines will never rule the world book review [01:20:06] Engineers should play in the scientists Venn Diagram! Panel; Dr. Tim Scarfe Dr. Keith Duggar Mark Mcguill

Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning". Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications. Episode recorded live at NeurIPS.  YT: https://youtu.be/9dLd6n9yT8U (references are there) Support us! https://www.patreon.com/mlst  Host: Dr. Tim Scarfe TOC: [00:00:00] LeCun interview [00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)

Dr. Petar Veličković  is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks. https://petar-v.com/ https://twitter.com/PetarV_93/ TOC: Categories  (Cats for AI) [00:00:00] Reasoning [00:14:44] Extrapolation [00:19:09] Ishan Misra Skit [00:27:50] Graphs (Expander Graph Propagation) [00:29:18] YT: https://youtu.be/1lkdWduuN14 MLST Discord: https://discord.gg/V25vQeFwhS Support us! https://www.patreon.com/mlst References on YT description, lots of them!  Host: Dr. Tim Scarfe

In this NeurIPSs interview, we speak with Laura Ruis about her research on the ability of language models to interpret language in context. She has designed a simple task to evaluate the performance of widely used state-of-the-art language models and has found that they struggle to make pragmatic inferences (implicatures). Tune in to learn more about her findings and what they mean for the future of conversational AI. Laura Ruis https://www.lauraruis.com/ https://twitter.com/LauraRuis BLOOM https://bigscience.huggingface.co/blog/bloom Large language models are not zero-shot communicators [Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, Edward Grefenstette] https://arxiv.org/abs/2210.14986 [Zhang et al] OPT: Open Pre-trained Transformer Language Models https://arxiv.org/pdf/2205.01068.pdf [Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/pdf/2210.15303.pdf [Gary Marcus] Horse rides astronaut https://garymarcus.substack.com/p/horse-rides-astronaut [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922  [janus] Simulators (Less Wrong) https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

First in our unplugged series live from #NeurIPS2022 We discuss natural language understanding, symbol meaning and grounding and Chomsky with Dr. Andrew Lampinen from DeepMind.  We recorded a LOT of material from NeurIPS, keep an eye out for the uploads.  YT version: https://youtu.be/46A-BcBbMnA References [Paul Cisek] Beyond the computer metaphor: Behaviour as interaction https://philpapers.org/rec/CISBTC Linguistic Competence (Chomsky reference) https://en.wikipedia.org/wiki/Linguistic_competence [Andrew Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/abs/2210.15303 [Fodor et al] Connectionism and Cognitive Architecture: A Critical Analysis https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models https://arxiv.org/abs/2210.13966 [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922 [Adam Santoro, Andrew Lampinen et al] Symbolic Behaviour in Artificial Intelligence https://arxiv.org/abs/2102.03406 [Ishita Dasgupta, Lampinen et al] Language models show human-like content effects on reasoning https://arxiv.org/abs/2207.07051 REACT - Synergizing Reasoning and Acting in Language Models https://arxiv.org/pdf/2210.03629.pdf https://ai.googleblog.com/2022/11/react-synergizing-reasoning-and-acting.html [Fabian Paischer] HELM - History Compression via Language Models in Reinforcement Learning https://ml-jku.github.io/blog/2022/helm/ https://arxiv.org/abs/2205.12258 [Laura Ruis] Large language models are not zero-shot communicators https://arxiv.org/pdf/2210.14986.pdf [Kumar] Using natural language and program abstractions to instill human inductive biases in machines https://arxiv.org/pdf/2205.11558.pdf Juho Kim https://juhokim.com/

AI Helps Ukraine - Charity Conference A charity conference on AI to raise funds for medical and humanitarian aid for Ukraine https://aihelpsukraine.cc/ YT version: https://youtu.be/LgwjcqhkOA4 Support us! https://www.patreon.com/mlst  Dr. Joscha Bach (born 1973 in Weimar, Germany) is a German artificial intelligence researcher and cognitive scientist focusing on cognitive architectures, mental representation, emotion, social modelling, and multi-agent systems.  http://bach.ai/ https://twitter.com/plinz TOC: [00:00:00] Ukraine Charity Conference and NeurIPS 2022 [00:03:40] Theory of computation, Godel, Penrose [00:11:44] Modelling physical reality [00:15:19] Is our universe infinite? [00:24:30] Large language models, and on DL / is Gary Marcus hitting a wall? [00:45:17] Generative models / Codex / Language of thought [00:58:46] Consciousness (with Friston references) References: Am I Self-Conscious? (Or Does Self-Organization Entail Self-Consciousness?) [Friston] https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi] https://arxiv.org/abs/2202.07206 Deep Learning Is Hitting a Wall [Gary Marcus] https://nautil.us/deep-learning-is-hitting-a-wall-238440/ Turing machines https://en.wikipedia.org/wiki/Turing_machine Lambda Calculus https://en.wikipedia.org/wiki/Lambda_calculus Godel's incompletness theorem https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems Oracle machine https://en.wikipedia.org/wiki/Oracle_machine

Support us (and please rate on podcast app) https://www.patreon.com/mlst  In this show tonight with Prof. Julian Togelius (NYU) and Prof. Ken Stanley we discuss open-endedness, AGI, game AI and reinforcement learning.   [Prof Julian Togelius] https://engineering.nyu.edu/faculty/julian-togelius https://twitter.com/togelius [Prof Ken Stanley] https://www.cs.ucf.edu/~kstanley/ https://twitter.com/kenneth0stanley TOC: [00:00:00] Introduction [00:01:07] AI and computer games [00:12:23] Intelligence [00:21:27] Intelligence Explosion [00:25:37] What should we be aspiring towards? [00:29:14] Should AI contribute to culture? [00:32:12] On creativity and open-endedness [00:36:11] RL overfitting [00:44:02] Diversity preservation [00:51:18] Empiricism vs rationalism , in gradient descent the data pushes you around [00:55:49] Creativity and interestingness (does complexity / information increase) [01:03:20] What does a population give us? [01:05:58] Emergence / generalisation snobbery References; [Hutter/Legg] Universal Intelligence: A Definition of Machine Intelligence https://arxiv.org/abs/0712.3329 https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/I._J._Good https://en.wikipedia.org/wiki/G%C3%B6del_machine [Chollet] Impossibility of intelligence explosion https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec [Alex Irpan] - RL is hard https://www.alexirpan.com/2018/02/14/rl-hard.html https://nethackchallenge.com/ Map elites https://arxiv.org/abs/1504.04909 Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space https://arxiv.org/abs/1912.02400 [Stanley] - Why greatness cannot be planned https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 [Lehman/Stanley] Abandoning Objectives: Evolution through the Search for Novelty Alone https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf

We had a conversation with Aidan Gomez, the CEO of language-based AI platform Cohere. Cohere is a startup which uses artificial intelligence to help users build the next generation of language-based applications. It's headquartered in Toronto. The company has raised $175 million in funding so far. Language may well become a key new substrate for software building, both in its representation and how we build the software. It may democratise software building so that more people can build software, and we can build new types of software. Aidan and I discuss this in detail in this episode of MLST. Check out Cohere -- https://dashboard.cohere.ai/welcome/register?utm_source=influencer&utm_medium=social&utm_campaign=mlst Support us! https://www.patreon.com/mlst  YT version: https://youtu.be/ooBt_di8DLs TOC: [00:00:00] Aidan Gomez intro [00:02:12] What's it like being a CEO? [00:02:52] Transformers [00:09:33] Deepmind Chomsky Hierarchy [00:14:58] Cohere roadmap [00:18:18] Friction using LLMs for startups [00:25:31] How different from OpenAI / GPT-3 [00:29:31] Engineering questions on Cohere [00:35:13] Francois Chollet says that LLMs are like databases [00:38:34] Next frontier of language models [00:42:04] Different modes of understanding in LLMs [00:47:04] LLMs are the new extended mind [00:50:03] Is language the next interface, and why might that be bad? References: [Balestriero] Spine theory of NNs https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf [Delétang et al] Neural Networks and the Chomsky Hierarchy https://arxiv.org/abs/2207.02098 [Fodor, Pylyshyn] Connectionism and Cognitive Architecture: A Critical Analysis https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf [Chalmers, Clark] The extended mind https://icds.uoregon.edu/wp-content/uploads/2014/06/Clark-and-Chalmers-The-Extended-Mind.pdf [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models https://arxiv.org/abs/2210.13966 [Jay Alammar] Illustrated stable diffusion https://jalammar.github.io/illustrated-stable-diffusion/ Illustrated transformer https://jalammar.github.io/illustrated-transformer/ https://www.youtube.com/channel/UCmOwsoHty5PrmE-3QhUBfPQ [Sandra Kublik] (works at Cohere!) https://www.youtube.com/channel/UCjG6QzmabZrBEeGh3vi-wDQ

This video is demonetised on music copyright so we would appreciate support on our Patreon! https://www.patreon.com/mlst  We would also appreciate it if you rated us on your podcast platform.  YT: https://youtu.be/_KVAzAzO5HU Panel: Dr. Tim Scarfe, Dr. Keith Duggar Guests: Prof. J. Mark Bishop, Francois Chollet, Prof. David Chalmers, Dr. Joscha Bach, Prof. Karl Friston, Alexander Mattick, Sam Roffey The Chinese Room Argument was first proposed by philosopher John Searle in 1980. It is an argument against the possibility of artificial intelligence (AI) – that is, the idea that a machine could ever be truly intelligent, as opposed to just imitating intelligence. The argument goes like this: Imagine a room in which a person sits at a desk, with a book of rules in front of them. This person does not understand Chinese. Someone outside the room passes a piece of paper through a slot in the door. On this paper is a Chinese character. The person in the room consults the book of rules and, following these rules, writes down another Chinese character and passes it back out through the slot. To someone outside the room, it appears that the person in the room is engaging in a conversation in Chinese. In reality, they have no idea what they are doing – they are just following the rules in the book. The Chinese Room Argument is an argument against the idea that a machine could ever be truly intelligent. It is based on the idea that intelligence requires understanding, and that following rules is not the same as understanding. in this detailed investigation into the Chinese Room, Consciousness and Syntax vs Semantics, we interview luminaries J.Mark Bishop and Francois Chollet and use unreleased footage from our interviews with David Chalmers, Joscha Bach and Karl Friston. We also cover material from Walid Saba and interview Alex Mattick from Yannic's Discord.  This is probably my favourite ever episode of MLST. I hope you enjoy it!  With Keith Duggar.  Note that we are using clips from our unreleased interviews from David Chalmers and Joscha Bach -- we will release those shows properly in the coming weeks. We apologise for delay releasing our backlog, we have been busy building a startup company in the background. TOC:  [00:00:00] Kick off [00:00:46] Searle [00:05:09] Bishop introduces CRA [00:00:00] Stevan Hardad take on CRA  [00:14:03] Francois Chollet dissects CRA [00:34:16] Chalmers on consciousness [00:36:27] Joscha Bach on consciousness [00:42:01] Bishop introduction [00:51:51] Karl Friston on consciousness [00:55:19] Bishop on consciousness and comments on Chalmers  [01:21:37] Private language games (including clip with Sam Roffey) [01:27:27] Dr. Walid Saba on the chinese room (gofai/systematicity take) [00:34:36] Bishop: on agency / teleology [01:36:38] Bishop: back to CRA [01:40:53] Noam Chomsky on mysteries  [01:45:56] Eric Curiel on math does not represent [01:48:14] Alexander Mattick on syntax vs semantics Thanks to: Mark MC on Discord for stimulating conversation, Alexander Mattick, Dr. Keith Duggar, Sam Roffey. Sam's YouTube channel is https://www.youtube.com/channel/UCjRNMsglFYFwNsnOWIOgt1Q

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB In this special edition episode, we have a conversation with Prof. Noam Chomsky, the father of modern linguistics and the most important intellectual of the 20th century.  With a career spanning the better part of a century, we took the chance to ask Prof. Chomsky his thoughts not only on the progress of linguistics and cognitive science but also the deepest enduring mysteries of science and philosophy as a whole - exploring what may lie beyond our limits of understanding. We also discuss the rise of connectionism and large language models, our quest to discover an intelligible world, and the boundaries between silicon and biology. We explore some of the profound misunderstandings of linguistics in general and Chomsky’s own work specifically which have persisted, at the highest levels of academia for over sixty years.   We have produced a significant introduction section where we discuss in detail Yann LeCun’s recent position paper on AGI, a recent paper on emergence in LLMs, empiricism related to cognitive science, cognitive templates, “the ghost in the machine” and language.  Panel:  Dr. Tim Scarfe Dr. Keith Duggar Dr. Walid Saba  YT version: https://youtu.be/-9I4SgkHpcA 00:00:00 Kick off 00:02:24 C1: LeCun's recent position paper on AI, JEPA, Schmidhuber, EBMs 00:48:38 C2: Emergent abilities in LLMs paper 00:51:32 C3: Empiricism 01:25:33 C4: Cognitive Templates 01:35:47 C5: The Ghost in the Machine 01:59:21 C6: Connectionism and Cognitive Architecture: A Critical Analysis by Fodor and Pylyshyn 02:19:25 C7: We deep-faked Chomsky 02:29:11 C8: Language 02:34:41 C9: Chomsky interview kick-off! 02:35:39 Large Language Models such as GPT-3 02:39:14 Connectionism and radical empiricism 02:44:44 Hybrid systems such as neurosymbolic 02:48:47 Computationalism silicon vs biological 02:53:28 Limits of human understanding 03:00:46 Semantics state-of-the-art 03:06:43 Universal grammar, I-Language, and language of thought 03:16:27 Profound and enduring misunderstandings 03:25:41 Greatest remaining mysteries science and philosophy 03:33:10 Debrief and 'Chuckles' from Chomsky

Vitaliy Chiley  is a Machine Learning Research Engineer at the next-generation computing hardware company Cerebras Systems. We spoke about how DL workloads including sparse workloads can run faster on Cerebras hardware. [00:00:00] Housekeeping [00:01:08] Preamble [00:01:50] Vitaliy Chiley Introduction [00:03:11] Cerebrus architecture [00:08:12] Memory management and FLOP utilisation [00:18:01] Centralised vs decentralised compute architecture [00:21:12] Sparsity [00:23:47] Does Sparse NN imply Heterogeneous compute? [00:29:21] Cost of distributed memory stores? [00:31:01] Activation vs weight sparsity [00:37:52] What constitutes a dead weight to be pruned? [00:39:02] Is it still a saving if we have to choose between weight and activation sparsity? [00:41:02] Cerebras is a cool place to work [00:44:05] What is sparsity? Why do we need to start dense?  [00:46:36] Evolutionary algorithms on Cerebras? [00:47:57] How can we start sparse? Google RIGL [00:51:44] Inductive priors, why do we need them if we can start sparse? [00:56:02] Why anthropomorphise inductive priors? [01:02:13] Could Cerebras run a cyclic computational graph? [01:03:16] Are NNs locality sensitive hashing tables? References; Rigging the Lottery: Making All Tickets Winners [RIGL] https://arxiv.org/pdf/1911.11134.pdf [D] DanNet, the CUDA CNN of Dan Ciresan in Jurgen Schmidhuber's team, won 4 image recognition challenges prior to AlexNet https://www.reddit.com/r/MachineLearning/comments/dwnuwh/d_dannet_the_cuda_cnn_of_dan_ciresan_in_jurgen/  A Spline Theory of Deep Learning [Balestriero] https://proceedings.mlr.press/v80/balestriero18b.html 

Check out Weights and Biases here!  https://wandb.me/MLST Lukas Biewald is an entrepreneur living in San Francisco. He was the founder and CEO of Figure Eight an Internet company that collects training data for machine learning.  In 2018, he founded Weights and Biases, a company that creates developer tools for machine learning. Recently WandB got a cash injection of 15 million dollars in its second funding round.  Lukas has a bachelors and masters in mathematics and computer science respectively from Stanford university.  He was a research student under the tutelage of the legendary Daphne Koller.  Lukas Biewald https://twitter.com/l2k [00:00:00] Preamble [00:01:27] Intro to Lukas [00:02:46] How did Lukas build 2 sucessful startups? [00:05:49] Rebalancing games with ML [00:08:14] Elevator pitch for WandB [00:10:38] Science vs Engineering divide in ML DevOps [00:14:11] Too much focus on the minutiae? [00:18:03] Vertical information sharing in large enterprises (metrics) [00:20:37] Centralised vs Decentralised topology [00:24:02] Generalisation vs specialisation [00:28:59] Enhancing explainability [00:33:14] Should we try and understand "the machine" or is testing / behaviourism enough? [00:36:55] WandB roadmap [00:39:06] WandB / ML Ops competitor space? [00:44:10] How is WandB differentiated over Sagemaker / AzureML [00:46:02] WandB Sponsorship of ML YT channels [00:48:43] Alternatives to deep learning? [00:53:47] How to build a business like WandB Panel: Tim Scarfe Ph.D and Keith Duggar Ph.D Note we didn't get paid by Weights and Biases to conduct this interview.

An emergent behavior or emergent property can appear when a number of simple entities operate in an environment, forming more complex behaviours as a collective. If emergence happens over disparate size scales, then the reason is usually a causal relation across different scales. Weak emergence describes new properties arising in systems as a result of the low-level interactions, these might be interactions between components of the system or the components and their environment.  In our epic introduction we focus a lot on the concept of self-organisation, complex systems, cellular automata and strong vs weak emergence. In the main show we discuss this more in detail with Dr. Daniele Grattarola and cover his recent NeurIPS paper on learning graph cellular automata.  YT version: https://youtu.be/MDt2e8XtUcA Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Featuring; Dr. Daniele Grattarola Dr. Tim Scarfe Dr. Keith Duggar Prof. David Chalmers Prof. Ken Stanley Prof. Julian Togelius Dr. Joscha Bach David Ha Dr. Pei Wang [00:00:00] Special Edition Intro: Emergence and Cellular Automata [00:49:02] Intro to Daniele and CAs [00:57:23] Numerical analysis link with CA (PDEs) [00:59:50] The representational dichotomy of discrete and continuous at different scales [01:05:21] Universal computation in CAs [01:10:27] Computational irreducibility  [01:16:33] Is the universe discrete? [01:20:49] Emergence but with the same computational principle [01:23:10] How do you formalise the emergent phenomenon  [01:25:44] Growing cellular automata [01:33:53] Openeded and unbounded computation is required for this kind of behaviour [01:37:31] Graph cellula automata [01:43:40] Connection to protein folding [01:46:24] Are CAs the best tool for the job? [01:49:37] Where to go to find more information

Please note that in this interview Dr. Lampinen was expressing his personal opinions and they do not necessarily represent those of DeepMind.  Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB YT version: https://youtu.be/yPMtSXXn4OY  Dr. Andrew Lampinen is a Senior Research Scientist at DeepMind, and he thinks that symbols are subjective in the relativistic sense. Dr. Lampinen completed his PhD in Cognitive Psychology at Stanford University. His background is in mathematics, physics, and machine learning. Andrew has said that his research interests are in cognitive flexibililty and generalization, and how these abilities are enabled by factors like language, memory, and embodiment.  Andrew with his coauthors has just released a paper called symbolic behaviour in artificial intelligence. Andrew lead in the paper by saying the human ability to use symbols has yet to be replicated in machines. He thinks that one of the key areas to bridge the gap here is considering how symbol meaning is established, and he strongly believes it is the symbol users themselves who agree upon the symbol meaning, And that the use of symbols entails behaviours which coalesce agreements about their meaning. Which in plain English means that symbols are defined by behaviours rather than their content. [00:00:00] Intro to Andrew and Symbolic Behaviour paper [00:07:01] Semantics underpins the unreasonable effectiveness of symbols [00:12:56] The Depth of Subjectivity [00:21:03] Walid Saba - universal cognitive templates [00:27:47] Insufficiently Darwinian  [00:30:52] Discovered vs invented [00:34:19] Does language have primacy [00:35:59] Research directions [00:39:43] Comparison to BenG OpenCog and human compatible AI [00:42:53] Aligning AI with our culture [00:47:55] Do we need to model the worst aspects of human behaviour?  [00:50:57] Fairness [00:54:24] Memorisatation on LLMs [01:00:38] Wason selection task [01:03:45] Would an Andrew hashtable robot be intelligent? Dr. Andrew Lampinen https://lampinen.github.io/ https://twitter.com/AndrewLampinen Symbolic Behaviour in Artificial Intelligence https://arxiv.org/abs/2102.03406 Imitating Interactive Intelligence https://arxiv.org/abs/2012.05672 https://www.deepmind.com/publications/imitating-interactive-intelligence Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi] https://arxiv.org/abs/2202.07206 Big bench dataset https://github.com/google/BIG-bench Teaching Autoregressive Language Models Complex Tasks By Demonstration [Recchia] https://arxiv.org/pdf/2109.02102.pdf Wason selection task https://en.wikipedia.org/wiki/Wason_selection_task Gary Lupyan https://psych.wisc.edu/staff/lupyan-gary/

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB YT version: https://youtu.be/RzGaI7vXrkk This week we speak with Yasaman Razeghi and Prof. Sameer Singh from UC Urvine. Yasaman recently published a paper called Impact of Pretraining Term Frequencies on Few-Shot Reasoning where she demonstrated comprehensively that large language models only perform well on reasoning tasks because they memorise the dataset. For the first time she showed the accuracy was linearly correlated to the occurance rate in the training corpus, something which OpenAI should have done in the first place!  We also speak with Sameer who has been a pioneering force in the area of machine learning interpretability for many years now, he created LIME with Marco Riberio and also had his hands all over the famous Checklist paper and many others.  We also get into the metric obsession in the NLP world and whether metrics are one of the principle reasons why we are failing to make any progress in NLU.  [00:00:00] Impact of Pretraining Term Frequencies on Few-Shot Reasoning [00:14:59] Metrics [00:18:55] Definition of reasoning [00:25:12] Metrics (again) [00:28:52] On true believers  [00:33:04] Sameers work on model explainability / LIME  [00:36:58] Computational irreducability  [00:41:07] ML DevOps and Checklist [00:45:58] Future of ML devops [00:49:34] Thinking about future Prof. Sameer Singh https://sameersingh.org/ Yasaman Razeghi https://yasamanrazeghi.com/ References; Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Razeghi et al with Singh] https://arxiv.org/pdf/2202.07206.pdf Beyond Accuracy: Behavioral Testing of NLP Models with CheckList [Riberio et al with Singh] https://arxiv.org/pdf/2005.04118.pdf “Why Should I Trust You?” Explaining the Predictions of Any Classifier (LIME) [Riberio et al with Singh] https://arxiv.org/abs/1602.04938 Tim interviewing LIME Creator Marco Ribeiro in 2019 https://www.youtube.com/watch?v=6aUU-Ob4a8I Tim video on LIME/SHAP on his other channel https://www.youtube.com/watch?v=jhopjN08lTM Our interview with Christoph Molar https://www.youtube.com/watch?v=0LIACHcxpHU Interpretable Machine Learning book @ChristophMolnar https://christophm.github.io/interpretable-ml-book/ Machine Teaching: A New Paradigm for Building Machine Learning Systems [Simard] https://arxiv.org/abs/1707.06742 Whimsical notes on machine teaching https://whimsical.com/machine-teaching-Ntke9EHHSR25yHnsypHnth Gopher paper (Deepmind) https://www.deepmind.com/blog/language-modelling-at-scale-gopher-ethical-considerations-and-retrieval https://arxiv.org/pdf/2112.11446.pdf EleutherAI https://www.eleuther.ai/ https://github.com/kingoflolz/mesh-transformer-jax/ https://pile.eleuther.ai/ A Theory of Universal Artificial Intelligence based on Algorithmic Complexity [Hutter] https://arxiv.org/pdf/cs/0004001.pdf

YT version: https://youtu.be/DxBZORM9F-8 Patreon: https://www.patreon.com/mlst  Discord: https://discord.gg/ESrGqhf5CB Prof. Ken Stanley argued in his book that our world has become saturated with objectives. The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture. He’s not saying that objectives are bad per se, especially if they’re modest, but he thinks that when goals are ambitious then the search space becomes deceptive. Is the key to artificial intelligence really related to intelligence? Does taking a job with a higher salary really bring you closer to being a millionaire? The problem is that the stepping stones which lead to ambitious objectives tend to be pretty strange, they don't resemble the final end state at all. Vaccum tubes led to computers for example and Youtube started as a dating website.  What fascinated us about this conversation with Ken is that we got a much deeper understanding of his philosophy. He lead by saying that he thought it's worth questioning whether artificial intelligence is even a science or not. Ken thinks that the secret to future progress is for us to embrace more subjectivity.  [00:00:00] Tim Intro [00:12:54] Intro [00:17:08] Seeing ideas everywhere - AI and art are highly connected [00:28:40] Creativity in Mathematics [00:30:14] Where is the intelligence in art? [00:38:49] Is AI disappointingly simple to mechanise? [00:42:48] Slightly conscious [00:46:27] Do we have subjective experience? [00:50:23] Fear of the unknown [00:51:48] Free Will [00:54:22] Chalmers [00:55:08] What's happening now in open-endedness [00:58:31] Generalisation [01:06:34] Representation primitives and what it means to understand [01:12:37] Appeal to definitions, knowledge itself blocks discovery Make sure you buy Kenneth's book! Why Greatness Cannot Be Planned: The Myth of the Objective [Stanley, Lehman] https://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 Abandoning Objectives: Evolution through the Search for Novelty Alone [Lehman, Stanley] https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf Twitter https://twitter.com/kenneth0stanley

Special discount link for Zak's GNN course - https://bit.ly/3uqmYVq Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB YT version: https://youtu.be/jAGIuobLp60 (there are lots of helper graphics there, recommended if poss) Want to sponsor MLST!? Let us know on Linkedin / Twitter.  [00:00:00] Preamble [00:03:12] Geometric deep learning [00:10:04] Message passing [00:20:42] Top down vs bottom up [00:24:59] All NN architectures are different forms of information diffusion processes (squashing and smoothing problem) [00:29:51] Graph rewiring [00:31:38] Back to information diffusion  [00:42:43] Transformers vs GNNs [00:47:10] Equivariant subgraph aggregation networks + WL test [00:55:36] Do equivariant layers aggregate too? [00:57:49] Zak's GNN course Exhaustive list of references on the YT show URL (https://youtu.be/jAGIuobLp60)

Today we are having a discussion with Letitia Parcalabescu from the AI Coffee Break youtube channel! We discuss linguistics, symbolic AI and our respective Youtube channels. Make sure you subscribe to her channel! In the first 15 minutes Tim dissects the recent article from Gary Marcus "Deep learning has hit a wall".  Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB YT: https://youtu.be/p2D2duT-R2E [00:00:00] Comments on Gary Marcus Article / Symbolic AI [00:14:57] Greetings [00:17:40] Introduction [00:18:48] A shared journey towards computation [00:22:10] A linguistics outsider [00:24:11] Is computational linguistics AI? [00:28:23] swinging pendulums of dogma and resource allocation [00:31:16] the road less travelled [00:34:35] pitching grants with multimodality ... and then the truth [00:40:50] some aspects of language are statistically learnable [00:44:58] ... and some aspects of language are dimensionally cursed [00:48:24] it's good to have both approaches to machine intelligence [00:51:14] the world runs on symbols [00:54:28] there is much more to learn biology [00:59:26] Letitia's creation process [01:02:23] don't overfit content, instead publish and iterate [01:07:48] merging the big picture arrow from the small direction arrows [01:11:02] use passion to drive through failure to success [01:12:56] stay positive [01:16:02] closing remarks

Today we are speaking with Dr. Thomas Lux, a research scientist at Meta in Silicon Valley.  In some sense, all of supervised machine learning can be framed through the lens of geometry. All training data exists as points in euclidean space, and we want to predict the value of a function at all those points. Neural networks appear to be the modus operandi these days for many domains of prediction. In that light; we might ask ourselves — what makes neural networks better than classical techniques like K nearest neighbour from a geometric perspective. Our guest today has done research on exactly that problem, trying to define error bounds for approximations in terms of directions, distances, and derivatives.   The insights from Thomas's work point at why neural networks are so good at problems which everything else fails at, like image recognition. The key is in their ability to ignore parts of the input space, do nonlinear dimension reduction, and concentrate their approximation power on important parts of the function.  [00:00:00] Intro to Show [00:04:11] Intro to Thomas (Main show kick off) [00:04:56] Interpolation of Sparse High-Dimensional Data [00:12:19] Where does one place the basis functions to partition the space, the perennial question [00:16:20] The sampling phenomenon -- where did all those dimensions come from? [00:17:40] The placement of the MLP basis functions, they are not where you think they are [00:23:15] NNs only extrapolate when given explicit priors to do so, CNNs in the translation domain [00:25:31] Transformers extrapolate in the permutation domain [00:28:26] NN priors work by creating space junk everywhere [00:36:44] Are vector spaces the way to go? On discrete problems [00:40:23] Activation functioms [00:45:57] What can we prove about NNs? Gradients without backprop Interpolation of Sparse High-Dimensional Data [Lux] https://tchlux.github.io/papers/tchlux-2020-NUMA.pdf A Spline Theory of Deep Learning [_Balestriero_] https://proceedings.mlr.press/v80/balestriero18b.html Gradients without Backpropagation ‘22 https://arxiv.org/pdf/2202.08587.pdf

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/HNnAwSduud YT version: https://youtu.be/pMtk-iUaEuQ Dr. Walid Saba is an old-school polymath. He has a background in cognitive  psychology, linguistics, philosophy, computer science and logic and he’s is now a Senior Scientist at Sorcero. Walid is perhaps the most outspoken critic of BERTOLOGY, which is to say trying to solve the problem of natural language understanding with application of large statistical language models. Walid thinks this approach is cursed to failure because it’s analogous to memorising infinity with a large hashtable. Walid thinks that the various appeals to infinity by some deep learning researchers are risible. [00:00:00] MLST Housekeeping [00:08:03] Dr. Walid Saba Intro [00:11:56] AI Cannot Ignore Symbolic Logic, and Here’s Why [00:23:39] Main show - Proposition: Statistical learning doesn't work [01:04:44] Discovering a sorting algorithm bottom-up is hard [01:17:36] The axioms of nature (universal cognitive templates) [01:31:06] MLPs are locality sensitive hashing tables References; The Missing Text Phenomenon, Again: the case of Compound Nominals https://ontologik.medium.com/the-missing-text-phenomenon-again-the-case-of-compound-nominals-abb6ece3e205 A Spline Theory of Deep Networks https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf The Defeat of the Winograd Schema Challenge https://arxiv.org/pdf/2201.02387.pdf Impact of Pretraining Term Frequencies on Few-Shot Reasoning https://twitter.com/yasaman_razeghi/status/1495112604854882304?s=21 https://arxiv.org/abs/2202.07206 AI Cannot Ignore Symbolic Logic, and Here’s Why https://medium.com/ontologik/ai-cannot-ignore-symbolic-logic-and-heres-why-1f896713525b Learnability can be undecidable http://gtts.ehu.es/German/Docencia/1819/AC/extras/s42256-018-0002-3.pdf Scaling Language Models: Methods, Analysis & Insights from Training Gopher https://arxiv.org/pdf/2112.11446.pdf DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning https://arxiv.org/abs/2006.08381 On the Measure of Intelligence [Chollet] https://arxiv.org/abs/1911.01547 A Formal Theory of Commonsense Psychology: How People Think People Think https://www.amazon.co.uk/Formal-Theory-Commonsense-Psychology-People/dp/1107151007 Continuum hypothesis https://en.wikipedia.org/wiki/Continuum_hypothesis Gödel numbering + completness theorems https://en.wikipedia.org/wiki/G%C3%B6del_numbering https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems Concepts: Where Cognitive Science Went Wrong [Jerry A. Fodor] https://oxford.universitypressscholarship.com/view/10.1093/0198236360.001.0001/acprof-9780198236368

We engage in a bit of epistemic foraging with Prof. Karl Friston! In this show; we discuss the free energy principle in detail, also emergence, cognition, consciousness and Karl's burden of knowledge! YT: https://youtu.be/xKQ-F2-o8uM Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/HNnAwSduud [00:00:00] Introduction to FEP/Friston [00:06:53] Cheers to Epistemic Foraging! [00:09:17] The Burden of Knowledge Across Disciplines [00:12:55] On-show introduction to Friston [00:14:23] Simple does NOT mean Easy [00:21:25] Searching for a Mathematics of Cognition [00:26:44] The Low Road and The High Road to the Principle [00:28:27] What's changed for the FEP in the last year [00:39:36] FEP as stochastic systems with a pullback attractor [00:44:03] An attracting set at multiple time scales and time infinity [00:53:56] What about fuzzy Markov boundaries? [00:59:17] Is reality densely or sparsely coupled? [01:07:00] Is a Strong and Weak Emergence distinction useful? [01:13:25] a Philosopher, a Zombie, and a Sentient Consciousness walk into a bar ...  [01:24:28] Can we recreate consciousness in silico? Will it have qualia? [01:28:29] Subjectivity and building hypotheses [01:34:17] Subject specific realizations to minimize free energy [01:37:21] Free will in a deterministic Universe The free energy principle made simpler but not too simple https://arxiv.org/abs/2201.06387

We have a chat with Alexander Mattick aka ZickZack from Yannic's Discord community. Alex is one of the leading voices in that community and has an impressive technical depth. Don't forget MLST has now started it's own Discord server too, come and join us! We are going to run regular events, our first big event on Wednesday 9th 1700-1900 UK time.  Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/HNnAwSduud YT version: https://youtu.be/rGOOLC8cIO4 [00:00:00] Introduction to Alex  [00:02:16] Spline theory of NNs  [00:05:19] Do NNs abstract?  [00:08:27] Tim's exposition of spline theory of NNs [00:11:11] Semantics in NNs  [00:13:37] Continuous vs discrete  [00:19:00] Open-ended Search [00:22:54] Inductive logic programming [00:25:00] Control to gain knowledge and knowledge to gain control [00:30:22] Being a generalist with a breadth of knowledge and knowledge transfer [00:36:29] Causality [00:43:14] Discrete program synthesis + theorem solvers

Note: there are no politics discussed in this show and please do not interpret this show as any kind of a political statement from us.  We have decided not to discuss politics on MLST anymore due to its divisive nature.  Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/HNnAwSduud [00:00:00] Intro [00:01:36] What we all need to understand about machine learning [00:06:05] The Master Algorithm Target Audience [00:09:50] Deeply Connected Algorithms seen from Divergent Frames of Reference [00:12:49] There is a Master Algorithm; and it's mine! [00:14:59] The Tribe of Evolution [00:17:17] Biological Inspirations and Predictive Coding [00:22:09] Shoe-Horning Gradient Descent [00:27:12] Sparsity at Training Time vs Prediction Time [00:30:00] World Models and Predictive Coding [00:33:24] The Cartoons of System 1 and System 2 [00:40:37] AlphaGo Searching vs Learning [00:45:56] Discriminative Models evolve into Generative Models [00:50:36] Generative Models, Predictive Coding, GFlowNets [00:55:50] Sympathy for a Thousand Brains [00:59:05] A Spectrum of Tribes [01:04:29] Causal Structure and Modelling [01:09:39] Entropy and The Duality of Past vs Future, Knowledge vs Control [01:16:14] A Discrete Universe? [01:19:49] And yet continuous models work so well [01:23:31] Finding a Discretised Theory of Everything

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/HNnAwSduud YT: https://www.youtube.com/watch?v=ZDY2nhkPZxw We have a chat with Prof. Gary Marcus about everything which is currently top of mind for him, consciousness  [00:00:00] Gary intro [00:01:25] Slightly conscious [00:24:59] Abstract, compositional models [00:32:46] Spline theory of NNs [00:36:17] Self driving cars / algebraic reasoning  [00:39:43] Extrapolation [00:44:15] Scaling laws [00:49:50] Maximum likelihood estimation References: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177 DEEP DOUBLE DESCENT: WHERE BIGGER MODELS AND MORE DATA HURT https://arxiv.org/pdf/1912.02292.pdf Bayesian Deep Learning and a Probabilistic Perspective of Generalization https://arxiv.org/pdf/2002.08791.pdf

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Patreon: https://www.patreon.com/mlst For Yoshua Bengio, GFlowNets are the most exciting thing on the horizon of Machine Learning today. He believes they can solve previously intractable problems and hold the key to unlocking machine abstract reasoning itself. This discussion explores the promise of GFlowNets and the personal journey Prof. Bengio traveled to reach them. Panel: Dr. Tim Scarfe Dr. Keith Duggar Dr. Yannic Kilcher Our special thanks to:  - Alexander Mattick (Zickzack) References: Yoshua Bengio @ MILA (https://mila.quebec/en/person/bengio-yoshua/) GFlowNet Foundations (https://arxiv.org/pdf/2111.09266.pdf) Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (https://arxiv.org/pdf/2106.04399.pdf) Interpolation Consistency Training for Semi-Supervised Learning (https://arxiv.org/pdf/1903.03825.pdf) Towards Causal Representation Learning (https://arxiv.org/pdf/2102.11107.pdf) Causal inference using invariant prediction: identification and confidence intervals (https://arxiv.org/pdf/1501.01332.pdf)

Dr. Guy Emerson is a computational linguist and obtained his Ph.D from Cambridge university where he is now a research fellow and lecturer. On panel we also have myself, Dr. Tim Scarfe, as well as Dr. Keith Duggar and the veritable Dr. Walid Saba. We dive into distributional semantics, probability theory, fuzzy logic, grounding, vagueness and the grammar/cognition connection. The aim of distributional semantics is to design computational techniques that can automatically learn the meanings of words from a body of text. The twin challenges are: how do we represent meaning, and how do we learn these representations? We want to learn the meanings of words from a corpus by exploiting the fact that the context of a word tells us something about its meaning. This is known as the distributional hypothesis. In his Ph.D thesis, Dr. Guy Emerson presented a distributional model which can learn truth-conditional semantics which are grounded by objects in the real world. Hope you enjoy the show! https://www.cai.cam.ac.uk/people/dr-guy-emerson https://www.repository.cam.ac.uk/handle/1810/284882?show=full https://www.semanticscholar.org/paper/Computational-linguistics-and-grammar-engineering-Bender-Emerson/bbd6f3b92a0f1ea8212f383cc4719bfe86b3588c Patreon: https://www.patreon.com/mlst

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Patreon: https://www.patreon.com/mlst Yann LeCun thinks that it's specious to say neural network models are interpolating because in high dimensions, everything is extrapolation. Recently Dr. Randall Balestriero, Dr. Jerome Pesente and prof. Yann LeCun released their paper learning in high dimensions always amounts to extrapolation. This discussion has completely changed how we think about neural networks and their behaviour. [00:00:00] Pre-intro [00:11:58] Intro Part 1: On linearisation in NNs [00:28:17] Intro Part 2: On interpolation in NNs [00:47:45] Intro Part 3: On the curse [00:48:19] LeCun [01:40:51] Randall B YouTube version: https://youtu.be/86ib0sfdFtw

Patreon: https://www.patreon.com/mlst The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact tractable given enough computational horsepower. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not uniform and have strong repeating patterns as a result of the low-dimensionality and structure of the physical world. Geometric Deep Learning unifies a broad class of ML problems from the perspectives of symmetry and invariance. These principles not only underlie the breakthrough performance of convolutional neural networks and the recent success of graph neural networks but also provide a principled way to construct new types of problem-specific inductive biases. This week we spoke with Professor Michael Bronstein (head of graph ML at Twitter) and Dr. Petar Veličković (Senior Research Scientist at DeepMind), and Dr. Taco Cohen and Prof. Joan Bruna about their new proto-book Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. See the table of contents for this (long) show at https://youtu.be/bIZB1hIJ4u8 

Patreon: https://www.patreon.com/mlst The ultimate goal of neuroscience is to learn how the human brain gives rise to human intelligence and what it means to be intelligent. Understanding how the brain works is considered one of humanity’s greatest challenges.  Jeff Hawkins thinks that the reality we perceive is a kind of simulation, a hallucination, a confabulation. He thinks that our brains are a model reality based on thousands of information streams originating from the sensors in our body.  Critically - Hawkins doesn’t think there is just one model but rather; thousands.  Jeff has just released his new book, A thousand brains: a new theory of intelligence. It’s an inspiring and well-written book and I hope after watching this show; you will be inspired to read it too.  https://numenta.com/a-thousand-brains-by-jeff-hawkins/ https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/ Panel: Dr. Keith Duggar https://twitter.com/DoctorDuggar Connor Leahy https://twitter.com/npcollapse

The field of Artificial Intelligence was founded in the mid 1950s with the aim of constructing “thinking machines” - that is to say, computer systems with human-like general intelligence. Think of humanoid robots that not only look but act and think with intelligence equal to and ultimately greater than that of human beings. But in the intervening years, the field has drifted far from its ambitious old-fashioned roots. Dr. Ben Goertzel is an artificial intelligence researcher, CEO and founder of SingularityNET. A project combining artificial intelligence and blockchain to democratize access to artificial intelligence. Ben seeks to fulfil the original ambitions of the field.  Ben graduated with a PhD in Mathematics from Temple University in 1990. Ben’s approach to AGI over many decades now has been inspired by many disciplines, but in particular from human cognitive psychology and computer science perspective. To date Ben’s work has been mostly theoretically-driven. Ben thinks that most of the deep learning approaches to AGI today try to model the brain. They may have a loose analogy to human neuroscience but they have not tried to derive the details of an AGI architecture from an overall conception of what a mind is. Ben thinks that what matters for creating human-level (or greater) intelligence is having the right information processing architecture, not the underlying mechanics via which the architecture is implemented. Ben thinks that there is a certain set of key cognitive processes and interactions that AGI systems must implement explicitly such as; working and long-term memory, deliberative and reactive processing, perc biological systems tend to be messy, complex and integrative; searching for a single “algorithm of general intelligence” is an inappropriate attempt to project the aesthetics of physics or theoretical computer science into a qualitatively different domain. TOC is on the YT show description https://www.youtube.com/watch?v=sw8IE3MX1SY Panel: Dr. Tim Scarfe, Dr. Yannic Kilcher, Dr. Keith Duggar Artificial General Intelligence: Concept, State of the Art, and Future Prospects https://sciendo.com/abstract/journals... The General Theory of General Intelligence: A Pragmatic Patternist Perspective https://arxiv.org/abs/2103.15100

Since its beginning in the 1950s, the field of artificial intelligence has vacillated between periods of optimistic predictions and massive investment and periods of disappointment, loss of confidence, and reduced funding. Even with today’s seemingly fast pace of AI breakthroughs, the development of long-promised technologies such as self-driving cars, housekeeping robots, and conversational companions has turned out to be much harder than many people expected.  Professor Melanie Mitchell thinks one reason for these repeating cycles is our limited understanding of the nature and complexity of intelligence itself. YT vid- https://www.youtube.com/watch?v=A8m1Oqz2HKc Main show kick off [00:26:51] Panel: Dr. Tim Scarfe, Dr. Keith Duggar, Letitia Parcalabescu (https://www.youtube.com/c/AICoffeeBreak/)

It has been over three decades since the statistical revolution overtook AI by a storm and over two  decades since deep learning (DL) helped usher the latest resurgence of artificial intelligence (AI). However, the disappointing progress in conversational agents, NLU, and self-driving cars, has made it clear that progress has not lived up to the promise of these empirical and data-driven methods. DARPA has suggested that it is time for a third wave in AI, one that would be characterized by hybrid models – models that combine knowledge-based approaches with data-driven machine learning techniques.  Joining us on this panel discussion is polymath and linguist Walid Saba - Co-founder ONTOLOGIK.AI, Gadi Singer - VP & Director, Cognitive Computing Research, Intel Labs and J. Mark Bishop - Professor of Cognitive Computing (Emeritus), Goldsmiths, University of London and Scientific Adviser to FACT360. Moderated by Dr. Keith Duggar and Dr. Tim Scarfe https://www.linkedin.com/in/gadi-singer/ https://www.linkedin.com/in/walidsaba/ https://www.linkedin.com/in/profjmarkbishop/ #machinelearning #artificialintelligence

Dr. Ishan Misra is a Research Scientist at Facebook AI Research where he works on Computer Vision and Machine Learning. His main research interest is reducing the need for human supervision, and indeed, human knowledge in visual learning systems. He finished his PhD at the Robotics Institute at Carnegie Mellon. He has done stints at Microsoft Research, INRIA and Yale. His bachelors is in computer science where he achieved the highest GPA in his cohort.  Ishan is fast becoming a prolific scientist, already with more than 3000 citations under his belt and co-authoring with Yann LeCun; the godfather of deep learning.  Today though we will be focusing an exciting cluster of recent papers around unsupervised representation learning for computer vision released from FAIR. These are; DINO: Emerging Properties in Self-Supervised Vision Transformers, BARLOW TWINS: Self-Supervised Learning via Redundancy Reduction and PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples. All of these papers are hot off the press, just being officially released in the last month or so. Many of you will remember PIRL: Self-Supervised Learning of Pretext-Invariant Representations which Ishan was the primary author of in 2019. References; Shuffle and Learn - https://arxiv.org/abs/1603.08561 DepthContrast - https://arxiv.org/abs/2101.02691 DINO - https://arxiv.org/abs/2104.14294 Barlow Twins - https://arxiv.org/abs/2103.03230 SwAV - https://arxiv.org/abs/2006.09882 PIRL - https://arxiv.org/abs/1912.01991 AVID - https://arxiv.org/abs/2004.12943 (best paper candidate at CVPR'21 (just announced over the weekend) - http://cvpr2021.thecvf.com/node/290)   Alexei (Alyosha) Efros http://people.eecs.berkeley.edu/~efros/ http://www.cs.cmu.edu/~tmalisie/projects/nips09/   Exemplar networks https://arxiv.org/abs/1406.6909   The bitter lesson - Rich Sutton http://www.incompleteideas.net/IncIdeas/BitterLesson.html   Machine Teaching: A New Paradigm for Building Machine Learning Systems https://arxiv.org/abs/1707.06742   POET https://arxiv.org/pdf/1901.01753.pdf

Professor Gary Marcus is a scientist, best-selling author, and entrepreneur. He is Founder and CEO of Robust.AI, and was Founder and CEO of Geometric Intelligence, a machine learning company acquired by Uber in 2016. Gary said in his recent next decade paper that — without us, or other creatures like us, the world would continue to exist, but it would not be described, distilled, or understood.  Human lives are filled with abstraction and causal description. This is so powerful. Francois Chollet the other week said that intelligence is literally sensitivity to abstract analogies, and that is all there is to it. It's almost as if one of the most important features of intelligence is to be able to abstract knowledge, this drives the generalisation which will allow you to mine previous experience to make sense of many future novel situations.   Also joining us today is Professor Luis Lamb — Secretary of Innovation for Science and Technology of the State of Rio Grande do Sul, Brazil. His Research Interests are Machine Learning and Reasoning, Neuro-Symbolic Computing, Logic in Computation and Artificial Intelligence, Cognitive and Neural Computation and also AI Ethics and Social Computing. Luis released his new paper Neurosymbolic AI: the third wave at the end of last year. It beautifully articulated the key ingredients needed in the next generation of AI systems, integrating type 1 and type 2 approaches to AI and it summarises all the of the achievements of the last 20 years of research.   We cover a lot of ground in today's show. Explaining the limitations of deep learning, Rich Sutton's the bitter lesson and "reward is enough", and the semantic foundation which is required for us to build robust AI.

Bob Coercke is a celebrated physicist, he's been a Physics and Quantum professor at Oxford University for the last 20 years. He is particularly interested in Structure which is to say, Logic, Order, and Category Theory. He is well known for work involving compositional distributional models of natural language meaning and he is also fascinated with understanding how our brains work. Bob was recently appointed as the Chief Scientist at Cambridge Quantum Computing. Bob thinks that interactions between systems in Quantum Mechanics carries naturally over to how word meanings interact in natural language. Bob argues that this interaction embodies the phenomenon of quantum teleportation. Bob invented ZX-calculus, a graphical calculus for revealing the compositional structure inside quantum circuits - to show entanglement states and protocols in a visually succinct but logically complete way. Von Neumann himself didn't even like his own original symbolic formalism of quantum theory, despite it being widely used! We hope you enjoy this fascinating conversation which might give you a lot of insight into natural language processing.  Tim Intro [00:00:00] The topological brain (Post-record button skit) [00:13:22] Show kick off [00:19:31] Bob introduction [00:22:37] Changing culture in universities [00:24:51] Machine Learning is like electricity [00:31:50] NLP -- what is Bob's Quantum conception? [00:34:50] The missing text problem [00:52:59] Can statistical induction be trusted? [00:59:49] On pragmatism and hybrid systems [01:04:42] Parlour tricks, parsing and information flows [01:07:43] How much human input is required with Bob's method? [01:11:29] Reality, meaning, structure and language [01:14:42] Replacing complexity with quantum entanglement, emergent complexity [01:17:45] Loading quantum data requires machine learning [01:19:49]  QC is happy math coincidence for NLP [01:22:30] The Theory of English (ToE) [01:28:23]  ... or can we learn the ToE? [01:29:56]  How did diagrammatic quantum calculus come about? [01:31:04 The state of quantum computing today [01:37:49]  NLP on QC might be doable even in the NISQ era [01:40:48]  Hype and private investment are driving progress [01:48:34]  Crypto discussion (moved to post-show) [01:50:38]  Kilcher is in a startup (moved to post show) [01:53:40 Debrief [01:55:26] 

Performing reliably on unseen or shifting data distributions is a difficult challenge for modern vision systems, even slight corruptions or transformations of images are enough to slash the accuracy of state-of-the-art classifiers. When an adversary is allowed to modify an input image directly, models can be manipulated into predicting anything even when there is no perceptible change, this is known an adversarial example. The ideal definition of an adversarial example is when humans consistently say two pictures are the same but a machine disagrees. Hadi Salman, a Ph.D student at MIT (ex-Uber and Microsoft Research) started thinking about how adversarial robustness  could be leveraged beyond security. He realised that the phenomenon of adversarial examples could actually be turned upside down to lead to more robust models instead of breaking them. Hadi actually utilized the brittleness of neural networks to design unadversarial examples or robust objects which_ are objects designed specifically to be robustly recognized by neural networks.  Introduction [00:00:00] DR KILCHER'S PHD HAT [00:11:18] Main Introduction [00:11:38] Hadi's Introduction [00:14:43] More robust models == transfer better [00:46:41] Features not bugs paper [00:49:13] Manifolds [00:55:51] Robustness and Transferability [00:58:00] Do non-robust features generalize worse than robust? [00:59:52] The unreasonable predicament of entangled features [01:01:57] We can only find adversarial examples in the vicinity [01:09:30] Certifiability of models for robustness [01:13:55] Carlini is coming for you! And we are screwed [01:23:21] Distribution shift and corruptions are a bigger problem than adversarial examples [01:25:34] All roads lead to generalization [01:26:47] Unadversarial examples [01:27:26]

In today's show we are joined by Francois Chollet, I have been inspired by Francois ever since I read his Deep Learning with Python book and started using the Keras library which he invented many, many years ago. Francois has a clarity of thought that I've never seen in any other human being! He has extremely interesting views on intelligence as generalisation, abstraction and an information conversation ratio. He wrote on the measure of intelligence at the end of 2019 and it had a huge impact on my thinking. He thinks that NNs can only model continuous problems, which have a smooth learnable manifold and that many "type 2" problems which involve reasoning and/or planning are not suitable for NNs. He thinks that many problems have type 1 and type 2 enmeshed together. He thinks that the future of AI must include program synthesis to allow us to generalise broadly from a few examples, but the search could be guided by neural networks because the search space is interpolative to some extent. https://youtu.be/J0p_thJJnoo Tim's Whimsical notes; https://whimsical.com/chollet-show-QQ2atZUoRR3yFDsxKVzCbj

Dr. Christian Szegedy from Google Research is a deep learning heavyweight. He invented adversarial examples, one of the first object detection algorithms, the inceptionnet architecture, and co-invented batchnorm. He thinks that if you bet on computers and software in 1990 you would have been as right as if you bet on AI now. But he thinks that we have been programming computers the same way since the 1950s and there has been a huge stagnation ever since. Mathematics is the process of taking a fuzzy thought and formalising it. But could we automate that? Could we create a system which will act like a super human mathematician but you can talk to it in natural language? This is what Christian calls autoformalisation. Christian thinks that automating many of the things we do in mathematics is the first step towards software synthesis and building human-level AGI. Mathematics ability is the litmus test for general reasoning ability. Christian has a fascinating take on transformers too.  With Yannic Lightspeed Kilcher and Dr. Mathew Salvaris Whimsical Canvas with Tim's Notes: https://whimsical.com/mar-26th-christian-szegedy-CpgGhnEYDBrDMFoATU6XYC YouTube version (with detailed table of contents) https://youtu.be/ehNGGYFO6ms