
Wednesday Jan 29, 2025
AI Computing Hardware - Past, Present, and Future
A special one-off episode with a deep dive into the past, present, and future of how computer hardware makes AI possible.
Join our brand new Discord here! https://discord.gg/nTyezGSKwP
Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.
Sponsors:
- The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence.
In this episode:
- Google and Mistral sign deals with AP and AFP, respectively, to deliver up-to-date news through their AI platforms.
- ChatGPT introduces a tasks feature for reminders and to-dos, positioning itself more as a personal assistant.
- Synthesia raises $180 million to enhance its AI video platform for generating videos of human avatars.
- New U.S. guidelines restrict exporting AI chips to various countries, impacting Nvidia and other tech firms.
If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.
Timestamps:
- 00:00:00 Introduction
- 00:03:08 Historical Recap: Early AI and Hardware
- 00:11:51 The Rise of GPUs and Deep Learning
- 00:15:39 Scaling Laws and the Evolution of AI Models
- 00:24:05 The Bitter Lesson and the Future of AI Compute
- 00:25:58 Moore's Law and Huang's Law
- 00:30:12 Memory and Logic in AI Hardware
- 00:34:53 Challenges in AI Hardware: The Memory Wall
- 00:37:08 The Role of GPUs in Modern AI
- 00:42:27 Fitting Neural Nets in GPUs
- 00:48:04 Batch Sizes and GPU Utilization
- 00:52:47 Parallelism in AI Models
- 00:55:53 Matrix Multiplications and GPUs
- 00:59:57 Understanding B200 and GB200
- 01:05:41 Data Center Hierarchy
- 01:13:42 High Bandwidth Memory (HBM)
- 01:16:45 Fabrication and Packaging
- 01:20:17 The Complexity of Semiconductor Fabrication
- 01:24:34 Understanding Process Nodes
- 01:28:26 The Art of Fabrication
- 01:33:17 The Role of Yield in Fabrication
- 01:35:47 The Photolithography Process
- 01:40:38 Deep Ultraviolet Lithography (DUV)
- 01:43:58 Extreme Ultraviolet Lithography (EUV)
- 01:51:46 Export Controls and Their Impact
- 01:54:22 The Rise of Custom AI Hardware
- 02:00:10 The Future of AI and Hardware
Comments (0)
To leave or reply to comments, please download free Podbean or
No Comments
To leave or reply to comments,
please download free Podbean App.