Full course coming soon

Memory Architecture for AI Systems

7 chapters · 34 lessons

1. 1. Memory Hierarchy Fundamentals

   5 lessons

   Understand the layers of memory from registers to storage and how proximity to compute affects speed, capacity, and cost.

2. 2. High Bandwidth Memory and 3D Stacking

   5 lessons

   Learn how vertical integration and wide data paths solve bandwidth constraints in AI accelerators.

3. 3. Profiling Memory Bottlenecks in AI Workloads

   5 lessons

   Use profiling tools and metrics to identify where memory limits performance in training and inference.

4. 4. Memory Architecture in AI Training

   5 lessons

   Optimize training pipelines by understanding how batch size, model size, and gradient accumulation interact with memory.

5. 5. Memory Architecture in AI Inference

   5 lessons

   Design inference systems that maximize throughput and minimize latency by matching memory to serving patterns.

6. 6. Evaluating AI Accelerator Specifications

   4 lessons

   Read datasheets critically and compare hardware options based on memory hierarchy, not just FLOPS.

7. 7. Total Cost of Ownership and Infrastructure Decisions

   5 lessons

   Calculate TCO by factoring memory architecture into power, cooling, utilization, and procurement strategy.