{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实验 02 · Python 与数学最小坡道 (Google Colab 交互式实验场)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/zenHeart/learn-llm/blob/main/notebooks/02_python_math.ipynb)\n",
    "\n",
    "本 Notebook 专为配套 **Learn LLM** 第 2 章设计。读者可直接在 Google Colab（支持 Google Pro 会员的高性能 GPU 与 High-RAM 运行时）上一键运行。\n",
    "\n",
    "### 学习目标：\n",
    "1. 掌握中心差商（Central Difference Quotient）的数值微分原理与 $O(h^2)$ 截断误差；\n",
    "2. 亲身体验 IEEE 754 浮点下溢与灾难性抵消（Catastrophic Cancellation）；\n",
    "3. 手写向量点积、余弦相似度与 Softmax 基础数学原语。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# [单元 1] 运行环境与 Google Pro GPU 算力检测\n",
    "import math\n",
    "import sys\n",
    "import torch\n",
    "\n",
    "print(f\"Python 版本: {sys.version.split()[0]}\")\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "print(f\"计算设备: {device}\")\n",
    "if device == \"cuda\":\n",
    "    gpu_name = torch.cuda.get_device_name(0)\n",
    "    vram_gb = torch.cuda.get_device_properties(0).total_memory / 1e9\n",
    "    print(f\"🚀 已识别 Google Pro GPU: {gpu_name} (显存: {vram_gb:.2f} GB)\")\n",
    "    # 针对现代 GPU (A100/L4) 开启高精度矩阵加速\n",
    "    torch.backends.cuda.matmul.allow_tf32 = True\n",
    "else:\n",
    "    print(\"ℹ️ 当前为 CPU 模式。本章属于轻量数学计算，CPU 即可极速完成。后文涉及 TinyGPT 与 LoRA 推荐在顶部菜单切换 GPU。\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习一：对称中心差商（Difference Quotient）\n",
    "\n",
    "导数定义为极限 $\\lim_{h \\to 0} \\frac{f(x+h) - f(x)}{h}$。但在计算机浮点计算中：\n",
    "- **前向差商**截断误差为 $O(h)$；\n",
    "- **中心差商**：$f'(x) \\approx \\frac{f(x+h) - f(x-h)}{2h}$，二阶泰勒项相互抵消，截断误差跃升至 $O(h^2)$！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def difference_quotient(f, x: float, h: float = 1e-5) -> float:\n",
    "    \"\"\"对称中心差商计算数值有限差分导数\n",
    "    \n",
    "    公式: (f(x + h) - f(x - h)) / (2 * h)\n",
    "    \"\"\"\n",
    "    return (f(x + h) - f(x - h)) / (2.0 * h)\n",
    "\n",
    "# 验证用例 1: f(t) = t^3 在 x = 2.0 处的导数 (解析真值 3*2^2 = 12.0)\n",
    "f_cube = lambda t: t**3\n",
    "approx = difference_quotient(f_cube, 2.0)\n",
    "expected = 12.0\n",
    "abs_err = abs(approx - expected)\n",
    "\n",
    "print(f\"数值计算值: {approx:.8f}\")\n",
    "print(f\"理论真实值: {expected:.8f}\")\n",
    "print(f\"绝对误差:   {abs_err:.2e}\")\n",
    "assert abs_err < 1e-5, f\"中心差商精度不足: {abs_err}\"\n",
    "print(\"✅ 练习一检验通过！\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习二：探索极端步长下的浮点灾难（IEEE 754 下溢）\n",
    "\n",
    "很多人误以为步长 $h$ 越小越好。我们用循环从 $1.0$ 一路缩小到 $10^{-17}$，观察误差变化曲线："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h_candidates = [1.0, 0.1, 1e-3, 1e-5, 1e-8, 1e-12, 1e-15, 1e-16, 1e-17]\n",
    "\n",
    "print(\"  h 步长      |  中心差商导数值  |  绝对误差      | 状态说明\")\n",
    "print(\"-\" * 60)\n",
    "for h in h_candidates:\n",
    "    val = difference_quotient(f_cube, 2.0, h=h)\n",
    "    err = abs(val - 12.0)\n",
    "    status = \"最佳工作区\" if h == 1e-5 else (\"截断误差主导\" if h > 1e-3 else \"舍入误差/下溢\")\n",
    "    if h <= 1e-16:\n",
    "        status = \"⚠️ 浮点抵消归零！\"\n",
    "    print(f\"{h:<13.1e} | {val:<16.8f} | {err:<14.2e} | {status}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习三：手写自注意力核心数学原语（Softmax 与点积）\n",
    "\n",
    "为了避免数值溢出（Overflow），标准 Softmax 实现必须使用平移技巧：$\\mathrm{Softmax}(z_i) = \\frac{e^{z_i - \\max(z)}}{\\sum_j e^{z_j - \\max(z)}}$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def safe_softmax(logits: list[float]) -> list[float]:\n",
    "    max_val = max(logits)\n",
    "    exp_vals = [math.exp(x - max_val) for x in logits]\n",
    "    total = sum(exp_vals)\n",
    "    return [v / total for v in exp_vals]\n",
    "\n",
    "# 即使输入极大数值 (如 1000.0) 也不会触发 float overflow\n",
    "huge_logits = [1000.0, 1002.0, 1005.0]\n",
    "probs = safe_softmax(huge_logits)\n",
    "print(f\"极大 Logits: {huge_logits}\")\n",
    "print(f\"归一化概率: {[round(p, 4) for p in probs]}\")\n",
    "print(f\"概率总和:   {sum(probs):.6f}\")\n",
    "assert math.isclose(sum(probs), 1.0, rel_tol=1e-7)\n",
    "print(\"✅ 稳定 Softmax 检验通过！恭喜完成实验 02！\")"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}