<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Sqlite on Tarragon</title><link>https://tarrragon.github.io/blog/tags/sqlite/</link><description>Recent content in Sqlite on Tarragon</description><generator>Hugo -- gohugo.io</generator><language>zh-TW</language><copyright>Tarragon (CC BY 4.0)</copyright><lastBuildDate>Sat, 20 Jun 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://tarrragon.github.io/blog/tags/sqlite/index.xml" rel="self" type="application/rss+xml"/><item><title>規模演進</title><link>https://tarrragon.github.io/blog/monitoring/04-collector/scaling-evolution/</link><pubDate>Fri, 19 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/04-collector/scaling-evolution/</guid><description>&lt;p>Collector 的儲存方案是可插拔 storage backend — 同一個 binary 透過啟動參數選擇不同的 storage implementation。Go 的 interface composition 讓 storage 分成 BasicStorage（所有 backend 共用）和 AnalyticsStorage（PostgreSQL 層新增），內部實作（SQLite / PostgreSQL / 時間序列 DB）分離，切換是 config change 而非重寫程式碼。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-go" data-lang="go">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="kd">type&lt;/span> &lt;span class="nx">BasicStorage&lt;/span> &lt;span class="kd">interface&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl"> &lt;span class="nf">Store&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">event&lt;/span> &lt;span class="nx">Event&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="kt">error&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl"> &lt;span class="nf">Query&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">filter&lt;/span> &lt;span class="nx">QueryFilter&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">([]&lt;/span>&lt;span class="nx">Event&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="kt">error&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl"> &lt;span class="nf">Close&lt;/span>&lt;span class="p">()&lt;/span> &lt;span class="kt">error&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl"> &lt;span class="nf">Downsample&lt;/span>&lt;span class="p">()&lt;/span> &lt;span class="kt">error&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl"> &lt;span class="nf">Purge&lt;/span>&lt;span class="p">()&lt;/span> &lt;span class="kt">error&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="kd">type&lt;/span> &lt;span class="nx">AnalyticsStorage&lt;/span> &lt;span class="kd">interface&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl"> &lt;span class="nx">BasicStorage&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl"> &lt;span class="nf">Aggregate&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">spec&lt;/span> &lt;span class="nx">AggregateSpec&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">(&lt;/span>&lt;span class="nx">AggregateResult&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="kt">error&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl"> &lt;span class="nf">Funnel&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">steps&lt;/span> &lt;span class="p">[]&lt;/span>&lt;span class="kt">string&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">timeWindow&lt;/span> &lt;span class="nx">Duration&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">(&lt;/span>&lt;span class="nx">FunnelResult&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="kt">error&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl"> &lt;span class="nf">Cohort&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">groupBy&lt;/span> &lt;span class="kt">string&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">metric&lt;/span> &lt;span class="kt">string&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">(&lt;/span>&lt;span class="nx">CohortResult&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="kt">error&lt;/span>&lt;span class="p">)&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>SQLite implementation 只實作 &lt;code>BasicStorage&lt;/code>。PostgreSQL implementation 實作 &lt;code>AnalyticsStorage&lt;/code>。Dashboard 用 Go 的 type assertion（&lt;code>if as, ok := storage.(AnalyticsStorage); ok { ... }&lt;/code>）判斷能力 — funnel/cohort 視圖在 SQLite 模式下不顯示入口，而非顯示後報錯。&lt;/p>
&lt;p>選擇哪個 backend 取決於部署場景和查詢需求：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>場景&lt;/th>
 &lt;th>Backend&lt;/th>
 &lt;th>啟動參數&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>自架簡單版（零依賴）&lt;/td>
 &lt;td>SQLite&lt;/td>
 &lt;td>&lt;code>--storage=sqlite&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>需要聚合分析的自用版&lt;/td>
 &lt;td>PostgreSQL&lt;/td>
 &lt;td>&lt;code>--storage=postgres --dsn=...&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>高併發 + 長期保留&lt;/td>
 &lt;td>時間序列 DB&lt;/td>
 &lt;td>&lt;code>--storage=timescale --dsn=...&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h2 id="sqlite-backendday-one-預設">SQLite Backend（day-one 預設）&lt;/h2>
&lt;p>SQLite 是嵌入式資料庫，編譯進 collector binary 中，不需要額外 server。Go 用 &lt;code>modernc.org/sqlite&lt;/code>（pure Go、無 CGO 依賴、效能約為 CGO driver mattn/go-sqlite3 的 60-80%，自用規模下足夠），開源使用者 &lt;code>go build &amp;amp;&amp;amp; ./collector&lt;/code> 就能跑，部署步驟為零。WAL mode 允許讀寫並行 — dashboard 的 SELECT 查詢不會被 ingestion 的 INSERT 阻塞，反之亦然。寫入之間的競爭由 busy_timeout 處理。&lt;/p>
&lt;h3 id="能力範圍">能力範圍&lt;/h3>
&lt;ul>
&lt;li>&lt;strong>索引查詢&lt;/strong>：按 type、name、timestamp 建索引，查詢從全表掃描變成索引查找&lt;/li>
&lt;li>&lt;strong>SQL 聚合&lt;/strong>：&lt;code>SELECT name, COUNT(*) FROM events WHERE type='error' GROUP BY name&lt;/code> — 一行 SQL 完成分群計數&lt;/li>
&lt;li>&lt;strong>跨欄位過濾&lt;/strong>：&lt;code>WHERE type='error' AND name LIKE 'terminal.%' AND ts &amp;gt; '2026-06-18'&lt;/code>&lt;/li>
&lt;li>&lt;strong>寫入&lt;/strong>：WAL mode 下每秒數千筆 append 寫入&lt;/li>
&lt;/ul>
&lt;h3 id="events-主表-ddl">Events 主表 DDL&lt;/h3>
&lt;p>Events 表的欄位從 &lt;a href="https://tarrragon.github.io/blog/monitoring/02-log-schema/event-schema-fields/" data-link-title="event.schema.json 完整欄位解說" data-link-desc="監控事件的 JSON Schema 定義 — 每個欄位的語意、必填/選填、資料型別和設計理由">event.schema.json&lt;/a> 的 JSON 結構推導。Source 的 nested object 攤平成獨立 column — 方便 SQL 查詢和索引，不需要每次從 JSON 裡 extract。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="k">CREATE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">TABLE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">events&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">id&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">INTEGER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">PRIMARY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">KEY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">AUTOINCREMENT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">v&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">INTEGER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">DEFAULT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">1&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">type&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">ts&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">source_sdk&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">source_app&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">source_version&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">source_platform&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">source_os&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">session_id&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">session_started&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">level&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">data&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">error_message&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">error_stack&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">error_type&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">receive_ts&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="p">);&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;code>source_sdk&lt;/code> 獨立成 column 讓「按 SDK 來源篩選」（&lt;code>WHERE source_sdk = 'python'&lt;/code>）不需要從 JSON extract。&lt;code>data&lt;/code> 用 TEXT 存 JSON。SQLite 沒有原生 JSON 型別，但 3.38+ 支援 &lt;code>json_extract()&lt;/code> 函式做查詢（&lt;code>WHERE json_extract(data, '$.duration_ms') &amp;gt; 1000&lt;/code>）。&lt;code>session_id&lt;/code> 獨立成 column 讓 session 回放的 JOIN 不需要 JSON extract。&lt;code>error_stack&lt;/code> 獨立成 column 讓 error 調查時全文搜尋 stack trace 不需要 JSON extract。&lt;code>receive_ts&lt;/code> 是 collector 收到事件的時間，和 SDK 端的 &lt;code>ts&lt;/code> 對照可估算 clock drift。&lt;/p></description><content:encoded><![CDATA[<p>Collector 的儲存方案是可插拔 storage backend — 同一個 binary 透過啟動參數選擇不同的 storage implementation。Go 的 interface composition 讓 storage 分成 BasicStorage（所有 backend 共用）和 AnalyticsStorage（PostgreSQL 層新增），內部實作（SQLite / PostgreSQL / 時間序列 DB）分離，切換是 config change 而非重寫程式碼。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="kd">type</span> <span class="nx">BasicStorage</span> <span class="kd">interface</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">    <span class="nf">Store</span><span class="p">(</span><span class="nx">event</span> <span class="nx">Event</span><span class="p">)</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">    <span class="nf">Query</span><span class="p">(</span><span class="nx">filter</span> <span class="nx">QueryFilter</span><span class="p">)</span> <span class="p">([]</span><span class="nx">Event</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">    <span class="nf">Close</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl">    <span class="nf">Downsample</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">    <span class="nf">Purge</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="kd">type</span> <span class="nx">AnalyticsStorage</span> <span class="kd">interface</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">    <span class="nx">BasicStorage</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">    <span class="nf">Aggregate</span><span class="p">(</span><span class="nx">spec</span> <span class="nx">AggregateSpec</span><span class="p">)</span> <span class="p">(</span><span class="nx">AggregateResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">    <span class="nf">Funnel</span><span class="p">(</span><span class="nx">steps</span> <span class="p">[]</span><span class="kt">string</span><span class="p">,</span> <span class="nx">timeWindow</span> <span class="nx">Duration</span><span class="p">)</span> <span class="p">(</span><span class="nx">FunnelResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">    <span class="nf">Cohort</span><span class="p">(</span><span class="nx">groupBy</span> <span class="kt">string</span><span class="p">,</span> <span class="nx">metric</span> <span class="kt">string</span><span class="p">)</span> <span class="p">(</span><span class="nx">CohortResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p>SQLite implementation 只實作 <code>BasicStorage</code>。PostgreSQL implementation 實作 <code>AnalyticsStorage</code>。Dashboard 用 Go 的 type assertion（<code>if as, ok := storage.(AnalyticsStorage); ok { ... }</code>）判斷能力 — funnel/cohort 視圖在 SQLite 模式下不顯示入口，而非顯示後報錯。</p>
<p>選擇哪個 backend 取決於部署場景和查詢需求：</p>
<table>
  <thead>
      <tr>
          <th>場景</th>
          <th>Backend</th>
          <th>啟動參數</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>自架簡單版（零依賴）</td>
          <td>SQLite</td>
          <td><code>--storage=sqlite</code></td>
      </tr>
      <tr>
          <td>需要聚合分析的自用版</td>
          <td>PostgreSQL</td>
          <td><code>--storage=postgres --dsn=...</code></td>
      </tr>
      <tr>
          <td>高併發 + 長期保留</td>
          <td>時間序列 DB</td>
          <td><code>--storage=timescale --dsn=...</code></td>
      </tr>
  </tbody>
</table>
<h2 id="sqlite-backendday-one-預設">SQLite Backend（day-one 預設）</h2>
<p>SQLite 是嵌入式資料庫，編譯進 collector binary 中，不需要額外 server。Go 用 <code>modernc.org/sqlite</code>（pure Go、無 CGO 依賴、效能約為 CGO driver mattn/go-sqlite3 的 60-80%，自用規模下足夠），開源使用者 <code>go build &amp;&amp; ./collector</code> 就能跑，部署步驟為零。WAL mode 允許讀寫並行 — dashboard 的 SELECT 查詢不會被 ingestion 的 INSERT 阻塞，反之亦然。寫入之間的競爭由 busy_timeout 處理。</p>
<h3 id="能力範圍">能力範圍</h3>
<ul>
<li><strong>索引查詢</strong>：按 type、name、timestamp 建索引，查詢從全表掃描變成索引查找</li>
<li><strong>SQL 聚合</strong>：<code>SELECT name, COUNT(*) FROM events WHERE type='error' GROUP BY name</code> — 一行 SQL 完成分群計數</li>
<li><strong>跨欄位過濾</strong>：<code>WHERE type='error' AND name LIKE 'terminal.%' AND ts &gt; '2026-06-18'</code></li>
<li><strong>寫入</strong>：WAL mode 下每秒數千筆 append 寫入</li>
</ul>
<h3 id="events-主表-ddl">Events 主表 DDL</h3>
<p>Events 表的欄位從 <a href="/blog/monitoring/02-log-schema/event-schema-fields/" data-link-title="event.schema.json 完整欄位解說" data-link-desc="監控事件的 JSON Schema 定義 — 每個欄位的語意、必填/選填、資料型別和設計理由">event.schema.json</a> 的 JSON 結構推導。Source 的 nested object 攤平成獨立 column — 方便 SQL 查詢和索引，不需要每次從 JSON 裡 extract。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">events</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="w">    </span><span class="n">id</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="n">AUTOINCREMENT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">    </span><span class="n">v</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">    </span><span class="n">name</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">    </span><span class="n">ts</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">    </span><span class="n">source_sdk</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w">    </span><span class="n">source_app</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w">    </span><span class="n">source_version</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w">    </span><span class="n">source_platform</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="w">    </span><span class="n">source_os</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="w">    </span><span class="n">session_id</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w">    </span><span class="n">session_started</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="w">    </span><span class="k">level</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w">    </span><span class="k">data</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w">    </span><span class="n">error_message</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w">    </span><span class="n">error_stack</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="w">    </span><span class="n">error_type</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="w">    </span><span class="n">receive_ts</span><span class="w"> </span><span class="nb">TEXT</span><span class="w">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="w"></span><span class="p">);</span></span></span></code></pre></div><p><code>source_sdk</code> 獨立成 column 讓「按 SDK 來源篩選」（<code>WHERE source_sdk = 'python'</code>）不需要從 JSON extract。<code>data</code> 用 TEXT 存 JSON。SQLite 沒有原生 JSON 型別，但 3.38+ 支援 <code>json_extract()</code> 函式做查詢（<code>WHERE json_extract(data, '$.duration_ms') &gt; 1000</code>）。<code>session_id</code> 獨立成 column 讓 session 回放的 JOIN 不需要 JSON extract。<code>error_stack</code> 獨立成 column 讓 error 調查時全文搜尋 stack trace 不需要 JSON extract。<code>receive_ts</code> 是 collector 收到事件的時間，和 SDK 端的 <code>ts</code> 對照可估算 clock drift。</p>
<p>PostgreSQL 版本的差異：<code>data</code> 改成 <code>JSONB</code> 型別（原生索引和查詢）、<code>source_*</code> 可保持為 nested JSON（PostgreSQL 的 JSONB 查詢效能足夠）或維持攤平（和 SQLite 版本保持一致）。</p>
<h3 id="建議索引">建議索引</h3>
<p>建表時一起建索引，覆蓋 dashboard 的核心查詢模式：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">CREATE</span><span class="w"> </span><span class="k">INDEX</span><span class="w"> </span><span class="n">idx_type_ts</span><span class="w"> </span><span class="k">ON</span><span class="w"> </span><span class="n">events</span><span class="p">(</span><span class="k">type</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="p">);</span><span class="w">    </span><span class="c1">-- 按 type + 時間過濾（error 列表、趨勢圖）
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"></span><span class="k">CREATE</span><span class="w"> </span><span class="k">INDEX</span><span class="w"> </span><span class="n">idx_session</span><span class="w"> </span><span class="k">ON</span><span class="w"> </span><span class="n">events</span><span class="p">(</span><span class="n">session_id</span><span class="p">);</span><span class="w">   </span><span class="c1">-- 按 session 回放
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="c1"></span><span class="k">CREATE</span><span class="w"> </span><span class="k">INDEX</span><span class="w"> </span><span class="n">idx_name</span><span class="w"> </span><span class="k">ON</span><span class="w"> </span><span class="n">events</span><span class="p">(</span><span class="n">name</span><span class="p">);</span><span class="w">            </span><span class="c1">-- 按 name 分群計數（功能使用排行）</span></span></span></code></pre></div><p>Day-one 建表時就建，不是效能出問題後才加。</p>
<h3 id="適用規模">適用規模</h3>
<p>單日事件量在十萬筆以下、SQLite 資料庫在 1GB 以下。索引查詢在毫秒級完成。自用工具和小型團隊的日常使用通常在這個範圍。</p>
<h3 id="分層保留與降採樣">分層保留與降採樣</h3>
<p>保留策略從查詢需求反推，每一種查詢需要的資料粒度和回溯深度不同。回溯越深的查詢需要的粒度越粗 — debug 需要最近幾天的逐筆事件，cohort 留存需要一整年的資料但每週一筆聚合數字就夠。</p>
<table>
  <thead>
      <tr>
          <th>查詢用途</th>
          <th>需要的粒度</th>
          <th>回溯深度</th>
          <th>對應表</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Debug 定位</td>
          <td>逐筆原始</td>
          <td>天</td>
          <td>events</td>
      </tr>
      <tr>
          <td>Funnel</td>
          <td>逐筆 event</td>
          <td>週～月</td>
          <td>events</td>
      </tr>
      <tr>
          <td>Error 趨勢</td>
          <td>每小時計數</td>
          <td>月～季</td>
          <td>hourly_summary</td>
      </tr>
      <tr>
          <td>Cohort</td>
          <td>每天計數</td>
          <td>季～年</td>
          <td>daily_summary</td>
      </tr>
      <tr>
          <td>RFM 分群</td>
          <td>每月聚合</td>
          <td>年</td>
          <td>monthly_summary</td>
      </tr>
  </tbody>
</table>
<p>SQLite 中的實作是三張摘要表加定期 job：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1">-- 摘要表
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="c1"></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">hourly_summary</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">    </span><span class="n">hour</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">    </span><span class="k">count</span><span class="w"> </span><span class="nb">INTEGER</span><span class="p">,</span><span class="w"> </span><span class="n">error_count</span><span class="w"> </span><span class="nb">INTEGER</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">    </span><span class="k">UNIQUE</span><span class="p">(</span><span class="n">hour</span><span class="p">,</span><span class="w"> </span><span class="k">type</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w"></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">daily_summary</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w">    </span><span class="nb">date</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w">    </span><span class="k">count</span><span class="w"> </span><span class="nb">INTEGER</span><span class="p">,</span><span class="w"> </span><span class="n">unique_sessions</span><span class="w"> </span><span class="nb">INTEGER</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w">    </span><span class="k">UNIQUE</span><span class="p">(</span><span class="nb">date</span><span class="p">,</span><span class="w"> </span><span class="k">type</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w"></span><span class="c1">-- 降採樣（Downsample，每小時跑一次，幂等 — 重跑只更新不重複）
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="c1"></span><span class="k">INSERT</span><span class="w"> </span><span class="k">OR</span><span class="w"> </span><span class="k">REPLACE</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">hourly_summary</span><span class="w"> </span><span class="p">(</span><span class="n">hour</span><span class="p">,</span><span class="w"> </span><span class="k">type</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="p">,</span><span class="w"> </span><span class="k">count</span><span class="p">,</span><span class="w"> </span><span class="n">error_count</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="n">strftime</span><span class="p">(</span><span class="s1">&#39;%Y-%m-%dT%H:00:00&#39;</span><span class="p">,</span><span class="w"> </span><span class="n">ts</span><span class="p">),</span><span class="w"> </span><span class="k">type</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w">       </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">),</span><span class="w"> </span><span class="k">SUM</span><span class="p">(</span><span class="k">CASE</span><span class="w"> </span><span class="k">WHEN</span><span class="w"> </span><span class="k">type</span><span class="o">=</span><span class="s1">&#39;error&#39;</span><span class="w"> </span><span class="k">THEN</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="k">ELSE</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="k">END</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="w"></span><span class="k">WHERE</span><span class="w"> </span><span class="n">ts</span><span class="w"> </span><span class="o">&gt;=</span><span class="w"> </span><span class="n">datetime</span><span class="p">(</span><span class="s1">&#39;now&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;-1 hour&#39;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="w"></span><span class="k">GROUP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="w"></span><span class="c1">-- 清理（Purge，每天跑一次，分批刪除避免長時間鎖定）
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="c1"></span><span class="k">DELETE</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w"> </span><span class="k">WHERE</span><span class="w"> </span><span class="n">rowid</span><span class="w"> </span><span class="k">IN</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="w">  </span><span class="k">SELECT</span><span class="w"> </span><span class="n">rowid</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w"> </span><span class="k">WHERE</span><span class="w"> </span><span class="n">ts</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">datetime</span><span class="p">(</span><span class="s1">&#39;now&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;-7 days&#39;</span><span class="p">)</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="mi">10000</span><span class="w">
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="w"></span><span class="c1">-- 重複執行直到影響行數為 0
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="c1"></span><span class="k">DELETE</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">hourly_summary</span><span class="w"> </span><span class="k">WHERE</span><span class="w"> </span><span class="n">hour</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">datetime</span><span class="p">(</span><span class="s1">&#39;now&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;-90 days&#39;</span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="w"></span><span class="k">DELETE</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">daily_summary</span><span class="w"> </span><span class="k">WHERE</span><span class="w"> </span><span class="nb">date</span><span class="w"> </span><span class="o">&lt;</span><span class="w"> </span><span class="n">datetime</span><span class="p">(</span><span class="s1">&#39;now&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;-365 days&#39;</span><span class="p">);</span></span></span></code></pre></div><p>保留期限由 collector config 設定，數字的來源是「哪些查詢需要回溯多遠」：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="ln">1</span><span class="cl"><span class="nt">retention</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w">  </span><span class="nt">raw_events</span><span class="p">:</span><span class="w"> </span><span class="l">7d</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w">  </span><span class="nt">hourly_summary</span><span class="p">:</span><span class="w"> </span><span class="l">90d</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w">  </span><span class="nt">daily_summary</span><span class="p">:</span><span class="w"> </span><span class="l">365d</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w">  </span><span class="nt">monthly_summary</span><span class="p">:</span><span class="w"> </span><span class="l">forever</span></span></span></code></pre></div><p>Storage interface 的 <code>Downsample()</code> 和 <code>Purge()</code> 由 collector 的定時排程觸發（Go 的 <code>time.Ticker</code>）。每個 storage backend 各自實作 — SQLite 用上述 SQL、PostgreSQL 用相同邏輯但可以加 partial index 加速、時間序列 DB 的 continuous aggregate 和 retention policy 原生支援。</p>
<h3 id="為什麼是聚合而非抽樣">為什麼是聚合而非抽樣</h3>
<p>原始事件的保留期到期後，需要決定如何保留歷史統計。降採樣有兩種思路。<strong>抽樣保留</strong>是同事件名稱（<code>name</code> 欄位）同小時保留一筆原始事件、刪除其餘，保留了逐筆查詢能力但喪失準確計數。<strong>聚合摘要</strong>是把一小時內的事件壓成一筆計數記錄，喪失逐筆細節但保留準確統計。</p>
<p>Collector 選擇聚合摘要——捨棄逐筆細節，換取準確計數。降採樣後的資料用途是趨勢圖和長期統計，這些查詢需要「過去 30 天每小時的 error 總數」而非「某一筆原始 error 的 stack trace」。</p>
<p>這意味著原始事件 purge（定期清理過期事件）後，超過保留期的逐筆查詢會回傳空結果。Dashboard 在回溯超過原始事件保留期的時間範圍時，應切換到上方的摘要表（<code>hourly_summary</code>/<code>daily_summary</code>）查詢——顯示趨勢圖而非事件列表。設計方向是查詢 API 的 <code>from</code> 參數超過 <code>retention.raw_events</code> 時自動降級到摘要表，或回傳提示告知 client 該時間範圍只有聚合資料（初版 collector 尚未實作此降級邏輯）。</p>
<h3 id="觸發切換到-postgresql-的訊號">觸發切換到 PostgreSQL 的訊號</h3>
<p><strong>寫入爭搶</strong>：SQLite 是單寫者模型。高併發寫入（多個 SDK 同時 flush、每秒數百筆以上持續發生）會出現 <code>database is locked</code> 錯誤。WAL mode 能緩解但不能根治。</p>
<p><strong>聚合查詢效能不足</strong>：Dashboard 需要的聚合查詢（「過去 30 天每小時的 error 數量趨勢」「funnel 的每步轉換率」）在資料量成長後變慢。SQLite 沒有 parallel query 和 partial index 等進階 OLAP 能力。</p>
<p><strong>跨實例需求</strong>：需要多個 collector 實例共用同一個資料庫時，SQLite 的單檔案模型無法跨主機存取。</p>
<h2 id="postgresql-backend分析觸發">PostgreSQL Backend（分析觸發）</h2>
<p>PostgreSQL 是獨立的資料庫 server，提供多連線並行寫入、進階索引（GIN for JSONB、partial index）和完整的 SQL 分析能力。切換到 PostgreSQL 意味著 collector 從「零依賴單一 binary」變成「binary + 外部 DB」，運維複雜度上升。</p>
<h3 id="觸發條件">觸發條件</h3>
<p>SQLite 的寫入爭搶或聚合效能成為瓶頸時切換。具體訊號：<code>database is locked</code> 錯誤頻率超過每分鐘一次、或 dashboard 的聚合查詢超過 3 秒。</p>
<h3 id="切換方式">切換方式</h3>
<p>切換是 config change：把 <code>--storage=sqlite</code> 改成 <code>--storage=postgres --dsn=postgres://...</code>。資料遷移用匯出 + 匯入完成：</p>
<ol>
<li>從 SQLite 匯出事件為 JSONL（<code>monitor export --format=jsonl</code>）</li>
<li>在 PostgreSQL 建立 events 表（schema 和 SQLite 相同，data 欄位改用 JSONB）</li>
<li>匯入 JSONL 到 PostgreSQL（<code>monitor import --storage=postgres --file=events.jsonl</code>）</li>
<li>切換啟動參數、確認查詢正常後停用 SQLite 檔案</li>
</ol>
<p>Storage interface 保證 collector 的 ingestion、query、rule engine 邏輯不需要改動 — 只有 storage implementation 層切換。</p>
<h3 id="能力增量">能力增量</h3>
<ul>
<li><strong>並行寫入</strong>：多個 SDK 同時 flush 不會 lock</li>
<li><strong>JSONB 索引</strong>：對 data 欄位的特定 key 建索引（<code>CREATE INDEX ON events ((data-&gt;&gt;'name'))</code>）</li>
<li><strong>Window function</strong>：funnel 和 cohort 分析的 SQL 基礎</li>
<li><strong>Read replica</strong>：寫入和查詢分離，dashboard 的查詢不影響 ingestion 效能</li>
</ul>
<h2 id="時間序列-db-backend長期演進">時間序列 DB Backend（長期演進）</h2>
<p>時間序列資料庫（TimescaleDB、InfluxDB、VictoriaMetrics）專門為高頻 append 寫入和時間分桶聚合設計。TimescaleDB 基於 PostgreSQL 擴展，Storage interface 的 PostgreSQL implementation 可以直接複用、加上 hypertable 和 continuous aggregate。</p>
<h3 id="觸發條件-1">觸發條件</h3>
<p>每秒數萬筆以上的持續寫入、或需要自動 downsampling（每分鐘的原始資料保留 7 天、每小時的聚合保留 90 天、每天的聚合永久保留）。多數自用工具和小型團隊不會到達這個規模。</p>
<h3 id="能力增量-1">能力增量</h3>
<ul>
<li><strong>時間分桶原生操作</strong>：<code>time_bucket('1 hour', ts)</code> 替代手動 DATE_TRUNC</li>
<li><strong>Continuous aggregate</strong>：預計算的聚合結果自動更新</li>
<li><strong>壓縮</strong>：歷史資料自動壓縮，TB 級資料可查詢</li>
<li><strong>Retention policy</strong>：按時間自動清理舊資料</li>
</ul>
<h2 id="jsonl-匯出debug-用途">JSONL 匯出（debug 用途）</h2>
<p>JSONL 不作為主要 storage backend，而是作為匯出格式保留人類可讀性和 grep 友好性。<code>monitor export --format=jsonl</code> 把 storage 中的事件匯出為每行一個 JSON 物件的檔案，讓開發者可以用 grep / jq 做臨時查詢或把資料搬到其他工具。</p>
<p>JSONL 匯出也是備份和遷移的中介格式 — SQLite 損壞時從 JSONL 重建、切換到 PostgreSQL 時從 JSONL 匯入。</p>
<p>匯出使用 streaming — 從 storage 逐筆讀取、逐行寫出檔案，記憶體使用和事件總量無關。300 萬筆事件（約 900MB JSONL）的匯出不需要載入全部資料到記憶體。匯出的 JSONL 檔案包含事件明文（已 redaction 的欄位除外），匯出後不受 collector 的存取控制保護，應注意存放位置和存取權限。</p>
<h2 id="演進原則">演進原則</h2>
<p><strong>按觀察到的瓶頸切換</strong>。<code>database is locked</code> 錯誤頻率、聚合查詢延遲、磁碟使用量 — 這些是可觀察的訊號。「未來可能有百萬筆事件」是預測。按訊號行動，不按預測行動。</p>
<p><strong>切換是 config change</strong>。Storage interface 確保切換 backend 時 collector 的其他邏輯（ingestion、query API、rule engine、dashboard）不需要改動。切換的成本是資料遷移，不是程式碼重寫。</p>
<p><strong>SQLite 是安全的起點</strong>。多數開源使用者會停留在 SQLite backend — 單日萬筆以下、索引查詢毫秒級、零依賴部署。只有明確的效能瓶頸才值得引入外部 DB 的運維成本。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>Collector 的整體架構 → <a href="/blog/monitoring/04-collector/architecture/" data-link-title="Collector 架構" data-link-desc="HTTP endpoint → JSON Schema 驗證 → 儲存 → 查詢 → rule engine 的五段式處理鏈路">Collector 架構</a></li>
<li>查詢 API 的設計（跨 backend 統一） → <a href="/blog/monitoring/04-collector/query-api/" data-link-title="查詢 API 設計" data-link-desc="CLI grep 友好的 JSONL 結構 &#43; HTTP 查詢 endpoint — 兩種查詢介面各自的適用場景和設計要點">查詢 API 設計</a></li>
<li>資料庫選型的通用指南 → <a href="/blog/backend/01-database/" data-link-title="模組一：資料庫與持久化" data-link-desc="整理 SQL、transaction、migration 與 repository adapter 的後端實務">backend 01 資料庫</a></li>
<li>效能瓶頸的判讀方法 → <a href="/blog/backend/09-performance-capacity/" data-link-title="模組九：效能工程與容量規劃" data-link-desc="把『目前配置能撐多少、要加多少機器』變成可量化、可驗證、可改進的工程流程">backend 09 效能容量</a></li>
<li>水平擴展的基礎概念 → <a href="/blog/devops/02-horizontal-scaling/" data-link-title="模組二：水平擴展" data-link-desc="一個實例不夠時怎麼加第二個 — stateless 設計、shared storage、session 處理的工程約束">DevOps 水平擴展</a></li>
<li>Error fingerprint 的 DDL 擴充 → <a href="/blog/monitoring/04-collector/error-fingerprint/" data-link-title="Error Fingerprint 與去重分群" data-link-desc="把大量 error 事件歸組成可管理的 issue 列表 — fingerprint 演算法、message normalization、error_groups 表設計、自架方案的務實邊界">Error Fingerprint 與去重分群</a></li>
</ul>
]]></content:encoded></item><item><title>功能分層與 Backend 選擇</title><link>https://tarrragon.github.io/blog/monitoring/04-collector/feature-tier-boundary/</link><pubDate>Sat, 20 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/04-collector/feature-tier-boundary/</guid><description>&lt;p>Collector 的可插拔 Storage Backend 分成兩個功能層級。分界線是查詢模式 — SQLite 能高效處理的查詢定義了簡單版的功能邊界，超出的查詢需求觸發 PostgreSQL 的引入。所有事件都經過同一個 Ingestion domain，差異在 Query 和 Dashboard domain 能提供什麼能力。&lt;/p>
&lt;h2 id="sqlite-層開發者工具">SQLite 層：開發者工具&lt;/h2>
&lt;p>SQLite 層提供的功能聚焦在「開發者自己 debug 和監控」。所有查詢都是單一維度的 — 按時間、按類型、按名稱過濾，不需要跨事件 JOIN 或跨使用者聚合。&lt;/p>
&lt;h3 id="承載的功能">承載的功能&lt;/h3>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>功能&lt;/th>
 &lt;th>查詢模式&lt;/th>
 &lt;th>SQL 範例&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>最近 error 列表&lt;/td>
 &lt;td>按 type + 時間過濾&lt;/td>
 &lt;td>&lt;code>WHERE type='error' ORDER BY ts DESC LIMIT 20&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Error 計數（按 name 分群）&lt;/td>
 &lt;td>單表 GROUP BY&lt;/td>
 &lt;td>&lt;code>SELECT name, COUNT(*) FROM events WHERE type='error' GROUP BY name&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>單次 session 回放&lt;/td>
 &lt;td>按 session_id 過濾&lt;/td>
 &lt;td>&lt;code>WHERE session_id='xxx' ORDER BY ts&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>事件時間軸&lt;/td>
 &lt;td>按時間排序&lt;/td>
 &lt;td>&lt;code>WHERE ts BETWEEN ? AND ? ORDER BY ts&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>基本 rule engine&lt;/td>
 &lt;td>逐筆事件評估&lt;/td>
 &lt;td>收到事件時逐條比對 rule（不需要查歷史）&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>CLI 查詢&lt;/td>
 &lt;td>任意過濾&lt;/td>
 &lt;td>&lt;code>WHERE type=? AND name LIKE ? AND ts &amp;gt; ?&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這些功能覆蓋開發者日常 debug 和監控的核心操作 — 查錯誤、看時間軸、回放 session、設規則告警。&lt;/p>
&lt;h3 id="對應的-dashboard-視圖">對應的 Dashboard 視圖&lt;/h3>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>視圖&lt;/th>
 &lt;th>顯示&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>總覽頁&lt;/td>
 &lt;td>最近 1 小時的事件計數（按 type 分）+ 最近 error 列表&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>事件詳情&lt;/td>
 &lt;td>單筆事件的完整 JSON&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Session 回放&lt;/td>
 &lt;td>單次 session 內的事件序列&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h3 id="對應的事件消費">對應的事件消費&lt;/h3>
&lt;p>SQLite 層消費所有四類事件，但消費方式是「單筆或單 session 級查詢」：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>事件類型&lt;/th>
 &lt;th>消費方式&lt;/th>
 &lt;th>保留需求&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>event&lt;/td>
 &lt;td>按名稱計數、按 session 排列&lt;/td>
 &lt;td>原始 7 天（debug）&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>error&lt;/td>
 &lt;td>按名稱分群、按時間排列、看 stack trace&lt;/td>
 &lt;td>原始 30 天（error 追蹤價值較長）&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>metric&lt;/td>
 &lt;td>按名稱查最近 N 筆的值&lt;/td>
 &lt;td>原始 7 天 + 每小時聚合 90 天&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>lifecycle&lt;/td>
 &lt;td>按 session 排列、看狀態轉換&lt;/td>
 &lt;td>原始 7 天&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h2 id="postgresql-層行為分析">PostgreSQL 層：行為分析&lt;/h2>
&lt;p>PostgreSQL 層在 SQLite 層的基礎上加入「跨 session、跨使用者的聚合分析」。這些查詢需要 JOIN 多張表、計算時間窗口、處理大量資料的 GROUP BY — SQLite 的單寫者模型和有限的查詢最佳化器在這些場景下效能不足。&lt;/p>
&lt;h3 id="觸發引入-postgresql-的功能需求">觸發引入 PostgreSQL 的功能需求&lt;/h3>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>功能需求&lt;/th>
 &lt;th>為什麼 SQLite 不夠&lt;/th>
 &lt;th>PostgreSQL 提供什麼&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;strong>Funnel 分析&lt;/strong>&lt;/td>
 &lt;td>跨大量 session 的 multi-step JOIN 和聚合效能不足&lt;/td>
 &lt;td>Window functions + 高效 JOIN&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;strong>Cohort 留存&lt;/strong>&lt;/td>
 &lt;td>需要按「註冊週」分群、計算每週的回訪率&lt;/td>
 &lt;td>Date functions + 大規模 GROUP BY&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;strong>RFM 分群&lt;/strong>&lt;/td>
 &lt;td>需要跨所有使用者計算 recency/frequency/monetary&lt;/td>
 &lt;td>全表聚合 + 分位數計算&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;strong>時間趨勢 dashboard&lt;/strong>&lt;/td>
 &lt;td>需要「過去 30 天每小時的 error P95」&lt;/td>
 &lt;td>時間分桶 + percentile 函數&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;strong>高併發寫入&lt;/strong>&lt;/td>
 &lt;td>多個 SDK 同時 flush 且持續出現 database is locked&lt;/td>
 &lt;td>連線池 + 並行寫入&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;strong>長期保留 + 聚合&lt;/strong>&lt;/td>
 &lt;td>降採樣的 materialized view&lt;/td>
 &lt;td>REFRESH MATERIALIZED VIEW&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h3 id="判斷公式">判斷公式&lt;/h3>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-text" data-lang="text">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">需要 funnel / cohort / RFM 任一 → PostgreSQL
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">需要跨使用者聚合（不只看自己的資料） → PostgreSQL
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">需要高併發寫入（多個 SDK 同時 flush 且持續出現 database is locked 錯誤） → PostgreSQL
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">以上都不需要 → SQLite 足夠&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h3 id="對應的-dashboard-視圖sqlite-層不提供">對應的 Dashboard 視圖（SQLite 層不提供）&lt;/h3>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>視圖&lt;/th>
 &lt;th>查詢模式&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Funnel 漏斗&lt;/td>
 &lt;td>多步驟轉換率（session 級 JOIN）&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Cohort 留存表&lt;/td>
 &lt;td>時間窗口 × 群組矩陣&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>RFM 分群散佈&lt;/td>
 &lt;td>三維度分位數計算&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Error 趨勢圖（長期）&lt;/td>
 &lt;td>30 天 × 每小時的時間序列&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>效能 P95 趨勢&lt;/td>
 &lt;td>percentile_cont 視窗函數&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h3 id="對應的事件消費-1">對應的事件消費&lt;/h3>
&lt;p>PostgreSQL 層消費的事件和 SQLite 相同（Ingestion 不變），但消費方式從「單筆/單 session」擴展到「跨 session/跨使用者」：&lt;/p></description><content:encoded><![CDATA[<p>Collector 的可插拔 Storage Backend 分成兩個功能層級。分界線是查詢模式 — SQLite 能高效處理的查詢定義了簡單版的功能邊界，超出的查詢需求觸發 PostgreSQL 的引入。所有事件都經過同一個 Ingestion domain，差異在 Query 和 Dashboard domain 能提供什麼能力。</p>
<h2 id="sqlite-層開發者工具">SQLite 層：開發者工具</h2>
<p>SQLite 層提供的功能聚焦在「開發者自己 debug 和監控」。所有查詢都是單一維度的 — 按時間、按類型、按名稱過濾，不需要跨事件 JOIN 或跨使用者聚合。</p>
<h3 id="承載的功能">承載的功能</h3>
<table>
  <thead>
      <tr>
          <th>功能</th>
          <th>查詢模式</th>
          <th>SQL 範例</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>最近 error 列表</td>
          <td>按 type + 時間過濾</td>
          <td><code>WHERE type='error' ORDER BY ts DESC LIMIT 20</code></td>
      </tr>
      <tr>
          <td>Error 計數（按 name 分群）</td>
          <td>單表 GROUP BY</td>
          <td><code>SELECT name, COUNT(*) FROM events WHERE type='error' GROUP BY name</code></td>
      </tr>
      <tr>
          <td>單次 session 回放</td>
          <td>按 session_id 過濾</td>
          <td><code>WHERE session_id='xxx' ORDER BY ts</code></td>
      </tr>
      <tr>
          <td>事件時間軸</td>
          <td>按時間排序</td>
          <td><code>WHERE ts BETWEEN ? AND ? ORDER BY ts</code></td>
      </tr>
      <tr>
          <td>基本 rule engine</td>
          <td>逐筆事件評估</td>
          <td>收到事件時逐條比對 rule（不需要查歷史）</td>
      </tr>
      <tr>
          <td>CLI 查詢</td>
          <td>任意過濾</td>
          <td><code>WHERE type=? AND name LIKE ? AND ts &gt; ?</code></td>
      </tr>
  </tbody>
</table>
<p>這些功能覆蓋開發者日常 debug 和監控的核心操作 — 查錯誤、看時間軸、回放 session、設規則告警。</p>
<h3 id="對應的-dashboard-視圖">對應的 Dashboard 視圖</h3>
<table>
  <thead>
      <tr>
          <th>視圖</th>
          <th>顯示</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>總覽頁</td>
          <td>最近 1 小時的事件計數（按 type 分）+ 最近 error 列表</td>
      </tr>
      <tr>
          <td>事件詳情</td>
          <td>單筆事件的完整 JSON</td>
      </tr>
      <tr>
          <td>Session 回放</td>
          <td>單次 session 內的事件序列</td>
      </tr>
  </tbody>
</table>
<h3 id="對應的事件消費">對應的事件消費</h3>
<p>SQLite 層消費所有四類事件，但消費方式是「單筆或單 session 級查詢」：</p>
<table>
  <thead>
      <tr>
          <th>事件類型</th>
          <th>消費方式</th>
          <th>保留需求</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>event</td>
          <td>按名稱計數、按 session 排列</td>
          <td>原始 7 天（debug）</td>
      </tr>
      <tr>
          <td>error</td>
          <td>按名稱分群、按時間排列、看 stack trace</td>
          <td>原始 30 天（error 追蹤價值較長）</td>
      </tr>
      <tr>
          <td>metric</td>
          <td>按名稱查最近 N 筆的值</td>
          <td>原始 7 天 + 每小時聚合 90 天</td>
      </tr>
      <tr>
          <td>lifecycle</td>
          <td>按 session 排列、看狀態轉換</td>
          <td>原始 7 天</td>
      </tr>
  </tbody>
</table>
<h2 id="postgresql-層行為分析">PostgreSQL 層：行為分析</h2>
<p>PostgreSQL 層在 SQLite 層的基礎上加入「跨 session、跨使用者的聚合分析」。這些查詢需要 JOIN 多張表、計算時間窗口、處理大量資料的 GROUP BY — SQLite 的單寫者模型和有限的查詢最佳化器在這些場景下效能不足。</p>
<h3 id="觸發引入-postgresql-的功能需求">觸發引入 PostgreSQL 的功能需求</h3>
<table>
  <thead>
      <tr>
          <th>功能需求</th>
          <th>為什麼 SQLite 不夠</th>
          <th>PostgreSQL 提供什麼</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><strong>Funnel 分析</strong></td>
          <td>跨大量 session 的 multi-step JOIN 和聚合效能不足</td>
          <td>Window functions + 高效 JOIN</td>
      </tr>
      <tr>
          <td><strong>Cohort 留存</strong></td>
          <td>需要按「註冊週」分群、計算每週的回訪率</td>
          <td>Date functions + 大規模 GROUP BY</td>
      </tr>
      <tr>
          <td><strong>RFM 分群</strong></td>
          <td>需要跨所有使用者計算 recency/frequency/monetary</td>
          <td>全表聚合 + 分位數計算</td>
      </tr>
      <tr>
          <td><strong>時間趨勢 dashboard</strong></td>
          <td>需要「過去 30 天每小時的 error P95」</td>
          <td>時間分桶 + percentile 函數</td>
      </tr>
      <tr>
          <td><strong>高併發寫入</strong></td>
          <td>多個 SDK 同時 flush 且持續出現 database is locked</td>
          <td>連線池 + 並行寫入</td>
      </tr>
      <tr>
          <td><strong>長期保留 + 聚合</strong></td>
          <td>降採樣的 materialized view</td>
          <td>REFRESH MATERIALIZED VIEW</td>
      </tr>
  </tbody>
</table>
<h3 id="判斷公式">判斷公式</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">需要 funnel / cohort / RFM 任一 → PostgreSQL
</span></span><span class="line"><span class="ln">2</span><span class="cl">需要跨使用者聚合（不只看自己的資料） → PostgreSQL
</span></span><span class="line"><span class="ln">3</span><span class="cl">需要高併發寫入（多個 SDK 同時 flush 且持續出現 database is locked 錯誤） → PostgreSQL
</span></span><span class="line"><span class="ln">4</span><span class="cl">以上都不需要 → SQLite 足夠</span></span></code></pre></div><h3 id="對應的-dashboard-視圖sqlite-層不提供">對應的 Dashboard 視圖（SQLite 層不提供）</h3>
<table>
  <thead>
      <tr>
          <th>視圖</th>
          <th>查詢模式</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Funnel 漏斗</td>
          <td>多步驟轉換率（session 級 JOIN）</td>
      </tr>
      <tr>
          <td>Cohort 留存表</td>
          <td>時間窗口 × 群組矩陣</td>
      </tr>
      <tr>
          <td>RFM 分群散佈</td>
          <td>三維度分位數計算</td>
      </tr>
      <tr>
          <td>Error 趨勢圖（長期）</td>
          <td>30 天 × 每小時的時間序列</td>
      </tr>
      <tr>
          <td>效能 P95 趨勢</td>
          <td>percentile_cont 視窗函數</td>
      </tr>
  </tbody>
</table>
<h3 id="對應的事件消費-1">對應的事件消費</h3>
<p>PostgreSQL 層消費的事件和 SQLite 相同（Ingestion 不變），但消費方式從「單筆/單 session」擴展到「跨 session/跨使用者」：</p>
<table>
  <thead>
      <tr>
          <th>事件類型</th>
          <th>SQLite 層消費</th>
          <th>PostgreSQL 層新增消費</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>event</td>
          <td>按名稱計數</td>
          <td>funnel 步驟轉換、cohort 行為分群</td>
      </tr>
      <tr>
          <td>error</td>
          <td>按名稱分群</td>
          <td>跨版本 error 率比較、P95 回應時間趨勢</td>
      </tr>
      <tr>
          <td>metric</td>
          <td>最近 N 筆值</td>
          <td>長期趨勢（materialized view 預聚合）</td>
      </tr>
      <tr>
          <td>lifecycle</td>
          <td>單 session 排列</td>
          <td>session 長度分佈、留存率計算</td>
      </tr>
  </tbody>
</table>
<h2 id="domain-的分層影響">Domain 的分層影響</h2>
<table>
  <thead>
      <tr>
          <th>Domain</th>
          <th>SQLite 層</th>
          <th>PostgreSQL 層新增</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><strong>Ingestion</strong></td>
          <td>HTTP POST → 驗證 → 寫入</td>
          <td>不變（寫入目標換 backend）</td>
      </tr>
      <tr>
          <td><strong>Storage</strong></td>
          <td>SQLite embedded</td>
          <td>PostgreSQL + 連線池</td>
      </tr>
      <tr>
          <td><strong>Query</strong></td>
          <td>單表過濾 + 單表 GROUP BY</td>
          <td>JOIN + window function + percentile</td>
      </tr>
      <tr>
          <td><strong>Rule</strong></td>
          <td>逐筆事件即時評估</td>
          <td>不變（rule 不依賴聚合查詢）</td>
      </tr>
      <tr>
          <td><strong>Dashboard</strong></td>
          <td>總覽 + 事件詳情 + session 回放</td>
          <td>新增 funnel / cohort / RFM / 趨勢圖</td>
      </tr>
  </tbody>
</table>
<p>Ingestion 和 Rule 兩個 domain 和 storage backend 無關 — 事件進來的方式和規則評估的邏輯不因 backend 改變。Query 和 Dashboard 是分層影響最大的兩個 domain — PostgreSQL 層的查詢能力決定了 Dashboard 能提供什麼視圖。</p>
<h2 id="實作邊界">實作邊界</h2>
<p>Storage interface 用 Go 的 interface composition 分成兩層：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="kd">type</span> <span class="nx">BasicStorage</span> <span class="kd">interface</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">    <span class="nf">Store</span><span class="p">(</span><span class="nx">event</span> <span class="nx">Event</span><span class="p">)</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">    <span class="nf">Query</span><span class="p">(</span><span class="nx">filter</span> <span class="nx">QueryFilter</span><span class="p">)</span> <span class="p">([]</span><span class="nx">Event</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">    <span class="nf">Close</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl">    <span class="nf">Downsample</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">    <span class="nf">Purge</span><span class="p">()</span> <span class="kt">error</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="kd">type</span> <span class="nx">AnalyticsStorage</span> <span class="kd">interface</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">    <span class="nx">BasicStorage</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">    <span class="nf">Aggregate</span><span class="p">(</span><span class="nx">spec</span> <span class="nx">AggregateSpec</span><span class="p">)</span> <span class="p">(</span><span class="nx">AggregateResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">    <span class="nf">Funnel</span><span class="p">(</span><span class="nx">steps</span> <span class="p">[]</span><span class="kt">string</span><span class="p">,</span> <span class="nx">timeWindow</span> <span class="nx">Duration</span><span class="p">)</span> <span class="p">(</span><span class="nx">FunnelResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">    <span class="nf">Cohort</span><span class="p">(</span><span class="nx">groupBy</span> <span class="kt">string</span><span class="p">,</span> <span class="nx">metric</span> <span class="kt">string</span><span class="p">)</span> <span class="p">(</span><span class="nx">CohortResult</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p>SQLite implementation 只實作 <code>BasicStorage</code>。PostgreSQL implementation 實作 <code>AnalyticsStorage</code>。Dashboard 用 Go 的 type assertion（<code>if as, ok := storage.(AnalyticsStorage); ok { ... }</code>）判斷能力 — funnel/cohort 視圖在 SQLite 模式下不顯示入口，而非顯示後報錯。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>可插拔 Storage Backend 的架構 → <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a></li>
<li>事件枚舉方法（哪些事件要收） → <a href="/blog/monitoring/01-mental-model/event-enumeration-method/" data-link-title="事件枚舉與補齊檢查" data-link-desc="從操作盤點系統性地推導出完整的事件清單 — 四類補齊檢查確保沒有遺漏、粒度判準確保每個事件只記一個事實">事件枚舉與補齊檢查</a></li>
<li>分層保留策略 → <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進的分層保留段</a></li>
<li>Funnel 分析的完整方法論 → <a href="/blog/monitoring/08-business-analytics/funnel-analysis/" data-link-title="Funnel Analysis" data-link-desc="使用者在哪一步流失 — 從事件序列計算每步轉換率、找出流失最嚴重的步驟、區分設計問題和技術問題">Funnel analysis</a></li>
<li>查詢消費模式（各場景需要什麼事件）→ <a href="/blog/monitoring/04-collector/query-consumption-patterns/" data-link-title="查詢消費模式" data-link-desc="Debug / Alerting / 產品決策 / 安全審計 / 效能監控 — 五種查詢場景各需要什麼事件、什麼欄位、什麼查詢模式">查詢消費模式</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/</link><pubDate>Wed, 13 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/</guid><description>&lt;p>SQLite 是世界上部署最多的 DB（手機、瀏覽器、car、IoT 都有）。傳統定位是 embedded、單檔案與低操作成本資料庫；multi-tenant 網路服務通常會先看 PostgreSQL、MySQL 或 managed SQL。但近年因 Cloudflare D1（serverless SQLite）、Turso（distributed SQLite）、Litestream（SQLite replication）等服務興起，出現「SQLite as production DB」的新場景。&lt;/p>
&lt;h2 id="教學路線單檔正式狀態與-local-first">教學路線：單檔正式狀態與 local-first&lt;/h2>
&lt;p>SQLite 服務頁的教學目標是把單機、單檔案、edge、desktop、test fixture 的正式狀態責任說清楚。讀者讀完後要能判斷 SQLite 何時是 production state，何時要轉向 server database、edge KV 或分散式 SQLite 變體。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>學習段&lt;/th>
 &lt;th>核心問題&lt;/th>
 &lt;th>對應段落&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Embedded state&lt;/td>
 &lt;td>單檔案資料庫如何成為 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">source of truth&lt;/a>&lt;/td>
 &lt;td>定位、適用場景&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Local-first&lt;/td>
 &lt;td>device、edge、desktop、test fixture 的責任形狀&lt;/td>
 &lt;td>適用場景、案例對照&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Writer boundary&lt;/td>
 &lt;td>single writer、file lock、WAL 如何決定服務上限&lt;/td>
 &lt;td>容量特性、容量規劃要點&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Distributed variants&lt;/td>
 &lt;td>Turso、LiteFS、rqlite、D1 解決哪類同步或 edge 問題&lt;/td>
 &lt;td>跟其他 vendor 的取捨、章節群結構&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>替代路由&lt;/td>
 &lt;td>何時升級 PostgreSQL、MySQL、DynamoDB 或 edge KV&lt;/td>
 &lt;td>不適用場景、下一步路由&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h2 id="定位單檔案-embedded--新興分散式-sqlite-生態">定位：單檔案 embedded + 新興分散式 SQLite 生態&lt;/h2>
&lt;p>SQLite 跟 PostgreSQL / MySQL 承擔不同層級的資料責任：&lt;/p>
&lt;ul>
&lt;li>以 function-call API 使用，省掉 server process&lt;/li>
&lt;li>單一檔案（含 schema、data、index、metadata）&lt;/li>
&lt;li>無 user / role / connection 概念&lt;/li>
&lt;li>同 process 同時 read / write 受 file lock 限制&lt;/li>
&lt;/ul>
&lt;p>傳統定位：test fixture、CLI tool data store、mobile app（iOS / Android 內建）、edge device。&lt;/p>
&lt;p>新興定位：edge serverless（Cloudflare D1）、distributed SQLite（Turso、rqlite）、replicated SQLite（Litestream）。&lt;/p>
&lt;h2 id="容量特性">容量特性&lt;/h2>
&lt;p>&lt;strong>單檔案上限&lt;/strong>：&lt;/p>
&lt;ul>
&lt;li>DB 最大 281 TB（理論）&lt;/li>
&lt;li>實務上單表 &amp;gt; 100 GB 開始有 vacuum / index 問題&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>並發寫&lt;/strong>：&lt;/p>
&lt;ul>
&lt;li>WAL mode：可同時多 reader + 1 writer&lt;/li>
&lt;li>寫入仍由 single writer boundary 控制&lt;/li>
&lt;li>寫吞吐受 disk fsync 限制（通常 &amp;lt; 1K WPS）&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>並發讀&lt;/strong>：&lt;/p>
&lt;ul>
&lt;li>WAL mode 多 reader 可同時跑&lt;/li>
&lt;li>read-only workload 可以撐高吞吐&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>Cross-process / cross-instance&lt;/strong>：&lt;/p>
&lt;ul>
&lt;li>多個 process / instance 同時寫同一檔案會破壞 single writer boundary&lt;/li>
&lt;li>需要分散時用 Litestream（replication）或 Turso（distributed）&lt;/li>
&lt;/ul>
&lt;h2 id="適用場景">適用場景&lt;/h2>
&lt;p>&lt;strong>1. Test fixture / CI 用 DB&lt;/strong>：&lt;/p>
&lt;ul>
&lt;li>整合測試需要的 fixed DB&lt;/li>
&lt;li>比 spin up PostgreSQL container 快&lt;/li>
&lt;li>對應 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/repository-adapter/" data-link-title="1.4 Repository Adapter 實作" data-link-desc="Port / Adapter 邊界、row mapping、error translation、ORM vs query builder 選型、contract test 設計">1.4 Repository Adapter&lt;/a> 的 contract test 模式&lt;/li>
&lt;/ul>
&lt;p>&lt;strong>2. CLI tool / desktop app 內建 store&lt;/strong>：&lt;/p></description><content:encoded><![CDATA[<p>SQLite 是世界上部署最多的 DB（手機、瀏覽器、car、IoT 都有）。傳統定位是 embedded、單檔案與低操作成本資料庫；multi-tenant 網路服務通常會先看 PostgreSQL、MySQL 或 managed SQL。但近年因 Cloudflare D1（serverless SQLite）、Turso（distributed SQLite）、Litestream（SQLite replication）等服務興起，出現「SQLite as production DB」的新場景。</p>
<h2 id="教學路線單檔正式狀態與-local-first">教學路線：單檔正式狀態與 local-first</h2>
<p>SQLite 服務頁的教學目標是把單機、單檔案、edge、desktop、test fixture 的正式狀態責任說清楚。讀者讀完後要能判斷 SQLite 何時是 production state，何時要轉向 server database、edge KV 或分散式 SQLite 變體。</p>
<table>
  <thead>
      <tr>
          <th>學習段</th>
          <th>核心問題</th>
          <th>對應段落</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Embedded state</td>
          <td>單檔案資料庫如何成為 <a href="/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">source of truth</a></td>
          <td>定位、適用場景</td>
      </tr>
      <tr>
          <td>Local-first</td>
          <td>device、edge、desktop、test fixture 的責任形狀</td>
          <td>適用場景、案例對照</td>
      </tr>
      <tr>
          <td>Writer boundary</td>
          <td>single writer、file lock、WAL 如何決定服務上限</td>
          <td>容量特性、容量規劃要點</td>
      </tr>
      <tr>
          <td>Distributed variants</td>
          <td>Turso、LiteFS、rqlite、D1 解決哪類同步或 edge 問題</td>
          <td>跟其他 vendor 的取捨、章節群結構</td>
      </tr>
      <tr>
          <td>替代路由</td>
          <td>何時升級 PostgreSQL、MySQL、DynamoDB 或 edge KV</td>
          <td>不適用場景、下一步路由</td>
      </tr>
  </tbody>
</table>
<h2 id="定位單檔案-embedded--新興分散式-sqlite-生態">定位：單檔案 embedded + 新興分散式 SQLite 生態</h2>
<p>SQLite 跟 PostgreSQL / MySQL 承擔不同層級的資料責任：</p>
<ul>
<li>以 function-call API 使用，省掉 server process</li>
<li>單一檔案（含 schema、data、index、metadata）</li>
<li>無 user / role / connection 概念</li>
<li>同 process 同時 read / write 受 file lock 限制</li>
</ul>
<p>傳統定位：test fixture、CLI tool data store、mobile app（iOS / Android 內建）、edge device。</p>
<p>新興定位：edge serverless（Cloudflare D1）、distributed SQLite（Turso、rqlite）、replicated SQLite（Litestream）。</p>
<h2 id="容量特性">容量特性</h2>
<p><strong>單檔案上限</strong>：</p>
<ul>
<li>DB 最大 281 TB（理論）</li>
<li>實務上單表 &gt; 100 GB 開始有 vacuum / index 問題</li>
</ul>
<p><strong>並發寫</strong>：</p>
<ul>
<li>WAL mode：可同時多 reader + 1 writer</li>
<li>寫入仍由 single writer boundary 控制</li>
<li>寫吞吐受 disk fsync 限制（通常 &lt; 1K WPS）</li>
</ul>
<p><strong>並發讀</strong>：</p>
<ul>
<li>WAL mode 多 reader 可同時跑</li>
<li>read-only workload 可以撐高吞吐</li>
</ul>
<p><strong>Cross-process / cross-instance</strong>：</p>
<ul>
<li>多個 process / instance 同時寫同一檔案會破壞 single writer boundary</li>
<li>需要分散時用 Litestream（replication）或 Turso（distributed）</li>
</ul>
<h2 id="適用場景">適用場景</h2>
<p><strong>1. Test fixture / CI 用 DB</strong>：</p>
<ul>
<li>整合測試需要的 fixed DB</li>
<li>比 spin up PostgreSQL container 快</li>
<li>對應 <a href="/blog/backend/01-database/repository-adapter/" data-link-title="1.4 Repository Adapter 實作" data-link-desc="Port / Adapter 邊界、row mapping、error translation、ORM vs query builder 選型、contract test 設計">1.4 Repository Adapter</a> 的 contract test 模式</li>
</ul>
<p><strong>2. CLI tool / desktop app 內建 store</strong>：</p>
<ul>
<li>Chrome / Firefox（cookies、history、bookmark）、Fossil SCM、iOS app</li>
<li>省掉 server、單檔案攜帶</li>
</ul>
<p><strong>3. Mobile app（iOS / Android）</strong>：</p>
<ul>
<li>iOS Core Data 底層用 SQLite</li>
<li>Android 自帶 SQLite API</li>
<li>offline-first app 的標準</li>
</ul>
<p><strong>4. Single-instance backend（特殊場景）</strong>：</p>
<ul>
<li>流量小 + HA 由備份 / restore / redeploy 流程承擔</li>
<li>例：Sidekick / 個人 SaaS / family-scale app</li>
<li>配合 Litestream 做 backup / DR</li>
</ul>
<p><strong>5. Edge / serverless（新興）</strong>：</p>
<ul>
<li>Cloudflare D1：edge SQLite、跟 Workers 整合</li>
<li>Turso：distributed SQLite、跨 region replication</li>
<li>跟傳統 SQLite 不同等級、是 <em>新的 product</em></li>
</ul>
<p><strong>6. Embedded device / IoT</strong>：</p>
<ul>
<li>沒網路或要降低 server 依賴</li>
<li>SQLite 內建、無 external dependency</li>
</ul>
<h2 id="不適用場景">不適用場景</h2>
<p><strong>1. 多 instance / 多 region web service</strong>：</p>
<ul>
<li>SQLite 的單檔模型以單 instance writer 為主要邊界</li>
<li>替代：PostgreSQL、Aurora、Spanner、CockroachDB</li>
</ul>
<p><strong>2. 高寫入吞吐（&gt; 1K WPS）</strong>：</p>
<ul>
<li>fsync 限制</li>
<li>替代：任何 server-based RDBMS</li>
</ul>
<p><strong>3. Multi-user 權限管理</strong>：</p>
<ul>
<li>無 user / role 概念</li>
<li>替代：PostgreSQL / MySQL</li>
</ul>
<p><strong>4. 跨機器 transaction</strong>：</p>
<ul>
<li>SQLite 是 single-machine</li>
<li>替代：分散式 SQL</li>
</ul>
<p><strong>5. 大規模 production OLTP</strong>：</p>
<ul>
<li>大規模 production OLTP 需要 server database 的 HA、replica、權限與操作邊界</li>
<li>替代：MySQL / PostgreSQL / Aurora</li>
</ul>
<h2 id="跟其他-vendor-的取捨">跟其他 vendor 的取捨</h2>
<p><strong>vs PostgreSQL（作為 test DB）</strong>：</p>
<ul>
<li>SQLite：快 spin up、SQL dialect 接近但有差異</li>
<li>PostgreSQL：跟 production 一致、發現的 bug 真實</li>
<li>選 SQLite：speed of iteration、簡單 query</li>
<li>選 PostgreSQL：catch production-like bug、PostgreSQL-specific 特性測試</li>
</ul>
<p><strong>vs Cloudflare D1</strong>：</p>
<ul>
<li>SQLite（local）：單機、自管</li>
<li>D1：edge serverless、跟 Workers 整合</li>
<li>選 SQLite：embedded / CLI / app 場景</li>
<li>選 D1：edge web service、跟 Cloudflare 生態整合</li>
</ul>
<p><strong>vs Turso（distributed SQLite）</strong>：</p>
<ul>
<li>SQLite：單機、單檔案</li>
<li>Turso：distributed、跨 region replication、SQLite-compatible</li>
<li>選 SQLite：simple use case</li>
<li>選 Turso：需要 SQLite simplicity + 全球分散</li>
</ul>
<p><strong>vs Litestream（replicated SQLite）</strong>：</p>
<ul>
<li>SQLite：單檔案</li>
<li>Litestream：把 SQLite 變成 streaming replicated 到 S3</li>
<li>選 Litestream：想要 SQLite simplicity + DR</li>
</ul>
<p><strong>vs Firebase / Firestore（mobile app）</strong>：</p>
<ul>
<li>SQLite：embedded、offline-first、無 sync</li>
<li>Firestore：realtime、自動 sync、雲端 store</li>
<li>選 SQLite：offline-first、單機</li>
<li>選 Firestore：multi-device sync、realtime</li>
</ul>
<h2 id="容量規劃要點">容量規劃要點</h2>
<p><strong>1. WAL mode 是 production baseline</strong>：</p>
<ul>
<li>default journal mode 是 rollback journal（每寫都 lock）</li>
<li>WAL（Write-Ahead Log）讓多 reader 可同時跑</li>
<li><code>PRAGMA journal_mode = WAL</code></li>
</ul>
<p><strong>2. fsync 配置</strong>：</p>
<ul>
<li><code>PRAGMA synchronous = FULL</code>（durable、慢）</li>
<li><code>PRAGMA synchronous = NORMAL</code>（faster、少數情況可能掉資料）</li>
<li><code>PRAGMA synchronous = OFF</code>（最快、不安全）</li>
</ul>
<p><strong>3. mmap 加速 read</strong>：</p>
<ul>
<li><code>PRAGMA mmap_size = 268435456</code>（256 MB）</li>
<li>把 DB 部分內容 mmap 進 RAM、加速 read</li>
</ul>
<p><strong>4. Cache size</strong>：</p>
<ul>
<li><code>PRAGMA cache_size = -64000</code>（64 MB cache）</li>
<li>大 cache 對 read-heavy workload 有幫助</li>
</ul>
<p><strong>5. Auto-vacuum</strong>：</p>
<ul>
<li>預設 off、delete 後檔案不縮小</li>
<li><code>PRAGMA auto_vacuum = INCREMENTAL</code> + 定期 <code>PRAGMA incremental_vacuum</code></li>
</ul>
<h2 id="章節群結構">章節群結構</h2>
<p>SQLite 章節群的責任是把單檔正式狀態、embedded process、writer boundary、backup / restore、test fixture、local-first 與 edge SQLite 變體拆成可教學路線。完整結構見 <a href="teaching-structure/">SQLite Teaching Structure</a>；下表列出目前已建立的 deep article、hands-on 與 migration route。</p>
<table>
  <thead>
      <tr>
          <th>層級</th>
          <th>文件</th>
          <th>狀態</th>
          <th>教學責任</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>結構總覽</td>
          <td><a href="teaching-structure/">Teaching Structure</a></td>
          <td>已有正文</td>
          <td>對齊 PG / MySQL 與 LLM 架構，固定 SQLite 後續讀法</td>
      </tr>
      <tr>
          <td>Core deep</td>
          <td><a href="file-lifecycle-backup-boundary/">File lifecycle / backup boundary</a></td>
          <td>已有正文</td>
          <td>WAL sidecar、backup API、restore drill、corruption route</td>
      </tr>
      <tr>
          <td>Hands-on</td>
          <td><a href="hands-on/">Hands-on 操作路線</a></td>
          <td>已有正文</td>
          <td>local file、backup restore、WAL busy、migration fixture</td>
      </tr>
      <tr>
          <td>Concurrency</td>
          <td><a href="wal-concurrency-locking/">WAL concurrency / locking</a></td>
          <td>已有正文</td>
          <td>single writer、file lock、<code>SQLITE_BUSY</code>、checkpoint</td>
      </tr>
      <tr>
          <td>Performance</td>
          <td><a href="pragma-tuning-performance/">PRAGMA tuning / performance</a></td>
          <td>已有正文</td>
          <td>journal、sync、cache、mmap、vacuum 的取捨</td>
      </tr>
      <tr>
          <td>Migration</td>
          <td><a href="schema-migration-versioning/">Schema migration / versioning</a></td>
          <td>已有正文</td>
          <td>app release、schema version、rollback、migration evidence</td>
      </tr>
      <tr>
          <td>Testing</td>
          <td><a href="test-fixture-best-practice/">Test fixture best practice</a></td>
          <td>已有正文</td>
          <td>SQLite 測試便利性與 production dialect gap</td>
      </tr>
      <tr>
          <td>Embedded app</td>
          <td><a href="mobile-desktop-embedded-store/">Mobile / desktop embedded store</a></td>
          <td>已有正文</td>
          <td>device local state、privacy、backup、app version</td>
      </tr>
      <tr>
          <td>Sync</td>
          <td><a href="local-first-sync-boundary/">Local-first sync boundary</a></td>
          <td>已有正文</td>
          <td>多裝置同步、conflict、server authority</td>
      </tr>
      <tr>
          <td>Edge variant</td>
          <td><a href="d1-turso-libsql-comparison/">D1 / Turso / libSQL comparison</a></td>
          <td>已有正文</td>
          <td>edge SQLite 產品與 local SQLite 的責任差異</td>
      </tr>
      <tr>
          <td>Replication</td>
          <td><a href="litestream-litefs-replication/">Litestream / LiteFS replication</a></td>
          <td>已有正文</td>
          <td>continuous backup、read replica、failover boundary</td>
      </tr>
      <tr>
          <td>SQL compatibility</td>
          <td><a href="sql-dialect-index-limits/">SQL dialect and index limits</a></td>
          <td>已有正文</td>
          <td>type affinity、index、constraint、PostgreSQL / MySQL gap</td>
      </tr>
      <tr>
          <td>Operations</td>
          <td><a href="observability-runbook/">Observability / runbook</a></td>
          <td>已有正文</td>
          <td>busy errors、WAL growth、backup evidence、incident route</td>
      </tr>
      <tr>
          <td>Migration route</td>
          <td><a href="migrate-to-postgresql/">SQLite to PostgreSQL</a></td>
          <td>已有正文</td>
          <td>多 tenant、權限、HA、audit 出現時的升級路線</td>
      </tr>
      <tr>
          <td>Migration route</td>
          <td><a href="migrate-to-d1-turso/">SQLite to D1 / Turso</a></td>
          <td>已有正文</td>
          <td>edge / serverless 化路線</td>
      </tr>
      <tr>
          <td>Migration route</td>
          <td><a href="migrate-from-postgresql-simplification/">PostgreSQL to SQLite simplification</a></td>
          <td>已有正文</td>
          <td>single-user / embedded 工具的反向簡化路線</td>
      </tr>
  </tbody>
</table>
<p>章節群的讀法是先讀 file lifecycle，再按壓力選 deep article。若問題是 write contention，讀 WAL locking；若問題是測試，讀 test fixture；若問題是 edge / serverless，讀 D1 / Turso comparison；若問題是服務長大，讀 SQLite to PostgreSQL migration。</p>
<h2 id="anti-recommendation-與升級路由">Anti-recommendation 與升級路由</h2>
<p>SQLite 的低操作成本容易讓團隊忽略它的 writer boundary。這一段先說何時維持 SQLite，再說何時升級到 server SQL、edge SQLite 變體或 managed KV。</p>
<table>
  <thead>
      <tr>
          <th>機制 / 路線</th>
          <th>維持簡單設計的條件</th>
          <th>升級訊號</th>
          <th>主要引用路徑</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Local SQLite</td>
          <td>單 process、單 writer、資料可用檔案備份保護</td>
          <td>多 instance 寫入、需要 HA、需要資料層權限</td>
          <td><a href="/blog/backend/knowledge-cards/database/" data-link-title="Database" data-link-desc="說明 database 在後端系統中如何承擔正式狀態、查詢與一致性責任">Database</a>、<a href="/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">Source of Truth</a></td>
      </tr>
      <tr>
          <td>WAL + file backup</td>
          <td>read-heavy、寫入量低、RPO 可接受定期 snapshot</td>
          <td>restore 演練失敗、WAL growth 失控、RPO / RTO 變嚴格</td>
          <td><a href="/blog/backend/knowledge-cards/rpo/" data-link-title="RPO" data-link-desc="說明恢復點目標如何定義可接受資料損失範圍">RPO</a>、<a href="/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO</a></td>
      </tr>
      <tr>
          <td>Litestream / LiteFS</td>
          <td>單 primary 寫入清楚、主要需求是 backup 或 read replica</td>
          <td>需要多地 active write、跨 region transaction</td>
          <td><a href="/blog/backend/knowledge-cards/replication-lag/" data-link-title="Replication Lag" data-link-desc="說明資料副本落後正式來源多久，以及它如何影響讀取正確性">Replication Lag</a>、<a href="/blog/backend/knowledge-cards/stale-read/" data-link-title="Stale Read" data-link-desc="讀取到落後於最新寫入版本的舊資料">Stale Read</a></td>
      </tr>
      <tr>
          <td>Cloudflare D1 / Turso</td>
          <td>edge / serverless 生態已是主平台</td>
          <td>SQL 特性、migration、observability 或 vendor 限制卡住</td>
          <td><a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">1.11 全球分散式 OLTP</a></td>
      </tr>
      <tr>
          <td>PostgreSQL / MySQL</td>
          <td>application 已進入多服務、多 tenant、權限與備份治理需求</td>
          <td>schema migration、connection、audit 與 failover 成主題</td>
          <td><a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL vendor</a>、<a href="/blog/backend/01-database/vendors/mysql/" data-link-title="MySQL" data-link-desc="高併發網路服務常用關聯式資料庫、Vitess / PlanetScale 分片生態、GitHub / Shopify / Facebook 規模驗證">MySQL vendor</a></td>
      </tr>
  </tbody>
</table>
<p>SQLite 的簡單路徑是讓檔案生命週期成為正式操作流程。只要單一 writer、備份、restore、migration 與 file ownership 都能被 runbook 控制，SQLite 可以是正式狀態，而非臨時 cache。</p>
<p>升級到 server SQL 的訊號是操作責任超過檔案邊界。當團隊需要資料庫帳號、權限分層、read replica、線上 schema migration、集中 audit 或跨 instance failover 時，PostgreSQL / MySQL / Aurora 會比繼續包裝 SQLite 更清楚。</p>
<h2 id="已知-limitation-與後續路由">已知 limitation 與後續路由</h2>
<p>SQLite overview 目前已完成服務判斷與章節群正文路由。File lifecycle、WAL locking、PRAGMA tuning、schema migration、test fixture、local-first sync、edge product 差異、observability、hands-on 與 migration route 都已有對應正文；下一輪審查可集中在案例補強、引用精度與跨章重複整理。</p>
<h2 id="案例對照">案例對照</h2>
<p>SQLite 不在 09 case 庫的「規模化 vendor」類別、但作為 <em>embedded 跟 test</em> 廣泛使用：</p>
<ul>
<li>iOS Core Data：所有 iOS app 的 default DB</li>
<li>Chrome / Firefox：cookie、history、bookmark</li>
<li>Fossil SCM：repository metadata 與 application-file use case</li>
<li>Cloudflare D1：edge serverless（新興 production 場景）</li>
<li>Turso：distributed SQLite（新興 production 場景）</li>
</ul>
<h2 id="常見陷阱">常見陷阱</h2>
<ul>
<li><strong>default journal mode 不改 WAL</strong>：read 跟 write 互相 block、performance 差</li>
<li><strong>多 process / instance 同時寫同檔</strong>：corruption</li>
<li><strong>delete 後檔案沒縮小</strong>：忘了 vacuum</li>
<li><strong>synchronous=OFF 給 production</strong>：power loss 可能掉資料</li>
<li><strong>SQLite 跟 PostgreSQL 行為差異測試不足</strong>：SQLite test 過、PostgreSQL production 出 bug（特別是 date / time、NULL 處理、type coercion）</li>
</ul>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>完整 T1 對照：<a href="/blog/backend/01-database/vendors/" data-link-title="資料庫 Vendor 清單" data-link-desc="規劃 SQL、managed SQL、document、KV 與 distributed SQL 的服務頁撰寫順序與教學大綱">01-database vendors index</a></li>
<li>平行：<a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL vendor</a> / <a href="/blog/backend/01-database/vendors/mysql/" data-link-title="MySQL" data-link-desc="高併發網路服務常用關聯式資料庫、Vitess / PlanetScale 分片生態、GitHub / Shopify / Facebook 規模驗證">MySQL vendor</a>（production server-based RDBMS）</li>
<li>上游：<a href="/blog/backend/01-database/repository-adapter/" data-link-title="1.4 Repository Adapter 實作" data-link-desc="Port / Adapter 邊界、row mapping、error translation、ORM vs query builder 選型、contract test 設計">1.4 Repository Adapter</a>（test fixture 模式）</li>
<li>結構：<a href="/blog/backend/01-database/vendors/sqlite/teaching-structure/" data-link-title="SQLite Teaching Structure" data-link-desc="SQLite 服務章節群的大綱：從 embedded formal state、WAL、backup、test fixture、local-first、edge SQLite 到遷移路由">SQLite Teaching Structure</a>（完整章節群與寫作順序）</li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a>（local file、backup restore、WAL busy reproduction、migration fixture、D1 / Turso preview）</li>
<li>深入：<a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">SQLite file lifecycle 與 backup boundary</a>（WAL、backup、restore、file ownership）</li>
<li>官方：<a href="https://sqlite.org/docs.html">SQLite Documentation</a>、<a href="https://litestream.io/">Litestream</a>、<a href="https://turso.tech/">Turso</a>、<a href="https://developers.cloudflare.com/d1/">Cloudflare D1</a></li>
</ul>
]]></content:encoded></item><item><title>從 collector 資料做基礎 funnel 分析</title><link>https://tarrragon.github.io/blog/monitoring/08-business-analytics/self-hosted-funnel/</link><pubDate>Fri, 19 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/08-business-analytics/self-hosted-funnel/</guid><description>&lt;p>自架 collector 收集的事件資料可以做基礎的 funnel 分析，不需要商業方案。分析的深度取決於 storage backend 的查詢能力 — SQLite 層能做每步事件計數，PostgreSQL 層能做 session 級轉換率分析。功能分層的完整定義見 &lt;a href="https://tarrragon.github.io/blog/monitoring/04-collector/feature-tier-boundary/" data-link-title="功能分層與 Backend 選擇" data-link-desc="SQLite 層和 PostgreSQL 層各自承載哪些功能 — 分界線是查詢模式而非資料量、觸發升級的是功能需求而非規模成長">功能分層與 Backend 選擇&lt;/a>。&lt;/p>
&lt;h2 id="定義-funnel-步驟">定義 funnel 步驟&lt;/h2>
&lt;p>Funnel 分析的第一步是列出每一步和對應的事件名稱。以一個透過 WebSocket 連接遠端終端機的 app 連線流程為例：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>步驟&lt;/th>
 &lt;th>事件名稱&lt;/th>
 &lt;th>意義&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>1&lt;/td>
 &lt;td>terminal.connect.start&lt;/td>
 &lt;td>使用者點擊連線&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>2&lt;/td>
 &lt;td>auth.biometric.success&lt;/td>
 &lt;td>生物辨識通過&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>3&lt;/td>
 &lt;td>terminal.connect.done&lt;/td>
 &lt;td>WebSocket 連線成功&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>4&lt;/td>
 &lt;td>terminal.input.submit&lt;/td>
 &lt;td>使用者開始打字&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h2 id="sqlite-層每步事件計數">SQLite 層：每步事件計數&lt;/h2>
&lt;p>SQLite backend 能做的 funnel 是「每步有多少事件觸發」— 單表 GROUP BY，不需要跨事件 JOIN。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="k">SELECT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">COUNT&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="o">*&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">as&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">count&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">FROM&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">events&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">WHERE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">IN&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="s1">&amp;#39;terminal.connect.start&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;auth.biometric.success&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;terminal.connect.done&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;terminal.input.submit&amp;#39;&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">5&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">AND&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">ts&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">&amp;gt;=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">datetime&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="s1">&amp;#39;now&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;-7 days&amp;#39;&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">6&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">GROUP&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="p">;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>步驟 N 的轉換率 = 步驟 N 的事件數 / 步驟 N-1 的事件數。流失率 = 1 - 轉換率。&lt;/p>
&lt;h3 id="能做的">能做的&lt;/h3>
&lt;ul>
&lt;li>每步事件計數（單表 GROUP BY）&lt;/li>
&lt;li>按 source.version 或 source.platform 分群（加 WHERE 條件）&lt;/li>
&lt;li>按天/按週看趨勢（strftime 分桶 + GROUP BY）&lt;/li>
&lt;/ul>
&lt;h3 id="做不到的">做不到的&lt;/h3>
&lt;ul>
&lt;li>&lt;strong>Session 級轉換率&lt;/strong>：「同一個 session 完成步驟 1 到步驟 4 的比例」需要 JOIN 同 session 的多個事件、跨所有 session 聚合。SQLite 能做這個 JOIN，但在大量 session 時效能不足。&lt;/li>
&lt;li>&lt;strong>步驟間耗時&lt;/strong>：「使用者在步驟 1 和步驟 2 之間等了多久」需要 self-join on session_id + timestamp 差值計算。&lt;/li>
&lt;li>&lt;strong>漏斗順序驗證&lt;/strong>：確認使用者是按 1→2→3→4 順序完成、不是跳步。&lt;/li>
&lt;/ul>
&lt;h2 id="postgresql-層session-級-funnel">PostgreSQL 層：Session 級 funnel&lt;/h2>
&lt;p>PostgreSQL backend 提供 window function 和高效 JOIN，能做完整的 session 級 funnel 分析。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="k">WITH&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_steps&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">AS&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">SELECT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_id&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">ROW_NUMBER&lt;/span>&lt;span class="p">()&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">OVER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">PARTITION&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_id&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">ORDER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">ts&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">as&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">step_order&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">FROM&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">events&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">WHERE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">name&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">IN&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="s1">&amp;#39;terminal.connect.start&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;auth.biometric.success&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;terminal.connect.done&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;terminal.input.submit&amp;#39;&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">AND&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">ts&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">&amp;gt;=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">NOW&lt;/span>&lt;span class="p">()&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">-&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">INTERVAL&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;7 days&amp;#39;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="p">),&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">session_max_step&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">AS&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">SELECT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_id&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">MAX&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">step_order&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">as&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">reached&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">FROM&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_steps&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="k">GROUP&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_id&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">SELECT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">reached&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">COUNT&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="o">*&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">as&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">sessions&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">FROM&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">session_max_step&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">GROUP&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">reached&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">ORDER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">BY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">reached&lt;/span>&lt;span class="p">;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h3 id="新增能力">新增能力&lt;/h3>
&lt;ul>
&lt;li>&lt;strong>Session 級轉換率&lt;/strong>：每個 session 到達了哪一步、在哪一步流失&lt;/li>
&lt;li>&lt;strong>步驟間耗時&lt;/strong>：LAG window function 計算相鄰步驟的 timestamp 差值&lt;/li>
&lt;li>&lt;strong>漏斗順序驗證&lt;/strong>：用 ROW_NUMBER + CASE 確認步驟順序&lt;/li>
&lt;li>&lt;strong>Cohort 分群的 funnel&lt;/strong>：按使用者註冊日期 / 版本 / 平台分群看不同 cohort 的 funnel 差異&lt;/li>
&lt;/ul>
&lt;h2 id="jsonl-匯出後的臨時分析">JSONL 匯出後的臨時分析&lt;/h2>
&lt;p>Collector 的 &lt;code>monitor export --format=jsonl&lt;/code> 可以匯出事件為 JSONL 格式。匯出後用 grep + jq 做一次性的臨時分析：&lt;/p></description><content:encoded><![CDATA[<p>自架 collector 收集的事件資料可以做基礎的 funnel 分析，不需要商業方案。分析的深度取決於 storage backend 的查詢能力 — SQLite 層能做每步事件計數，PostgreSQL 層能做 session 級轉換率分析。功能分層的完整定義見 <a href="/blog/monitoring/04-collector/feature-tier-boundary/" data-link-title="功能分層與 Backend 選擇" data-link-desc="SQLite 層和 PostgreSQL 層各自承載哪些功能 — 分界線是查詢模式而非資料量、觸發升級的是功能需求而非規模成長">功能分層與 Backend 選擇</a>。</p>
<h2 id="定義-funnel-步驟">定義 funnel 步驟</h2>
<p>Funnel 分析的第一步是列出每一步和對應的事件名稱。以一個透過 WebSocket 連接遠端終端機的 app 連線流程為例：</p>
<table>
  <thead>
      <tr>
          <th>步驟</th>
          <th>事件名稱</th>
          <th>意義</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>1</td>
          <td>terminal.connect.start</td>
          <td>使用者點擊連線</td>
      </tr>
      <tr>
          <td>2</td>
          <td>auth.biometric.success</td>
          <td>生物辨識通過</td>
      </tr>
      <tr>
          <td>3</td>
          <td>terminal.connect.done</td>
          <td>WebSocket 連線成功</td>
      </tr>
      <tr>
          <td>4</td>
          <td>terminal.input.submit</td>
          <td>使用者開始打字</td>
      </tr>
  </tbody>
</table>
<h2 id="sqlite-層每步事件計數">SQLite 層：每步事件計數</h2>
<p>SQLite backend 能做的 funnel 是「每步有多少事件觸發」— 單表 GROUP BY，不需要跨事件 JOIN。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">SELECT</span><span class="w"> </span><span class="n">name</span><span class="p">,</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="k">count</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">WHERE</span><span class="w"> </span><span class="n">name</span><span class="w"> </span><span class="k">IN</span><span class="w"> </span><span class="p">(</span><span class="s1">&#39;terminal.connect.start&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;auth.biometric.success&#39;</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w">               </span><span class="s1">&#39;terminal.connect.done&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;terminal.input.submit&#39;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w">  </span><span class="k">AND</span><span class="w"> </span><span class="n">ts</span><span class="w"> </span><span class="o">&gt;=</span><span class="w"> </span><span class="n">datetime</span><span class="p">(</span><span class="s1">&#39;now&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;-7 days&#39;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="w"></span><span class="k">GROUP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">name</span><span class="p">;</span></span></span></code></pre></div><p>步驟 N 的轉換率 = 步驟 N 的事件數 / 步驟 N-1 的事件數。流失率 = 1 - 轉換率。</p>
<h3 id="能做的">能做的</h3>
<ul>
<li>每步事件計數（單表 GROUP BY）</li>
<li>按 source.version 或 source.platform 分群（加 WHERE 條件）</li>
<li>按天/按週看趨勢（strftime 分桶 + GROUP BY）</li>
</ul>
<h3 id="做不到的">做不到的</h3>
<ul>
<li><strong>Session 級轉換率</strong>：「同一個 session 完成步驟 1 到步驟 4 的比例」需要 JOIN 同 session 的多個事件、跨所有 session 聚合。SQLite 能做這個 JOIN，但在大量 session 時效能不足。</li>
<li><strong>步驟間耗時</strong>：「使用者在步驟 1 和步驟 2 之間等了多久」需要 self-join on session_id + timestamp 差值計算。</li>
<li><strong>漏斗順序驗證</strong>：確認使用者是按 1→2→3→4 順序完成、不是跳步。</li>
</ul>
<h2 id="postgresql-層session-級-funnel">PostgreSQL 層：Session 級 funnel</h2>
<p>PostgreSQL backend 提供 window function 和高效 JOIN，能做完整的 session 級 funnel 分析。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">WITH</span><span class="w"> </span><span class="n">session_steps</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="w">  </span><span class="k">SELECT</span><span class="w"> </span><span class="n">session_id</span><span class="p">,</span><span class="w"> </span><span class="n">name</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">         </span><span class="n">ROW_NUMBER</span><span class="p">()</span><span class="w"> </span><span class="n">OVER</span><span class="w"> </span><span class="p">(</span><span class="n">PARTITION</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">session_id</span><span class="w"> </span><span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">ts</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">step_order</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">  </span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">  </span><span class="k">WHERE</span><span class="w"> </span><span class="n">name</span><span class="w"> </span><span class="k">IN</span><span class="w"> </span><span class="p">(</span><span class="s1">&#39;terminal.connect.start&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;auth.biometric.success&#39;</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">                 </span><span class="s1">&#39;terminal.connect.done&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;terminal.input.submit&#39;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">    </span><span class="k">AND</span><span class="w"> </span><span class="n">ts</span><span class="w"> </span><span class="o">&gt;=</span><span class="w"> </span><span class="n">NOW</span><span class="p">()</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="nb">INTERVAL</span><span class="w"> </span><span class="s1">&#39;7 days&#39;</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w"></span><span class="p">),</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w"></span><span class="n">session_max_step</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w">  </span><span class="k">SELECT</span><span class="w"> </span><span class="n">session_id</span><span class="p">,</span><span class="w"> </span><span class="k">MAX</span><span class="p">(</span><span class="n">step_order</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">reached</span><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="w">  </span><span class="k">FROM</span><span class="w"> </span><span class="n">session_steps</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="w">  </span><span class="k">GROUP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">session_id</span><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w"></span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="n">reached</span><span class="p">,</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">sessions</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">session_max_step</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w"></span><span class="k">GROUP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">reached</span><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w"></span><span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">reached</span><span class="p">;</span></span></span></code></pre></div><h3 id="新增能力">新增能力</h3>
<ul>
<li><strong>Session 級轉換率</strong>：每個 session 到達了哪一步、在哪一步流失</li>
<li><strong>步驟間耗時</strong>：LAG window function 計算相鄰步驟的 timestamp 差值</li>
<li><strong>漏斗順序驗證</strong>：用 ROW_NUMBER + CASE 確認步驟順序</li>
<li><strong>Cohort 分群的 funnel</strong>：按使用者註冊日期 / 版本 / 平台分群看不同 cohort 的 funnel 差異</li>
</ul>
<h2 id="jsonl-匯出後的臨時分析">JSONL 匯出後的臨時分析</h2>
<p>Collector 的 <code>monitor export --format=jsonl</code> 可以匯出事件為 JSONL 格式。匯出後用 grep + jq 做一次性的臨時分析：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">for</span> step in terminal.connect.start auth.biometric.success terminal.connect.done terminal.input.submit<span class="p">;</span> <span class="k">do</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">  <span class="nv">count</span><span class="o">=</span><span class="k">$(</span>grep <span class="s2">&#34;\&#34;name\&#34;:\&#34;</span><span class="nv">$step</span><span class="s2">\&#34;&#34;</span> exported-events.jsonl <span class="p">|</span> wc -l<span class="k">)</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">  <span class="nb">echo</span> <span class="s2">&#34;</span><span class="nv">$step</span><span class="s2">: </span><span class="nv">$count</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="k">done</span></span></span></code></pre></div><p>JSONL 臨時分析適合「快速看一眼大概數字」的場景。持續性的 funnel 監控應該用 SQLite 或 PostgreSQL 的 SQL 查詢，結果穩定且可重現。</p>
<h2 id="自架-vs-商業方案">自架 vs 商業方案</h2>
<table>
  <thead>
      <tr>
          <th>需求</th>
          <th>自架能力</th>
          <th>商業方案</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>每步事件計數</td>
          <td>SQLite GROUP BY</td>
          <td>Mixpanel / Amplitude 內建</td>
      </tr>
      <tr>
          <td>Session 級轉換率</td>
          <td>PostgreSQL window function</td>
          <td>Mixpanel / Amplitude 內建</td>
      </tr>
      <tr>
          <td>視覺化 funnel 漏斗圖</td>
          <td>自建 dashboard</td>
          <td>商業方案內建、拖拉設定</td>
      </tr>
      <tr>
          <td>即時更新</td>
          <td>定期重算 + dashboard 刷新</td>
          <td>商業方案即時</td>
      </tr>
      <tr>
          <td>A/B test 分群 funnel</td>
          <td>PostgreSQL + feature flag</td>
          <td>Optimizely / LaunchDarkly 整合</td>
      </tr>
  </tbody>
</table>
<p>自用工具場景下，SQLite 層的每步事件計數通常足夠。商業產品需要 session 級分析時，PostgreSQL 層的 SQL 能力和商業方案的分析能力在功能上對等，差異在 UI 和設定便利性。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>Funnel 分析的完整方法論 → <a href="/blog/monitoring/08-business-analytics/funnel-analysis/" data-link-title="Funnel Analysis" data-link-desc="使用者在哪一步流失 — 從事件序列計算每步轉換率、找出流失最嚴重的步驟、區分設計問題和技術問題">Funnel analysis</a></li>
<li>事件設計如何影響分析品質 → <a href="/blog/monitoring/08-business-analytics/behavior-event-design/" data-link-title="行為事件設計" data-link-desc="事件命名規範、屬性設計、funnel 定義 — 行為分析的品質取決於事件設計的品質">行為事件設計</a></li>
<li>功能分層定義 → <a href="/blog/monitoring/04-collector/feature-tier-boundary/" data-link-title="功能分層與 Backend 選擇" data-link-desc="SQLite 層和 PostgreSQL 層各自承載哪些功能 — 分界線是查詢模式而非資料量、觸發升級的是功能需求而非規模成長">功能分層與 Backend 選擇</a></li>
<li>去識別化是分析的入場條件 → <a href="/blog/monitoring/07-security-privacy/" data-link-title="模組七：資安與隱私" data-link-desc="SDK redaction / transport 加密 / collector access control / 去識別化 — 蒐集的資料本身就是風險資產">模組七 資安與隱私</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Backend 效能基準</title><link>https://tarrragon.github.io/blog/monitoring/04-collector/sqlite-performance-baseline/</link><pubDate>Sat, 20 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/04-collector/sqlite-performance-baseline/</guid><description>&lt;p>SQLite Backend 的效能受三個因素影響：儲存裝置（SSD vs HDD vs SD card）、Go driver 選擇（modernc.org/sqlite pure Go vs mattn/go-sqlite3 CGO）、並發模型（WAL mode + single-writer）。本章根據 SQLite 的技術特性和業界基準推導預期效能範圍，並提供實測方法讓使用者在自己的環境驗證。所有數字是預期範圍而非實測值 — 實際效能依硬體和 workload 而定。&lt;/p>
&lt;h2 id="寫入吞吐">寫入吞吐&lt;/h2>
&lt;p>寫入吞吐決定 collector 每秒能消化多少事件。SQLite 的寫入效能主要受 fsync 頻率和 WAL checkpoint 影響。&lt;/p>
&lt;h3 id="單筆-insert">單筆 INSERT&lt;/h3>
&lt;p>每筆 INSERT 獨立一個 transaction 時，每次 commit 都會 fsync。WAL mode 的 fsync 成本比 journal mode 低（append-only），但仍是寫入的主要瓶頸。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>儲存裝置&lt;/th>
 &lt;th>單筆 INSERT 延遲&lt;/th>
 &lt;th>理論上限&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>NVMe SSD&lt;/td>
 &lt;td>10-30 μs&lt;/td>
 &lt;td>30,000-100,000 inserts/sec&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>SATA SSD&lt;/td>
 &lt;td>30-50 μs&lt;/td>
 &lt;td>20,000-30,000 inserts/sec&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>HDD&lt;/td>
 &lt;td>50-200 μs&lt;/td>
 &lt;td>5,000-20,000 inserts/sec&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>SD card&lt;/td>
 &lt;td>500-2000 μs&lt;/td>
 &lt;td>500-2,000 inserts/sec&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>modernc.org/sqlite（pure Go）的效能約為 CGO driver（mattn/go-sqlite3）的 60-80%。上表數字基於 CGO driver，pure Go 需打八折。Go HTTP handler 的開銷（JSON 解碼、schema 驗證、goroutine 調度）再扣 10-20%。&lt;/p>
&lt;h3 id="批次-insert">批次 INSERT&lt;/h3>
&lt;p>一個 transaction 包裹多筆 INSERT，只做一次 fsync。Collector 接收 SDK 的 flush batch（一個 HTTP request 帶一批事件）天然適合批次寫入。&lt;/p>
&lt;p>吞吐提升幅度和批次大小的關係：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>批次大小&lt;/th>
 &lt;th>相對單筆的吞吐提升&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>10 筆/tx&lt;/td>
 &lt;td>3-5x&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>100 筆/tx&lt;/td>
 &lt;td>5-10x&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>1000 筆/tx&lt;/td>
 &lt;td>8-15x&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>提升來自 fsync 次數從「每筆一次」降到「每批一次」。超過 100 筆/tx 後邊際收益遞減。&lt;/p>
&lt;h3 id="實際預期">實際預期&lt;/h3>
&lt;p>結合 pure Go driver、HTTP handler 開銷和批次寫入，不同環境下的預期吞吐：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>環境&lt;/th>
 &lt;th>單筆&lt;/th>
 &lt;th>批次（100/tx）&lt;/th>
 &lt;th>適合場景&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Mac M1/M2 NVMe + pure Go&lt;/td>
 &lt;td>~5,000/sec&lt;/td>
 &lt;td>~30,000/sec&lt;/td>
 &lt;td>開發機&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Linux VPS SATA SSD&lt;/td>
 &lt;td>~3,000/sec&lt;/td>
 &lt;td>~20,000/sec&lt;/td>
 &lt;td>小型部署&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Raspberry Pi 4 SD card&lt;/td>
 &lt;td>~200/sec&lt;/td>
 &lt;td>~1,000/sec&lt;/td>
 &lt;td>邊緣設備&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h3 id="和事件產生速率的對照">和事件產生速率的對照&lt;/h3>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>場景&lt;/th>
 &lt;th>預估 events/sec&lt;/th>
 &lt;th>SQLite 批次能撐嗎&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>自用 1 個 app&lt;/td>
 &lt;td>&amp;lt; 10&lt;/td>
 &lt;td>遠超需求&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>小團隊 5 人各跑 1 個 app&lt;/td>
 &lt;td>&amp;lt; 50&lt;/td>
 &lt;td>綽綽有餘&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>10 SDK 同時 flush&lt;/td>
 &lt;td>100-1000 burst&lt;/td>
 &lt;td>批次 INSERT 撐得住&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>100+ 使用者持續活躍&lt;/td>
 &lt;td>500+ 持續&lt;/td>
 &lt;td>邊界 — 觀察 database is locked&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>burst 和持續的差異在於：burst 是短暫的高峰（flush batch 到達後數秒內消化完），持續是長時間的穩定高流量。SQLite 的 WAL mode 對 burst 容忍度高（write lock 等待時間短），對持續高流量容忍度有限（write lock 等待累積）。&lt;/p></description><content:encoded><![CDATA[<p>SQLite Backend 的效能受三個因素影響：儲存裝置（SSD vs HDD vs SD card）、Go driver 選擇（modernc.org/sqlite pure Go vs mattn/go-sqlite3 CGO）、並發模型（WAL mode + single-writer）。本章根據 SQLite 的技術特性和業界基準推導預期效能範圍，並提供實測方法讓使用者在自己的環境驗證。所有數字是預期範圍而非實測值 — 實際效能依硬體和 workload 而定。</p>
<h2 id="寫入吞吐">寫入吞吐</h2>
<p>寫入吞吐決定 collector 每秒能消化多少事件。SQLite 的寫入效能主要受 fsync 頻率和 WAL checkpoint 影響。</p>
<h3 id="單筆-insert">單筆 INSERT</h3>
<p>每筆 INSERT 獨立一個 transaction 時，每次 commit 都會 fsync。WAL mode 的 fsync 成本比 journal mode 低（append-only），但仍是寫入的主要瓶頸。</p>
<table>
  <thead>
      <tr>
          <th>儲存裝置</th>
          <th>單筆 INSERT 延遲</th>
          <th>理論上限</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>NVMe SSD</td>
          <td>10-30 μs</td>
          <td>30,000-100,000 inserts/sec</td>
      </tr>
      <tr>
          <td>SATA SSD</td>
          <td>30-50 μs</td>
          <td>20,000-30,000 inserts/sec</td>
      </tr>
      <tr>
          <td>HDD</td>
          <td>50-200 μs</td>
          <td>5,000-20,000 inserts/sec</td>
      </tr>
      <tr>
          <td>SD card</td>
          <td>500-2000 μs</td>
          <td>500-2,000 inserts/sec</td>
      </tr>
  </tbody>
</table>
<p>modernc.org/sqlite（pure Go）的效能約為 CGO driver（mattn/go-sqlite3）的 60-80%。上表數字基於 CGO driver，pure Go 需打八折。Go HTTP handler 的開銷（JSON 解碼、schema 驗證、goroutine 調度）再扣 10-20%。</p>
<h3 id="批次-insert">批次 INSERT</h3>
<p>一個 transaction 包裹多筆 INSERT，只做一次 fsync。Collector 接收 SDK 的 flush batch（一個 HTTP request 帶一批事件）天然適合批次寫入。</p>
<p>吞吐提升幅度和批次大小的關係：</p>
<table>
  <thead>
      <tr>
          <th>批次大小</th>
          <th>相對單筆的吞吐提升</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>10 筆/tx</td>
          <td>3-5x</td>
      </tr>
      <tr>
          <td>100 筆/tx</td>
          <td>5-10x</td>
      </tr>
      <tr>
          <td>1000 筆/tx</td>
          <td>8-15x</td>
      </tr>
  </tbody>
</table>
<p>提升來自 fsync 次數從「每筆一次」降到「每批一次」。超過 100 筆/tx 後邊際收益遞減。</p>
<h3 id="實際預期">實際預期</h3>
<p>結合 pure Go driver、HTTP handler 開銷和批次寫入，不同環境下的預期吞吐：</p>
<table>
  <thead>
      <tr>
          <th>環境</th>
          <th>單筆</th>
          <th>批次（100/tx）</th>
          <th>適合場景</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Mac M1/M2 NVMe + pure Go</td>
          <td>~5,000/sec</td>
          <td>~30,000/sec</td>
          <td>開發機</td>
      </tr>
      <tr>
          <td>Linux VPS SATA SSD</td>
          <td>~3,000/sec</td>
          <td>~20,000/sec</td>
          <td>小型部署</td>
      </tr>
      <tr>
          <td>Raspberry Pi 4 SD card</td>
          <td>~200/sec</td>
          <td>~1,000/sec</td>
          <td>邊緣設備</td>
      </tr>
  </tbody>
</table>
<h3 id="和事件產生速率的對照">和事件產生速率的對照</h3>
<table>
  <thead>
      <tr>
          <th>場景</th>
          <th>預估 events/sec</th>
          <th>SQLite 批次能撐嗎</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>自用 1 個 app</td>
          <td>&lt; 10</td>
          <td>遠超需求</td>
      </tr>
      <tr>
          <td>小團隊 5 人各跑 1 個 app</td>
          <td>&lt; 50</td>
          <td>綽綽有餘</td>
      </tr>
      <tr>
          <td>10 SDK 同時 flush</td>
          <td>100-1000 burst</td>
          <td>批次 INSERT 撐得住</td>
      </tr>
      <tr>
          <td>100+ 使用者持續活躍</td>
          <td>500+ 持續</td>
          <td>邊界 — 觀察 database is locked</td>
      </tr>
  </tbody>
</table>
<p>burst 和持續的差異在於：burst 是短暫的高峰（flush batch 到達後數秒內消化完），持續是長時間的穩定高流量。SQLite 的 WAL mode 對 burst 容忍度高（write lock 等待時間短），對持續高流量容忍度有限（write lock 等待累積）。</p>
<h2 id="查詢延遲">查詢延遲</h2>
<p>查詢延遲決定 dashboard 的刷新體驗。SQLite 的查詢效能取決於索引覆蓋和掃描行數。</p>
<h3 id="有索引的查詢">有索引的查詢</h3>
<p>建議的索引（見 <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a> 的建議索引段）覆蓋 dashboard 的核心查詢模式。有索引時的預期延遲：</p>
<table>
  <thead>
      <tr>
          <th>查詢模式</th>
          <th>10 萬筆</th>
          <th>50 萬筆</th>
          <th>100 萬筆</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>等值查詢（WHERE session_id = ?）</td>
          <td>&lt; 1ms</td>
          <td>&lt; 1ms</td>
          <td>&lt; 1ms</td>
      </tr>
      <tr>
          <td>範圍查詢（WHERE ts BETWEEN ? AND ?）</td>
          <td>&lt; 10ms</td>
          <td>10-50ms</td>
          <td>50-100ms</td>
      </tr>
      <tr>
          <td>GROUP BY name</td>
          <td>10-50ms</td>
          <td>50-200ms</td>
          <td>200-500ms</td>
      </tr>
      <tr>
          <td>COUNT DISTINCT session_id</td>
          <td>50-100ms</td>
          <td>200-500ms</td>
          <td>500ms-1s</td>
      </tr>
      <tr>
          <td>JOIN + window function</td>
          <td>100ms-1s</td>
          <td>1-3s</td>
          <td>3-10s</td>
      </tr>
  </tbody>
</table>
<h3 id="無索引的查詢">無索引的查詢</h3>
<p>無索引時 SQLite 做全表掃描。掃描速度約 50-100 MB/sec（SSD）、10-30 MB/sec（HDD）。</p>
<table>
  <thead>
      <tr>
          <th>資料量</th>
          <th>預估大小</th>
          <th>SSD 全掃延遲</th>
          <th>HDD 全掃延遲</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>10 萬筆</td>
          <td>~40 MB</td>
          <td>200-500ms</td>
          <td>1-3s</td>
      </tr>
      <tr>
          <td>100 萬筆</td>
          <td>~400 MB</td>
          <td>2-5s</td>
          <td>10-30s</td>
      </tr>
      <tr>
          <td>300 萬筆</td>
          <td>~1.2 GB</td>
          <td>5-15s</td>
          <td>30-90s</td>
      </tr>
  </tbody>
</table>
<p>超過 100 萬筆無索引查詢會超出 dashboard 可接受的刷新延遲 — 這是 day-one 就建索引的理由。</p>
<h3 id="dashboard-刷新頻率-vs-查詢延遲">Dashboard 刷新頻率 vs 查詢延遲</h3>
<p>Dashboard 的每個視圖有不同的刷新間隔和可接受延遲。查詢延遲超過可接受值時，dashboard 體驗變差（等待轉圈、資料過時）。</p>
<table>
  <thead>
      <tr>
          <th>Dashboard 視圖</th>
          <th>刷新間隔</th>
          <th>可接受延遲</th>
          <th>10 萬筆有索引</th>
          <th>100 萬筆有索引</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>即時狀態卡</td>
          <td>1-5 秒</td>
          <td>&lt; 100ms</td>
          <td>滿足</td>
          <td>滿足</td>
      </tr>
      <tr>
          <td>Error 列表</td>
          <td>5-10 秒</td>
          <td>&lt; 500ms</td>
          <td>滿足</td>
          <td>滿足</td>
      </tr>
      <tr>
          <td>趨勢圖（最近 24h）</td>
          <td>30 秒</td>
          <td>&lt; 1s</td>
          <td>滿足</td>
          <td>邊界</td>
      </tr>
      <tr>
          <td>長期聚合（最近 30 天）</td>
          <td>5 分鐘</td>
          <td>&lt; 3s</td>
          <td>滿足</td>
          <td>需要預聚合</td>
      </tr>
  </tbody>
</table>
<p>「需要預聚合」代表原始事件的聚合查詢超過可接受延遲，應該依賴分層保留策略中的 hourly_summary / daily_summary 表（見 <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a> 的分層保留段）。</p>
<h2 id="資源消耗">資源消耗</h2>
<h3 id="記憶體">記憶體</h3>
<table>
  <thead>
      <tr>
          <th>元件</th>
          <th>佔用</th>
          <th>備註</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Go HTTP server</td>
          <td>20-50 MB</td>
          <td>基礎開銷</td>
      </tr>
      <tr>
          <td>SQLite page cache</td>
          <td>2 MB（預設）</td>
          <td><code>PRAGMA cache_size</code> 可調</td>
      </tr>
      <tr>
          <td>寫入 buffer（channel）</td>
          <td>1-10 MB</td>
          <td>取決於 channel 容量和事件大小</td>
      </tr>
      <tr>
          <td>查詢結果暫存</td>
          <td>和結果集成正比</td>
          <td>GROUP BY 10 萬筆 ~10 MB</td>
      </tr>
      <tr>
          <td><strong>Collector 整體</strong></td>
          <td><strong>50-100 MB</strong></td>
          <td>自用場景</td>
      </tr>
  </tbody>
</table>
<p>Raspberry Pi（1 GB RAM）上建議把 page cache 調小（<code>PRAGMA cache_size = -512</code> = 512 KB），避免大結果集查詢（加 LIMIT），dashboard 刷新頻率降低。</p>
<h3 id="cpu">CPU</h3>
<table>
  <thead>
      <tr>
          <th>操作</th>
          <th>CPU 使用</th>
          <th>備註</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>INSERT（寫入）</td>
          <td>可忽略</td>
          <td>I/O bound，CPU 不是瓶頸</td>
      </tr>
      <tr>
          <td>SELECT（查詢）</td>
          <td>和掃描行數正比</td>
          <td>有索引時可忽略</td>
      </tr>
      <tr>
          <td>Downsample（每小時）</td>
          <td>短暫 spike &lt; 1s</td>
          <td>處理最近一小時的事件</td>
      </tr>
      <tr>
          <td>Purge（每天）</td>
          <td>短暫 spike 1-3s</td>
          <td>分批 DELETE</td>
      </tr>
      <tr>
          <td><strong>整體</strong></td>
          <td><strong>&lt; 5%</strong></td>
          <td>自用場景</td>
      </tr>
  </tbody>
</table>
<h3 id="磁碟">磁碟</h3>
<table>
  <thead>
      <tr>
          <th>日事件量</th>
          <th>原始資料/天</th>
          <th>原始資料/月</th>
          <th>含索引/月</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>1,000（極低）</td>
          <td>0.3-0.5 MB</td>
          <td>9-15 MB</td>
          <td>11-18 MB</td>
      </tr>
      <tr>
          <td>10,000（自用）</td>
          <td>3-5 MB</td>
          <td>90-150 MB</td>
          <td>110-180 MB</td>
      </tr>
      <tr>
          <td>100,000（小團隊）</td>
          <td>30-50 MB</td>
          <td>0.9-1.5 GB</td>
          <td>1.1-1.8 GB</td>
      </tr>
  </tbody>
</table>
<p>WAL 檔案通常 &lt; 10 MB（auto-checkpoint 在 WAL 達到 1000 pages 時觸發）。分層保留策略下，原始事件只保留 7 天，長期佔用由聚合摘要表決定（遠小於原始事件）。</p>
<h2 id="邊緣設備場景">邊緣設備場景</h2>
<p>Raspberry Pi、低配 VPS（1 核 / 1 GB RAM）、甚至 NAS 上跑 collector 時的特殊考量：</p>
<p><strong>SD card 的隨機寫入</strong>：SD card 的隨機寫入 IOPS 極低（100-500 IOPS），WAL mode 的 checkpoint（把 WAL 內容合併回主資料庫檔案）可能卡住 1-5 秒。期間新的寫入等待 checkpoint 完成。建議調高 <code>wal_autocheckpoint</code> 的閾值（如 5000 pages），讓 checkpoint 頻率降低但每次時間更長 — 在非活躍時段（凌晨）手動觸發 <code>PRAGMA wal_checkpoint(TRUNCATE)</code>。</p>
<p><strong>1 GB RAM</strong>：cache_size 調小（512 KB）、避免 <code>SELECT *</code> 不帶 LIMIT、GROUP BY 的結果集用 HAVING 條件過濾減少暫存。Dashboard 的長期聚合直接查 hourly_summary 表而非原始事件。</p>
<p><strong>ARM CPU</strong>：pure Go SQLite driver（modernc.org/sqlite）在 ARM 上的效能差距可能比 x86 更大（pure Go 的 C-to-Go 翻譯在 ARM 的指令最佳化較少）。實測確認。</p>
<p><strong>建議配置</strong>：邊緣設備上 collector 的 dashboard 刷新頻率從預設值降低（即時狀態卡 5 秒 → 30 秒，趨勢圖 30 秒 → 5 分鐘），降採樣 job 頻率從每小時改為每 6 小時。</p>
<h2 id="實測方法指引">實測方法指引</h2>
<p>教學的預期數字是推導值，實際效能取決於使用者的硬體和 workload。Collector 提供內建的 benchmark 命令讓使用者在自己的環境實測。</p>
<h3 id="寫入-benchmark">寫入 benchmark</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># 單筆寫入：10000 筆，每筆獨立 transaction</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">./collector benchmark write --events<span class="o">=</span><span class="m">10000</span> --batch<span class="o">=</span><span class="m">1</span> --storage<span class="o">=</span>sqlite
</span></span><span class="line"><span class="ln">3</span><span class="cl">
</span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="c1"># 批次寫入：10000 筆，每 100 筆一個 transaction</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl">./collector benchmark write --events<span class="o">=</span><span class="m">10000</span> --batch<span class="o">=</span><span class="m">100</span> --storage<span class="o">=</span>sqlite</span></span></code></pre></div><p>輸出：total duration、events/sec、p50/p95/p99 latency per event。</p>
<h3 id="查詢-benchmark">查詢 benchmark</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># 先灌入測試資料</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">./collector benchmark seed --events<span class="o">=</span><span class="m">100000</span> --storage<span class="o">=</span>sqlite
</span></span><span class="line"><span class="ln">3</span><span class="cl">
</span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="c1"># 跑查詢 benchmark</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl">./collector benchmark query --type<span class="o">=</span>error --group-by<span class="o">=</span>name --storage<span class="o">=</span>sqlite
</span></span><span class="line"><span class="ln">6</span><span class="cl">./collector benchmark query --session-id<span class="o">=</span>random --storage<span class="o">=</span>sqlite</span></span></code></pre></div><p>輸出：query duration、rows scanned、rows returned。</p>
<h3 id="production-觀察指標">Production 觀察指標</h3>
<p>部署後用 DevOps dashboard（見 <a href="/blog/monitoring/04-collector/dashboard-devops/" data-link-title="DevOps Dashboard 設計" data-link-desc="Collector 和 SDK 是否健康 — 日常監控的服務狀態卡、吞吐量曲線、儲存用量，以及告警觸發後的排障視圖">DevOps Dashboard 設計</a>）觀察 collector 自身的效能 metric：</p>
<ul>
<li><code>collector.storage.write_duration_ms</code>：每次寫入的延遲。P95 超過 100ms 是瓶頸訊號。</li>
<li><code>collector.storage.query_duration_ms</code>：每次查詢的延遲。P95 超過 dashboard 刷新間隔是瓶頸訊號。</li>
<li><code>collector.storage.db_size_bytes</code>：資料庫大小。接近磁碟可用空間的 80% 時觸發 purge 或擴容。</li>
<li><code>collector.storage.wal_size_bytes</code>：WAL 檔案大小。持續 &gt; 50 MB 代表 checkpoint 跟不上寫入速度。</li>
</ul>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>切換到 PostgreSQL 的觸發條件 → <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a></li>
<li>SQLite 和 PostgreSQL 的功能分層 → <a href="/blog/monitoring/04-collector/feature-tier-boundary/" data-link-title="功能分層與 Backend 選擇" data-link-desc="SQLite 層和 PostgreSQL 層各自承載哪些功能 — 分界線是查詢模式而非資料量、觸發升級的是功能需求而非規模成長">功能分層與 Backend 選擇</a></li>
<li>Ingestion 端的擴展設計 → <a href="/blog/monitoring/04-collector/ingestion-scaling/" data-link-title="Ingestion Scaling" data-link-desc="四層防線應對 ingestion 端的流量擴展 — SDK 取樣、Collector 背壓、水平擴展、Queue 解耦">Ingestion Scaling</a></li>
</ul>
]]></content:encoded></item><item><title>Container 部署設計</title><link>https://tarrragon.github.io/blog/monitoring/04-collector/container-deployment/</link><pubDate>Sat, 20 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/04-collector/container-deployment/</guid><description>&lt;p>Container 部署讓 collector 完全隔離於 host 環境，開源使用者用 &lt;code>docker run&lt;/code> 一行部署，不需要安裝 Go 或管理 binary 版本。但 SQLite 在 container 中有特殊的 I/O 和持久化考量 — overlay filesystem 的寫入延遲和 container 生命週期對資料持久性的影響需要在部署設計中處理。&lt;/p>
&lt;h2 id="dockerfile-設計">Dockerfile 設計&lt;/h2>
&lt;p>Multi-stage build 把編譯環境和執行環境分離。Build stage 用 Go 官方 image 編譯 binary，runtime stage 只包含 binary 和必要的 CA 憑證。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-dockerfile" data-lang="dockerfile">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="k">FROM&lt;/span>&lt;span class="s"> golang:1.22-alpine AS build&lt;/span>&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">WORKDIR&lt;/span>&lt;span class="s"> /src&lt;/span>&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">COPY&lt;/span> go.mod go.sum ./&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">RUN&lt;/span> go mod download&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">COPY&lt;/span> . .&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">RUN&lt;/span> &lt;span class="nv">CGO_ENABLED&lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="m">0&lt;/span> go build -o /collector ./cmd/collector&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">FROM&lt;/span>&lt;span class="s"> alpine:3.20&lt;/span>&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">RUN&lt;/span> apk add --no-cache ca-certificates tzdata&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">COPY&lt;/span> --from&lt;span class="o">=&lt;/span>build /collector /usr/local/bin/collector&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">RUN&lt;/span> adduser -D -u &lt;span class="m">1000&lt;/span> monitor&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">USER&lt;/span>&lt;span class="s"> monitor&lt;/span>&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">EXPOSE&lt;/span>&lt;span class="s"> 8080&lt;/span>&lt;span class="err">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="err">&lt;/span>&lt;span class="k">ENTRYPOINT&lt;/span> &lt;span class="p">[&lt;/span>&lt;span class="s2">&amp;#34;collector&amp;#34;&lt;/span>&lt;span class="p">]&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>最終 image 包含 Go binary（~15MB）+ alpine base（~7MB）+ ca-certificates，總大小目標 &amp;lt; 25MB。用 &lt;code>scratch&lt;/code> 替代 &lt;code>alpine&lt;/code> 可以再小 7MB，但失去 shell debug 能力。&lt;/p>
&lt;h2 id="sqlite-在-container-中的-io-考量">SQLite 在 Container 中的 I/O 考量&lt;/h2>
&lt;p>Docker 的 overlay2 storage driver 在每次 fsync 時經過 overlay 層。SQLite 的 WAL mode 依賴 fsync 確保寫入持久性 — 每筆 transaction commit 觸發一次 fsync。Overlay 層增加的延遲讓每筆 fsync 慢 20-40%（取決於 host 的 storage driver 和檔案系統）。&lt;/p>
&lt;h3 id="volume-mount-繞過-overlay">Volume mount 繞過 overlay&lt;/h3>
&lt;p>把 SQLite 的資料目錄掛載為 host volume（&lt;code>-v /host/data:/data&lt;/code>），SQLite 直接寫 host 檔案系統、繞過 overlay 層。寫入效能和同機部署的 binary 版本相當。&lt;/p>
&lt;p>不用 volume mount 的風險：container 刪除時 overlay 層的資料一起消失。&lt;code>docker rm&lt;/code> = 所有事件資料消失。即使只是 &lt;code>docker run&lt;/code> 新版本的 image 也會建立新 container，舊 container 的資料不會自動遷移。&lt;/p>
&lt;h2 id="volume-mount-設計">Volume Mount 設計&lt;/h2>
&lt;p>兩個目錄分開掛載，職責和權限不同：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Mount&lt;/th>
 &lt;th>Container 路徑&lt;/th>
 &lt;th>Host 路徑（範例）&lt;/th>
 &lt;th>權限&lt;/th>
 &lt;th>內容&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>資料&lt;/td>
 &lt;td>&lt;code>/data&lt;/code>&lt;/td>
 &lt;td>&lt;code>./monitor-data&lt;/code>&lt;/td>
 &lt;td>read-write&lt;/td>
 &lt;td>SQLite DB + WAL + 匯出檔&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>設定&lt;/td>
 &lt;td>&lt;code>/config&lt;/code>&lt;/td>
 &lt;td>&lt;code>./monitor-config&lt;/code>&lt;/td>
 &lt;td>read-only&lt;/td>
 &lt;td>retention config + rule config + sensor config&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Container 內用非 root user（UID 1000）執行。Host 的 volume 目錄 ownership 需要對應：&lt;/p></description><content:encoded><![CDATA[<p>Container 部署讓 collector 完全隔離於 host 環境，開源使用者用 <code>docker run</code> 一行部署，不需要安裝 Go 或管理 binary 版本。但 SQLite 在 container 中有特殊的 I/O 和持久化考量 — overlay filesystem 的寫入延遲和 container 生命週期對資料持久性的影響需要在部署設計中處理。</p>
<h2 id="dockerfile-設計">Dockerfile 設計</h2>
<p>Multi-stage build 把編譯環境和執行環境分離。Build stage 用 Go 官方 image 編譯 binary，runtime stage 只包含 binary 和必要的 CA 憑證。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-dockerfile" data-lang="dockerfile"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">FROM</span><span class="s"> golang:1.22-alpine AS build</span><span class="err">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="err"></span><span class="k">WORKDIR</span><span class="s"> /src</span><span class="err">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="err"></span><span class="k">COPY</span> go.mod go.sum ./<span class="err">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="err"></span><span class="k">RUN</span> go mod download<span class="err">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="err"></span><span class="k">COPY</span> . .<span class="err">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="err"></span><span class="k">RUN</span> <span class="nv">CGO_ENABLED</span><span class="o">=</span><span class="m">0</span> go build -o /collector ./cmd/collector<span class="err">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="err"></span><span class="k">FROM</span><span class="s"> alpine:3.20</span><span class="err">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="err"></span><span class="k">RUN</span> apk add --no-cache ca-certificates tzdata<span class="err">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="err"></span><span class="k">COPY</span> --from<span class="o">=</span>build /collector /usr/local/bin/collector<span class="err">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="err"></span><span class="k">RUN</span> adduser -D -u <span class="m">1000</span> monitor<span class="err">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="err"></span><span class="k">USER</span><span class="s"> monitor</span><span class="err">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="err"></span><span class="k">EXPOSE</span><span class="s"> 8080</span><span class="err">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="err"></span><span class="k">ENTRYPOINT</span> <span class="p">[</span><span class="s2">&#34;collector&#34;</span><span class="p">]</span></span></span></code></pre></div><p>最終 image 包含 Go binary（~15MB）+ alpine base（~7MB）+ ca-certificates，總大小目標 &lt; 25MB。用 <code>scratch</code> 替代 <code>alpine</code> 可以再小 7MB，但失去 shell debug 能力。</p>
<h2 id="sqlite-在-container-中的-io-考量">SQLite 在 Container 中的 I/O 考量</h2>
<p>Docker 的 overlay2 storage driver 在每次 fsync 時經過 overlay 層。SQLite 的 WAL mode 依賴 fsync 確保寫入持久性 — 每筆 transaction commit 觸發一次 fsync。Overlay 層增加的延遲讓每筆 fsync 慢 20-40%（取決於 host 的 storage driver 和檔案系統）。</p>
<h3 id="volume-mount-繞過-overlay">Volume mount 繞過 overlay</h3>
<p>把 SQLite 的資料目錄掛載為 host volume（<code>-v /host/data:/data</code>），SQLite 直接寫 host 檔案系統、繞過 overlay 層。寫入效能和同機部署的 binary 版本相當。</p>
<p>不用 volume mount 的風險：container 刪除時 overlay 層的資料一起消失。<code>docker rm</code> = 所有事件資料消失。即使只是 <code>docker run</code> 新版本的 image 也會建立新 container，舊 container 的資料不會自動遷移。</p>
<h2 id="volume-mount-設計">Volume Mount 設計</h2>
<p>兩個目錄分開掛載，職責和權限不同：</p>
<table>
  <thead>
      <tr>
          <th>Mount</th>
          <th>Container 路徑</th>
          <th>Host 路徑（範例）</th>
          <th>權限</th>
          <th>內容</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>資料</td>
          <td><code>/data</code></td>
          <td><code>./monitor-data</code></td>
          <td>read-write</td>
          <td>SQLite DB + WAL + 匯出檔</td>
      </tr>
      <tr>
          <td>設定</td>
          <td><code>/config</code></td>
          <td><code>./monitor-config</code></td>
          <td>read-only</td>
          <td>retention config + rule config + sensor config</td>
      </tr>
  </tbody>
</table>
<p>Container 內用非 root user（UID 1000）執行。Host 的 volume 目錄 ownership 需要對應：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">mkdir -p monitor-data monitor-config
</span></span><span class="line"><span class="ln">2</span><span class="cl">chown 1000:1000 monitor-data</span></span></code></pre></div><h2 id="graceful-shutdown">Graceful Shutdown</h2>
<p><code>docker stop</code> 送 SIGTERM → collector 收到後執行 shutdown 序列：</p>
<ol>
<li>停止接受新的 HTTP request（listener close）</li>
<li>等待 in-flight request 完成（5 秒 context timeout）</li>
<li>Flush pending writes（尚未寫入 storage 的事件，5 秒）</li>
<li>停止定期 job（downsample / purge / rule engine 定期評估）</li>
<li>SQLite WAL checkpoint（TRUNCATE mode，15 秒）</li>
<li>關閉 DB connection</li>
<li>退出</li>
</ol>
<p>步驟 2-5 合計超時上限 25 秒。這個序列對應 <a href="/blog/backend/05-deployment-platform/platform-lifecycle-contract/" data-link-title="5.6 Platform Lifecycle Contract" data-link-desc="說明 runtime、startup、readiness、liveness、shutdown 與 drain 如何組成平台生命週期合約。">Backend 5.6 Platform Lifecycle Contract</a> 的 shutdown → drain 狀態：步驟 1-2 是 drain（停接新工作、等在途完成），步驟 3-6 是 shutdown（flush 狀態和釋放資源）。Collector 屬於短 request API 的 workload 類型（drain 窗口 5-30 秒），但多了 WAL checkpoint 步驟，讓 shutdown 時間可能超過一般 HTTP 服務。PID 1 信號處理的設計考量（exec form、避免 shell 攔截 SIGTERM）見 <a href="/blog/backend/05-deployment-platform/container-runtime/" data-link-title="5.1 container 與 runtime" data-link-desc="整理 image、resource limit 與啟動行為">Backend 5.1 PID 1 與信號處理</a>。</p>
<p><code>docker stop</code> 預設等 10 秒後送 SIGKILL。如果 WAL checkpoint 在大量未 checkpoint 的資料下需要超過 10 秒，Docker Compose 可以調 <code>stop_grace_period: 30s</code>。</p>
<p>SQLite 的 WAL 設計支援 crash recovery — SIGKILL 後 WAL 檔案仍在，下次開啟 DB 時自動 replay。但非 graceful shutdown 可能丟失 channel 中尚未寫入的事件（已收到 HTTP 202 但還在 buffer 中的事件）。</p>
<h2 id="資源限制">資源限制</h2>
<table>
  <thead>
      <tr>
          <th>資源</th>
          <th>建議值（自用）</th>
          <th>建議值（小團隊）</th>
          <th>理由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Memory</td>
          <td>256MB</td>
          <td>512MB</td>
          <td>Collector + SQLite page cache + Go runtime</td>
      </tr>
      <tr>
          <td>CPU</td>
          <td>0.5 核</td>
          <td>1 核</td>
          <td>I/O bound、CPU 通常不是瓶頸</td>
      </tr>
      <tr>
          <td>磁碟</td>
          <td>volume mount 容量</td>
          <td>volume mount 容量</td>
          <td>保留策略控制、和 host 磁碟共享</td>
      </tr>
  </tbody>
</table>
<p>Memory 限制設太緊會觸發 OOMKill — container 突然消失且無 log。設定 memory limit 前先觀察 collector 的 baseline 記憶體使用（<code>docker stats</code>），再乘以 1.5 安全係數。CPU request/limit 的設定策略（guaranteed vs burstable QoS）和 memory limit 與 OOM 的判讀見 <a href="/blog/backend/05-deployment-platform/container-runtime/" data-link-title="5.1 container 與 runtime" data-link-desc="整理 image、resource limit 與啟動行為">Backend 5.1 Resource Limit</a>。</p>
<h2 id="docker-compose-範例">Docker Compose 範例</h2>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="nt">services</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="w">  </span><span class="nt">collector</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">    </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">tarrragon/monitor:latest</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">    </span><span class="nt">ports</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">      </span>- <span class="s2">&#34;8080:8080&#34;</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">    </span><span class="nt">volumes</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">      </span>- <span class="l">./monitor-data:/data</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w">      </span>- <span class="l">./monitor-config:/config:ro</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w">    </span><span class="nt">environment</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w">      </span>- <span class="l">MONITOR_STORAGE=sqlite</span><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="w">      </span>- <span class="l">MONITOR_DB_PATH=/data/events.db</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="w">    </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="l">unless-stopped</span><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w">    </span><span class="nt">stop_grace_period</span><span class="p">:</span><span class="w"> </span><span class="l">30s</span><span class="w">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="w">    </span><span class="nt">deploy</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w">      </span><span class="nt">resources</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w">        </span><span class="nt">limits</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w">          </span><span class="nt">memory</span><span class="p">:</span><span class="w"> </span><span class="l">256M</span><span class="w">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="w">          </span><span class="nt">cpus</span><span class="p">:</span><span class="w"> </span><span class="s1">&#39;0.5&#39;</span><span class="w">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="w">    </span><span class="nt">healthcheck</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="w">      </span><span class="nt">test</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">&#34;CMD&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;wget&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;-q&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;--spider&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;http://localhost:8080/health&#34;</span><span class="p">]</span><span class="w">
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="w">      </span><span class="nt">interval</span><span class="p">:</span><span class="w"> </span><span class="l">30s</span><span class="w">
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="w">      </span><span class="nt">timeout</span><span class="p">:</span><span class="w"> </span><span class="l">5s</span><span class="w">
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="w">      </span><span class="nt">retries</span><span class="p">:</span><span class="w"> </span><span class="m">3</span></span></span></code></pre></div><p><code>restart: unless-stopped</code> 讓 container 在 crash 或 host 重啟後自動恢復。<code>healthcheck</code> 讓 Docker 偵測 collector 是否真的在回應 — 只有 process 活著但 HTTP 不回應的場景也會被標記為 unhealthy。</p>
<h2 id="和同機部署的效能對照">和同機部署的效能對照</h2>
<table>
  <thead>
      <tr>
          <th>指標</th>
          <th>同機 binary</th>
          <th>Container + volume mount</th>
          <th>Container 無 volume（overlay）</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>寫入吞吐（Mac SSD）</td>
          <td>~5,000/sec</td>
          <td>~4,500/sec（-10%）</td>
          <td>~3,000/sec（-40%）</td>
      </tr>
      <tr>
          <td>寫入吞吐（Linux VPS）</td>
          <td>~3,000/sec</td>
          <td>~2,700/sec（-10%）</td>
          <td>~1,800/sec（-40%）</td>
      </tr>
      <tr>
          <td>查詢延遲</td>
          <td>baseline</td>
          <td>baseline（volume = 直接讀 host）</td>
          <td>+20%（overlay 讀取開銷小）</td>
      </tr>
      <tr>
          <td>啟動時間</td>
          <td>&lt; 100ms</td>
          <td>&lt; 500ms（container 啟動開銷）</td>
          <td>同左</td>
      </tr>
      <tr>
          <td>記憶體額外開銷</td>
          <td>0</td>
          <td>~10-20MB（container runtime）</td>
          <td>同左</td>
      </tr>
  </tbody>
</table>
<p>Volume mount 後效能差異只有 ~10%（Go HTTP handler 的 overhead 大於 volume mount 的 overhead）。不用 volume mount 時 overlay fs 的 fsync 開銷顯著 — 寫入吞吐降 40%。</p>
<h2 id="何時用-container何時用-binary">何時用 container、何時用 binary</h2>
<table>
  <thead>
      <tr>
          <th>場景</th>
          <th>建議</th>
          <th>理由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>開源使用者快速試用</td>
          <td>Container</td>
          <td><code>docker run</code> 一行、不需裝 Go</td>
      </tr>
      <tr>
          <td>長期自用部署</td>
          <td>Binary + systemd</td>
          <td>效能最佳、無 container overhead</td>
      </tr>
      <tr>
          <td>CI/CD 測試環境</td>
          <td>Container</td>
          <td>可拋棄式、每次乾淨環境</td>
      </tr>
      <tr>
          <td>Kubernetes 部署</td>
          <td>Container</td>
          <td>pod spec 標準化</td>
      </tr>
      <tr>
          <td>Raspberry Pi / 邊緣設備</td>
          <td>Binary</td>
          <td>低資源環境避免 container overhead</td>
      </tr>
  </tbody>
</table>
<h2 id="斷網環境的部署考量">斷網環境的部署考量</h2>
<p>Collector 在斷網環境（air-gapped）裡的部署跟連網環境的主要差異有三點。第一，SDK 的 endpoint 從外部 URL（<code>https://collect.example.com</code>）改為內網地址（<code>http://collector.internal:8080</code>），SDK 設定檔裡的 endpoint 要能按環境切換。第二，Collector 的 container image 無法從 Docker Hub 拉取——需要透過 content ferry 搬運映像、推送到內網的 private registry（Harbor 或 Docker Registry），Dockerfile 的 base image 來源也要改指 private registry。第三，Collector 的 storage backend 只能用本地磁碟或 NFS，不能用雲端物件儲存——SQLite backend 在斷網環境反而是優勢（零外部依賴），儲存容量規劃要在部署前就確定，因為斷網環境的磁碟擴容流程可能需要數週。</p>
<p>SDK 的 offline buffer（見<a href="/blog/monitoring/03-sdk-design/offline-buffer/" data-link-title="離線 buffer 與重試" data-link-desc="網路不可用時的事件保存策略 — FIFO 丟棄、本地 persistence、恢復後補發的取捨">SDK 設計：offline-buffer</a>）在斷網環境更重要——如果 Collector 重啟或暫時不可達，SDK 端的 buffer 是唯一能保住事件的機制。</p>
<p>斷網環境的 infra 層監控（Prometheus / Grafana / Loki）設定見<a href="/blog/infra/air-gapped/air-gapped-monitoring/" data-link-title="斷網環境的監控與可觀測性" data-link-desc="Self-hosted 監控（Prometheus &#43; Grafana）、離線 log 收集（Loki / ELK）、不能 phone home 的告警、NTP 時間同步">斷網環境的監控與可觀測性</a>。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>SQLite 效能基準的詳細數字 → <a href="/blog/monitoring/04-collector/sqlite-performance-baseline/" data-link-title="SQLite Backend 效能基準" data-link-desc="寫入吞吐 / 查詢延遲 / 資源消耗的量化預期 — 不同硬體環境下 SQLite 能撐多少、邊界在哪、怎麼實測">SQLite Backend 效能基準</a></li>
<li>可插拔 Storage Backend 架構 → <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a></li>
<li>Container runtime 通用原則（base image 選擇、build 可重現性、PID 1 信號處理）→ <a href="/blog/backend/05-deployment-platform/container-runtime/" data-link-title="5.1 container 與 runtime" data-link-desc="整理 image、resource limit 與啟動行為">Backend 5.1 Container 與 Runtime</a></li>
<li>生命週期合約（startup / readiness / drain / shutdown 的責任分類）→ <a href="/blog/backend/05-deployment-platform/platform-lifecycle-contract/" data-link-title="5.6 Platform Lifecycle Contract" data-link-desc="說明 runtime、startup、readiness、liveness、shutdown 與 drain 如何組成平台生命週期合約。">Backend 5.6 Platform Lifecycle Contract</a></li>
<li>容器化資源設計的通用原則 → <a href="/blog/devops/05-capacity-planning/container-resource-design/" data-link-title="容器化資源設計" data-link-desc="Container 的 memory / CPU / 磁碟限制設計 — 資源限制設太緊 OOMKill、設太鬆擠壓其他服務、overlay filesystem 的 I/O 影響">DevOps 容器化資源設計</a></li>
<li>服務探活和自動恢復 → <a href="/blog/devops/04-service-health/" data-link-title="模組四：服務探活與自動恢復" data-link-desc="服務掛了怎麼自動發現和恢復 — health check 設計、liveness vs readiness、systemd watchdog、process supervisor">DevOps 服務探活</a></li>
</ul>
]]></content:encoded></item><item><title>PostgreSQL to SQLite Simplification</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-from-postgresql-simplification/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-from-postgresql-simplification/</guid><description>&lt;p>PostgreSQL to SQLite simplification 的核心責任是處理反向路線：服務責任縮小後，評估 SQLite 是否能降低操作成本。這條路線適合 single-user app、CLI、desktop app、內部工具、read-mostly artifact store、demo environment、local-first prototype 或 edge-local utility。&lt;/p>
&lt;p>本文的判讀錨點是：降級到 SQLite 是責任縮小，也是讓資料模型回到 single-process / file-owned / local-state 的工程選擇。只要正式需求從 multi-user server DB 回到這個範圍，SQLite 可以提供更低元件數、更容易搬移與更低維護成本。&lt;/p>
&lt;h2 id="simplification-drivers">Simplification Drivers&lt;/h2>
&lt;p>Simplification drivers 的核心責任是確認 PostgreSQL 的能力已超過服務需求。若 server DB 的 HA、role、replica、pool、vacuum、PITR、schema governance 都變成維運負擔，而產品只需要單一 process 持有資料，就可以評估 SQLite。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Driver&lt;/th>
 &lt;th>代表情境&lt;/th>
 &lt;th>SQLite 帶來的收益&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Single-user app&lt;/td>
 &lt;td>desktop、CLI、local admin tool&lt;/td>
 &lt;td>file portability、offline use&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Read-mostly artifact&lt;/td>
 &lt;td>build metadata、catalog snapshot&lt;/td>
 &lt;td>deployment simple、低 runtime dependency&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Internal tool&lt;/td>
 &lt;td>小團隊使用、資料量小、低寫入&lt;/td>
 &lt;td>降低 DB server operation&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Demo / fixture&lt;/td>
 &lt;td>每個 environment 一份可重建資料&lt;/td>
 &lt;td>quick reset、deterministic seed&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Edge-local utility&lt;/td>
 &lt;td>request-local / device-local state&lt;/td>
 &lt;td>low latency、local ownership&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Driver 要連到 ownership。SQLite 適合「這份資料由某個 process / device / artifact 明確持有」；若資料仍屬於多服務共同真相，保留 PostgreSQL 或改成 managed SQL 會更穩定。&lt;/p>
&lt;h2 id="no-go-conditions">No-Go Conditions&lt;/h2>
&lt;p>No-go condition 的核心責任是保護仍需要 server DB 的服務。若 PostgreSQL 的核心能力仍被業務依賴，遷到 SQLite 會把風險轉移到 application code、file backup 與人工流程。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>No-go 訊號&lt;/th>
 &lt;th>代表責任&lt;/th>
 &lt;th>保留路由&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>多 tenant 與 centralized permission&lt;/td>
 &lt;td>DB role、grant、audit 仍有價值&lt;/td>
 &lt;td>PostgreSQL&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>多 instance concurrent writer&lt;/td>
 &lt;td>SQLite writer boundary 壓力過高&lt;/td>
 &lt;td>PostgreSQL / MySQL&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>PITR / HA 是合約要求&lt;/td>
 &lt;td>server DB operation 是正式責任&lt;/td>
 &lt;td>Managed PostgreSQL / Aurora&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Analyst / job 直接查 DB&lt;/td>
 &lt;td>access control 與 query isolation&lt;/td>
 &lt;td>PostgreSQL read replica / warehouse&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Cross-service source of truth&lt;/td>
 &lt;td>單檔 ownership 與服務邊界衝突&lt;/td>
 &lt;td>保留 server DB 或拆 bounded context&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>No-go 條件要寫進 migration proposal。Simplification 的目標是降低操作成本；若降級後要用大量自製機制補回 role、audit、HA 與 concurrent write，成本會回到系統裡。&lt;/p>
&lt;h2 id="diff-audit">Diff Audit&lt;/h2>
&lt;p>Diff audit 的核心責任是把 PostgreSQL 語意縮到 SQLite 可以清楚承擔的範圍。PostgreSQL extension、function、type、index、constraint、sequence、view、trigger、role 與 transaction behavior 都要盤點。&lt;/p></description><content:encoded><![CDATA[<p>PostgreSQL to SQLite simplification 的核心責任是處理反向路線：服務責任縮小後，評估 SQLite 是否能降低操作成本。這條路線適合 single-user app、CLI、desktop app、內部工具、read-mostly artifact store、demo environment、local-first prototype 或 edge-local utility。</p>
<p>本文的判讀錨點是：降級到 SQLite 是責任縮小，也是讓資料模型回到 single-process / file-owned / local-state 的工程選擇。只要正式需求從 multi-user server DB 回到這個範圍，SQLite 可以提供更低元件數、更容易搬移與更低維護成本。</p>
<h2 id="simplification-drivers">Simplification Drivers</h2>
<p>Simplification drivers 的核心責任是確認 PostgreSQL 的能力已超過服務需求。若 server DB 的 HA、role、replica、pool、vacuum、PITR、schema governance 都變成維運負擔，而產品只需要單一 process 持有資料，就可以評估 SQLite。</p>
<table>
  <thead>
      <tr>
          <th>Driver</th>
          <th>代表情境</th>
          <th>SQLite 帶來的收益</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Single-user app</td>
          <td>desktop、CLI、local admin tool</td>
          <td>file portability、offline use</td>
      </tr>
      <tr>
          <td>Read-mostly artifact</td>
          <td>build metadata、catalog snapshot</td>
          <td>deployment simple、低 runtime dependency</td>
      </tr>
      <tr>
          <td>Internal tool</td>
          <td>小團隊使用、資料量小、低寫入</td>
          <td>降低 DB server operation</td>
      </tr>
      <tr>
          <td>Demo / fixture</td>
          <td>每個 environment 一份可重建資料</td>
          <td>quick reset、deterministic seed</td>
      </tr>
      <tr>
          <td>Edge-local utility</td>
          <td>request-local / device-local state</td>
          <td>low latency、local ownership</td>
      </tr>
  </tbody>
</table>
<p>Driver 要連到 ownership。SQLite 適合「這份資料由某個 process / device / artifact 明確持有」；若資料仍屬於多服務共同真相，保留 PostgreSQL 或改成 managed SQL 會更穩定。</p>
<h2 id="no-go-conditions">No-Go Conditions</h2>
<p>No-go condition 的核心責任是保護仍需要 server DB 的服務。若 PostgreSQL 的核心能力仍被業務依賴，遷到 SQLite 會把風險轉移到 application code、file backup 與人工流程。</p>
<table>
  <thead>
      <tr>
          <th>No-go 訊號</th>
          <th>代表責任</th>
          <th>保留路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>多 tenant 與 centralized permission</td>
          <td>DB role、grant、audit 仍有價值</td>
          <td>PostgreSQL</td>
      </tr>
      <tr>
          <td>多 instance concurrent writer</td>
          <td>SQLite writer boundary 壓力過高</td>
          <td>PostgreSQL / MySQL</td>
      </tr>
      <tr>
          <td>PITR / HA 是合約要求</td>
          <td>server DB operation 是正式責任</td>
          <td>Managed PostgreSQL / Aurora</td>
      </tr>
      <tr>
          <td>Analyst / job 直接查 DB</td>
          <td>access control 與 query isolation</td>
          <td>PostgreSQL read replica / warehouse</td>
      </tr>
      <tr>
          <td>Cross-service source of truth</td>
          <td>單檔 ownership 與服務邊界衝突</td>
          <td>保留 server DB 或拆 bounded context</td>
      </tr>
  </tbody>
</table>
<p>No-go 條件要寫進 migration proposal。Simplification 的目標是降低操作成本；若降級後要用大量自製機制補回 role、audit、HA 與 concurrent write，成本會回到系統裡。</p>
<h2 id="diff-audit">Diff Audit</h2>
<p>Diff audit 的核心責任是把 PostgreSQL 語意縮到 SQLite 可以清楚承擔的範圍。PostgreSQL extension、function、type、index、constraint、sequence、view、trigger、role 與 transaction behavior 都要盤點。</p>
<table>
  <thead>
      <tr>
          <th>PostgreSQL feature</th>
          <th>SQLite 轉換策略</th>
          <th>審查問題</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>timestamptz</code></td>
          <td>UTC ISO text 或 integer epoch</td>
          <td>timezone policy 是否固定</td>
      </tr>
      <tr>
          <td><code>jsonb</code> + GIN</td>
          <td>JSON text + limited query / app filter</td>
          <td>query 是否仍需 index</td>
      </tr>
      <tr>
          <td>Sequence / identity</td>
          <td>INTEGER PRIMARY KEY 或 app ID</td>
          <td>id stability 與 import collision</td>
      </tr>
      <tr>
          <td>Partial index</td>
          <td>SQLite partial index</td>
          <td>predicate 與 query planner 是否對齊</td>
      </tr>
      <tr>
          <td>Role / grant</td>
          <td>filesystem permission + app auth</td>
          <td>權限是否可移到 application boundary</td>
      </tr>
      <tr>
          <td>Extension</td>
          <td>application logic 或放棄 feature</td>
          <td>feature 是否仍是正式需求</td>
      </tr>
  </tbody>
</table>
<p>Diff audit 的輸出是一份保留 / 移除 / 改寫清單。每個 PostgreSQL feature 都要回答：這是正式需求、歷史殘留，還是可以移到 application layer 的便利功能。</p>
<h2 id="phase-plan">Phase Plan</h2>
<p>Phase plan 的核心責任是把 server DB 退場變成可回復流程。反向 migration 要超過一次性 dump：先收斂寫入、建立 SQLite schema、匯入資料、跑 adapter test、演練 backup，再退役 PostgreSQL。</p>
<table>
  <thead>
      <tr>
          <th>Phase</th>
          <th>目的</th>
          <th>Evidence</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Scope reduction</td>
          <td>確認資料責任已縮小</td>
          <td>ownership doc、no-go review</td>
      </tr>
      <tr>
          <td>Schema rewrite</td>
          <td>建立 SQLite schema</td>
          <td>migration dry run、STRICT / constraint</td>
      </tr>
      <tr>
          <td>Data export</td>
          <td>從 PostgreSQL 匯出 snapshot</td>
          <td>row count、checksum、dump metadata</td>
      </tr>
      <tr>
          <td>Data import</td>
          <td>寫入 SQLite file</td>
          <td>integrity check、foreign key check</td>
      </tr>
      <tr>
          <td>Adapter switch</td>
          <td>app 改用 SQLite repository</td>
          <td>contract test、error mapping</td>
      </tr>
      <tr>
          <td>Backup runbook</td>
          <td>建立 file lifecycle evidence</td>
          <td>backup restore drill</td>
      </tr>
      <tr>
          <td>Server retirement</td>
          <td>關閉 PostgreSQL 寫入與 credential</td>
          <td>retention、credential removal、incident route</td>
      </tr>
  </tbody>
</table>
<p>Scope reduction 是第一關。若資料仍被多個服務寫入，應先拆出 bounded context 或建立 event / export boundary；SQLite file 才能成為明確 owned artifact。</p>
<h2 id="data-movement">Data Movement</h2>
<p>Data movement 的核心責任是把 PostgreSQL snapshot 轉成 SQLite file 並保留驗證。可用 <code>COPY</code> / CSV、application ETL 或 dedicated migration tool；選擇取決於 type conversion 與資料量。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">psql <span class="s2">&#34;</span><span class="nv">$DATABASE_URL</span><span class="s2">&#34;</span> -c <span class="s2">&#34;\\copy orders TO &#39;orders.csv&#39; CSV HEADER&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.mode csv&#34;</span> <span class="s2">&#34;.import --skip 1 orders.csv orders&#34;</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>這段命令是教學骨架。正式流程要處理 NULL、delimiter、timezone、numeric precision、FK order、transaction、temporary disk、sensitive data 與 import log。</p>
<p>Import 後要跑三種 evidence：database integrity、row count / checksum、business invariant。Business invariant 例如 active user count、total balance、latest event id、pending job count；這些比單純 row count 更能抓到語意錯誤。</p>
<h2 id="runbook-shift">Runbook Shift</h2>
<p>Runbook shift 的核心責任是把 PostgreSQL operation 移轉成 SQLite file operation。Server DB 的 backup / role / monitoring 退場後，要補上 SQLite 的 backup、restore、file permission、WAL、migration 與 disk 觀測。</p>
<p>最小 SQLite runbook 包含：</p>
<ol>
<li>Database file path、owner process、filesystem permission。</li>
<li>Journal mode、busy timeout、foreign key、schema version。</li>
<li>Backup command、restore drill、retention、checksum。</li>
<li>Migration command、pre-migration snapshot、rollback path。</li>
<li>Observability：busy、WAL size、disk free、backup age。</li>
<li>Incident route：disk full、bad migration、corruption signal。</li>
</ol>
<p>Runbook shift 要同步移除 PostgreSQL credential。Server database 退役時，保留 read-only archive、刪除 application secret、關閉 scheduled job、更新 dashboard 與 incident routing。</p>
<h2 id="cleanup-and-retention">Cleanup and Retention</h2>
<p>Cleanup and retention 的核心責任是讓舊 PostgreSQL 不再成為影子真相。Migration 後若舊 DB 長期可寫，團隊會在事故中分不清哪份資料有效。</p>
<table>
  <thead>
      <tr>
          <th>Cleanup 項目</th>
          <th>操作</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Write disable</td>
          <td>PostgreSQL role 改 read-only 或關閉 app access</td>
      </tr>
      <tr>
          <td>Archive snapshot</td>
          <td>保存最後 dump、checksum、schema</td>
      </tr>
      <tr>
          <td>Credential removal</td>
          <td>移除 app secret、CI secret、admin token</td>
      </tr>
      <tr>
          <td>Dashboard update</td>
          <td>停用 PostgreSQL alert、啟用 SQLite alert</td>
      </tr>
      <tr>
          <td>Documentation</td>
          <td>更新 source-of-truth 與 restore route</td>
      </tr>
  </tbody>
</table>
<p>Retention 要和 data protection 對齊。若 PostgreSQL 內有 PII、audit log 或 legal retention，退役流程要依 retention policy 保存或銷毀，而非直接刪除。</p>
<h2 id="decision-route">Decision Route</h2>
<p>Decision route 的核心責任是讓 simplification 保持可逆。若未來 concurrent writer、central audit、PITR 或 multi-service source-of-truth 回來，系統要能沿 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration</a> 重新升級。</p>
<table>
  <thead>
      <tr>
          <th>現況</th>
          <th>建議</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Single-user / local artifact</td>
          <td>SQLite simplification</td>
      </tr>
      <tr>
          <td>Small internal tool + low write</td>
          <td>SQLite + restore drill</td>
      </tr>
      <tr>
          <td>Read-mostly dataset for app bundle</td>
          <td>SQLite artifact + release version</td>
      </tr>
      <tr>
          <td>Multi-user SaaS</td>
          <td>保留 PostgreSQL</td>
      </tr>
      <tr>
          <td>Audit / HA / role 是正式要求</td>
          <td>保留 managed PostgreSQL</td>
      </tr>
  </tbody>
</table>
<p>Simplification 的完成標準是：SQLite file 可以被重建、備份、恢復、升級與交接。只要這些 evidence 完整，從 PostgreSQL 退到 SQLite 是清楚的工程決策。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>PostgreSQL to SQLite simplification 完成後，先讀 <a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">file lifecycle / backup boundary</a> 建立 file operation；再讀 <a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">SQLite observability / runbook</a> 補 evidence；若之後需求再成長，回到 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration</a>。</p>
]]></content:encoded></item><item><title>SQLite Backup Restore Drill</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/backup-restore-drill/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/backup-restore-drill/</guid><description>&lt;p>SQLite backup restore drill 的核心責任是證明單檔 database 可以被一致備份並還原。這篇承接 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary&lt;/a>，把備份從概念轉成 artifact、validation query 與 RPO / RTO note。&lt;/p>
&lt;p>本文的驗收標準是：你能從 live &lt;code>app.db&lt;/code> 建立 backup，將它還原到隔離路徑，通過 &lt;code>integrity_check&lt;/code> 與核心查詢，並記錄 restore duration。&lt;/p>
&lt;h2 id="prepare-source">Prepare Source&lt;/h2>
&lt;p>Prepare source 的核心責任是建立一個有 WAL 與資料變化的 live database。若你已跑過 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/" data-link-title="SQLite Local File Quickstart" data-link-desc="SQLite local .db file、schema、seed data、PRAGMA baseline、query sample 與 cleanup 的操作說明">local file quickstart&lt;/a>，可以直接沿用 &lt;code>/tmp/sqlite-lab/app.db&lt;/code>。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">mkdir -p /tmp/sqlite-lab/backup /tmp/sqlite-lab/restore
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;PRAGMA journal_mode = WAL;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (2, 100, &amp;#39;backup-drill-1&amp;#39;, &amp;#39;2026-05-21T01:00:00Z&amp;#39;);&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>這一步讓 source database 有新的資料。後續會用 backup snapshot 和 source 後續寫入做對照。&lt;/p>
&lt;h2 id="create-backup">Create Backup&lt;/h2>
&lt;p>Create backup 的核心責任是用 SQLite-aware 方法建立一致 snapshot。SQLite CLI &lt;code>.backup&lt;/code> 會透過 SQLite backup API 產出目標檔案。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;.backup &amp;#39;backup/app-backup.db&amp;#39;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">sqlite3 backup/app-backup.db &lt;span class="s2">&amp;#34;PRAGMA integrity_check;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>預期 &lt;code>integrity_check&lt;/code> 輸出 &lt;code>ok&lt;/code>。這是最小 backup evidence。&lt;/p>
&lt;p>&lt;code>VACUUM INTO&lt;/code> 也可以產出 compact copy，適合想順便整理檔案大小的情境。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;VACUUM INTO &amp;#39;backup/app-vacuum-copy.db&amp;#39;;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">sqlite3 backup/app-vacuum-copy.db &lt;span class="s2">&amp;#34;PRAGMA integrity_check;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;code>.backup&lt;/code> 與 &lt;code>VACUUM INTO&lt;/code> 都要在 runbook 中標明使用條件、耗時、目標路徑與失敗處理。正式環境還要記錄檔案大小、checksum 與 storage retention。&lt;/p>
&lt;h2 id="mutate-source-after-backup">Mutate Source After Backup&lt;/h2>
&lt;p>Mutate source 的核心責任是確認 backup 是時間點 snapshot。備份後對 source 寫入新資料，再用 restore 驗證 backup 保持原時間點。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 777, &amp;#39;after-backup-write&amp;#39;, &amp;#39;2026-05-21T01:05:00Z&amp;#39;);&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;SELECT COUNT(*) FROM ledger_entries;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">sqlite3 backup/app-backup.db &lt;span class="s2">&amp;#34;SELECT COUNT(*) FROM ledger_entries;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>Source count 應比 backup count 多一筆。這個差異讓 RPO 討論具體化：backup 只保護到它建立的時間點。&lt;/p>
&lt;h2 id="restore-isolated-copy">Restore Isolated Copy&lt;/h2>
&lt;p>Restore isolated copy 的核心責任是避免把演練和 source 混在一起。把 backup 複製到 restore path，所有 validation 都對 restore file 執行。&lt;/p></description><content:encoded><![CDATA[<p>SQLite backup restore drill 的核心責任是證明單檔 database 可以被一致備份並還原。這篇承接 <a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary</a>，把備份從概念轉成 artifact、validation query 與 RPO / RTO note。</p>
<p>本文的驗收標準是：你能從 live <code>app.db</code> 建立 backup，將它還原到隔離路徑，通過 <code>integrity_check</code> 與核心查詢，並記錄 restore duration。</p>
<h2 id="prepare-source">Prepare Source</h2>
<p>Prepare source 的核心責任是建立一個有 WAL 與資料變化的 live database。若你已跑過 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/" data-link-title="SQLite Local File Quickstart" data-link-desc="SQLite local .db file、schema、seed data、PRAGMA baseline、query sample 與 cleanup 的操作說明">local file quickstart</a>，可以直接沿用 <code>/tmp/sqlite-lab/app.db</code>。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">mkdir -p /tmp/sqlite-lab/backup /tmp/sqlite-lab/restore
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-lab
</span></span><span class="line"><span class="ln">3</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA journal_mode = WAL;&#34;</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl">sqlite3 app.db <span class="s2">&#34;INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (2, 100, &#39;backup-drill-1&#39;, &#39;2026-05-21T01:00:00Z&#39;);&#34;</span></span></span></code></pre></div><p>這一步讓 source database 有新的資料。後續會用 backup snapshot 和 source 後續寫入做對照。</p>
<h2 id="create-backup">Create Backup</h2>
<p>Create backup 的核心責任是用 SQLite-aware 方法建立一致 snapshot。SQLite CLI <code>.backup</code> 會透過 SQLite backup API 產出目標檔案。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.backup &#39;backup/app-backup.db&#39;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 backup/app-backup.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>預期 <code>integrity_check</code> 輸出 <code>ok</code>。這是最小 backup evidence。</p>
<p><code>VACUUM INTO</code> 也可以產出 compact copy，適合想順便整理檔案大小的情境。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;VACUUM INTO &#39;backup/app-vacuum-copy.db&#39;;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 backup/app-vacuum-copy.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p><code>.backup</code> 與 <code>VACUUM INTO</code> 都要在 runbook 中標明使用條件、耗時、目標路徑與失敗處理。正式環境還要記錄檔案大小、checksum 與 storage retention。</p>
<h2 id="mutate-source-after-backup">Mutate Source After Backup</h2>
<p>Mutate source 的核心責任是確認 backup 是時間點 snapshot。備份後對 source 寫入新資料，再用 restore 驗證 backup 保持原時間點。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 777, &#39;after-backup-write&#39;, &#39;2026-05-21T01:05:00Z&#39;);&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;SELECT COUNT(*) FROM ledger_entries;&#34;</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">sqlite3 backup/app-backup.db <span class="s2">&#34;SELECT COUNT(*) FROM ledger_entries;&#34;</span></span></span></code></pre></div><p>Source count 應比 backup count 多一筆。這個差異讓 RPO 討論具體化：backup 只保護到它建立的時間點。</p>
<h2 id="restore-isolated-copy">Restore Isolated Copy</h2>
<p>Restore isolated copy 的核心責任是避免把演練和 source 混在一起。把 backup 複製到 restore path，所有 validation 都對 restore file 執行。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">cp backup/app-backup.db restore/app-restored.db
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">sqlite3 restore/app-restored.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">sqlite3 restore/app-restored.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">.headers on
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">.mode column
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">SELECT account_id, SUM(amount_cents) AS balance_cents
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">FROM ledger_entries
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">GROUP BY account_id
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">ORDER BY account_id;
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>正式 restore drill 還要啟動 application 指向 <code>restore/app-restored.db</code>，跑核心 read/write smoke test。若 application 需要 migration，也要確認 restore file 的 <code>PRAGMA user_version</code> 與 app version 相容。</p>
<h2 id="rpo--rto-note">RPO / RTO Note</h2>
<p>RPO / RTO note 的核心責任是把演練結果轉成服務承諾。RPO 是可接受資料遺失窗口，RTO 是可接受恢復時間。</p>
<table>
  <thead>
      <tr>
          <th>指標</th>
          <th>本 lab 記錄方式</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>RPO</td>
          <td>backup 建立時間到事故時間的資料差距</td>
      </tr>
      <tr>
          <td>RTO</td>
          <td>從取得 backup 到 app smoke test 成功耗時</td>
      </tr>
  </tbody>
</table>
<p>可以用 shell 的 <code>time</code> 記錄 restore duration。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="nb">time</span> sqlite3 restore/app-restored.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>正式服務要把 RPO / RTO 寫進 <a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">observability / runbook</a>。</p>
<h2 id="known-gap">Known Gap</h2>
<p>Known gap 的核心責任是讓 lab 結果誠實。這個 drill 驗證 SQLite-aware backup 與 restore path；它尚未覆蓋 object storage credential、remote retention、large database restore time、encrypted disk、user device support flow 與 legal retention。</p>
<p>完成本篇後，下一步可以進入 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a> 觀察 writer boundary，或進入 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/migration-fixture-lab/" data-link-title="SQLite Migration Fixture Lab" data-link-desc="SQLite user_version、table rebuild migration、fixture snapshot、rollback note 與 CI evidence 的操作說明">migration fixture lab</a> 建立 schema change evidence。</p>
]]></content:encoded></item><item><title>SQLite D1 / Turso / libSQL Comparison</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/</guid><description>&lt;p>D1 / Turso / libSQL comparison 的核心責任是把 SQLite-compatible edge products 和 local SQLite 分開判讀。它們共享 SQLite 開發體驗的一部分，但它們承擔的服務責任不同：Cloudflare D1 把 SQLite-like database 放進 Workers 生態與 managed edge platform；Turso / libSQL 把 SQLite family 延伸到 remote primary、embedded replica 與同步模型；local SQLite 則是 application process 直接管理單一 database file。&lt;/p>
&lt;p>本文的判讀錨點是：SQLite compatibility 代表開發入口接近，服務責任仍要重新審查。採用 edge SQLite 前，要先確認 write authority、read freshness、migration limit、backup evidence、observability、cost 與 vendor exit，而非只看 SQL 語法能否執行。&lt;/p>
&lt;h2 id="product-boundary">Product Boundary&lt;/h2>
&lt;p>Product boundary 的核心責任是定義誰持有資料、誰執行 SQL、誰負責恢復。Local SQLite 的資料在你的 filesystem；D1 的資料由 Cloudflare D1 平台管理並和 Workers binding 整合；Turso / libSQL 的資料通常有 remote database 與 client / embedded replica 的分工。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>選項&lt;/th>
 &lt;th>主要責任&lt;/th>
 &lt;th>適合情境&lt;/th>
 &lt;th>關鍵審查點&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Local SQLite&lt;/td>
 &lt;td>Process-local formal state&lt;/td>
 &lt;td>CLI、desktop、single-node app&lt;/td>
 &lt;td>file lifecycle、backup、WAL、lock&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Cloudflare D1&lt;/td>
 &lt;td>Workers-integrated database&lt;/td>
 &lt;td>edge app、serverless API、low ops&lt;/td>
 &lt;td>platform limit、migration、binding&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Turso / libSQL&lt;/td>
 &lt;td>Remote primary + replicas&lt;/td>
 &lt;td>low-latency read、embedded replica&lt;/td>
 &lt;td>freshness、sync、driver semantics&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Litestream / LiteFS&lt;/td>
 &lt;td>Backup / replica operation&lt;/td>
 &lt;td>single-node app with recovery / read&lt;/td>
 &lt;td>RPO、RTO、primary ownership&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>PostgreSQL&lt;/td>
 &lt;td>Server SQL operation&lt;/td>
 &lt;td>multi-tenant、central audit、HA、role&lt;/td>
 &lt;td>operation team、PITR、schema gate&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Local SQLite 的判斷重點是 file ownership。若 app 與 database file 位於同一個 host，備份、restore、disk full、permission 與 app upgrade 都在你的 runbook 裡；這條路線承接 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">file lifecycle / backup boundary&lt;/a>。&lt;/p>
&lt;p>D1 的判斷重點是 platform integration。Cloudflare 官方 D1 docs 把 D1 放在 Workers 與 Wrangler workflow 內，並公開 &lt;a href="https://developers.cloudflare.com/d1/platform/limits/">D1 limits&lt;/a>；因此採用 D1 時要把 database decision 與 Workers deployment、local preview、batch migration、import/export limit 一起審查。&lt;/p></description><content:encoded><![CDATA[<p>D1 / Turso / libSQL comparison 的核心責任是把 SQLite-compatible edge products 和 local SQLite 分開判讀。它們共享 SQLite 開發體驗的一部分，但它們承擔的服務責任不同：Cloudflare D1 把 SQLite-like database 放進 Workers 生態與 managed edge platform；Turso / libSQL 把 SQLite family 延伸到 remote primary、embedded replica 與同步模型；local SQLite 則是 application process 直接管理單一 database file。</p>
<p>本文的判讀錨點是：SQLite compatibility 代表開發入口接近，服務責任仍要重新審查。採用 edge SQLite 前，要先確認 write authority、read freshness、migration limit、backup evidence、observability、cost 與 vendor exit，而非只看 SQL 語法能否執行。</p>
<h2 id="product-boundary">Product Boundary</h2>
<p>Product boundary 的核心責任是定義誰持有資料、誰執行 SQL、誰負責恢復。Local SQLite 的資料在你的 filesystem；D1 的資料由 Cloudflare D1 平台管理並和 Workers binding 整合；Turso / libSQL 的資料通常有 remote database 與 client / embedded replica 的分工。</p>
<table>
  <thead>
      <tr>
          <th>選項</th>
          <th>主要責任</th>
          <th>適合情境</th>
          <th>關鍵審查點</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Local SQLite</td>
          <td>Process-local formal state</td>
          <td>CLI、desktop、single-node app</td>
          <td>file lifecycle、backup、WAL、lock</td>
      </tr>
      <tr>
          <td>Cloudflare D1</td>
          <td>Workers-integrated database</td>
          <td>edge app、serverless API、low ops</td>
          <td>platform limit、migration、binding</td>
      </tr>
      <tr>
          <td>Turso / libSQL</td>
          <td>Remote primary + replicas</td>
          <td>low-latency read、embedded replica</td>
          <td>freshness、sync、driver semantics</td>
      </tr>
      <tr>
          <td>Litestream / LiteFS</td>
          <td>Backup / replica operation</td>
          <td>single-node app with recovery / read</td>
          <td>RPO、RTO、primary ownership</td>
      </tr>
      <tr>
          <td>PostgreSQL</td>
          <td>Server SQL operation</td>
          <td>multi-tenant、central audit、HA、role</td>
          <td>operation team、PITR、schema gate</td>
      </tr>
  </tbody>
</table>
<p>Local SQLite 的判斷重點是 file ownership。若 app 與 database file 位於同一個 host，備份、restore、disk full、permission 與 app upgrade 都在你的 runbook 裡；這條路線承接 <a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">file lifecycle / backup boundary</a>。</p>
<p>D1 的判斷重點是 platform integration。Cloudflare 官方 D1 docs 把 D1 放在 Workers 與 Wrangler workflow 內，並公開 <a href="https://developers.cloudflare.com/d1/platform/limits/">D1 limits</a>；因此採用 D1 時要把 database decision 與 Workers deployment、local preview、batch migration、import/export limit 一起審查。</p>
<p>Turso / libSQL 的判斷重點是 replica freshness 與 client semantics。Turso docs 對 <a href="https://docs.turso.tech/features/embedded-replicas/introduction">embedded replicas</a> 的描述顯示：application 可以持有 local replica 並透過同步取得資料；這會把「讀得快」和「讀到多新」變成同一個設計問題。</p>
<h2 id="edge-data-model">Edge Data Model</h2>
<p>Edge data model 的核心責任是把 latency 改善與一致性責任拆開。Edge database 的價值常來自 closer read path、serverless deployment 與較低操作表面；風險則集中在 write authority、replication lag、region routing 與平台限制。</p>
<table>
  <thead>
      <tr>
          <th>問題</th>
          <th>要觀察的訊號</th>
          <th>設計含義</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>誰可以寫</td>
          <td>single primary、remote write、queue</td>
          <td>決定 conflict、retry、idempotency 設計</td>
      </tr>
      <tr>
          <td>讀取要多新</td>
          <td>read-after-write、sync interval</td>
          <td>決定 UI freshness、cache invalidation、fallback</td>
      </tr>
      <tr>
          <td>migration 怎麼跑</td>
          <td>CLI、batch limit、preview / prod gap</td>
          <td>決定 release gate 與 rollback plan</td>
      </tr>
      <tr>
          <td>失敗時如何恢復</td>
          <td>export、backup、restore command</td>
          <td>決定 RPO / RTO 與 vendor exit</td>
      </tr>
      <tr>
          <td>observability 在哪一層</td>
          <td>platform metrics、app log、query log</td>
          <td>決定 incident triage 從 app 還是 platform 開始查</td>
      </tr>
  </tbody>
</table>
<p>Write authority 是 edge SQLite 的第一個分水嶺。若所有 write 都集中到 remote primary，application 要處理 network error、retry、idempotency 與 read freshness；若 write 發生在 local replica，系統要有 conflict resolution、sync ordering 與 delete propagation。</p>
<p>Read locality 是 edge SQLite 的主要收益。它適合 session-local preference、read-mostly catalog、低風險 personalization、feature flag snapshot、tenant-local small dataset；這些情境的共同點是資料量小、write rate 低、freshness 可以定義。</p>
<p>Global transaction 是 edge SQLite 的高風險區。若產品需求包含跨 region balance transfer、inventory reservation、ledger posting、strongly consistent permission decision，設計應路由到 <a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">Global Distributed OLTP</a> 或 PostgreSQL / CockroachDB / Spanner 的 transactional model。</p>
<h2 id="migration-gap">Migration Gap</h2>
<p>Migration gap 的核心責任是確認 SQLite file 可以搬到 edge product 後，release workflow 仍可驗證。SQL syntax compatibility 只解決起點；真正會造成事故的是 batch limit、extension 差異、driver API、local preview 與 production platform 行為差異。</p>
<table>
  <thead>
      <tr>
          <th>差異面</th>
          <th>審查問題</th>
          <th>Evidence</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>SQL dialect</td>
          <td>schema、index、trigger、JSON 是否可用</td>
          <td>compatibility matrix + migration dry run</td>
      </tr>
      <tr>
          <td>Data movement</td>
          <td>seed / import / export 的容量與時間</td>
          <td>sample import、row count、checksum</td>
      </tr>
      <tr>
          <td>Runtime binding</td>
          <td>app 如何取得 database connection</td>
          <td>staging deployment + smoke test</td>
      </tr>
      <tr>
          <td>Transaction</td>
          <td>write path 是否跨 request / region</td>
          <td>failure injection、retry log、freshness test</td>
      </tr>
      <tr>
          <td>Backup / exit</td>
          <td>如何拿回 SQLite-compatible artifact</td>
          <td>export file、restore drill、retention note</td>
      </tr>
  </tbody>
</table>
<p>D1 migration 要把 Wrangler workflow 納入 release gate。Cloudflare D1 的 limits 文件明確列出 import、query、batch 等限制；因此大型 update / delete 要拆 batch，migration 要有 staging dry run 與 production rollback step。</p>
<p>Turso / libSQL migration 要把 driver semantics 納入 release gate。Local SQLite driver 直連 file；libSQL client 可能連 remote endpoint 或 embedded replica；application 要把 connection lifecycle、sync timing、auth token、network failure 與 local cache freshness 寫進測試。</p>
<h2 id="operational-model">Operational Model</h2>
<p>Operational model 的核心責任是把 managed convenience 轉成 ownership map。Edge SQLite 減少了部分 server operation，但新增 platform limit、billing、region behavior、vendor incident、CLI workflow 與 local preview mismatch。</p>
<p>Production runbook 至少要保存五種證據：</p>
<ol>
<li>Schema migration history 與每次 release 的 dry-run result。</li>
<li>Data import / export 指令、檔案大小、row count 與 checksum。</li>
<li>Region latency、read freshness、write error rate 與 retry count。</li>
<li>Platform limit 命中紀錄、batch policy 與成本警戒線。</li>
<li>Vendor exit route：回 local SQLite、PostgreSQL 或另一個 edge database 的最小搬遷步驟。</li>
</ol>
<p>成本模型要同時看 request、storage、egress、operation time 與工程鎖定。Edge product 常把起步成本壓低，但當資料變大、batch migration 變長、observability 需要外掛、vendor API 滲入 repository layer 時，長期成本會出現在 release 與 incident。</p>
<h2 id="decision-route">Decision Route</h2>
<p>Decision route 的核心責任是把需求送到相符的資料模型。D1 / Turso / libSQL 適合 edge locality 與低操作表面；當需求轉向 high-write OLTP、central audit、role-based permission、global transaction 或跨服務資料治理，應轉向 server SQL 或 distributed OLTP。</p>
<table>
  <thead>
      <tr>
          <th>需求訊號</th>
          <th>優先路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Workers app 需要小型 relational data</td>
          <td>Cloudflare D1 + explicit limits review</td>
      </tr>
      <tr>
          <td>App 需要 local read latency + remote sync</td>
          <td>Turso / libSQL + freshness contract</td>
      </tr>
      <tr>
          <td>Single-node app 只需要備份與恢復</td>
          <td>Local SQLite + <a href="/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS</a></td>
      </tr>
      <tr>
          <td>多 tenant、central audit、DB role</td>
          <td><a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a></td>
      </tr>
      <tr>
          <td>Global write consistency</td>
          <td><a href="/blog/backend/01-database/vendors/cockroachdb/" data-link-title="CockroachDB" data-link-desc="分散式 SQL、PostgreSQL 相容、跨區強一致、Spanner 的開源 / 跨雲替代">CockroachDB</a> 或 Spanner</td>
      </tr>
  </tbody>
</table>
<p>D1 的採用條件是 edge runtime 本身就是主平台。若 application 已在 Workers 上、資料量可控、query pattern 清楚、migration 可 batch，D1 可以把 database operation 融入 deployment workflow。</p>
<p>Turso / libSQL 的採用條件是 local read value 高於同步複雜度。若產品可明確定義 stale read window、write path 與 conflict policy，embedded replica 可以降低 latency；若使用者需要立即看見跨裝置變更，就要先設計 freshness evidence。</p>
<h2 id="production-tripwires">Production Tripwires</h2>
<p>Production tripwires 的核心責任是指出何時重新評估 edge SQLite。這些訊號出現時，系統通常已從「SQLite-compatible convenience」進入正式 database governance。</p>
<table>
  <thead>
      <tr>
          <th>Tripwire</th>
          <th>意義</th>
          <th>下一步</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Migration batch 經常碰 limit</td>
          <td>schema 與資料量超過 edge workflow</td>
          <td>評估 PostgreSQL / managed SQL</td>
      </tr>
      <tr>
          <td>Read freshness ticket 增加</td>
          <td>replica / sync 語意影響產品體驗</td>
          <td>建 freshness SLO 或改集中讀寫</td>
      </tr>
      <tr>
          <td>Export / restore 未演練</td>
          <td>vendor exit 與災難恢復缺 evidence</td>
          <td>補 restore drill 與 retention policy</td>
      </tr>
      <tr>
          <td>Driver API 滲入 domain</td>
          <td><a href="/blog/backend/knowledge-cards/vendor-lock-in/" data-link-title="Vendor Lock-In" data-link-desc="說明採用供應商產品後，其 API 與格式滲入程式碼造成的退出成本">vendor lock-in</a> 進入核心程式碼</td>
          <td>建 repository adapter 與 compatibility test</td>
      </tr>
      <tr>
          <td>Cross-region write 需求出現</td>
          <td>edge-local read 已不足</td>
          <td>路由到 distributed OLTP</td>
      </tr>
  </tbody>
</table>
<p>這些 tripwire 要寫進設計文件與 runbook。Edge SQLite 的優勢在於低摩擦起步；它的長期品質來自早期把 ownership、limits、exit 與 evidence 設計清楚。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>D1 / Turso / libSQL comparison 完成後，下一步要依壓力路由。要處理 local file 與 backup，讀 <a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">file lifecycle / backup boundary</a>；要處理 replica / restore，讀 <a href="/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS replication</a>；要從 local SQLite 移到 edge product，讀 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso migration</a>；要處理 global write，回到 <a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">Global Distributed OLTP</a>。</p>
]]></content:encoded></item><item><title>SQLite D1 / Turso Preview Lab</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/d1-turso-preview-lab/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/d1-turso-preview-lab/</guid><description>&lt;p>SQLite D1 / Turso preview lab 的核心責任是把 local SQLite 轉向 edge SQLite product 前的 compatibility gap 找出來。這篇承接 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL Comparison&lt;/a> 與 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso Migration&lt;/a>，把 edge migration 變成可回報的 query matrix。&lt;/p>
&lt;p>本文的驗收標準是：你能從 local SQLite 匯出 schema / seed，匯入 D1 或 Turso preview database，跑相同 query set，記錄 unsupported SQL、latency、error mapping 與 rollback route。&lt;/p>
&lt;h2 id="preview-scope">Preview Scope&lt;/h2>
&lt;p>Preview scope 的核心責任是把 lab 限制在 staging / preview。D1 與 Turso 都是平台產品，實際命令會依 CLI version、帳號、region 與專案設定改變；本文提供操作骨架與 evidence 格式，正式命令以官方文件為準。&lt;/p>
&lt;p>官方文件路由：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>產品&lt;/th>
 &lt;th>官方文件&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Cloudflare D1&lt;/td>
 &lt;td>&lt;a href="https://developers.cloudflare.com/d1/">Cloudflare D1 docs&lt;/a>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>D1 limits&lt;/td>
 &lt;td>&lt;a href="https://developers.cloudflare.com/d1/platform/limits/">Cloudflare D1 limits&lt;/a>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Turso&lt;/td>
 &lt;td>&lt;a href="https://docs.turso.tech/">Turso docs&lt;/a>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Turso embedded replicas&lt;/td>
 &lt;td>&lt;a href="https://docs.turso.tech/features/embedded-replicas/introduction">Embedded replicas&lt;/a>&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Preview lab 要先確認資料不含 production PII。若 seed data 來自正式資料，先做 masking 或 synthetic data。&lt;/p>
&lt;h2 id="export-local-sqlite">Export Local SQLite&lt;/h2>
&lt;p>Export local SQLite 的核心責任是建立 target platform 的 seed input。沿用 &lt;code>/tmp/sqlite-lab/app.db&lt;/code> 或 migration fixture。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">mkdir -p /tmp/sqlite-edge-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-edge-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">cp /tmp/sqlite-lab/app.db ./app.db
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;.schema&amp;#34;&lt;/span> &amp;gt; schema.sql
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">5&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;.dump&amp;#34;&lt;/span> &amp;gt; seed.sql
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">6&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;SELECT COUNT(*) FROM accounts;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">7&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;SELECT COUNT(*) FROM ledger_entries;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;code>schema.sql&lt;/code> 用來審查 DDL，&lt;code>seed.sql&lt;/code> 用來匯入 preview database。正式 migration 可能要拆 schema / data / index，並處理 target platform limits。&lt;/p>
&lt;h2 id="build-query-matrix">Build Query Matrix&lt;/h2>
&lt;p>Build query matrix 的核心責任是定義 preview 要驗證什麼。query set 要代表產品行為，而非只跑一個 &lt;code>SELECT 1&lt;/code>。&lt;/p></description><content:encoded><![CDATA[<p>SQLite D1 / Turso preview lab 的核心責任是把 local SQLite 轉向 edge SQLite product 前的 compatibility gap 找出來。這篇承接 <a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL Comparison</a> 與 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso Migration</a>，把 edge migration 變成可回報的 query matrix。</p>
<p>本文的驗收標準是：你能從 local SQLite 匯出 schema / seed，匯入 D1 或 Turso preview database，跑相同 query set，記錄 unsupported SQL、latency、error mapping 與 rollback route。</p>
<h2 id="preview-scope">Preview Scope</h2>
<p>Preview scope 的核心責任是把 lab 限制在 staging / preview。D1 與 Turso 都是平台產品，實際命令會依 CLI version、帳號、region 與專案設定改變；本文提供操作骨架與 evidence 格式，正式命令以官方文件為準。</p>
<p>官方文件路由：</p>
<table>
  <thead>
      <tr>
          <th>產品</th>
          <th>官方文件</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Cloudflare D1</td>
          <td><a href="https://developers.cloudflare.com/d1/">Cloudflare D1 docs</a></td>
      </tr>
      <tr>
          <td>D1 limits</td>
          <td><a href="https://developers.cloudflare.com/d1/platform/limits/">Cloudflare D1 limits</a></td>
      </tr>
      <tr>
          <td>Turso</td>
          <td><a href="https://docs.turso.tech/">Turso docs</a></td>
      </tr>
      <tr>
          <td>Turso embedded replicas</td>
          <td><a href="https://docs.turso.tech/features/embedded-replicas/introduction">Embedded replicas</a></td>
      </tr>
  </tbody>
</table>
<p>Preview lab 要先確認資料不含 production PII。若 seed data 來自正式資料，先做 masking 或 synthetic data。</p>
<h2 id="export-local-sqlite">Export Local SQLite</h2>
<p>Export local SQLite 的核心責任是建立 target platform 的 seed input。沿用 <code>/tmp/sqlite-lab/app.db</code> 或 migration fixture。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">mkdir -p /tmp/sqlite-edge-lab
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-edge-lab
</span></span><span class="line"><span class="ln">3</span><span class="cl">cp /tmp/sqlite-lab/app.db ./app.db
</span></span><span class="line"><span class="ln">4</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.schema&#34;</span> &gt; schema.sql
</span></span><span class="line"><span class="ln">5</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.dump&#34;</span> &gt; seed.sql
</span></span><span class="line"><span class="ln">6</span><span class="cl">sqlite3 app.db <span class="s2">&#34;SELECT COUNT(*) FROM accounts;&#34;</span>
</span></span><span class="line"><span class="ln">7</span><span class="cl">sqlite3 app.db <span class="s2">&#34;SELECT COUNT(*) FROM ledger_entries;&#34;</span></span></span></code></pre></div><p><code>schema.sql</code> 用來審查 DDL，<code>seed.sql</code> 用來匯入 preview database。正式 migration 可能要拆 schema / data / index，並處理 target platform limits。</p>
<h2 id="build-query-matrix">Build Query Matrix</h2>
<p>Build query matrix 的核心責任是定義 preview 要驗證什麼。query set 要代表產品行為，而非只跑一個 <code>SELECT 1</code>。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">Q1 list account balances
</span></span><span class="line"><span class="ln">2</span><span class="cl">Q2 insert ledger entry with unique idempotency key
</span></span><span class="line"><span class="ln">3</span><span class="cl">Q3 insert duplicate idempotency key and capture error
</span></span><span class="line"><span class="ln">4</span><span class="cl">Q4 foreign key violation
</span></span><span class="line"><span class="ln">5</span><span class="cl">Q5 transaction rollback
</span></span><span class="line"><span class="ln">6</span><span class="cl">Q6 pagination by created_at
</span></span><span class="line"><span class="ln">7</span><span class="cl">Q7 explain / performance sample if platform supports it</span></span></code></pre></div><p>這份 matrix 要保存 expected result。Local SQLite 先跑一次，把 row count、error category、latency baseline 記下來。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">.timer on
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">SELECT a.id, a.owner_name, SUM(l.amount_cents) AS balance_cents
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">FROM accounts a
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">JOIN ledger_entries l ON l.account_id = a.id
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">GROUP BY a.id, a.owner_name
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">ORDER BY a.id;
</span></span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><h2 id="import-to-d1-preview">Import to D1 Preview</h2>
<p>Import to D1 preview 的核心責任是驗證 Cloudflare D1 workflow。以下是操作骨架，正式命令與 flags 以 Cloudflare D1 docs 和 Wrangler 版本為準。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># Example shape only. Use your project naming and official Wrangler docs.</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">wrangler d1 create sqlite_edge_preview
</span></span><span class="line"><span class="ln">3</span><span class="cl">wrangler d1 execute sqlite_edge_preview --file<span class="o">=</span>seed.sql
</span></span><span class="line"><span class="ln">4</span><span class="cl">wrangler d1 execute sqlite_edge_preview --command<span class="o">=</span><span class="s2">&#34;SELECT COUNT(*) FROM accounts;&#34;</span></span></span></code></pre></div><p>D1 preview evidence 要記錄：</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>內容</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>CLI version</td>
          <td>Wrangler version、account / project</td>
      </tr>
      <tr>
          <td>Import log</td>
          <td>duration、file size、error</td>
      </tr>
      <tr>
          <td>Query result</td>
          <td>每個 query 的 row count / error</td>
      </tr>
      <tr>
          <td>Limit hit</td>
          <td>D1 limits 是否影響 seed 或 query</td>
      </tr>
      <tr>
          <td>Rollback</td>
          <td>刪除 preview DB 或重建 seed</td>
      </tr>
  </tbody>
</table>
<p>若 seed file 太大或某些 SQL 需要改寫，就把 gap 寫進 compatibility matrix，先保留 production migration 的審查邊界。</p>
<h2 id="import-to-turso-preview">Import to Turso Preview</h2>
<p>Import to Turso preview 的核心責任是驗證 remote database、client SDK 與 embedded replica 行為。以下是操作骨架，正式命令以 Turso docs 與 CLI version 為準。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># Example shape only. Use your org, group, region and official Turso docs.</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">turso db create sqlite-edge-preview
</span></span><span class="line"><span class="ln">3</span><span class="cl">turso db shell sqlite-edge-preview &lt; seed.sql
</span></span><span class="line"><span class="ln">4</span><span class="cl">turso db shell sqlite-edge-preview <span class="s2">&#34;SELECT COUNT(*) FROM accounts;&#34;</span></span></span></code></pre></div><p>Turso preview evidence 要多記 replica freshness。若使用 embedded replica，測試流程要包含 bootstrap、sync、read query、write delegation 與 sync 後 read。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">embedded replica evidence:
</span></span><span class="line"><span class="ln">2</span><span class="cl">  bootstrap duration
</span></span><span class="line"><span class="ln">3</span><span class="cl">  first read latency
</span></span><span class="line"><span class="ln">4</span><span class="cl">  write path
</span></span><span class="line"><span class="ln">5</span><span class="cl">  sync command / interval
</span></span><span class="line"><span class="ln">6</span><span class="cl">  read freshness after write</span></span></code></pre></div><p>Freshness 是 product decision。若 query matrix 只測 remote primary，仍需要追加 embedded replica 的使用者體驗驗證。</p>
<h2 id="compatibility-matrix">Compatibility Matrix</h2>
<p>Compatibility matrix 的核心責任是把 local SQLite 與 edge target 的差異留下來。建議表格欄位如下：</p>
<table>
  <thead>
      <tr>
          <th>Query / operation</th>
          <th>Local SQLite</th>
          <th>D1 preview</th>
          <th>Turso preview</th>
          <th>Decision</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Balance list</td>
          <td>pass</td>
          <td>pass / diff</td>
          <td>pass / diff</td>
          <td>keep / rewrite</td>
      </tr>
      <tr>
          <td>Unique violation</td>
          <td>error class</td>
          <td>error class</td>
          <td>error class</td>
          <td>map error</td>
      </tr>
      <tr>
          <td>FK violation</td>
          <td>error class</td>
          <td>error class</td>
          <td>error class</td>
          <td>enable / validate</td>
      </tr>
      <tr>
          <td>Transaction rollback</td>
          <td>pass</td>
          <td>pass / diff</td>
          <td>pass / diff</td>
          <td>rewrite workflow</td>
      </tr>
      <tr>
          <td>Import seed</td>
          <td>pass</td>
          <td>duration / limit</td>
          <td>duration / limit</td>
          <td>split batch</td>
      </tr>
  </tbody>
</table>
<p>Decision 欄要寫具體下一步。<code>rewrite workflow</code> 代表 application adapter 要改；<code>split batch</code> 代表 migration runbook 要改；<code>map error</code> 代表 repository error classification 要改。</p>
<h2 id="latency-and-cost-sample">Latency and Cost Sample</h2>
<p>Latency and cost sample 的核心責任是避免只看功能相容。Edge SQLite migration 的收益常來自 region latency 或 managed operation，因此 preview 要量測主要使用者區域的 read / write latency。</p>
<p>最小量測：</p>
<ol>
<li>Local baseline latency。</li>
<li>Preview target read latency。</li>
<li>Preview target write latency。</li>
<li>Error rate / retry count。</li>
<li>Estimated request / storage / egress cost。</li>
</ol>
<p>Latency sample 要搭配 freshness。快速讀到舊資料和稍慢讀到最新資料是不同產品體驗；query matrix 要標註哪個 workflow 可以接受 stale read。</p>
<h2 id="rollback-route">Rollback Route</h2>
<p>Rollback route 的核心責任是保留 local SQLite 退路。Preview lab 完成後，要能刪除 preview database、保留 local seed、重跑 local app。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;SELECT COUNT(*) FROM accounts;&#34;</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">sqlite3 app.db <span class="s2">&#34;SELECT COUNT(*) FROM ledger_entries;&#34;</span></span></span></code></pre></div><p>正式 cutover 的 rollback 還要處理 target-only writes。Preview 階段應避免讓真實使用者寫入 target；若需要 shadow traffic，先用 read-only 或 synthetic write。</p>
<h2 id="completion-note">Completion Note</h2>
<p>Completion note 的核心責任是決定是否進入正式 migration。Lab 完成後應輸出四個 artifact：<code>seed.sql</code>、import log、compatibility matrix、rollback note。</p>
<p>進入正式 migration 的條件：</p>
<ol>
<li>Query matrix 主要 workflow 通過或已有 rewrite plan。</li>
<li>Platform limits 對資料量與 migration time 可接受。</li>
<li>Error mapping 已接到 repository adapter。</li>
<li>Freshness / latency 符合產品需求。</li>
<li>Export / rollback route 已演練。</li>
</ol>
<p>完成本篇後，回到 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso Migration</a> 補正式 phase plan。</p>
]]></content:encoded></item><item><title>SQLite file lifecycle 與 backup boundary</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 embedded、local-first、edge 與低操作成本場景；本文聚焦 &lt;em>SQLite 檔案生命週期 + backup / restore 邊界&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite 的 file lifecycle 是把「一個資料庫檔案」升級成正式狀態的操作契約。SQLite 省掉 server process、帳號管理與網路連線，但它把 durability、backup、restore、locking 與 corruption recovery 放回 application process、filesystem 與 runbook；讀者要判斷的是這些責任是否已經有人承擔。&lt;/p>
&lt;p>這篇文章適合三種情境。第一種是 CLI、desktop、mobile 或 edge service 已經用 SQLite 保存正式資料；第二種是 single-instance backend 想用 SQLite 降低操作成本；第三種是 test fixture 用 SQLite，但需要知道哪些差異會讓 production database 的 bug 漏掉。&lt;/p>
&lt;h2 id="核心模型資料庫檔案是一組受-sqlite-管理的狀態檔">核心模型：資料庫檔案是一組受 SQLite 管理的狀態檔&lt;/h2>
&lt;p>SQLite 的資料庫狀態由 main database file 與 journal / WAL sidecar 共同構成。Rollback journal mode 會在寫入期間產生 journal file；WAL mode 會讓寫入先進入 &lt;code>-wal&lt;/code> 檔，並用 &lt;code>-shm&lt;/code> 檔協調 reader / writer。操作上看似「一個 &lt;code>.db&lt;/code> 檔」，production runbook 要把 sidecar file、checkpoint、backup API 與 restore test 一起納入。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>檔案 / 機制&lt;/th>
 &lt;th>服務責任&lt;/th>
 &lt;th>操作判讀&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>.db&lt;/code>&lt;/td>
 &lt;td>持久化資料、schema、index&lt;/td>
 &lt;td>file owner、permission、storage durability、snapshot 位置&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>-wal&lt;/code>&lt;/td>
 &lt;td>WAL mode 下尚未 checkpoint 的寫入&lt;/td>
 &lt;td>WAL growth、checkpoint cadence、backup 是否包含一致快照&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>-shm&lt;/code>&lt;/td>
 &lt;td>WAL index 與跨 connection 協調&lt;/td>
 &lt;td>local filesystem lock 是否可靠、部署是否跨 process 共用檔案&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>checkpoint&lt;/td>
 &lt;td>把 WAL 內容合併回 main database&lt;/td>
 &lt;td>checkpoint latency、writer pause、檔案大小是否持續膨脹&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>backup API&lt;/td>
 &lt;td>線上複製一致 snapshot&lt;/td>
 &lt;td>backup 是否在 application 還活著時仍能取得一致狀態&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的讀法是先找「誰有權改檔案」。SQLite 的核心風險多半來自繞過 SQLite library 的檔案操作，例如直接 copy 活躍 WAL database、把 database 放在 lock 語意不可靠的 filesystem、或讓多個不協調的 process 同時寫同一份檔案。&lt;/p>
&lt;h2 id="wal-mode讀取並發提升後writer-boundary-仍然存在">WAL mode：讀取並發提升後，writer boundary 仍然存在&lt;/h2>
&lt;p>WAL mode 的工程價值是讓 reader 與 writer 的衝突下降。讀取可以看 main database 加上 WAL 中的 snapshot，寫入則 append 到 WAL；這讓 read-heavy workload 比 rollback journal mode 更容易撐住互動式服務。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 embedded、local-first、edge 與低操作成本場景；本文聚焦 <em>SQLite 檔案生命週期 + backup / restore 邊界</em>。</p></blockquote>
<p>SQLite 的 file lifecycle 是把「一個資料庫檔案」升級成正式狀態的操作契約。SQLite 省掉 server process、帳號管理與網路連線，但它把 durability、backup、restore、locking 與 corruption recovery 放回 application process、filesystem 與 runbook；讀者要判斷的是這些責任是否已經有人承擔。</p>
<p>這篇文章適合三種情境。第一種是 CLI、desktop、mobile 或 edge service 已經用 SQLite 保存正式資料；第二種是 single-instance backend 想用 SQLite 降低操作成本；第三種是 test fixture 用 SQLite，但需要知道哪些差異會讓 production database 的 bug 漏掉。</p>
<h2 id="核心模型資料庫檔案是一組受-sqlite-管理的狀態檔">核心模型：資料庫檔案是一組受 SQLite 管理的狀態檔</h2>
<p>SQLite 的資料庫狀態由 main database file 與 journal / WAL sidecar 共同構成。Rollback journal mode 會在寫入期間產生 journal file；WAL mode 會讓寫入先進入 <code>-wal</code> 檔，並用 <code>-shm</code> 檔協調 reader / writer。操作上看似「一個 <code>.db</code> 檔」，production runbook 要把 sidecar file、checkpoint、backup API 與 restore test 一起納入。</p>
<table>
  <thead>
      <tr>
          <th>檔案 / 機制</th>
          <th>服務責任</th>
          <th>操作判讀</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>.db</code></td>
          <td>持久化資料、schema、index</td>
          <td>file owner、permission、storage durability、snapshot 位置</td>
      </tr>
      <tr>
          <td><code>-wal</code></td>
          <td>WAL mode 下尚未 checkpoint 的寫入</td>
          <td>WAL growth、checkpoint cadence、backup 是否包含一致快照</td>
      </tr>
      <tr>
          <td><code>-shm</code></td>
          <td>WAL index 與跨 connection 協調</td>
          <td>local filesystem lock 是否可靠、部署是否跨 process 共用檔案</td>
      </tr>
      <tr>
          <td>checkpoint</td>
          <td>把 WAL 內容合併回 main database</td>
          <td>checkpoint latency、writer pause、檔案大小是否持續膨脹</td>
      </tr>
      <tr>
          <td>backup API</td>
          <td>線上複製一致 snapshot</td>
          <td>backup 是否在 application 還活著時仍能取得一致狀態</td>
      </tr>
  </tbody>
</table>
<p>這張表的讀法是先找「誰有權改檔案」。SQLite 的核心風險多半來自繞過 SQLite library 的檔案操作，例如直接 copy 活躍 WAL database、把 database 放在 lock 語意不可靠的 filesystem、或讓多個不協調的 process 同時寫同一份檔案。</p>
<h2 id="wal-mode讀取並發提升後writer-boundary-仍然存在">WAL mode：讀取並發提升後，writer boundary 仍然存在</h2>
<p>WAL mode 的工程價值是讓 reader 與 writer 的衝突下降。讀取可以看 main database 加上 WAL 中的 snapshot，寫入則 append 到 WAL；這讓 read-heavy workload 比 rollback journal mode 更容易撐住互動式服務。</p>
<p>WAL mode 同時保留 single writer boundary。SQLite 仍以檔案鎖與 transaction serialisation 控制寫入；寫入交易越長，其他 writer 等待時間越長，application 看到的訊號通常是 <code>SQLITE_BUSY</code>、latency spike 或 background job 卡住。</p>
<table>
  <thead>
      <tr>
          <th>訊號</th>
          <th>常見原因</th>
          <th>第一輪處理</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>SQLITE_BUSY</code> 增加</td>
          <td>長交易、background migration、慢 disk</td>
          <td>縮短 write transaction、加 busy timeout、把批次寫入切小</td>
      </tr>
      <tr>
          <td><code>-wal</code> 檔持續變大</td>
          <td>checkpoint 追不上、long reader 卡住</td>
          <td>找出長讀取、調整 checkpoint cadence、把 analytics query 移出路徑</td>
      </tr>
      <tr>
          <td>restore 後資料落差</td>
          <td>backup 沒取得一致 snapshot</td>
          <td>改用 <code>.backup</code> / backup API / <code>VACUUM INTO</code>，並演練 restore</td>
      </tr>
      <tr>
          <td>latency 受 fsync 拉高</td>
          <td><code>synchronous=FULL</code> + 高寫入頻率</td>
          <td>重新定義 durability 需求，評估 server SQL 或 managed service</td>
      </tr>
  </tbody>
</table>
<p>WAL mode 的 capacity gate 是「寫入是否仍能用一個 writer 排隊」。如果服務壓力來自大量並行寫入、多 instance active write 或跨 region 寫入，SQLite 的簡單性開始變成排隊與恢復成本；這時候要回到 <a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a>、<a href="/blog/backend/01-database/vendors/mysql/" data-link-title="MySQL" data-link-desc="高併發網路服務常用關聯式資料庫、Vitess / PlanetScale 分片生態、GitHub / Shopify / Facebook 規模驗證">MySQL</a> 或 <a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">global distributed OLTP</a>。</p>
<h2 id="backup-boundary複製檔案與取得一致-snapshot-是兩件事">Backup boundary：複製檔案與取得一致 snapshot 是兩件事</h2>
<p>SQLite backup 的核心責任是取得某一時間點的一致 snapshot。當 database live 且 WAL mode 開啟時，直接複製 <code>.db</code> 檔容易漏掉 <code>-wal</code> 中尚未 checkpoint 的寫入；即使同時複製 sidecar file，也要面對複製期間狀態變動的 race。正式服務應使用 SQLite 提供的 backup path 或可驗證的 filesystem snapshot。</p>
<table>
  <thead>
      <tr>
          <th>方法</th>
          <th>適合情境</th>
          <th>邊界</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>.backup</code> / Backup API</td>
          <td>live database、application 仍在服務</td>
          <td>SQLite 管理 source lock，產出開始備份時的一致 snapshot</td>
      </tr>
      <tr>
          <td><code>VACUUM INTO</code></td>
          <td>想同時 compact + 輸出新檔</td>
          <td>需要 I/O 空間與時間，適合 maintenance 或低流量窗口</td>
      </tr>
      <tr>
          <td>filesystem snapshot</td>
          <td>VM / volume 層已有一致 snapshot 能力</td>
          <td>要確認 snapshot 包含 main file 與 WAL sidecar，且 lock 語意清楚</td>
      </tr>
      <tr>
          <td>Litestream</td>
          <td>single-primary SQLite 的持續備份</td>
          <td>適合 DR / restore，不把 SQLite 變成 multi-primary database</td>
      </tr>
      <tr>
          <td>手動 <code>cp</code></td>
          <td>database 已關閉或已完成 checkpoint</td>
          <td>live WAL database 的一致性風險高，production runbook 應改路由</td>
      </tr>
  </tbody>
</table>
<p>Backup method 的選擇要先回到 <a href="/blog/backend/knowledge-cards/rpo/" data-link-title="RPO" data-link-desc="說明恢復點目標如何定義可接受資料損失範圍">RPO</a> 與 <a href="/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO</a>。如果產品可以接受每天一次快照，<code>VACUUM INTO</code> 或 scheduled backup 足夠；如果資料損失窗口要降到分鐘級或秒級，就要看 Litestream 類連續複製，或直接升級到 server database 的 PITR / replica 模型。</p>
<h2 id="restore-drillsqlite-production-readiness-看還原不只看備份成功">Restore drill：SQLite production readiness 看還原，不只看備份成功</h2>
<p>Restore drill 的責任是證明備份能在事故時接回服務。SQLite 的備份檔通常只有一個 target file，表面上比 PostgreSQL PITR 或 MySQL binlog recovery 簡單；真正的風險在 application binary、schema migration version、file permission、deployment path 與舊 WAL sidecar 是否一起對齊。</p>
<p>一個最小 restore drill 應保留五個檢查點：</p>
<ol>
<li>從備份產出新的 database file，不覆蓋 production path。</li>
<li>用 application binary 啟動 read-only smoke test，確認 schema version 與 migration table。</li>
<li>跑 row count、critical query、checksum 或 domain validation query。</li>
<li>驗證 file owner、permission、disk path、SELinux / container mount 或 volume 設定。</li>
<li>以 incident decision log 記錄 restore time、data freshness、known gap 與 owner。</li>
</ol>
<p>Restore drill 的交付物應接回 <a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">Observability Evidence Package</a> 與 <a href="/blog/backend/08-incident-response/incident-decision-log/" data-link-title="8.19 Incident Decision Log" data-link-desc="把事中假設、決策、證據、回退條件與責任人留下可復盤紀錄">Incident Decision Log</a>。SQLite 的低操作成本來自日常元件少；事故時仍需要 evidence、owner 與 rollback condition。</p>
<h2 id="corruption-recovery先保全證據再決定修復或還原">Corruption recovery：先保全證據，再決定修復或還原</h2>
<p>SQLite <a href="/blog/backend/knowledge-cards/corruption-recovery/" data-link-title="Corruption Recovery" data-link-desc="說明資料損毀事故如何先辨識來源、保全證據，再決定修復或還原">corruption recovery</a> 的核心責任是區分「資料庫檔案本身受損」與「application 寫入了錯誤資料」。前者要走 file-level evidence、<code>.recover</code>、backup restore 與 filesystem / hardware investigation；後者要走資料修復、migration rollback 或 business reconciliation。</p>
<table>
  <thead>
      <tr>
          <th>觀察訊號</th>
          <th>優先判讀</th>
          <th>下一步路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>SQLITE_CORRUPT</code></td>
          <td>database page / btree 受損</td>
          <td>複製原檔保存證據、用 <code>.recover</code> 嘗試導出、從最近 backup 建新檔</td>
      </tr>
      <tr>
          <td>power loss 後啟動異常</td>
          <td>journal / WAL recovery 問題</td>
          <td>確認 sidecar file 是否仍在、檢查 storage sync 與 <code>synchronous</code> 設定</td>
      </tr>
      <tr>
          <td>restore 後 business data 錯誤</td>
          <td>備份點或 migration 錯誤</td>
          <td>對照 validation query、migration log、事件補償與 <a href="/blog/backend/01-database/reconciliation-data-repair/" data-link-title="1.9 Reconciliation 與 Data Repair" data-link-desc="資料不一致的分類、偵測模式、修復策略、audit trail、跟 backup / PITR 整合">reconciliation</a></td>
      </tr>
      <tr>
          <td>network filesystem 上偶發錯誤</td>
          <td>lock 語意與 filesystem 問題</td>
          <td>把 SQLite 移回 local disk，或升級 server database</td>
      </tr>
  </tbody>
</table>
<p>Corruption 事件的第一個操作是保存原始檔案與 sidecar。直接在疑似受損檔案上跑修復、vacuum 或 application migration，會讓後續 root cause analysis 失去證據；比較穩定的流程是複製原檔、在副本上嘗試 <code>.recover</code>，同時從備份恢復服務路徑。</p>
<h2 id="anti-recommendation維持-sqlite-的條件要可被操作驗證">Anti-recommendation：維持 SQLite 的條件要可被操作驗證</h2>
<p>SQLite 的合理使用條件是「單一 writer、檔案生命週期清楚、restore drill 成立」。只要這三件事能被 runbook 驗證，SQLite 在 embedded、desktop、mobile、edge-local 或 small backend 場景可以是 production state。</p>
<p>升級條件則來自操作責任外溢。需要 database user / role、中心化 audit、多人同時寫、跨 instance failover、online schema migration、PITR、read replica 或跨 region transaction 時，server SQL 或 managed SQL 的操作模型會比繼續包裝 SQLite 清楚。</p>
<table>
  <thead>
      <tr>
          <th>目前壓力</th>
          <th>留在 SQLite 的條件</th>
          <th>升級路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>read-heavy local store</td>
          <td>WAL + restore drill 成立</td>
          <td>維持 SQLite，補 observability 與 backup evidence</td>
      </tr>
      <tr>
          <td>single-instance backend</td>
          <td>writer queue 可接受、RPO / RTO 明確</td>
          <td>SQLite + Litestream；或升級 PostgreSQL / MySQL</td>
      </tr>
      <tr>
          <td>edge / serverless</td>
          <td>平台已提供 SQLite-compatible 運作模型</td>
          <td>Cloudflare D1 / Turso；跨 region transaction 回到 global DB</td>
      </tr>
      <tr>
          <td>multi-tenant SaaS</td>
          <td>tenant 數少且 file ownership 清楚</td>
          <td>PostgreSQL / Aurora / CockroachDB</td>
      </tr>
      <tr>
          <td>regulated data</td>
          <td>backup encryption、audit、restore 可驗證</td>
          <td>PostgreSQL / managed SQL + audit / PITR</td>
      </tr>
  </tbody>
</table>
<p>這張表的核心是把操作責任具體化，而非替 SQLite 設流量天花板。小型服務可能用 SQLite 長期穩定運作；同樣流量下，一旦合規、稽核、多人操作或 HA 需求進來，server database 的長期成本會更容易被治理。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>SQLite production runbook 至少要能回答下列問題：</p>
<ol>
<li>Database file、WAL sidecar 與 backup target 在哪個 volume、由誰擁有。</li>
<li><code>journal_mode</code>、<code>synchronous</code>、busy timeout、checkpoint cadence 與 migration policy 如何設定。</li>
<li>Backup 用 <code>.backup</code> / backup API / <code>VACUUM INTO</code> / Litestream 的哪一條路徑。</li>
<li>Restore drill 最近一次何時執行，RPO / RTO 是否符合產品承諾。</li>
<li><code>SQLITE_BUSY</code>、WAL growth、disk full、backup failure 與 restore failure 如何告警。</li>
<li>Corruption recovery 時誰保存原檔、誰啟動 restore、誰決定修復或 fail-forward。</li>
</ol>
<p>這份清單要接到服務 ownership，而非留在工程師個人習慣。SQLite 的優勢是 deployment surface 小；production 化的代價是把檔案、備份與恢復流程寫進同一份可交接 runbook。</p>
<h2 id="引用路徑">引用路徑</h2>
<ul>
<li>上游 overview：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite vendor page</a></li>
<li>服務責任：<a href="/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">Source of Truth</a>、<a href="/blog/backend/knowledge-cards/database/" data-link-title="Database" data-link-desc="說明 database 在後端系統中如何承擔正式狀態、查詢與一致性責任">Database</a></li>
<li>恢復目標：<a href="/blog/backend/knowledge-cards/rpo/" data-link-title="RPO" data-link-desc="說明恢復點目標如何定義可接受資料損失範圍">RPO</a>、<a href="/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO</a></li>
<li>證據交接：<a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">Observability Evidence Package</a>、<a href="/blog/backend/08-incident-response/incident-decision-log/" data-link-title="8.19 Incident Decision Log" data-link-desc="把事中假設、決策、證據、回退條件與責任人留下可復盤紀錄">Incident Decision Log</a></li>
<li>官方文件：<a href="https://www.sqlite.org/wal.html">SQLite Write-Ahead Logging</a>、<a href="https://www.sqlite.org/backup.html">SQLite Backup API</a>、<a href="https://www.sqlite.org/howtocorrupt.html">How To Corrupt An SQLite Database File</a>、<a href="https://www.sqlite.org/recovery.html">Recovering Data From A Corrupt SQLite Database</a>、<a href="https://www.sqlite.org/whentouse.html">Appropriate Uses For SQLite</a>、<a href="https://www.sqlite.org/mostdeployed.html">Most Widely Deployed SQL Database Engine</a></li>
<li>延伸工具：<a href="https://litestream.io/reference/restore/">Litestream restore reference</a>、<a href="https://litestream.io/getting-started/">Litestream getting started</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Hands-on 操作路線</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/</guid><description>&lt;p>SQLite hands-on 操作路線的核心責任是把單檔正式狀態轉成可演練流程。這一層對齊 LLM &lt;code>hands-on/&lt;/code>：讀者能建立一個 SQLite 檔案、製造 WAL / lock 訊號、跑 backup / restore、套 migration，並知道何時該升級到 server SQL 或 edge SQLite。&lt;/p>
&lt;h2 id="章節列表">章節列表&lt;/h2>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>章節&lt;/th>
 &lt;th>主題&lt;/th>
 &lt;th>產出 artifact&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;a href="local-file-quickstart/">Local file quickstart&lt;/a>&lt;/td>
 &lt;td>建立 &lt;code>.db&lt;/code>、schema、seed data、basic query&lt;/td>
 &lt;td>database file、schema version、query sample&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;a href="backup-restore-drill/">Backup restore drill&lt;/a>&lt;/td>
 &lt;td>&lt;code>.backup&lt;/code> / &lt;code>VACUUM INTO&lt;/code> / restore validation&lt;/td>
 &lt;td>backup file、restore record、validation query&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;a href="wal-busy-reproduction/">WAL busy reproduction&lt;/a>&lt;/td>
 &lt;td>long transaction、&lt;code>SQLITE_BUSY&lt;/code>、checkpoint growth&lt;/td>
 &lt;td>busy error sample、WAL size evidence&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;a href="migration-fixture-lab/">Migration fixture lab&lt;/a>&lt;/td>
 &lt;td>&lt;code>user_version&lt;/code>、table rebuild、fixture snapshot&lt;/td>
 &lt;td>migration log、fixture DB、rollback note&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;a href="d1-turso-preview-lab/">D1 / Turso preview lab&lt;/a>&lt;/td>
 &lt;td>local SQLite 到 edge SQLite product 的 compatibility preview&lt;/td>
 &lt;td>export / import note、compatibility gap&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h2 id="設計原則">設計原則&lt;/h2>
&lt;p>SQLite hands-on 章節要以檔案生命週期為中心。操作指令只在能產出 evidence 時出現；每篇都要回答 database file 在哪裡、sidecar file 如何處理、restore 如何驗證，以及 application release 如何知道它仍相容。&lt;/p>
&lt;h2 id="引用路徑">引用路徑&lt;/h2>
&lt;ul>
&lt;li>上游：&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview&lt;/a>&lt;/li>
&lt;li>Structure：&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/teaching-structure/" data-link-title="SQLite Teaching Structure" data-link-desc="SQLite 服務章節群的大綱：從 embedded formal state、WAL、backup、test fixture、local-first、edge SQLite 到遷移路由">SQLite Teaching Structure&lt;/a>&lt;/li>
&lt;li>Deep article：&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary&lt;/a>&lt;/li>
&lt;/ul></description><content:encoded><![CDATA[<p>SQLite hands-on 操作路線的核心責任是把單檔正式狀態轉成可演練流程。這一層對齊 LLM <code>hands-on/</code>：讀者能建立一個 SQLite 檔案、製造 WAL / lock 訊號、跑 backup / restore、套 migration，並知道何時該升級到 server SQL 或 edge SQLite。</p>
<h2 id="章節列表">章節列表</h2>
<table>
  <thead>
      <tr>
          <th>章節</th>
          <th>主題</th>
          <th>產出 artifact</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><a href="local-file-quickstart/">Local file quickstart</a></td>
          <td>建立 <code>.db</code>、schema、seed data、basic query</td>
          <td>database file、schema version、query sample</td>
      </tr>
      <tr>
          <td><a href="backup-restore-drill/">Backup restore drill</a></td>
          <td><code>.backup</code> / <code>VACUUM INTO</code> / restore validation</td>
          <td>backup file、restore record、validation query</td>
      </tr>
      <tr>
          <td><a href="wal-busy-reproduction/">WAL busy reproduction</a></td>
          <td>long transaction、<code>SQLITE_BUSY</code>、checkpoint growth</td>
          <td>busy error sample、WAL size evidence</td>
      </tr>
      <tr>
          <td><a href="migration-fixture-lab/">Migration fixture lab</a></td>
          <td><code>user_version</code>、table rebuild、fixture snapshot</td>
          <td>migration log、fixture DB、rollback note</td>
      </tr>
      <tr>
          <td><a href="d1-turso-preview-lab/">D1 / Turso preview lab</a></td>
          <td>local SQLite 到 edge SQLite product 的 compatibility preview</td>
          <td>export / import note、compatibility gap</td>
      </tr>
  </tbody>
</table>
<h2 id="設計原則">設計原則</h2>
<p>SQLite hands-on 章節要以檔案生命週期為中心。操作指令只在能產出 evidence 時出現；每篇都要回答 database file 在哪裡、sidecar file 如何處理、restore 如何驗證，以及 application release 如何知道它仍相容。</p>
<h2 id="引用路徑">引用路徑</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview</a></li>
<li>Structure：<a href="/blog/backend/01-database/vendors/sqlite/teaching-structure/" data-link-title="SQLite Teaching Structure" data-link-desc="SQLite 服務章節群的大綱：從 embedded formal state、WAL、backup、test fixture、local-first、edge SQLite 到遷移路由">SQLite Teaching Structure</a></li>
<li>Deep article：<a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Litestream / LiteFS Replication</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/</guid><description>&lt;p>Litestream / LiteFS replication 的核心責任是把 SQLite 的 single-file operation 補成可恢復、可部署、可讀擴展的服務形狀。這類工具延伸 SQLite，但它們解決的問題不同：Litestream 主要把 WAL 變化持續送到 replica storage，強化 backup 與 restore；LiteFS 主要在 Fly.io 生態中透過 primary lease 與 filesystem layer 支援 replicated SQLite deployment。&lt;/p>
&lt;p>本文的判讀錨點是：replicated SQLite 要先說明 replica 的服務責任。它可能是 continuous backup、warm restore source、read replica、primary failover helper 或 deployment topology；每一種責任都有不同的 RPO、RTO、freshness 與 incident runbook。&lt;/p>
&lt;h2 id="replication-taxonomy">Replication Taxonomy&lt;/h2>
&lt;p>Replication taxonomy 的核心責任是把「有複本」拆成可操作的幾種能力。SQLite 周邊工具常用 replication 這個字，但 operator 需要知道它到底保護哪個風險。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>類型&lt;/th>
 &lt;th>主要責任&lt;/th>
 &lt;th>成功訊號&lt;/th>
 &lt;th>常見誤判&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Continuous backup&lt;/td>
 &lt;td>降低資料遺失窗口&lt;/td>
 &lt;td>replica lag、restore 成功&lt;/td>
 &lt;td>把 replica 當 active-active database&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Read replica&lt;/td>
 &lt;td>降低 read latency / 壓力&lt;/td>
 &lt;td>freshness、read error rate&lt;/td>
 &lt;td>忽略 stale read&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Warm standby&lt;/td>
 &lt;td>縮短 restore / failover&lt;/td>
 &lt;td>promotion drill、DNS / routing&lt;/td>
 &lt;td>只備份檔案、未演練切換&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Primary lease&lt;/td>
 &lt;td>控制單一 writer ownership&lt;/td>
 &lt;td>writer lease、fencing log&lt;/td>
 &lt;td>多個 node 同時寫同一份邏輯狀態&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Consensus SQL&lt;/td>
 &lt;td>多節點一致性寫入&lt;/td>
 &lt;td>quorum、leader election&lt;/td>
 &lt;td>用 WAL shipping 取代 distributed OLTP&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Continuous backup 的語言是 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/rpo/" data-link-title="RPO" data-link-desc="說明恢復點目標如何定義可接受資料損失範圍">RPO&lt;/a> 與 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO&lt;/a>。它關心最近一次成功送出的 WAL、snapshot freshness、object storage credential、restore 指令與演練結果。&lt;/p>
&lt;p>Read replica 的語言是 freshness。Replica 能降低 read latency 或保護 primary workload，但讀者要知道 stale window、read-after-write policy、fallback to primary 與 cache invalidation。&lt;/p>
&lt;p>Primary lease 的語言是 writer ownership。SQLite 的服務形狀仍適合 single writer；工具可以協助 deployment 切換，但 application 要配合 fencing、retry 與 promotion evidence。&lt;/p>
&lt;h2 id="litestream-boundary">Litestream Boundary&lt;/h2>
&lt;p>Litestream boundary 的核心責任是把 SQLite WAL 變成可持續複製的 backup stream。Litestream 官方說明把它定位為 SQLite streaming replication tool，並在 &lt;a href="https://litestream.io/how-it-works/">How it works&lt;/a> 與 &lt;a href="https://litestream.io/reference/restore/">restore command&lt;/a> 文件中強調 replica 與 restore workflow。&lt;/p>
&lt;p>Litestream 適合下列情境：&lt;/p>
&lt;ol>
&lt;li>單節點 SQLite app 要降低資料遺失窗口。&lt;/li>
&lt;li>系統可接受 restore 後重新啟動 service。&lt;/li>
&lt;li>Object storage credential、retention、restore drill 可以被管理。&lt;/li>
&lt;li>Write pattern 適中，WAL stream 與 snapshot 維護成本可控。&lt;/li>
&lt;/ol>
&lt;p>Litestream 的設計重點是 backup evidence。Runbook 要記錄 replica destination、last replicated generation、last restore test、expected RPO、expected RTO、restore target path、credential rotation 與 corruption triage。&lt;/p></description><content:encoded><![CDATA[<p>Litestream / LiteFS replication 的核心責任是把 SQLite 的 single-file operation 補成可恢復、可部署、可讀擴展的服務形狀。這類工具延伸 SQLite，但它們解決的問題不同：Litestream 主要把 WAL 變化持續送到 replica storage，強化 backup 與 restore；LiteFS 主要在 Fly.io 生態中透過 primary lease 與 filesystem layer 支援 replicated SQLite deployment。</p>
<p>本文的判讀錨點是：replicated SQLite 要先說明 replica 的服務責任。它可能是 continuous backup、warm restore source、read replica、primary failover helper 或 deployment topology；每一種責任都有不同的 RPO、RTO、freshness 與 incident runbook。</p>
<h2 id="replication-taxonomy">Replication Taxonomy</h2>
<p>Replication taxonomy 的核心責任是把「有複本」拆成可操作的幾種能力。SQLite 周邊工具常用 replication 這個字，但 operator 需要知道它到底保護哪個風險。</p>
<table>
  <thead>
      <tr>
          <th>類型</th>
          <th>主要責任</th>
          <th>成功訊號</th>
          <th>常見誤判</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Continuous backup</td>
          <td>降低資料遺失窗口</td>
          <td>replica lag、restore 成功</td>
          <td>把 replica 當 active-active database</td>
      </tr>
      <tr>
          <td>Read replica</td>
          <td>降低 read latency / 壓力</td>
          <td>freshness、read error rate</td>
          <td>忽略 stale read</td>
      </tr>
      <tr>
          <td>Warm standby</td>
          <td>縮短 restore / failover</td>
          <td>promotion drill、DNS / routing</td>
          <td>只備份檔案、未演練切換</td>
      </tr>
      <tr>
          <td>Primary lease</td>
          <td>控制單一 writer ownership</td>
          <td>writer lease、fencing log</td>
          <td>多個 node 同時寫同一份邏輯狀態</td>
      </tr>
      <tr>
          <td>Consensus SQL</td>
          <td>多節點一致性寫入</td>
          <td>quorum、leader election</td>
          <td>用 WAL shipping 取代 distributed OLTP</td>
      </tr>
  </tbody>
</table>
<p>Continuous backup 的語言是 <a href="/blog/backend/knowledge-cards/rpo/" data-link-title="RPO" data-link-desc="說明恢復點目標如何定義可接受資料損失範圍">RPO</a> 與 <a href="/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO</a>。它關心最近一次成功送出的 WAL、snapshot freshness、object storage credential、restore 指令與演練結果。</p>
<p>Read replica 的語言是 freshness。Replica 能降低 read latency 或保護 primary workload，但讀者要知道 stale window、read-after-write policy、fallback to primary 與 cache invalidation。</p>
<p>Primary lease 的語言是 writer ownership。SQLite 的服務形狀仍適合 single writer；工具可以協助 deployment 切換，但 application 要配合 fencing、retry 與 promotion evidence。</p>
<h2 id="litestream-boundary">Litestream Boundary</h2>
<p>Litestream boundary 的核心責任是把 SQLite WAL 變成可持續複製的 backup stream。Litestream 官方說明把它定位為 SQLite streaming replication tool，並在 <a href="https://litestream.io/how-it-works/">How it works</a> 與 <a href="https://litestream.io/reference/restore/">restore command</a> 文件中強調 replica 與 restore workflow。</p>
<p>Litestream 適合下列情境：</p>
<ol>
<li>單節點 SQLite app 要降低資料遺失窗口。</li>
<li>系統可接受 restore 後重新啟動 service。</li>
<li>Object storage credential、retention、restore drill 可以被管理。</li>
<li>Write pattern 適中，WAL stream 與 snapshot 維護成本可控。</li>
</ol>
<p>Litestream 的設計重點是 backup evidence。Runbook 要記錄 replica destination、last replicated generation、last restore test、expected RPO、expected RTO、restore target path、credential rotation 與 corruption triage。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">litestream restore -o /var/lib/app/restored.db s3://example-bucket/app.db
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 /var/lib/app/restored.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>這段命令是 restore drill 的最小骨架。正式 runbook 要補上 service stop、database path、sidecar file、permission、checksum、application smoke test 與 rollback decision。</p>
<p>Litestream 的風險集中在 restore path。備份存在和服務可恢復是兩件事；每次 release 或 schema migration 後，都應用 staging data 跑一次 restore、integrity check、row count 與 application smoke test。</p>
<h2 id="litefs-boundary">LiteFS Boundary</h2>
<p>LiteFS boundary 的核心責任是支援 replicated deployment topology，而非只做 backup。LiteFS 在 Fly.io 文件中被定位為 SQLite replication layer，透過 FUSE filesystem 與 primary lease 模型協助應用在多個 instance 間運作。</p>
<p>LiteFS 適合下列情境：</p>
<ol>
<li>App 仍希望使用 SQLite file 與 local SQL path。</li>
<li>Deployment 有多個 instance，但 write authority 可以集中到 primary。</li>
<li>Read replica freshness 可以被產品接受。</li>
<li>Team 願意把 filesystem layer、primary lease、promotion 與 platform operation 納入 runbook。</li>
</ol>
<p>LiteFS 的設計重點是 primary ownership。Application 要知道 write request 到哪裡執行、primary 切換時如何重試、read replica 讀到舊資料時如何回應，以及 promotion 完成前哪些 endpoint 要進入 degraded mode。</p>
<p>LiteFS 的 incident route 要從 writer ownership 開始查。若出現 write error、stale read 或 suspected split brain，先查看 primary lease、instance health、replication lag、pending writes 與 platform network，再處理 application retry。</p>
<h2 id="failure-modes">Failure Modes</h2>
<p>Failure modes 的核心責任是把 replicated SQLite 的事故從「資料庫壞了」拆成可排查訊號。SQLite file、WAL、object storage、filesystem layer、deployment platform 與 application retry 都可能是問題來源。</p>
<table>
  <thead>
      <tr>
          <th>Failure mode</th>
          <th>判讀訊號</th>
          <th>立即處理</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Replica lag</td>
          <td>last replicated time 落後</td>
          <td>降低 write rate、檢查 credential / network</td>
      </tr>
      <tr>
          <td>Restore lag</td>
          <td>WAL files 過多、restore time 變長</td>
          <td>觸發 snapshot、演練 restore</td>
      </tr>
      <tr>
          <td>Stale read</td>
          <td>使用者讀到舊資料</td>
          <td>fallback primary read、標記 freshness</td>
      </tr>
      <tr>
          <td>Writer lease confusion</td>
          <td>多 instance write error</td>
          <td>暫停寫入、確認 primary、fencing old writer</td>
      </tr>
      <tr>
          <td>Object storage failure</td>
          <td>backup upload error</td>
          <td>切換 credential / destination、補上重送</td>
      </tr>
      <tr>
          <td>Sidecar file mismatch</td>
          <td>restore / copy 後 integrity fail</td>
          <td>回到 backup API / official restore path</td>
      </tr>
  </tbody>
</table>
<p>Replica lag 要接到 alert。對 Litestream，它意味著 RPO 正在擴大；對 LiteFS，它可能同時影響 read freshness 與 failover confidence。</p>
<p>Restore lag 要接到 release gate。若 restore time 已超過目標 <a href="/blog/backend/knowledge-cards/rto/" data-link-title="RTO" data-link-desc="說明恢復時間目標如何約束事故回復策略">RTO</a>，就要調整 snapshot frequency、資料保留策略或搬到 server database。</p>
<p>Stale read 要接到產品語言。使用者看到舊資料時，系統可以顯示 sync state、重讀 primary、限制 critical action 或提供 refresh；這些策略要在設計階段決定。</p>
<h2 id="no-go-conditions">No-Go Conditions</h2>
<p>No-go condition 的核心責任是避免把 replicated SQLite 推到 distributed OLTP 的位置。SQLite 周邊 replication 工具可以強化單節點與 read replica，但高寫入、多 writer、強一致跨 region transaction 需要不同資料庫模型。</p>
<table>
  <thead>
      <tr>
          <th>No-go 訊號</th>
          <th>原因</th>
          <th>路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>多 region 都要接受交易性寫入</td>
          <td>single writer / primary lease 壓力過高</td>
          <td><a href="/blog/backend/01-database/vendors/cockroachdb/" data-link-title="CockroachDB" data-link-desc="分散式 SQL、PostgreSQL 相容、跨區強一致、Spanner 的開源 / 跨雲替代">CockroachDB</a> 或 Spanner</td>
      </tr>
      <tr>
          <td>每秒大量 concurrent writer</td>
          <td>lock contention 與 replica lag 擴大</td>
          <td>PostgreSQL / MySQL / managed OLTP</td>
      </tr>
      <tr>
          <td>Central audit / DB role 是硬需求</td>
          <td>SQLite file model 缺少 server role</td>
          <td><a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a></td>
      </tr>
      <tr>
          <td>Restore drill 經常超過 RTO</td>
          <td>file size / WAL backlog 已超界</td>
          <td>server DB、sharding 或資料生命週期重整</td>
      </tr>
      <tr>
          <td>Incident team 缺少 filesystem layer 維護能力</td>
          <td>operation model 超過組織能力</td>
          <td>managed SQL 或 D1 / Turso managed path</td>
      </tr>
  </tbody>
</table>
<p>No-go 條件要在 design review 階段列出。SQLite replication 的好處是低成本與低元件數；當核心需求變成跨節點一致性寫入，繼續調工具會把風險藏在 incident 時刻。</p>
<h2 id="decision-route">Decision Route</h2>
<p>Decision route 的核心責任是把資料保護、讀擴展與高可用分開選型。Litestream / LiteFS 位置清楚時，SQLite 可以保持簡潔；位置混淆時，系統會同時缺 backup evidence 與 transaction guarantee。</p>
<table>
  <thead>
      <tr>
          <th>需求</th>
          <th>建議路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>單節點 SQLite 需要 continuous backup</td>
          <td>Litestream + restore drill</td>
      </tr>
      <tr>
          <td>多 instance deployment 需要 primary lease</td>
          <td>LiteFS + write routing / promotion runbook</td>
      </tr>
      <tr>
          <td>Edge app 需要 managed SQL-like platform</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL comparison</a></td>
      </tr>
      <tr>
          <td>多 tenant OLTP 需要 central operation</td>
          <td>PostgreSQL / MySQL / Aurora</td>
      </tr>
      <tr>
          <td>Global transaction 是核心需求</td>
          <td>Distributed OLTP</td>
      </tr>
  </tbody>
</table>
<p>選擇 Litestream 時，完成標準是能在 staging 從 replica restore 出可用 DB。選擇 LiteFS 時，完成標準是能演練 primary 切換、read freshness、write retry 與 degraded mode。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>Litestream / LiteFS replication 完成後，下一步要回到 SQLite operation evidence。File copy、backup API 與 WAL sidecar 請讀 <a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">file lifecycle / backup boundary</a>；busy、lock 與 writer 壓力請讀 <a href="/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking</a>；完整 runbook 請讀 <a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">SQLite observability / runbook</a>。</p>
]]></content:encoded></item><item><title>SQLite Local File Quickstart</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/</guid><description>&lt;p>SQLite local file quickstart 的核心責任是建立後續 backup、WAL、migration 與 fixture lab 共用的 database file。這個 lab 把 SQLite 從抽象服務選型轉成可觀察的檔案、schema、PRAGMA、transaction 與 sidecar artifact。&lt;/p>
&lt;p>本文的驗收標準是：你能建立一個可重建的 &lt;code>app.db&lt;/code>，知道它的 schema version、journal mode、foreign key 設定、seed data 與 cleanup 路徑。&lt;/p>
&lt;h2 id="lab-directory">Lab Directory&lt;/h2>
&lt;p>Lab directory 的核心責任是把 SQLite artifact 放在隔離資料夾，避免和正式檔案混淆。以下命令建立一個可刪除的本地工作區。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">mkdir -p /tmp/sqlite-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">rm -f app.db app.db-wal app.db-shm&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>驗收 artifact 是 &lt;code>/tmp/sqlite-lab/app.db&lt;/code>。後續 lab 可以沿用這個路徑，也可以每次從頭建立。&lt;/p>
&lt;h2 id="baseline-schema">Baseline Schema&lt;/h2>
&lt;p>Baseline schema 的核心責任是建立一組能測 transaction、constraint、index 與 query 的小型資料模型。這裡使用 &lt;code>accounts&lt;/code> 與 &lt;code>ledger_entries&lt;/code>，因為它們能清楚展示 foreign key 與金額 invariant。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s">&amp;lt;&amp;lt;&amp;#39;SQL&amp;#39;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA journal_mode = WAL;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA foreign_keys = ON;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA user_version = 1;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="s">CREATE TABLE accounts (
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="s"> id INTEGER PRIMARY KEY,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="s"> owner_name TEXT NOT NULL,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="s"> status TEXT NOT NULL CHECK (status IN (&amp;#39;active&amp;#39;, &amp;#39;closed&amp;#39;)),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="s"> created_at TEXT NOT NULL
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="s">) STRICT;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="s">CREATE TABLE ledger_entries (
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="s"> id INTEGER PRIMARY KEY,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="s"> account_id INTEGER NOT NULL REFERENCES accounts(id),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="s"> amount_cents INTEGER NOT NULL CHECK (amount_cents != 0),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">&lt;span class="s"> idempotency_key TEXT NOT NULL UNIQUE,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="s"> created_at TEXT NOT NULL
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="s">) STRICT;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">21&lt;/span>&lt;span class="cl">&lt;span class="s">CREATE INDEX idx_ledger_entries_account_created
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">22&lt;/span>&lt;span class="cl">&lt;span class="s">ON ledger_entries(account_id, created_at);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">23&lt;/span>&lt;span class="cl">&lt;span class="s">SQL&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>這段 schema 的重點是明確資料合約。&lt;code>STRICT&lt;/code>、&lt;code>CHECK&lt;/code>、&lt;code>FOREIGN KEY&lt;/code> 與 &lt;code>UNIQUE&lt;/code> 讓 fixture 更接近正式資料責任，也讓後續 migration lab 有可驗證的 invariant。&lt;/p>
&lt;h2 id="seed-data">Seed Data&lt;/h2>
&lt;p>Seed data 的核心責任是建立可重跑的測試資料。每筆 ledger entry 都有 idempotency key，讓後續 edge / retry 設計可以沿用。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s">&amp;lt;&amp;lt;&amp;#39;SQL&amp;#39;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA foreign_keys = ON;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="s">BEGIN;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="s">INSERT INTO accounts(id, owner_name, status, created_at)
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="s">VALUES
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="s"> (1, &amp;#39;Ada&amp;#39;, &amp;#39;active&amp;#39;, &amp;#39;2026-05-21T00:00:00Z&amp;#39;),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="s"> (2, &amp;#39;Lin&amp;#39;, &amp;#39;active&amp;#39;, &amp;#39;2026-05-21T00:05:00Z&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="s">INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at)
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="s">VALUES
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="s"> (1, 1200, &amp;#39;seed-ada-credit-1&amp;#39;, &amp;#39;2026-05-21T00:10:00Z&amp;#39;),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="s"> (1, -200, &amp;#39;seed-ada-debit-1&amp;#39;, &amp;#39;2026-05-21T00:12:00Z&amp;#39;),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="s"> (2, 900, &amp;#39;seed-lin-credit-1&amp;#39;, &amp;#39;2026-05-21T00:15:00Z&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="s">COMMIT;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="s">SQL&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>Seed 完成後先跑基本查詢。這一步確認 schema、constraint 與 index 入口都可用。&lt;/p></description><content:encoded><![CDATA[<p>SQLite local file quickstart 的核心責任是建立後續 backup、WAL、migration 與 fixture lab 共用的 database file。這個 lab 把 SQLite 從抽象服務選型轉成可觀察的檔案、schema、PRAGMA、transaction 與 sidecar artifact。</p>
<p>本文的驗收標準是：你能建立一個可重建的 <code>app.db</code>，知道它的 schema version、journal mode、foreign key 設定、seed data 與 cleanup 路徑。</p>
<h2 id="lab-directory">Lab Directory</h2>
<p>Lab directory 的核心責任是把 SQLite artifact 放在隔離資料夾，避免和正式檔案混淆。以下命令建立一個可刪除的本地工作區。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">mkdir -p /tmp/sqlite-lab
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-lab
</span></span><span class="line"><span class="ln">3</span><span class="cl">rm -f app.db app.db-wal app.db-shm</span></span></code></pre></div><p>驗收 artifact 是 <code>/tmp/sqlite-lab/app.db</code>。後續 lab 可以沿用這個路徑，也可以每次從頭建立。</p>
<h2 id="baseline-schema">Baseline Schema</h2>
<p>Baseline schema 的核心責任是建立一組能測 transaction、constraint、index 與 query 的小型資料模型。這裡使用 <code>accounts</code> 與 <code>ledger_entries</code>，因為它們能清楚展示 foreign key 與金額 invariant。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="s">PRAGMA journal_mode = WAL;
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">PRAGMA user_version = 1;
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">CREATE TABLE accounts (
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">  id INTEGER PRIMARY KEY,
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">  owner_name TEXT NOT NULL,
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">  status TEXT NOT NULL CHECK (status IN (&#39;active&#39;, &#39;closed&#39;)),
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">  created_at TEXT NOT NULL
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">) STRICT;
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">CREATE TABLE ledger_entries (
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">  id INTEGER PRIMARY KEY,
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">  account_id INTEGER NOT NULL REFERENCES accounts(id),
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">  amount_cents INTEGER NOT NULL CHECK (amount_cents != 0),
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s">  idempotency_key TEXT NOT NULL UNIQUE,
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s">  created_at TEXT NOT NULL
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s">) STRICT;
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s">CREATE INDEX idx_ledger_entries_account_created
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s">ON ledger_entries(account_id, created_at);
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>這段 schema 的重點是明確資料合約。<code>STRICT</code>、<code>CHECK</code>、<code>FOREIGN KEY</code> 與 <code>UNIQUE</code> 讓 fixture 更接近正式資料責任，也讓後續 migration lab 有可驗證的 invariant。</p>
<h2 id="seed-data">Seed Data</h2>
<p>Seed data 的核心責任是建立可重跑的測試資料。每筆 ledger entry 都有 idempotency key，讓後續 edge / retry 設計可以沿用。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">BEGIN;
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">INSERT INTO accounts(id, owner_name, status, created_at)
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">VALUES
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">  (1, &#39;Ada&#39;, &#39;active&#39;, &#39;2026-05-21T00:00:00Z&#39;),
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">  (2, &#39;Lin&#39;, &#39;active&#39;, &#39;2026-05-21T00:05:00Z&#39;);
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at)
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">VALUES
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">  (1, 1200, &#39;seed-ada-credit-1&#39;, &#39;2026-05-21T00:10:00Z&#39;),
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">  (1, -200, &#39;seed-ada-debit-1&#39;, &#39;2026-05-21T00:12:00Z&#39;),
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">  (2, 900, &#39;seed-lin-credit-1&#39;, &#39;2026-05-21T00:15:00Z&#39;);
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">COMMIT;
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>Seed 完成後先跑基本查詢。這一步確認 schema、constraint 與 index 入口都可用。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">.headers on
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">.mode column
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">SELECT a.id, a.owner_name, SUM(l.amount_cents) AS balance_cents
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">FROM accounts a
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">JOIN ledger_entries l ON l.account_id = a.id
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">GROUP BY a.id, a.owner_name
</span></span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="s">ORDER BY a.id;
</span></span></span><span class="line"><span class="ln">9</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>預期輸出應顯示 Ada 餘額 <code>1000</code>，Lin 餘額 <code>900</code>。</p>
<h2 id="pragma-snapshot">PRAGMA Snapshot</h2>
<p>PRAGMA snapshot 的核心責任是把連線設定變成 evidence。SQLite 的部分設定與 connection 有關，因此 lab 要明確查出當前狀態。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">.headers on
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">.mode column
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">PRAGMA journal_mode;
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">PRAGMA foreign_keys;
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">PRAGMA user_version;
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">PRAGMA integrity_check;
</span></span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>驗收重點如下：</p>
<table>
  <thead>
      <tr>
          <th>欄位</th>
          <th>期望結果</th>
          <th>意義</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>journal_mode</code></td>
          <td><code>wal</code></td>
          <td>後續可觀察 <code>-wal</code> sidecar</td>
      </tr>
      <tr>
          <td><code>foreign_keys</code></td>
          <td><code>1</code></td>
          <td>constraint 在連線上已啟用</td>
      </tr>
      <tr>
          <td><code>user_version</code></td>
          <td><code>1</code></td>
          <td>migration 起點清楚</td>
      </tr>
      <tr>
          <td>integrity</td>
          <td><code>ok</code></td>
          <td>database file 基本健康</td>
      </tr>
  </tbody>
</table>
<h2 id="transaction-sample">Transaction Sample</h2>
<p>Transaction sample 的核心責任是建立後續 busy / migration lab 的共同語言。SQLite transaction 成功時要同時更新資料與保護 invariant。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">BEGIN IMMEDIATE;
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at)
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">VALUES (1, 300, &#39;manual-ada-credit-1&#39;, &#39;2026-05-21T00:20:00Z&#39;);
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">COMMIT;
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p><code>BEGIN IMMEDIATE</code> 會提早取得 write lock。這讓後續 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a> 可以直接展示 single writer boundary。</p>
<h2 id="file-artifact-check">File Artifact Check</h2>
<p>File artifact check 的核心責任是讓讀者看到 SQLite 由 <code>.db</code> 與可能存在的 sidecar 共同構成。WAL mode 可能建立 <code>-wal</code> 與 <code>-shm</code> sidecar，backup / copy / restore runbook 要理解這些檔案。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">ls -lh app.db app.db-wal app.db-shm</span></span></code></pre></div><p>若 sidecar 暫時未出現，可以再寫入一筆資料或保持連線開啟。Sidecar 是否存在取決於 WAL 狀態、checkpoint 與 connection lifecycle。</p>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup 的核心責任是讓 lab 可以重跑。若要重新開始，刪除 database 與 sidecar。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">rm -f /tmp/sqlite-lab/app.db /tmp/sqlite-lab/app.db-wal /tmp/sqlite-lab/app.db-shm</span></span></code></pre></div><p>完成本篇後，下一步可以進入 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/backup-restore-drill/" data-link-title="SQLite Backup Restore Drill" data-link-desc="SQLite .backup、VACUUM INTO、restore validation、sidecar file handling 與 RPO / RTO note 的操作說明">backup restore drill</a> 或 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a>。</p>
]]></content:encoded></item><item><title>SQLite Local-first Sync Boundary</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 local-first / offline-first 場景；本文聚焦 &lt;em>SQLite local store 與 multi-device sync protocol 的責任分界&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite local-first sync boundary 的核心責任是把「本機可用」和「多端一致」分成兩個問題。SQLite 很適合保存 device-local state；但它不提供 identity、transport、&lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/conflict-resolution/" data-link-title="Conflict Resolution" data-link-desc="說明並發或離線寫入產生衝突時，如何偵測、呈現與合併成可接受狀態">conflict resolution&lt;/a>、delete propagation、server authority 或 audit trail。當資料要跨裝置、跨使用者或跨服務同步時，SQLite 只是 local replica / working copy。&lt;/p>
&lt;p>本文的判讀錨點是：&lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/local-first/" data-link-title="Local-First" data-link-desc="說明本機優先的資料架構如何讓離線可用，並把同步當成獨立問題">local-first&lt;/a> 的產品價值來自離線可用，工程成本來自同步語意。SQLite 解的是 local durability；sync layer 解的是資料合併、順序、權威來源與錯誤修復。&lt;/p>
&lt;h2 id="local-state-taxonomy">Local state taxonomy&lt;/h2>
&lt;p>Local-first 設計的第一步是標記本機資料角色。不同資料角色對 sync、backup、conflict 與 delete 的要求不同。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>資料角色&lt;/th>
 &lt;th>例子&lt;/th>
 &lt;th>Sync 語意&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Local cache&lt;/td>
 &lt;td>API response cache、thumbnail metadata&lt;/td>
 &lt;td>可清除、可重抓&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Draft / working copy&lt;/td>
 &lt;td>草稿、離線表單、未送出 action&lt;/td>
 &lt;td>需要 upload / retry / conflict handling&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Local source of truth&lt;/td>
 &lt;td>單裝置日記、CLI state&lt;/td>
 &lt;td>需要 backup / export，可能不需要 server&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Local replica&lt;/td>
 &lt;td>server record 的本地副本&lt;/td>
 &lt;td>server authority、stale read、sync lag&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Sync queue&lt;/td>
 &lt;td>pending mutation / event log&lt;/td>
 &lt;td>ordering、idempotency、replay&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的重點是資料角色先於 sync 工具。若所有資料都只是 cache，SQLite + TTL 足夠；若有 pending mutation 或 multi-device edit，就需要 sync protocol。&lt;/p>
&lt;h2 id="authority-boundary">Authority boundary&lt;/h2>
&lt;p>Authority boundary 的核心責任是決定衝突時誰說了算。Local-first app 可以讓 device、server、field-level merge 或 CRDT 成為不同層的 authority；SQLite 本身只保存狀態，不替系統決策。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Authority model&lt;/th>
 &lt;th>適合情境&lt;/th>
 &lt;th>代價&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Server authority&lt;/td>
 &lt;td>帳務、權限、共享資料&lt;/td>
 &lt;td>離線寫入要排隊，回線後可能被拒絕&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Device authority&lt;/td>
 &lt;td>單使用者、單裝置資料&lt;/td>
 &lt;td>多裝置同步能力弱&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Last-write-wins&lt;/td>
 &lt;td>低價值設定、簡單 preference&lt;/td>
 &lt;td>資料覆蓋風險&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Field merge&lt;/td>
 &lt;td>profile、表單、可分欄位資料&lt;/td>
 &lt;td>merge rule 要測，使用者理解成本上升&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>CRDT / operation log&lt;/td>
 &lt;td>協作編輯、順序敏感操作&lt;/td>
 &lt;td>實作與除錯成本高&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Authority model 要和 product semantics 對齊。庫存、付款、權限這類資料通常需要 server authority；notes、draft、local settings 可以接受更偏 local 的權威模型。&lt;/p>
&lt;h2 id="sync-transport-與-local-log">Sync transport 與 local log&lt;/h2>
&lt;p>Sync transport 的核心責任是把 SQLite local state 轉成可重送、可去重、可驗證的資料流。最常見做法是本地維護 pending mutation table 或 change log，再由 background sync worker 送到 server。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 local-first / offline-first 場景；本文聚焦 <em>SQLite local store 與 multi-device sync protocol 的責任分界</em>。</p></blockquote>
<p>SQLite local-first sync boundary 的核心責任是把「本機可用」和「多端一致」分成兩個問題。SQLite 很適合保存 device-local state；但它不提供 identity、transport、<a href="/blog/backend/knowledge-cards/conflict-resolution/" data-link-title="Conflict Resolution" data-link-desc="說明並發或離線寫入產生衝突時，如何偵測、呈現與合併成可接受狀態">conflict resolution</a>、delete propagation、server authority 或 audit trail。當資料要跨裝置、跨使用者或跨服務同步時，SQLite 只是 local replica / working copy。</p>
<p>本文的判讀錨點是：<a href="/blog/backend/knowledge-cards/local-first/" data-link-title="Local-First" data-link-desc="說明本機優先的資料架構如何讓離線可用，並把同步當成獨立問題">local-first</a> 的產品價值來自離線可用，工程成本來自同步語意。SQLite 解的是 local durability；sync layer 解的是資料合併、順序、權威來源與錯誤修復。</p>
<h2 id="local-state-taxonomy">Local state taxonomy</h2>
<p>Local-first 設計的第一步是標記本機資料角色。不同資料角色對 sync、backup、conflict 與 delete 的要求不同。</p>
<table>
  <thead>
      <tr>
          <th>資料角色</th>
          <th>例子</th>
          <th>Sync 語意</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Local cache</td>
          <td>API response cache、thumbnail metadata</td>
          <td>可清除、可重抓</td>
      </tr>
      <tr>
          <td>Draft / working copy</td>
          <td>草稿、離線表單、未送出 action</td>
          <td>需要 upload / retry / conflict handling</td>
      </tr>
      <tr>
          <td>Local source of truth</td>
          <td>單裝置日記、CLI state</td>
          <td>需要 backup / export，可能不需要 server</td>
      </tr>
      <tr>
          <td>Local replica</td>
          <td>server record 的本地副本</td>
          <td>server authority、stale read、sync lag</td>
      </tr>
      <tr>
          <td>Sync queue</td>
          <td>pending mutation / event log</td>
          <td>ordering、idempotency、replay</td>
      </tr>
  </tbody>
</table>
<p>這張表的重點是資料角色先於 sync 工具。若所有資料都只是 cache，SQLite + TTL 足夠；若有 pending mutation 或 multi-device edit，就需要 sync protocol。</p>
<h2 id="authority-boundary">Authority boundary</h2>
<p>Authority boundary 的核心責任是決定衝突時誰說了算。Local-first app 可以讓 device、server、field-level merge 或 CRDT 成為不同層的 authority；SQLite 本身只保存狀態，不替系統決策。</p>
<table>
  <thead>
      <tr>
          <th>Authority model</th>
          <th>適合情境</th>
          <th>代價</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Server authority</td>
          <td>帳務、權限、共享資料</td>
          <td>離線寫入要排隊，回線後可能被拒絕</td>
      </tr>
      <tr>
          <td>Device authority</td>
          <td>單使用者、單裝置資料</td>
          <td>多裝置同步能力弱</td>
      </tr>
      <tr>
          <td>Last-write-wins</td>
          <td>低價值設定、簡單 preference</td>
          <td>資料覆蓋風險</td>
      </tr>
      <tr>
          <td>Field merge</td>
          <td>profile、表單、可分欄位資料</td>
          <td>merge rule 要測，使用者理解成本上升</td>
      </tr>
      <tr>
          <td>CRDT / operation log</td>
          <td>協作編輯、順序敏感操作</td>
          <td>實作與除錯成本高</td>
      </tr>
  </tbody>
</table>
<p>Authority model 要和 product semantics 對齊。庫存、付款、權限這類資料通常需要 server authority；notes、draft、local settings 可以接受更偏 local 的權威模型。</p>
<h2 id="sync-transport-與-local-log">Sync transport 與 local log</h2>
<p>Sync transport 的核心責任是把 SQLite local state 轉成可重送、可去重、可驗證的資料流。最常見做法是本地維護 pending mutation table 或 change log，再由 background sync worker 送到 server。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">pending_mutations</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="w">  </span><span class="n">id</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">  </span><span class="n">entity_type</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">  </span><span class="n">entity_id</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">  </span><span class="k">operation</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">  </span><span class="n">payload</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">  </span><span class="n">created_at</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w">  </span><span class="n">retry_count</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="w"> </span><span class="k">DEFAULT</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w">  </span><span class="n">last_error</span><span class="w"> </span><span class="nb">TEXT</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w"></span><span class="p">);</span></span></span></code></pre></div><table>
  <thead>
      <tr>
          <th>設計點</th>
          <th>判讀</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>idempotency</td>
          <td>每個 mutation 需要穩定 id，避免重送副作用</td>
      </tr>
      <tr>
          <td>ordering</td>
          <td>同 entity 操作是否必須按順序</td>
      </tr>
      <tr>
          <td>retry</td>
          <td>transient failure、backoff、dead-letter</td>
      </tr>
      <tr>
          <td>compaction</td>
          <td>已同步 local log 何時清除</td>
      </tr>
      <tr>
          <td>reconciliation</td>
          <td>server / local 差異如何修復</td>
      </tr>
  </tbody>
</table>
<p>這裡和 backend queue 概念相通：pending mutation table 是本機版 durable queue。它需要 <a href="/blog/backend/knowledge-cards/idempotency/" data-link-title="Idempotency" data-link-desc="說明同一操作執行多次時如何保持結果一致">idempotency</a>、retry 與 replay 思維，而不只是「存一張表」。</p>
<h2 id="conflict-resolution">Conflict resolution</h2>
<p>Conflict resolution 的核心責任是讓兩個合法 local write 合併成可接受狀態。SQLite 可以保存 local write；sync layer 要決定衝突偵測、呈現與合併。</p>
<table>
  <thead>
      <tr>
          <th>衝突型態</th>
          <th>例子</th>
          <th>處理策略</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Same field update</td>
          <td>兩台裝置改同一個 display name</td>
          <td>LWW、server reject、manual merge</td>
      </tr>
      <tr>
          <td>Disjoint field update</td>
          <td>一台改 phone，一台改 address</td>
          <td>field merge</td>
      </tr>
      <tr>
          <td>Delete vs update</td>
          <td>一台刪除，一台修改</td>
          <td>tombstone、manual review</td>
      </tr>
      <tr>
          <td>Ordered operation</td>
          <td>task reorder、ledger append</td>
          <td>operation log、server sequence</td>
      </tr>
  </tbody>
</table>
<p>Conflict policy 要在資料模型設計時決定。等衝突發生後才補策略，通常會導致資料修復、客服流程與 audit evidence 同時缺位。</p>
<h2 id="delete-propagation-與-privacy">Delete propagation 與 privacy</h2>
<p>Delete propagation 的核心責任是讓 server、device、backup 與 sync queue 對「刪除」有一致語意。Local-first app 常見風險是 server 已刪，但 device local DB、pending queue 或 OS backup 還留著資料。</p>
<table>
  <thead>
      <tr>
          <th>刪除語意</th>
          <th>適合情境</th>
          <th>SQLite 設計</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Soft delete</td>
          <td>可恢復、需要 sync tombstone</td>
          <td><code>deleted_at</code>、sync tombstone、retention job</td>
      </tr>
      <tr>
          <td>Hard delete</td>
          <td>privacy / compliance</td>
          <td>local purge、backup exclusion、sync confirmation</td>
      </tr>
      <tr>
          <td>Redaction</td>
          <td>support bundle / log</td>
          <td>export 時遮罩 sensitive fields</td>
      </tr>
  </tbody>
</table>
<p>刪除在同步系統裡是一個跨裝置生命週期。若資料跨裝置同步，delete 需要 <a href="/blog/backend/knowledge-cards/tombstone/" data-link-title="Tombstone" data-link-desc="說明刪除如何用一筆標記記錄下來，讓刪除事件能跨副本與裝置傳播">tombstone</a>、ack、retry、backup retention 與 evidence；這些責任要接到 <a href="/blog/backend/07-security-data-protection/data-protection-and-masking-governance/" data-link-title="7.4 資料保護與遮罩治理" data-link-desc="以問題驅動方式整理資料分級、遮罩、匯出與備份治理">Data Protection</a>。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1pending-mutation-沒有-idempotency-key">Case 1：pending mutation 沒有 idempotency key</h3>
<p>Pending mutation 沒有 idempotency key 的核心風險是重送造成重複副作用。網路 timeout 後 worker 重送，server 已經處理第一次請求，第二次又建立一筆資料或扣一次庫存。</p>
<p>修正方向是每個 mutation 生成 stable id，server 以 idempotency key 去重，local SQLite 保存 retry state 與 server ack。</p>
<h3 id="case-2lww-覆蓋使用者資料">Case 2：LWW 覆蓋使用者資料</h3>
<p>Last-write-wins 的核心風險是把衝突靜默變成資料遺失。Preference 類資料可接受；草稿、文件、表單、付款資料通常需要更清楚的 conflict handling。</p>
<p>修正方向是依資料價值分層。低價值設定用 LWW；高價值內容用 field merge、manual conflict 或 operation log。</p>
<h3 id="case-3delete-沒傳到離線裝置">Case 3：delete 沒傳到離線裝置</h3>
<p>Delete propagation 失敗的核心風險是 privacy / compliance 失效。使用者刪除 server 資料後，一台長期離線裝置重新上線又把舊資料同步回來。</p>
<p>修正方向是 tombstone + server authority。Server 要能拒絕過期 mutation，device 要能接收 delete tombstone 並 purge local state。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>Local-first SQLite 設計要回答：</p>
<ol>
<li>哪些 table 是 local source of truth，哪些是 server replica。</li>
<li>Pending mutation 是否有 idempotency key 與 retry state。</li>
<li>Conflict policy 是 LWW、field merge、manual merge 還是 operation log。</li>
<li>Delete 是否有 tombstone、ack 與 local purge。</li>
<li>Sync worker 是否有 backoff、dead-letter、reconciliation。</li>
<li>Device backup 是否會保存已刪資料。</li>
<li>Server 是否能拒絕過期 local write。</li>
</ol>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/mobile-desktop-embedded-store/" data-link-title="SQLite Mobile / Desktop Embedded Store" data-link-desc="SQLite 在 mobile、desktop、CLI、browser profile 與 embedded device 中承擔 local formal state 的資料責任、backup、privacy 與 sync boundary">Mobile / Desktop Embedded Store</a></li>
<li>Sibling：<a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso</a>、<a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL Comparison</a></li>
<li>卡片：<a href="/blog/backend/knowledge-cards/idempotency/" data-link-title="Idempotency" data-link-desc="說明同一操作執行多次時如何保持結果一致">Idempotency</a>、<a href="/blog/backend/knowledge-cards/eventual-consistency/" data-link-title="Eventual Consistency" data-link-desc="允許短暫不一致、最終收斂到同一資料狀態的一致性語意">Eventual Consistency</a>、<a href="/blog/backend/knowledge-cards/stale-read/" data-link-title="Stale Read" data-link-desc="讀取到落後於最新寫入版本的舊資料">Stale Read</a></li>
<li>跨模組：<a href="/blog/backend/07-security-data-protection/data-protection-and-masking-governance/" data-link-title="7.4 資料保護與遮罩治理" data-link-desc="以問題驅動方式整理資料分級、遮罩、匯出與備份治理">Data Protection</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Migration Fixture Lab</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/migration-fixture-lab/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/migration-fixture-lab/</guid><description>&lt;p>SQLite migration fixture lab 的核心責任是把 schema migration 與 test fixture 放進同一個可重建流程。這篇承接 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema Migration / Versioning&lt;/a> 與 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test Fixture Best Practice&lt;/a>，讓 migration 有版本、snapshot、validation 與 rollback note。&lt;/p>
&lt;p>本文的驗收標準是：你能建立 v1 fixture、套用 v2 migration、產生 v2 snapshot，並用 validation query 證明資料合約仍成立。&lt;/p>
&lt;h2 id="create-fixture">Create Fixture&lt;/h2>
&lt;p>Create fixture 的核心責任是建立乾淨、可重建的 source fixture。沿用 quickstart schema，或重新建立一份 fixture DB。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">mkdir -p /tmp/sqlite-fixture-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-fixture-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">rm -f fixture-v1.db fixture-v2.db
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">sqlite3 fixture-v1.db &lt;span class="s">&amp;lt;&amp;lt;&amp;#39;SQL&amp;#39;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA foreign_keys = ON;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="s">PRAGMA user_version = 1;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="s">CREATE TABLE accounts (
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="s"> id INTEGER PRIMARY KEY,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="s"> owner_name TEXT NOT NULL,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="s"> status TEXT NOT NULL CHECK (status IN (&amp;#39;active&amp;#39;, &amp;#39;closed&amp;#39;)),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="s"> created_at TEXT NOT NULL
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="s">) STRICT;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="s">CREATE TABLE ledger_entries (
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="s"> id INTEGER PRIMARY KEY,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">&lt;span class="s"> account_id INTEGER NOT NULL REFERENCES accounts(id),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="s"> amount_cents INTEGER NOT NULL CHECK (amount_cents != 0),
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="s"> idempotency_key TEXT NOT NULL UNIQUE,
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl">&lt;span class="s"> created_at TEXT NOT NULL
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">21&lt;/span>&lt;span class="cl">&lt;span class="s">) STRICT;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">22&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">23&lt;/span>&lt;span class="cl">&lt;span class="s">INSERT INTO accounts VALUES (1, &amp;#39;Ada&amp;#39;, &amp;#39;active&amp;#39;, &amp;#39;2026-05-21T00:00:00Z&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">24&lt;/span>&lt;span class="cl">&lt;span class="s">INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at)
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">25&lt;/span>&lt;span class="cl">&lt;span class="s">VALUES (1, 1000, &amp;#39;fixture-v1-ada&amp;#39;, &amp;#39;2026-05-21T00:10:00Z&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">26&lt;/span>&lt;span class="cl">&lt;span class="s">SQL&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>這個 fixture 是 v1 source of truth。CI 可以每次從 SQL 重建，也可以保存 &lt;code>fixture-v1.db&lt;/code> 作為 binary fixture；兩者都要有版本與 checksum。&lt;/p></description><content:encoded><![CDATA[<p>SQLite migration fixture lab 的核心責任是把 schema migration 與 test fixture 放進同一個可重建流程。這篇承接 <a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema Migration / Versioning</a> 與 <a href="/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test Fixture Best Practice</a>，讓 migration 有版本、snapshot、validation 與 rollback note。</p>
<p>本文的驗收標準是：你能建立 v1 fixture、套用 v2 migration、產生 v2 snapshot，並用 validation query 證明資料合約仍成立。</p>
<h2 id="create-fixture">Create Fixture</h2>
<p>Create fixture 的核心責任是建立乾淨、可重建的 source fixture。沿用 quickstart schema，或重新建立一份 fixture DB。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">mkdir -p /tmp/sqlite-fixture-lab
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-fixture-lab
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">rm -f fixture-v1.db fixture-v2.db
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">sqlite3 fixture-v1.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">PRAGMA user_version = 1;
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">CREATE TABLE accounts (
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">  id INTEGER PRIMARY KEY,
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">  owner_name TEXT NOT NULL,
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">  status TEXT NOT NULL CHECK (status IN (&#39;active&#39;, &#39;closed&#39;)),
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">  created_at TEXT NOT NULL
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">) STRICT;
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">CREATE TABLE ledger_entries (
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">  id INTEGER PRIMARY KEY,
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s">  account_id INTEGER NOT NULL REFERENCES accounts(id),
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s">  amount_cents INTEGER NOT NULL CHECK (amount_cents != 0),
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s">  idempotency_key TEXT NOT NULL UNIQUE,
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s">  created_at TEXT NOT NULL
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s">) STRICT;
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s">INSERT INTO accounts VALUES (1, &#39;Ada&#39;, &#39;active&#39;, &#39;2026-05-21T00:00:00Z&#39;);
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="s">INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at)
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="s">VALUES (1, 1000, &#39;fixture-v1-ada&#39;, &#39;2026-05-21T00:10:00Z&#39;);
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>這個 fixture 是 v1 source of truth。CI 可以每次從 SQL 重建，也可以保存 <code>fixture-v1.db</code> 作為 binary fixture；兩者都要有版本與 checksum。</p>
<h2 id="pre-migration-snapshot">Pre-Migration Snapshot</h2>
<p>Pre-migration snapshot 的核心責任是建立 rollback 起點。正式 migration 前應先保存 source DB。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 fixture-v1.db <span class="s2">&#34;.backup &#39;fixture-v1-before-migration.db&#39;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 fixture-v1-before-migration.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>這份 snapshot 代表 migration 失敗時的回退點。CI log 要保留 snapshot path、schema version 與 migration id。</p>
<h2 id="apply-add-column-migration">Apply Add Column Migration</h2>
<p>Apply add column migration 的核心責任是展示低風險 schema change。先複製 v1，再套用 v2。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">cp fixture-v1.db fixture-v2.db
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 fixture-v2.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">BEGIN;
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">ALTER TABLE accounts ADD COLUMN email TEXT;
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">PRAGMA user_version = 2;
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">COMMIT;
</span></span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>驗證 schema version 與新欄位：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 fixture-v2.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">PRAGMA user_version;
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">PRAGMA table_info(accounts);
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>Add column 是較簡單的 migration。涉及 drop column、rename、constraint 重建或資料 reshape 時，應改用 table rebuild 策略。</p>
<h2 id="table-rebuild-example">Table Rebuild Example</h2>
<p>Table rebuild 的核心責任是展示 SQLite schema migration 的高風險路徑。以下範例把 <code>accounts.status</code> 的 allowed value 加入 <code>suspended</code>，透過新表重建 constraint。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">sqlite3 fixture-v2.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="s">PRAGMA foreign_keys = OFF;
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s">BEGIN;
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">CREATE TABLE accounts_new (
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">  id INTEGER PRIMARY KEY,
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">  owner_name TEXT NOT NULL,
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">  status TEXT NOT NULL CHECK (status IN (&#39;active&#39;, &#39;closed&#39;, &#39;suspended&#39;)),
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">  created_at TEXT NOT NULL,
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">  email TEXT
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">) STRICT;
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">INSERT INTO accounts_new(id, owner_name, status, created_at, email)
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">SELECT id, owner_name, status, created_at, email
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">FROM accounts;
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s">DROP TABLE accounts;
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s">ALTER TABLE accounts_new RENAME TO accounts;
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s">PRAGMA user_version = 3;
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s">COMMIT;
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s">PRAGMA foreign_keys = ON;
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>Table rebuild 要保存 index、trigger、view 與 FK reference。這個 lab 只有小型 schema；正式 migration 要先列出所有 dependent object。</p>
<h2 id="validation-query">Validation Query</h2>
<p>Validation query 的核心責任是證明 migration 後資料仍符合 domain invariant。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 fixture-v2.db <span class="s">&lt;&lt;&#39;SQL&#39;
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="s">PRAGMA integrity_check;
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="s">PRAGMA foreign_key_check;
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="s">SELECT COUNT(*) AS account_count FROM accounts;
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="s">SELECT COUNT(*) AS ledger_count FROM ledger_entries;
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="s">SELECT SUM(amount_cents) AS total_balance FROM ledger_entries;
</span></span></span><span class="line"><span class="ln">7</span><span class="cl"><span class="s">PRAGMA user_version;
</span></span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="s">SQL</span></span></span></code></pre></div><p>驗收結果應包含 integrity <code>ok</code>、foreign key check 空結果、account count <code>1</code>、ledger count <code>1</code>、total balance <code>1000</code>、user version <code>3</code>。</p>
<h2 id="contract-test-hook">Contract Test Hook</h2>
<p>Contract test hook 的核心責任是讓 fixture 進入 CI。語言與 framework 可以不同，但測試要固定做三件事：開啟 FK、確認 schema version、跑 repository contract。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">test setup:
</span></span><span class="line"><span class="ln">2</span><span class="cl">  copy fixture-v2.db to temp path
</span></span><span class="line"><span class="ln">3</span><span class="cl">  open SQLite connection
</span></span><span class="line"><span class="ln">4</span><span class="cl">  execute PRAGMA foreign_keys = ON
</span></span><span class="line"><span class="ln">5</span><span class="cl">  assert PRAGMA user_version = 3
</span></span><span class="line"><span class="ln">6</span><span class="cl">  run repository contract tests</span></span></code></pre></div><p>每個 test 使用 temp copy 可以避免資料污染。需要測 concurrency 時，改用 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a>。</p>
<h2 id="rollback-note">Rollback Note</h2>
<p>Rollback note 的核心責任是把 migration 失敗時的處理寫清楚。這個 lab 的 rollback 是保留 <code>fixture-v1-before-migration.db</code>，在 migration validation 失敗時停止 release 並保存 failed DB。</p>
<p>正式 runbook 要記錄：</p>
<ol>
<li>Migration id 與 source / target <code>user_version</code>。</li>
<li>Pre-migration backup path。</li>
<li>Validation query 與結果。</li>
<li>Failed DB 保存路徑。</li>
<li>Release block / rollback 條件。</li>
</ol>
<p>完成本篇後，下一步可以讀 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration</a> 或 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso migration</a>。</p>
]]></content:encoded></item><item><title>SQLite Mobile / Desktop Embedded Store</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/mobile-desktop-embedded-store/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/mobile-desktop-embedded-store/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 mobile、desktop、CLI 與 embedded device；本文聚焦 &lt;em>device-local formal state 的資料責任、backup、privacy 與 sync boundary&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite embedded store 的核心責任是讓 application process 在本機持有正式狀態。Mobile app、desktop app、browser profile、CLI tool 與 embedded device 常用 SQLite 保存 local data；這些資料可能只是 cache，也可能是使用者唯一資料來源。教學上要先判斷它是否承擔 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">source of truth&lt;/a>，再決定 backup、sync、privacy 與 migration 責任。&lt;/p>
&lt;p>本文的判讀錨點是：embedded SQLite 的 production boundary 在 device lifecycle，database server 層的邊界在這裡不適用。OS backup、app upgrade、device loss、profile corruption、local PII、multi-device sync 與 user export / delete 都是資料庫責任的一部分。&lt;/p>
&lt;h2 id="embedded-state-model">Embedded state model&lt;/h2>
&lt;p>Embedded state model 的核心責任是把 local database file 放回 application lifecycle。SQLite 是典型的 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/embedded-database/" data-link-title="Embedded Database" data-link-desc="說明嵌入式資料庫如何隨 application process 運作，並把檔案生命週期責任交回應用">embedded database&lt;/a>：database file 通常跟著 app sandbox、user profile、CLI config directory 或 device storage 存在，它的 owner 是 application，而非獨立 DBA。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>場景&lt;/th>
 &lt;th>SQLite 資料角色&lt;/th>
 &lt;th>主要風險&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Mobile app&lt;/td>
 &lt;td>offline state、draft、cache、local profile&lt;/td>
 &lt;td>app upgrade、device loss、cloud backup leakage&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Desktop app&lt;/td>
 &lt;td>user profile、history、settings&lt;/td>
 &lt;td>profile corruption、manual file copy、multi-version app&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>CLI tool&lt;/td>
 &lt;td>local index、metadata、state cache&lt;/td>
 &lt;td>command interruption、portable file path&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Browser / profile&lt;/td>
 &lt;td>cookies、history、bookmark 類資料&lt;/td>
 &lt;td>privacy、profile migration、lock collision&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Embedded device&lt;/td>
 &lt;td>offline event、sensor / config state&lt;/td>
 &lt;td>power loss、flash wear、delayed sync&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的重點是資料角色而非產品名稱。同樣是 SQLite file，cache 可以清掉重建；draft、local-only note、sensor event 或 user history 可能需要正式 backup / export / delete。&lt;/p>
&lt;h2 id="backup-與-export">Backup 與 export&lt;/h2>
&lt;p>Embedded backup 的核心責任是讓使用者或服務能從 device / profile failure 復原。Mobile / desktop / CLI 的 backup 路徑常和 OS backup、app export、cloud sync 或手動複製混在一起；SQLite file lifecycle 要明確。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合 mobile、desktop、CLI 與 embedded device；本文聚焦 <em>device-local formal state 的資料責任、backup、privacy 與 sync boundary</em>。</p></blockquote>
<p>SQLite embedded store 的核心責任是讓 application process 在本機持有正式狀態。Mobile app、desktop app、browser profile、CLI tool 與 embedded device 常用 SQLite 保存 local data；這些資料可能只是 cache，也可能是使用者唯一資料來源。教學上要先判斷它是否承擔 <a href="/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">source of truth</a>，再決定 backup、sync、privacy 與 migration 責任。</p>
<p>本文的判讀錨點是：embedded SQLite 的 production boundary 在 device lifecycle，database server 層的邊界在這裡不適用。OS backup、app upgrade、device loss、profile corruption、local PII、multi-device sync 與 user export / delete 都是資料庫責任的一部分。</p>
<h2 id="embedded-state-model">Embedded state model</h2>
<p>Embedded state model 的核心責任是把 local database file 放回 application lifecycle。SQLite 是典型的 <a href="/blog/backend/knowledge-cards/embedded-database/" data-link-title="Embedded Database" data-link-desc="說明嵌入式資料庫如何隨 application process 運作，並把檔案生命週期責任交回應用">embedded database</a>：database file 通常跟著 app sandbox、user profile、CLI config directory 或 device storage 存在，它的 owner 是 application，而非獨立 DBA。</p>
<table>
  <thead>
      <tr>
          <th>場景</th>
          <th>SQLite 資料角色</th>
          <th>主要風險</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Mobile app</td>
          <td>offline state、draft、cache、local profile</td>
          <td>app upgrade、device loss、cloud backup leakage</td>
      </tr>
      <tr>
          <td>Desktop app</td>
          <td>user profile、history、settings</td>
          <td>profile corruption、manual file copy、multi-version app</td>
      </tr>
      <tr>
          <td>CLI tool</td>
          <td>local index、metadata、state cache</td>
          <td>command interruption、portable file path</td>
      </tr>
      <tr>
          <td>Browser / profile</td>
          <td>cookies、history、bookmark 類資料</td>
          <td>privacy、profile migration、lock collision</td>
      </tr>
      <tr>
          <td>Embedded device</td>
          <td>offline event、sensor / config state</td>
          <td>power loss、flash wear、delayed sync</td>
      </tr>
  </tbody>
</table>
<p>這張表的重點是資料角色而非產品名稱。同樣是 SQLite file，cache 可以清掉重建；draft、local-only note、sensor event 或 user history 可能需要正式 backup / export / delete。</p>
<h2 id="backup-與-export">Backup 與 export</h2>
<p>Embedded backup 的核心責任是讓使用者或服務能從 device / profile failure 復原。Mobile / desktop / CLI 的 backup 路徑常和 OS backup、app export、cloud sync 或手動複製混在一起；SQLite file lifecycle 要明確。</p>
<table>
  <thead>
      <tr>
          <th>路徑</th>
          <th>適合資料</th>
          <th>注意事項</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>OS / device backup</td>
          <td>user-owned local state</td>
          <td>local PII、encryption、restore compatibility</td>
      </tr>
      <tr>
          <td>App export</td>
          <td>使用者可攜資料</td>
          <td>schema version、format stability、privacy</td>
      </tr>
      <tr>
          <td><code>.backup</code> / snapshot</td>
          <td>application-managed backup</td>
          <td>live DB consistency、WAL sidecar handling</td>
      </tr>
      <tr>
          <td>Cloud sync</td>
          <td>multi-device state</td>
          <td>conflict、server authority、delete propagation</td>
      </tr>
  </tbody>
</table>
<p>Backup 設計要先決定 restore target。Restore 到同 app version、未來 app version、或不同 device，會帶來不同 schema compatibility 與 privacy requirement。</p>
<h2 id="privacy-與-local-pii">Privacy 與 local PII</h2>
<p>Embedded SQLite 的 privacy 責任是治理 device-local data。資料在 server DB 中通常有 access log、IAM、DLP 與 retention policy；進入 SQLite file 後，風險轉到 device encryption、app sandbox、backup retention、debug export 與 support bundle。</p>
<table>
  <thead>
      <tr>
          <th>風險</th>
          <th>真實情境</th>
          <th>控制方向</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Local PII</td>
          <td>profile、token、message、draft</td>
          <td>最小化欄位、加密敏感值、限制 export</td>
      </tr>
      <tr>
          <td>Backup leakage</td>
          <td>OS cloud backup 含 database file</td>
          <td>設定 backup exclusion 或加密</td>
      </tr>
      <tr>
          <td>Support bundle</td>
          <td>使用者回報問題附上 DB</td>
          <td>scrub / redaction、只匯出必要 table</td>
      </tr>
      <tr>
          <td>Delete request</td>
          <td>server 刪除但 device local 留存</td>
          <td>sync delete、local purge、retention evidence</td>
      </tr>
  </tbody>
</table>
<p>SQLite file 要進入資料保護盤點。若 local DB 保存敏感資料，應連到 <a href="/blog/backend/07-security-data-protection/data-protection-and-masking-governance/" data-link-title="7.4 資料保護與遮罩治理" data-link-desc="以問題驅動方式整理資料分級、遮罩、匯出與備份治理">Data Protection</a> 與 <a href="/blog/backend/knowledge-cards/audit-log/" data-link-title="Audit Log" data-link-desc="說明高風險操作如何留下可追溯、可稽核的紀錄">Audit Log</a> 的相同問題，只是控制面改在 device / app。</p>
<h2 id="app-upgrade-與-schema-compatibility">App upgrade 與 schema compatibility</h2>
<p>App upgrade 的核心責任是保證新版 binary 能安全打開舊 database file。Mobile / desktop app 的使用者不會按照 backend deployment order 升級；同一時間可能存在多個 app version 與多個 DB schema version。</p>
<table>
  <thead>
      <tr>
          <th>問題</th>
          <th>設計策略</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>新 app 打舊 DB</td>
          <td>startup migration、<code>user_version</code>、backup before migration</td>
      </tr>
      <tr>
          <td>舊 app 打新 DB</td>
          <td>backward-compatible column、feature gate、minimum supported version</td>
      </tr>
      <tr>
          <td>使用者降版</td>
          <td>export / import、read-only fallback、no-downgrade notice</td>
      </tr>
      <tr>
          <td>多裝置不同版本</td>
          <td>sync protocol version、server-side compatibility</td>
      </tr>
  </tbody>
</table>
<p>這些策略要和 <a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema Migration / Versioning</a> 對齊。Embedded app 的 migration failure 通常直接影響使用者啟動體驗，因此 migration 要能快速、可恢復、可診斷。</p>
<h2 id="sync-boundary">Sync boundary</h2>
<p>Sync boundary 的核心責任是把 single-device SQLite 和 multi-device state 分開。SQLite 保存本地狀態；跨裝置同步需要 transport、identity、conflict resolution、delete propagation 與 server authority。</p>
<table>
  <thead>
      <tr>
          <th>Sync 需求</th>
          <th>SQLite 角色</th>
          <th>下一步路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>單裝置 offline</td>
          <td>local source of truth</td>
          <td>SQLite + backup / export</td>
      </tr>
      <tr>
          <td>多裝置同步</td>
          <td>local replica / cache</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/" data-link-title="SQLite Local-first Sync Boundary" data-link-desc="SQLite local-first app、multi-device sync、server authority、conflict resolution、delete propagation 與 offline-first trade-off">Local-first sync boundary</a></td>
      </tr>
      <tr>
          <td>即時多人協作</td>
          <td>local working copy</td>
          <td>server authority、CRDT、event log</td>
      </tr>
      <tr>
          <td>Server reporting</td>
          <td>local data upload / ETL</td>
          <td>API sync、queue、analytics store</td>
      </tr>
  </tbody>
</table>
<p>當 sync 需求出現時，SQLite 仍可作為 local store，但不再單獨承擔完整資料一致性。完整性要由 sync protocol 與 server-side validation 補上。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1把-cache-當正式資料">Case 1：把 cache 當正式資料</h3>
<p>Cache 被誤當正式資料的核心風險是清除 local DB 會造成不可恢復資料損失。許多 app 初期把 SQLite 當 cache；後來加入 draft、offline action 或 local-only setting，資料責任就改變了。</p>
<p>修正方向是逐 table 標示資料角色。Cache table 可清；formal state table 要 backup、migration、export 與 delete policy。</p>
<h3 id="case-2os-backup-帶走敏感資料">Case 2：OS backup 帶走敏感資料</h3>
<p>OS backup 的核心風險是 device-local PII 進入使用者或平台雲端備份。Server 端已刪除的資料，可能仍存在 device backup。</p>
<p>修正方向是決定哪些資料可被備份。Token、secret、敏感 PII 可排除或加密；user-owned content 則要提供 export / restore 語意。</p>
<h3 id="case-3app-upgrade-migration-失敗讓使用者卡在啟動頁">Case 3：App upgrade migration 失敗讓使用者卡在啟動頁</h3>
<p>Startup migration 失敗的核心風險是使用者卡在 app 啟動前，且修復能力有限。SQLite file 在使用者裝置上，SRE 通常需要透過 app update、support bundle 或 restore flow 處理。</p>
<p>修正方向是保留 pre-migration snapshot、提供 safe mode、收集匿名 schema / error evidence，並避免長 migration 放在 cold start。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>Embedded SQLite 設計要回答：</p>
<ol>
<li>每張 table 是 cache、formal state、derived state 還是 sync queue。</li>
<li>Database file 在 app / OS 的哪個 storage boundary。</li>
<li>OS backup 是否包含 database file。</li>
<li>敏感欄位是否加密、排除或可清除。</li>
<li>App upgrade migration 是否有 pre-migration backup。</li>
<li>使用者 export / delete / support bundle 如何處理 SQLite data。</li>
<li>Multi-device sync 是否有 conflict 與 server authority 設計。</li>
</ol>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview</a></li>
<li>Sibling：<a href="/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/" data-link-title="SQLite Local-first Sync Boundary" data-link-desc="SQLite local-first app、multi-device sync、server authority、conflict resolution、delete propagation 與 offline-first trade-off">Local-first Sync Boundary</a>、<a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema Migration / Versioning</a></li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a></li>
<li>跨模組：<a href="/blog/backend/07-security-data-protection/data-protection-and-masking-governance/" data-link-title="7.4 資料保護與遮罩治理" data-link-desc="以問題驅動方式整理資料分級、遮罩、匯出與備份治理">Data Protection</a></li>
<li>官方：<a href="https://www.sqlite.org/whentouse.html">SQLite Appropriate Uses</a>、<a href="https://www.sqlite.org/backup.html">SQLite Backup API</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Observability and Runbook</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/observability-runbook/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/observability-runbook/</guid><description>&lt;p>SQLite observability and runbook 的核心責任是把低操作成本服務補成可交接的 production evidence。SQLite 的元件少，但正式服務仍需要觀測 busy errors、WAL growth、backup freshness、restore drill、disk usage、migration result、file permission 與 application-level query health。&lt;/p>
&lt;p>本文的判讀錨點是：SQLite 的 observability 要貼近 file、process、filesystem 與 application。它通常沒有 server DB 那種長駐監控平面，因此 runbook 要把 signal 從 app metrics、log、scheduled job、file metadata 與 restore evidence 裡組出來。&lt;/p>
&lt;h2 id="signal-inventory">Signal Inventory&lt;/h2>
&lt;p>Signal inventory 的核心責任是列出 SQLite production 化後最能預告事故的訊號。這些訊號要放進 dashboard、log search 或 scheduled report，讓事故前後都能直接查。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Signal&lt;/th>
 &lt;th>來源&lt;/th>
 &lt;th>代表風險&lt;/th>
 &lt;th>建議反應&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>SQLITE_BUSY&lt;/code> count&lt;/td>
 &lt;td>app log / metric&lt;/td>
 &lt;td>writer contention、long reader&lt;/td>
 &lt;td>查 transaction duration、busy timeout&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>WAL file size&lt;/td>
 &lt;td>filesystem metric&lt;/td>
 &lt;td>checkpoint lag、long reader&lt;/td>
 &lt;td>查 checkpoint result、reader age&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Backup age&lt;/td>
 &lt;td>scheduled job metric&lt;/td>
 &lt;td>RPO 擴大&lt;/td>
 &lt;td>重跑 backup、檢查 storage&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Restore drill age&lt;/td>
 &lt;td>release evidence&lt;/td>
 &lt;td>RTO 信心下降&lt;/td>
 &lt;td>排程 restore drill&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Disk free&lt;/td>
 &lt;td>host / platform metric&lt;/td>
 &lt;td>write failure、checkpoint failure&lt;/td>
 &lt;td>清理、擴容、降級寫入&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Migration version&lt;/td>
 &lt;td>app startup / metadata&lt;/td>
 &lt;td>schema drift&lt;/td>
 &lt;td>block release、跑 validation&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Integrity check result&lt;/td>
 &lt;td>maintenance job&lt;/td>
 &lt;td>corruption / storage issue&lt;/td>
 &lt;td>進入 restore decision&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>&lt;code>SQLITE_BUSY&lt;/code> 是 writer boundary 的最直接訊號。它可能代表長交易、read cursor 未關、parallel test 共用 DB、checkpoint 壓力或 write burst；runbook 要先查 query duration 與 transaction boundary，再調 busy timeout。&lt;/p>
&lt;p>WAL size 是 checkpoint 與 reader 壓力的綜合訊號。WAL 持續成長時，先確認是否有長 reader、backup process、未完成 transaction 或 checkpoint 失敗；接著才考慮手動 checkpoint。&lt;/p>
&lt;p>Backup age 是 RPO 的可觀測版本。若目標 RPO 是 5 分鐘，dashboard 就要顯示 last successful backup / replica time 與警戒線。&lt;/p>
&lt;h2 id="backup-evidence">Backup Evidence&lt;/h2>
&lt;p>Backup evidence 的核心責任是證明資料可被拿回來。SQLite backup 的完成標準包含成功建立備份、保存 sidecar 語意、恢復到新路徑、通過 integrity check、跑 application smoke test。&lt;/p></description><content:encoded><![CDATA[<p>SQLite observability and runbook 的核心責任是把低操作成本服務補成可交接的 production evidence。SQLite 的元件少，但正式服務仍需要觀測 busy errors、WAL growth、backup freshness、restore drill、disk usage、migration result、file permission 與 application-level query health。</p>
<p>本文的判讀錨點是：SQLite 的 observability 要貼近 file、process、filesystem 與 application。它通常沒有 server DB 那種長駐監控平面，因此 runbook 要把 signal 從 app metrics、log、scheduled job、file metadata 與 restore evidence 裡組出來。</p>
<h2 id="signal-inventory">Signal Inventory</h2>
<p>Signal inventory 的核心責任是列出 SQLite production 化後最能預告事故的訊號。這些訊號要放進 dashboard、log search 或 scheduled report，讓事故前後都能直接查。</p>
<table>
  <thead>
      <tr>
          <th>Signal</th>
          <th>來源</th>
          <th>代表風險</th>
          <th>建議反應</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>SQLITE_BUSY</code> count</td>
          <td>app log / metric</td>
          <td>writer contention、long reader</td>
          <td>查 transaction duration、busy timeout</td>
      </tr>
      <tr>
          <td>WAL file size</td>
          <td>filesystem metric</td>
          <td>checkpoint lag、long reader</td>
          <td>查 checkpoint result、reader age</td>
      </tr>
      <tr>
          <td>Backup age</td>
          <td>scheduled job metric</td>
          <td>RPO 擴大</td>
          <td>重跑 backup、檢查 storage</td>
      </tr>
      <tr>
          <td>Restore drill age</td>
          <td>release evidence</td>
          <td>RTO 信心下降</td>
          <td>排程 restore drill</td>
      </tr>
      <tr>
          <td>Disk free</td>
          <td>host / platform metric</td>
          <td>write failure、checkpoint failure</td>
          <td>清理、擴容、降級寫入</td>
      </tr>
      <tr>
          <td>Migration version</td>
          <td>app startup / metadata</td>
          <td>schema drift</td>
          <td>block release、跑 validation</td>
      </tr>
      <tr>
          <td>Integrity check result</td>
          <td>maintenance job</td>
          <td>corruption / storage issue</td>
          <td>進入 restore decision</td>
      </tr>
  </tbody>
</table>
<p><code>SQLITE_BUSY</code> 是 writer boundary 的最直接訊號。它可能代表長交易、read cursor 未關、parallel test 共用 DB、checkpoint 壓力或 write burst；runbook 要先查 query duration 與 transaction boundary，再調 busy timeout。</p>
<p>WAL size 是 checkpoint 與 reader 壓力的綜合訊號。WAL 持續成長時，先確認是否有長 reader、backup process、未完成 transaction 或 checkpoint 失敗；接著才考慮手動 checkpoint。</p>
<p>Backup age 是 RPO 的可觀測版本。若目標 RPO 是 5 分鐘，dashboard 就要顯示 last successful backup / replica time 與警戒線。</p>
<h2 id="backup-evidence">Backup Evidence</h2>
<p>Backup evidence 的核心責任是證明資料可被拿回來。SQLite backup 的完成標準包含成功建立備份、保存 sidecar 語意、恢復到新路徑、通過 integrity check、跑 application smoke test。</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>最小內容</th>
          <th>失敗時路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Backup job result</td>
          <td>timestamp、duration、file size、target</td>
          <td>重跑 job、檢查 credential / disk</td>
      </tr>
      <tr>
          <td>Restore artifact</td>
          <td>restored path、checksum、row count</td>
          <td>回前一份 backup、檢查 WAL / snapshot</td>
      </tr>
      <tr>
          <td>Integrity result</td>
          <td><code>PRAGMA integrity_check;</code></td>
          <td>停止寫入、進入 corruption triage</td>
      </tr>
      <tr>
          <td>Application smoke test</td>
          <td>啟動、讀核心頁、寫測試資料</td>
          <td>rollback、保留 evidence</td>
      </tr>
      <tr>
          <td>Retention note</td>
          <td>保存天數、刪除策略、legal hold</td>
          <td>更新 data protection policy</td>
      </tr>
  </tbody>
</table>
<p>SQLite 官方 <a href="https://www.sqlite.org/backup.html">backup API</a> 與 CLI <code>.backup</code> 是備份設計的基礎路由。WAL mode 下，直接複製單一 <code>.db</code> 檔容易漏掉 sidecar file 的時序；runbook 應使用 SQLite-aware backup 或經過 checkpoint / stop-the-world 的 snapshot。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.backup &#39;backup/app-2026-05-21.db&#39;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 backup/app-2026-05-21.db <span class="s2">&#34;PRAGMA integrity_check;&#34;</span></span></span></code></pre></div><p>這段命令提供最小 restore evidence 的起點。正式演練要把備份檔複製到隔離路徑，使用相同 application version 啟動，跑核心 read/write smoke test，再記錄耗時與失敗條件。</p>
<h2 id="migration-evidence">Migration Evidence</h2>
<p>Migration evidence 的核心責任是讓 SQLite schema change 可回退、可審查、可交接。單檔 DB 在使用者裝置或服務節點上升級時，migration 失敗會直接影響啟動、資料讀取與同步。</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>內容</th>
          <th>Release gate</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Schema version</td>
          <td><code>PRAGMA user_version</code> 或 migration table</td>
          <td>app startup 比對 expected version</td>
      </tr>
      <tr>
          <td>Pre-migration snapshot</td>
          <td>backup path、size、checksum</td>
          <td>migration 前完成</td>
      </tr>
      <tr>
          <td>Validation query</td>
          <td>row count、FK check、domain invariant</td>
          <td>migration 後立即執行</td>
      </tr>
      <tr>
          <td>Smoke test</td>
          <td>核心 read/write workflow</td>
          <td>app release gate</td>
      </tr>
      <tr>
          <td>Rollback route</td>
          <td>restore snapshot 或 block startup</td>
          <td>migration 失敗時啟動</td>
      </tr>
  </tbody>
</table>
<p>Migration log 要包含版本、耗時、row count、錯誤、validation result 與 rollback decision。若 SQLite file 位於 end-user device，log 還要能被使用者支援流程收集，避免事故只停在「app 開不起來」。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">user_version</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_key_check</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span></span></span></code></pre></div><p>這些 query 是 migration 後的最小 evidence。正式服務要再補 domain-specific invariant，例如「所有 active subscription 都有 owner」、「所有 pending mutation 都有 idempotency key」。</p>
<h2 id="incident-runbook">Incident Runbook</h2>
<p>Incident runbook 的核心責任是把 SQLite 事故分流到正確處置。SQLite 常見事故包含 disk full、busy storm、WAL growth、bad migration、corruption suspicion、backup failure 與 permission error。</p>
<table>
  <thead>
      <tr>
          <th>Incident</th>
          <th>第一個判讀問題</th>
          <th>立即處置</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Busy storm</td>
          <td>有長 transaction 或 write burst 嗎</td>
          <td>暫停非必要寫入、查 transaction duration</td>
      </tr>
      <tr>
          <td>Disk full</td>
          <td>DB / WAL / backup 哪個吃掉空間</td>
          <td>停止寫入、清理 backup、擴容</td>
      </tr>
      <tr>
          <td>WAL growth</td>
          <td>checkpoint 被誰阻擋</td>
          <td>查 reader、跑 checkpoint evidence</td>
      </tr>
      <tr>
          <td>Bad migration</td>
          <td>schema version 與 app version 是否一致</td>
          <td>停止 rollout、restore snapshot、保留 failed DB</td>
      </tr>
      <tr>
          <td>Corruption signal</td>
          <td>integrity check 是否失敗</td>
          <td>進入 read-only、restore last good backup</td>
      </tr>
      <tr>
          <td>Backup failure</td>
          <td>credential、network、destination 是否可用</td>
          <td>切換 destination、補跑 restore drill</td>
      </tr>
  </tbody>
</table>
<p>Busy storm 要先保護使用者操作。可以降低 write endpoint、停用背景 job、延長 retry backoff，然後用 log 查最長 transaction 與最多重試的 query。</p>
<p>Disk full 要先停止寫入。SQLite 在 disk full 時可能讓 write / checkpoint / backup 同時失敗；runbook 要保留剩餘空間、DB file、WAL file、backup directory 與 tmp directory 的大小。</p>
<p>Bad migration 要保留 failed artifact。先複製 failed DB 到 evidence path，記錄 schema version、app version、migration id、validation error，再執行 rollback。</p>
<h2 id="dashboard-and-alert-route">Dashboard and Alert Route</h2>
<p>Dashboard and alert route 的核心責任是讓 SQLite 被納入正式服務的可觀測系統。SQLite signal 常來自 application，因此 metric 命名要接近操作問題。</p>
<table>
  <thead>
      <tr>
          <th>Metric name example</th>
          <th>類型</th>
          <th>用途</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>sqlite_busy_total</code></td>
          <td>counter</td>
          <td>writer contention</td>
      </tr>
      <tr>
          <td><code>sqlite_query_duration_ms</code></td>
          <td>histogram</td>
          <td>slow query / long transaction</td>
      </tr>
      <tr>
          <td><code>sqlite_wal_size_bytes</code></td>
          <td>gauge</td>
          <td>checkpoint pressure</td>
      </tr>
      <tr>
          <td><code>sqlite_backup_age_seconds</code></td>
          <td>gauge</td>
          <td>RPO evidence</td>
      </tr>
      <tr>
          <td><code>sqlite_restore_drill_age_days</code></td>
          <td>gauge</td>
          <td>RTO confidence</td>
      </tr>
      <tr>
          <td><code>sqlite_disk_free_bytes</code></td>
          <td>gauge</td>
          <td>disk full prevention</td>
      </tr>
      <tr>
          <td><code>sqlite_migration_version</code></td>
          <td>gauge</td>
          <td>schema drift</td>
      </tr>
  </tbody>
</table>
<p>Alert 要連到 runbook，並提供可執行的第一步。每個 alert 至少要有 owner、severity、first query、rollback condition 與 escalation route。</p>
<p>Log schema 要保留 query category，而非只記原始 SQL。正式服務通常應避免把完整 SQL 與 PII 直接寫入 log；可以記 operation name、duration、row count、error code、busy retry count 與 correlation id。</p>
<h2 id="handoff">Handoff</h2>
<p>Handoff 的核心責任是讓下一個維護者知道 SQLite service 的邊界。交接文件要把「誰負責檔案」、「誰負責備份」、「誰能執行 restore」、「何時升級資料庫」寫清楚。</p>
<p>最小 handoff 包含：</p>
<ol>
<li>Database file path、sidecar file policy、journal mode 與 PRAGMA baseline。</li>
<li>Backup command、destination、retention、last restore drill。</li>
<li>Migration command、schema version、rollback route。</li>
<li>Alert list、dashboard link、incident owner。</li>
<li>Known limits：writer concurrency、file size、edge / sync boundary。</li>
<li>Next route：PostgreSQL、D1 / Turso、Litestream / LiteFS 的評估條件。</li>
</ol>
<p>Handoff 的重點是把低操作成本保留下來。SQLite 的好處來自少元件；可交接文件讓少元件不等於少 evidence。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>Observability / runbook 完成後，下一步要接到具體演練。Backup 與 restore 讀 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/backup-restore-drill/" data-link-title="SQLite Backup Restore Drill" data-link-desc="SQLite .backup、VACUUM INTO、restore validation、sidecar file handling 與 RPO / RTO note 的操作說明">SQLite backup restore drill</a>；WAL 與 busy 讀 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a>；正式服務的 evidence 可對齊 <a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">Observability Evidence Package</a> 與 <a href="/blog/backend/08-incident-response/incident-decision-log/" data-link-title="8.19 Incident Decision Log" data-link-desc="把事中假設、決策、證據、回退條件與責任人留下可復盤紀錄">Incident Decision Log</a>。</p>
]]></content:encoded></item><item><title>SQLite PRAGMA Tuning and Performance</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/pragma-tuning-performance/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/pragma-tuning-performance/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的容量規劃要點；本文聚焦 &lt;em>PRAGMA 設定如何變成 durability、latency、檔案大小與 restore risk 的取捨&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite PRAGMA tuning 的核心責任是把單檔資料庫的行為固定成可重複、可觀測、可回退的操作契約。SQLite 的許多重要行為由 connection-level 或 database-level PRAGMA 控制；這些設定看起來像小開關，實際上會影響 crash recovery、commit latency、reader / writer 衝突、檔案大小與測試一致性。&lt;/p>
&lt;p>本文的判讀錨點是：PRAGMA 是 durability / latency / maintenance 的顯性取捨，而非效能魔法。Production runbook 要記錄設定值、設定時機、驗證 query 與回退條件，避免不同 process、test runner 或 migration tool 用不同 SQLite 行為。&lt;/p>
&lt;h2 id="baseline-pragma">Baseline PRAGMA&lt;/h2>
&lt;p>SQLite baseline PRAGMA 的責任是讓 application 每次啟動都進入同一個資料庫模式。對 production-like local store、small backend 或 test fixture，建議把 journal、sync、foreign key、busy timeout 與 checkpoint 明確設定，而非依賴語言 binding 預設值。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">journal_mode&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">WAL&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">synchronous&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">NORMAL&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">foreign_keys&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">ON&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">busy_timeout&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">5000&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">5&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">wal_autocheckpoint&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">1000&lt;/span>&lt;span class="p">;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>設定&lt;/th>
 &lt;th>服務責任&lt;/th>
 &lt;th>驗證方式&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>journal_mode=WAL&lt;/code>&lt;/td>
 &lt;td>降低 reader / writer 衝突&lt;/td>
 &lt;td>回傳值為 &lt;code>wal&lt;/code>，觀察 &lt;code>-wal&lt;/code> file&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>synchronous=NORMAL&lt;/code>&lt;/td>
 &lt;td>平衡 fsync cost 與 crash durability&lt;/td>
 &lt;td>查 &lt;code>PRAGMA synchronous&lt;/code>，跑 restore drill&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>foreign_keys=ON&lt;/code>&lt;/td>
 &lt;td>啟用 FK enforcement&lt;/td>
 &lt;td>&lt;code>PRAGMA foreign_key_check&lt;/code>&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>busy_timeout&lt;/code>&lt;/td>
 &lt;td>吸收短暫 writer queue&lt;/td>
 &lt;td>記錄 busy wait 與 timeout rate&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>wal_autocheckpoint&lt;/code>&lt;/td>
 &lt;td>控制 WAL growth cadence&lt;/td>
 &lt;td>觀察 WAL size 與 checkpoint duration&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的重點是把設定與 evidence 綁在一起。若某個 PRAGMA 缺少成功訊號與失敗訊號，就先維持保守預設；盲目追求「最快」通常會把風險推到 power loss、restore 或長尾 latency。&lt;/p>
&lt;h2 id="journal_mode-與-wal-boundary">&lt;code>journal_mode&lt;/code> 與 WAL boundary&lt;/h2>
&lt;p>&lt;code>journal_mode&lt;/code> 的核心責任是決定 transaction 如何保護原始資料。SQLite 預設 rollback journal 對簡單場景合理；WAL mode 則讓 reader 可以在 writer append WAL 時保有 snapshot，適合多 reader、短寫入、互動式 workload。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>模式&lt;/th>
 &lt;th>適合情境&lt;/th>
 &lt;th>注意事項&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>DELETE&lt;/code>&lt;/td>
 &lt;td>最簡單、低併發、短生命週期檔案&lt;/td>
 &lt;td>write / read 衝突較明顯&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>WAL&lt;/code>&lt;/td>
 &lt;td>read-heavy、local app、小型 API&lt;/td>
 &lt;td>需要治理 &lt;code>-wal&lt;/code>、&lt;code>-shm&lt;/code>、checkpoint&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>MEMORY&lt;/code>&lt;/td>
 &lt;td>暫存測試、可丟資料&lt;/td>
 &lt;td>crash 後 recovery 風險高&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>&lt;code>OFF&lt;/code>&lt;/td>
 &lt;td>可重建資料、一次性 bulk load&lt;/td>
 &lt;td>production formal state 應避開&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>WAL mode 是多數 production-like SQLite 的 baseline，但它也引入 sidecar file 與 checkpoint 責任。完整判讀見 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking&lt;/a>。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的容量規劃要點；本文聚焦 <em>PRAGMA 設定如何變成 durability、latency、檔案大小與 restore risk 的取捨</em>。</p></blockquote>
<p>SQLite PRAGMA tuning 的核心責任是把單檔資料庫的行為固定成可重複、可觀測、可回退的操作契約。SQLite 的許多重要行為由 connection-level 或 database-level PRAGMA 控制；這些設定看起來像小開關，實際上會影響 crash recovery、commit latency、reader / writer 衝突、檔案大小與測試一致性。</p>
<p>本文的判讀錨點是：PRAGMA 是 durability / latency / maintenance 的顯性取捨，而非效能魔法。Production runbook 要記錄設定值、設定時機、驗證 query 與回退條件，避免不同 process、test runner 或 migration tool 用不同 SQLite 行為。</p>
<h2 id="baseline-pragma">Baseline PRAGMA</h2>
<p>SQLite baseline PRAGMA 的責任是讓 application 每次啟動都進入同一個資料庫模式。對 production-like local store、small backend 或 test fixture，建議把 journal、sync、foreign key、busy timeout 與 checkpoint 明確設定，而非依賴語言 binding 預設值。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">journal_mode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">WAL</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">synchronous</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">NORMAL</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">ON</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">busy_timeout</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">5000</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">wal_autocheckpoint</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">1000</span><span class="p">;</span></span></span></code></pre></div><table>
  <thead>
      <tr>
          <th>設定</th>
          <th>服務責任</th>
          <th>驗證方式</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>journal_mode=WAL</code></td>
          <td>降低 reader / writer 衝突</td>
          <td>回傳值為 <code>wal</code>，觀察 <code>-wal</code> file</td>
      </tr>
      <tr>
          <td><code>synchronous=NORMAL</code></td>
          <td>平衡 fsync cost 與 crash durability</td>
          <td>查 <code>PRAGMA synchronous</code>，跑 restore drill</td>
      </tr>
      <tr>
          <td><code>foreign_keys=ON</code></td>
          <td>啟用 FK enforcement</td>
          <td><code>PRAGMA foreign_key_check</code></td>
      </tr>
      <tr>
          <td><code>busy_timeout</code></td>
          <td>吸收短暫 writer queue</td>
          <td>記錄 busy wait 與 timeout rate</td>
      </tr>
      <tr>
          <td><code>wal_autocheckpoint</code></td>
          <td>控制 WAL growth cadence</td>
          <td>觀察 WAL size 與 checkpoint duration</td>
      </tr>
  </tbody>
</table>
<p>這張表的重點是把設定與 evidence 綁在一起。若某個 PRAGMA 缺少成功訊號與失敗訊號，就先維持保守預設；盲目追求「最快」通常會把風險推到 power loss、restore 或長尾 latency。</p>
<h2 id="journal_mode-與-wal-boundary"><code>journal_mode</code> 與 WAL boundary</h2>
<p><code>journal_mode</code> 的核心責任是決定 transaction 如何保護原始資料。SQLite 預設 rollback journal 對簡單場景合理；WAL mode 則讓 reader 可以在 writer append WAL 時保有 snapshot，適合多 reader、短寫入、互動式 workload。</p>
<table>
  <thead>
      <tr>
          <th>模式</th>
          <th>適合情境</th>
          <th>注意事項</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>DELETE</code></td>
          <td>最簡單、低併發、短生命週期檔案</td>
          <td>write / read 衝突較明顯</td>
      </tr>
      <tr>
          <td><code>WAL</code></td>
          <td>read-heavy、local app、小型 API</td>
          <td>需要治理 <code>-wal</code>、<code>-shm</code>、checkpoint</td>
      </tr>
      <tr>
          <td><code>MEMORY</code></td>
          <td>暫存測試、可丟資料</td>
          <td>crash 後 recovery 風險高</td>
      </tr>
      <tr>
          <td><code>OFF</code></td>
          <td>可重建資料、一次性 bulk load</td>
          <td>production formal state 應避開</td>
      </tr>
  </tbody>
</table>
<p>WAL mode 是多數 production-like SQLite 的 baseline，但它也引入 sidecar file 與 checkpoint 責任。完整判讀見 <a href="/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking</a>。</p>
<h2 id="synchronouscommit-latency-與資料損失窗口"><code>synchronous</code>：commit latency 與資料損失窗口</h2>
<p><code>synchronous</code> 的核心責任是控制 SQLite 在關鍵時刻要求 storage flush 的強度。官方 PRAGMA 文件說明 WAL mode 下 <code>NORMAL</code> 會把 sync 主要放在 checkpoint 路徑；這通常讓 commit 更快，但 crash durability 的語意要由 service owner 接受。</p>
<table>
  <thead>
      <tr>
          <th>設定</th>
          <th>服務語意</th>
          <th>適合情境</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>FULL</code></td>
          <td>更保守的 durability</td>
          <td>金錢、ledger、不可重建 local state</td>
      </tr>
      <tr>
          <td><code>NORMAL</code></td>
          <td>多數 WAL production-like baseline</td>
          <td>local app、小型服務、可接受極小 crash window</td>
      </tr>
      <tr>
          <td><code>OFF</code></td>
          <td>追求速度，放棄重要 durability</td>
          <td>scratch DB、可重建 cache、bulk import staging</td>
      </tr>
  </tbody>
</table>
<p><code>synchronous=OFF</code> 要被視為明確風險接受。若資料是 <a href="/blog/backend/knowledge-cards/source-of-truth/" data-link-title="Source of Truth" data-link-desc="說明正式資料來源如何決定資料判斷、修復與一致性責任">source of truth</a>，設定檔、runbook 與 review 都應避免把 staging 的快速設定帶進 production。</p>
<h2 id="cachemmap-與-memory-pressure">Cache、mmap 與 memory pressure</h2>
<p>SQLite memory tuning 的核心責任是降低 read path I/O，同時避免把 device / container memory 壓到不可控。<code>cache_size</code> 控制 SQLite page cache；<code>mmap_size</code> 讓讀取可透過 memory-mapped I/O 加速，但仍受平台、檔案大小與 memory budget 影響。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">cache_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">64000</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">mmap_size</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">268435456</span><span class="p">;</span></span></span></code></pre></div><table>
  <thead>
      <tr>
          <th>設定</th>
          <th>改善目標</th>
          <th>觀測訊號</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>cache_size</code></td>
          <td>減少重複 page read</td>
          <td>query latency、disk read、memory usage</td>
      </tr>
      <tr>
          <td><code>mmap_size</code></td>
          <td>降低 read syscall cost</td>
          <td>p95 / p99 read latency、address space</td>
      </tr>
      <tr>
          <td><code>temp_store</code></td>
          <td>控制 temp table 位置</td>
          <td>sort / join query latency、memory pressure</td>
      </tr>
  </tbody>
</table>
<p>Memory 設定要和 workload size 一起看。Desktop app、mobile app、edge worker、container service 的 memory ceiling 不同；把 server 上的設定複製到 mobile 或 edge runtime 會讓風險轉移到 OOM 或 OS reclaim。</p>
<h2 id="vacuum-與檔案大小治理">Vacuum 與檔案大小治理</h2>
<p>Vacuum 設定的核心責任是控制 delete 後的空間回收。SQLite delete row 後，database file 不會自然縮小；<code>auto_vacuum</code> 要在 database 建立早期決定，後續切換通常需要 <code>VACUUM</code> 重整整個 database。</p>
<table>
  <thead>
      <tr>
          <th>設定 / 操作</th>
          <th>適合情境</th>
          <th>風險 / 成本</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>auto_vacuum=NONE</code></td>
          <td>資料量穩定、delete 少</td>
          <td>檔案可能長期保持高水位</td>
      </tr>
      <tr>
          <td><code>auto_vacuum=INCREMENTAL</code></td>
          <td>需要逐步回收空間</td>
          <td>需要排程 <code>incremental_vacuum</code></td>
      </tr>
      <tr>
          <td><code>VACUUM</code></td>
          <td>maintenance window、重整資料庫</td>
          <td>需要額外空間與 I/O，可能影響服務</td>
      </tr>
      <tr>
          <td><code>VACUUM INTO</code></td>
          <td>compact copy / backup</td>
          <td>產出新檔，適合 restore drill 或 export</td>
      </tr>
  </tbody>
</table>
<p>檔案大小治理要接到 backup 成本。Database file 長期膨脹會放大備份時間、restore 時間與 edge deploy artifact size；若服務有大量 delete / churn，vacuum policy 要被寫進 runbook。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1pragma-只在某個-connection-設定">Case 1：PRAGMA 只在某個 connection 設定</h3>
<p>Connection-level PRAGMA 的核心風險是不同程式路徑行為不一致。Application 啟動時設了 <code>foreign_keys=ON</code>，migration tool 或 test runner 沒設，就會出現 production / migration / test 三種語意。</p>
<p>修正方向是把 baseline PRAGMA 放進 shared DB open path，並在 startup health check 印出設定值。Migration CLI、background worker、test fixture 都要共用同一份 connection initialization。</p>
<h3 id="case-2synchronousoff-從測試環境流到正式資料">Case 2：<code>synchronous=OFF</code> 從測試環境流到正式資料</h3>
<p>快速測試設定外流的核心風險是資料損失只在 crash 後出現。平常 query 都正常，直到 power loss、container kill 或 host crash 後，資料庫出現落差。</p>
<p>修正方向是設定分層。Test / benchmark 可以用 faster profile；formal state profile 要用 <code>NORMAL</code> 或 <code>FULL</code>，並要求 restore drill。</p>
<h3 id="case-3wal-growth-被誤判成資料成長">Case 3：WAL growth 被誤判成資料成長</h3>
<p>WAL growth 的核心風險是 checkpoint 問題被當成容量問題。Disk alert 看到 <code>db-wal</code> 變大，若只擴 disk，長 reader 或 checkpoint starvation 仍會持續。</p>
<p>修正方向是把 WAL size、checkpoint return 與 long reader 一起看。先找 reader lifecycle，再調 checkpoint cadence。</p>
<h3 id="case-4vacuum-在高峰期執行">Case 4：Vacuum 在高峰期執行</h3>
<p>Vacuum 的核心風險是把 maintenance I/O 放到使用者路徑。檔案縮小是好事，但 full vacuum 會消耗 I/O 與時間，對 mobile / desktop / small backend 都可能造成卡頓。</p>
<p>修正方向是把 vacuum 當 maintenance job。大檔案用 <code>incremental_vacuum</code> 或低流量窗口；備份前的 compact copy 可考慮 <code>VACUUM INTO</code>。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>SQLite PRAGMA runbook 至少要記錄：</p>
<ol>
<li>所有 connection 初始化時執行的 baseline PRAGMA。</li>
<li><code>journal_mode</code> 實際回傳值與 sidecar file 位置。</li>
<li><code>synchronous</code> profile 與資料風險接受者。</li>
<li><code>busy_timeout</code> 值、busy wait metric、timeout threshold。</li>
<li><code>wal_autocheckpoint</code>、manual checkpoint cadence 與 WAL size alert。</li>
<li><code>cache_size</code> / <code>mmap_size</code> 對 memory budget 的影響。</li>
<li><code>auto_vacuum</code> / <code>VACUUM</code> / <code>VACUUM INTO</code> 的 maintenance window。</li>
</ol>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview</a></li>
<li>前置：<a href="/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking</a></li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a></li>
<li>平行：<a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">Observability / runbook</a></li>
<li>官方：<a href="https://www.sqlite.org/pragma.html">SQLite PRAGMA</a>、<a href="https://www.sqlite.org/lang_vacuum.html">SQLite VACUUM</a>、<a href="https://www.sqlite.org/wal.html">SQLite WAL</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite Schema Migration and Versioning</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的 embedded / single-file 定位；本文聚焦 &lt;em>schema version、ALTER TABLE boundary、table rebuild migration 與 application release compatibility&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite schema migration 的核心責任是讓單檔資料庫隨 application release 安全演進。SQLite 沒有獨立 database server，也沒有 DBA 在 server 端統一套 migration；migration 常在 application startup、CLI command、mobile app upgrade 或 desktop app launch 時發生，因此 schema version、binary compatibility、backup 與 rollback 要放在同一個 release contract 中設計。&lt;/p>
&lt;p>本文的判讀錨點是：SQLite migration 同時改資料庫檔案與 application 能讀的資料格式。只要使用者或服務可能拿舊 binary 打開新 database，或新 binary 打開舊 database，migration 就要處理 forward / backward compatibility，而不只是 SQL 成功執行。&lt;/p>
&lt;h2 id="version-model">Version model&lt;/h2>
&lt;p>SQLite schema versioning 的服務責任是讓 application 能判斷 database file 目前處於哪個契約。SQLite 提供 &lt;code>PRAGMA user_version&lt;/code> 作為 application-controlled integer；更複雜的服務也可以用 migration table 記錄多步驟版本、checksum 與執行時間。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">user_version&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">user_version&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">2026052101&lt;/span>&lt;span class="p">;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>方式&lt;/th>
 &lt;th>適合情境&lt;/th>
 &lt;th>優點&lt;/th>
 &lt;th>邊界&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>user_version&lt;/code>&lt;/td>
 &lt;td>mobile / desktop / CLI single file&lt;/td>
 &lt;td>簡單、內建、開檔即可讀&lt;/td>
 &lt;td>只能存一個整數，缺 migration history&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>migration table&lt;/td>
 &lt;td>small backend、多人維護 schema&lt;/td>
 &lt;td>可記錄每步 migration 與 owner&lt;/td>
 &lt;td>需要先建立 table 與初始化流程&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>external manifest&lt;/td>
 &lt;td>fixture、artifact、read-only DB&lt;/td>
 &lt;td>可和 release artifact 綁定&lt;/td>
 &lt;td>DB file 本身不含完整 history&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Version model 要在第一版就定義。沒有版本欄位的 SQLite file 仍可 migration，但 application 只能靠 introspection 猜 schema，會讓 upgrade / downgrade runbook 複雜化。&lt;/p>
&lt;h2 id="alter-table-boundary">ALTER TABLE boundary&lt;/h2>
&lt;p>SQLite ALTER TABLE 的核心責任是處理有限集合的 schema 變更。官方文件說明 SQLite 支援 rename table、rename column、add column、drop column；更複雜的變更要走 table rebuild pattern。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>變更類型&lt;/th>
 &lt;th>SQLite 支援形態&lt;/th>
 &lt;th>操作判讀&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Rename table / column&lt;/td>
 &lt;td>直接 ALTER，版本差異影響 trigger / view&lt;/td>
 &lt;td>需要測 trigger、view、FK reference&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Add column&lt;/td>
 &lt;td>多數情境很快，受 default / constraint 限制&lt;/td>
 &lt;td>適合 expand migration&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Drop column&lt;/td>
 &lt;td>需要檢查 index、constraint、trigger、view&lt;/td>
 &lt;td>可能掃資料，需 maintenance window&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Change type / constraint&lt;/td>
 &lt;td>通常走 table rebuild&lt;/td>
 &lt;td>需要完整 copy、foreign key check、validation&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>SQLite schema 存在 &lt;code>sqlite_schema&lt;/code> 的 SQL text 中；這讓檔案格式簡潔，但也讓 ALTER TABLE 的安全條件和 server SQL 不同。Production migration 應優先用官方建議的 rebuild procedure，而非直接修改 &lt;code>sqlite_schema&lt;/code>。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的 embedded / single-file 定位；本文聚焦 <em>schema version、ALTER TABLE boundary、table rebuild migration 與 application release compatibility</em>。</p></blockquote>
<p>SQLite schema migration 的核心責任是讓單檔資料庫隨 application release 安全演進。SQLite 沒有獨立 database server，也沒有 DBA 在 server 端統一套 migration；migration 常在 application startup、CLI command、mobile app upgrade 或 desktop app launch 時發生，因此 schema version、binary compatibility、backup 與 rollback 要放在同一個 release contract 中設計。</p>
<p>本文的判讀錨點是：SQLite migration 同時改資料庫檔案與 application 能讀的資料格式。只要使用者或服務可能拿舊 binary 打開新 database，或新 binary 打開舊 database，migration 就要處理 forward / backward compatibility，而不只是 SQL 成功執行。</p>
<h2 id="version-model">Version model</h2>
<p>SQLite schema versioning 的服務責任是讓 application 能判斷 database file 目前處於哪個契約。SQLite 提供 <code>PRAGMA user_version</code> 作為 application-controlled integer；更複雜的服務也可以用 migration table 記錄多步驟版本、checksum 與執行時間。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">user_version</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">user_version</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">2026052101</span><span class="p">;</span></span></span></code></pre></div><table>
  <thead>
      <tr>
          <th>方式</th>
          <th>適合情境</th>
          <th>優點</th>
          <th>邊界</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>user_version</code></td>
          <td>mobile / desktop / CLI single file</td>
          <td>簡單、內建、開檔即可讀</td>
          <td>只能存一個整數，缺 migration history</td>
      </tr>
      <tr>
          <td>migration table</td>
          <td>small backend、多人維護 schema</td>
          <td>可記錄每步 migration 與 owner</td>
          <td>需要先建立 table 與初始化流程</td>
      </tr>
      <tr>
          <td>external manifest</td>
          <td>fixture、artifact、read-only DB</td>
          <td>可和 release artifact 綁定</td>
          <td>DB file 本身不含完整 history</td>
      </tr>
  </tbody>
</table>
<p>Version model 要在第一版就定義。沒有版本欄位的 SQLite file 仍可 migration，但 application 只能靠 introspection 猜 schema，會讓 upgrade / downgrade runbook 複雜化。</p>
<h2 id="alter-table-boundary">ALTER TABLE boundary</h2>
<p>SQLite ALTER TABLE 的核心責任是處理有限集合的 schema 變更。官方文件說明 SQLite 支援 rename table、rename column、add column、drop column；更複雜的變更要走 table rebuild pattern。</p>
<table>
  <thead>
      <tr>
          <th>變更類型</th>
          <th>SQLite 支援形態</th>
          <th>操作判讀</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Rename table / column</td>
          <td>直接 ALTER，版本差異影響 trigger / view</td>
          <td>需要測 trigger、view、FK reference</td>
      </tr>
      <tr>
          <td>Add column</td>
          <td>多數情境很快，受 default / constraint 限制</td>
          <td>適合 expand migration</td>
      </tr>
      <tr>
          <td>Drop column</td>
          <td>需要檢查 index、constraint、trigger、view</td>
          <td>可能掃資料，需 maintenance window</td>
      </tr>
      <tr>
          <td>Change type / constraint</td>
          <td>通常走 table rebuild</td>
          <td>需要完整 copy、foreign key check、validation</td>
      </tr>
  </tbody>
</table>
<p>SQLite schema 存在 <code>sqlite_schema</code> 的 SQL text 中；這讓檔案格式簡潔，但也讓 ALTER TABLE 的安全條件和 server SQL 不同。Production migration 應優先用官方建議的 rebuild procedure，而非直接修改 <code>sqlite_schema</code>。</p>
<h2 id="table-rebuild-migration">Table rebuild migration</h2>
<p>Table rebuild migration 的服務責任是安全完成 SQLite 直接 ALTER 難以表達的變更。官方 ALTER TABLE 文件建議的 generalized procedure 是建立新 table、copy data、drop old、rename new、重建 index / trigger / view、跑 foreign key check、commit。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">BEGIN</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">OFF</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w"></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">new_orders</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">  </span><span class="n">id</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">  </span><span class="n">status</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">  </span><span class="n">paid_at</span><span class="w"> </span><span class="nb">TEXT</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w"></span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w"></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">new_orders</span><span class="w"> </span><span class="p">(</span><span class="n">id</span><span class="p">,</span><span class="w"> </span><span class="n">status</span><span class="p">,</span><span class="w"> </span><span class="n">paid_at</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="n">id</span><span class="p">,</span><span class="w"> </span><span class="n">status</span><span class="p">,</span><span class="w"> </span><span class="n">paid_at</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="w"></span><span class="k">DROP</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w"></span><span class="k">ALTER</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">new_orders</span><span class="w"> </span><span class="k">RENAME</span><span class="w"> </span><span class="k">TO</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_key_check</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">user_version</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">2026052101</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="w"></span><span class="k">COMMIT</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">ON</span><span class="p">;</span></span></span></code></pre></div><p>這段範例是教學骨架，而非可直接複製到所有 schema 的萬用腳本。真實 migration 要先保存 index、trigger、view 與 FK reference，再依 schema 重建；有資料量時還要考慮 copy duration、disk 空間與 rollback snapshot。</p>
<h2 id="app-release-compatibility">App release compatibility</h2>
<p>SQLite migration 的 application compatibility 來自 binary 與 DB file 的同步問題。Server SQL migration 通常有 central deploy order；SQLite file 可能跟著使用者裝置、desktop profile、CLI artifact 或 edge deploy 留在不同版本。</p>
<table>
  <thead>
      <tr>
          <th>相容性問題</th>
          <th>真實情境</th>
          <th>設計策略</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>新 app 打開舊 DB</td>
          <td>使用者升級 app</td>
          <td>startup migration、read compatibility</td>
      </tr>
      <tr>
          <td>舊 app 打開新 DB</td>
          <td>使用者 downgrade、同步舊 binary</td>
          <td>保留 backward-compatible column、feature gate</td>
      </tr>
      <tr>
          <td>多裝置不同版本</td>
          <td>local-first / sync app</td>
          <td>sync protocol version、server authority</td>
      </tr>
      <tr>
          <td>fixture 與 production drift</td>
          <td>test fixture 沒更新</td>
          <td>fixture version、contract test、migration smoke</td>
      </tr>
  </tbody>
</table>
<p>Compatibility 的核心是先決定支援範圍。Mobile app 常要支援舊版資料庫升級；internal CLI 可能只支援最新版本；test fixture 則需要每次 migration 後重新產生。</p>
<h2 id="migration-evidence">Migration evidence</h2>
<p>Migration evidence 的責任是證明 schema 變更已完成且資料仍可用。SQLite migration evidence 比 server DB 簡單，但更依賴 application-level validation。</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>目的</th>
          <th>範例</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>schema version</td>
          <td>確認 DB file 契約</td>
          <td><code>PRAGMA user_version</code></td>
      </tr>
      <tr>
          <td>row count</td>
          <td>確認 copy / rebuild 無漏資料</td>
          <td><code>SELECT COUNT(*) FROM orders</code></td>
      </tr>
      <tr>
          <td>domain query</td>
          <td>確認重要 business invariant</td>
          <td>unpaid / paid 狀態數量</td>
      </tr>
      <tr>
          <td>foreign key check</td>
          <td>確認 reference integrity</td>
          <td><code>PRAGMA foreign_key_check</code></td>
      </tr>
      <tr>
          <td>integrity check</td>
          <td>檢查 DB 結構</td>
          <td><code>PRAGMA integrity_check</code></td>
      </tr>
      <tr>
          <td>backup marker</td>
          <td>回退點</td>
          <td>pre-migration <code>.backup</code> file</td>
      </tr>
  </tbody>
</table>
<p>這些 evidence 應接到 <a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">Observability Evidence Package</a> 或 release note。SQLite migration 失敗時，最清楚的 rollback 通常是回到 migration 前 snapshot，而非在同一檔案上繼續試錯。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1startup-migration-讓-app-啟動卡住">Case 1：startup migration 讓 app 啟動卡住</h3>
<p>Startup migration 的核心風險是把長時間 table rebuild 放在使用者啟動路徑。小表新增 column 可能很快；大表 rebuild、index 重建或 vacuum 類操作會讓 app 啟動、CLI command 或 API cold start 變慢。</p>
<p>修正方向是先估資料量。短 migration 可在 startup；長 migration 要有 explicit command、progress、backup 與 rollback route。</p>
<h3 id="case-2fixture-schema-升級漏掉-production-gap">Case 2：fixture schema 升級漏掉 production gap</h3>
<p>Fixture schema drift 的核心風險是測試 DB 和 production DB 的 dialect / constraint 不一致。SQLite fixture 很快，但 production 若是 PostgreSQL / MySQL，type、date、NULL、constraint 與 transaction 行為都可能不同。</p>
<p>修正方向是把 SQLite fixture 明確標成 contract test 層。Repository error mapping、domain invariant 可以用 SQLite；production-specific SQL 要用 production database container 驗證。</p>
<h3 id="case-3直接改-sqlite_schema">Case 3：直接改 <code>sqlite_schema</code></h3>
<p>直接改 <code>sqlite_schema</code> 的核心風險是產生語法正確但語意破壞的 database file。SQLite 官方文件提供 writable schema route，但同時強調錯誤修改可能讓 database corrupt / unreadable。</p>
<p>修正方向是讓 writable schema 成為最後手段。一般 migration 優先用 ALTER TABLE 或 table rebuild；需要特殊修復時先複製原檔，在副本驗證。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>SQLite migration runbook 至少要記錄：</p>
<ol>
<li>DB file 目前 <code>user_version</code> 與 application release version。</li>
<li>Migration 是否可重入、是否可中斷後恢復。</li>
<li>Migration 前 backup / snapshot 位置。</li>
<li>需要 table rebuild 的 table、資料量、index / trigger / view 清單。</li>
<li>Validation query、row count、foreign key check、integrity check。</li>
<li>舊 binary / 新 binary 的相容策略。</li>
<li>Fixture DB 是否已重新產生並被 contract test 使用。</li>
</ol>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview</a></li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/migration-fixture-lab/" data-link-title="SQLite Migration Fixture Lab" data-link-desc="SQLite user_version、table rebuild migration、fixture snapshot、rollback note 與 CI evidence 的操作說明">Migration fixture lab</a></li>
<li>平行：<a href="/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test Fixture Best Practice</a></li>
<li>遷移：<a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL</a></li>
<li>官方：<a href="https://www.sqlite.org/lang_altertable.html">SQLite ALTER TABLE</a>、<a href="https://www.sqlite.org/pragma.html">SQLite PRAGMA</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite SQL Dialect and Index Limits</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/</guid><description>&lt;p>SQLite SQL dialect and index limits 的核心責任是說明 SQLite 和 server SQL 的語意差異。SQLite 可以執行大量 SQL，也支援 transaction、index、trigger、view、window function 與 JSON；但它的 typing、constraint、file-level operation、query planner 與 extension model 會影響測試可信度、migration 成本與 production adapter。&lt;/p>
&lt;p>本文的判讀錨點是：SQLite 測過代表某個 repository contract 在 SQLite 語意下成立。當 production target 是 PostgreSQL、MySQL、D1、Turso 或其他 server database 時，測試與 migration 要補上 dialect gap evidence。&lt;/p>
&lt;h2 id="type-affinity">Type Affinity&lt;/h2>
&lt;p>&lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/type-affinity/" data-link-title="Type Affinity" data-link-desc="說明 SQLite 如何用 type affinity 決定欄位的型別傾向與值的儲存方式">Type affinity&lt;/a> 的核心責任是定義資料寫入時如何被保存與比較。SQLite 官方 &lt;a href="https://www.sqlite.org/datatype3.html">Datatypes&lt;/a> 文件說明 SQLite 使用 dynamic typing，型別關聯在 value 層與 column affinity 層共同作用；&lt;a href="https://www.sqlite.org/stricttables.html">STRICT tables&lt;/a> 則提供較嚴格的型別檢查。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>議題&lt;/th>
 &lt;th>SQLite 行為重點&lt;/th>
 &lt;th>Production 影響&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Integer&lt;/td>
 &lt;td>value type 可依寫入內容變化&lt;/td>
 &lt;td>test fixture 可能放過錯誤型別&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Text&lt;/td>
 &lt;td>collation 與比較語意需明確設定&lt;/td>
 &lt;td>排序、大小寫、unique 判斷要對照 target DB&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Date/time&lt;/td>
 &lt;td>常以 TEXT / REAL / INTEGER 表示&lt;/td>
 &lt;td>timezone、range query、serialization 要一致&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Boolean&lt;/td>
 &lt;td>常以 integer convention 表示&lt;/td>
 &lt;td>adapter 要定義 true / false encoding&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>STRICT&lt;/td>
 &lt;td>提供更接近 server DB 的型別 guard&lt;/td>
 &lt;td>適合作為 fixture 預設，仍需 production test&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Type affinity 的教學重點是把資料合約放在 application boundary。若 domain 說 &lt;code>created_at&lt;/code> 是 timestamp，就要定義 storage format、timezone、precision、comparison query 與 serialization，而非只讓 SQLite 接受任意 value。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="k">CREATE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">TABLE&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">orders&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">id&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">INTEGER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">PRIMARY&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">KEY&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">created_at&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">TEXT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">&lt;span class="w"> &lt;/span>&lt;span class="n">total_cents&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="nb">INTEGER&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NOT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">NULL&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">CHECK&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">total_cents&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">&amp;gt;=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">0&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">5&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">STRICT&lt;/span>&lt;span class="p">;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>這段 schema 用 &lt;code>STRICT&lt;/code>、&lt;code>NOT NULL&lt;/code> 與 &lt;code>CHECK&lt;/code> 讓 fixture 更接近正式資料合約。Production target 仍要跑 PostgreSQL / MySQL container test，確認 timestamp、integer range 與 constraint error mapping。&lt;/p>
&lt;h2 id="constraint-behavior">Constraint Behavior&lt;/h2>
&lt;p>Constraint behavior 的核心責任是確保資料完整性由 database 和 application 共同維護。SQLite 支援 primary key、unique、check、foreign key 與 deferred constraint，但 foreign key enforcement 需要明確啟用，migration / test runner 也要確認連線設定。&lt;/p></description><content:encoded><![CDATA[<p>SQLite SQL dialect and index limits 的核心責任是說明 SQLite 和 server SQL 的語意差異。SQLite 可以執行大量 SQL，也支援 transaction、index、trigger、view、window function 與 JSON；但它的 typing、constraint、file-level operation、query planner 與 extension model 會影響測試可信度、migration 成本與 production adapter。</p>
<p>本文的判讀錨點是：SQLite 測過代表某個 repository contract 在 SQLite 語意下成立。當 production target 是 PostgreSQL、MySQL、D1、Turso 或其他 server database 時，測試與 migration 要補上 dialect gap evidence。</p>
<h2 id="type-affinity">Type Affinity</h2>
<p><a href="/blog/backend/knowledge-cards/type-affinity/" data-link-title="Type Affinity" data-link-desc="說明 SQLite 如何用 type affinity 決定欄位的型別傾向與值的儲存方式">Type affinity</a> 的核心責任是定義資料寫入時如何被保存與比較。SQLite 官方 <a href="https://www.sqlite.org/datatype3.html">Datatypes</a> 文件說明 SQLite 使用 dynamic typing，型別關聯在 value 層與 column affinity 層共同作用；<a href="https://www.sqlite.org/stricttables.html">STRICT tables</a> 則提供較嚴格的型別檢查。</p>
<table>
  <thead>
      <tr>
          <th>議題</th>
          <th>SQLite 行為重點</th>
          <th>Production 影響</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Integer</td>
          <td>value type 可依寫入內容變化</td>
          <td>test fixture 可能放過錯誤型別</td>
      </tr>
      <tr>
          <td>Text</td>
          <td>collation 與比較語意需明確設定</td>
          <td>排序、大小寫、unique 判斷要對照 target DB</td>
      </tr>
      <tr>
          <td>Date/time</td>
          <td>常以 TEXT / REAL / INTEGER 表示</td>
          <td>timezone、range query、serialization 要一致</td>
      </tr>
      <tr>
          <td>Boolean</td>
          <td>常以 integer convention 表示</td>
          <td>adapter 要定義 true / false encoding</td>
      </tr>
      <tr>
          <td>STRICT</td>
          <td>提供更接近 server DB 的型別 guard</td>
          <td>適合作為 fixture 預設，仍需 production test</td>
      </tr>
  </tbody>
</table>
<p>Type affinity 的教學重點是把資料合約放在 application boundary。若 domain 說 <code>created_at</code> 是 timestamp，就要定義 storage format、timezone、precision、comparison query 與 serialization，而非只讓 SQLite 接受任意 value。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">orders</span><span class="w"> </span><span class="p">(</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w">  </span><span class="n">id</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w">  </span><span class="n">created_at</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="p">,</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w">  </span><span class="n">total_cents</span><span class="w"> </span><span class="nb">INTEGER</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">NULL</span><span class="w"> </span><span class="k">CHECK</span><span class="w"> </span><span class="p">(</span><span class="n">total_cents</span><span class="w"> </span><span class="o">&gt;=</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w"></span><span class="p">)</span><span class="w"> </span><span class="k">STRICT</span><span class="p">;</span></span></span></code></pre></div><p>這段 schema 用 <code>STRICT</code>、<code>NOT NULL</code> 與 <code>CHECK</code> 讓 fixture 更接近正式資料合約。Production target 仍要跑 PostgreSQL / MySQL container test，確認 timestamp、integer range 與 constraint error mapping。</p>
<h2 id="constraint-behavior">Constraint Behavior</h2>
<p>Constraint behavior 的核心責任是確保資料完整性由 database 和 application 共同維護。SQLite 支援 primary key、unique、check、foreign key 與 deferred constraint，但 foreign key enforcement 需要明確啟用，migration / test runner 也要確認連線設定。</p>
<table>
  <thead>
      <tr>
          <th>Constraint</th>
          <th>SQLite 審查點</th>
          <th>操作判準</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Foreign key</td>
          <td><code>PRAGMA foreign_keys = ON</code></td>
          <td>每個 connection / test setup 都要驗證</td>
      </tr>
      <tr>
          <td>Unique</td>
          <td>NULL、collation、expression</td>
          <td>對照 target DB 的 NULL uniqueness 與 collation</td>
      </tr>
      <tr>
          <td>Check</td>
          <td>type affinity 互動</td>
          <td>用 domain invalid case 驗證</td>
      </tr>
      <tr>
          <td>Deferred</td>
          <td>transaction boundary</td>
          <td>用 multi-step workflow 測 commit-time failure</td>
      </tr>
  </tbody>
</table>
<p>Foreign key 是 SQLite fixture 最常漏掉的設定。每個測試連線開啟後應立刻查 <code>PRAGMA foreign_keys;</code>，並用一個故意違反 FK 的 fixture case 確認錯誤會出現。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">ON</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">pragma_foreign_keys</span><span class="p">;</span></span></span></code></pre></div><p>Constraint error 要在 repository adapter 層被歸類。若 production target 會把 duplicate key、foreign key、check violation 映射成不同 error code，SQLite fixture 也要至少保留 domain-level classification test。</p>
<h2 id="transaction-behavior">Transaction Behavior</h2>
<p>Transaction behavior 的核心責任是定義讀寫隔離、savepoint、nested workflow 與 retry。SQLite 官方 <a href="https://www.sqlite.org/isolation.html">isolation</a> 文件說明 connection 之間的隔離語意；WAL mode 下 reader / writer behavior 也會影響 concurrent test。</p>
<table>
  <thead>
      <tr>
          <th>行為</th>
          <th>SQLite 判讀</th>
          <th>測試影響</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Single writer</td>
          <td>同一時間只有一個 writer 取得寫鎖</td>
          <td>concurrent writer test 要顯式設計</td>
      </tr>
      <tr>
          <td>Snapshot read</td>
          <td>WAL mode 下 reader 可讀舊 snapshot</td>
          <td>freshness 與 read-after-write 要分開測</td>
      </tr>
      <tr>
          <td>Savepoint</td>
          <td>適合 nested workflow</td>
          <td>repository transaction helper 要支援</td>
      </tr>
      <tr>
          <td>Busy timeout</td>
          <td>lock wait policy</td>
          <td>integration test 要設定固定 timeout</td>
      </tr>
  </tbody>
</table>
<p>Savepoint 可以讓 application 實作可組合的 transaction helper。若上層 workflow 已在 transaction 內，內層 repository 可以使用 savepoint 承接局部 rollback，而非開另一個 database transaction。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">SAVEPOINT</span><span class="w"> </span><span class="n">create_order</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">orders</span><span class="p">(</span><span class="n">id</span><span class="p">,</span><span class="w"> </span><span class="n">created_at</span><span class="p">,</span><span class="w"> </span><span class="n">total_cents</span><span class="p">)</span><span class="w"> </span><span class="k">VALUES</span><span class="w"> </span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;2026-05-21T00:00:00Z&#39;</span><span class="p">,</span><span class="w"> </span><span class="mi">1200</span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="n">RELEASE</span><span class="w"> </span><span class="n">create_order</span><span class="p">;</span></span></span></code></pre></div><p>Busy timeout 是測試穩定性的關鍵設定。若 fixture 會平行跑測試，應每個 temp DB 獨立，或在專門 concurrency lab 裡測 <code>SQLITE_BUSY</code>；一般 contract test 要追求 deterministic result。</p>
<h2 id="index-model">Index Model</h2>
<p>Index model 的核心責任是把查詢形狀與資料量變成可觀測的計畫。SQLite 支援 B-tree index、covering index、partial index、expression index 與 query planner；但 planner choice、統計資訊與 function support 會和 target DB 不同。</p>
<table>
  <thead>
      <tr>
          <th>Index 類型</th>
          <th>適用情境</th>
          <th>審查問題</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Composite index</td>
          <td>多欄位 equality / range query</td>
          <td>欄位順序是否符合主要 query pattern</td>
      </tr>
      <tr>
          <td>Partial index</td>
          <td>active / pending / soft-delete row</td>
          <td>predicate 是否穩定、target DB 是否支援</td>
      </tr>
      <tr>
          <td>Expression index</td>
          <td>normalized email、date bucket</td>
          <td>function deterministic 與 migration 支援</td>
      </tr>
      <tr>
          <td>Covering index</td>
          <td>read-mostly list page</td>
          <td>index size 與 write overhead</td>
      </tr>
  </tbody>
</table>
<p>Index review 要從 query pattern 開始，而非從「常用欄位」開始。SQLite 可以用 <code>EXPLAIN QUERY PLAN</code> 檢查是否掃 index；production target 要用自己的 explain 工具重跑。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">EXPLAIN</span><span class="w"> </span><span class="n">QUERY</span><span class="w"> </span><span class="n">PLAN</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="n">id</span><span class="p">,</span><span class="w"> </span><span class="n">total_cents</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">orders</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w"></span><span class="k">WHERE</span><span class="w"> </span><span class="n">created_at</span><span class="w"> </span><span class="o">&gt;=</span><span class="w"> </span><span class="s1">&#39;2026-05-01T00:00:00Z&#39;</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w"></span><span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">created_at</span><span class="w"> </span><span class="k">DESC</span><span class="w">
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="w"></span><span class="k">LIMIT</span><span class="w"> </span><span class="mi">50</span><span class="p">;</span></span></span></code></pre></div><p>Index drift 是 migration 的常見風險。SQLite fixture 裡的 index 可以讓測試變快，但若 production schema 缺少同等 index，正式服務會在資料量成長後出現 latency spike；因此 index 要進入 schema diff audit。</p>
<h2 id="dialect-gap">Dialect Gap</h2>
<p>Dialect gap 的核心責任是把 SQLite 與 target database 的差異寫成 matrix。這份 matrix 應跟 repository adapter、migration plan 與 CI test suite 綁定。</p>
<table>
  <thead>
      <tr>
          <th>面向</th>
          <th>SQLite 審查點</th>
          <th>對照路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>ALTER TABLE</td>
          <td>支援範圍、table rebuild</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema migration / versioning</a></td>
      </tr>
      <tr>
          <td>JSON</td>
          <td>function availability、index support</td>
          <td>production container test</td>
      </tr>
      <tr>
          <td>Generated column</td>
          <td>expression、storage、index</td>
          <td>migration dry run</td>
      </tr>
      <tr>
          <td>Window function</td>
          <td>target DB 支援與 planner</td>
          <td>query compatibility suite</td>
      </tr>
      <tr>
          <td>Extension</td>
          <td>FTS、vector、custom function</td>
          <td>vendor extension policy</td>
      </tr>
  </tbody>
</table>
<p>Dialect matrix 要以 query contract 為單位。每個 repository method 至少列出 SQL feature、SQLite behavior、production behavior、test layer 與 fallback strategy。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">Contract: Search active documents by tenant and prefix
</span></span><span class="line"><span class="ln">2</span><span class="cl">SQLite: FTS5 virtual table in fixture
</span></span><span class="line"><span class="ln">3</span><span class="cl">PostgreSQL: tsvector + GIN index
</span></span><span class="line"><span class="ln">4</span><span class="cl">Risk: ranking / tokenizer / collation differ
</span></span><span class="line"><span class="ln">5</span><span class="cl">Evidence: golden result set + production container explain</span></span></code></pre></div><p>這種寫法讓測試負責驗證 domain contract，避免把兩個 SQL engine 的搜尋語意視為完全一致。</p>
<h2 id="test--migration-impact">Test / Migration Impact</h2>
<p>Test / migration impact 的核心責任是決定哪些東西可以用 SQLite 快速驗證，哪些東西要交給 production-like database。SQLite 很適合 repository contract、migration fixture、local development 與 file lifecycle drill；涉及 planner、extension、collation、locking、permission、role 與 HA 時，需要追加 target DB evidence。</p>
<table>
  <thead>
      <tr>
          <th>測試層</th>
          <th>SQLite 適合度</th>
          <th>必補 evidence</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Domain repository</td>
          <td>高</td>
          <td>invalid data、constraint、transaction case</td>
      </tr>
      <tr>
          <td>Migration syntax</td>
          <td>中</td>
          <td>target DB dry run</td>
      </tr>
      <tr>
          <td>Query performance</td>
          <td>中</td>
          <td>target DB explain + realistic data volume</td>
      </tr>
      <tr>
          <td>Permission / role</td>
          <td>低</td>
          <td>server DB integration test</td>
      </tr>
      <tr>
          <td>HA / failover</td>
          <td>低</td>
          <td>vendor-specific drill</td>
      </tr>
  </tbody>
</table>
<p>SQLite fixture 的價值在於快、穩、便宜。它應承擔「資料合約是否被 repository 保護」；production container 或 staging database 承擔「正式 engine 是否用同樣方式執行」。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>SQL dialect and index limits 完成後，下一步要把 gap 接到實作層。測試設計讀 <a href="/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test Fixture Best Practice</a>；migration 實作讀 <a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema migration / versioning</a>；要升級到 PostgreSQL，讀 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration</a>。</p>
]]></content:encoded></item><item><title>SQLite Teaching Structure</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/teaching-structure/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/teaching-structure/</guid><description>&lt;p>SQLite teaching structure 的核心責任是把 SQLite 從單篇 vendor overview 擴成可教學的服務章節群。PostgreSQL / MySQL 的完整度來自 overview、deep article、migration playbook 與案例路由；SQLite 的完整度也要保留同樣層級，但正文重點要貼合它自己的服務語言：single file、embedded process、writer boundary、backup / restore、test fixture、local-first 與 edge SQLite 變體。&lt;/p>
&lt;h2 id="完成標準">完成標準&lt;/h2>
&lt;p>SQLite 章節群的完成標準是讀者能回答三個問題。第一，SQLite 何時是正式狀態而非臨時檔案；第二，SQLite production 化後要如何處理 WAL、backup、restore、migration、測試與觀測；第三，SQLite 成長後該升到 PostgreSQL / MySQL、Cloudflare D1、Turso / libSQL、Litestream / LiteFS 或 mobile sync。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>層級&lt;/th>
 &lt;th>SQLite 對應文件&lt;/th>
 &lt;th>教學責任&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Service overview&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a>&lt;/td>
 &lt;td>第一輪服務定位、適用壓力、替代邊界與下一步路由&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Core deep article&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary&lt;/a>&lt;/td>
 &lt;td>WAL sidecar、backup API、restore drill、corruption recovery&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Hands-on&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on&lt;/a>&lt;/td>
 &lt;td>local file、backup restore、WAL busy、migration fixture&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Operations&lt;/td>
 &lt;td>WAL / locking、PRAGMA tuning、schema migration、observability&lt;/td>
 &lt;td>日常設定、排錯、容量訊號與 release gate&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Application shape&lt;/td>
 &lt;td>test fixture、mobile / desktop store、local-first sync&lt;/td>
 &lt;td>SQLite 跟 application process / device / test workflow 的關係&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Edge / variants&lt;/td>
 &lt;td>D1 / Turso / libSQL、Litestream / LiteFS&lt;/td>
 &lt;td>分散式或 replicated SQLite 變體的責任邊界&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Migration route&lt;/td>
 &lt;td>SQLite → PostgreSQL、SQLite → D1 / Turso、PostgreSQL → SQLite&lt;/td>
 &lt;td>成長、edge 化或降操作成本時的階段化搬遷&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這份結構的重點是避免把 SQLite 寫成小型 PostgreSQL。SQLite deep article 要先處理檔案、process、filesystem、device、test 與 edge runtime；SQL dialect、index 與 migration 工具只有在這些責任成立後才展開。&lt;/p>
&lt;h2 id="推薦撰寫順序">推薦撰寫順序&lt;/h2>
&lt;p>撰寫順序要從正式狀態的最低操作責任開始，再逐步擴到應用形狀、edge 變體與 migration。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>順序&lt;/th>
 &lt;th>文件&lt;/th>
 &lt;th>狀態&lt;/th>
 &lt;th>為什麼排在這裡&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>1&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>先回答 SQLite 如何成為可恢復的正式狀態&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>2&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>writer boundary 是 SQLite production 判斷的核心&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>3&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/pragma-tuning-performance/" data-link-title="SQLite PRAGMA Tuning and Performance" data-link-desc="SQLite journal_mode、synchronous、busy_timeout、wal_autocheckpoint、cache_size、mmap_size、auto_vacuum 與 performance evidence 的操作判準">PRAGMA tuning / performance&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>把 journal、sync、cache、mmap 轉成可驗證的設定&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>4&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema migration / versioning&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>單檔案 DB 仍需要版本、rollback 與 app release 配合&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>5&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test fixture best practice&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>SQLite 最常被語言教材引用，需要明確 production gap&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>6&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/mobile-desktop-embedded-store/" data-link-title="SQLite Mobile / Desktop Embedded Store" data-link-desc="SQLite 在 mobile、desktop、CLI、browser profile 與 embedded device 中承擔 local formal state 的資料責任、backup、privacy 與 sync boundary">Mobile / desktop embedded store&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>說明 device local state、backup、sync 與 privacy 責任&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>7&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/" data-link-title="SQLite Local-first Sync Boundary" data-link-desc="SQLite local-first app、multi-device sync、server authority、conflict resolution、delete propagation 與 offline-first trade-off">Local-first sync boundary&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>把 single-device SQLite 與 multi-device sync 分開&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>8&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL comparison&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>edge SQLite 變體需要獨立比較，和本地 SQLite 分開&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>9&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS replication&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>backup / read replica / failover 的語意要跟 multi-write 分開&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>10&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/" data-link-title="SQLite SQL Dialect and Index Limits" data-link-desc="SQLite type affinity、NULL / date handling、constraint、index、query planner 與 PostgreSQL / MySQL 差異">SQL dialect and index limits&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>對照 PostgreSQL / MySQL 測試與 migration gap&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>11&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">Observability / runbook&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>把 SQLite 的低操作成本補成可交接 evidence&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>12&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">Hands-on 操作路線&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>把 local file、backup、WAL busy、migration fixture 變成演練&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>13&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>多 tenant、權限、HA、schema governance 出現時的主要升級路徑&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>14&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso route&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>edge / serverless 化時的 migration route&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>15&lt;/td>
 &lt;td>&lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-from-postgresql-simplification/" data-link-title="PostgreSQL to SQLite Simplification" data-link-desc="PostgreSQL 降低操作成本轉向 SQLite 的適用條件、資料責任縮小、export/import、runbook 與 no-go condition">PostgreSQL to SQLite simplification&lt;/a>&lt;/td>
 &lt;td>已有正文&lt;/td>
 &lt;td>小型工具、single-user app 或 embedded 需求的反向路徑&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這個順序讓 SQLite 先完成自己的核心語言，再處理相鄰產品。D1、Turso、LiteFS、Litestream 都帶有 SQLite 相容性，但教學上要先問它們承擔的是 backup、replication、edge locality、read replica 還是 distributed write。&lt;/p></description><content:encoded><![CDATA[<p>SQLite teaching structure 的核心責任是把 SQLite 從單篇 vendor overview 擴成可教學的服務章節群。PostgreSQL / MySQL 的完整度來自 overview、deep article、migration playbook 與案例路由；SQLite 的完整度也要保留同樣層級，但正文重點要貼合它自己的服務語言：single file、embedded process、writer boundary、backup / restore、test fixture、local-first 與 edge SQLite 變體。</p>
<h2 id="完成標準">完成標準</h2>
<p>SQLite 章節群的完成標準是讀者能回答三個問題。第一，SQLite 何時是正式狀態而非臨時檔案；第二，SQLite production 化後要如何處理 WAL、backup、restore、migration、測試與觀測；第三，SQLite 成長後該升到 PostgreSQL / MySQL、Cloudflare D1、Turso / libSQL、Litestream / LiteFS 或 mobile sync。</p>
<table>
  <thead>
      <tr>
          <th>層級</th>
          <th>SQLite 對應文件</th>
          <th>教學責任</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Service overview</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a></td>
          <td>第一輪服務定位、適用壓力、替代邊界與下一步路由</td>
      </tr>
      <tr>
          <td>Core deep article</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary</a></td>
          <td>WAL sidecar、backup API、restore drill、corruption recovery</td>
      </tr>
      <tr>
          <td>Hands-on</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a></td>
          <td>local file、backup restore、WAL busy、migration fixture</td>
      </tr>
      <tr>
          <td>Operations</td>
          <td>WAL / locking、PRAGMA tuning、schema migration、observability</td>
          <td>日常設定、排錯、容量訊號與 release gate</td>
      </tr>
      <tr>
          <td>Application shape</td>
          <td>test fixture、mobile / desktop store、local-first sync</td>
          <td>SQLite 跟 application process / device / test workflow 的關係</td>
      </tr>
      <tr>
          <td>Edge / variants</td>
          <td>D1 / Turso / libSQL、Litestream / LiteFS</td>
          <td>分散式或 replicated SQLite 變體的責任邊界</td>
      </tr>
      <tr>
          <td>Migration route</td>
          <td>SQLite → PostgreSQL、SQLite → D1 / Turso、PostgreSQL → SQLite</td>
          <td>成長、edge 化或降操作成本時的階段化搬遷</td>
      </tr>
  </tbody>
</table>
<p>這份結構的重點是避免把 SQLite 寫成小型 PostgreSQL。SQLite deep article 要先處理檔案、process、filesystem、device、test 與 edge runtime；SQL dialect、index 與 migration 工具只有在這些責任成立後才展開。</p>
<h2 id="推薦撰寫順序">推薦撰寫順序</h2>
<p>撰寫順序要從正式狀態的最低操作責任開始，再逐步擴到應用形狀、edge 變體與 migration。</p>
<table>
  <thead>
      <tr>
          <th>順序</th>
          <th>文件</th>
          <th>狀態</th>
          <th>為什麼排在這裡</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>1</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary</a></td>
          <td>已有正文</td>
          <td>先回答 SQLite 如何成為可恢復的正式狀態</td>
      </tr>
      <tr>
          <td>2</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking</a></td>
          <td>已有正文</td>
          <td>writer boundary 是 SQLite production 判斷的核心</td>
      </tr>
      <tr>
          <td>3</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/pragma-tuning-performance/" data-link-title="SQLite PRAGMA Tuning and Performance" data-link-desc="SQLite journal_mode、synchronous、busy_timeout、wal_autocheckpoint、cache_size、mmap_size、auto_vacuum 與 performance evidence 的操作判準">PRAGMA tuning / performance</a></td>
          <td>已有正文</td>
          <td>把 journal、sync、cache、mmap 轉成可驗證的設定</td>
      </tr>
      <tr>
          <td>4</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema migration / versioning</a></td>
          <td>已有正文</td>
          <td>單檔案 DB 仍需要版本、rollback 與 app release 配合</td>
      </tr>
      <tr>
          <td>5</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/" data-link-title="SQLite Test Fixture Best Practice" data-link-desc="SQLite 作為 test fixture、repository contract test、production dialect gap、seed data、fixture snapshot 與 CI evidence 的操作判準">Test fixture best practice</a></td>
          <td>已有正文</td>
          <td>SQLite 最常被語言教材引用，需要明確 production gap</td>
      </tr>
      <tr>
          <td>6</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/mobile-desktop-embedded-store/" data-link-title="SQLite Mobile / Desktop Embedded Store" data-link-desc="SQLite 在 mobile、desktop、CLI、browser profile 與 embedded device 中承擔 local formal state 的資料責任、backup、privacy 與 sync boundary">Mobile / desktop embedded store</a></td>
          <td>已有正文</td>
          <td>說明 device local state、backup、sync 與 privacy 責任</td>
      </tr>
      <tr>
          <td>7</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/local-first-sync-boundary/" data-link-title="SQLite Local-first Sync Boundary" data-link-desc="SQLite local-first app、multi-device sync、server authority、conflict resolution、delete propagation 與 offline-first trade-off">Local-first sync boundary</a></td>
          <td>已有正文</td>
          <td>把 single-device SQLite 與 multi-device sync 分開</td>
      </tr>
      <tr>
          <td>8</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL comparison</a></td>
          <td>已有正文</td>
          <td>edge SQLite 變體需要獨立比較，和本地 SQLite 分開</td>
      </tr>
      <tr>
          <td>9</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS replication</a></td>
          <td>已有正文</td>
          <td>backup / read replica / failover 的語意要跟 multi-write 分開</td>
      </tr>
      <tr>
          <td>10</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/" data-link-title="SQLite SQL Dialect and Index Limits" data-link-desc="SQLite type affinity、NULL / date handling、constraint、index、query planner 與 PostgreSQL / MySQL 差異">SQL dialect and index limits</a></td>
          <td>已有正文</td>
          <td>對照 PostgreSQL / MySQL 測試與 migration gap</td>
      </tr>
      <tr>
          <td>11</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">Observability / runbook</a></td>
          <td>已有正文</td>
          <td>把 SQLite 的低操作成本補成可交接 evidence</td>
      </tr>
      <tr>
          <td>12</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">Hands-on 操作路線</a></td>
          <td>已有正文</td>
          <td>把 local file、backup、WAL busy、migration fixture 變成演練</td>
      </tr>
      <tr>
          <td>13</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL migration</a></td>
          <td>已有正文</td>
          <td>多 tenant、權限、HA、schema governance 出現時的主要升級路徑</td>
      </tr>
      <tr>
          <td>14</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso route</a></td>
          <td>已有正文</td>
          <td>edge / serverless 化時的 migration route</td>
      </tr>
      <tr>
          <td>15</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-from-postgresql-simplification/" data-link-title="PostgreSQL to SQLite Simplification" data-link-desc="PostgreSQL 降低操作成本轉向 SQLite 的適用條件、資料責任縮小、export/import、runbook 與 no-go condition">PostgreSQL to SQLite simplification</a></td>
          <td>已有正文</td>
          <td>小型工具、single-user app 或 embedded 需求的反向路徑</td>
      </tr>
  </tbody>
</table>
<p>這個順序讓 SQLite 先完成自己的核心語言，再處理相鄰產品。D1、Turso、LiteFS、Litestream 都帶有 SQLite 相容性，但教學上要先問它們承擔的是 backup、replication、edge locality、read replica 還是 distributed write。</p>
<h2 id="文件命名規則">文件命名規則</h2>
<p>SQLite 章節群的檔名用服務責任命名，product-first 命名只留給 D1 / Turso / libSQL 這類 product boundary 本身就是教學主題的文件。</p>
<table>
  <thead>
      <tr>
          <th>類型</th>
          <th>命名方式</th>
          <th>範例</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Core deep</td>
          <td><code>{mechanism}-{responsibility}</code></td>
          <td><code>wal-concurrency-locking.md</code></td>
      </tr>
      <tr>
          <td>Operation</td>
          <td><code>{operation}-{decision-signal}</code></td>
          <td><code>pragma-tuning-performance.md</code></td>
      </tr>
      <tr>
          <td>Application</td>
          <td><code>{context}-{state-role}</code></td>
          <td><code>mobile-desktop-embedded-store.md</code></td>
      </tr>
      <tr>
          <td>Variant</td>
          <td><code>{products}-comparison</code></td>
          <td><code>d1-turso-libsql-comparison.md</code></td>
      </tr>
      <tr>
          <td>Migration</td>
          <td><code>migrate-to-{target}</code></td>
          <td><code>migrate-to-postgresql.md</code></td>
      </tr>
  </tbody>
</table>
<h2 id="cross-module-路由">Cross-module 路由</h2>
<p>SQLite 章節群要固定連到四個 backend 模組。Backup / restore 連到 04 evidence 與 08 incident；test fixture 連到語言教材與 repository adapter；edge / local-first 連到 05 deployment / 07 data protection；performance tuning 連到 09 capacity。</p>
<table>
  <thead>
      <tr>
          <th>SQLite 議題</th>
          <th>主要跨模組路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Backup / restore</td>
          <td><a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">Observability Evidence Package</a>、<a href="/blog/backend/08-incident-response/incident-decision-log/" data-link-title="8.19 Incident Decision Log" data-link-desc="把事中假設、決策、證據、回退條件與責任人留下可復盤紀錄">Incident Decision Log</a></td>
      </tr>
      <tr>
          <td>Test fixture</td>
          <td><a href="/blog/backend/01-database/repository-adapter/" data-link-title="1.4 Repository Adapter 實作" data-link-desc="Port / Adapter 邊界、row mapping、error translation、ORM vs query builder 選型、contract test 設計">Repository Adapter</a>、語言教材的 contract test</td>
      </tr>
      <tr>
          <td>Local-first / sync</td>
          <td><a href="/blog/backend/07-security-data-protection/data-protection-and-masking-governance/" data-link-title="7.4 資料保護與遮罩治理" data-link-desc="以問題驅動方式整理資料分級、遮罩、匯出與備份治理">Data Protection</a>、offline / device privacy</td>
      </tr>
      <tr>
          <td>Edge SQLite</td>
          <td><a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">Global Distributed OLTP</a>、deployment platform</td>
      </tr>
      <tr>
          <td>Performance</td>
          <td><a href="/blog/backend/09-performance-capacity/bottleneck-localization/" data-link-title="9.5 瓶頸定位流程" data-link-desc="從 app 到 DB / cache / broker / 第三方 quota 的逐層瓶頸定位">Bottleneck Localization</a></td>
      </tr>
  </tbody>
</table>
<h2 id="後續審查點">後續審查點</h2>
<p>SQLite 章節群完稿後要特別審查三個偏誤。第一是把 SQLite 過度美化成 production SQL 替代品；第二是把 edge SQLite 產品跟本地 SQLite 混成同一種能力；第三是把 test fixture 的便利性誤寫成 production equivalence。</p>
]]></content:encoded></item><item><title>SQLite Test Fixture Best Practice</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/test-fixture-best-practice/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合作為 test fixture；本文聚焦 &lt;em>如何用 SQLite 加速測試，同時保留 production database 的語意邊界&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite test fixture 的核心責任是讓 repository / adapter 測試快速、可重複、可攜帶。SQLite 的單檔特性讓 CI 可以快速建立 DB、載入 seed、跑 contract test；但它的 type affinity、SQL dialect、locking 與 constraint behavior 和 PostgreSQL / MySQL 不完全相同，因此 fixture 要被定位為一層測試工具，而非 production equivalence。&lt;/p>
&lt;p>本文的判讀錨點是：SQLite fixture 適合驗證 application contract，不適合取代 production database compatibility test。若測試目標是 repository error mapping、domain invariant、migration fixture 或 deterministic seed，SQLite 很划算；若測試目標是 PostgreSQL extension、MySQL lock、query planner 或 SQL dialect，應使用 production-like container。&lt;/p>
&lt;h2 id="test-fixture-的位置">Test fixture 的位置&lt;/h2>
&lt;p>SQLite fixture 的服務責任是提供快、穩定、可重建的本地資料狀態。它通常位於 unit test 與 full integration test 之間，承擔 repository adapter 的 contract test。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>測試層級&lt;/th>
 &lt;th>SQLite 適合度&lt;/th>
 &lt;th>判讀&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Pure unit test&lt;/td>
 &lt;td>低&lt;/td>
 &lt;td>fake / in-memory object 通常更快&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Repository contract&lt;/td>
 &lt;td>高&lt;/td>
 &lt;td>驗證 CRUD、constraint mapping、transaction behavior&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Service integration&lt;/td>
 &lt;td>中&lt;/td>
 &lt;td>適合簡單流程，不覆蓋 production-specific SQL&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Production compatibility&lt;/td>
 &lt;td>低&lt;/td>
 &lt;td>用 PostgreSQL / MySQL container 或 staging DB&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Migration smoke&lt;/td>
 &lt;td>中&lt;/td>
 &lt;td>適合 fixture migration，不代表 production DDL&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的重點是把測試目的說清楚。SQLite fixture 讓語言教材與 backend 教材接起來；語言端測 interface / adapter，backend 端保留 production database 的深度文章與 migration playbook。&lt;/p>
&lt;h2 id="fixture-lifecycle">Fixture lifecycle&lt;/h2>
&lt;p>Fixture lifecycle 的核心責任是讓每次測試拿到已知資料狀態。常見策略有三種：每 test 建新 in-memory DB、每 suite 複製 template file、每 CI job 產生 versioned fixture。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>策略&lt;/th>
 &lt;th>適合情境&lt;/th>
 &lt;th>優點&lt;/th>
 &lt;th>邊界&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>&lt;code>:memory:&lt;/code> per test&lt;/td>
 &lt;td>小 schema、快速 unit-like contract&lt;/td>
 &lt;td>隔離最好、清理簡單&lt;/td>
 &lt;td>跨 connection / WAL 行為不同&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>template file copy&lt;/td>
 &lt;td>中等 seed、需要真實檔案行為&lt;/td>
 &lt;td>快速、可測 file lifecycle&lt;/td>
 &lt;td>要避免多 test 共用同一檔案&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>generated fixture&lt;/td>
 &lt;td>migration / seed 驗證&lt;/td>
 &lt;td>和 migration 同步&lt;/td>
 &lt;td>CI 時間較長&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>read-only fixture&lt;/td>
 &lt;td>查詢 / report 測試&lt;/td>
 &lt;td>避免 writer collision&lt;/td>
 &lt;td>不測 mutation&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Fixture file 應和 schema version 綁定。檔名、metadata 或 &lt;code>user_version&lt;/code> 要能回答「這個 fixture 對應哪個 migration 版本」，避免測試資料在多次 schema 變更後變成隱性技術債。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 適合作為 test fixture；本文聚焦 <em>如何用 SQLite 加速測試，同時保留 production database 的語意邊界</em>。</p></blockquote>
<p>SQLite test fixture 的核心責任是讓 repository / adapter 測試快速、可重複、可攜帶。SQLite 的單檔特性讓 CI 可以快速建立 DB、載入 seed、跑 contract test；但它的 type affinity、SQL dialect、locking 與 constraint behavior 和 PostgreSQL / MySQL 不完全相同，因此 fixture 要被定位為一層測試工具，而非 production equivalence。</p>
<p>本文的判讀錨點是：SQLite fixture 適合驗證 application contract，不適合取代 production database compatibility test。若測試目標是 repository error mapping、domain invariant、migration fixture 或 deterministic seed，SQLite 很划算；若測試目標是 PostgreSQL extension、MySQL lock、query planner 或 SQL dialect，應使用 production-like container。</p>
<h2 id="test-fixture-的位置">Test fixture 的位置</h2>
<p>SQLite fixture 的服務責任是提供快、穩定、可重建的本地資料狀態。它通常位於 unit test 與 full integration test 之間，承擔 repository adapter 的 contract test。</p>
<table>
  <thead>
      <tr>
          <th>測試層級</th>
          <th>SQLite 適合度</th>
          <th>判讀</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Pure unit test</td>
          <td>低</td>
          <td>fake / in-memory object 通常更快</td>
      </tr>
      <tr>
          <td>Repository contract</td>
          <td>高</td>
          <td>驗證 CRUD、constraint mapping、transaction behavior</td>
      </tr>
      <tr>
          <td>Service integration</td>
          <td>中</td>
          <td>適合簡單流程，不覆蓋 production-specific SQL</td>
      </tr>
      <tr>
          <td>Production compatibility</td>
          <td>低</td>
          <td>用 PostgreSQL / MySQL container 或 staging DB</td>
      </tr>
      <tr>
          <td>Migration smoke</td>
          <td>中</td>
          <td>適合 fixture migration，不代表 production DDL</td>
      </tr>
  </tbody>
</table>
<p>這張表的重點是把測試目的說清楚。SQLite fixture 讓語言教材與 backend 教材接起來；語言端測 interface / adapter，backend 端保留 production database 的深度文章與 migration playbook。</p>
<h2 id="fixture-lifecycle">Fixture lifecycle</h2>
<p>Fixture lifecycle 的核心責任是讓每次測試拿到已知資料狀態。常見策略有三種：每 test 建新 in-memory DB、每 suite 複製 template file、每 CI job 產生 versioned fixture。</p>
<table>
  <thead>
      <tr>
          <th>策略</th>
          <th>適合情境</th>
          <th>優點</th>
          <th>邊界</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><code>:memory:</code> per test</td>
          <td>小 schema、快速 unit-like contract</td>
          <td>隔離最好、清理簡單</td>
          <td>跨 connection / WAL 行為不同</td>
      </tr>
      <tr>
          <td>template file copy</td>
          <td>中等 seed、需要真實檔案行為</td>
          <td>快速、可測 file lifecycle</td>
          <td>要避免多 test 共用同一檔案</td>
      </tr>
      <tr>
          <td>generated fixture</td>
          <td>migration / seed 驗證</td>
          <td>和 migration 同步</td>
          <td>CI 時間較長</td>
      </tr>
      <tr>
          <td>read-only fixture</td>
          <td>查詢 / report 測試</td>
          <td>避免 writer collision</td>
          <td>不測 mutation</td>
      </tr>
  </tbody>
</table>
<p>Fixture file 應和 schema version 綁定。檔名、metadata 或 <code>user_version</code> 要能回答「這個 fixture 對應哪個 migration 版本」，避免測試資料在多次 schema 變更後變成隱性技術債。</p>
<h2 id="production-dialect-gap">Production dialect gap</h2>
<p>Production dialect gap 的核心責任是避免 SQLite 測試通過後，PostgreSQL / MySQL production 出現不同語意。SQLite 的 dynamic typing、date / time representation、foreign key pragma、ALTER TABLE 支援與 lock model 都會影響測試可信度。</p>
<table>
  <thead>
      <tr>
          <th>Gap 類型</th>
          <th>SQLite 行為</th>
          <th>Production 風險</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Type affinity</td>
          <td>欄位有 affinity，值本身仍有 storage class</td>
          <td>PostgreSQL / MySQL type error 沒被測到</td>
      </tr>
      <tr>
          <td>Date / time</td>
          <td>常以 TEXT / REAL / INTEGER 表示</td>
          <td>timezone、precision、function 差異</td>
      </tr>
      <tr>
          <td>Foreign key</td>
          <td>需要 <code>PRAGMA foreign_keys=ON</code></td>
          <td>fixture 忘記開 FK，constraint bug 漏掉</td>
      </tr>
      <tr>
          <td>ALTER TABLE</td>
          <td>支援 subset，複雜變更需 rebuild</td>
          <td>production migration 工具行為不同</td>
      </tr>
      <tr>
          <td>Locking</td>
          <td>single-file lock / single writer</td>
          <td>server DB connection / lock model 不同</td>
      </tr>
      <tr>
          <td>SQL feature</td>
          <td>extension / JSON / index 差異</td>
          <td>vendor-specific query 需要 production evidence</td>
      </tr>
  </tbody>
</table>
<p>這張表的用法是決定哪些測試留在 SQLite，哪些要升級到 production-like DB。Repository contract 可用 SQLite；query optimization、vendor SQL、online schema change、CDC、replication、pooling 都應回到 PostgreSQL / MySQL 章節。</p>
<h2 id="contract-test-設計">Contract test 設計</h2>
<p>Contract test 的核心責任是讓不同 DB adapter 對 application 呈現同一組語意。SQLite fixture 測的是 application port 的行為，例如 duplicate key、not found、transaction rollback、pagination、domain invariant，而非底層 engine 的所有細節。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">Repository contract
</span></span><span class="line"><span class="ln">2</span><span class="cl">├── Create / read / update / delete
</span></span><span class="line"><span class="ln">3</span><span class="cl">├── Unique conflict → ErrAlreadyExists
</span></span><span class="line"><span class="ln">4</span><span class="cl">├── Missing row → ErrNotFound
</span></span><span class="line"><span class="ln">5</span><span class="cl">├── Transaction rollback restores domain invariant
</span></span><span class="line"><span class="ln">6</span><span class="cl">├── Pagination order stable
</span></span><span class="line"><span class="ln">7</span><span class="cl">└── Migration version matches fixture</span></span></code></pre></div><p>如果 production adapter 是 PostgreSQL / MySQL，contract test 應至少在 nightly 或 CI matrix 裡跑一輪 production-like database。SQLite 提供快速回饋，production-like test 提供 dialect confidence。</p>
<h2 id="ci-evidence">CI evidence</h2>
<p>SQLite fixture 的 CI evidence 要證明資料狀態和 schema version 一致。測試失敗時，讀者要能知道是 application contract 失效、fixture 過期、migration 漏跑，還是 SQLite / production dialect gap。</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>目的</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>fixture version</td>
          <td>對齊 migration / app release</td>
      </tr>
      <tr>
          <td>seed checksum</td>
          <td>確認測試資料穩定</td>
      </tr>
      <tr>
          <td>migration log</td>
          <td>確認 fixture 可由 migration 重建</td>
      </tr>
      <tr>
          <td>contract test output</td>
          <td>確認 repository behavior</td>
      </tr>
      <tr>
          <td>dialect gap note</td>
          <td>標示未覆蓋 production behavior</td>
      </tr>
  </tbody>
</table>
<p>CI 產物不一定要很複雜，但要能被下一個維護者重建。SQLite fixture 的優勢是可攜帶；若 fixture 只能靠某個人的本機狀態生成，就失去教學與維護價值。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1共用同一個-db-檔跑平行測試">Case 1：共用同一個 <code>.db</code> 檔跑平行測試</h3>
<p>平行測試共用檔案的核心風險是 test runner 製造和 production 不同的 writer collision。測試偶發 <code>SQLITE_BUSY</code>，團隊可能以為 application 有 race；實際上是測試隔離不足。</p>
<p>修正方向是 per-test temp DB 或 read-only template copy。需要測 WAL / busy 行為時，用專門 hands-on lab，讓一般 contract test 專注在 repository contract。</p>
<h3 id="case-2忘記開-foreign-keys">Case 2：忘記開 foreign keys</h3>
<p>Foreign key pragma 漏開的核心風險是 constraint bug 被 fixture 隱藏。SQLite foreign key enforcement 需要明確啟用；若 production DB 一定 enforce FK，fixture 也要在 connection initialization 中開啟。</p>
<p>修正方向是 baseline PRAGMA 和 startup assertion。每個 test DB open 後都跑 <code>PRAGMA foreign_keys</code> 並驗證結果。</p>
<h3 id="case-3sqlite-fixture-掩蓋-vendor-specific-sql">Case 3：SQLite fixture 掩蓋 vendor-specific SQL</h3>
<p>Vendor-specific SQL 被 SQLite 掩蓋的核心風險是 query 到 production 才失敗。例如 PostgreSQL JSONB、partial index、full-text search 或 MySQL generated column、optimizer hint 都應在 vendor DB 測。</p>
<p>修正方向是把 SQL 分層。Portable repository contract 可以用 SQLite；vendor-specific query 要有 PostgreSQL / MySQL test container。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>SQLite fixture 設計前要回答：</p>
<ol>
<li>這個測試驗證 application contract 還是 production dialect。</li>
<li>Fixture 是 in-memory、template copy、generated file 還是 read-only。</li>
<li><code>PRAGMA foreign_keys</code>、<code>journal_mode</code>、<code>busy_timeout</code> 是否固定。</li>
<li>Fixture version 如何對齊 migration version。</li>
<li>Parallel test 是否每個 worker 有獨立 DB file。</li>
<li>哪些 query 必須在 PostgreSQL / MySQL container 再跑。</li>
<li>CI artifact 是否保留 migration log 與 dialect gap note。</li>
</ol>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/repository-adapter/" data-link-title="1.4 Repository Adapter 實作" data-link-desc="Port / Adapter 邊界、row mapping、error translation、ORM vs query builder 選型、contract test 設計">Repository Adapter</a></li>
<li>Sibling：<a href="/blog/backend/01-database/vendors/sqlite/schema-migration-versioning/" data-link-title="SQLite Schema Migration and Versioning" data-link-desc="SQLite schema migration、user_version、table rebuild、ALTER TABLE 限制、app release compatibility 與 migration evidence">Schema Migration / Versioning</a>、<a href="/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/" data-link-title="SQLite SQL Dialect and Index Limits" data-link-desc="SQLite type affinity、NULL / date handling、constraint、index、query planner 與 PostgreSQL / MySQL 差異">SQL Dialect and Index Limits</a></li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/migration-fixture-lab/" data-link-title="SQLite Migration Fixture Lab" data-link-desc="SQLite user_version、table rebuild migration、fixture snapshot、rollback note 與 CI evidence 的操作說明">Migration Fixture Lab</a></li>
<li>平行：<a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a>、<a href="/blog/backend/01-database/vendors/mysql/" data-link-title="MySQL" data-link-desc="高併發網路服務常用關聯式資料庫、Vitess / PlanetScale 分片生態、GitHub / Shopify / Facebook 規模驗證">MySQL</a></li>
<li>官方：<a href="https://www.sqlite.org/datatype3.html">SQLite Datatypes</a>、<a href="https://www.sqlite.org/stricttables.html">SQLite STRICT Tables</a>、<a href="https://www.sqlite.org/pragma.html">SQLite PRAGMA</a></li>
</ul>
]]></content:encoded></item><item><title>SQLite to D1 / Turso Migration</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/</guid><description>&lt;p>SQLite to D1 / Turso migration 的核心責任是把 local SQLite 轉成 edge / serverless / distributed SQLite-compatible product。這條路線的 driver 通常是 edge locality、Workers integration、managed operation、global read latency、embedded replica 或 serverless deployment workflow。&lt;/p>
&lt;p>本文的判讀錨點是：D1 / Turso migration 是 runtime boundary 變更。Local file 直連變成 platform binding、remote endpoint 或 embedded replica；因此 migration 要同時審查 SQL support、data movement、driver API、auth、latency、freshness、backup 與 vendor exit。&lt;/p>
&lt;h2 id="migration-drivers">Migration Drivers&lt;/h2>
&lt;p>Migration drivers 的核心責任是確認 edge SQLite 產品解決的是哪個服務壓力。D1 與 Turso / libSQL 都接近 SQLite experience，但它們的採用理由應寫成具體 workload。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Driver&lt;/th>
 &lt;th>適合產品&lt;/th>
 &lt;th>判讀訊號&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Workers integration&lt;/td>
 &lt;td>Cloudflare D1&lt;/td>
 &lt;td>App 已在 Workers、資料量小、query 清楚&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Serverless low ops&lt;/td>
 &lt;td>D1 / Turso&lt;/td>
 &lt;td>不想維護 host DB、可接受 platform limit&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Low-latency read&lt;/td>
 &lt;td>Turso / embedded replica&lt;/td>
 &lt;td>read-heavy、freshness window 明確&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Edge-local app&lt;/td>
 &lt;td>D1 / Turso&lt;/td>
 &lt;td>使用者分散、write rate 可控&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Portable SQLite base&lt;/td>
 &lt;td>Turso / libSQL&lt;/td>
 &lt;td>想保留 SQLite-like schema 與 local dev&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>D1 的 migration driver 要和 Cloudflare platform 綁定。若 app 已用 Workers routing、KV、Queues 或 Pages，D1 可以降低跨平台整合成本；若 app 不在 Cloudflare 生態，D1 的價值要用 latency、operation 與成本證明。&lt;/p>
&lt;p>Turso / libSQL 的 migration driver 要和 replica freshness 綁定。若使用者需要 local read speed，embedded replica 有價值；若產品要求每次讀都立即看到最新 global state，就要先設計 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/read-after-write/" data-link-title="Read-After-Write Consistency" data-link-desc="說明寫入後能否立即讀到該筆寫入的一致性保證">read-after-write&lt;/a> path。&lt;/p>
&lt;h2 id="compatibility-audit">Compatibility Audit&lt;/h2>
&lt;p>Compatibility audit 的核心責任是確認 local SQLite schema、query 與 migration workflow 可在 target product 上運作。官方文件要作為 limits 與 feature 的單一來源：D1 參考 &lt;a href="https://developers.cloudflare.com/d1/">Cloudflare D1 docs&lt;/a> 與 &lt;a href="https://developers.cloudflare.com/d1/platform/limits/">D1 limits&lt;/a>；Turso 參考 &lt;a href="https://docs.turso.tech/">Turso docs&lt;/a> 與 libSQL client reference。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>面向&lt;/th>
 &lt;th>審查問題&lt;/th>
 &lt;th>Evidence&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>SQL support&lt;/td>
 &lt;td>schema、trigger、index、JSON、FK&lt;/td>
 &lt;td>migration dry run、query suite&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Size / batch&lt;/td>
 &lt;td>import file、query duration、batch size&lt;/td>
 &lt;td>limit review、sample import&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Driver API&lt;/td>
 &lt;td>local file path 變成 binding / endpoint&lt;/td>
 &lt;td>repository adapter test&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Auth&lt;/td>
 &lt;td>token、binding、environment secret&lt;/td>
 &lt;td>staging deployment&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Transaction&lt;/td>
 &lt;td>request boundary、retry、write location&lt;/td>
 &lt;td>failure injection&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Backup&lt;/td>
 &lt;td>export、restore、retention&lt;/td>
 &lt;td>restore drill&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Compatibility audit 要以 production query 為單位。只跑 &lt;code>CREATE TABLE&lt;/code> 會漏掉最重要的差異；query suite 要包含 list page、pagination、unique violation、FK violation、transaction rollback、large batch 與 slow query。&lt;/p></description><content:encoded><![CDATA[<p>SQLite to D1 / Turso migration 的核心責任是把 local SQLite 轉成 edge / serverless / distributed SQLite-compatible product。這條路線的 driver 通常是 edge locality、Workers integration、managed operation、global read latency、embedded replica 或 serverless deployment workflow。</p>
<p>本文的判讀錨點是：D1 / Turso migration 是 runtime boundary 變更。Local file 直連變成 platform binding、remote endpoint 或 embedded replica；因此 migration 要同時審查 SQL support、data movement、driver API、auth、latency、freshness、backup 與 vendor exit。</p>
<h2 id="migration-drivers">Migration Drivers</h2>
<p>Migration drivers 的核心責任是確認 edge SQLite 產品解決的是哪個服務壓力。D1 與 Turso / libSQL 都接近 SQLite experience，但它們的採用理由應寫成具體 workload。</p>
<table>
  <thead>
      <tr>
          <th>Driver</th>
          <th>適合產品</th>
          <th>判讀訊號</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Workers integration</td>
          <td>Cloudflare D1</td>
          <td>App 已在 Workers、資料量小、query 清楚</td>
      </tr>
      <tr>
          <td>Serverless low ops</td>
          <td>D1 / Turso</td>
          <td>不想維護 host DB、可接受 platform limit</td>
      </tr>
      <tr>
          <td>Low-latency read</td>
          <td>Turso / embedded replica</td>
          <td>read-heavy、freshness window 明確</td>
      </tr>
      <tr>
          <td>Edge-local app</td>
          <td>D1 / Turso</td>
          <td>使用者分散、write rate 可控</td>
      </tr>
      <tr>
          <td>Portable SQLite base</td>
          <td>Turso / libSQL</td>
          <td>想保留 SQLite-like schema 與 local dev</td>
      </tr>
  </tbody>
</table>
<p>D1 的 migration driver 要和 Cloudflare platform 綁定。若 app 已用 Workers routing、KV、Queues 或 Pages，D1 可以降低跨平台整合成本；若 app 不在 Cloudflare 生態，D1 的價值要用 latency、operation 與成本證明。</p>
<p>Turso / libSQL 的 migration driver 要和 replica freshness 綁定。若使用者需要 local read speed，embedded replica 有價值；若產品要求每次讀都立即看到最新 global state，就要先設計 <a href="/blog/backend/knowledge-cards/read-after-write/" data-link-title="Read-After-Write Consistency" data-link-desc="說明寫入後能否立即讀到該筆寫入的一致性保證">read-after-write</a> path。</p>
<h2 id="compatibility-audit">Compatibility Audit</h2>
<p>Compatibility audit 的核心責任是確認 local SQLite schema、query 與 migration workflow 可在 target product 上運作。官方文件要作為 limits 與 feature 的單一來源：D1 參考 <a href="https://developers.cloudflare.com/d1/">Cloudflare D1 docs</a> 與 <a href="https://developers.cloudflare.com/d1/platform/limits/">D1 limits</a>；Turso 參考 <a href="https://docs.turso.tech/">Turso docs</a> 與 libSQL client reference。</p>
<table>
  <thead>
      <tr>
          <th>面向</th>
          <th>審查問題</th>
          <th>Evidence</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>SQL support</td>
          <td>schema、trigger、index、JSON、FK</td>
          <td>migration dry run、query suite</td>
      </tr>
      <tr>
          <td>Size / batch</td>
          <td>import file、query duration、batch size</td>
          <td>limit review、sample import</td>
      </tr>
      <tr>
          <td>Driver API</td>
          <td>local file path 變成 binding / endpoint</td>
          <td>repository adapter test</td>
      </tr>
      <tr>
          <td>Auth</td>
          <td>token、binding、environment secret</td>
          <td>staging deployment</td>
      </tr>
      <tr>
          <td>Transaction</td>
          <td>request boundary、retry、write location</td>
          <td>failure injection</td>
      </tr>
      <tr>
          <td>Backup</td>
          <td>export、restore、retention</td>
          <td>restore drill</td>
      </tr>
  </tbody>
</table>
<p>Compatibility audit 要以 production query 為單位。只跑 <code>CREATE TABLE</code> 會漏掉最重要的差異；query suite 要包含 list page、pagination、unique violation、FK violation、transaction rollback、large batch 與 slow query。</p>
<h2 id="data-movement">Data Movement</h2>
<p>Data movement 的核心責任是把 SQLite file 轉成 target platform 可接受的 seed。Local SQLite 可以先 export 成 SQL dump、CSV 或 platform CLI 支援的 import format，再進 target product。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.dump&#34;</span> &gt; seed.sql</span></span></code></pre></div><p>這段命令只是 seed 起點。正式流程要處理 schema ordering、unsupported SQL、large transaction、batch split、sensitive data masking、import duration、row count 與 checksum。</p>
<p>D1 migration 要把 Wrangler / platform workflow 納入 runbook。Cloudflare D1 的 limits 文件列出 import 與 query 限制；大型資料變更應切 batch，並在 preview / staging database 跑完整 dry run。</p>
<p>Turso migration 要把 remote database 與 embedded replica 分開驗證。Seed 完 remote primary 後，要測 local embedded replica 的 bootstrap、sync、read freshness、write delegation 與 offline behavior。</p>
<h2 id="application-change">Application Change</h2>
<p>Application change 的核心責任是把 database access 從 file path 改成可替換 adapter。Local SQLite 常用 file path 與 process-local connection；D1 / Turso 會加入 binding、endpoint、token、client SDK、network failure 與 platform runtime。</p>
<table>
  <thead>
      <tr>
          <th>改動層</th>
          <th>Local SQLite</th>
          <th>D1 / Turso route</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Connection</td>
          <td>file path</td>
          <td>Workers binding、HTTP / libSQL endpoint</td>
      </tr>
      <tr>
          <td>Auth</td>
          <td>filesystem permission</td>
          <td>platform secret、token、binding</td>
      </tr>
      <tr>
          <td>Error model</td>
          <td>SQLite error code</td>
          <td>SDK / platform error + SQLite-like error</td>
      </tr>
      <tr>
          <td>Retry</td>
          <td>local busy / lock retry</td>
          <td>network retry、idempotency、timeout</td>
      </tr>
      <tr>
          <td>Observability</td>
          <td>app log + file metric</td>
          <td>app log + platform metric</td>
      </tr>
  </tbody>
</table>
<p>Repository adapter 要承擔 driver 差異。Domain layer 應看到穩定的 repository contract，例如 duplicate key、stale read、temporary unavailable、retryable write；底層才處理 D1 binding 或 libSQL client。</p>
<p>Idempotency 是 edge migration 的關鍵。Write request 進入 network / serverless runtime 後，retry 可能在 client、platform 或 application 層發生；每個 critical write 都應有 idempotency key 或 natural unique key。</p>
<h2 id="evidence">Evidence</h2>
<p>Evidence 的核心責任是證明 edge migration 帶來的收益大於新風險。D1 / Turso 的成功要同時看功能可用、region latency、freshness、error rate、cost、migration time 與 exit route。</p>
<table>
  <thead>
      <tr>
          <th>Evidence</th>
          <th>最小驗證方式</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Latency by region</td>
          <td>從主要 user region 跑 read/write test</td>
      </tr>
      <tr>
          <td>Freshness</td>
          <td>write 後在 replica / edge read 檢查</td>
      </tr>
      <tr>
          <td>Migration repeatability</td>
          <td>staging database 從空庫重跑 seed</td>
      </tr>
      <tr>
          <td>Error mapping</td>
          <td>duplicate、constraint、timeout、auth</td>
      </tr>
      <tr>
          <td>Cost</td>
          <td>request、storage、egress、operation</td>
      </tr>
      <tr>
          <td>Exit route</td>
          <td>export file + restore to local SQLite</td>
      </tr>
  </tbody>
</table>
<p>Freshness evidence 要用產品語言寫。若 UI 可以顯示「同步中」，freshness window 可被使用者理解；若是付款、庫存、權限決策，讀舊資料會直接造成業務錯誤，這類 workflow 要走 primary read 或 server SQL。</p>
<p>Exit route 要被演練。Edge product 的 adoption cost 低，exit cost 會出現在 driver API、migration workflow、platform binding 與 data export；至少要能把 staging data export 回 SQLite file 並通過 smoke test。</p>
<h2 id="rollback">Rollback</h2>
<p>Rollback 的核心責任是保留 local SQLite snapshot 與 read-only fallback。Edge migration 若在 cutover 後遇到 auth、latency、limit 或 query error，團隊要能快速回到上一個可用資料狀態。</p>
<table>
  <thead>
      <tr>
          <th>Rollback 觸發</th>
          <th>回退策略</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Import / migration 失敗</td>
          <td>清空 target、修 migration、重跑 seed</td>
      </tr>
      <tr>
          <td>Query error spike</td>
          <td>切回 local SQLite / previous endpoint</td>
      </tr>
      <tr>
          <td>Freshness issue</td>
          <td>critical read 改 primary path</td>
      </tr>
      <tr>
          <td>Cost / limit spike</td>
          <td>降低 traffic、batch migration、重評估</td>
      </tr>
      <tr>
          <td>Vendor incident</td>
          <td>read-only mode、fallback endpoint</td>
      </tr>
  </tbody>
</table>
<p>Local snapshot 要保存到 cutover 後的觀察窗口結束。若 cutover 期間已有 target-only writes，要設計回放或 reconciliation；高風險 workflow 可以先進 read-only cutover，再逐步開寫。</p>
<h2 id="decision-route">Decision Route</h2>
<p>Decision route 的核心責任是把 edge migration 和 server DB migration 分開。D1 / Turso 適合 edge runtime 與 SQLite-like workflow；當需求轉向 central audit、server role、high-write OLTP 或 distributed transaction，應改走 PostgreSQL / CockroachDB / Spanner。</p>
<table>
  <thead>
      <tr>
          <th>需求</th>
          <th>路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Workers app + small relational data</td>
          <td>D1</td>
      </tr>
      <tr>
          <td>Read-heavy app + local replica value</td>
          <td>Turso / libSQL</td>
      </tr>
      <tr>
          <td>Backup / restore 是主要問題</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS</a></td>
      </tr>
      <tr>
          <td>多 tenant + permission + audit</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL</a></td>
      </tr>
      <tr>
          <td>Global write transaction</td>
          <td><a href="/blog/backend/01-database/global-distributed-oltp/" data-link-title="1.11 全球分散式 OLTP" data-link-desc="Spanner / Aurora DSQL / Cosmos DB multi-region write / CockroachDB / TiDB 的全球一致性取捨">Global Distributed OLTP</a></td>
      </tr>
  </tbody>
</table>
<h2 id="下一步路由">下一步路由</h2>
<p>SQLite to D1 / Turso migration 完成後，先讀 <a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso / libSQL comparison</a> 釐清 product boundary；再用 <a href="/blog/backend/01-database/vendors/sqlite/sql-dialect-index-limits/" data-link-title="SQLite SQL Dialect and Index Limits" data-link-desc="SQLite type affinity、NULL / date handling、constraint、index、query planner 與 PostgreSQL / MySQL 差異">SQL dialect and index limits</a> 做 compatibility audit；需要操作演練時讀 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/d1-turso-preview-lab/" data-link-title="SQLite D1 / Turso Preview Lab" data-link-desc="SQLite local DB 匯出到 Cloudflare D1 或 Turso preview environment 的 compatibility、latency 與 rollback 操作說明">D1 / Turso preview lab</a>。</p>
]]></content:encoded></item><item><title>SQLite to PostgreSQL Migration</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/</guid><description>&lt;p>SQLite to PostgreSQL migration 的核心責任是把 embedded single-file state 升級成 server SQL operational model。這條路線通常由 multi-user access、HA、central audit、permission、online schema governance、write concurrency 或 team handoff 壓力觸發。&lt;/p>
&lt;p>本文的判讀錨點是：升級到 PostgreSQL 是服務責任擴大，而非單純換 driver。Migration 要同時處理 schema 語意、資料搬遷、application adapter、backup / PITR、role、observability、cutover 與 rollback。&lt;/p>
&lt;h2 id="migration-drivers">Migration Drivers&lt;/h2>
&lt;p>Migration drivers 的核心責任是確認 PostgreSQL 真的承擔新增責任。SQLite 在 single-node、single-file、low-concurrency 場景很強；PostgreSQL 的價值出現在 server database governance。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>Driver&lt;/th>
 &lt;th>代表需求&lt;/th>
 &lt;th>PostgreSQL 承擔的責任&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Concurrent writers&lt;/td>
 &lt;td>多 instance / 多使用者同時寫入&lt;/td>
 &lt;td>MVCC、connection management、lock insight&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>HA / PITR&lt;/td>
 &lt;td>需要時間點恢復與 managed backup&lt;/td>
 &lt;td>WAL archiving、replica、restore drill&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Central audit&lt;/td>
 &lt;td>需要查詢與變更證據&lt;/td>
 &lt;td>role、log、extension、SIEM integration&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Permission boundary&lt;/td>
 &lt;td>app / analyst / job 權限分離&lt;/td>
 &lt;td>DB role、grant、row / schema boundary&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Schema governance&lt;/td>
 &lt;td>migration 要 online 且可審查&lt;/td>
 &lt;td>migration tool、lock review、rollback&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Shared data platform&lt;/td>
 &lt;td>多服務共用正式資料&lt;/td>
 &lt;td>connection pool、capacity、ownership&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Driver 要被量化。若問題只是單一 CLI 檔案變大，先改善 backup、VACUUM、index 與 WAL runbook；若問題是多 instance 同時寫、權限分離、audit 與 PITR，PostgreSQL 才是正確路由。&lt;/p>
&lt;h2 id="diff-audit">Diff Audit&lt;/h2>
&lt;p>Diff audit 的核心責任是把 SQLite 語意轉成 PostgreSQL 語意。SQLite 的 type affinity、date / time convention、auto-increment、foreign key、index、JSON、transaction 與 extension 都要逐項審查。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>面向&lt;/th>
 &lt;th>SQLite source 問題&lt;/th>
 &lt;th>PostgreSQL target 決策&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Type&lt;/td>
 &lt;td>dynamic typing、STRICT usage&lt;/td>
 &lt;td>integer / bigint / numeric / timestamptz&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Primary key&lt;/td>
 &lt;td>rowid、INTEGER PRIMARY KEY&lt;/td>
 &lt;td>identity、sequence、UUID&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Date/time&lt;/td>
 &lt;td>TEXT / INTEGER convention&lt;/td>
 &lt;td>timestamptz、timezone policy&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>JSON&lt;/td>
 &lt;td>JSON text / function usage&lt;/td>
 &lt;td>jsonb、GIN index、query rewrite&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Constraint&lt;/td>
 &lt;td>FK pragma、check、unique collation&lt;/td>
 &lt;td>enforced FK、deferrable、collation&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Index&lt;/td>
 &lt;td>partial / expression / covering index&lt;/td>
 &lt;td>equivalent index + explain&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>Transaction&lt;/td>
 &lt;td>single writer、savepoint&lt;/td>
 &lt;td>isolation level、deadlock retry&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>Type mapping 要先保護 domain invariant。金額欄位用 integer cents 或 numeric、時間欄位用 timestamptz 或明確 UTC text、boolean 用 boolean；每個轉換都要有 invalid sample 與 round-trip test。&lt;/p></description><content:encoded><![CDATA[<p>SQLite to PostgreSQL migration 的核心責任是把 embedded single-file state 升級成 server SQL operational model。這條路線通常由 multi-user access、HA、central audit、permission、online schema governance、write concurrency 或 team handoff 壓力觸發。</p>
<p>本文的判讀錨點是：升級到 PostgreSQL 是服務責任擴大，而非單純換 driver。Migration 要同時處理 schema 語意、資料搬遷、application adapter、backup / PITR、role、observability、cutover 與 rollback。</p>
<h2 id="migration-drivers">Migration Drivers</h2>
<p>Migration drivers 的核心責任是確認 PostgreSQL 真的承擔新增責任。SQLite 在 single-node、single-file、low-concurrency 場景很強；PostgreSQL 的價值出現在 server database governance。</p>
<table>
  <thead>
      <tr>
          <th>Driver</th>
          <th>代表需求</th>
          <th>PostgreSQL 承擔的責任</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Concurrent writers</td>
          <td>多 instance / 多使用者同時寫入</td>
          <td>MVCC、connection management、lock insight</td>
      </tr>
      <tr>
          <td>HA / PITR</td>
          <td>需要時間點恢復與 managed backup</td>
          <td>WAL archiving、replica、restore drill</td>
      </tr>
      <tr>
          <td>Central audit</td>
          <td>需要查詢與變更證據</td>
          <td>role、log、extension、SIEM integration</td>
      </tr>
      <tr>
          <td>Permission boundary</td>
          <td>app / analyst / job 權限分離</td>
          <td>DB role、grant、row / schema boundary</td>
      </tr>
      <tr>
          <td>Schema governance</td>
          <td>migration 要 online 且可審查</td>
          <td>migration tool、lock review、rollback</td>
      </tr>
      <tr>
          <td>Shared data platform</td>
          <td>多服務共用正式資料</td>
          <td>connection pool、capacity、ownership</td>
      </tr>
  </tbody>
</table>
<p>Driver 要被量化。若問題只是單一 CLI 檔案變大，先改善 backup、VACUUM、index 與 WAL runbook；若問題是多 instance 同時寫、權限分離、audit 與 PITR，PostgreSQL 才是正確路由。</p>
<h2 id="diff-audit">Diff Audit</h2>
<p>Diff audit 的核心責任是把 SQLite 語意轉成 PostgreSQL 語意。SQLite 的 type affinity、date / time convention、auto-increment、foreign key、index、JSON、transaction 與 extension 都要逐項審查。</p>
<table>
  <thead>
      <tr>
          <th>面向</th>
          <th>SQLite source 問題</th>
          <th>PostgreSQL target 決策</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Type</td>
          <td>dynamic typing、STRICT usage</td>
          <td>integer / bigint / numeric / timestamptz</td>
      </tr>
      <tr>
          <td>Primary key</td>
          <td>rowid、INTEGER PRIMARY KEY</td>
          <td>identity、sequence、UUID</td>
      </tr>
      <tr>
          <td>Date/time</td>
          <td>TEXT / INTEGER convention</td>
          <td>timestamptz、timezone policy</td>
      </tr>
      <tr>
          <td>JSON</td>
          <td>JSON text / function usage</td>
          <td>jsonb、GIN index、query rewrite</td>
      </tr>
      <tr>
          <td>Constraint</td>
          <td>FK pragma、check、unique collation</td>
          <td>enforced FK、deferrable、collation</td>
      </tr>
      <tr>
          <td>Index</td>
          <td>partial / expression / covering index</td>
          <td>equivalent index + explain</td>
      </tr>
      <tr>
          <td>Transaction</td>
          <td>single writer、savepoint</td>
          <td>isolation level、deadlock retry</td>
      </tr>
  </tbody>
</table>
<p>Type mapping 要先保護 domain invariant。金額欄位用 integer cents 或 numeric、時間欄位用 timestamptz 或明確 UTC text、boolean 用 boolean；每個轉換都要有 invalid sample 與 round-trip test。</p>
<p>Index mapping 要用 production query 重跑 explain。SQLite 的 <code>EXPLAIN QUERY PLAN</code> 只能說明 SQLite planner；PostgreSQL 需要自己的 <code>EXPLAIN (ANALYZE, BUFFERS)</code>，並使用接近真實分布的資料量。</p>
<h2 id="phase-plan">Phase Plan</h2>
<p>Phase plan 的核心責任是降低一次性 cutover 風險。SQLite to PostgreSQL migration 通常可以分成 schema 建模、資料匯出、adapter 切換、shadow read、freeze / cutover 與 cleanup。</p>
<table>
  <thead>
      <tr>
          <th>Phase</th>
          <th>目的</th>
          <th>Evidence</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Schema rewrite</td>
          <td>建立 PostgreSQL target schema</td>
          <td>migration dry run、schema review</td>
      </tr>
      <tr>
          <td>Data export</td>
          <td>從 SQLite 取出穩定 snapshot</td>
          <td>source checksum、row count、export log</td>
      </tr>
      <tr>
          <td>Data import</td>
          <td>寫入 PostgreSQL</td>
          <td>target checksum、constraint validation</td>
      </tr>
      <tr>
          <td>Adapter layer</td>
          <td>將 repository 改為可切換</td>
          <td>dual test suite、error mapping</td>
      </tr>
      <tr>
          <td>Shadow read</td>
          <td>比對新舊 query result</td>
          <td>mismatch report、latency profile</td>
      </tr>
      <tr>
          <td>Cutover</td>
          <td>切正式寫入</td>
          <td>freeze window、rollback snapshot</td>
      </tr>
      <tr>
          <td>Cleanup</td>
          <td>退役 SQLite write path</td>
          <td>retention、credential、runbook update</td>
      </tr>
  </tbody>
</table>
<p>Adapter layer 是風險控制點。Repository 應把 SQLite 與 PostgreSQL driver 差異藏在 infrastructure layer，domain 不直接依賴 vendor-specific SQL exception 或 connection object。</p>
<p>Shadow read 適合先驗證 read contract。正式寫入仍留在 SQLite 時，background job 可以把相同 query 跑到 PostgreSQL mirror，記錄 row count、field diff、排序差異與 latency。</p>
<h2 id="data-movement">Data Movement</h2>
<p>Data movement 的核心責任是讓搬遷結果可驗證。SQLite database file 可以透過 <code>.dump</code>、CSV export、application-level export 或 custom ETL 搬入 PostgreSQL；選擇取決於資料量、型別轉換、FK order 與 downtime window。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;.mode csv&#34;</span> <span class="s2">&#34;.headers on&#34;</span> <span class="s2">&#34;.once orders.csv&#34;</span> <span class="s2">&#34;SELECT * FROM orders ORDER BY id;&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">psql <span class="s2">&#34;</span><span class="nv">$DATABASE_URL</span><span class="s2">&#34;</span> -c <span class="s2">&#34;\\copy orders FROM &#39;orders.csv&#39; CSV HEADER&#34;</span></span></span></code></pre></div><p>這段命令是教學骨架。正式 migration 要處理 quoting、NULL、timezone、large object、FK order、batch size、transaction size、retry、import log 與 sensitive data handling。</p>
<p>Row count 是基本證據，checksum 是更強證據。可以針對每張表計算穩定排序後的 hash，或在 application layer 對 domain key 與重要欄位做 checksum。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">SELECT</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">SELECT</span><span class="w"> </span><span class="k">SUM</span><span class="p">(</span><span class="n">total_cents</span><span class="p">)</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">orders</span><span class="p">;</span></span></span></code></pre></div><p>Aggregate checksum 適合快速抓大錯。正式驗證還要補抽樣 row diff、edge case row、foreign key check 與 business invariant。</p>
<h2 id="cutover">Cutover</h2>
<p>Cutover 的核心責任是控制最後一次寫入切換。SQLite source 在 cutover 前應進入 read-only 或 writer freeze，確保最後 snapshot、import 與 validation 對齊。</p>
<table>
  <thead>
      <tr>
          <th>Cutover step</th>
          <th>操作</th>
          <th>Rollback 條件</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Freeze writers</td>
          <td>停止背景 job、API write、admin tool</td>
          <td>source 寫入仍持續或 freeze 失敗</td>
      </tr>
      <tr>
          <td>Final snapshot</td>
          <td>SQLite backup / export</td>
          <td>checksum 失敗</td>
      </tr>
      <tr>
          <td>Final import</td>
          <td>PostgreSQL transaction / batch import</td>
          <td>constraint error、row mismatch</td>
      </tr>
      <tr>
          <td>Smoke test</td>
          <td>核心 read/write workflow</td>
          <td>error rate、latency、permission failure</td>
      </tr>
      <tr>
          <td>Switch traffic</td>
          <td>更新 config / secret / deployment</td>
          <td>application error rate 超過 tripwire</td>
      </tr>
      <tr>
          <td>Monitor</td>
          <td>query latency、lock、connection pool</td>
          <td>pool exhaustion、deadlock spike、data diff</td>
      </tr>
  </tbody>
</table>
<p>Rollback 要保存 source snapshot。若 cutover 後發現 PostgreSQL error mapping、permission 或 performance 問題，可以切回 SQLite read/write snapshot；前提是 cutover window 內所有新寫入都能回放或被阻擋。</p>
<h2 id="postgresql-operation-gate">PostgreSQL Operation Gate</h2>
<p>PostgreSQL operation gate 的核心責任是確認團隊準備好接手 server DB。Migration 成功要包含資料進入 target 與 operation readiness；PostgreSQL 需要 connection pool、backup / PITR、vacuum、index bloat、role、migration lock review 與 alert。</p>
<p>最小 operation checklist：</p>
<ol>
<li>Connection pool 設計：max connections、pool size、timeout、transaction pooling policy。</li>
<li>Backup / PITR：restore drill、retention、RPO / RTO。</li>
<li>Role / grant：application role、migration role、read-only role。</li>
<li>Migration lock review：DDL impact、online migration strategy。</li>
<li>Observability：slow query、lock wait、deadlock、replica lag、disk。</li>
<li>Incident route：rollback、restore、read-only mode、on-call owner。</li>
</ol>
<p>這個 gate 要在 cutover 前完成。SQLite 讓 operation surface 很小；PostgreSQL 擴大能力的同時，也擴大維護責任。</p>
<h2 id="no-go-conditions">No-Go Conditions</h2>
<p>No-go condition 的核心責任是阻止過早升級。若服務仍是 single-user、local-first、low-write、可用簡單 backup 解決，PostgreSQL 可能引入比問題更大的 operation cost。</p>
<table>
  <thead>
      <tr>
          <th>No-go 訊號</th>
          <th>更合適路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Single-user app 或 desktop app</td>
          <td>保留 SQLite + backup / migration runbook</td>
      </tr>
      <tr>
          <td>主要壓力是備份</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/litestream-litefs-replication/" data-link-title="SQLite Litestream / LiteFS Replication" data-link-desc="Litestream、LiteFS、SQLite backup replication、read replica、failover 與 restore route">Litestream / LiteFS</a></td>
      </tr>
      <tr>
          <td>主要壓力是 edge locality</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">D1 / Turso route</a></td>
      </tr>
      <tr>
          <td>Team 尚未準備 server DB operation</td>
          <td>先補 observability / restore drill</td>
      </tr>
      <tr>
          <td>Schema / query 還在快速探索</td>
          <td>先穩定 domain model，再做正式 migration</td>
      </tr>
  </tbody>
</table>
<p>No-go 條件要轉成 tripwire。當 writer concurrency、audit、PITR、role 或 HA 需求跨過明確門檻，再啟動 migration。</p>
<h2 id="下一步路由">下一步路由</h2>
<p>SQLite to PostgreSQL migration 完成後，下一步要看 target operation。PostgreSQL 能力讀 <a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a>；migration 方法讀 <a href="/blog/backend/01-database/database-migration-playbook/" data-link-title="1.6 資料庫轉換實作：雙寫、回填、切流與回滾" data-link-desc="同 DB 內 schema 演進與資料變更的可分段驗證流程、跟 1.12 cross-DB migration 分工">Database Migration Playbook</a>；若需求只是 edge platform，改讀 <a href="/blog/backend/01-database/vendors/sqlite/migrate-to-d1-turso/" data-link-title="SQLite to D1 / Turso Migration" data-link-desc="SQLite 轉向 Cloudflare D1、Turso / libSQL 的 edge driver、compatibility audit、data movement 與 rollback">SQLite to D1 / Turso migration</a>。</p>
]]></content:encoded></item><item><title>SQLite WAL Busy Reproduction</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/</guid><description>&lt;p>SQLite WAL busy reproduction 的核心責任是讓讀者親眼看到 single writer boundary。這篇承接 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking&lt;/a>，把 &lt;code>SQLITE_BUSY&lt;/code> 從文字警告轉成可重現 timeline。&lt;/p>
&lt;p>本文的驗收標準是：你能用兩個 sqlite3 session 重現 writer contention，觀察 busy timeout 行為，並用 WAL size 與 checkpoint result 連回 production runbook。&lt;/p>
&lt;h2 id="prepare-database">Prepare Database&lt;/h2>
&lt;p>Prepare database 的核心責任是建立可重現的 WAL mode database。若已跑過 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/" data-link-title="SQLite Local File Quickstart" data-link-desc="SQLite local .db file、schema、seed data、PRAGMA baseline、query sample 與 cleanup 的操作說明">local file quickstart&lt;/a>，可以沿用 &lt;code>/tmp/sqlite-lab/app.db&lt;/code>。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;PRAGMA journal_mode = WAL;&amp;#34;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;PRAGMA busy_timeout = 1000;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>確認 WAL mode：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;PRAGMA journal_mode;&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>預期輸出是 &lt;code>wal&lt;/code>。&lt;/p>
&lt;h2 id="session-a-hold-writer-lock">Session A: Hold Writer Lock&lt;/h2>
&lt;p>Session A 的核心責任是刻意持有 write transaction。開第一個 terminal，執行：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">sqlite3 app.db&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>在 sqlite prompt 內輸入：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="n">PRAGMA&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">foreign_keys&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="o">=&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">ON&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">BEGIN&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">IMMEDIATE&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">INSERT&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="k">INTO&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">ledger_entries&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="n">account_id&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">amount_cents&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">idempotency_key&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="n">created_at&lt;/span>&lt;span class="p">)&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">4&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="k">VALUES&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="mi">1&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="mi">11&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;busy-session-a&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span>&lt;span class="w"> &lt;/span>&lt;span class="s1">&amp;#39;2026-05-21T02:00:00Z&amp;#39;&lt;/span>&lt;span class="p">);&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>先保持 transaction 開啟，暫時延後 &lt;code>COMMIT&lt;/code>。&lt;code>BEGIN IMMEDIATE&lt;/code> 會取得 writer lock，讓第二個 writer 需要等待或失敗。&lt;/p>
&lt;h2 id="session-b-observe-busy">Session B: Observe Busy&lt;/h2>
&lt;p>Session B 的核心責任是用第二個 connection 觀察 single writer boundary。開第二個 terminal，執行：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/sqlite-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">sqlite3 app.db &lt;span class="s2">&amp;#34;PRAGMA busy_timeout = 1000; INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 22, &amp;#39;busy-session-b&amp;#39;, &amp;#39;2026-05-21T02:01:00Z&amp;#39;);&amp;#34;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>預期結果是等待約 1 秒後出現 busy / locked 類錯誤。不同 sqlite3 版本的錯誤文字可能略有差異，核心訊號是第二個 writer 在 Session A commit 前拿不到 write lock。&lt;/p>
&lt;h2 id="release-lock">Release Lock&lt;/h2>
&lt;p>Release lock 的核心責任是確認 contention 來自 writer transaction。回到 Session A，輸入：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-sql" data-lang="sql">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="k">COMMIT&lt;/span>&lt;span class="p">;&lt;/span>&lt;span class="w">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="w">&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="n">quit&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>再次執行 Session B 的 insert，這次應成功。&lt;/p></description><content:encoded><![CDATA[<p>SQLite WAL busy reproduction 的核心責任是讓讀者親眼看到 single writer boundary。這篇承接 <a href="/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/" data-link-title="SQLite WAL Concurrency and Locking" data-link-desc="SQLite WAL mode 如何降低 reader / writer 衝突、保留 single writer boundary，並用 SQLITE_BUSY、WAL growth、checkpoint 訊號判斷 production 上限">WAL concurrency / locking</a>，把 <code>SQLITE_BUSY</code> 從文字警告轉成可重現 timeline。</p>
<p>本文的驗收標準是：你能用兩個 sqlite3 session 重現 writer contention，觀察 busy timeout 行為，並用 WAL size 與 checkpoint result 連回 production runbook。</p>
<h2 id="prepare-database">Prepare Database</h2>
<p>Prepare database 的核心責任是建立可重現的 WAL mode database。若已跑過 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/local-file-quickstart/" data-link-title="SQLite Local File Quickstart" data-link-desc="SQLite local .db file、schema、seed data、PRAGMA baseline、query sample 與 cleanup 的操作說明">local file quickstart</a>，可以沿用 <code>/tmp/sqlite-lab/app.db</code>。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-lab
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA journal_mode = WAL;&#34;</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA busy_timeout = 1000;&#34;</span></span></span></code></pre></div><p>確認 WAL mode：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA journal_mode;&#34;</span></span></span></code></pre></div><p>預期輸出是 <code>wal</code>。</p>
<h2 id="session-a-hold-writer-lock">Session A: Hold Writer Lock</h2>
<p>Session A 的核心責任是刻意持有 write transaction。開第一個 terminal，執行：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db</span></span></code></pre></div><p>在 sqlite prompt 內輸入：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">foreign_keys</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">ON</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">BEGIN</span><span class="w"> </span><span class="k">IMMEDIATE</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">ledger_entries</span><span class="p">(</span><span class="n">account_id</span><span class="p">,</span><span class="w"> </span><span class="n">amount_cents</span><span class="p">,</span><span class="w"> </span><span class="n">idempotency_key</span><span class="p">,</span><span class="w"> </span><span class="n">created_at</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w"></span><span class="k">VALUES</span><span class="w"> </span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">11</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;busy-session-a&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;2026-05-21T02:00:00Z&#39;</span><span class="p">);</span></span></span></code></pre></div><p>先保持 transaction 開啟，暫時延後 <code>COMMIT</code>。<code>BEGIN IMMEDIATE</code> 會取得 writer lock，讓第二個 writer 需要等待或失敗。</p>
<h2 id="session-b-observe-busy">Session B: Observe Busy</h2>
<p>Session B 的核心責任是用第二個 connection 觀察 single writer boundary。開第二個 terminal，執行：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="nb">cd</span> /tmp/sqlite-lab
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA busy_timeout = 1000; INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 22, &#39;busy-session-b&#39;, &#39;2026-05-21T02:01:00Z&#39;);&#34;</span></span></span></code></pre></div><p>預期結果是等待約 1 秒後出現 busy / locked 類錯誤。不同 sqlite3 版本的錯誤文字可能略有差異，核心訊號是第二個 writer 在 Session A commit 前拿不到 write lock。</p>
<h2 id="release-lock">Release Lock</h2>
<p>Release lock 的核心責任是確認 contention 來自 writer transaction。回到 Session A，輸入：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">COMMIT</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="p">.</span><span class="n">quit</span></span></span></code></pre></div><p>再次執行 Session B 的 insert，這次應成功。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA foreign_keys = ON; INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 22, &#39;busy-session-b&#39;, &#39;2026-05-21T02:01:00Z&#39;);&#34;</span></span></span></code></pre></div><p>若 idempotency key 已在前一次嘗試中寫入，改成新的 key。這個細節也提醒 production write 要有 idempotency 設計。</p>
<h2 id="busy-timeout-comparison">Busy Timeout Comparison</h2>
<p>Busy timeout comparison 的核心責任是區分「等一下」和「解決 writer contention」。Timeout 可以讓短暫鎖等待更平滑，但長交易仍會造成延遲或失敗。</p>
<p>重開 Session A 並持有 transaction：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">BEGIN</span><span class="w"> </span><span class="k">IMMEDIATE</span><span class="p">;</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">ledger_entries</span><span class="p">(</span><span class="n">account_id</span><span class="p">,</span><span class="w"> </span><span class="n">amount_cents</span><span class="p">,</span><span class="w"> </span><span class="n">idempotency_key</span><span class="p">,</span><span class="w"> </span><span class="n">created_at</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">VALUES</span><span class="w"> </span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="w"> </span><span class="mi">33</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;busy-session-a-long&#39;</span><span class="p">,</span><span class="w"> </span><span class="s1">&#39;2026-05-21T02:10:00Z&#39;</span><span class="p">);</span></span></span></code></pre></div><p>在 Session B 測不同 timeout：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="nb">time</span> sqlite3 app.db <span class="s2">&#34;PRAGMA busy_timeout = 5000; INSERT INTO ledger_entries(account_id, amount_cents, idempotency_key, created_at) VALUES (1, 44, &#39;busy-session-b-long&#39;, &#39;2026-05-21T02:11:00Z&#39;);&#34;</span></span></span></code></pre></div><p>若 Session A 在 5 秒內 commit，Session B 可能成功；若持續持有 transaction，Session B 會在 timeout 後失敗。這就是 production 裡 busy timeout 的邊界：它緩衝短鎖，長 transaction 仍要被設計移除。</p>
<h2 id="wal-and-checkpoint">WAL and Checkpoint</h2>
<p>WAL and checkpoint 的核心責任是把 writer activity 和 file artifact 連起來。多做幾次寫入後觀察 sidecar。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">ls -lh app.db app.db-wal app.db-shm
</span></span><span class="line"><span class="ln">2</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA wal_checkpoint(PASSIVE);&#34;</span></span></span></code></pre></div><p><code>wal_checkpoint</code> 會回傳 checkpoint 狀態。正式 runbook 要記錄 WAL size、checkpoint duration、reader age 與 checkpoint failure。</p>
<p>可以手動觸發 truncate checkpoint：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">sqlite3 app.db <span class="s2">&#34;PRAGMA wal_checkpoint(TRUNCATE);&#34;</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">ls -lh app.db app.db-wal app.db-shm</span></span></code></pre></div><p>TRUNCATE 適合 lab 觀察。Production 使用時要評估 reader、latency 與維護窗口。</p>
<h2 id="mitigation-note">Mitigation Note</h2>
<p>Mitigation note 的核心責任是把 lab 結果轉成設計策略。看到 <code>SQLITE_BUSY</code> 後，優先檢查 long transaction、未關閉 cursor、背景 job、write burst、parallel test 共用 DB 與 checkpoint pressure。</p>
<p>常見策略包含：</p>
<ol>
<li>縮短 transaction，將外部 API call 移到 transaction 外。</li>
<li>設定合理 busy timeout 與 retry backoff。</li>
<li>把 write queue 序列化，讓高風險 workflow 先排隊。</li>
<li>將 heavy read 移到 snapshot 或 replica。</li>
<li>當 concurrent writer 成為常態，評估 PostgreSQL / MySQL。</li>
</ol>
<p>完成本篇後，下一步讀 <a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">observability / runbook</a> 把 busy、WAL 與 checkpoint 變成正式監控訊號。</p>
]]></content:encoded></item><item><title>SQLite WAL Concurrency and Locking</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/</link><pubDate>Thu, 21 May 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/wal-concurrency-locking/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite&lt;/a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的 single-file / embedded 定位；本文聚焦 &lt;em>WAL concurrency、single writer boundary、&lt;code>SQLITE_BUSY&lt;/code> 與 checkpoint strategy&lt;/em>。&lt;/p>&lt;/blockquote>
&lt;p>SQLite WAL concurrency 的核心責任是讓 reader / writer 衝突下降，同時保留單檔案資料庫的寫入邊界。&lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/write-ahead-log/" data-link-title="Write-Ahead Log" data-link-desc="說明資料庫如何先寫入 log 再合併回主資料，以提供持久性與崩潰復原">WAL mode&lt;/a> 把寫入 append 到 &lt;code>-wal&lt;/code> sidecar file，reader 可以從 main database file 加 WAL snapshot 讀取一致視圖；這讓 read-heavy workload 能比 rollback journal mode 更順。但 SQLite 仍遵循 &lt;a href="https://tarrragon.github.io/blog/backend/knowledge-cards/single-writer-model/" data-link-title="Single Writer Model" data-link-desc="說明單寫者模型如何序列化寫入，並成為系統的容量邊界">single writer model&lt;/a>、只有一條 writer path，長交易、背景 migration、慢 disk 或多 process 寫入都會在這條 path 上排隊。&lt;/p>
&lt;p>本文的判讀錨點是：WAL 提升的是 reader concurrency，治理的是 writer queue。當服務看到 &lt;code>SQLITE_BUSY&lt;/code>、WAL file 持續變大、checkpoint duration 變長或偶發 commit latency spike，問題通常在 transaction duration、checkpoint cadence、filesystem lock 或 process ownership，而非單純「資料庫太小」。&lt;/p>
&lt;h2 id="wal-mode-的服務責任">WAL mode 的服務責任&lt;/h2>
&lt;p>WAL mode 的服務責任是把「寫入直接改 main database file」改成「寫入先 append 到 WAL，再由 checkpoint 合併回 main database」。SQLite 官方文件把 WAL 模型拆成 reading、writing、checkpointing 三個 primitive；這個 framing 對 production runbook 很重要，因為 checkpoint 會變成獨立的操作訊號。&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>模式&lt;/th>
 &lt;th>寫入路徑&lt;/th>
 &lt;th>Reader 影響&lt;/th>
 &lt;th>Production 判讀&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>Rollback journal&lt;/td>
 &lt;td>寫入前保存原始 page，再修改 main file&lt;/td>
 &lt;td>write 期間更容易和 reader 互相等待&lt;/td>
 &lt;td>適合簡單、低並發、短交易路徑&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>WAL&lt;/td>
 &lt;td>寫入 append 到 &lt;code>-wal&lt;/code>，checkpoint 後合併&lt;/td>
 &lt;td>reader 可看自己的 WAL snapshot&lt;/td>
 &lt;td>適合 read-heavy、互動式、短寫交易 workload&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;p>這張表的讀法是先看服務是否主要受 read / write 衝突影響。Read-heavy CLI、desktop、mobile、edge-local API 或 small backend 往往能從 WAL mode 受益；write-heavy queue consumer、batch import、multi-process writer 或 high-concurrency OLTP 則會先撞到 single writer boundary。&lt;/p>
&lt;h2 id="locking-model多-reader-與單-writer-是同時成立的">Locking model：多 reader 與單 writer 是同時成立的&lt;/h2>
&lt;p>SQLite locking model 的核心責任是保護單一 database file 的 ACID 邊界。Rollback journal mode 的官方 locking 文件描述了 SHARED、RESERVED、PENDING、EXCLUSIVE 等狀態；WAL mode 的細節另由 WAL 文件說明，但服務判讀上仍要記住同一件事：跨 connection / process 的寫入要被序列化。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite</a> overview 的 implementation-layer deep article。Overview 已說明 SQLite 的 single-file / embedded 定位；本文聚焦 <em>WAL concurrency、single writer boundary、<code>SQLITE_BUSY</code> 與 checkpoint strategy</em>。</p></blockquote>
<p>SQLite WAL concurrency 的核心責任是讓 reader / writer 衝突下降，同時保留單檔案資料庫的寫入邊界。<a href="/blog/backend/knowledge-cards/write-ahead-log/" data-link-title="Write-Ahead Log" data-link-desc="說明資料庫如何先寫入 log 再合併回主資料，以提供持久性與崩潰復原">WAL mode</a> 把寫入 append 到 <code>-wal</code> sidecar file，reader 可以從 main database file 加 WAL snapshot 讀取一致視圖；這讓 read-heavy workload 能比 rollback journal mode 更順。但 SQLite 仍遵循 <a href="/blog/backend/knowledge-cards/single-writer-model/" data-link-title="Single Writer Model" data-link-desc="說明單寫者模型如何序列化寫入，並成為系統的容量邊界">single writer model</a>、只有一條 writer path，長交易、背景 migration、慢 disk 或多 process 寫入都會在這條 path 上排隊。</p>
<p>本文的判讀錨點是：WAL 提升的是 reader concurrency，治理的是 writer queue。當服務看到 <code>SQLITE_BUSY</code>、WAL file 持續變大、checkpoint duration 變長或偶發 commit latency spike，問題通常在 transaction duration、checkpoint cadence、filesystem lock 或 process ownership，而非單純「資料庫太小」。</p>
<h2 id="wal-mode-的服務責任">WAL mode 的服務責任</h2>
<p>WAL mode 的服務責任是把「寫入直接改 main database file」改成「寫入先 append 到 WAL，再由 checkpoint 合併回 main database」。SQLite 官方文件把 WAL 模型拆成 reading、writing、checkpointing 三個 primitive；這個 framing 對 production runbook 很重要，因為 checkpoint 會變成獨立的操作訊號。</p>
<table>
  <thead>
      <tr>
          <th>模式</th>
          <th>寫入路徑</th>
          <th>Reader 影響</th>
          <th>Production 判讀</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Rollback journal</td>
          <td>寫入前保存原始 page，再修改 main file</td>
          <td>write 期間更容易和 reader 互相等待</td>
          <td>適合簡單、低並發、短交易路徑</td>
      </tr>
      <tr>
          <td>WAL</td>
          <td>寫入 append 到 <code>-wal</code>，checkpoint 後合併</td>
          <td>reader 可看自己的 WAL snapshot</td>
          <td>適合 read-heavy、互動式、短寫交易 workload</td>
      </tr>
  </tbody>
</table>
<p>這張表的讀法是先看服務是否主要受 read / write 衝突影響。Read-heavy CLI、desktop、mobile、edge-local API 或 small backend 往往能從 WAL mode 受益；write-heavy queue consumer、batch import、multi-process writer 或 high-concurrency OLTP 則會先撞到 single writer boundary。</p>
<h2 id="locking-model多-reader-與單-writer-是同時成立的">Locking model：多 reader 與單 writer 是同時成立的</h2>
<p>SQLite locking model 的核心責任是保護單一 database file 的 ACID 邊界。Rollback journal mode 的官方 locking 文件描述了 SHARED、RESERVED、PENDING、EXCLUSIVE 等狀態；WAL mode 的細節另由 WAL 文件說明，但服務判讀上仍要記住同一件事：跨 connection / process 的寫入要被序列化。</p>
<table>
  <thead>
      <tr>
          <th>角色</th>
          <th>WAL mode 下的責任</th>
          <th>常見失效訊號</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Reader</td>
          <td>讀取開始時固定自己的 snapshot end mark</td>
          <td>長讀取讓 checkpoint 停在舊 snapshot，WAL file 持續變大</td>
      </tr>
      <tr>
          <td>Writer</td>
          <td>append 新 transaction 到同一個 WAL file</td>
          <td>其他 writer 看到 <code>SQLITE_BUSY</code> 或 write latency spike</td>
      </tr>
      <tr>
          <td>Checkpoint</td>
          <td>把 WAL frame 合併回 main database file</td>
          <td>checkpoint duration 拉長、commit 偶發變慢</td>
      </tr>
      <tr>
          <td>Filesystem</td>
          <td>提供可靠 file lock 與 shared-memory 支援</td>
          <td>network filesystem、container mount 或權限造成異常</td>
      </tr>
  </tbody>
</table>
<p>多 reader 與單 writer 的組合是 SQLite 的正常設計。讀者在查問題時，要避免把 <code>SQLITE_BUSY</code> 直接解讀成資料毀損；它多半代表某個 connection 正在持有 writer 所需的 lock，或 checkpoint / transaction 正在等待可前進的窗口。</p>
<h2 id="sqlite_busy-的第一輪排查"><code>SQLITE_BUSY</code> 的第一輪排查</h2>
<p><code>SQLITE_BUSY</code> 的核心意義是某個 connection 當下拿不到需要的 lock。SQLite 提供 <code>busy_timeout</code> 讓 connection 等待一段時間；這能吸收短暫 writer queue，但它只是等待策略，single writer boundary 仍然存在。</p>
<table>
  <thead>
      <tr>
          <th>觀察訊號</th>
          <th>可能原因</th>
          <th>第一輪處理</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>短暫 <code>SQLITE_BUSY</code></td>
          <td>多個短寫入撞在一起</td>
          <td>設定 bounded busy timeout，縮短 transaction duration</td>
      </tr>
      <tr>
          <td>持續 <code>SQLITE_BUSY</code></td>
          <td>長交易、migration、batch import</td>
          <td>找出持鎖 connection，拆小 transaction 或移到 maintenance window</td>
      </tr>
      <tr>
          <td>commit latency 偶發變慢</td>
          <td>auto-checkpoint 在 commit path 上</td>
          <td>調整 auto-checkpoint，改由 background checkpoint</td>
      </tr>
      <tr>
          <td>read query 讓 WAL 變大</td>
          <td>long reader 卡住 checkpoint</td>
          <td>限制長查詢、拆 reporting query、設定 reader timeout</td>
      </tr>
      <tr>
          <td>部署後 busy rate 上升</td>
          <td>instance 數增加、multi-process write</td>
          <td>重新檢查 writer ownership，必要時升級 server SQL</td>
      </tr>
  </tbody>
</table>
<p>這張表的重點是先找「誰持有 writer path」。如果問題來自單一長 transaction，修 transaction boundary；如果問題來自多個 process 同時寫同檔，修 process ownership；如果問題來自真實高寫入吞吐，SQLite 已經接近服務邊界。</p>
<h2 id="busy-timeout-是緩衝器容量邊界仍在-writer-path">Busy timeout 是緩衝器，容量邊界仍在 writer path</h2>
<p>Busy timeout 的服務責任是吸收短時間 lock collision。它適合 desktop app autosave、mobile local store、短 API write、測試 fixture 或偶發 background job；它不適合作為高寫入吞吐的主要容量策略。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">busy_timeout</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">5000</span><span class="p">;</span></span></span></code></pre></div><p>這個設定代表 connection 最多等待 5000 ms。Production runbook 要同時記錄三個訊號：busy 次數、等待時間分布、等待後成功率。若等待後成功率高且 p99 可接受，代表 writer queue 仍在服務邊界內；若等待常超時，代表 transaction duration 或 writer 並發已經超出單檔模型。</p>
<h2 id="checkpoint-strategywal-growth-是操作訊號">Checkpoint strategy：WAL growth 是操作訊號</h2>
<p>Checkpoint 的核心責任是把 WAL 中的 committed frames 合併回 main database file。SQLite 預設會在 WAL file 達到約 1000 pages 後自動 checkpoint；這個預設適合多數小型場景，但 production 服務要把 checkpoint 視為獨立操作。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="n">PRAGMA</span><span class="w"> </span><span class="n">wal_checkpoint</span><span class="p">(</span><span class="n">PASSIVE</span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">wal_checkpoint</span><span class="p">(</span><span class="k">FULL</span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">wal_checkpoint</span><span class="p">(</span><span class="k">RESTART</span><span class="p">);</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w"></span><span class="n">PRAGMA</span><span class="w"> </span><span class="n">wal_checkpoint</span><span class="p">(</span><span class="k">TRUNCATE</span><span class="p">);</span></span></span></code></pre></div><table>
  <thead>
      <tr>
          <th>Checkpoint 型態</th>
          <th>操作語意</th>
          <th>適合場景</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>PASSIVE</td>
          <td>盡量前進，避免主動阻塞 reader / writer</td>
          <td>日常觀測、低風險背景 checkpoint</td>
      </tr>
      <tr>
          <td>FULL</td>
          <td>等待 writer，嘗試完成更多 checkpoint</td>
          <td>maintenance window、WAL growth 需要收斂</td>
      </tr>
      <tr>
          <td>RESTART</td>
          <td>完成後讓後續 writer 可重新使用 WAL</td>
          <td>想降低 WAL 持續膨脹，能接受等待</td>
      </tr>
      <tr>
          <td>TRUNCATE</td>
          <td>完成後截斷 WAL file</td>
          <td>低流量窗口、需要回收檔案空間</td>
      </tr>
  </tbody>
</table>
<p>Checkpoint 策略的判讀要看 workload cadence。互動式服務通常保留 auto-checkpoint，再加上低流量時段的 background checkpoint；長查詢或 reporting workload 需要避免讓 long reader 長期佔住 snapshot；batch import 則要把 transaction 切小，避免 WAL file 在單一交易期間快速膨脹。</p>
<h2 id="checkpoint-starvation長-reader-會讓-wal-持續長大">Checkpoint starvation：長 reader 會讓 WAL 持續長大</h2>
<p>Checkpoint starvation 的核心概念是：只要總有 reader 還在使用舊 snapshot，checkpoint 就可能停在 reset 之前。SQLite 官方 WAL 文件明確指出，checkpoint 可以和 reader 並行，但遇到仍被 reader 使用的 WAL 位置時要停下來；如果長時間沒有 reader gap，WAL file 會持續成長。</p>
<table>
  <thead>
      <tr>
          <th>情境</th>
          <th>真實服務長相</th>
          <th>修正方向</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Desktop app 開著長報表</td>
          <td>使用者查詢大列表，背景寫入持續發生</td>
          <td>報表分頁、限制 read transaction duration</td>
      </tr>
      <tr>
          <td>API handler 把 cursor 留太久</td>
          <td>streaming response 邊讀邊回，交易未結束</td>
          <td>先 materialize 結果、縮短 DB read transaction</td>
      </tr>
      <tr>
          <td>Background sync 長讀取</td>
          <td>sync worker 掃全表，UI 仍在寫資料</td>
          <td>分批讀取、讀寫排程、低流量 checkpoint</td>
      </tr>
      <tr>
          <td>Test suite 平行讀寫 fixture</td>
          <td>測試共用同一 <code>.db</code>，多 worker 交錯</td>
          <td>per-test DB、read-only fixture、獨立 temp file</td>
      </tr>
  </tbody>
</table>
<p>這些情境的共同點是 reader lifecycle 沒有被 application 控制。SQLite 的 concurrency 問題常發生在 application boundary，而非 database engine 本身；修法也應回到 handler、worker、test runner 或 UI lifecycle。</p>
<h2 id="filesystem-與-deployment-boundary">Filesystem 與 deployment boundary</h2>
<p>SQLite WAL 的 deployment boundary 是 local filesystem 與可靠 shared-memory / file-locking primitive。官方 WAL 文件指出 wal-index 使用 shared memory，所有 reader 要位於同一台機器；這也是 WAL mode 不適合放在一般 network filesystem 上的主要原因。</p>
<table>
  <thead>
      <tr>
          <th>部署方式</th>
          <th>判讀</th>
          <th>建議路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>單 process / 單機 local disk</td>
          <td>SQLite 最自然的部署形狀</td>
          <td>WAL + backup / restore runbook</td>
      </tr>
      <tr>
          <td>多 process / 同機 local disk</td>
          <td>可行，但要清楚 writer ownership 與 timeout</td>
          <td>WAL + busy timeout + checkpoint evidence</td>
      </tr>
      <tr>
          <td>多 instance / shared volume</td>
          <td>lock 與 writer ownership 風險上升</td>
          <td>升級 PostgreSQL / MySQL，或改用明確 primary pattern</td>
      </tr>
      <tr>
          <td>network filesystem</td>
          <td>WAL shared-memory 與 file lock 語意風險高</td>
          <td>改 local disk + replication，或 server database</td>
      </tr>
      <tr>
          <td>container ephemeral disk</td>
          <td>durability 與 restore 路徑要重新設計</td>
          <td>persistent volume、backup drill、restore evidence</td>
      </tr>
  </tbody>
</table>
<p>Deployment review 要問的第一個問題是「同一時間誰會寫這個檔案」。如果答案是多個 instance、跨機器 process 或不受控 job，SQLite 的服務邊界已經需要重新評估。</p>
<h2 id="production-踩雷">Production 踩雷</h2>
<h3 id="case-1多個-worker-同時寫同一個-sqlite-檔">Case 1：多個 worker 同時寫同一個 SQLite 檔</h3>
<p>多 worker 寫入同一個 SQLite 檔的核心風險是 writer ownership 消失。常見情境是小型服務從單 instance 擴到多 instance，但仍把 database file 放在 shared volume；早期看起來可運作，流量上升後開始出現 busy timeout、WAL growth 與偶發資料修復壓力。</p>
<p>修正方向是重新定義 writer。若服務仍是 small backend，可以收斂到單 writer process + queue；若 multi-instance 是長期需求，應遷移到 <a href="/blog/backend/01-database/vendors/postgresql/" data-link-title="PostgreSQL" data-link-desc="多用途 OLTP 主流關聯式資料庫、MVCC、豐富 SQL 特性、是 Aurora / Cosmos DB / Spanner / CockroachDB / Aurora DSQL 的相容目標">PostgreSQL</a> 或 <a href="/blog/backend/01-database/vendors/mysql/" data-link-title="MySQL" data-link-desc="高併發網路服務常用關聯式資料庫、Vitess / PlanetScale 分片生態、GitHub / Shopify / Facebook 規模驗證">MySQL</a>。</p>
<h3 id="case-2長讀取卡住-checkpoint磁碟被-wal-吃滿">Case 2：長讀取卡住 checkpoint，磁碟被 WAL 吃滿</h3>
<p>長讀取卡 checkpoint 的核心風險是 WAL file 成為隱性容量消耗。讀者可能只看到 disk usage 增長，誤以為是資料量變大；實際上 main database file 沒有明顯增長，<code>-wal</code> sidecar 持續膨脹。</p>
<p>修正方向是先找到長 reader，再調整 query lifecycle。Reporting query、background sync、streaming response、互動式 UI 大列表都要有 pagination、timeout 或低流量窗口；checkpoint 只負責收斂 WAL，application 仍要主動結束長讀取。</p>
<h3 id="case-3把-busy-timeout-當成擴容策略">Case 3：把 busy timeout 當成擴容策略</h3>
<p>Busy timeout 被當成擴容策略的核心風險是延遲被隱藏到使用者路徑。短暫 lock collision 可以等待；長期 write queue 則會把 API p99、UI freeze 或 worker backlog 拉高。</p>
<p>修正方向是把 busy wait 當 metric。設定 timeout 後要記錄等待時間與超時率；當 busy wait 成為常態，下一步是拆交易、調整 writer process、移走 batch job，或升級到 server database。</p>
<h3 id="case-4checkpoint-放在高流量-commit-path">Case 4：checkpoint 放在高流量 commit path</h3>
<p>Checkpoint 放在高流量 commit path 的核心風險是少數 commit 變得很慢。SQLite 預設 auto-checkpoint 對多數場景合理，但互動式服務可能看到偶發 latency spike；這時可以把 checkpoint 移到背景 thread / process 或低流量窗口。</p>
<p>修正方向是把 checkpoint duration 變成 evidence。觀察 WAL size、checkpoint return、commit latency 與 disk sync；若尖峰可接受，維持預設；若尖峰影響 UX，調整 checkpoint cadence。</p>
<h3 id="case-5wal-mode-版本與部署條件未納入維護">Case 5：WAL mode 版本與部署條件未納入維護</h3>
<p>WAL mode 的維護責任包含 SQLite runtime version、filesystem、sidecar file 與 release notes。SQLite 官方 WAL 文件記錄 2026-03 修正過罕見 WAL-reset bug；雖然觸發條件很窄，production runbook 仍應記錄 SQLite version、runtime package 與更新策略。</p>
<p>修正方向是把 SQLite runtime 當成 dependency。Mobile、desktop、embedded、language binding、OS bundled SQLite 可能各自帶不同版本；需要在 support matrix 中標明版本來源、WAL mode 行為與升級路徑。</p>
<h2 id="操作檢查清單">操作檢查清單</h2>
<p>SQLite WAL / locking runbook 至少要能回答下列問題：</p>
<ol>
<li>Database file、<code>-wal</code>、<code>-shm</code> 是否位於 local durable filesystem。</li>
<li>同一時間哪些 process / thread 會寫入 database file。</li>
<li><code>PRAGMA journal_mode</code>、<code>busy_timeout</code>、<code>wal_autocheckpoint</code> 如何設定。</li>
<li><code>SQLITE_BUSY</code> 次數、等待時間、超時率是否被記錄。</li>
<li>WAL file size、checkpoint duration、disk usage 是否被觀測。</li>
<li>長 read transaction 的來源與 timeout 如何治理。</li>
<li>Batch import、migration、background sync 是否避開互動式高峰。</li>
<li>SQLite runtime version 與 WAL 相關 release notes 如何追蹤。</li>
</ol>
<p>這份清單要接到 <a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">Observability / runbook</a> 與 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a>；正文教判讀，hands-on 負責讓讀者重現 <code>SQLITE_BUSY</code>、WAL growth 與 checkpoint 行為。</p>
<h2 id="何時維持-sqlite何時升級">何時維持 SQLite，何時升級</h2>
<p>SQLite WAL mode 適合單機、短交易、read-heavy、writer ownership 清楚的服務。只要 busy wait 可控、checkpoint 能完成、backup / restore drill 成立，SQLite 可以承擔正式狀態。</p>
<p>升級訊號來自 writer boundary 外溢。多 instance write、多 region write、high-write OLTP、集中權限治理、read replica、PITR、DB account / role 與 audit requirement 都會把服務推向 server SQL、edge SQLite product 或 distributed SQL。</p>
<table>
  <thead>
      <tr>
          <th>壓力</th>
          <th>SQLite 內修正</th>
          <th>升級路由</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>偶發 <code>SQLITE_BUSY</code></td>
          <td>busy timeout、縮短 transaction</td>
          <td>維持 SQLite</td>
      </tr>
      <tr>
          <td>WAL growth</td>
          <td>找長 reader、manual checkpoint</td>
          <td>維持 SQLite，補 observability</td>
      </tr>
      <tr>
          <td>多 worker 寫入</td>
          <td>收斂單 writer、queue 化</td>
          <td>PostgreSQL / MySQL</td>
      </tr>
      <tr>
          <td>Edge locality</td>
          <td>D1 / Turso compatibility audit</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/d1-turso-libsql-comparison/" data-link-title="SQLite D1 / Turso / libSQL Comparison" data-link-desc="Cloudflare D1、Turso、libSQL 與 local SQLite 在 edge、replication、consistency、migration 與 vendor boundary 的比較">D1 / Turso route</a></td>
      </tr>
      <tr>
          <td>HA / PITR / audit governance</td>
          <td>file backup 已經難以治理</td>
          <td><a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL</a></td>
      </tr>
  </tbody>
</table>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/sqlite/" data-link-title="SQLite" data-link-desc="embedded、單檔案、test / CLI / edge 場景的標準選擇、近年因 Cloudflare D1 / Turso 等服務復興">SQLite overview</a></li>
<li>前置：<a href="/blog/backend/01-database/vendors/sqlite/file-lifecycle-backup-boundary/" data-link-title="SQLite file lifecycle 與 backup boundary" data-link-desc="把 SQLite 單檔案正式狀態拆成 WAL、backup API、restore drill、corruption recovery 與操作責任邊界">File lifecycle / backup boundary</a></li>
<li>操作：<a href="/blog/backend/01-database/vendors/sqlite/hands-on/" data-link-title="SQLite Hands-on 操作路線" data-link-desc="SQLite local file lab、backup / restore drill、WAL busy reproduction、migration fixture、D1 / Turso preview 的操作型章節設計">SQLite Hands-on</a> 與 <a href="/blog/backend/01-database/vendors/sqlite/hands-on/wal-busy-reproduction/" data-link-title="SQLite WAL Busy Reproduction" data-link-desc="SQLite long transaction、SQLITE_BUSY、busy_timeout、checkpoint growth 與 writer queue 的操作說明">WAL busy reproduction</a></li>
<li>平行：<a href="/blog/backend/01-database/vendors/sqlite/pragma-tuning-performance/" data-link-title="SQLite PRAGMA Tuning and Performance" data-link-desc="SQLite journal_mode、synchronous、busy_timeout、wal_autocheckpoint、cache_size、mmap_size、auto_vacuum 與 performance evidence 的操作判準">PRAGMA tuning / performance</a>、<a href="/blog/backend/01-database/vendors/sqlite/observability-runbook/" data-link-title="SQLite Observability and Runbook" data-link-desc="SQLite production runbook、backup evidence、WAL growth、busy errors、disk usage、restore drill 與 incident route">Observability / runbook</a></li>
<li>遷移：<a href="/blog/backend/01-database/vendors/sqlite/migrate-to-postgresql/" data-link-title="SQLite to PostgreSQL Migration" data-link-desc="SQLite 升級到 PostgreSQL 的 driver、schema diff、data copy、dual run、cutover、rollback 與 cleanup">SQLite to PostgreSQL</a></li>
<li>官方：<a href="https://www.sqlite.org/wal.html">SQLite Write-Ahead Logging</a>、<a href="https://www.sqlite.org/lockingv3.html">SQLite File Locking</a>、<a href="https://www.sqlite.org/isolation.html">SQLite Isolation</a>、<a href="https://www.sqlite.org/pragma.html">SQLite PRAGMA</a></li>
</ul>
]]></content:encoded></item></channel></rss>