<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Distributed-Counter on Tarragon</title><link>https://tarrragon.github.io/blog/tags/distributed-counter/</link><description>Recent content in Distributed-Counter on Tarragon</description><generator>Hugo -- gohugo.io</generator><language>zh-TW</language><copyright>Tarragon (CC BY 4.0)</copyright><lastBuildDate>Tue, 16 Jun 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://tarrragon.github.io/blog/tags/distributed-counter/index.xml" rel="self" type="application/rss+xml"/><item><title>Firestore 高頻寫入與 distributed counter：單 document contention 邊界與分片計數</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/</link><pubDate>Tue, 16 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/" data-link-title="Firestore" data-link-desc="Firebase / Google Cloud 的 serverless document database、collection / document 模型、client 直連 &amp;#43; Security Rules、realtime listener 與 offline 同步、BaaS bundle 的資料層面">Firestore&lt;/a> overview 的 deep article。寫作參照 &lt;a href="https://tarrragon.github.io/blog/posts/vendor-%E6%B7%B1%E5%BA%A6%E6%8A%80%E8%A1%93%E6%96%87%E7%AB%A0%E6%96%B9%E6%B3%95%E8%AB%96%E7%9A%84%E6%BC%94%E5%8C%96%E7%B4%80%E9%8C%84%E5%90%8C-vendor-%E7%B3%BB%E5%88%97%E7%9A%84%E9%96%8B%E5%A0%B4%E8%BC%AA%E6%9B%BF%E9%A9%97%E8%AD%89/" data-link-title="Vendor 深度技術文章方法論的演化紀錄：同 vendor 系列的開場輪替驗證" data-link-desc="vendor overview 飽和後要寫單一功能深度文章、需要選題與結構依據時回來。這套方法論的驗證來源與 cadence variant 在高風險場景（同 vendor sub-tool 系列）的實證。">Vendor 深度技術文章寫作方法論&lt;/a>。寫入限制以 &lt;a href="https://firebase.google.com/docs/firestore/best-practices">官方 best practices&lt;/a> 為準、最後檢查日 2026-06-16。&lt;/p>&lt;/blockquote>
&lt;h2 id="問題情境一個讚數欄位拖垮整條寫入">問題情境：一個讚數欄位拖垮整條寫入&lt;/h2>
&lt;p>直播平台上線一個「即時按讚數」功能：每個貼文一個 document，按讚就 &lt;code>update&lt;/code> 它的 &lt;code>likes&lt;/code> 欄位 &lt;code>+1&lt;/code>。內測沒問題，上了熱門直播——同一個貼文每秒湧入上千次按讚，寫入開始大量失敗、retry，延遲飆高，連帶其他寫入路徑被拖累。&lt;/p>
&lt;p>根因是流量全壓在&lt;strong>單一 document&lt;/strong> 上，而非流量總量超過 Firestore。Firestore 對單一 document 的持續寫入有軟上限（官方長期建議維持在每秒個位數量級、以當前文件為準），因為每次寫入要更新該 document 的所有索引、且並行寫同一 document 會觸發 contention 重試。把高頻變動的值塞進一個 document，等於替自己造一個寫入熱點。這篇處理 contention 的成因、用 distributed counter 把熱點打散的實作，以及這個手段的能力邊界。&lt;/p>
&lt;h2 id="核心概念寫入-contention-從哪來">核心概念：寫入 contention 從哪來&lt;/h2>
&lt;p>Firestore 的寫入成本不只是「寫一個值」。理解 contention 要抓三點：&lt;/p>
&lt;p>&lt;strong>每次寫入維護該 document 的所有索引&lt;/strong>。document 上有幾個被索引的欄位，一次寫入就要更新幾份索引條目。索引越多、單次寫入越重，這是寫入吞吐與索引數量綁定的根因。&lt;/p>
&lt;p>&lt;strong>並行寫同一 document 會序列化&lt;/strong>。Firestore 保證單一 document 的寫入一致性，並行的 &lt;code>+1&lt;/code> 不能各寫各的——它們競爭同一份狀態，後到的要重試。&lt;code>transaction&lt;/code> 與 &lt;code>FieldValue.increment()&lt;/code> 都受這個限制：&lt;code>increment&lt;/code> 省掉「讀-改-寫」的來回，但多個 increment 打同一 document 仍在同一個寫入熱點上排隊。&lt;/p>
&lt;p>&lt;strong>熱點是 per-document，不是 per-collection&lt;/strong>。把 1000 個貼文的讚數分在 1000 個 document，每個 document 每秒個位數寫入，完全沒問題；問題只在「單一 document 每秒上千寫入」。所以解法的方向是&lt;strong>把一個邏輯計數拆成多個物理 document&lt;/strong>。&lt;/p>
&lt;h2 id="配置distributed-counter-分片計數">配置：distributed counter 分片計數&lt;/h2>
&lt;p>distributed counter 的核心是把「一個計數」拆成 N 個 shard document，寫入時隨機挑一個 shard &lt;code>+1&lt;/code>，讀取時把所有 shard 加總。寫入壓力被分散到 N 個 document，每個 shard 的寫入頻率降為原本的 1/N。&lt;/p>
&lt;p>資料結構：在計數目標下建一個 &lt;code>shards&lt;/code> subcollection，N 個 shard document，每個存一段 partial count。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-javascript" data-lang="javascript">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="c1">// counter.js（用 Firebase Web SDK v9 modular API）
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="c1">&lt;/span>&lt;span class="kr">import&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl"> &lt;span class="nx">doc&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">collection&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">runTransaction&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">getDocs&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl"> &lt;span class="nx">writeBatch&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">increment&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span> &lt;span class="nx">from&lt;/span> &lt;span class="s1">&amp;#39;firebase/firestore&amp;#39;&lt;/span>&lt;span class="p">;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="kr">const&lt;/span> &lt;span class="nx">NUM_SHARDS&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="mi">10&lt;/span>&lt;span class="p">;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="c1">// 初始化：建立 N 個 shard、每個 count = 0
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="c1">&lt;/span>&lt;span class="kr">export&lt;/span> &lt;span class="kr">async&lt;/span> &lt;span class="kd">function&lt;/span> &lt;span class="nx">createCounter&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">db&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">counterRef&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl"> &lt;span class="kr">const&lt;/span> &lt;span class="nx">batch&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="nx">writeBatch&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">db&lt;/span>&lt;span class="p">);&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl"> &lt;span class="k">for&lt;/span> &lt;span class="p">(&lt;/span>&lt;span class="kd">let&lt;/span> &lt;span class="nx">i&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="mi">0&lt;/span>&lt;span class="p">;&lt;/span> &lt;span class="nx">i&lt;/span> &lt;span class="o">&amp;lt;&lt;/span> &lt;span class="nx">NUM_SHARDS&lt;/span>&lt;span class="p">;&lt;/span> &lt;span class="nx">i&lt;/span>&lt;span class="o">++&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl"> &lt;span class="nx">batch&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">set&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">doc&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">counterRef&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="s1">&amp;#39;shards&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nb">String&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">i&lt;/span>&lt;span class="p">)),&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nx">count&lt;/span>&lt;span class="o">:&lt;/span> &lt;span class="mi">0&lt;/span> &lt;span class="p">});&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl"> &lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl"> &lt;span class="kr">await&lt;/span> &lt;span class="nx">batch&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">commit&lt;/span>&lt;span class="p">();&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="c1">// 寫入：隨機挑一個 shard +1（用 increment 省掉 read-modify-write）
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="c1">&lt;/span>&lt;span class="kr">export&lt;/span> &lt;span class="kr">async&lt;/span> &lt;span class="kd">function&lt;/span> &lt;span class="nx">incrementCounter&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">db&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">counterRef&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl"> &lt;span class="kr">const&lt;/span> &lt;span class="nx">shardId&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="nb">Math&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">floor&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nb">Math&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">random&lt;/span>&lt;span class="p">()&lt;/span> &lt;span class="o">*&lt;/span> &lt;span class="nx">NUM_SHARDS&lt;/span>&lt;span class="p">);&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">21&lt;/span>&lt;span class="cl"> &lt;span class="kr">const&lt;/span> &lt;span class="nx">shardRef&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="nx">doc&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">counterRef&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="s1">&amp;#39;shards&amp;#39;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nb">String&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">shardId&lt;/span>&lt;span class="p">));&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">22&lt;/span>&lt;span class="cl"> &lt;span class="kr">await&lt;/span> &lt;span class="nx">setDoc&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">shardRef&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nx">count&lt;/span>&lt;span class="o">:&lt;/span> &lt;span class="nx">increment&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="mi">1&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">},&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nx">merge&lt;/span>&lt;span class="o">:&lt;/span> &lt;span class="kc">true&lt;/span> &lt;span class="p">});&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">23&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">24&lt;/span>&lt;span class="cl">
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">25&lt;/span>&lt;span class="cl">&lt;span class="c1">// 讀取：加總所有 shard
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">26&lt;/span>&lt;span class="cl">&lt;span class="c1">&lt;/span>&lt;span class="kr">export&lt;/span> &lt;span class="kr">async&lt;/span> &lt;span class="kd">function&lt;/span> &lt;span class="nx">getCount&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">db&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nx">counterRef&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">27&lt;/span>&lt;span class="cl"> &lt;span class="kr">const&lt;/span> &lt;span class="nx">snap&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="kr">await&lt;/span> &lt;span class="nx">getDocs&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">collection&lt;/span>&lt;span class="p">(&lt;/span>&lt;span class="nx">counterRef&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="s1">&amp;#39;shards&amp;#39;&lt;/span>&lt;span class="p">));&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">28&lt;/span>&lt;span class="cl"> &lt;span class="kd">let&lt;/span> &lt;span class="nx">total&lt;/span> &lt;span class="o">=&lt;/span> &lt;span class="mi">0&lt;/span>&lt;span class="p">;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">29&lt;/span>&lt;span class="cl"> &lt;span class="nx">snap&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">forEach&lt;/span>&lt;span class="p">((&lt;/span>&lt;span class="nx">s&lt;/span>&lt;span class="p">)&lt;/span> &lt;span class="p">=&amp;gt;&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nx">total&lt;/span> &lt;span class="o">+=&lt;/span> &lt;span class="nx">s&lt;/span>&lt;span class="p">.&lt;/span>&lt;span class="nx">data&lt;/span>&lt;span class="p">().&lt;/span>&lt;span class="nx">count&lt;/span>&lt;span class="p">;&lt;/span> &lt;span class="p">});&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">30&lt;/span>&lt;span class="cl"> &lt;span class="k">return&lt;/span> &lt;span class="nx">total&lt;/span>&lt;span class="p">;&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">31&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>三個設計點要展開。第一，寫入用 &lt;code>increment(1)&lt;/code> 而非 transaction 的讀-改-寫：&lt;code>increment&lt;/code> 是 atomic 的 server-side 操作，省掉一次讀取，且本身就避開了「讀到舊值再寫」的 race。第二，shard 選擇用隨機分佈，讓寫入均勻打散到 N 個 shard——這是分片有效的前提，若選 shard 有偏（例如按 user id hash 但 user 分佈不均），熱點會在某幾個 shard 復現。第三，讀取要讀 N 個 document 加總，這是分片的代價：寫入便宜了，讀取從「讀 1 筆」變成「讀 N 筆」，計費與延遲都乘以 N。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/firestore/" data-link-title="Firestore" data-link-desc="Firebase / Google Cloud 的 serverless document database、collection / document 模型、client 直連 &#43; Security Rules、realtime listener 與 offline 同步、BaaS bundle 的資料層面">Firestore</a> overview 的 deep article。寫作參照 <a href="/blog/posts/vendor-%E6%B7%B1%E5%BA%A6%E6%8A%80%E8%A1%93%E6%96%87%E7%AB%A0%E6%96%B9%E6%B3%95%E8%AB%96%E7%9A%84%E6%BC%94%E5%8C%96%E7%B4%80%E9%8C%84%E5%90%8C-vendor-%E7%B3%BB%E5%88%97%E7%9A%84%E9%96%8B%E5%A0%B4%E8%BC%AA%E6%9B%BF%E9%A9%97%E8%AD%89/" data-link-title="Vendor 深度技術文章方法論的演化紀錄：同 vendor 系列的開場輪替驗證" data-link-desc="vendor overview 飽和後要寫單一功能深度文章、需要選題與結構依據時回來。這套方法論的驗證來源與 cadence variant 在高風險場景（同 vendor sub-tool 系列）的實證。">Vendor 深度技術文章寫作方法論</a>。寫入限制以 <a href="https://firebase.google.com/docs/firestore/best-practices">官方 best practices</a> 為準、最後檢查日 2026-06-16。</p></blockquote>
<h2 id="問題情境一個讚數欄位拖垮整條寫入">問題情境：一個讚數欄位拖垮整條寫入</h2>
<p>直播平台上線一個「即時按讚數」功能：每個貼文一個 document，按讚就 <code>update</code> 它的 <code>likes</code> 欄位 <code>+1</code>。內測沒問題，上了熱門直播——同一個貼文每秒湧入上千次按讚，寫入開始大量失敗、retry，延遲飆高，連帶其他寫入路徑被拖累。</p>
<p>根因是流量全壓在<strong>單一 document</strong> 上，而非流量總量超過 Firestore。Firestore 對單一 document 的持續寫入有軟上限（官方長期建議維持在每秒個位數量級、以當前文件為準），因為每次寫入要更新該 document 的所有索引、且並行寫同一 document 會觸發 contention 重試。把高頻變動的值塞進一個 document，等於替自己造一個寫入熱點。這篇處理 contention 的成因、用 distributed counter 把熱點打散的實作，以及這個手段的能力邊界。</p>
<h2 id="核心概念寫入-contention-從哪來">核心概念：寫入 contention 從哪來</h2>
<p>Firestore 的寫入成本不只是「寫一個值」。理解 contention 要抓三點：</p>
<p><strong>每次寫入維護該 document 的所有索引</strong>。document 上有幾個被索引的欄位，一次寫入就要更新幾份索引條目。索引越多、單次寫入越重，這是寫入吞吐與索引數量綁定的根因。</p>
<p><strong>並行寫同一 document 會序列化</strong>。Firestore 保證單一 document 的寫入一致性，並行的 <code>+1</code> 不能各寫各的——它們競爭同一份狀態，後到的要重試。<code>transaction</code> 與 <code>FieldValue.increment()</code> 都受這個限制：<code>increment</code> 省掉「讀-改-寫」的來回，但多個 increment 打同一 document 仍在同一個寫入熱點上排隊。</p>
<p><strong>熱點是 per-document，不是 per-collection</strong>。把 1000 個貼文的讚數分在 1000 個 document，每個 document 每秒個位數寫入，完全沒問題；問題只在「單一 document 每秒上千寫入」。所以解法的方向是<strong>把一個邏輯計數拆成多個物理 document</strong>。</p>
<h2 id="配置distributed-counter-分片計數">配置：distributed counter 分片計數</h2>
<p>distributed counter 的核心是把「一個計數」拆成 N 個 shard document，寫入時隨機挑一個 shard <code>+1</code>，讀取時把所有 shard 加總。寫入壓力被分散到 N 個 document，每個 shard 的寫入頻率降為原本的 1/N。</p>
<p>資料結構：在計數目標下建一個 <code>shards</code> subcollection，N 個 shard document，每個存一段 partial count。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-javascript" data-lang="javascript"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1">// counter.js（用 Firebase Web SDK v9 modular API）
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="c1"></span><span class="kr">import</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">  <span class="nx">doc</span><span class="p">,</span> <span class="nx">collection</span><span class="p">,</span> <span class="nx">runTransaction</span><span class="p">,</span> <span class="nx">getDocs</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">  <span class="nx">writeBatch</span><span class="p">,</span> <span class="nx">increment</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="p">}</span> <span class="nx">from</span> <span class="s1">&#39;firebase/firestore&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="kr">const</span> <span class="nx">NUM_SHARDS</span> <span class="o">=</span> <span class="mi">10</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="c1">// 初始化：建立 N 個 shard、每個 count = 0
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="c1"></span><span class="kr">export</span> <span class="kr">async</span> <span class="kd">function</span> <span class="nx">createCounter</span><span class="p">(</span><span class="nx">db</span><span class="p">,</span> <span class="nx">counterRef</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">  <span class="kr">const</span> <span class="nx">batch</span> <span class="o">=</span> <span class="nx">writeBatch</span><span class="p">(</span><span class="nx">db</span><span class="p">);</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">  <span class="k">for</span> <span class="p">(</span><span class="kd">let</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&lt;</span> <span class="nx">NUM_SHARDS</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">    <span class="nx">batch</span><span class="p">.</span><span class="nx">set</span><span class="p">(</span><span class="nx">doc</span><span class="p">(</span><span class="nx">counterRef</span><span class="p">,</span> <span class="s1">&#39;shards&#39;</span><span class="p">,</span> <span class="nb">String</span><span class="p">(</span><span class="nx">i</span><span class="p">)),</span> <span class="p">{</span> <span class="nx">count</span><span class="o">:</span> <span class="mi">0</span> <span class="p">});</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl">  <span class="kr">await</span> <span class="nx">batch</span><span class="p">.</span><span class="nx">commit</span><span class="p">();</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">
</span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="c1">// 寫入：隨機挑一個 shard +1（用 increment 省掉 read-modify-write）
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="c1"></span><span class="kr">export</span> <span class="kr">async</span> <span class="kd">function</span> <span class="nx">incrementCounter</span><span class="p">(</span><span class="nx">db</span><span class="p">,</span> <span class="nx">counterRef</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">20</span><span class="cl">  <span class="kr">const</span> <span class="nx">shardId</span> <span class="o">=</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">floor</span><span class="p">(</span><span class="nb">Math</span><span class="p">.</span><span class="nx">random</span><span class="p">()</span> <span class="o">*</span> <span class="nx">NUM_SHARDS</span><span class="p">);</span>
</span></span><span class="line"><span class="ln">21</span><span class="cl">  <span class="kr">const</span> <span class="nx">shardRef</span> <span class="o">=</span> <span class="nx">doc</span><span class="p">(</span><span class="nx">counterRef</span><span class="p">,</span> <span class="s1">&#39;shards&#39;</span><span class="p">,</span> <span class="nb">String</span><span class="p">(</span><span class="nx">shardId</span><span class="p">));</span>
</span></span><span class="line"><span class="ln">22</span><span class="cl">  <span class="kr">await</span> <span class="nx">setDoc</span><span class="p">(</span><span class="nx">shardRef</span><span class="p">,</span> <span class="p">{</span> <span class="nx">count</span><span class="o">:</span> <span class="nx">increment</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">},</span> <span class="p">{</span> <span class="nx">merge</span><span class="o">:</span> <span class="kc">true</span> <span class="p">});</span>
</span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="ln">24</span><span class="cl">
</span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="c1">// 讀取：加總所有 shard
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="c1"></span><span class="kr">export</span> <span class="kr">async</span> <span class="kd">function</span> <span class="nx">getCount</span><span class="p">(</span><span class="nx">db</span><span class="p">,</span> <span class="nx">counterRef</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">27</span><span class="cl">  <span class="kr">const</span> <span class="nx">snap</span> <span class="o">=</span> <span class="kr">await</span> <span class="nx">getDocs</span><span class="p">(</span><span class="nx">collection</span><span class="p">(</span><span class="nx">counterRef</span><span class="p">,</span> <span class="s1">&#39;shards&#39;</span><span class="p">));</span>
</span></span><span class="line"><span class="ln">28</span><span class="cl">  <span class="kd">let</span> <span class="nx">total</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">29</span><span class="cl">  <span class="nx">snap</span><span class="p">.</span><span class="nx">forEach</span><span class="p">((</span><span class="nx">s</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="p">{</span> <span class="nx">total</span> <span class="o">+=</span> <span class="nx">s</span><span class="p">.</span><span class="nx">data</span><span class="p">().</span><span class="nx">count</span><span class="p">;</span> <span class="p">});</span>
</span></span><span class="line"><span class="ln">30</span><span class="cl">  <span class="k">return</span> <span class="nx">total</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">31</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p>三個設計點要展開。第一，寫入用 <code>increment(1)</code> 而非 transaction 的讀-改-寫：<code>increment</code> 是 atomic 的 server-side 操作，省掉一次讀取，且本身就避開了「讀到舊值再寫」的 race。第二，shard 選擇用隨機分佈，讓寫入均勻打散到 N 個 shard——這是分片有效的前提，若選 shard 有偏（例如按 user id hash 但 user 分佈不均），熱點會在某幾個 shard 復現。第三，讀取要讀 N 個 document 加總，這是分片的代價：寫入便宜了，讀取從「讀 1 筆」變成「讀 N 筆」，計費與延遲都乘以 N。</p>
<p>如果即時讀取頻率也很高（每個觀眾畫面都要顯示即時讚數），讀 N 個 shard 的成本會反過來變成瓶頸。這時把彙總值定期寫回一個 summary document，client 訂閱 summary 而非每次加總：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-javascript" data-lang="javascript"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1">// 由 Cloud Function 定時（或 onWrite 觸發 + debounce）彙總寫回 summary
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"></span><span class="kr">export</span> <span class="kr">async</span> <span class="kd">function</span> <span class="nx">aggregateToSummary</span><span class="p">(</span><span class="nx">db</span><span class="p">,</span> <span class="nx">counterRef</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">  <span class="kr">const</span> <span class="nx">total</span> <span class="o">=</span> <span class="kr">await</span> <span class="nx">getCount</span><span class="p">(</span><span class="nx">db</span><span class="p">,</span> <span class="nx">counterRef</span><span class="p">);</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl">  <span class="kr">await</span> <span class="nx">setDoc</span><span class="p">(</span><span class="nx">doc</span><span class="p">(</span><span class="nx">counterRef</span><span class="p">,</span> <span class="s1">&#39;summary&#39;</span><span class="p">,</span> <span class="s1">&#39;current&#39;</span><span class="p">),</span> <span class="p">{</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl">    <span class="nx">count</span><span class="o">:</span> <span class="nx">total</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">6</span><span class="cl">    <span class="nx">updatedAt</span><span class="o">:</span> <span class="nx">serverTimestamp</span><span class="p">(),</span>
</span></span><span class="line"><span class="ln">7</span><span class="cl">  <span class="p">});</span>
</span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p>這把「即時精確」換成「近即時」：summary 有刷新間隔的延遲，但讀取從 N 筆降回 1 筆。讚數、觀看數這類「差幾個不影響體驗」的計數，這個取捨幾乎總是對的。</p>
<h2 id="故障演練五個高頻寫入踩坑">故障演練：五個高頻寫入踩坑</h2>
<h4 id="case-1直接-increment-單一-document-沒分片">Case 1：直接 <code>increment</code> 單一 document 沒分片</h4>
<p>最常見的起手——以為 <code>FieldValue.increment()</code> 就解決了並行，忽略它仍在單一 document 的寫入熱點上。低流量沒事、熱門事件寫爆。修法：判斷該計數的峰值寫入頻率，超過單 document 軟上限就上 distributed counter；不確定峰值就先分片，分片對低流量無害（只是多讀幾筆）。</p>
<h4 id="case-2shard-數量拍腦袋定太小">Case 2：shard 數量拍腦袋定太小</h4>
<p>設了 3 個 shard，峰值流量下每個 shard 仍每秒上百寫入、照樣 contention。修法：shard 數要對齊峰值寫入頻率除以單 shard 安全寫入率（每秒個位數）。預期峰值每秒 500 寫入、單 shard 安全 5/s，就需要約 100 個 shard。寧可估高。</p>
<h4 id="case-3shard-太多拖垮讀取">Case 3：shard 太多拖垮讀取</h4>
<p>反向錯誤——為了保險設 1000 個 shard，結果每次讀計數要讀 1000 個 document，讀取計費與延遲爆炸。修法：shard 數是寫入分散與讀取成本的取捨；高寫入低讀取用多 shard + 直接加總，高寫入高讀取用多 shard + summary 彙總，別用「讀 N 筆加總」硬扛高頻讀取。</p>
<h4 id="case-4選-shard-有偏導致熱點復現">Case 4：選 shard 有偏導致熱點復現</h4>
<p>用 <code>userId</code> 的 hash 選 shard、但活躍 user 集中在少數，寫入仍打在某幾個 shard 上。修法：shard 選擇要與寫入來源無關的隨機分佈，不要綁任何可能傾斜的 key。</p>
<h4 id="case-5把分片計數當強一致餘額用">Case 5：把分片計數當強一致餘額用</h4>
<p>把 distributed counter 拿來記帳戶餘額、庫存這類需要強一致與精確讀的值。分片計數的讀取是「加總當下各 shard」，並行寫入下讀到的是近似值，不適合做扣款判斷。修法：強一致的計數（餘額、庫存、配額）不該用分片計數，也通常不該用 Firestore 的單欄位累加——這類值要走 transaction 嚴格控制、或放關聯式資料庫用 row lock，見邊界段。</p>
<h2 id="容量與觀測shard-數的估算與監控">容量與觀測：shard 數的估算與監控</h2>
<p>shard 數量的估算從峰值寫入頻率反推：<code>shard 數 ≈ 峰值每秒寫入 / 單 shard 安全寫入率</code>。單 shard 安全寫入率以官方當前的單 document 持續寫入建議為基準（個位數量級），估算時取保守值。讀取成本同步要算：每次讀計數 = N 次 document read，乘上讀取頻率與日活，這是 distributed counter 的隱性帳。</p>
<p>監控的訊號是寫入失敗率與 contention 重試。寫入大量失敗 + retry 是 contention 的直接徵兆；單一 shard 的寫入頻率若明顯高於其他 shard，是 shard 選擇有偏的徵兆。這些訊號接回 <a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">4.20 Observability Evidence Package</a>，把高頻寫入的健康度當成可觀測指標而非事故才發現。</p>
<p>容量規劃還要考慮 shard 數的可調整性：shard 數寫死在 client 程式裡，事後要加 shard 得同時改寫入與讀取邏輯、並補建新 shard document。預期會成長的計數，起步就把 shard 數設在峰值對應的量級，比事後擴容省事。</p>
<h2 id="邊界與整合什麼計數不該用分片什麼該離開-firestore">邊界與整合：什麼計數不該用分片，什麼該離開 Firestore</h2>
<p>distributed counter 解的是「高頻、可接受近似、不需強一致」的計數——讚數、觀看數、瀏覽量、即時參與人數。它的邊界很清楚：</p>
<ul>
<li><strong>需要強一致與精確的計數</strong>：帳戶餘額、庫存、配額扣減。這些要嘛用 Firestore transaction 嚴格序列化（但就回到單 document 寫入上限的限制、不適合高頻），要嘛放關聯式資料庫用 row-level lock 與交易保護（見 <a href="/blog/backend/01-database/transaction-boundary/" data-link-title="1.3 Transaction 與一致性邊界" data-link-desc="交易邊界、isolation level、retry 策略、distributed transaction（2PC、Saga）與跨 region 強一致取捨">1.3 transaction 與一致性邊界</a>）</li>
<li><strong>需要任意維度聚合的計數</strong>：要算「各地區、各時段的累計」這類多維彙總，分片計數表達不了，該把事件流寫進分析系統或關聯式資料庫做 aggregation</li>
<li><strong>計數本身是核心交易資料</strong>：當計數驅動扣款、結算這類有金錢後果的流程，把它留在 client 直連的 Firestore 是控制面風險，該移到後端——這呼應 <a href="/blog/backend/01-database/vendors/firestore/migrate-to-relational/" data-link-title="從 Firestore 遷往自建 relational：撞牆驅動的 Type E 重建模、存取模型反轉與並行期" data-link-desc="Firestore → 自建後端 &#43; relational 不是匯資料而是反轉存取模型：client 直連變 API 中介、Security Rules 授權變後端授權、document 反正規化變正規 schema、realtime listener 與 offline 同步要重建；本文走 Type E paradigm shift 結構、展開為何字面遷移不成立、哪些該遷哪些先留、dual-write &#43; shadow read 階段化與遷出代價判讀">Firestore → 自建 relational</a> 的成本與授權 driver</li>
</ul>
<p>判讀順序是先問「這個計數能不能容忍近似與最終一致」。能，distributed counter 是 Firestore 內的正解；不能，這個計數從一開始就不該用 Firestore 的單欄位累加表達。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>上層：<a href="/blog/backend/01-database/vendors/firestore/" data-link-title="Firestore" data-link-desc="Firebase / Google Cloud 的 serverless document database、collection / document 模型、client 直連 &#43; Security Rules、realtime listener 與 offline 同步、BaaS bundle 的資料層面">Firestore overview</a>（容量特性與寫入熱點）</li>
<li>一致性邊界：<a href="/blog/backend/01-database/transaction-boundary/" data-link-title="1.3 Transaction 與一致性邊界" data-link-desc="交易邊界、isolation level、retry 策略、distributed transaction（2PC、Saga）與跨 region 強一致取捨">1.3 transaction 與一致性邊界</a>（強一致計數的去處）</li>
<li>容量背景：<a href="/blog/backend/01-database/kv-document-capacity-planning/" data-link-title="1.10 KV / Document DB 容量規劃" data-link-desc="DynamoDB / Cosmos DB / Bigtable / MongoDB 等 KV / Document DB 的容量設計、partition key 取捨、capacity mode 選擇">1.10 KV / Document DB 容量規劃</a></li>
<li>觀測：<a href="/blog/backend/04-observability/observability-evidence-package/" data-link-title="4.20 Observability Evidence Package" data-link-desc="把 log、metric、trace、audit 與資料品質限制包成可交接證據">4.20 Observability Evidence Package</a>（寫入失敗率與 contention 監控）</li>
<li>官方：<a href="https://firebase.google.com/docs/firestore/best-practices">Firestore best practices</a>、<a href="https://firebase.google.com/docs/firestore/solutions/counters">Distributed counters solution</a></li>
</ul>
]]></content:encoded></item><item><title>Firestore Distributed Counter Lab</title><link>https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/hands-on/distributed-counter-lab/</link><pubDate>Tue, 16 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/hands-on/distributed-counter-lab/</guid><description>&lt;blockquote>
&lt;p>本文是 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/hands-on/" data-link-title="Firestore Hands-on 操作路線" data-link-desc="用 Firebase Emulator Suite 在本地演練 Firestore：emulator quickstart、Security Rules 單元測試、distributed counter 分片計數，全程零雲端成本、可重跑、產出可驗證 artifact">Firestore Hands-on 操作路線&lt;/a> 的 lab，實作 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/" data-link-title="Firestore 高頻寫入與 distributed counter：單 document contention 邊界與分片計數" data-link-desc="Firestore 單一 document 有持續寫入的軟上限、高頻計數寫爆 contention 是常見事故；本文展開寫入 contention 的成因、distributed counter 分片計數的實作與讀取彙總、shard 數量與讀寫成本的取捨、五個高頻寫入踩坑，以及計數需求超過分片能處理時改走外部聚合的邊界">distributed counter 高頻寫入&lt;/a> deep article 的機制。前置環境見 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/hands-on/local-emulator-quickstart/" data-link-title="Firestore Local Emulator Quickstart" data-link-desc="用 Firebase CLI 啟動 Firestore emulator、寫 firestore.rules、用 admin SDK seed 資料、跑 query baseline 與 cleanup，建立後續 Security Rules 與 distributed counter lab 共用的本地環境">Local emulator quickstart&lt;/a>。&lt;/p>&lt;/blockquote>
&lt;p>Firestore distributed counter lab 的核心責任是把「分片計數」從概念變成可觀察的寫入分佈與彙總結果。這個 lab 在 emulator 上建立 N 個 shard、隨機分片寫入大量 increment、檢查寫入是否均勻打散到各 shard、再讀取彙總驗證總和正確。&lt;/p>
&lt;p>本文的驗收標準是：你能跑出一個 sharded counter、看到 N 個 shard 各自累積了大致均勻的 partial count、彙總後等於總寫入次數，並理解 emulator 能驗什麼、不能驗什麼。&lt;/p>
&lt;h2 id="先講清楚-emulator-的邊界">先講清楚 emulator 的邊界&lt;/h2>
&lt;p>這個 lab 驗證的是&lt;strong>分片計數的機制正確性&lt;/strong>：寫入是否均勻分佈、彙總是否等於總和、讀取要讀幾個 document。它不驗證的是 &lt;strong>contention 本身&lt;/strong>——emulator 不強制 production 的單 document 持續寫入軟上限，所以「不分片會寫爆」這件事在 emulator 跑不出來。contention 是 production 的規模特性，要在雲端真實負載下才會出現。&lt;/p>
&lt;p>這個分界本身是要學的判讀：emulator 證明「分片計數做對了」，雲端負載測試才證明「不分片會撞牆」。把兩者混為一談會誤以為 emulator 全綠就代表 production 安全。&lt;/p>
&lt;h2 id="lab-環境">Lab 環境&lt;/h2>
&lt;p>沿用 &lt;a href="https://tarrragon.github.io/blog/backend/01-database/vendors/firestore/hands-on/local-emulator-quickstart/" data-link-title="Firestore Local Emulator Quickstart" data-link-desc="用 Firebase CLI 啟動 Firestore emulator、寫 firestore.rules、用 admin SDK seed 資料、跑 query baseline 與 cleanup，建立後續 Security Rules 與 distributed counter lab 共用的本地環境">quickstart&lt;/a> 的工作區與 emulator。確認 emulator 在跑（另一個 terminal）。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">&lt;span class="nb">cd&lt;/span> /tmp/firestore-lab
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">2&lt;/span>&lt;span class="cl">&lt;span class="c1"># 確認 emulator 已啟動：firebase emulators:start --only firestore --project demo-firestore-lab&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">3&lt;/span>&lt;span class="cl">&lt;span class="nb">export&lt;/span> &lt;span class="nv">FIRESTORE_EMULATOR_HOST&lt;/span>&lt;span class="o">=&lt;/span>localhost:8080&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h2 id="實作-sharded-counter">實作 sharded counter&lt;/h2>
&lt;p>counter 的核心責任是把一個邏輯計數拆成 N 個 shard document。寫入時隨機挑 shard &lt;code>increment(1)&lt;/code>，讀取時加總所有 shard。這份 script 用 admin SDK 直接對 emulator 操作。&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">cat &amp;gt; counter.js &lt;span class="s">&amp;lt;&amp;lt;&amp;#39;JS&amp;#39;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl">&lt;span class="s">const admin = require(&amp;#39;firebase-admin&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl">&lt;span class="s">admin.initializeApp({ projectId: &amp;#39;demo-firestore-lab&amp;#39; });
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl">&lt;span class="s">const db = admin.firestore();
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl">&lt;span class="s">const FieldValue = admin.firestore.FieldValue;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl">&lt;span class="s">const NUM_SHARDS = 10;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl">&lt;span class="s">const counterRef = db.collection(&amp;#39;counters&amp;#39;).doc(&amp;#39;likes&amp;#39;);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl">&lt;span class="s">async function createCounter() {
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl">&lt;span class="s"> const batch = db.batch();
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl">&lt;span class="s"> for (let i = 0; i &amp;lt; NUM_SHARDS; i++) {
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl">&lt;span class="s"> batch.set(counterRef.collection(&amp;#39;shards&amp;#39;).doc(String(i)), { count: 0 });
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl">&lt;span class="s"> }
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl">&lt;span class="s"> await batch.commit();
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl">&lt;span class="s">}
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="s">async function incrementOnce() {
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">19&lt;/span>&lt;span class="cl">&lt;span class="s"> const shardId = Math.floor(Math.random() * NUM_SHARDS);
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">20&lt;/span>&lt;span class="cl">&lt;span class="s"> await counterRef.collection(&amp;#39;shards&amp;#39;).doc(String(shardId))
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">21&lt;/span>&lt;span class="cl">&lt;span class="s"> .set({ count: FieldValue.increment(1) }, { merge: true });
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">22&lt;/span>&lt;span class="cl">&lt;span class="s">}
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">23&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">24&lt;/span>&lt;span class="cl">&lt;span class="s">async function getCount() {
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">25&lt;/span>&lt;span class="cl">&lt;span class="s"> const snap = await counterRef.collection(&amp;#39;shards&amp;#39;).get();
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">26&lt;/span>&lt;span class="cl">&lt;span class="s"> let total = 0;
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">27&lt;/span>&lt;span class="cl">&lt;span class="s"> const perShard = {};
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">28&lt;/span>&lt;span class="cl">&lt;span class="s"> snap.forEach((s) =&amp;gt; { perShard[s.id] = s.data().count; total += s.data().count; });
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">29&lt;/span>&lt;span class="cl">&lt;span class="s"> return { total, perShard };
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">30&lt;/span>&lt;span class="cl">&lt;span class="s">}
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">31&lt;/span>&lt;span class="cl">&lt;span class="s">
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">32&lt;/span>&lt;span class="cl">&lt;span class="s">module.exports = { createCounter, incrementOnce, getCount, NUM_SHARDS };
&lt;/span>&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">33&lt;/span>&lt;span class="cl">&lt;span class="s">JS&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>三個設計點對應 deep article：用 &lt;code>FieldValue.increment(1)&lt;/code> 而非讀-改-寫（避開 race）；隨機選 shard 讓寫入均勻打散；讀取要讀 N 個 shard 加總（這是分片的代價）。&lt;/p></description><content:encoded><![CDATA[<blockquote>
<p>本文是 <a href="/blog/backend/01-database/vendors/firestore/hands-on/" data-link-title="Firestore Hands-on 操作路線" data-link-desc="用 Firebase Emulator Suite 在本地演練 Firestore：emulator quickstart、Security Rules 單元測試、distributed counter 分片計數，全程零雲端成本、可重跑、產出可驗證 artifact">Firestore Hands-on 操作路線</a> 的 lab，實作 <a href="/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/" data-link-title="Firestore 高頻寫入與 distributed counter：單 document contention 邊界與分片計數" data-link-desc="Firestore 單一 document 有持續寫入的軟上限、高頻計數寫爆 contention 是常見事故；本文展開寫入 contention 的成因、distributed counter 分片計數的實作與讀取彙總、shard 數量與讀寫成本的取捨、五個高頻寫入踩坑，以及計數需求超過分片能處理時改走外部聚合的邊界">distributed counter 高頻寫入</a> deep article 的機制。前置環境見 <a href="/blog/backend/01-database/vendors/firestore/hands-on/local-emulator-quickstart/" data-link-title="Firestore Local Emulator Quickstart" data-link-desc="用 Firebase CLI 啟動 Firestore emulator、寫 firestore.rules、用 admin SDK seed 資料、跑 query baseline 與 cleanup，建立後續 Security Rules 與 distributed counter lab 共用的本地環境">Local emulator quickstart</a>。</p></blockquote>
<p>Firestore distributed counter lab 的核心責任是把「分片計數」從概念變成可觀察的寫入分佈與彙總結果。這個 lab 在 emulator 上建立 N 個 shard、隨機分片寫入大量 increment、檢查寫入是否均勻打散到各 shard、再讀取彙總驗證總和正確。</p>
<p>本文的驗收標準是：你能跑出一個 sharded counter、看到 N 個 shard 各自累積了大致均勻的 partial count、彙總後等於總寫入次數，並理解 emulator 能驗什麼、不能驗什麼。</p>
<h2 id="先講清楚-emulator-的邊界">先講清楚 emulator 的邊界</h2>
<p>這個 lab 驗證的是<strong>分片計數的機制正確性</strong>：寫入是否均勻分佈、彙總是否等於總和、讀取要讀幾個 document。它不驗證的是 <strong>contention 本身</strong>——emulator 不強制 production 的單 document 持續寫入軟上限，所以「不分片會寫爆」這件事在 emulator 跑不出來。contention 是 production 的規模特性，要在雲端真實負載下才會出現。</p>
<p>這個分界本身是要學的判讀：emulator 證明「分片計數做對了」，雲端負載測試才證明「不分片會撞牆」。把兩者混為一談會誤以為 emulator 全綠就代表 production 安全。</p>
<h2 id="lab-環境">Lab 環境</h2>
<p>沿用 <a href="/blog/backend/01-database/vendors/firestore/hands-on/local-emulator-quickstart/" data-link-title="Firestore Local Emulator Quickstart" data-link-desc="用 Firebase CLI 啟動 Firestore emulator、寫 firestore.rules、用 admin SDK seed 資料、跑 query baseline 與 cleanup，建立後續 Security Rules 與 distributed counter lab 共用的本地環境">quickstart</a> 的工作區與 emulator。確認 emulator 在跑（另一個 terminal）。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="nb">cd</span> /tmp/firestore-lab
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"># 確認 emulator 已啟動：firebase emulators:start --only firestore --project demo-firestore-lab</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="nb">export</span> <span class="nv">FIRESTORE_EMULATOR_HOST</span><span class="o">=</span>localhost:8080</span></span></code></pre></div><h2 id="實作-sharded-counter">實作 sharded counter</h2>
<p>counter 的核心責任是把一個邏輯計數拆成 N 個 shard document。寫入時隨機挑 shard <code>increment(1)</code>，讀取時加總所有 shard。這份 script 用 admin SDK 直接對 emulator 操作。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">cat &gt; counter.js <span class="s">&lt;&lt;&#39;JS&#39;
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="s">const admin = require(&#39;firebase-admin&#39;);
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s">admin.initializeApp({ projectId: &#39;demo-firestore-lab&#39; });
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">const db = admin.firestore();
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">const FieldValue = admin.firestore.FieldValue;
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">const NUM_SHARDS = 10;
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">const counterRef = db.collection(&#39;counters&#39;).doc(&#39;likes&#39;);
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">async function createCounter() {
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">  const batch = db.batch();
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">  for (let i = 0; i &lt; NUM_SHARDS; i++) {
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">    batch.set(counterRef.collection(&#39;shards&#39;).doc(String(i)), { count: 0 });
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">  }
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">  await batch.commit();
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s">async function incrementOnce() {
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s">  const shardId = Math.floor(Math.random() * NUM_SHARDS);
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s">  await counterRef.collection(&#39;shards&#39;).doc(String(shardId))
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s">    .set({ count: FieldValue.increment(1) }, { merge: true });
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="s">async function getCount() {
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="s">  const snap = await counterRef.collection(&#39;shards&#39;).get();
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="s">  let total = 0;
</span></span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="s">  const perShard = {};
</span></span></span><span class="line"><span class="ln">28</span><span class="cl"><span class="s">  snap.forEach((s) =&gt; { perShard[s.id] = s.data().count; total += s.data().count; });
</span></span></span><span class="line"><span class="ln">29</span><span class="cl"><span class="s">  return { total, perShard };
</span></span></span><span class="line"><span class="ln">30</span><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="ln">31</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">32</span><span class="cl"><span class="s">module.exports = { createCounter, incrementOnce, getCount, NUM_SHARDS };
</span></span></span><span class="line"><span class="ln">33</span><span class="cl"><span class="s">JS</span></span></span></code></pre></div><p>三個設計點對應 deep article：用 <code>FieldValue.increment(1)</code> 而非讀-改-寫（避開 race）；隨機選 shard 讓寫入均勻打散；讀取要讀 N 個 shard 加總（這是分片的代價）。</p>
<h2 id="跑寫入並觀察分佈">跑寫入並觀察分佈</h2>
<p>driver 的核心責任是製造大量 increment、然後檢查寫入是否均勻落在各 shard。均勻分佈是分片有效的前提——若 shard 選擇有偏，熱點會在某幾個 shard 復現。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln"> 1</span><span class="cl">cat &gt; run.js <span class="s">&lt;&lt;&#39;JS&#39;
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="s">const { createCounter, incrementOnce, getCount, NUM_SHARDS } = require(&#39;./counter&#39;);
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="s">const TOTAL_WRITES = 1000;
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="s">async function main() {
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="s">  await createCounter();
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="s">  console.log(`created ${NUM_SHARDS} shards`);
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="s">  // 製造 1000 次 increment
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="s">  const tasks = [];
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="s">  for (let i = 0; i &lt; TOTAL_WRITES; i++) tasks.push(incrementOnce());
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="s">  await Promise.all(tasks);
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="s">  const { total, perShard } = await getCount();
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="s">  console.log(&#39;per-shard counts:&#39;, perShard);
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="s">  console.log(`total = ${total} (expected ${TOTAL_WRITES})`);
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="s">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="s">  // 均勻度檢查：每個 shard 期望 ~100，看極差
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="s">  const counts = Object.values(perShard);
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="s">  const min = Math.min(...counts), max = Math.max(...counts);
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="s">  console.log(`min=${min} max=${max} spread=${max - min} (expected mean ~${TOTAL_WRITES / NUM_SHARDS})`);
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="s">}
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="s">main().then(() =&gt; process.exit(0));
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="s">JS</span>
</span></span><span class="line"><span class="ln">26</span><span class="cl">
</span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="nb">export</span> <span class="nv">FIRESTORE_EMULATOR_HOST</span><span class="o">=</span>localhost:8080
</span></span><span class="line"><span class="ln">28</span><span class="cl">node run.js</span></span></code></pre></div><p>預期輸出類似（實際數字每次隨機分佈而異）：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">created 10 shards
</span></span><span class="line"><span class="ln">2</span><span class="cl">per-shard counts: { &#39;0&#39;: 98, &#39;1&#39;: 105, &#39;2&#39;: 92, ... }
</span></span><span class="line"><span class="ln">3</span><span class="cl">total = 1000 (expected 1000)
</span></span><span class="line"><span class="ln">4</span><span class="cl">min=88 max=112 spread=24 (expected mean ~100)</span></span></code></pre></div><p>兩個驗收點：<code>total</code> 等於總寫入次數（彙總正確、沒有 increment 遺失），以及各 shard 的 count 大致落在均值附近（隨機分佈均勻、沒有單一 shard 吸走大部分寫入）。</p>
<h2 id="對照實驗讀取成本隨-shard-數成長">對照實驗：讀取成本隨 shard 數成長</h2>
<p>讀取的核心代價是讀 N 個 document。把 <code>NUM_SHARDS</code> 改大（例如 100）重跑，<code>getCount</code> 要讀的 document 從 10 變 100——這就是 deep article 講的「寫入便宜了、讀取乘以 N」的取捨。在 production 這直接反映成 read 計費。</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># 編輯 counter.js 把 NUM_SHARDS 改為 100、重跑 run.js</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="c1"># 觀察 per-shard counts 物件變成 100 個 key、getCount 讀取量 10x</span></span></span></code></pre></div><p>這個對照讓「shard 數是寫入分散與讀取成本的取捨」從文字變成可觀察：多 shard 寫入更分散（每 shard 更少），但讀取要加總更多筆。高寫入高讀取的場景該配 summary 彙總（deep article 的進階手段），而非無限加 shard。</p>
<h2 id="artifact-與驗收">Artifact 與驗收</h2>
<table>
  <thead>
      <tr>
          <th>Artifact</th>
          <th>來源</th>
          <th>驗收</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>counter 實作</td>
          <td><code>counter.js</code></td>
          <td><code>increment</code> 分片寫入 + 彙總讀取</td>
      </tr>
      <tr>
          <td>寫入分佈</td>
          <td><code>run.js</code> output</td>
          <td>total = 寫入次數、各 shard 均勻</td>
      </tr>
      <tr>
          <td>讀寫取捨</td>
          <td>NUM_SHARDS 對照</td>
          <td>shard 數↑ → 讀取 document 數↑</td>
      </tr>
  </tbody>
</table>
<h2 id="回到-production-判讀">回到 production 判讀</h2>
<p>emulator lab 證明了機制正確，但三個 production 判讀要回雲端確認：單 document 寫入軟上限（決定 shard 數要多少）、read 計費（決定 shard 數別太多 / 要不要 summary）、shard 選擇在真實流量下是否仍均勻。把 emulator 的機制驗證當第一道關，production 的容量與成本判讀見 <a href="/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/#%e5%ae%b9%e9%87%8f%e8%88%87%e8%a7%80%e6%b8%acshard-%e6%95%b8%e7%9a%84%e4%bc%b0%e7%ae%97%e8%88%87%e7%9b%a3%e6%8e%a7" data-link-title="Firestore 高頻寫入與 distributed counter：單 document contention 邊界與分片計數" data-link-desc="Firestore 單一 document 有持續寫入的軟上限、高頻計數寫爆 contention 是常見事故；本文展開寫入 contention 的成因、distributed counter 分片計數的實作與讀取彙總、shard 數量與讀寫成本的取捨、五個高頻寫入踩坑，以及計數需求超過分片能處理時改走外部聚合的邊界">deep article 的容量段</a>。</p>
<h2 id="cleanup">Cleanup</h2>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl"><span class="c1"># 停 emulator（Ctrl-C）或清整個工作區</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">rm -rf /tmp/firestore-lab</span></span></code></pre></div><h2 id="引用路徑">引用路徑</h2>
<ul>
<li>上游：<a href="/blog/backend/01-database/vendors/firestore/hands-on/" data-link-title="Firestore Hands-on 操作路線" data-link-desc="用 Firebase Emulator Suite 在本地演練 Firestore：emulator quickstart、Security Rules 單元測試、distributed counter 分片計數，全程零雲端成本、可重跑、產出可驗證 artifact">Firestore Hands-on 操作路線</a></li>
<li>Deep article：<a href="/blog/backend/01-database/vendors/firestore/distributed-counter-high-frequency-write/" data-link-title="Firestore 高頻寫入與 distributed counter：單 document contention 邊界與分片計數" data-link-desc="Firestore 單一 document 有持續寫入的軟上限、高頻計數寫爆 contention 是常見事故；本文展開寫入 contention 的成因、distributed counter 分片計數的實作與讀取彙總、shard 數量與讀寫成本的取捨、五個高頻寫入踩坑，以及計數需求超過分片能處理時改走外部聚合的邊界">高頻寫入與 distributed counter</a></li>
<li>一致性邊界：<a href="/blog/backend/01-database/transaction-boundary/" data-link-title="1.3 Transaction 與一致性邊界" data-link-desc="交易邊界、isolation level、retry 策略、distributed transaction（2PC、Saga）與跨 region 強一致取捨">1.3 transaction 與一致性邊界</a></li>
<li>官方：<a href="https://firebase.google.com/docs/firestore/solutions/counters">Distributed counters</a>、<a href="https://firebase.google.com/docs/firestore/best-practices">Firestore best practices</a></li>
</ul>
]]></content:encoded></item></channel></rss>