<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Grep on Tarragon</title><link>https://tarrragon.github.io/blog/tags/grep/</link><description>Recent content in Grep on Tarragon</description><generator>Hugo -- gohugo.io</generator><language>zh-TW</language><copyright>Tarragon (CC BY 4.0)</copyright><lastBuildDate>Fri, 19 Jun 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://tarrragon.github.io/blog/tags/grep/index.xml" rel="self" type="application/rss+xml"/><item><title>查詢 API 設計</title><link>https://tarrragon.github.io/blog/monitoring/04-collector/query-api/</link><pubDate>Fri, 19 Jun 2026 00:00:00 +0000</pubDate><guid>https://tarrragon.github.io/blog/monitoring/04-collector/query-api/</guid><description>&lt;p>查詢是監控資料的消費介面。Collector 提供兩種查詢方式：CLI 直接操作 JSONL 檔案（grep + jq），和 HTTP 查詢 endpoint。兩種方式服務不同的消費者 — CLI 給開發者即時探索，HTTP endpoint 給自動化工具和非 CLI 使用者。&lt;/p>
&lt;h2 id="cli-查詢grep--jq">CLI 查詢：grep + jq&lt;/h2>
&lt;p>JSONL 格式的最大優勢是原生支援 Unix 文字處理工具。不需要額外的查詢語言、不需要客戶端工具、不需要連線到 database。&lt;/p>
&lt;h3 id="常見查詢模式">常見查詢模式&lt;/h3>
&lt;p>按事件類型過濾：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">grep &lt;span class="s1">&amp;#39;&amp;#34;type&amp;#34;:&amp;#34;error&amp;#34;&amp;#39;&lt;/span> events-2026-06-19.jsonl &lt;span class="p">|&lt;/span> jq .&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>按 namespace 過濾：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">grep &lt;span class="s1">&amp;#39;&amp;#34;name&amp;#34;:&amp;#34;terminal.connect&amp;#39;&lt;/span> events-2026-06-19.jsonl &lt;span class="p">|&lt;/span> jq .&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>按時間範圍過濾（跨檔案）：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">cat events-2026-06-1&lt;span class="o">{&lt;/span>8,9&lt;span class="o">}&lt;/span>.jsonl &lt;span class="p">|&lt;/span> jq &lt;span class="s1">&amp;#39;select(.ts &amp;gt;= &amp;#34;2026-06-18T18:00:00&amp;#34;)&amp;#39;&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>統計每種事件的數量：&lt;/p>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-bash" data-lang="bash">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">jq -r &lt;span class="s1">&amp;#39;.name&amp;#39;&lt;/span> events-2026-06-19.jsonl &lt;span class="p">|&lt;/span> sort &lt;span class="p">|&lt;/span> uniq -c &lt;span class="p">|&lt;/span> sort -rn&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;h3 id="grep-友好的-jsonl-設計">grep 友好的 JSONL 設計&lt;/h3>
&lt;p>JSONL 的每行 JSON 結構影響 grep 的查詢效率和準確性。&lt;/p>
&lt;p>&lt;strong>把常用過濾欄位放在 JSON 的前面&lt;/strong>。grep 是字串匹配，把 &lt;code>type&lt;/code> 和 &lt;code>name&lt;/code> 放在行首讓 grep pattern 更簡單、誤匹配更少。&lt;/p>
&lt;p>&lt;strong>避免 JSON 值中包含雙引號&lt;/strong>。事件名稱和型別用簡單字串（不含特殊字元），讓 grep 的 pattern 不需要處理 escape。&lt;/p>
&lt;p>&lt;strong>每行 JSON 不換行&lt;/strong>。JSONL 的定義就是每行一個 JSON，但格式化工具可能自動加換行。寫入時用 &lt;code>json.Marshal&lt;/code>（Go）或 &lt;code>JSON.stringify&lt;/code>（JS）確保單行輸出。&lt;/p>
&lt;h2 id="http-查詢-endpoint">HTTP 查詢 endpoint&lt;/h2>
&lt;p>HTTP 查詢 endpoint 讓非 CLI 使用者（dashboard、自動化腳本、其他服務）能查詢事件資料。&lt;/p>
&lt;h3 id="endpoint-設計">Endpoint 設計&lt;/h3>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-text" data-lang="text">&lt;span class="line">&lt;span class="ln">1&lt;/span>&lt;span class="cl">GET /v1/events?type=error&amp;amp;name=terminal.connect.*&amp;amp;from=2026-06-18T00:00:00Z&amp;amp;to=2026-06-19T00:00:00Z&amp;amp;limit=100&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>查詢參數：&lt;/p>
&lt;table>
 &lt;thead>
 &lt;tr>
 &lt;th>參數&lt;/th>
 &lt;th>說明&lt;/th>
 &lt;th>預設值&lt;/th>
 &lt;/tr>
 &lt;/thead>
 &lt;tbody>
 &lt;tr>
 &lt;td>type&lt;/td>
 &lt;td>事件類型（event/error/metric/lifecycle）&lt;/td>
 &lt;td>全部&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>name&lt;/td>
 &lt;td>事件名稱（支援 &lt;code>*&lt;/code> 萬用字元）&lt;/td>
 &lt;td>全部&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>from&lt;/td>
 &lt;td>起始時間（ISO 8601）&lt;/td>
 &lt;td>24 小時前&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>to&lt;/td>
 &lt;td>結束時間（ISO 8601）&lt;/td>
 &lt;td>現在&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>limit&lt;/td>
 &lt;td>回傳筆數上限&lt;/td>
 &lt;td>100&lt;/td>
 &lt;/tr>
 &lt;tr>
 &lt;td>offset&lt;/td>
 &lt;td>分頁偏移&lt;/td>
 &lt;td>0&lt;/td>
 &lt;/tr>
 &lt;/tbody>
&lt;/table>
&lt;h3 id="回應格式">回應格式&lt;/h3>





&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-json" data-lang="json">&lt;span class="line">&lt;span class="ln"> 1&lt;/span>&lt;span class="cl">&lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 2&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;events&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="p">[&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 3&lt;/span>&lt;span class="cl"> &lt;span class="p">{&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 4&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;v&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="mi">1&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 5&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;type&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;error&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 6&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;timestamp&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;2026-06-19T08:42:00Z&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 7&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;source&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nt">&amp;#34;sdk&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;python&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nt">&amp;#34;platform&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;macos&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nt">&amp;#34;app&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;claude-hooks&amp;#34;&lt;/span> &lt;span class="p">},&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 8&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;name&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;hook.failure&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln"> 9&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;level&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;error&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">10&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;data&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nt">&amp;#34;hook&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;branch-status-reminder&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nt">&amp;#34;step&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;validation&amp;#34;&lt;/span> &lt;span class="p">},&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">11&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;error&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nt">&amp;#34;message&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;KeyError: &amp;#39;status&amp;#39;&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nt">&amp;#34;stack&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;Traceback...&amp;#34;&lt;/span>&lt;span class="p">,&lt;/span> &lt;span class="nt">&amp;#34;type&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;KeyError&amp;#34;&lt;/span> &lt;span class="p">},&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">12&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;context&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="p">{&lt;/span> &lt;span class="nt">&amp;#34;session_id&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="s2">&amp;#34;sess-abc-123&amp;#34;&lt;/span> &lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">13&lt;/span>&lt;span class="cl"> &lt;span class="p">}&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">14&lt;/span>&lt;span class="cl"> &lt;span class="p">],&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">15&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;total&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="mi">42&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">16&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;limit&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="mi">100&lt;/span>&lt;span class="p">,&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">17&lt;/span>&lt;span class="cl"> &lt;span class="nt">&amp;#34;offset&amp;#34;&lt;/span>&lt;span class="p">:&lt;/span> &lt;span class="mi">0&lt;/span>
&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="ln">18&lt;/span>&lt;span class="cl">&lt;span class="p">}&lt;/span>&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;code>events&lt;/code> 陣列按 &lt;code>timestamp&lt;/code> 降序排列。&lt;code>total&lt;/code> 是符合篩選條件的全量筆數（不受 limit 截斷），讓呼叫端計算分頁（&lt;code>total_pages = ceil(total / limit)&lt;/code>）。分頁用 offset-based（&lt;code>offset=100&lt;/code> 取第二頁），適合資料量在十萬筆以下的場景。資料量大到 offset 效能不足時，改用 cursor-based（&lt;code>after=&amp;lt;last_event_id&amp;gt;&lt;/code>），但 cursor-based 是 PostgreSQL 層的演進，SQLite 層用 offset 足夠。&lt;/p>
&lt;h3 id="實作策略">實作策略&lt;/h3>
&lt;p>HTTP 查詢 endpoint 的底層實作可以直接讀取 JSONL 檔案 — 根據 from/to 確定要讀哪些日期的檔案，逐行 parse 並過濾。這個實作在資料量小（單日萬筆以下）時足夠快。&lt;/p>
&lt;p>當查詢效能成為問題時，在 JSONL 之上加一層索引（按 type/name 建立反向索引），或演進到 SQLite 儲存（見 &lt;a href="https://tarrragon.github.io/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進&lt;/a>）。&lt;/p></description><content:encoded><![CDATA[<p>查詢是監控資料的消費介面。Collector 提供兩種查詢方式：CLI 直接操作 JSONL 檔案（grep + jq），和 HTTP 查詢 endpoint。兩種方式服務不同的消費者 — CLI 給開發者即時探索，HTTP endpoint 給自動化工具和非 CLI 使用者。</p>
<h2 id="cli-查詢grep--jq">CLI 查詢：grep + jq</h2>
<p>JSONL 格式的最大優勢是原生支援 Unix 文字處理工具。不需要額外的查詢語言、不需要客戶端工具、不需要連線到 database。</p>
<h3 id="常見查詢模式">常見查詢模式</h3>
<p>按事件類型過濾：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">grep <span class="s1">&#39;&#34;type&#34;:&#34;error&#34;&#39;</span> events-2026-06-19.jsonl <span class="p">|</span> jq .</span></span></code></pre></div><p>按 namespace 過濾：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">grep <span class="s1">&#39;&#34;name&#34;:&#34;terminal.connect&#39;</span> events-2026-06-19.jsonl <span class="p">|</span> jq .</span></span></code></pre></div><p>按時間範圍過濾（跨檔案）：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">cat events-2026-06-1<span class="o">{</span>8,9<span class="o">}</span>.jsonl <span class="p">|</span> jq <span class="s1">&#39;select(.ts &gt;= &#34;2026-06-18T18:00:00&#34;)&#39;</span></span></span></code></pre></div><p>統計每種事件的數量：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="ln">1</span><span class="cl">jq -r <span class="s1">&#39;.name&#39;</span> events-2026-06-19.jsonl <span class="p">|</span> sort <span class="p">|</span> uniq -c <span class="p">|</span> sort -rn</span></span></code></pre></div><h3 id="grep-友好的-jsonl-設計">grep 友好的 JSONL 設計</h3>
<p>JSONL 的每行 JSON 結構影響 grep 的查詢效率和準確性。</p>
<p><strong>把常用過濾欄位放在 JSON 的前面</strong>。grep 是字串匹配，把 <code>type</code> 和 <code>name</code> 放在行首讓 grep pattern 更簡單、誤匹配更少。</p>
<p><strong>避免 JSON 值中包含雙引號</strong>。事件名稱和型別用簡單字串（不含特殊字元），讓 grep 的 pattern 不需要處理 escape。</p>
<p><strong>每行 JSON 不換行</strong>。JSONL 的定義就是每行一個 JSON，但格式化工具可能自動加換行。寫入時用 <code>json.Marshal</code>（Go）或 <code>JSON.stringify</code>（JS）確保單行輸出。</p>
<h2 id="http-查詢-endpoint">HTTP 查詢 endpoint</h2>
<p>HTTP 查詢 endpoint 讓非 CLI 使用者（dashboard、自動化腳本、其他服務）能查詢事件資料。</p>
<h3 id="endpoint-設計">Endpoint 設計</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">GET /v1/events?type=error&amp;name=terminal.connect.*&amp;from=2026-06-18T00:00:00Z&amp;to=2026-06-19T00:00:00Z&amp;limit=100</span></span></code></pre></div><p>查詢參數：</p>
<table>
  <thead>
      <tr>
          <th>參數</th>
          <th>說明</th>
          <th>預設值</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>type</td>
          <td>事件類型（event/error/metric/lifecycle）</td>
          <td>全部</td>
      </tr>
      <tr>
          <td>name</td>
          <td>事件名稱（支援 <code>*</code> 萬用字元）</td>
          <td>全部</td>
      </tr>
      <tr>
          <td>from</td>
          <td>起始時間（ISO 8601）</td>
          <td>24 小時前</td>
      </tr>
      <tr>
          <td>to</td>
          <td>結束時間（ISO 8601）</td>
          <td>現在</td>
      </tr>
      <tr>
          <td>limit</td>
          <td>回傳筆數上限</td>
          <td>100</td>
      </tr>
      <tr>
          <td>offset</td>
          <td>分頁偏移</td>
          <td>0</td>
      </tr>
  </tbody>
</table>
<h3 id="回應格式">回應格式</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl">  <span class="nt">&#34;events&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl">    <span class="p">{</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl">      <span class="nt">&#34;v&#34;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl">      <span class="nt">&#34;type&#34;</span><span class="p">:</span> <span class="s2">&#34;error&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl">      <span class="nt">&#34;timestamp&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-06-19T08:42:00Z&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl">      <span class="nt">&#34;source&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;sdk&#34;</span><span class="p">:</span> <span class="s2">&#34;python&#34;</span><span class="p">,</span> <span class="nt">&#34;platform&#34;</span><span class="p">:</span> <span class="s2">&#34;macos&#34;</span><span class="p">,</span> <span class="nt">&#34;app&#34;</span><span class="p">:</span> <span class="s2">&#34;claude-hooks&#34;</span> <span class="p">},</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">      <span class="nt">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;hook.failure&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl">      <span class="nt">&#34;level&#34;</span><span class="p">:</span> <span class="s2">&#34;error&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl">      <span class="nt">&#34;data&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;hook&#34;</span><span class="p">:</span> <span class="s2">&#34;branch-status-reminder&#34;</span><span class="p">,</span> <span class="nt">&#34;step&#34;</span><span class="p">:</span> <span class="s2">&#34;validation&#34;</span> <span class="p">},</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl">      <span class="nt">&#34;error&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;message&#34;</span><span class="p">:</span> <span class="s2">&#34;KeyError: &#39;status&#39;&#34;</span><span class="p">,</span> <span class="nt">&#34;stack&#34;</span><span class="p">:</span> <span class="s2">&#34;Traceback...&#34;</span><span class="p">,</span> <span class="nt">&#34;type&#34;</span><span class="p">:</span> <span class="s2">&#34;KeyError&#34;</span> <span class="p">},</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">      <span class="nt">&#34;context&#34;</span><span class="p">:</span> <span class="p">{</span> <span class="nt">&#34;session_id&#34;</span><span class="p">:</span> <span class="s2">&#34;sess-abc-123&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl">  <span class="p">],</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl">  <span class="nt">&#34;total&#34;</span><span class="p">:</span> <span class="mi">42</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl">  <span class="nt">&#34;limit&#34;</span><span class="p">:</span> <span class="mi">100</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl">  <span class="nt">&#34;offset&#34;</span><span class="p">:</span> <span class="mi">0</span>
</span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p><code>events</code> 陣列按 <code>timestamp</code> 降序排列。<code>total</code> 是符合篩選條件的全量筆數（不受 limit 截斷），讓呼叫端計算分頁（<code>total_pages = ceil(total / limit)</code>）。分頁用 offset-based（<code>offset=100</code> 取第二頁），適合資料量在十萬筆以下的場景。資料量大到 offset 效能不足時，改用 cursor-based（<code>after=&lt;last_event_id&gt;</code>），但 cursor-based 是 PostgreSQL 層的演進，SQLite 層用 offset 足夠。</p>
<h3 id="實作策略">實作策略</h3>
<p>HTTP 查詢 endpoint 的底層實作可以直接讀取 JSONL 檔案 — 根據 from/to 確定要讀哪些日期的檔案，逐行 parse 並過濾。這個實作在資料量小（單日萬筆以下）時足夠快。</p>
<p>當查詢效能成為問題時，在 JSONL 之上加一層索引（按 type/name 建立反向索引），或演進到 SQLite 儲存（見 <a href="/blog/monitoring/04-collector/scaling-evolution/" data-link-title="規模演進" data-link-desc="可插拔 Storage Backend 架構 — SQLite 預設、PostgreSQL 觸發切換、時間序列 DB 長期演進">規模演進</a>）。</p>
<h2 id="聚合查詢">聚合查詢</h2>
<p>逐筆查詢回答「發生了什麼」，聚合查詢回答「發生了多少」。Error 調查的第一步是定位最頻繁的 error — 「哪些 error 最多」需要按 name 分群計數的聚合結果，逐筆列表在這個階段資訊量太大。</p>
<h3 id="endpoint-設計-1">Endpoint 設計</h3>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="ln">1</span><span class="cl">GET /v1/events/summary?type=error&amp;from=2026-06-18T00:00:00Z&amp;to=2026-06-19T00:00:00Z&amp;group_by=name</span></span></code></pre></div><p>回傳按 name 分群的統計：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="ln">1</span><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">  <span class="nt">&#34;groups&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">    <span class="p">{</span> <span class="nt">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;hook.failure&#34;</span><span class="p">,</span> <span class="nt">&#34;count&#34;</span><span class="p">:</span> <span class="mi">15</span><span class="p">,</span> <span class="nt">&#34;last_seen&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-06-19T08:42:00Z&#34;</span> <span class="p">},</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl">    <span class="p">{</span> <span class="nt">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;terminal.connect.failed&#34;</span><span class="p">,</span> <span class="nt">&#34;count&#34;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span> <span class="nt">&#34;last_seen&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-06-19T07:10:00Z&#34;</span> <span class="p">}</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl">  <span class="p">],</span>
</span></span><span class="line"><span class="ln">6</span><span class="cl">  <span class="nt">&#34;total&#34;</span><span class="p">:</span> <span class="mi">18</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">7</span><span class="cl">  <span class="nt">&#34;from&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-06-18T00:00:00Z&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">8</span><span class="cl">  <span class="nt">&#34;to&#34;</span><span class="p">:</span> <span class="s2">&#34;2026-06-19T00:00:00Z&#34;</span>
</span></span><span class="line"><span class="ln">9</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p>查詢參數和逐筆查詢共用（type、name、from、to），額外的 <code>group_by</code> 指定分群欄位（name 或 type）。</p>
<h3 id="sql-實作">SQL 實作</h3>
<p>SQLite backend 下直接用 GROUP BY：</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sql" data-lang="sql"><span class="line"><span class="ln">1</span><span class="cl"><span class="k">SELECT</span><span class="w"> </span><span class="n">name</span><span class="p">,</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="k">count</span><span class="p">,</span><span class="w"> </span><span class="k">MAX</span><span class="p">(</span><span class="k">timestamp</span><span class="p">)</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">last_seen</span><span class="w">
</span></span></span><span class="line"><span class="ln">2</span><span class="cl"><span class="w"></span><span class="k">FROM</span><span class="w"> </span><span class="n">events</span><span class="w">
</span></span></span><span class="line"><span class="ln">3</span><span class="cl"><span class="w"></span><span class="k">WHERE</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">&#39;error&#39;</span><span class="w"> </span><span class="k">AND</span><span class="w"> </span><span class="k">timestamp</span><span class="w"> </span><span class="k">BETWEEN</span><span class="w"> </span><span class="o">?</span><span class="w"> </span><span class="k">AND</span><span class="w"> </span><span class="o">?</span><span class="w">
</span></span></span><span class="line"><span class="ln">4</span><span class="cl"><span class="w"></span><span class="k">GROUP</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">name</span><span class="w">
</span></span></span><span class="line"><span class="ln">5</span><span class="cl"><span class="w"></span><span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="k">count</span><span class="w"> </span><span class="k">DESC</span><span class="w">
</span></span></span><span class="line"><span class="ln">6</span><span class="cl"><span class="w"></span><span class="k">LIMIT</span><span class="w"> </span><span class="mi">100</span></span></span></code></pre></div><p>有 type + timestamp 複合索引時，這個查詢在 10 萬筆資料內的效能和逐筆查詢相當 — GROUP BY 在索引掃描後做，不需要全表掃描。</p>
<h3 id="和逐筆查詢的定位差異">和逐筆查詢的定位差異</h3>
<table>
  <thead>
      <tr>
          <th>面向</th>
          <th>逐筆查詢 <code>/v1/events</code></th>
          <th>聚合查詢 <code>/v1/events/summary</code></th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>回答</td>
          <td>發生了什麼（事件列表）</td>
          <td>發生了多少（統計摘要）</td>
      </tr>
      <tr>
          <td>用途</td>
          <td>看單筆 error 的 stack trace</td>
          <td>找出最頻繁的 error</td>
      </tr>
      <tr>
          <td>回傳</td>
          <td>事件陣列（含完整 JSON）</td>
          <td>分群摘要（name + count + last_seen）</td>
      </tr>
      <tr>
          <td>資料量</td>
          <td>大（完整事件 body）</td>
          <td>小（只有統計值）</td>
      </tr>
      <tr>
          <td>典型工作流</td>
          <td>聚合查詢找到問題 name → 逐筆查詢看細節</td>
          <td>首先使用</td>
      </tr>
  </tbody>
</table>
<p>兩者是互補的工作流 — 聚合查詢定位問題方向，逐筆查詢深入細節。Dashboard 的 Error 列表頁面直接消費聚合查詢的結果。</p>
<h2 id="cli-vs-http-的定位">CLI vs HTTP 的定位</h2>
<table>
  <thead>
      <tr>
          <th>面向</th>
          <th>CLI (grep + jq)</th>
          <th>HTTP endpoint</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>使用者</td>
          <td>開發者</td>
          <td>自動化工具、dashboard</td>
      </tr>
      <tr>
          <td>適合</td>
          <td>即時探索、ad-hoc 查詢</td>
          <td>結構化查詢、程式化存取</td>
      </tr>
      <tr>
          <td>優勢</td>
          <td>零安裝、可組合</td>
          <td>遠端存取、標準化</td>
      </tr>
      <tr>
          <td>限制</td>
          <td>需要 SSH 存取 server</td>
          <td>需要 collector 啟動</td>
      </tr>
  </tbody>
</table>
<p>兩種介面共存 — CLI 用於開發者日常 debug，HTTP endpoint 用於自動化和遠端存取。兩者底層讀取同一份 JSONL 檔案，結果一致。</p>
<h2 id="下一步路由">下一步路由</h2>
<ul>
<li>JSONL 儲存的設計 → <a href="/blog/monitoring/04-collector/jsonl-storage/" data-link-title="JSONL 匯出與備份格式" data-link-desc="JSONL 作為匯出和備份格式的設計 — 人類可讀、grep 友好、SQLite 損壞時的重建來源">JSONL 儲存設計</a></li>
<li>Rule engine 的自動化處理 → <a href="/blog/monitoring/04-collector/rule-engine/" data-link-title="Rule engine 設計" data-link-desc="條件 → 動作 → 模板的三段式規則結構 — 讓 collector 從被動儲存變成主動回應">Rule engine 設計</a></li>
<li>Collector 的完整架構 → <a href="/blog/monitoring/04-collector/architecture/" data-link-title="Collector 架構" data-link-desc="HTTP endpoint → JSON Schema 驗證 → 儲存 → 查詢 → rule engine 的五段式處理鏈路">Collector 架構</a></li>
</ul>
]]></content:encoded></item></channel></rss>