Microdeck.tools 100% 瀏覽器本地執行

本地 LLM 顯存需求計算器

New

計算語言模型在 FP16、8 位元或 4 位元與指定上下文長度下所需的 VRAM,並確認你的 GPU 是否放得下。

100% 瀏覽器本地執行 不上傳至伺服器 即時顯示結果

這個工具替您省下寶貴時間了嗎? 歡迎自願贊助,協助 Microdeck 維持免費與穩定運作 · 最低 $3 起。

底層運作原理

Model weights take parameters × bits per weight ÷ 8 bytes. The KV cache grows with context: 2 × layers × KV heads × head dimension × tokens × batch × bytes per value. The tool adds a runtime overhead of about 10% of the weights plus 0.5 GiB for CUDA context and activations, then compares the total with the VRAM you have.

詳細操作步驟

1

Pick a preset model or choose custom and enter parameters, layers, KV heads and head dimension.

2

Choose the quantization (FP16, Q8, Q6, Q5, Q4 or Q3 style) and the KV cache precision.

3

Set the context length and batch size you plan to use.

4

Enter the VRAM per GPU and the number of GPUs to see whether the model fits and how much is left.

核心功能與效能優勢

  • Presets for Llama 3.1, Mistral 7B, Qwen2.5 and Gemma 2 with their real layer and KV head counts
  • Separate lines for weights, KV cache and overhead so you see what to cut first
  • Fit check against common GPU sizes and multi-GPU setups
  • Runs locally; nothing you type is sent anywhere

運算方法、公式與技術標準

Effective bits per weight for GGUF-style quantizations are averages (Q4_K_M is about 4.85 bits, Q8_0 about 8.5). Real files differ slightly by architecture.

常見實務應用情境

  • Deciding whether a 24 GB card can run a 32B model at 4-bit with a long context.
  • Sizing a multi-GPU server before renting hardware.
  • Seeing how much memory a longer context window costs.

實作範例與邊界案例

Llama 3.1 8B KV cache

輸入
Llama 3.1 8B, 8,192 tokens, batch 1, FP16 cache
輸出
1.00 GiB of KV cache

128 KiB per token: 2 × 32 layers × 8 KV heads × 128 dims × 2 bytes.

Weights of a 70B model

輸入
70B parameters at 16 bits
輸出
140 GB of weights

Far beyond one consumer GPU, which is why 70B is usually run at 4-bit or split across cards.

使用限制與精準度說明

  • • This is an estimate. Inference engines differ in overhead, and activation memory depends on the backend and batch shape.
  • • It does not model offloading to system RAM, tensor-parallel duplication or training memory (gradients and optimizer state).

瀏覽器本地運算與隱私保障

傳統線上轉換網站會將您的文件、文字片段或影音檔案上傳至遠端伺服器。本地 LLM 顯存需求計算器 完全在您的瀏覽器分頁內運行:

於此分頁本地處理

您的檔案與文字都在裝置本機計算,不會傳送內容至任何後端伺服器。

絕不記錄操作內容

Microdeck 不會記錄或儲存您在工具內貼上的任何檔案或程式碼。

分頁記憶體即時釋放

所有暫存結果僅存在目前分頁記憶體中,重設或關閉分頁後即刻清除。

常見問題 (FAQ)

Why does the KV cache matter?

It grows linearly with context length and batch size. At very long contexts it can rival the weights, so a model that fits at 4K tokens may not fit at 128K.

Does a 4-bit model lose quality?

Usually a little. Q4_K_M and Q5 variants are widely used because the quality loss is small compared with the memory saved, but it depends on the model and task.

Can I run a model that is larger than my VRAM?

Some runtimes offload layers to system RAM, which works but is much slower. This tool only checks whether everything fits on the GPU.

探索相關微型工具

支持 Microdeck

完全自願 · $3 起 · 免註冊帳號

選擇金額 (USD):已選金額:$5

單次最低贊助金額為 $3 USD。

完全自願 · $3 起 · 將透過 Gumroad 安全結帳頁面完成。