Writing by Oney Erge

Writing by Oney Erge https://oneyerge.com/writing/ Technical articles by Oney Erge on running large language models with limited GPU memory: memory budgets, context length, streamed inference, weight placement, and speculative decoding. en-us Thu, 22 Oct 2026 17:04:43 GMT Speculative Decoding: Draft Models, Draft Length, and Trees https://oneyerge.com/writing/speculative-decoding-variants/ https://oneyerge.com/writing/speculative-decoding-variants/ Thu, 15 Oct 2026 00:00:00 GMT How draft source, draft length, and tree verification affect speculative decoding for streamed models. Speculative decoding Self-speculation Token trees How Speculative Decoding Speeds Up Inference https://oneyerge.com/writing/speculative-decoding-streamed-inference/ https://oneyerge.com/writing/speculative-decoding-streamed-inference/ Thu, 15 Oct 2026 00:00:00 GMT How speculative decoding reduces target-model passes, measured with Qwen3-14B on an 8 GB laptop GPU. Speculative decoding LLM inference Streaming How H6.5 Optimizes Weight Placement https://oneyerge.com/writing/schedule-aware-exact-weight-placement/ https://oneyerge.com/writing/schedule-aware-exact-weight-placement/ Wed, 14 Oct 2026 23:00:00 GMT How H6.5 searches for weight placements that reduce GPU waits, with measured memory and latency results. Weight placement H6.5 Why Weight Offloading Slows Inference https://oneyerge.com/writing/moving-weights-storage-to-gpu/ https://oneyerge.com/writing/moving-weights-storage-to-gpu/ Wed, 14 Oct 2026 23:00:00 GMT Storage reads, decompression, and transfers in streamed LLM inference, with measured bottlenecks. Memory hierarchy Lossless compression Prefetching How Context Length Affects GPU Memory https://oneyerge.com/writing/kv-cache-streamed-inference-consumer-gpu/ https://oneyerge.com/writing/kv-cache-streamed-inference-consumer-gpu/ Wed, 14 Oct 2026 23:00:00 GMT How the KV cache grows with context length and changes the available memory for a streamed Qwen3-14B run. KV cache LLM inference Streaming How to Run a 14B Model on an 8 GB GPU https://oneyerge.com/writing/llm-memory-requirements-consumer-gpu/ https://oneyerge.com/writing/llm-memory-requirements-consumer-gpu/ Sun, 13 Sep 2026 17:00:00 GMT A memory budget and streamed execution for running Qwen3-14B's released BF16 weights when they exceed GPU memory. GPU memory LLM inference BF16