Thinking out loud

Blog

Notes on AI, GPUs and engineering, written to check whether I really understood something.

Learn it.Break it.Explain it.

If I can’texplain it,I don’tget it yet.

Latest entry · Oct 2026

Stop Giving Every Model Its Own GPU: Serving Embeddings, Rerankers, OCR and LLMs on One Card

Karpathy recently made the same bet at the level of the model itself, arguing that a “cognitive core” of roughly a billion parameters, distilled from a larger model, is…

Read on Medium

Oct 2026

Stop Recomputing Tokens: KV, Prefix, Prompt, and Semantic Caching Breakdown

Everything you need to understand where your input tokens are being recomputed and what to do about it. It covers the four cache layers from first principles, their…

  • caching
  • vllm
  • llm inference

Oct 2026

Keep the GPU Busy: A Hardware-First Guide to CUDA Kernels

The best way to understand and go about writing CUDA kernels would be to understand the hardware, look at how the compiler instructions work and connect them both.…

  • kernel
  • llm
  • gpu

Oct 2026

Scaling LLM Reasoning with Amazon SageMaker RLVR

Reinforcement Learning from Verifiable Rewards (RLVR) is useful when an answer can be checked automatically, but writing an ideal response is expensive.

  • fine tuning llm
  • llm
  • aws sagemaker

Oct 2026

Stop Waiting on Local LLMs: How Uzu Supercharges Inference Speed

In 2023, Karpathy explained why local LLMs are slow. When a model generates one stream of text on your own computer, the chip spends most of its time waiting for weights…

  • inference
  • optimization
  • gpu

Oct 2026

Everything an AI Engineer Needs to Know About GPUs

What is inside a GPU, how training and inference use it differently, the generations, what you actually rent, NVIDIA’s competitors, and running models locally.

  • llm
  • ai engineering
  • cuda

Oct 2026

CUDA from zero to hero #1

These are my notes as I learn CUDA; here I will try to break down things and represent them in a way that makes the most sense to me. I like to truly understand things…

  • gpu
  • inference
  • cuda

Sep 2026

Jev without Jargons

We have been using LLMs like a hammer for every AI problem, even simple decisions. Jev handles those decisions in milliseconds at a fraction of the cost. Let’s…

  • jevs
  • agentic ai
  • llm

Sep 2026

Winning the Modern AI Search Landscape

For decades, the fundamental search was relatively straightforward: crawl the web, index documents, retrieve relevant pages, rank them, and present the results to the…

  • nlp
  • retrieval augmented gen
  • vector quantization

Aug 2026

Maps, Keys, and Digital Fingerprints: Demystifying Linux Security and Safe App Installation…

Every time you install modern, secure software outside the default Ubuntu/Debian store, you follow the exact same 3-step pattern:

  • linux
  • cryptography
  • linux security

All posts on Medium