Wizardry Labs
Wizardry Labs

Wizardry Labs / field notes

Ideas from the systems we build.

Technical notes on models, runtimes, data systems, and the product decisions that make AI useful.

Filter by topic:
AILLMOpsInferenceApple Silicon

KV-cache management: keeping long-context inference within a fixed memory budget

A practical look at active KV-cache limits, paging, and the tradeoffs behind longer conversations on constrained hardware.

Research note · 7 min read

AIInferenceMetalApple Silicon

Layer streaming on Apple Silicon: scheduling around memory pressure

Model weights do not all need to be equally hot. Layer-aware prefetching and eviction can give a local runtime more room to work.

Research note · 6 min read

AIInferenceModelsInfrastructure

GGUF as a runtime contract: parsing model metadata before compute

Before a local model can run, the runtime needs to understand its tensors, metadata, alignment, quantization, and layer boundaries.

Engineering note · 5 min read

RAGData platformsSearchAI

RAG starts before retrieval: building the data layer first

Grounded answers depend on the quality, freshness, and shape of the information that enters the index.

Field note · 6 min read

macOSComputer visionAI productsSwift

AI in the notch: why the interface is part of the model experience

A capable assistant becomes more useful when it is placed where the user already works.

Product note · 5 min read

ModelsEvaluationFine-tuningAI

Evaluation before fine-tuning: make adaptation measurable

Fine-tuning is a tool, not a default. A clear evaluation loop helps teams decide whether adaptation is needed and whether it helped.

Practice note · 6 min read