Chunking rarely gets the attention embeddings and retrieval get, but it's usually the thing quietly breaking a RAG system. Cut a product description in the wrong place and the embedding for that chunk captures neither the product name nor the spec that mattered — and no embedding model, however good, can retrieve context that was never encoded to begin with.
The basic tension
Smaller chunks — in the 100 to 256 token range — give precise semantic matching, since each chunk represents one specific idea. Larger chunks, 1024 tokens and up, preserve enough surrounding context for the model to actually understand what it retrieved. Every chunking strategy is a negotiation between those two pulls, and a product catalog makes the negotiation harder because a single product page mixes short structured facts (price, size, SKU) with longer free-text description in the same document.
Two approaches, not one
Dedicated chunking library
A tool like Chonkie sits in front of your embedding call. Its token chunker splits text into fixed token-sized pieces with configurable overlap; its semantic chunker groups text by similarity instead of a fixed size. It plugs into whichever embedding provider you're using — Voyage, Cohere, OpenAI, or an open-source model — so the chunking strategy stays decoupled from the embedding choice.
Embeddings that chunk for you
Newer models like Voyage's voyage-context-4 fold chunking into the embedding step itself: send the whole document, and it returns chunk-level embeddings that already carry full-document context, with no separate chunking pipeline to build or tune.
The trade-off in practice: a dedicated chunking library gives you full control and provider flexibility, at the cost of a pipeline stage you have to maintain and re-tune as your content changes. An embedding model with built-in chunking removes that maintenance burden, but ties your chunking strategy to that specific provider's model.
What this means for a product catalog specifically
Document-aware chunking — splitting along the structure the document already has, like headings, spec tables, and description sections, rather than blindly cutting at a token count — tends to hold up best for product data, because product pages already have implicit structure worth preserving. For a catalog that changes as often as inventory does, the maintenance cost of a chunking pipeline is a real, ongoing cost — which is exactly why the built-in-chunking approach is worth serious consideration for teams that don't want chunking logic to become its own subsystem to maintain.