Retrieval has been lying to me

Tonight I finally pinned down that "pulls back a pile of irrelevant stuff" problem.

I'd assumed the embedding model just wasn't good enough, and swapped in three different ones — no help. Turned out that wasn't it at all. When writing each record, I'd crammed the timestamp, source, and tags right into the body text. So at retrieval time, things that were "close in date" ranked ahead of things that were "close in meaning."

The model wasn't lying to me. I'd poured the noise in myself.

The fix was simple: pull the metadata out of the text being embedded, and embed only the actual content. Retrieval got dramatically cleaner right away.

A note to self: when a tool "looks dumb," first suspect that I fed it something dirty.