Large source’s chunks are overrides small source’s chunks

Research
1

Hello guys,

I got some serious issue in my RAG pipeline.

Here is the case,

I’ve already trained almost 10 large source urls (~1500 pages each), and then I’ve indexed one small pdf document which has hardly 5 chunks there (~3 pages pdf).

Then I’ve asked some queries in my assistant regarding that pdf content, but either it says query is out of context which is our default line when answer not found or it gives information from sources but chunks are there.

Even I’ve asked same query content from heading of the pdf it still can’t able to find that chunk.

Is there any way to define the priority of sources when user asks the query or at the time of embeddings, or any other ways?

If you guys can do any help, It would be great.

Thank you.

2

This sounds like a retrieval-ranking issue rather than an embedding-generation issue.

Since your large sources contribute significantly more chunks, they may dominate the candidate pool during similarity search. Vector search doesn’t automatically prioritize smaller or newer documents.

A few things you could try:

  1. Metadata filtering: Attach a source_id or document identifier to each chunk and filter retrieval to the relevant PDF when the user specifies it.

  2. Hybrid retrieval: Combine dense vector search with BM25 so exact heading and keyword matches aren’t missed.

  3. Reranking: Retrieve a broader set of candidates and use a cross-encoder reranker to improve relevance.

  4. Source-aware ranking: Apply a controlled source-priority score when you know which document the user wants.

  5. Check chunking and indexing: Verify that the PDF’s five chunks were successfully embedded, stored and included in the retrieval results.

One important debugging step: run the query directly against your vector database and inspect the top 10 retrieved chunks, including their source metadata and similarity scores.

That will help establish whether the PDF chunks are missing from retrieval or whether the assistant is losing them later in the RAG pipeline.

Which vector database and retrieval framework are you using?

3

Hi @bennibyx — thanks, that diagnosis matches what I’m seeing.

Quick answers to your questions:

  • Vector DB: Elasticsearch 8 (dense vectors for chunks + embeddings)
  • Retrieval: custom FastAPI RAG pipeline — parallel semantic + keyword, fused with RRF, optional bge cross-encoder rerank, plus source/document metadata on each chunk

So we already use hybrid retrieval, reranking, and source-aware metadata. Chunking/indexing looks fine too: the PDF’s ~5 chunks are embedded and present in ES.

What’s still unclear is why those chunks don’t surface for queries that match the PDF heading / content, while large crawled sources (~1500 pages × many URLs) dominate.

I’ll follow your debug step next: run the same query against ES, inspect top-10 hits (source id, scores), and check whether the PDF chunks are missing from retrieval or get dropped later (threshold / fusion / answer generation).

If you have tips for stopping large corpora from drowning a newly indexed small doc in this setup, I’d really appreciate it.

Thanks again.

4

With hybrid retrieval, reranking, and metadata already in place, the failure is probably one step earlier: if both searches take a single global top-K, the large sources can fill the candidate list before fusion runs, and a reranker can only reorder chunks that already made that list. Test it in Elasticsearch by running the heading query twice — unfiltered, and filtered to only that PDF’s source_id. If the chunks rank well filtered but vanish unfiltered, switch to per-source retrieval — a guaranteed top-N from each source, fused and reranked after — and if they miss even filtered, the heading likely isn’t in the searched or embedded text, so prepend the document title and heading path to each chunk’s indexed text and reindex just those five chunks.