GitHub Repo
LangChain Community Reports a Chunk-Positioning Bug; Repeated Short Phrases May Point RAG Citations to the Wrong Location
A LangChain community report on October 10 said that, with source-position metadata enabled, repeated short phrases may be assigned the position of an earlier chunk without raising an exception. The case was reproduced in a released package and a development branch, but a fix remains under discussion.
On October 10, the LangChain community reported a document-chunk positioning issue: when RecursiveCharacterTextSplitter is used with add_start_index=True, later chunks may receive the character position of an earlier occurrence if the source text contains repeated short phrases. The reporter said the issue could be reproduced with langchain-text-splitters 1.1.3 and the then-current master branch. At the time of review, the issue remained open, with no linked patch branch or PR. Community report
The official API defines add_start_index as an option to add each chunk’s starting position to its metadata; it is disabled by default. chunk_overlap controls the number of overlapping characters between adjacent chunks. These positions let applications map retrieved passages back to the source text, so position itself is an output of the document-processing pipeline that needs validation, alongside the passage content. Official API documentation
The minimal reproduction sets the chunk size limit to 30 and the overlap to 20, with the same short phrase appearing twice in succession in the source text. The expected start positions for the three chunks are 0, 22, and 44, but the actual positions are 0, 0, and 44. No exception is raised. The report attributes this to a string search performed after chunking: the code estimates a search start from “the previous chunk’s start position + its length − the overlap,” then uses str.find to recover the position. A short passage can cause the search range to reach back into earlier content and match the previous copy again. Reproduction and root-cause analysis
For RAG engineering, this could cause citation highlighting, source-text navigation, or retrieval of nearby passages to point to the wrong region. This risk is inferred from how the position field is used; the report did not measure answer quality. Because the chunk text is identical, comparing content alone may not detect the misalignment.
The reporter recommends preserving offsets during splitting and merging instead of searching for them afterward. They also note that forcing start positions to increase is unsafe, since different chunks may legitimately start at the same position. Teams can begin by adding cases with repeated short phrases, whitespace cleanup, and overlap boundaries, then checking each chunk’s actual position in the source text. Next, they should track upstream regression tests and release versions before assessing whether existing indexes need to be rebuilt. Maintainer confirmation and a full assessment of the impact are still pending. Fix discussion