Configure Chunk Rules
Chunk settings affect retrieval accuracy, answer context, and processing cost. When EZChat creates a dataset, it submits chunk settings and uses them to split document content into searchable chunks.
Current Default Settings
The current new dataset flow uses these default values:
| Setting | Default | Description |
|---|---|---|
| Maximum chunk length | 1024 | Maximum length of each chunk. |
| Chunk overlap | 50 | Overlapping content kept between adjacent chunks. |
| Delimiter | \n\n | Splits content by paragraph by default. |
| Keep separators | true | Keeps separators inside generated chunks. |
| Language hint | Follows the current UI language | Chinese UI uses chinese; other cases use english. |
TIP
Chunk overlap must be smaller than chunk size. The default values are a good starting point. If document meaning often spans across paragraphs, consider increasing overlap moderately.
Field Details
Delimiter
Delimiter is the character used to split text. Common values include:
| Value | Usage |
|---|---|
\n\n | Split by paragraphs. |
\n | Split by lines. |
*** | Use a custom separator. |
Maximum Chunk Length
Controls the maximum length of one chunk. If it is too small, the chunk may not contain enough context. If it is too large, retrieval cost increases and unrelated content may be included in answers.
Chunk Overlap
Controls how much overlapping content is retained between adjacent chunks. A moderate overlap helps preserve semantic continuity, especially for FAQs, tutorials, or documents that require surrounding context.
Keep Separators
When enabled, separators are kept inside chunks. This helps preserve paragraph structure. If a document contains many meaningless separators, it may reduce chunk quality.
When to Adjust
- Documents have very short paragraphs: lower chunk size to reduce unrelated content in retrieval.
- Documents have very long paragraphs: increase chunk size to avoid splitting sentences or sections too aggressively.
- Answers often miss context: increase chunk overlap.
- Documents use explicit markers such as
###or---: consider using a custom delimiter.
Screenshot pending: the current production UI does not expose a chunk setting page. After the UI is available, capture the settings area that includes Delimiter, Maximum chunk length, Chunk overlap, and Keep separators.