fix(table): row-tokenizer chunks must start at index 0 (header was left uncovered)

[lntutor]

What's broken

TableChunker.chunk has two paths. The character-tokenizer path makes the first chunk start at index 0 and include the header in its span (its tests enforce chunks[0].start_index == 0). The default row-tokenizer path instead initializes the running offset at len(header):

current_char_index = header_len   # <- first chunk starts after the header

So the first chunk's start_index is the header length instead of 0, and the header region [0, len(header)) is never covered by any chunk. original_text[start:end] then returns the data rows without the header the chunk actually contains:

table = "| A | B |\n|---|---|\n| 1 | 2 |\n| 3 | 4 |\n| 5 | 6 |\n| 7 | 8 |"
chunks = TableChunker(tokenizer="row", chunk_size=2).chunk(table)
chunks[0].start_index                 # 20  -- should be 0
"".join(table[c.start_index:c.end_index] for c in chunks) == table   # False

Fix

Start the offset at 0 and add the header length to the first chunk's span, mirroring the token path, so chunk spans are contiguous and tile the whole table from index 0.

Tests

Adds test_table_chunker_row_tokenizer_indices (first chunk starts at 0; spans contiguous; reconstruct the full table). Full tests/chunkers/test_table_chunker.py passes (40). Verified via a ~2000-table fuzz that row-path spans now reconstruct from index 0 and the character path is unchanged.

Summary by CodeRabbit

• Bug Fixes

  • Corrected table chunk span calculations for the row-tokenizer path so chunks now cover contiguous character ranges from the table’s start through the full input.
  • Improved accuracy of chunk boundaries to ensure header/prefix text is reflected in the first chunk and the last chunk absorbs any trailing content.

• Tests

  • Added new index-consistency and full-reconstruction checks for both Markdown and HTML tables (including expected chunk counts).

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 554e8744-18b8-40de-9029-30272075956d

📥 Commits

Reviewing files that changed from the base of the PR and between 2bce5b4 and 4222901.

📒 Files selected for processing (2)

• src/chonkie/chunker/table.py
• tests/chunkers/test_table_chunker.py

---

📝 Walkthrough

Walkthrough

The row-tokenizer now computes contiguous table chunk spans from index 0, includes trailing text in the final span, and validates complete reconstruction for Markdown and HTML tables.

Changes

Table chunk span fix

Layer / File(s)	Summary
Span accounting and validation src/chonkie/chunker/table.py, tests/chunkers/test_table_chunker.py	Row-tokenizer spans include the header in the first chunk, cover trailing text in the final chunk, remain contiguous, and are tested for complete Markdown and HTML table reconstruction.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: chonk-lain

Poem

I’m a bunny with a table to mend,
Spans now start where the rows begin.
Header tucked in the first neat slice,
No gaps or overlaps—very nice!
Tests hop along from start to end.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main fix: row-tokenizer table chunks now start at index 0 and include the header.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request updates the TableChunker to ensure chunk spans are contiguous and start at index 0 by adjusting the character tracking logic. A new test case is also added to verify these indices. The review feedback points out that the current implementation of span_len does not account for the footer length in the last chunk, which is especially important for HTML tables. It suggests storing the joined chunk rows in a variable to avoid redundant joins, calculating the span length to include the footer for the last chunk, and adding a corresponding test case for HTML tables.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/chonkie/chunker/table.py`:
- Around line 152-168: Update the chunk span calculations in the table chunking
loop to account for leading whitespace in the first chunk and the footer or
trailing remainder in the final chunk, so the first span begins at the table
text start and the final end_index reaches len(text). Preserve intermediate
chunk boundaries while ensuring both Markdown and HTML tables fully cover the
original input.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 60187606-1aa0-4f90-b7a3-332e318b964a

📥 Commits

Reviewing files that changed from the base of the PR and between 0a6baea and 2bce5b4.

📒 Files selected for processing (2)

• src/chonkie/chunker/table.py
• tests/chunkers/test_table_chunker.py