Fix CSV iterator test to not use std::max_element

Author: vincentlaucsb

.claude/rules/csv_reader_rules.md

@@ -0,0 +1,39 @@
+---
+paths:
+  - "include/internal/csv_reader.hpp"
+  - "include/internal/csv_reader.cpp"
+  - "include/internal/csv_reader_iterator.cpp"
+---
+
+# CSVReader Iterator Rules
+
+## CRITICAL CONSTRAINT: DO NOT Cache All RawCSVDataPtr Chunks
+
+### The Streaming Architecture
+- Library parses 50+ GB CSV files with bounded memory (e.g., 8GB RAM)
+- Previous data chunks are freed as iterator advances
+- `CSVReader::iterator` is `std::input_iterator_tag` by design (single-pass)
+
+### When You See Heap-Use-After-Free with std::max_element or ForwardIterator Algorithms
+
+**❌ WRONG FIX (defeats streaming)**:
+```cpp
+class iterator {
+    std::vector<RawCSVDataPtr> _all_data_chunks;  // NO!
+    // Result: 50GB CSV requires 50GB RAM
+};
+```
+
+**✅ CORRECT FIXES**:
+1. **Document limitation**: ForwardIterator algorithms not supported on CSVReader::iterator
+2. **Fix test**: Copy to vector first: `std::vector<CSVRow> rows(reader.begin(), reader.end());`
+3. **Update README**: Show vector-based workaround for std::max_element
+
+### Valid RawCSVDataPtr Usage
+- ✅ `CSVRow::iterator` CAN have RawCSVDataPtr member (single row scope, negligible memory)
+- ❌ `CSVReader::iterator` CANNOT cache all chunks (file scope, unbounded memory)
+
+### Root Cause Analysis
+- Small test files (< chunk size) fit in one chunk → ForwardIterator algorithms appear to work
+- Large files span multiple chunks → heap-use-after-free when algorithm accesses freed chunk
+- Solution: Document as unsupported OR provide vector workaround, NOT cache all chunks

.github/workflows/sanitizers.yml

@@ -51,8 +51,8 @@ jobs:
           -DCSV_CXX_STANDARD=${{ matrix.cxx_standard }} \
           -DCMAKE_CXX_COMPILER=g++ \
           -DCMAKE_C_COMPILER=gcc \
-          -DCMAKE_CXX_FLAGS="${{ matrix.flag }} -g" \
-          -DCMAKE_C_FLAGS="${{ matrix.flag }} -g"
+          -DCMAKE_CXX_FLAGS="${{ matrix.flag }} -g -fno-coverage" \
+          -DCMAKE_C_FLAGS="${{ matrix.flag }} -g -fno-coverage"
     
     - name: Build with ${{ matrix.name }}
       run: cmake --build build --config Debug
@@ -92,8 +92,8 @@ jobs:
           -DCSV_CXX_STANDARD=17 \
           -DCMAKE_CXX_COMPILER=g++ \
           -DCMAKE_C_COMPILER=gcc \
-          -DCMAKE_CXX_FLAGS="-g -O1" \
-          -DCMAKE_C_FLAGS="-g -O1"
+          -DCMAKE_CXX_FLAGS="-g -O1 -fno-coverage" \
+          -DCMAKE_C_FLAGS="-g -O1 -fno-coverage"
     
     - name: Build
       run: cmake --build build --config Debug

README.md

@@ -147,17 +147,31 @@ However, `std::ifstream` may also be used as well as in-memory sources via `std:
 **Note**: Currently CSV guessing only works for memory-mapped files. The CSV dialect
 must be manually defined for other sources.
 
-**Note on Iterator Type**: `CSVReader::iterator` is an **input iterator**, not a forward iterator.
-This design enables streaming large CSV files without loading them entirely into memory.
-While some algorithms requiring forward iterators (e.g., `std::max_element`) may work in practice,
-only input iterator algorithms are guaranteed to work reliably. If you need guaranteed forward-iterator
-or random-access support, collect rows into a container first:
+**⚠️ IMPORTANT - Iterator Type and Memory Safety**:  
+`CSVReader::iterator` is an **input iterator** (`std::input_iterator_tag`), NOT a forward iterator.
+This design enables streaming large CSV files (50+ GB) without loading them entirely into memory.
+
+**Why Forward Iterator Algorithms Don't Work**:
+- As the iterator advances, underlying data chunks are automatically freed to bound memory usage
+- Algorithms like `std::max_element` require ForwardIterator semantics (multi-pass, hold multiple positions)
+- Using such algorithms directly on `CSVReader::iterator` will cause **heap-use-after-free** when the
+  algorithm tries to access iterators pointing to already-freed data chunks
+- While it may appear to work with small files that fit in a single chunk, it WILL fail with larger files
+
+**✅ Correct Approach for ForwardIterator Algorithms**:
 ```cpp
-std::vector<CSVRow> rows;
-for (auto& row : reader) rows.push_back(row);
-// Now use std::max_element or other forward-iterator algorithms reliably
+// Copy rows to vector first (enables multi-pass iteration)
+CSVReader reader("large_file.csv");
+std::vector<CSVRow> rows(reader.begin(), reader.end());
+
+// Now safely use any algorithm requiring ForwardIterator
+auto max_row = std::max_element(rows.begin(), rows.end(), 
+    [](const CSVRow& a, const CSVRow& b) { 
+        return a["salary"].get<double>() < b["salary"].get<double>(); 
+    });
 ```
 
+
 ```cpp
 CSVFormat format;
 // custom formatting options go here

include/internal/csv_reader.hpp

@@ -85,6 +85,34 @@ namespace csv {
          *
          * @par Using with `<algorithm>` library
          * @snippet tests/test_csv_iterator.cpp CSVReader Iterator 2
+         * 
+         * @warning STREAMING CONSTRAINT - DO NOT ATTEMPT TO CACHE ALL DATA
+         * This iterator is intentionally std::input_iterator_tag (single-pass) to support
+         * streaming large CSV files that may exceed available RAM (e.g., 50+ GB files).
+         * 
+         * @par CRITICAL DESIGN CONSTRAINT:
+         * - The underlying RawCSVData chunks are automatically freed as the iterator advances
+         * - Only the CURRENT row's data is kept alive (via CSVRow's RawCSVDataPtr member)
+         * - Previous chunks are freed to allow unlimited file sizes with bounded memory
+         * 
+         * @par WHY FORWARD ITERATOR IS NOT POSSIBLE:
+         * - ForwardIterator requires multi-pass guarantees (can hold multiple valid positions)
+         * - Supporting this would require caching ALL RawCSVDataPtr chunks in the iterator
+         * - This defeats the streaming purpose: a 50 GB CSV would require 50+ GB of RAM
+         * - The entire library design depends on automatic chunk cleanup for memory efficiency
+         * 
+         * @par IMPLICATIONS FOR ALGORITHM USE:
+         * - Algorithms requiring ForwardIterator (std::max_element, std::sort, etc.) may
+         *   appear to work in tests with small files, but will cause heap-use-after-free
+         *   when the algorithm tries to access iterators pointing to freed chunks
+         * - CORRECT approach: Copy rows to std::vector first, then use algorithms
+         * - Example: auto rows = std::vector<CSVRow>(reader.begin(), reader.end());
+         *            auto max_row = std::max_element(rows.begin(), rows.end(), ...);
+         * 
+         * @par FOR AI AGENTS AND MAINTAINERS:
+         * If you encounter heap-use-after-free with ForwardIterator algorithms, the solution
+         * is NOT to cache all RawCSVDataPtr chunks - this violates the fundamental streaming
+         * architecture. Instead, document the limitation or provide vector-based workarounds.
          */
         class iterator {
         public:

tests/test_csv_iterator.cpp

@@ -107,31 +107,48 @@ TEST_CASE("Basic CSVReader Iterator Test", "[read_ints_iter]") {
 //! [CSVReader Iterator 1]
 
 //! [CSVReader Iterator 2]
-TEST_CASE("CSVReader Iterator + std::max_elem", "[iter_max_elem]") {
-    SECTION("Find Max Element with std::max_element") {
+/**
+ * IMPORTANT: CSVReader::iterator is std::input_iterator_tag (single-pass)
+ * to support streaming large files with bounded memory usage.
+ * 
+ * Algorithms requiring ForwardIterator (like std::max_element) may appear
+ * to work with small files, but cause heap-use-after-free when data spans
+ * multiple RawCSVData chunks that get freed as the iterator advances.
+ * 
+ * CORRECT approach: Copy to vector first, then use algorithms.
+ */
+TEST_CASE("CSVReader Iterator + Algorithms Requiring ForwardIterator", "[iter_algorithms]") {
+    SECTION("std::max_element - CORRECT approach using vector") {
         // The first is such that each value in the ith row is the number i
         // There are 100 rows
-        CSVReader r1("./tests/data/fake_data/ints.csv");
+        CSVReader reader("./tests/data/fake_data/ints.csv");
+
+        // Copy rows to vector to enable ForwardIterator algorithms
+        auto rows = std::vector<CSVRow>(reader.begin(), reader.end());
+        REQUIRE(rows.size() == 100);
 
         // Find largest number
-        auto int_finder = [](CSVRow& left, CSVRow& right) {
+        auto int_finder = [](const CSVRow& left, const CSVRow& right) {
             return (left["A"].get<int>() < right["A"].get<int>());
         };
 
-        auto max_int = std::max_element(r1.begin(), r1.end(), int_finder);
+        auto max_int = std::max_element(rows.begin(), rows.end(), int_finder);
         REQUIRE((*max_int)["A"] == 100);
     }
 
-    SECTION ("Find Max Element with std::max_element - Large File") {
+    SECTION("std::max_element - Large File using vector") {
         // The second file is a database of California state employee salaries
-        CSVReader r2("./tests/data/real_data/2015_StateDepartment.csv");
+        CSVReader reader("./tests/data/real_data/2015_StateDepartment.csv");
+        
+        // Copy rows to vector to enable ForwardIterator algorithms
+        auto rows = std::vector<CSVRow>(reader.begin(), reader.end());
 
         // Find highest salary
-        auto wage_finder = [](CSVRow& left, CSVRow& right) {
+        auto wage_finder = [](const CSVRow& left, const CSVRow& right) {
             return (left["Total Wages"].get<double>() < right["Total Wages"].get<double>());
         };
 
-        auto max_wage = std::max_element(r2.begin(), r2.end(), wage_finder);
+        auto max_wage = std::max_element(rows.begin(), rows.end(), wage_finder);
 
         REQUIRE((*max_wage)["Total Wages"] == "812064.87");
     }