CSV Parser 4.0.0 - Benchmarks, DataFrame gets parallelized, MSVC AV2 Fixes (#309)

* Add benchmarks + MSVC AXV2 fixes

* Simplify DataFrame

* JsonConverter refactor

* Fix compiler errors

* Euthanize CSVStat

* Update file2.cpp

* Make read_chunk() more efficient

* Remove DataFrame dummy key

* Clean up CSV writing duplicated logic

* Make sure DataFrameExecutor does not swallow exceptions

Fix writing edge case with zero

* Simplify DelimWriter factory functions

* Update csv_writer.hpp

* Made DataFrame edit overlay thread safe

* More unit tests

* More unit tests

* Update sanitizers.yml

Author: vincentlaucsb

.github/workflows/README.md

@@ -12,14 +12,18 @@ This directory contains GitHub Actions workflows for comprehensive memory and th
 - **UndefinedBehaviorSanitizer (UBSan)** - Catches undefined behavior: signed overflow, type mismatches
 
 **Config:**
-- Runs on Ubuntu with GCC
-- Tests C++17 and C++20 standards
+- Runs Linux sanitizer coverage on Ubuntu with GCC
+- Adds a dedicated Windows/MSVC AddressSanitizer leg on C++20
 - Debug builds for better diagnostics
 - Timeout: 20 minutes per configuration
 - Artifacts: Upload logs on failure
 
 **Key Features:**
-- Matrix testing: 3 sanitizers × 2 C++ standards = 6 parallel jobs
+- Linux matrix testing:
+  - ASan on C++20
+  - TSan on C++20
+  - UBSan on C++17
+- Dedicated Windows/MSVC ASan coverage for AVX2- and codegen-sensitive issues
 - Fail-fast disabled to see all results
 - Environment variables configured for halt-on-error behavior
 
@@ -54,6 +58,16 @@ cmake -B build/asan -DCMAKE_BUILD_TYPE=Debug \
   -DCMAKE_CXX_FLAGS="-fsanitize=address -fno-omit-frame-pointer -g"
 cmake --build build/asan
 cd build/asan && ctest --output-on-failure && cd ../..
+
+# Windows / MSVC AddressSanitizer
+cmake -S . -B build/msvc-asan ^
+  -DCSV_CXX_STANDARD=20 ^
+  -DCMAKE_BUILD_TYPE=RelWithDebInfo ^
+  -DCMAKE_CXX_FLAGS="/fsanitize=address /Zi" ^
+  -DCMAKE_C_FLAGS="/fsanitize=address /Zi" ^
+  -DCMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO"
+cmake --build build/msvc-asan --config RelWithDebInfo
+ctest --test-dir build/msvc-asan --build-config RelWithDebInfo --output-on-failure
 ```
 
 ### CI/CD Pipeline Order
@@ -75,6 +89,7 @@ cd build/asan && ctest --output-on-failure && cd ../..
 - **Why Important:** Catches CSVFieldList memory issues like issue #278
 - Cannot run simultaneously with TSan (different memory models)
 - Better performance than TSan for memory safety
+- The MSVC C++20 ASan leg is especially useful for AVX2- and codegen-sensitive regressions
 
 ### Valgrind
 - Slower than sanitizers but more mature tool

.github/workflows/codeql.yml

@@ -47,6 +47,9 @@ jobs:
         cmake .. \
           -DCMAKE_BUILD_TYPE=Release \
           -DCSV_CXX_STANDARD=20 \
+          -DCSV_BUILD_PROGRAMS=OFF \
+          -DCSV_BUILD_TESTS=OFF \
+          -DBUILD_PYTHON=OFF \
           -DCMAKE_CXX_COMPILER=g++ \
           -DCMAKE_C_COMPILER=gcc
     

.github/workflows/sanitizers.yml

@@ -10,22 +10,23 @@ permissions:
   contents: read
 
 jobs:
-  sanitizers:
+  linux-sanitizers:
     runs-on: ubuntu-latest
 
     strategy:
       fail-fast: false
       matrix:
-        sanitizer: [address, thread, undefined]
-        cxx_standard: [17, 20]
         include:
           - sanitizer: address
+            cxx_standard: 20
             flag: "-fsanitize=address -fno-omit-frame-pointer"
             name: "AddressSanitizer"
           - sanitizer: thread
+            cxx_standard: 20
             flag: "-fsanitize=thread"
             name: "ThreadSanitizer"
           - sanitizer: undefined
+            cxx_standard: 17
             flag: "-fsanitize=undefined -fno-omit-frame-pointer"
             name: "UndefinedBehaviorSanitizer"
 
@@ -69,9 +70,52 @@ jobs:
       if: failure()
       uses: actions/upload-artifact@v4
       with:
-        name: sanitizer-logs-${{ matrix.sanitizer }}-std${{ matrix.cxx_standard }}
+        name: linux-sanitizer-logs-${{ matrix.sanitizer }}-std${{ matrix.cxx_standard }}
         path: build/Testing/
 
+  windows-asan-msvc:
+    name: MSVC AddressSanitizer (C++20)
+    runs-on: windows-latest
+
+    env:
+      ASAN_OPTIONS: "halt_on_error=1"
+
+    steps:
+    - name: Checkout repository and submodules
+      uses: actions/checkout@v5
+      with:
+        submodules: recursive
+
+    - name: Set up MSVC developer command prompt
+      uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756
+
+    - name: Configure CMake with MSVC AddressSanitizer
+      shell: cmd
+      run: >
+        cmake -S %GITHUB_WORKSPACE%
+        -B %GITHUB_WORKSPACE%\build\msvc-asan
+        -DCSV_CXX_STANDARD=20
+        -DCMAKE_BUILD_TYPE=RelWithDebInfo
+        -DCMAKE_CXX_FLAGS="/fsanitize=address /Zi"
+        -DCMAKE_C_FLAGS="/fsanitize=address /Zi"
+        -DCMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO"
+        -DCMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO"
+
+    - name: Build with MSVC AddressSanitizer
+      run: cmake --build ${{ github.workspace }}/build/msvc-asan --config RelWithDebInfo
+
+    - name: Test with MSVC AddressSanitizer
+      working-directory: ${{ github.workspace }}/build/msvc-asan
+      run: ctest --build-config RelWithDebInfo --output-on-failure -V
+      timeout-minutes: 20
+
+    - name: Upload MSVC AddressSanitizer logs
+      if: failure()
+      uses: actions/upload-artifact@v4
+      with:
+        name: windows-msvc-asan-logs-std20
+        path: build/msvc-asan/Testing/
+
   valgrind:
     runs-on: ubuntu-latest
 

AGENTS.md

@@ -69,9 +69,12 @@ See `tests/AGENTS.md` for test strategy, checklist, and conventions.
    - **Templates must stay in `.hpp`** — the compiler needs the definition at instantiation time. `init_from_stream` is the standing example.
    - **Trivial one-liner accessors** may be unconditionally `inline` in the header when the call overhead is measurable and the body will never change.
    - **Consolidation:** If a `.cpp` would be under ~100 lines *and* the split causes excessive comment duplication between the two files, prefer a single `.hpp` with definitions marked `inline` (free functions and methods alike). Do not use `CSV_INLINE` for consolidated definitions — `CSV_INLINE` expands to empty in multi-header mode, which would produce ODR violations across TUs. Do not consolidate just for brevity — only when duplication is the dominant cost.
+7. **Prefer LF (`\n`) line endings for tracked source, test, CMake, and Markdown files.** When you touch a file with mixed line endings, normalize the edited file to LF unless there is a file-specific reason not to. Avoid introducing mixed CRLF/LF endings in the same file.
+8. **Keep preprocessor directives flush left.** `#define`, `#if`, `#ifdef`, `#else`, and `#endif` should start at column 0. Code inside multi-line macros should be indented exactly as the equivalent non-macro code would be; do not add extra indentation just because it lives inside a macro body.
 
 ### Rules for Comments
 1. **Always update or remove incorrect comments.**
 2. **Don't reference internal functions in public API comments.** Public API docs should describe user-visible behavior and contracts; internal helper/function details belong in internal docs.
 3. **Avoid meaningless @param and @return descriptions.** Do not add comments that could trivially be inferred by the function's name or other existing comments. When editing a function, remove any @param/@return descriptions that merely restate the function name or signature.
-4. **Don't delete or simplify comments** unless allowed by other rules in this section. Comments in this codebase frequently encode concurrency invariants, non-obvious design decisions, and hard-won bug context that cannot be recovered from the code alone.
+4. **Don't delete or simplify comments** unless allowed by other rules in this section. Comments in this codebase frequently encode concurrency invariants, non-obvious design decisions, and hard-won bug context that cannot be recovered from the code alone.
+5. **Public API docs belong on declarations in `.hpp` files.** When a class has both a header and implementation file, put user-facing/Doxygen documentation on the declaration in the header. Keep the `.cpp` focused on implementation notes, concurrency invariants, performance rationale, and bug-history comments.

ARCHITECTURE.md

@@ -18,8 +18,11 @@ Notes:
 - Queue synchronization details are maintained only in THREADSAFE_DEQUE_DESIGN.md to avoid duplication.
 - Always update or remove incorrect comments.
 - Public API comments should remain user-facing and avoid references to internal helper/function details.
+- Public API docs belong on declarations in `.hpp` files; keep `.cpp` comments focused on implementation notes, concurrency invariants, performance rationale, and bug history.
 - When editing a function, remove `@param` and `@return` descriptions that merely restate the function name or signature.
 - Private member naming should prefer trailing underscores; when editing mixed-style code, normalize the touched region toward that convention.
+- Prefer LF (`\n`) line endings for tracked source, test, CMake, and Markdown files; when touching a file with mixed endings, normalize it to LF unless there is a file-specific reason not to.
+- Keep preprocessor directives flush left; `#define`, `#if`, `#ifdef`, `#else`, and `#endif` should start at column 0, and code inside multi-line macros should be indented as if the macro wrapper were not present.
 - Compatibility macros defined in `common.hpp` must only be referenced after including `common.hpp`. See AGENTS.md and CLAUDE.md for details.
 - API constraints should be user-friendly: do not over-constrain templates unless needed for correctness, safety, or a measured performance win.
 - `CSVReader` is intentionally non-copyable and move-enabled; use explicit ownership transfer patterns (`std::move`, `std::unique_ptr`) at API boundaries.