WASM build can't read files — route ReadStat I/O through DuckDB's FileSystem

[rustyconover]

Hi! 👋 Thanks for the read-stat extension — ReadStat-backed access to SAS/SPSS/Stata files from DuckDB is really useful.

I was trying it under the DuckDB-WASM build and the file reads fail there. The reason is that the file path is handed straight to ReadStat, which opens it with its own (host OS) file I/O:

• src/duckdb_read_stat.c — readstat_parse_sas7bdat(parser, path, data); (and the _xport / _sav / _por / _dta siblings)

ReadStat's default I/O uses raw POSIX file calls, which don't exist in DuckDB-WASM — so the parse fails. It also means s3:// / https:// / DuckDB-registered paths can't be read even on native builds.

Suggested fix — feed ReadStat through DuckDB's FileSystem

ReadStat is nicely pluggable here: it lets you replace its I/O with custom handlers via

readstat_set_open_handler(parser, my_open);
readstat_set_close_handler(parser, my_close);
readstat_set_read_handler(parser, my_read);
readstat_set_seek_handler(parser, my_seek);
readstat_set_io_ctx(parser, ctx);   // carry a DuckDB FileHandle here

Backing those handlers with a DuckDB FileHandle (FileSystem::GetFileSystem(context).OpenFile(...), then Read / Seek / GetFileSize) routes all of ReadStat's reads through DuckDB's virtual filesystem. That makes the extension work under WASM, and on native it transparently gains httpfs (S3/HTTPS) and registered-file support.

It's a touch more involved than a one-line swap since it's the C handler API, but it's exactly what those hooks are for. Happy to help sketch a PR if that'd be welcome. Thanks for the extension! 🙏

[dylanmeysmans]

Hi! 👋 Thanks for the read-stat extension — ReadStat-backed access to SAS/SPSS/Stata files from DuckDB is really useful.

I was trying it under the DuckDB-WASM build and the file reads fail there. The reason is that the file path is handed straight to ReadStat, which opens it with its own (host OS) file I/O:

* `src/duckdb_read_stat.c` — `readstat_parse_sas7bdat(parser, path, data);` (and the `_xport` / `_sav` / `_por` / `_dta` siblings)

ReadStat's default I/O uses raw POSIX file calls, which don't exist in DuckDB-WASM — so the parse fails. It also means s3:// / https:// / DuckDB-registered paths can't be read even on native builds.

Suggested fix — feed ReadStat through DuckDB's FileSystem

ReadStat is nicely pluggable here: it lets you replace its I/O with custom handlers via

readstat_set_open_handler(parser, my_open);
readstat_set_close_handler(parser, my_close);
readstat_set_read_handler(parser, my_read);
readstat_set_seek_handler(parser, my_seek);
readstat_set_io_ctx(parser, ctx); // carry a DuckDB FileHandle here

Backing those handlers with a DuckDB FileHandle (FileSystem::GetFileSystem(context).OpenFile(...), then Read / Seek / GetFileSize) routes all of ReadStat's reads through DuckDB's virtual filesystem. That makes the extension work under WASM, and on native it transparently gains httpfs (S3/HTTPS) and registered-file support.

It's a touch more involved than a one-line swap since it's the C handler API, but it's exactly what those hooks are for. Happy to help sketch a PR if that'd be welcome. Thanks for the extension! 🙏

@rustyconover The functionality you describe is not part of DuckDB's stable C API, so the question becomes whether it is worth it to break backwards compatibility to read SAS, SPSS, and Stata files on a WASM build or over HTTP(S) or S3. As these are file formats from niche software, I imagine this use case never occurs. If you can point me at an instance of it, I would be happy to consider moving to the unstable C API.

[rustyconover]

Makes sense to me, do you want to disable the WASM platform builds then?