Skip to content

Structlog processor#5

Merged
Polliog merged 8 commits into
logtide-dev:mainfrom
apelsynca:structlog-processor
May 26, 2026
Merged

Structlog processor#5
Polliog merged 8 commits into
logtide-dev:mainfrom
apelsynca:structlog-processor

Conversation

@apelsynca
Copy link
Copy Markdown
Contributor

Short summary:

1: changed relative imports to absolute.

2:
added LogTideProcessor based on structlog, and tests for that. (and fixed tests)
does not requires structlog to be installed.

usage example:

from logtide_sdk.structlog import LogTideProcessor

# then somewhere in code:

structlog.configure_once(processors=[
    ...,
    ...,
    LogtideProcessor(client=logtide_client, service="service_name_string")
]

apelsynca added 3 commits May 25, 2026 08:13
Also changed pyproject.toml, to use dependency groups while testing.
Also generated uv.sync file (for those who use uv)

Fixed pytest asyncio mode to a normal one
Downsides: users might see unwantanted class names (python class names)
Upsides: users can now set non JSONable objects as metadata, and be
fine.
@Polliog
Copy link
Copy Markdown
Contributor

Polliog commented May 25, 2026

@
Thanks for this! A few things before merge:

  1. Blocker: middleware/__init__.py uses wrong absolute imports — logtide_sdk.flask should be logtide_sdk.middleware.flask (same for django/fastapi/starlette). Since they are in try/except ImportError: pass, all middleware silently fail to import (__all__ becomes []).

  2. The pydantic quickfix does not fully work. Adding default=repr only fixes the size calc. _process_value leaves unknown objects untouched, and the actual send (_send_logs/_send_batch) posts with json=payload and no default=, so it still raises TypeError at flush time. I would prefer your other idea: do the repr/serialization inside LogTideProcessor so metadata is JSON-safe before it reaches the SDK. Let us drop the json.dumps(..., default=repr) change.

  3. frozenset[str] annotation in structlog.py breaks Python 3.8 (PEP 585 needs 3.9+), and we declare requires-python = ">=3.8". Use typing.FrozenSet or from __future__ import annotations.

The structlog processor itself looks good 👍
@

@apelsynca
Copy link
Copy Markdown
Contributor Author

Hi, tomorrow will try to do that.

Also i forgore to update the ci.yml (will do)

Took json encoder with inspiration to logfire,
Added pytest-mock package and shortened some tests boilerplate,

Custom json encoder (both on size calc and send)
@Polliog
Copy link
Copy Markdown
Contributor

Polliog commented May 26, 2026

drop Python 3.8 and set requires-python = ">=3.10". 3.9 is EOL too (Oct 2025), and 3.10 covers Ubuntu 22.04 LTS since we are still alpha the compat cost is zero, and it removes the functools.cache / frozenset[str] 3.8 breakage for good.

Needs: requires-python, classifiers (drop 3.8/3.9, add 3.10-3.13), and the CI matrix.

Separately, one real bug: json_encoder.py uses chain(...) in _pandas_data_frame_encoder but never imports it add from itertools import chain. Large DataFrames currently raise a NameError that gets swallowed into a repr() fallback.

@Polliog
Copy link
Copy Markdown
Contributor

Polliog commented May 26, 2026

Almost there 👍 Two things left:

  1. CI matrix is out of sync .github/workflows/ci.yml still runs ["3.8", "3.9", ...], which now contradicts requires-python = ">=3.10" (pip will refuse to install / import will fail on those runners → red CI). Drop 3.8 and 3.9, and add 3.14 to match the classifiers.

  2. chain still unimported json_encoder.py:242 uses chain(...) in _pandas_data_frame_encoder but there is no from itertools import chain. Was not touched by the version bump.

@apelsynca apelsynca force-pushed the structlog-processor branch from 89bed7b to 1198d3c Compare May 26, 2026 16:09
@Polliog
Copy link
Copy Markdown
Contributor

Polliog commented May 26, 2026

All green now thanks a lot for the thorough work on this! 🙏 The structlog processor + JSON encoder are a really nice addition, and dropping the EOL Python versions was the right call. Merging.

@Polliog Polliog merged commit 569f77c into logtide-dev:main May 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants