Scraper fails on "Shared Memories" and Group Feeds

[khanguyenwk]

Thank you so much for the awesome work, I really like it. But during the run, I encounter some minor problem.

The scraper terminates the scrolling session prematurely because it relies on the creation_time of individual posts to determine when to stop. This logic fails in two common scenarios where the visual feed order does not match the chronological creation date of the posts.

If a days_limit is set (e.g., 30 days), the scraper stops immediately upon encountering a single post date older than the limit, ignoring the fact that valid, newer posts may exist further down the page.

Scenario 1: Shared Memories / Reshared Posts
When a Page shares an old post (e.g., a "Memory" from 2 years ago), the scraper extracts the date of the original content (2023) rather than the timestamp of the share action (2025).

• Result: The scraper sees "2023", thinks it has reached the end of the requested history, and stops.
  

Scenario 2: Facebook Group Feeds (Activity-based Sorting)
Facebook Groups often default to "New Activity" or "Most Relevant" sorting, not "Newest".

• Context: If a post created 2 years ago receives a new comment today, it jumps to the top of the feed.
  

• The Failure: The scraper reads the top post, sees the creation date is "2 years ago", and triggers the stop condition immediately.
  

• Result: The scraper collects 1 post (the old bumped one) and terminates, missing all the actual new posts located below it.
  

Expected Behavior

The scraper should not hard-stop on the first post that exceeds the days_limit.

• For Shares: It should prioritize the feed_story timestamp over the attached_story timestamp.
  

• For Groups/Feeds: It should likely implement a "threshold" or "patience" mechanism (e.g., only stop after seeing X consecutive posts older than the limit) to account for non-chronological feeds.
  

[FaustRen]

Thanks for the detailed report — this is very solid feedback.

You're correct that the current stopping logic relies too heavily on creation_time, which breaks in cases where the visual feed order and chronological timestamps diverge (shared memories, group activity sorting, bumped posts, etc.).

I’ll try to update the scraper with:

Timestamp priority fix
For shared posts, use the feed-level feed_story.timestamp instead of the attached/original post timestamp.

Non-chronological feed handling
Replace the hard stop with a tolerance-based mechanism (e.g., stop only after N consecutive posts older than days_limit) to avoid premature termination in groups or activity-sorted feeds.

This should greatly improve accuracy and prevent early termination. I’ll push an update once the new logic is implemented.