Skip to content

scraper takes too long to run #6

Description

@mrcnc

Currently, on my machine with a 2.2 GHz i7 Processor, it's crawling ~600 pages/minute. Since there are ~165000 parcels, it will take ~4.5 hours to complete. This is too long and it would be a lot better if we could speed up this process.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions