Skip to content

BjornMelin/ai-job-scraper

Repository files navigation

Job Tracker

Job Tracker is a local-first workspace for collecting, reviewing, and tracking job opportunities. The repository is named ai-job-scraper, but the product uses the shorter Job Tracker name.

The application keeps one SQLite database and one Streamlit interface. Saved searches define every collection run. Companies appear as read-only facets after jobs are collected.

Product workflow

The interface has three top-level pages:

  • Jobs: review jobs and move them through Inbox, Saved, Applied, Interviews, or Closed
  • Searches: create repeatable searches and run each one on demand
  • Insights: inspect workflow counts, listing trends, salary data, and company facets

Each job also supports an independent starred marker and private notes. A repeated scrape updates provider-owned fields without replacing your stage, star, or notes. If a provider omits details that were collected earlier, Job Tracker keeps the richer stored values.

Architecture

Job Tracker uses one data boundary:

flowchart LR
    UI[Streamlit UI] --> Saved[Saved searches]
    Saved --> Runner[Saved-search runner]
    Runner --> JobSpy[JobSpy]
    JobSpy --> Jobs[JobService transaction]
    Jobs --> DB[(SQLite)]
    DB --> Search[Job search]
    DB --> Analytics[Insights]
    DB --> Companies[Company facets]
    Search --> UI
    Analytics --> UI
    Companies --> UI
Loading

The main dependencies are:

  • Python 3.12
  • Streamlit for the web interface
  • SQLModel and SQLAlchemy for persistence
  • Alembic for schema migrations
  • JobSpy for job-board collection

See How the application is structured for component ownership and transaction rules.

Run locally

Install uv, then clone and configure the repository:

git clone https://github.com/BjornMelin/ai-job-scraper.git
cd ai-job-scraper
uv sync --locked
cp .env.example .env

Apply the database migrations:

uv run --locked alembic upgrade head

You can add the starter saved searches:

uv run --locked ai-job-seed

Start Job Tracker:

uv run --locked ai-job-scraper

Open http://localhost:8501. Create or review a saved search under Searches, then select Run now.

Configure the runtime

The application reads settings from .env. The default database URL is sqlite:///jobs.db.

DB_URL=sqlite:///jobs.db
SCRAPER_LOG_LEVEL=INFO

Keep machine-specific configuration in .env, not in the web interface.

Run with Docker

Build and start the container:

docker compose up --build

The container serves Job Tracker on port 8501.

Verify changes

Run formatting, static checks, migrations, and tests before opening a pull request:

uv run --locked ruff format --check .
uv run --locked ruff check .
uv run --locked alembic check
uv run --locked pytest -q

Documentation

License

Job Tracker is available under the MIT License.

About

🕵️‍♂️ Privacy-focused AI job scraper, local storage, and interactive dashboard. Auto-scrapes AI/ML roles from top companies using ScrapeGraph-AI + LLM and LangGraph Agents, filters for relevance, and provides a Streamlit UI for tracking applications. Built for developers seeking AI careers.

Topics

Resources

License

Stars

39 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages