Discord Semantic Search

This repository demonstrates how to do semantic search on Discord, relying on Pyserini for retrieval.

Prerequisites

• Python ≥ 3.10
• NVIDIA GPU with CUDA support
• Conda (recommended)
• Discord Account
• Ownership of Discord Server

Installation

1. Install Pyserini according to https://github.com/castorini/pyserini/blob/master/docs/installation.md#pypi-installation-walkthrough using Conda. (Preferably Linux)

   conda create -n pyserini python=3.11 -y
   conda activate pyserini
   
   # Inside the new environment...
   conda install -c conda-forge openjdk=21 maven -y
   
   # from https://pytorch.org/get-started/locally/ (it is important to have CUDA enabled)
   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
   conda install -c pytorch faiss-cpu -y
   
   # Good idea to always explicitly specify the latest version, found here: https://pypi.org/project/pyserini/
   pip install pyserini==1.3.0

2. Install the required packages:

   pip install -r requirements.txt
   

To reactivate environment

eval "$(/home/<username>/miniforge3/bin/conda shell.bash hook)"
conda activate pyserini

Configuration

• Set the following environment variables in the .env file:

  • CHANNEL_ID: The channel in a given Discord server that the searcher searches through
  • TOKEN: Auth token for your Discord bot
  • CACHE: True means that data is cached. False means that the data is updated each time the searcher is started up. (Default: True for the sake of demonstration)
  • TOP_K: Show the top TOP_K results (Default: 5 for the sake of demonstration)

• Create a Discord server if you already do not own a Discord Server: https://support.discord.com/hc/en-us/articles/204849977-How-do-I-create-a-server

• How To Find The Channel ID

  1. Navigate to the channel you want to copy the ID for.
  2. Right-click on the channel, and then tap on Copy Channel ID
  3. Now you can paste the channel ID into the .env file

Setup Discord bot

1. Make sure you’re logged on to the Discord website: https://discord.com
2. Navigate to https://discord.com/developers/applications
3. Create a New Application
4. Within your new application, on the left side, navigate to Settings > Bot
   1. Click on Reset Token under Token. This is important to grab this token for the TOKEN in the .env file
   2. Click on Message Content Intent to enable it
5. Navigate to Settings > OAuth2

   1. Click on the bot checkbox under the OAuth2 URL Generator > Scopes

      

   2. Then under OAuth2 URL Generator > Bot Permissions > Text Permissions, click on

      • Send Messages
      • Read Message History
      • Embed Links

      

   3. Copy the URL under Generated URL

   4. Paste the URL into a new window/tab so as to open the URL

   5. Add bot to the server set up in Configuration section above.

Usage

To run:

python main.py 

Depending on if you have allowed the cache to be reset or if this is the first time running the searcher, it may take a while before the searcher is ready; terminal should say when the searcher is ready:

Searcher is ready!

Once the searcher is running, the !search <query_text> is all you need

By default (as described in .env), the bot will return the top 5 results by semantics.

Project Structure

Discord-Semantic-Search/
├── data/               # Local document corpus
├── main.py             # Entry point
├── utils/
│   ├── config.py       # Configuration management
│   ├── bot.py          # Discord bot business logic
└── requirements.txt

The implementation can be broken down into two parts:

1. Retrieval The retrieval first grabs the chat history of a given channel and stores it locally. The business logic then creates indexes and embeddings of said chat history in a format that friendly is friendly to FAISS (Facebook AI Similarity Search). Upon a given query, it scores the chats based on semantic relevance to the query and returns the top k relevant chat messages.
2. Discord Using the Discord SDK, the bot is created. It is the entry point for accessing and reading the message history of a given Discord channel. It is also the interactive bot that accepts query strings and prints out the results of the FAISS retrieval to the Discord server.

Evaluations

The search has a NDCG@10 score of about 35.19% and precision of about 27.95%. In other words, this search does slightly worse than BM25 and other term matching methodologies. Nonetheless, heuristically, the search has helped me find results that I would otherwise not be able to do so via the built-in discord search. So practically speaking, semantic search is helpful as an additonal or supplementary approach to searching for chat rather than just being constrained to term matching search.

TODOs

Things to do if this was not just a demo, and I wanted to productionized it

• Deploy the bot in a cloud environment
• Setup a cloud vectorized database
• Enable it for more than just a singular channel search
• Use a hybrid model in an attempt to improve performance
• Rich Discord user interface
• Replace pyserini with something more performant
• Replace listdir with scandir

Video Demo

YouTube

Acknowledgement

MIT License