[Bug] KB deletion does not clean up orphaned asynq tasks, causing worker waste and queue starvation

[liangliwu]

Describe the bug

When a knowledge base is deleted (soft-delete), the asynq task queue is not cleaned up. Pending tasks targeting the deleted KB (e.g., wiki:ingest, question:generation, image:multimodal) remain in their respective queues and continue to be retried by workers until MaxRetry is exhausted. Each retry fails with knowledge base not found error, wasting worker capacity and polluting logs.

This is inconsistent with single-knowledge deletion (DeleteKnowledge), which correctly calls dequeueKnowledgeTasks to purge queued downstream tasks.

To reproduce

1. Create a KB with Wiki ingest enabled and some documents
2. While documents are still being processed (or have failed tasks queued), delete the KB
3. Observe the asynq queues — pending tasks for the deleted KB remain

Observed data (from a production instance):

12 soft-deleted KBs had 916 orphaned tasks across queues:
  - low queue:       542 wiki:ingest + list_delete tasks
  - question queue:  374 question:generation tasks
  - multimodal queue: 0

Worker logs repeat indefinitely until retry budget exhausted:

ERROR knowledgebase.go:234 | error=knowledge base not found knowledge_base_id=<deleted-kb-id>

Expected behavior

When a KB is soft-deleted, all pending asynq tasks targeting that KB should be dequeued (best-effort), similar to how DeleteKnowledge calls dequeueKnowledgeTasks for single-knowledge deletion.

Root cause

File: internal/application/service/knowledgebase.go:616-691 (DeleteKnowledgeBase)

err = s.repo.DeleteKnowledgeBase(ctx, id)           // DB soft-delete only
s.shareRepo.DeleteByKnowledgeBaseID(...)
s.deleteDataSourcesForKnowledgeBase(ctx, id)
task := asynq.NewTask(types.TypeKBDelete, ...)      // Enqueue async cleanup
// ❌ No dequeue / asynq.Inspector / DeleteTask call

File: internal/application/service/knowledgebase.go:695-828 (ProcessKBDelete)

The async cleanup path cleans up vectors, chunks, files, graph, and DB rows, but does not clean up asynq queue tasks for the deleted KB.

Contrast — internal/application/service/knowledge_delete.go:84-87 (DeleteKnowledge, single-knowledge):

// ✅ Correctly dequeues downstream tasks
if originalStatus == types.ParseStatusPending ||
    originalStatus == types.ParseStatusProcessing {
    s.dequeueKnowledgeTasks(ctx, id)
}

Suggested fix

In DeleteKnowledgeBase (or at the start of ProcessKBDelete), iterate over all pending tasks in each queue and remove those whose payload contains the deleted KB ID. This could reuse the existing taskInspector abstraction:

// In DeleteKnowledgeBase or ProcessKBDelete:
if s.taskInspector != nil {
    s.taskInspector.CancelTasksForKnowledgeBase(ctx, id)  // New method needed
}

Where CancelTasksForKnowledgeBase would use asynq.Inspector to scan and delete pending tasks matching the KB ID across all queues.

Alternatively, since ProcessKBDelete already iterates over the KB's knowledge entries, it could call dequeueKnowledgeTasks for each knowledge ID — but this would miss wiki:ingest tasks that are keyed by knowledge_base_id rather than knowledge_id.

Environment

• WeKnora version: 0.6.3
• Deployment: Docker Compose, 12 containers
• asynq concurrency: 48