#77

Add pruning and retention policy for devii_lessons table to prevent unbounded growth

Status: closed
retoor retoor ·

Summary

The devii_lessons table has grown to 3827 records (soft-deleted) with no pruning, retention, or deduplication policy. Every reflect() call stores a permanent lesson used by recall() queries, causing performance degradation and accumulation of duplicate/irrelevant lessons.

Steps to Reproduce

  1. Call reflect() multiple times for the same user/guest session.
  2. Observe that each call inserts a new record into devii_lessons without any limit or deduplication.
  3. Run recall() queries and note that old, duplicate, or irrelevant lessons are still returned.
  4. Check the table row count; it grows unbounded per account/session.

Expected Behaviour

  • The devii_lessons table should have a maximum lesson count per user/guest session (e.g., 500 active lessons).
  • Lessons older than a configurable number of days should be automatically pruned.
  • Similar or duplicate lessons should be merged or deduplicated instead of stacking.
  • A mechanism (e.g., quality/thumbs-up system) should allow irrelevant lessons to be silently dropped.
  • recall() queries should remain performant as the table size is bounded.

Actual Behaviour

  • The table contains 3827 records (soft-deleted) with no cap, retention policy, or deduplication.
  • recall() queries slow down as the table grows.
  • Duplicate lessons about the same topic (e.g., container sync issues) accumulate.
  • Old, irrelevant lessons continue to match in recall results.
  • The database size is 44 KB and grows with every session.

Environment

Not provided.


Reported by retoor via DevPlace.

Updates & comments

No updates yet.

Log in to comment.