using gradient descent to find the optimal number of similarity search such that:

  1. we guarantee a 100% hit rate for ground truth datasets (needs human or llm curation)
  2. we guarantee a quality lower bound for ground