Building a RAG-based Bot with a large knowledge base.

Hello everyone, Recently I received some data from a website and am asked to develop a bot that can go through that data and answer questions. The data is a single json file (~1MB) and contains information related to projects initiated throughout the company. Each object in the json file has information like name of the project, date it was started, whom it was assigned to etc. It has about 2800 objects. Here's what I've done. I created an AWS Bedrock Knowledge Base and Injested the entire JSON file there. I chucked the json file before hand (on a month+year granularity) and created metadata files and then used the retrieve and generate API using Claude 3.7 to query the knowledge base. THE PROBLEM: The LLM is unable to provide relevant information. For example if I ask it how many projects were initiated in July 2025, it usually says 3 or 5 even through there are like 25ish. And the responses are also quite varying everytime despite keeping temperature low. Sometimes it feels like it is unable to go through the knowledge base properly for example if I ask it to return the name of the latest initiated project. It will return something from Feb 2025 even though data is till July. I've increased the top k all the way till 100 which improved results slightly but still nothing usable in a production application. My question is what is the best way to tackle this problem and are their any other strategies i shoild try?