Log in to your Document360 account to give feedback

Feature Request

Anonymous

Capturing citation details in Eddy AI analytics
While working with the Eddy AI Analytics page in the Knowledge base portal, I noticed that the citation details for responses are not currently visible. At present, Eddy AI displays references like [citation 1], [citation 2], but it doesn’t show which specific articles those citations are pulled from. Why does it matter? We regularly review the queries users submit and how Eddy AI responds to them. If the information provided is accurate, there’s no concern. However, when the response is incorrect or out of context, we need to identify the source of that information to make necessary corrections or improvements. Currently, to find out where Eddy AI fetched the content from, we often need to manually re-enter the same query in the Knowledge base site search. This process: Consumes Eddy AI query credits May return different results due to frequent content updates or backend changes in Eddy AI Sometimes fails to reproduce the original citation, making it harder to trace and verify the response Suggested improvement: It would be very helpful if the Analytics section could include the exact article titles or links from which the citations were generated. Providing this visibility within the portal itself will: Help us validate responses more efficiently Enable quicker identification and correction of outdated or misleading content Enhance trust in Eddy AI outputs for both the team and our end-users Thank you.
5
·
AI - Eddy
·
complete
Ask Eddy testing feedback
We've been testing Ask Eddy using our content to see if it's something we'd want to add to our site. However, there are a few key issues we identified, As with other chatbots, Eddy has a major problem with hallucination. It often makes up answers, especially when leading questions are used. In one of our test questions (regarding deleting a category), Eddy makes up a response that could potentially cause huge problems for users with significant data loss. I checked the reference pages cited to make sure we didn’t include or allude to the incorrect information. Additionally, the reference articles are often unrelated to the answer provided (sometimes Eddy is able to provide the correct answer even though it is not included in the reference articles cited). In other instances, Eddy will pull an answer from one page, but miss that the correct answer was provided in another article, and will not include that page in the reference articles. Eddy also seems to have some difficulty pulling answers from HTML tables, which I also found when testing ChatGPT separately. For example, when I asked which versions of PHP are supported, the response ignored the version numbers provided in the table on the system requirements page, and instead generated a response based on the body text on this page. Finally, the responses are typically very short and give less detail than a standard GPT would - I assume this is done intentionally to reduce the risk of hallucination. This wouldn’t be much of an issue if the reference articles were more accurate. I don't know what the solutions are to these issues, but hopefully this feedback is useful. As others have mentioned in their feedback, having some control over the instructions provided to the GPT would help to alleviate some of these issues.
3
·
AI - Eddy
·
complete
Load More
→