when looking at this url https://www.crowdstrike.com/blog/crowdstrike-falcon-cloud-security-delivers-future-of-cnapp/ the chrome extension summarizes the page as
CrowdStrike is the only cloud security provider that is recognized as a Representative Vendor in the Gartner Market Guide, demonstrating their ability to meet or exceed Gartner’s criteria for CNAPP capabilities. [1]
but there is nothing in the article that mention them being THE ONLY. this is a mis-summarization of the content.
Hi kayvon, thank you very much and really appreciate your feedback.
In this case our model indeed created a hallucination. The article didn't mention on "only", the mode imagine the "only" because the word are referred many times (for different usages like Erika mentioned below).
You might heard that OpenAI/Google/Microsoft's model also suffered from similar hallucination. This is a pretty hard problem to solve by just adjusting models.
We would like to work on how to improve the verification of the result to make it easier to do fact checks from the source context. We would love to hear your feedback on this.
I wonder what made to start to think to fact check the "only" part? Is there any thing that we could surface that would make your fact check much easier?
Any feedback would be greatly appreciated. Thanks and looking forward to hearing from you!
Hi Kayvon, thanks so much for pointing this out! Could you tell us more about how you would like to work?
To validate what you went through, I just ran the extension on the content and had the same experience.
I did notice that "only" popped up a couple times in the text, though it's not specifically in reference to what you're pointing out. In a case like this, would you want to be able to inspect the content our algorithm is pulling from to create the summary?
I'll ask engineering to take a look to see how we ended up on this result -- your feedback is really helpful to that end. In terms of what we build and how we build it, we'd love to know more from you about your ideal experience.