Building a Document Q&A System with On-Device AI

Every now and then I build a demo for a presentation and think to myself - this deserves its own blog post. I then promptly forget to actually do that. Even better, I completely forget to even show the demo in the presentation. A few weeks ago I made my first presentation at RenderATL and gave my talk on Chrome's on-device AI technology. I've been blogging about this particular set of APIs for a few years now and I've given talks on it at numerous conferences. While these APIs are still evolving, many of them (and the ones I'm using today in particular) are now fully GA for Chrome. For this demo, I put together a few things I've done before into one cohesive tool - a completely client-side application that lets you pick a file, get a summary, and then chat with it. The Stack The demo is built from a few different things: First, I'm making use of the incredibly excellent officeParser library by Harsh Ankur. This library can take any Office document, or PDF, or ODT, or CSV, and so on,…