Open WebUI, Ollama, Docker, Portainer GenAI Demos - LMM and LLM PDF.

To improve commercial software quality, feedback was collected from U.S. users regarding two demos using Open WebUI and Ollama GenAI Models.

Demo 1: Users were asked to upload a photo and request its description, which were then processed by a server that generated text and was returned to the user. Some users reported on the novelty of the application, while at least one user had observed similar ai functionality. Users typically experienced issues with speed to receive their generation, and sometimes experienced an interruption in transmission, especially if images appeared more complex. Speed issues using the Moondream LMM were likely attributed to sending full images to the server, as processing data on the server was performed at typical speeds. Text generation quality was generally accurate, but more basic and brief than with leading LMMs such as Chat GPT4o and Gemini. 3 Users (MI, TX, CA)

Demo 2: All prior users and two new users were asked to submit their prompt regarding technology, but this time to a more efficient language only generative ai model. One returning user commented that the speed was much greater, and therefore preferred their experience with this simpler demo over issues using an image. A new user found utility asking several questions and could convert documents to text, but the model was not able to spell check or summarize an uploaded document that can be accomplished with larger models. Text generations were typically longer in words than with the prior demo; but were typically less detailed and missed some minor contextual nuances vs. larger models that run slower. Given the small 350MB size, Qwen2 performed well as a very fast basic LLM alternative to existing options. 5 Users (MI, TX, CA)

The overall feedback regarding both demos was that slow speed or interrupted transmissions were a negative experience, while receiving a faster text generation from a text prompt produced a more positive experience, despite more basic utility. Some users were interviewed in person or on the phone and typically were in one of three categories: 1) Interested or very interested in using GenAI for productivity or engaging with the technology to a greater degree, 2) Thought results had utility and were previously aware of the benefit, but still not fully committing to using the technology personally, or 3) Not comfortable with using AI at this time or did not complete the demo. Cases 2 and 3 are anticipated to improve due to expected broader GenAI adoption in industry later this year.

Based on this feedback, the smaller model from Demo 2 that ran fastest with longer text explanations using a more basic task and less network issues had the highest user satisfaction of 3 users interviewed, in general. Next steps include efforts to provide users with better, faster, and less expensive generative ai software to solve drug discovery problems using more appropriate hardware software combinations.
Reference: NetworkChuck. (2024b, May 3). host ALL your AI locally [Video]. YouTube. https://www.youtube.com/watch?v=Wjrdr0NU4Sk