* Deep refacto that disentangled logging logic between logging (informing the user via console outputs) and memory (storing what happened in the agent).
---------
Co-authored-by: benediktstroebl <50178209+benediktstroebl@users.noreply.github.com>
Co-authored-by: Aymeric <aymeric.roucher@gmail.com>
* vlm initial commit
* transformers integration for vlms
* Add webbrowser example and make it work 🥳🥳
* Refactor image support
* Allow modifying agent attributes in callback
* Improve vlm browser example
* time.sleep(0.5) before screenshot to let js animations happen
* test to validate internal workflow for passing images
* Update test_agents.py
* Improve error logging
* Switch to OpenAIServerModel
* Improve the example
* Format
* add docs about steps, callbacks & co
* Add precisions in doc
* Improve browser
* Tiny prompting update
* Fix style
* fix/add test
* refactor
* Fix write_inner_memory_from_logs for OpenAI format
* Add back summary mode
* Make it work with TransformersModel
* Fix test
* Fix loop
* Fix quality
* Fix mutable default argument
* Rename tool_response_message to error_message and append it
* Working browser with firefox
* Use flatten_messages_as_text passed to TransformersModel
* Fix quality
* Document flatten_messages_as_text in docstring
* Working ctrl + f in browser
* Make style
* Fix summary_mode type hint and add to docstring
* Move image functions to tools
* Update docstrings
* Fix type hint
* Fix typo
* Fix type hints
* Make callback call compatible with old single-argument functions
* Revert update_metrics to have a single arg
* Pass keyword args instead of args to callback
* Update webbrowser
* fix for single message case where final message list is empty
* forgot debugger lol
* accommodate VLM-like chat template and fix tests
* Improve example wording
* Style fixes
* clarify naming and fix tests
* test fix
* Fix style
* Add bm25 to fix one of the doc tests
* fix mocking in VL test
* fix bug in fallback
* add transformers model
* remove chrome dir from helium
* Update Transformers example with flatten_messages_as_text
* Add doc for flatten_messages_as_text
* Fix merge error
---------
Co-authored-by: Merve Noyan <mervenoyan@Merve-MacBook-Pro.local>
Co-authored-by: Aymeric <aymeric.roucher@gmail.com>
Co-authored-by: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Julien Chaumond
Clémentine Fourrier
Aymeric Roucher
Merve Noyan
Nicholas Broad
Lucain
Edward Beeching
Aymeric