408b52abec
* vlm initial commit * transformers integration for vlms * Add webbrowser example and make it work 🥳🥳 * Refactor image support * Allow modifying agent attributes in callback * Improve vlm browser example * time.sleep(0.5) before screenshot to let js animations happen * test to validate internal workflow for passing images * Update test_agents.py * Improve error logging * Switch to OpenAIServerModel * Improve the example * Format * add docs about steps, callbacks & co * Add precisions in doc * Improve browser * Tiny prompting update * Fix style * fix/add test * refactor * Fix write_inner_memory_from_logs for OpenAI format * Add back summary mode * Make it work with TransformersModel * Fix test * Fix loop * Fix quality * Fix mutable default argument * Rename tool_response_message to error_message and append it * Working browser with firefox * Use flatten_messages_as_text passed to TransformersModel * Fix quality * Document flatten_messages_as_text in docstring * Working ctrl + f in browser * Make style * Fix summary_mode type hint and add to docstring * Move image functions to tools * Update docstrings * Fix type hint * Fix typo * Fix type hints * Make callback call compatible with old single-argument functions * Revert update_metrics to have a single arg * Pass keyword args instead of args to callback * Update webbrowser * fix for single message case where final message list is empty * forgot debugger lol * accommodate VLM-like chat template and fix tests * Improve example wording * Style fixes * clarify naming and fix tests * test fix * Fix style * Add bm25 to fix one of the doc tests * fix mocking in VL test * fix bug in fallback * add transformers model * remove chrome dir from helium * Update Transformers example with flatten_messages_as_text * Add doc for flatten_messages_as_text * Fix merge error --------- Co-authored-by: Merve Noyan <mervenoyan@Merve-MacBook-Pro.local> Co-authored-by: Aymeric <aymeric.roucher@gmail.com> Co-authored-by: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| fixtures | ||
| __init__.py | ||
| test_agents.py | ||
| test_all_docs.py | ||
| test_default_tools.py | ||
| test_final_answer.py | ||
| test_function_type_hints_utils.py | ||
| test_models.py | ||
| test_monitoring.py | ||
| test_python_interpreter.py | ||
| test_search.py | ||
| test_tools.py | ||
| test_types.py | ||
| test_utils.py | ||