[Roadmap] LiteWebAgent Roadmap Q3 2024

[TataKKKL]

V1 release

v0 implementation

• improve the find and click agent #2
• add more navigation control #1

Baseline: function calling for action generation, browsergym for action grounding

• [roadmap] architecture design and roadmap #13
• [features] web agent baselines #5
• browser gym environment #3
• improve prompt #19
• refactor to use browsergym if not modified #60
• [feature] auto-login with state.json  #52
• [feature] select option (drop down options) #42
• [refactor] function response #37
• [refactor] pass goal instead of plan, mark plan as optional #33

Input feature

• [feature] test new input feature #14
• pass screenshot and SoM as input #18
• [feature] extract html identifier from interactive elements #29
• [refactor] make it more flexible to use different features from the webpage #24

Evaluation

• [feature] evaluate whether a task is completed #9

Planning

• [feature ] ContextAwarePlanningAgent #22
• [refactor] modify HighLevelPlanningAgent #23
• [feature] update workflow based on current input and current result #21
• make plan: [feature] workflow agent #16
• [fix] structured output taking too long #34
• [feature] ReAct Agent #25

Prompting Agent

• [feature] prompt engineering for prompt agent #58
• [feature] add prompt agent #53

Search agent & Replay

• add search_agent debugged repo #77
• [feature] fix replay #76
  • log ariaLabel to steps.json #69
  • [feature] refactoring functions and replay #56
  • [feature] workflow replay #30
  • [feature] record element id, task status, recording for replay #26

Interface

• [feature] modify cli_main and main #67
• [feature] simple chat interface #28
• [feature] enable human, agent interaction #12