Human x AI Interaction Patterns

These patterns highlight the diverse ways in which humans can interact with AI agents, ranging from direct control to more collaborative and conversational approaches. The choice of interaction pattern often depends on the complexity of the task, the level of autonomy required, and the need for human oversight and intervention. The design of the user interface plays a crucial role in mediating these interactions effectively.

Agents as Personal Assistants

Agents learn user preferences, track their history, and provide tailored experiences and proactive support. A memory-enhanced agent acting as a personal assistant would be an example of this.

Conversational Interaction

Humans engage in natural language dialogues with agents, providing instructions, asking questions, and receiving responses in a more human-like manner. This could involve a user chatting with a customer service agent to resolve an issue or asking a research agent open-ended questions. Several software frameworks are designed to facilitate these conversational workflows.

Delegation with High-Level Goals

Humans provide agents with general objectives or desired outcomes, and the agent autonomously determines the necessary steps to achieve them. For instance, a user could instruct a travel agent to “plan a weekend trip to San Francisco for two people with a budget of $1000,” and the agent would handle the itinerary, booking, and other details.

Direct Command and Control

Humans explicitly instruct agents to perform specific tasks or provide precise commands, similar to traditional software interaction but with the agent having more autonomy in executing the task. For example, a user might tell a coding agent to “write a Python script to analyze this data file”.

Human-in-the-Loop Collaboration

Humans and AI agents work together iteratively on a task, with humans providing guidance, feedback, and making critical decisions while the agent handles sub-tasks, gathers information, or executes actions. For example, in a claims processing scenario, an AI agent might gather initial information and prepare a draft, which a human expert then reviews and approves.

Providing Feedback and Oversight

Humans monitor the actions and outputs of AI agents, providing feedback to correct errors, refine behaviour, and ensure alignment with human values and goals. This is crucial for building trustworthy AI systems. Mechanisms for human approval workflows and escalation protocols fall under this pattern.

Humans can interact within a system where multiple AI agents are also present, potentially collaborating with or even observing the agent-agent interactions. The Conversational Chess example illustrates a scenario where humans and AI agents can play together using natural language.

Task Specification through User Interfaces

Humans use graphical or other intuitive interfaces to define tasks and workflows for agents without needing to write code or complex commands. This allows users who are not technical experts to leverage the capabilities of AI agents.

Teaching and Learning by Demonstration

The concept of agents learning from user behaviour implies a scenario where humans implicitly train agents through their actions and preferences.