What is openCHA?

openCHA is a foundational Large Language Models (LLMs)-powered framework, enabling Conversational Health Agents (CHAs) to deliver effective healthcare services, including patient support, health coaching, and promoting patient self-awareness. While LLMs excel in conversational tasks, they often struggle with analyzing diverse data types, delivering personalized services, and ensuring responses are reliable, safe, and trustworthy. openCHA addresses these challenges by equipping CHAs with advanced capabilities for multi-step problem-solving, multimodal data analysis, and external information access. By integrating LLMs with datasets, knowledge bases, and analytical and AI models, openCHA enables CHAs to process healthcare queries effectively, analyze inputs, gather relevant information, and deliver context-aware, empathetic, and personalized responses. The framework aims to enhance the adaptability and effectiveness of CHAs in meeting the unique healthcare needs of users.

Why openCHA?

openCHA enables the development of CHAs designed to address critical challenges in healthcare communication:

  • Trustworthiness: CHAs rely on externally validated sources, verified by domain experts, rather than depending solely on the internal knowledge of LLMs. This approach ensures that the accuracy and dependability of responses are grounded in the credibility of the chosen sources.
  • Personalization: By integrating with patient-specific data and tailored models, openCHA enables CHAs to deliver highly personalized interactions. Each response is crafted to align with the unique health context and requirements of the individual.
  • Multi-Modal Data Processing: openCHA supports the analysis of diverse data types through its ability to interface with AI models developed by domain specialists. This flexibility allows CHAs to process inputs such as text, time series, and images effectively.
  • Dynamic Planning and Information Retrieval: CHAs are designed to dynamically connect with various databases, knowledge systems, and AI models. This ensures access to the most relevant and up-to-date health information during interactions.
  • Explainability: Transparency is a core feature of openCHA. The logic and processes behind response generation are clear and traceable, allowing users to understand how information is gathered and utilized.
  • Empathetic Conversation: By leveraging contextual understanding, openCHA equips CHAs to engage in supportive, human-like conversations. This capability improves the quality of interactions and enhances the emotional support offered in health-related scenarios.

Demos

 

 

 

 

openCHA and Gen AI in Health Hackathon

We organized a one-week take-home Generative AI in Health Hackathon kicking off on November 9 and concluding on November 16-17, co-hosted by NVIDIA, StartX, and HealthUnity. Approximately 180 participants registered for the hackathon. Ten selected teams presented their projects on Nov 16, and winners were selected by an external judging panel. The panel included leading experts from Stanford University, StartX, Accenture, etc. Please find more details here. Throughout the event, we provided mentorship and support for openCHA. About 5-6 of the selected teams (out of 10) developed their agents using openCHA within just a week. For more details see this.

How does openCHA work?

We have created openCHA, an LLM-powered framework that operates using a service-based architecture. This framework enables the development of agents capable of interpreting and analyzing user queries, providing appropriate responses, and managing access to external resources through Application Programming Interfaces (APIs). The interaction between the user and the framework is bidirectional, enabling a conversational tone for continuous and subsequent dialogues. Its core components include:

  • The interface serves as a bridge between users and agents, providing interactive tools accessible through web-based applications. It integrates various communication methods, such as text and audio, to transmit user queries to the Orchestrator. Moreover, users can attach metadata to their queries, including biomedical data or images. For instance, a user might upload a photo of their meal to request details about its nutritional value or calorie count, using the image as supplementary metadata.
  • The Orchestrator enables problem-solving and decision-making, ensuring relevant responses are obtained based on user inquiries. It utilizes the Perceptual Cycle Model to process input data through perception, transformation, and analysis while engaging with external sources for information retrieval and insights. Below are its five key components:
    • Task Planner: Equipped with LLM capabilities, this component designs procedures to collect the necessary information required to address the user’s query.
    • Task Executor: Executes the procedures created by the Task Planner and interfaces with external sources via function calls.
    • Data Pipe: Functions as a temporary repository for metadata and data acquired from external sources, aiding in the generation of responses.
    • Promptist: Converts text or results from external sources into effective prompts for the Task Planner or Response Generator.
    • Response Generator: Utilizes LLM to synthesize collected information, provide accurate responses, and ensure a conversational tone that fosters empathy and companionship.
  • External Sources are essential to access required information from the broader world. These sources, often accessed via APIs, enabling the Orchestrator to retrieve, process, and derive meaningful health data. openCHA integrates four primary categories of external sources for CHAs:
    • Healthcare Data Sources: Facilitate the collection, ingestion, and integration of diverse health data from sources such as Electronic Health Records (EHR), smartphones, and wearable devices like smartwatches. Examples include mHealth platforms and healthcare databases.
    • Knowledge Base: Provides access to up-to-date and relevant healthcare information from diverse resources, including healthcare literature, trustworthy websites, and knowledge graphs, leveraging search engines or retrieval models.
    • AI and Analysis Models: Deliver analytical capabilities for extracting insights, identifying patterns, and performing advanced tasks like data denoising, abstraction, classification, and event detection.
    • Translators: Enhance accessibility and inclusivity by converting content from various languages into widely spoken ones, such as English, ensuring broader usability of conversational health agents (CHAs).

Learning materials

openCHA case studies

References