ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance
Loading...
Author/Creator
Author/Creator ORCID
Date
2023-01-09
Type of Work
Department
Program
Citation of Original Publication
Roy, Kaushik, Manas Gaur, Misagh Soltani, Vipula Rawte, Ashwin Kalyan, and Amit Sheth. “ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance.” Frontiers in Big Data 5 (2023). https://www.frontiersin.org/articles/10.3389/fdata.2022.1056728.
Rights
This item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
Attribution 4.0 International (CC BY 4.0)
https://creativecommons.org/licenses/by/4.0/
Attribution 4.0 International (CC BY 4.0)
https://creativecommons.org/licenses/by/4.0/
Subjects
Abstract
Current Virtual Mental Health Assistants (VMHAs) provide counseling and sug2 gestive care. They refrain from patient diagnostic assistance because of a lack of
3 training on safety-constrained and specialized clinical process knowledge (Pro4 Know). In this work, we define ProKnow as an ordered set of information that maps
5 to evidence-based guidelines or categories of conceptual understanding to experts
6 in a domain. We also introduce a new dataset of diagnostic conversations guided by
7 safety constraints and ProKnow that healthcare professionals use (ProKnow-data).
8 We develop a method for natural language question generation (NLG) that collects
9 diagnostic information from the patient interactively (ProKnow-algo). We demon10 strate the limitations of using state-of-the-art large-scale language models (LMs)
11 on this dataset. ProKnow-algo models the process knowledge through explicitly
12 modeling safety, knowledge capture, and explainability. LMs with ProKnow-algo
13 generated 89% safer questions in the depression and anxiety domain. Further,
14 without ProKnow-algo generations question did not adhere to clinical process
15 knowledge in ProKnow-data. In comparison, ProKnow-algo-based generations
16 yield a 96% reduction in averaged squared rank error.The Explainability of the gen17 erated question is assessed by computing similarity with concepts in depression and
18 anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow-algo
19 achieved an averaged 82% improvement over simple pre-trained LMs on safety,
20 explainability, and process-guided question generation. We qualitatively and quanti21 tatively evaluate the efficacy of ProKnow-algo by introducing three new evaluation metrics for safety, explainability, and process knowledge-adherence. For repro23 ducibility, we will make ProKnow-data and the code repository of ProKnow-algo
24 publicly available upon acceptance.