Improving Psychotherapy With AI: From the Couch to the Keyboard
利用 AI 改善心理治療：從躺椅到鍵盤
Enhancing quality, expanding access, and measuring outcomes with the aid of artificial intelligence means better care for patients
藉由人工智慧來提升品質、擴大服務範圍並衡量成效，代表患者能獲得更好的照護
A growing number of researchers and startup companies are looking to integrate artificial intelligence (AI) into perhaps the most human of places in health care: the therapy room [1]. The hope is that AI can help mitigate the time and expense it takes for humans to rate and code elements of the patient-provider interaction in a therapy session, a process that is vital for training new therapists, evaluating therapy quality, and conducting research studies and clinical trials of psychotherapy.
越來越多的研究人員和新創公司正尋求將人工智慧 (AI) 整合到醫療保健中最人性化的場所：治療室 [1]。 他們希望 AI 能協助減輕人類在治療過程中評估和編碼醫病互動要素所需的時間和費用，這個過程對於培訓新治療師、評估治療品質，以及進行心理治療的研究和臨床試驗至關重要。
Detecting empathy—and more
偵測同理心——以及更多
When Zac Imel, Ph.D., professor and director of clinical training at the University of Utah and co-founder and chief science officer at Lyssn, first considered integrating AI with psychotherapy, he began with a proof of principle study (Figure 1). Could AI be used to rate how empathetic a therapist was in a given session—a vital component of effective therapy? To test this, Imel and his colleagues trained a machine learning model using human ratings of counselor empathy and several hundred counseling session transcripts and recordings [2] (Figure 2). They found that the model could detect empathy from transcripts with a reliability similar to human raters, paving the way for more studies using machine learning to detect other therapy skills.
猶他大學教授、臨床訓練主任，同時也是 Lyssn 共同創辦人兼首席科學長的 Zac Imel 博士，在最初考慮將 AI 整合到心理治療時，他從一項原理驗證研究開始（圖 1）。AI 是否可用於評估治療師在特定療程中的同理心程度？這是有效治療的一個重要組成部分。為了驗證這一點，Imel 和他的同事使用諮商師同理心的人工評分以及數百個諮商療程的文字記錄和錄音，訓練了一個機器學習模組 [2]（圖 2）。他們發現，該模組可以從文字記錄中檢測到同理心，其可靠性與人工評分者相似，為使用機器學習檢測其他治療技巧的更多研究鋪平了道路。
In 2017, Imel and a group of other researchers founded Lyssn, a company which uses automatic speech recognition and AI to support the training and supervision of behavioral health specialists via a Health Insurance Portability and Accountability Act (HIPAA)-compliant cloud-based platform. Over 20,000 human-rated psychotherapy sessions were used to train Lyssn’s algorithms.
2017 年，Imel 和一群其他研究人員創立了 Lyssn 公司，該公司使用自動語音識別和 AI，透過符合《健康保險流通與責任法案》(HIPAA) 的雲端平台，來支援行為健康專家的培訓和監督。超過 20,000 個人工評分的心理治療療程被用於訓練 Lyssn 的演算法。
Figure 1. Zac Imel, Ph.D., professor and director of clinical training at the University of Utah and co-founder and chief science officer at Lyssn. (Photo courtesy of Z. Imel.)
圖 1. Zac Imel 博士，猶他大學教授兼臨床培訓主任，同時也是 Lyssn 的共同創辦人兼首席科學官。（圖片由 Z. Imel 提供。）
When fed a recording or transcript of a therapy session, Lyssn’s AI automatically detects more than 54 research-backed metrics of therapy quality, such as provider empathy, active listening skills, and engagement. Lyssn can also detect elements of evidence-based practices, including motivational interviewing and cognitive behavioral therapy (CBT), and can create session summaries and clinician notes.
當輸入一段治療過程的錄音或文字稿，Lyssn 的 AI 會自動偵測超過 54 種經研究證實的治療品質指標，例如提供者的同理心、積極傾聽技巧和投入程度。Lyssn 也能偵測實證醫學的要素，包括動機式晤談和認知行為療法 (CBT)，並且可以產生療程摘要和臨床醫師筆記。
Technical advancements in AI—in natural language processing and deep learning in particular—over the last five to six years have made Lyssn’s detailed analytics possible. “The revolution that’s happened in natural language processing is [that] we figured out ways to learn from unlabeled data and then transfer that learning to a specific problem,” says Michael Tanana, Ph.D., co-founder and chief technology officer at Lyssn (Figure 3). This advance means Lyssn’s AI is able to perform subtle, complicated tasks—like detecting empathy and writing clinical notes that look like they were generated by a person.
在過去五到六年裡，人工智慧的技術進步——特別是在自然語言處理和深度學習方面——使得 Lyssn 的詳細分析成為可能。Lyssn 的共同創辦人兼技術長 Michael Tanana 博士（圖 3）表示：「自然語言處理領域的革命在於，我們找到了從未標記資料中學習，然後將這些學習轉移到特定問題的方法。」這項進展意味著 Lyssn 的人工智慧能夠執行微妙而複雜的任務——例如偵測同理心和撰寫看起來像是人寫的臨床筆記。
By integrating AI into the clinic, Lyssn gives therapist trainees more opportunities to practice skills and receive immediate feedback and expands opportunities for quality monitoring of therapy. The platform allows supervisors to see detailed information about the performance of individual therapists such as how their listening skills have varied from session to session. Lyssn is also working with state governments to help evaluate evidence-based practices administered through federally-funded programs. “There were already mandates for measuring what was happening,” says Imel. “Instead of having to pay a motivational interviewing expert thousands of dollars to grade just a few sessions, we can get the cost per provider down quite low, and we can give them maximum visibility into the quality of services across their systems.”
透過將人工智慧整合到診所中，Lyssn 為治療師學員提供更多練習技巧和獲得即時回饋的機會，並擴大了治療品質監控的機會。該平台讓督導可以查看關於個別治療師表現的詳細資訊，例如他們的傾聽技巧在不同療程中如何變化。Lyssn 也在與州政府合作，以協助評估透過聯邦資助計畫管理的實證醫學。Imel 說：「已經有衡量正在發生的事情的授權。」「我們可以將每位提供者的成本降得很低，並且可以讓他們最大限度地了解其系統中服務的品質，而不必花費數千美元聘請一位動機式晤談專家來評估幾個療程。」
Now that this type of quality assurance evaluation is possible, it may lead to changes in mental health care administration. “There was no scalable way for people to demonstrate quality, and we’re starting to get into situations where payers are thinking about how can we actually pay for good quality services,” says Tanana. “As just a health care consumer, I think that’s what gets us out of bed in the morning.”
既然這種品質保證評估成為可能，它可能會導致心理健康照護管理方面的變革。「過去沒有可擴展的方式讓人們展示品質，而我們正開始進入付款人思考如何實際支付優質服務費用的情況，」塔納納說。「僅僅作為一個醫療保健消費者，我認為這就是我們早上起床的動力。」
Looking forward, Lyssn’s next big horizon is providing therapists with real-time suggestions for improving their interactions with patients. “We’re still not in the middle of the interaction, and we have the technological capability to help in real time with the conversation,” says Imel. “We’ve done some internal proof of concept work on that and are really excited about where that might go.”
展望未來，Lyssn 的下一個重大目標是為治療師提供即時建議，以改善他們與患者的互動。「我們仍然沒有參與到互動之中，但我們有技術能力可以在對話中即時提供幫助，」Imel 說。「我們已經在這方面做了一些內部的概念驗證工作，並且對它可能發展的方向感到非常興奮。」
Expanding access to therapy
擴大接受治療的機會
Another problem in mental health care that AI may help tackle is expanding access to care. That’s been the goal of ieso, a company based in the U.K. that has been providing assessment and treatment of mental health conditions primarily to patients within the National Health System (NHS) for the past 10 years.
AI 可能有助於解決的另一個心理健康照護問題是擴大接受照護的機會。這一直是 ieso 的目標，這是一家位於英國的公司，在過去 10 年中，主要為國民健康系統 (NHS) 內的患者提供心理健康狀況的評估和治療。
Figure 2. Overview of processing steps for moving from audio recording of session to predicted value of empathy. The lower portion of the figure represents the process for a single session recording, whereas the upper portion represents various speech signal processing tasks, learned from all available corpora (as indicated in the text). (Image and caption from [2]; Creative Commons Attribution License.)
圖 2. 從會話錄音到預測同理心值的處理步驟概述。圖的下半部分表示單個會話錄音的過程，而上半部分表示從所有可用語料庫（如文本所示）中學習的各種語音信號處理任務。（圖片和說明來自 [2]；創用 CC 姓名標示授權。）
Figure 3. Michael Tanana, Ph.D., co-founder and chief technology officer at Lyssn. (Photo courtesy of M. Tanana.)
圖 3. Michael Tanana 博士，Lyssn 的共同創辦人兼技術長。（圖片由 M. Tanana 提供。）
“In order to solve the access problem, the first thing we did is to provide therapy services online,” says Valentin Tablan, Ph.D., chief AI officer at ieso (Figure 4). Rather than rely on videoconferencing, therapists working for ieso provide patients with CBT via a text-based platform [3].
「為了要解決近用性的問題，我們做的第一件事就是提供線上諮商服務，」ieso 的人工智慧長 Valentin Tablan 博士說（圖 4）。ieso 的治療師並非仰賴視訊會議，而是透過文字平台為病患提供認知行為療法 [3]。
CBT therapists must transmit a lot of information to their patients, so ieso made the choice to record all therapy sessions so patients could review them later on. Over the past 10 years, ieso has seen some 100,000 patients and delivered—and recorded—more than half a million sessions of therapy. This ended up being a rich dataset for ieso to use AI to explore what works in therapy.
認知行為療法治療師必須傳輸大量資訊給病患，因此 ieso 選擇記錄所有諮商過程，讓病患日後可以複習。過去 10 年來，ieso 已服務約 10 萬名病患，並提供及記錄了超過 50 萬次的諮商。這最終成為 ieso 使用人工智慧探索諮商有效方法的一個豐富資料集。
Because ieso measures symptoms at every point of contact with a patient, the company is able to track how symptom intensity changes throughout treatment (Figure 5). “From a data science point of view, that’s an ideal situation. You have the full record of the intervention that was given to the patient and you have a measurement of the outcome, how the symptoms changed as a result of that intervention,” Tablan says.
由於 ieso 會在與病患的每次接觸點測量症狀，因此該公司能夠追蹤症狀強度在整個治療過程中如何變化（圖 5）。「從資料科學的角度來看，這是一個理想的情況。您有給予病患的<intervention>完整記錄，並且可以測量結果，也就是症狀如何因該<intervention>而改變，」Tablan 說。
Figure 4. Valentin Tablan, Ph.D., chief AI officer at ieso. (Photo courtesy of ieso.)
圖 4. ieso 人工智慧長 Valentin Tablan 博士。（圖片由 ieso 提供。）
In 2019, ieso researchers published a study where they used a deep learning model to automatically tag therapy sessions for different types of content and clinical events that occurred in the conversation between patient and therapist [4]. The researchers then looked at how the composition of the therapy session and the presence or absence of various elements of care correlated with outcomes for patients. For example, content related to therapeutic praise, discussing perceptions of change, and planning for the future were positively associated with improvement.
2019 年，ieso 的研究人員發表了一項研究，他們使用深度學習模型自動標記治療過程，以識別患者和治療師之間對話中發生的不同類型內容和臨床事件 [4]。然後，研究人員研究了治療過程的組成以及各種護理要素的存在與否，如何與患者的治療結果相關聯。例如，與治療性讚揚、討論對變化的看法以及規劃未來相關的內容，與病情改善呈正相關。
ieso took the results of this study and integrated them into a tool for the clinical supervision team that oversees ieso therapists. The tool analyzes every session of therapy, scores it against criteria of therapy effectiveness, and displays it on a data dashboard. Supervisors can use this information to check in with therapists who have patients who are not making the expected progress.
ieso 採用了這項研究的結果，並將其整合到一個工具中，供監督 ieso 治療師的臨床督導團隊使用。該工具會分析每次治療過程，根據治療效能的標準對其進行評分，並將其顯示在數據儀表板上。督導可以利用這些資訊來關心那些患者沒有達到預期進度的治療師。
Tablan says this consultation between human supervisor and therapist is key for understanding the nuances of a particular case. “The system may say that this therapist is not doing this particular element of CBT that we know is supposed to be good for patients, but there may be a good reason for that,” he says
Tablan 表示，這種人類督導和治療師之間的諮詢對於理解特定案例的細微差別至關重要。「系統可能會說這位治療師沒有做到我們知道對患者有益的 CBT（認知行為療法）的特定要素，但這可能有充分的理由，」他說。
Figure 5. User interface for the ieso therapy platform. (Image courtesy of ieso.)
圖 5. ieso 治療平台的使用者介面。（圖片由 ieso 提供。）
This AI-mediated feedback may be leading to better care for patients. For 2021, ieso reports a recovery rate for depression of 62% compared to the national average of 50%. For generalized anxiety disorder it was 73% versus a national average of 58%.
這種透過 AI 媒介的回饋可能會為患者帶來更好的照護。ieso 報告指出，2021 年的憂鬱症康復率為 62%，而全國平均水平為 50%。廣泛性焦慮症的康復率為 73%，而全國平均水平為 58%。
The hope is that AI tools can help clinicians offer more effective and efficient therapy, but that alone isn’t enough to adequately expand access to mental health care. “There are not sufficient therapists around the world,” says Tablan. He notes that the NHS is currently working toward a 25% access target—meaning that one in four people who need help will actually get it.
人們希望 AI 工具可以幫助臨床醫生提供更有效率的治療，但僅靠這一點還不足以充分擴大心理健康照護的覆蓋範圍。「世界各地的治療師數量不足，」Tablan 說。他指出，英國國民保健署 (NHS) 目前正朝著 25% 的覆蓋率目標努力——這意味著四分之一需要幫助的人實際上會得到幫助。
With that in mind, ieso has been working on a digital therapeutics program that will deliver care via virtual “conversational agents.” Tablan predicts that some patients may be able to receive all of their care through a conversational agent—and may actually prefer it—while others may use an agent as a supplement to their regular therapy, reducing the amount of time a therapist needs to work with a particular patient so they can help more people.
考慮到這一點，ieso 一直在開發一個數位治療計畫，該計畫將透過虛擬「對話式代理人」提供照護。Tablan 預測，有些患者可能能夠透過對話式代理人獲得所有照護——並且可能實際上更喜歡這樣——而另一些患者可能會使用代理人作為常規治療的補充，從而減少治療師需要與特定患者合作的時間，以便他們可以幫助更多的人。
Computerized CBT programs have been available for decades but have suffered from lack of patient engagement. Tablan says ieso is building something different. “We don’t start from the CBT book and deliver that in a digital format,” he says. “We start from what actually happens between real patients and real therapists, and we emulate that.”
電腦化的認知行為療法程式已經存在數十年，但一直苦於缺乏患者參與。Tablan 表示，ieso 正在建立一些不同的東西。「我們不是從認知行為療法的書本開始，然後以數位格式傳遞它，」他說。「我們從真實患者和真實治療師之間實際發生的事情開始，並模擬它。」
The company is currently testing a virtual support tool called the CBT Connect app. While the study is not yet done, engagement data looks promising. “Patients use the app to an amount of time that’s pretty much equivalent with an extra therapy session,” says Tablan. “It’s a way of increasing the dose without increasing the time demands on the therapist.”
該公司目前正在測試一種名為 CBT Connect 應用程式的虛擬支援工具。雖然研究尚未完成，但參與數據看起來很有希望。「患者使用該應用程式的時間量幾乎相當於額外的治療課程，」Tablan 說。「這是一種在不增加治療師時間需求的情況下增加劑量的方法。」
Training the next generation
培養下一代
Clinicians say that psychotherapy is both an art and a science, but sometimes the science gets lost in the shuffle. Could AI help bring it back?
臨床醫生表示，心理治療既是一門藝術，也是一門科學，但有時科學會在混亂中迷失。人工智慧能幫助找回它嗎？
Figure 6. Katie Aafjes-van Doorn, DClinPsy, assistant professor of clinical psychology at Yeshiva University and co-founder and clinical lead of Deliberate. (Photo courtesy of K. Aafjes-van Doorn.)
圖 6. Katie Aafjes-van Doorn，DClinPsy，葉史瓦大學臨床心理學助理教授，Deliberate 的共同創辦人兼臨床負責人。（照片由 K. Aafjes-van Doorn 提供。）
“As a trained clinical psychologist and as a clinician, I’ve always been surprised by how unscientific the field is,” says Katie Aafjes-van Doorn, DClinPsy, assistant professor of clinical psychology at Yeshiva University and co-founder and clinical lead of Deliberate (Figure 6). “It seems like we do a lot on intuition, what feels right, how the patient responds to you, but it seems like science and clinical practice are quite distant, or at least not working well together always.” Aafjes-van Doorn says she thinks this might especially be true for less regimented forms of therapy like psychoanalysis and psychodynamic therapy, which are her preferred modalities.
「身為一位受過訓練的臨床心理學家和臨床醫師，我一直對這個領域有多麼不科學感到驚訝，」葉史瓦大學臨床心理學助理教授，Deliberate 的共同創辦人兼臨床負責人 Katie Aafjes-van Doorn（圖 6）說。「我們似乎做了很多憑直覺的事情，什麼感覺是對的，病人如何回應你，但科學和臨床實踐似乎相當疏遠，或者至少並非總是能很好地協同工作。」Aafjes-van Doorn 說，她認為對於較不嚴謹的治療形式，如精神分析和精神動力治療，情況可能尤其如此，而這些是她偏好的模式。
To better integrate science with the practice of psychotherapy, Aafjes-van Doorn and a group of other researchers formed a company called Deliberate, which provides an AI-backed platform for analyzing objective measures for mental health diagnosis and treatment from video recordings and patient-provided ratings (Figure 7). The platform can be used to train and evaluate therapists as well as for clinical trials and research studies.
為了更好地將科學與心理治療的實踐相結合，Aafjes-van Doorn 和其他研究人員組成了一個名為 Deliberate 的公司，該公司提供一個 AI 支援的平台，用於分析影片記錄和患者提供的評分中，關於心理健康診斷和治療的客觀指標（圖 7）。該平台可用於培訓和評估治療師，以及用於臨床試驗和研究。
For clinicians, Deliberate allows for routine outcome monitoring based on session-by-session patient ratings of symptoms and the therapeutic relationship. The platform uses this information to flag patients who might be getting worse. “That is something that we know as clinicians we’re so blind [to],” she says. “We have no clue. We think we help everyone so this is something where we need help.”
對於臨床醫生來說，Deliberate 允許基於每次療程的患者對症狀和治療關係的評分，進行常規的結果監測。該平台使用這些資訊來標記可能正在惡化的患者。「身為臨床醫生，我們對此非常盲目，」她說。「我們毫無頭緒。我們認為我們幫助了所有人，所以這是我們需要幫助的地方。」
Figure 7. Example of data collection and user interface in the Deliberate platform. (Image courtesy of K. Aafjes-van Doorn.)
圖 7. Deliberate 平台中的資料收集和使用者介面範例。（圖片由 K. Aafjes-van Doorn 提供。）
When using this tool in her training clinic, Aafjes-van Doorn has learned that it is important to educate clinicians about the role of the AI-mediated feedback in their training and practice. “There are a million different reasons why symptoms might go up. It might be that the patient is actually more aware of their problems once they start talking about it, and that’s actually a good thing,” she says. “It is quite hard to explain to people that it’s not about their competency or being a bad therapist, but it’s actually just another data point that we can consider and think about.”
Aafjes-van Doorn 在她的訓練診所中使用這個工具時，了解到教育臨床醫師了解 AI 輔助回饋在其訓練和實踐中的作用非常重要。「症狀可能上升的原因有百萬種。有可能是患者在開始談論後，實際上更了解自己的問題，而這實際上是一件好事，」她說。「很難向人們解釋說，這與他們的能力或身為一個不好的治療師無關，而實際上這只是我們可以考慮和思考的另一個數據點。」
In a research project, Aafjes-van Doorn is using AI to detect the interpersonal skills that evidence suggests are the most important for all therapists to cultivate—regardless of the type of therapy they are delivering. These common factors are called Facilitative Interpersonal Skills (FIS) and include elements like focusing on the therapeutic relationship, showing empathy, and creating hope and positive expectations [5].
在一個研究項目中，Aafjes-van Doorn 正在使用 AI 來檢測證據表明對所有治療師來說，無論他們提供何種類型的治療，都最需要培養的人際**技巧**。這些共同因素被稱為促進性人際**技巧** (FIS)，包括關注治療關係、表現同理心以及創造希望和積極期望等要素 [5]。
A common training module for FIS involves showing trainees video clips of patients, asking them to respond as they would in a session, and then rating their skills. Aafjes-van Doorn’s group is working to automate this process by using the Deliberate platform to collect a large sample of recorded therapist responses with high-quality video that can be used to train a machine learning model to predict FIS.
FIS 的常見訓練模組包括向受訓者展示患者的影片片段，要求他們像在治療中一樣做出反應，然後對他們的技巧進行評分。Aafjes-van Doorn 的團隊正在努力將此過程自動化，方法是使用 Deliberate 平台收集大量錄製的治療師反應樣本，並提供高品質的影片，可用於訓練機器學習模型來預測 FIS。
Aafjes-van Doorn says that analyzing both movement and audio metrics will likely be key for creating a successful machine learning model and something that hasn’t been done in other attempts to use AI to automate FIS training. “We can look at head movement, body movements, even hands movement, and also the synchrony between the therapist and the patient and how they are attuned in their movements,” she says.
Aafjes-van Doorn 表示，分析動作和音訊指標很可能對於創建成功的機器學習模型至關重要，這也是其他嘗試使用 AI 自動化 FIS 訓練時沒有做到的。「我們可以觀察頭部動作、身體動作，甚至是手部動作，以及治療師和患者之間的同步性，以及他們在動作上的協調程度，」她說。
However, not all therapists are on board with adopting AI-based tools. Privacy and ethical concerns are barriers for some. That is one advantage of using a platform like Deliberate, which is HIPAA-compliant. Deliberate is also developing feature extraction methods that would allow researchers and clinicians to use interesting features from a video therapy session without being able to identify the patient or therapist. “That could change the field a lot if people can be totally certain that they wouldn’t be identified,” she says.
然而，並非所有治療師都贊成採用基於 AI 的工具。隱私和倫理方面的考量對某些人來說是阻礙。這也是使用像 Deliberate 這樣符合 HIPAA 規範的平台的一個優勢。Deliberate 也在開發特徵提取方法，讓研究人員和臨床醫生可以使用視訊治療過程中有趣的特徵，而無需識別患者或治療師。「如果人們可以完全確定自己不會被識別出來，這可能會大大改變這個領域，」她說。
Other clinicians may be concerned that they will one day be replaced by an AI therapist. While Aafjes-van Doorn can see a role for digital therapeutics in helping people with low-level symptoms, she does not anticipate an end to human therapists altogether. “The relational aspect of being in a room or even being in a zoom room together and feeling what it’s like to be with you so that I can understand what your interpersonal world might be like—I think that is still something that psychotherapy can offer that is not necessarily replaceable,” she says.
其他臨床醫生可能會擔心他們有一天會被 AI 治療師取代。雖然 Aafjes-van Doorn 認為數位療法可以在幫助有輕微症狀的人方面發揮作用，但她並不預期人類治療師會完全消失。「身處同一個房間，甚至是同一個 Zoom 房間，感受與你同在的感覺，以便我了解你的人際關係世界可能是什麼樣子——我認為這仍然是心理治療可以提供的東西，而且不一定可以取代，」她說。
Yet, it is a concern she has heard from others when talking about her work. “I think that is a fear that we do need to tackle and take seriously,” she says.
然而，這是她在談論她的工作時從其他人那裡聽到的擔憂。「我認為這是一種我們確實需要處理並認真對待的恐懼，」她說。
Aafjes-van Doorn hopes that these concerns can be adequately addressed so that more clinicians can benefit from the fruits of AI. “What I hope at least is that it’s heading toward not just being a fancy tool for some psychotherapy researchers,” she says. “I would hope that it’s a far more common practice with clinicians seeing that these clinical tools actually will help them and will help their patients. Because that’s ultimately what we want to do.”
Aafjes-van Doorn 希望這些疑慮能獲得充分解決，讓更多臨床醫生能從 AI 的成果中受益。「我至少希望它不僅僅是某些心理治療研究人員的精美工具，」她說。「我希望臨床醫生能更普遍地看到這些臨床工具實際上會幫助他們，也會幫助他們的患者。因為這最終是我們想做的事。」
References 參考文獻
- K. Aafjes-van Doorn et al., “A scoping review of machine learning in psychotherapy research,” Psychotherapy Res., vol. 31, no. 1, pp. 92–116, Jan. 2021. Accessed: Jul. 15, 2022, doi: 10.1080/10503307.2020.1808729.
K. Aafjes-van Doorn 等人，“機器學習在心理治療研究中的範圍回顧”，《心理治療研究》，第 31 卷，第 1 期，第 92–116 頁，2021 年 1 月。存取日期：2022 年 7 月 15 日，doi：10.1080/10503307.2020.1808729。 - B. Xiao, Z. Imel, P. Georgiou, D. Atkins, and S. Narayanan, “‘Rate my therapist’: Automated detection of empathy in drug and alcohol counseling via speech and language processing,” PLoS ONE, vol. 10, no. 12, 2015, Art. no. e0143055. Accessed: Jul. 15, 2022. [Online]. Available: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0143055
B. Xiao、Z. Imel、P. Georgiou、D. Atkins 和 S. Narayanan，“‘評價我的治療師’：透過語音和語言處理自動檢測藥物和酒精諮詢中的同理心”，《PLoS ONE》，第 10 卷，第 12 期，2015 年，文章編號 e0143055。存取日期：2022 年 7 月 15 日。[線上]。可於以下網址取得：https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0143055 - D. Kessler et al., “Therapist-delivered internet psychotherapy for depression in primary care: A randomized controlled trial,” Lancet, vol. 374, no. 9690, pp. 628–634, 2009. Accessed: Jul. 15, 2022, doi: 10.1016/s0140-6736(09)61257-5.
D. Kessler 等人，“基層醫療中針對憂鬱症的治療師主導型網路心理治療：一項隨機對照試驗”，《刺胳針》，第 374 卷，第 9690 期，第 628–634 頁，2009 年。存取日期：2022 年 7 月 15 日，doi：10.1016/s0140-6736(09)61257-5。 - M. P. Ewbank et al., “Quantifying the association between psychotherapy content and clinical outcomes using deep learning,” J. Amer. Med. Assoc. Psychiatry, vol. 77, no. 1, p. 35, Jan. 2020. Accessed: Jul. 15, 2022, doi: 10.1001/jamapsychiatry.2019.2664.
M. P. Ewbank 等人，“使用深度學習量化心理治療內容與臨床結果之間的關聯”，《美國醫學會雜誌精神病學》，第 77 卷，第 1 期，第 35 頁，2020 年 1 月。存取日期：2022 年 7 月 15 日，doi：10.1001/jamapsychiatry.2019.2664。 - T. Anderson et al., “Therapist effects: Facilitative interpersonal skills as a predictor of therapist success,” J. Clin. Psychol., vol. 65, no. 7, pp. 755–768, Jul. 2009. Accessed: Jul. 15, 2022, doi: 10.1002/jclp.20583.