Friday, April 26, 2024

UDLCO : Patient centered Critical appraisal of interventions, ambient intelligent data capture and clinical reasoning in doctors and LLMs

 

UDLC Summary :

The UDLC here shares a few patient requirements that are answered by  journal articles and conversely a few journal articles are shared that are critically or loosely appraised. Thoughts regarding open access and citations of authors work from their own online learning portfolios are developed and asserted. 


UDLC transcripts :


[3/26, 10:02 PM] +91: Looking for doctor recommendation for a friend. His condition, in his own words:

I was misdiagnosed for back issues while a few weeks back I was diagnosed with a very rare chronic pelvic pain issue called pudendal neuralgia. 

It’s been 2 months that I’ve been in debilitating pain and other distressing symptoms that stay throughout the day.. and the mental health is extremely bad due to accumulation of reactions 24X7 for so long.

Kindly DM if you can recommend a doctor anywhere in the world


[3/26, 11:21 PM] An AI : Go for a chronic pain specialist, if the patient is in Bengaluru then this is my colleague



[3/26, 11:24 PM] An AI :

He will do any interventions if indicated (sometimes a simple usg guided nerve block followed by radiofrequency is enough to end the suffering) and save the patient from high doses of medications like pregabalin and gabapentin


[3/27, 9:30 AM] Rakesh Biswas: How about a DIY AI product that can do the same? 

Product to zap/block DIY under AI guidance, any neuralgic pain in any nerve in the human body that could be as difficult to reach as the pudendal nerve in Alcock's canal or relatively easier such as median nerve in the carpal tunnel?



[3/27, 9:31 AM] An AI: You want to enrol in the trial? ๐Ÿค“



[3/27, 9:33 AM] Rakesh Biswas: I can lead it ๐Ÿ˜…

But we need the product first


[3/27, 12:34 PM] Uro : If he is Middle Age man ensure it is not due to chronic prostatitis - often missed and overlooked by most clinicians.


[3/27, 12:35 PM] Uro : The condition can be difficult to treat


[3/27, 11:24 PM] NPC: When selling AI products to "business" when we say buy this cool new technology, we get immediate push back- is it making me more money or saving me cost, is the retort from business? If the answer is yes to any of those two, then business is willing to listen and learn. Similarly on this community, can we talk purely from improving patient outcomes ++ etc


[3/28, 8:58 AM] Rakesh Biswas: Product buyer : Is it making my patient save more money? 

Seller : Definitely yes 

Product buyer : Will I as a provider make more money? 

Seller : Umm..er...not in the short term I guess but very likely in the long term 

Buyer: ๐Ÿง๐Ÿค”๐Ÿคจ


[3/28, 6:07 AM] +91 77: Has anyone seen this ? The PHTI looked at three categories of digital diabetes management solutions: remote patient monitoring, behavior and lifestyle modification, and nutritional ketosis.

The PHTI found that remote patient monitoring and behavior and lifestyle modification did not have a clinically meaningful reduction in HbA1c, a measure of blood sugar control. These interventions also increased healthcare spending. Nutritional ketosis, on the other hand, was found to have a clinically meaningful reduction in HbA1c and may lead to long-term cost savings. However, the evidence for nutritional ketosis is based on a lower certainty study.

The PHTI concludes that the current evidence does not support the broader adoption of remote patient monitoring or behavior and lifestyle modification for diabetes management. However, nutritional ketosis may be a promising option for some patients. More evidence is needed to determine the generalizability of the findings on nutritional ketosis.


[3/28, 9:07 AM] Rakesh Biswas: They may not have looked at the number of dangerous hypoglycemia events saved through remote patient monitoring @⁨Rahul healthcare 2.0⁩ ?


[3/28, 9:27 AM] XCMC Vellore: Not surprising. Minute to minute Remote monitoring does nothing to the disease only alerts you on the spikes and troughs - useful for brittle diabetics only. 
Nutritional Ketosis or hypoglycaemia is not such a frequent occurrence. In fact ketosis is latently present, though infrequent in Intermittent fasting which is another moniker for starvation ketosis, 
Like all public and community health health folks say -  a disciplined dietary style obviates unnecessary gadgetry which only alerts but adds to cost without mass benefits.


[3/28, 10:07 AM] Rakesh Biswas: Agree! Recall bias on my part as we are currently guiding the PHR driven hybrid management (the other component is the local physician)of a 2 year old girl through remote patient monitoring where we found dangerous levels of hypoglycemia documented by the father in the 2 year old child before we stepped in and optimized the insulin, else he would have continued with the same insulin regime before meeting the local doctor after a month!!


[3/28, 10:09 AM] XCMC Vellore: Yes there are flaws in this study. There is a selector bias , sample bias, recall bias and is an observational study. But there surely are points to be factored in the blind adoption of gadgetry as disease modifiers.


[3/28, 10:12 AM] An AI: Like this ๐Ÿ‘†


[3/28, 10:14 AM] XCMC Vellore: Every study is flawed to a certain extent as we have seen from those withdrawn from esteemed journals like Lancet, NEJM , Nature.  Evidence must align with logic , not overthrow the biological fundamentals.  We saw what happened to Sirolimus stents, IVC filters, biodegradable etc. Anecdotally fantastic , but not on a mass scale. Health transformation is essentially mass scale effects.


[3/28, 10:22 AM] Rakesh Biswas: The workaround to population based mass scaling is precision medicine focused on improving individual patient outcomes and trying to join the dots with the delivered interventions. 

An important agenda of pcori (patient centered outcomes research Institute) : https://sites.pitt.edu/~super1/lecture/lec53081/005.htm


[3/28, 10:23 AM] XCMC Vellore: Agree fully. Undoubtedly true. 
BUT What is happening today on the precocious pronouncements of ‘ healthcare transformations’ based on assumptions and anecdotal evidence is not scientific. Scientific rigour takes time , trial and testing.  Short cutting them will only delay the desired outcomes. When Fleming discovered penicillin , he did not trumpet it as revolutionary. Time did.  If we can just reduce the volume on ‘ Lo we have arrived’ it may serve health outcomes better.


[3/28, 10:26 AM] AIM : In continuum to all the comments coming in from the esteemed community experts, allow me to highlight that this report is comparing an apple with a pear in clinical grounds, so bias is obvious!!

 While digital diabetes solutions revolve more around monitoring of glycaemic status and focuses on improving glycaemic variability over and above the standards of treatment provided, nutritional ketosis through IF or time restricted eating is a completely different context of therapeutic intervention impacting the physiology of glucose metabolism!  I literally don't understand why they are being compared. 

More so, to implement good achievement of nutritional ketosis you need digital solutions to improve adherence and compliance. So these are complementary solutions!

To all the healthtech entrepreneurs of the group, please don't feel disappointed, there are plenty of research that proves the efficacy of digital solutions in chronic metabolic care! I'll share some of my own work here too as a reference. 

Dtx in diabetes care is here to stay! Cheers guys! ๐Ÿ˜…


[3/28, 10:29 AM] AnAI: Absolutely, moving towards precision medicine to get the individual dots to complete the big picture makes a lot of sense. Kudos to your approach ๐Ÿ‘



[3/28, 11:42 AM] NPC: Dear all, these communities are hard work. Being part of them and conversing over text is not easy and requires real commitment.  Each and every one of you adds value, and will add value. Each opinion and perspective, no matter how contrarian it may sound, is valuable. Your context and learning and journey from where you are coming from and going to is most important. These communities are about sharing and learning foremost. Growth mindset. We don't know, what we don't know. Hard debates and discussion is part and parcel of such communities (as long as we are respectful in our feedback). That is how we will learn new territory or reconcile with current ones etc. Isn't marrying the old and new the trick. I always think having a thick skin and taking feedback in good spirit is also useful to have. I just want to say that each and every member here, regardless of how vocal you are, or not, have an important role to play to realize the mission and vision of this great community assembled. As a fellow community member, I just want to say please do stay the course and continue to be part of community and help shape its direction, influence and execution. Thank you!


[3/28, 12:03 PM] I AI : We are deploying old school supervised machine learning models for diabetes related  eye disease screening in remote parts of South India.

In our last analysis we identified and referred DR, DME and AMD patients which otherwise would have never captured or at a much later stage in the disease progression.


[3/28, 12:06 PM] I AI : In absence of this, patients would have had to travel to tertiary care eye hospital for the screening which is now available at their PHC .. connected to Referral center via Telemedicine solution.


[3/28, 12:06 PM] AIM : That's good. ๐Ÿ‘ been using this in our clinics here (urban + rural) in WB over the last 5 years. Well proven use case of DL. Artelus and Remdio guys are more popular in this domain with diabetologists with their auto focus cameras. Good wishes!


[3/28, 12:40 PM] K : Two lessons we learnt hardway:

1. Crossing the AI Chasm as soon as possible. Beyond the buzz and hype the model needs to show visible proofs/evidences that it is adding value in "clinic's" workflow and not a generic standalone solution. For this we moved away from accuracy and primarily focused on false negative as the key metric, and got false positive to an acceptable threshold that the clinic was comfortable with. Lesson -  understand key drivers of workflow efficiency and priorities them.


2. ⁠Model drift/variance is real. Our solution doesn't support Active Learning by design to safeguard patient safety. In spite of that we observed model performance detoriate over a period of time as our model is device agnostic. Lesson - Plan for it in your regulatory filings and periodically update it with new training and test datasets.

Hope this helps. Until next deployment/learning :)


[3/28, 1:43 PM] R P : Very interesting JDA. Thanks for sharing.


[3/28, 5:06 PM] Rakesh Biswas: Point 2 was very interesting and insightful!๐Ÿ‘


[3/28, 8:41 PM] +9: Hi folks : Given the density of physician population here, looking for a nephrologist recommendation for someone I know with early stage Polycystic Kidney Disease 
Pls DM if you have recommendations 
(Apologies for the non AI question - pre patient care though is a great area for gen AI :))


[3/28, 8:57 PM] +91 9: Taking the liberty to ask a question here... Are there tech enabled solutions to help patients with DLBD ( Lewy Body Dementia ) ? More specifically, to mobilize them .. Something to  complement the efforts of physio & occupational therapists



[3/29, 6:56 AM] Rakesh Biswas: The search words

"assistive devices for mobilizing dementia patients"

brought this and I quote, 

"Competence-environmental Press Model provides a useful framework for understanding the potential role of assistive technologies in relationship to personal needs, abilities and social and environmental resources9. Applied to dementia, the model suggests that behavioral symptoms may reflect a mismatch between the person’s capabilities (cognitive and functional) and environmental demands. As Dr. Fozard has discussed, technology can lower sensory barriers to independent functioning10. Thus, using technologies to decrease sensorial, physical and cognitive demands and align environmental stressor   to fit patient abilities may reduce behavioral symptoms and enhance quality of life."

They :

"categorized devices into 3 domains and 11 categories: instrumental activities of daily living (IADLs) included devices for mobility (wheelchair), seating (for instance, padding, pillows), medication taking (medication dispensers), transfers (for instance, bed transfer handles), and leisure activities (for instance, videos, crafts, exercise equipment); Activities of Daily Living (ADLs) devices included those for eating (for instance, mug with lid), bathroom and toileting (for instance, grab bars, tub mat, tub rail), and grooming (for instance, sock donner); safety included medical alert identification bracelets, and monitors (for instance, motion detectors); and other devices (for instance, lost item finders)."



[3/29, 8:01 AM] Rakesh Biswas: A search for early stage polycystic kidney disease interventions  revealed a cochrane review that examined, "30 studies (2039 participants)  investigating 11 pharmacological interventions (angiotensin‐converting enzyme inhibitors (ACEi), angiotensin receptor blockers (ARBs), calcium channel blockers, beta blockers, vasopressin receptor 2 (V2R) antagonists, mammalian target of rapamycin (mTOR) inhibitors, somatostatin analogues, antiplatelet agents, eicosapentaenoic acids, statins and vitamin D compounds) and found no evidence of efficacy for any of the above. 




[3/29, 8:05 AM] +91 92: Dear Healthcare Leaders,

Greetings from the International Patients' Union.

Finally, the day is arriving, and the patients and the healthcare industry have waited for long. Together, we are bringing the much-awaited disruption to healthcare, led by patients. For the first time, a patient will decide which doctor or hospital to choose based on experience-based recommendations. Also, we will be launching the 'Patient Reported Outcomes (PROs) on the 6th of April, besides other initiatives that have the potential to disrupt healthcare and create a lasting impact. Join us to witness the revolution. Voice your views and meet the leaders who have brought this disruption. Based on the deliberations, the report will be sent to NITI Aayog, and we hope you will get your name there in this report.  

To register for the International Patients' Union Conference, please visit: www.patientsunion.org

Look forward to meeting you at the International Patients' Union Conference on April 6. Hosted at the PHD Chambers Auditorium, New Delhi 

-Ms. Priya Shukla
Team International Patients' Union
M: 9878741776



[3/29, 1:36 PM] Ko : Asked the same question to our Medical LLM "Cosmo". Here is what it recommends ...


[3/29, 1:36 PM] Ko : This is in beta undergoing testing and trials for public launch soon


[3/29, 1:37 PM] : In hindi
[3/29, 1:37 PM] : Feedback is welcome


[3/30, 8:57 AM]  XHcg Bangalore: I think we all are victims of mood swings..identifying it in ourselves and others is the key..
Not all swings are Bipolar disorders ? Dr J do opine


[3/30, 9:14 AM] Rakesh Biswas: Yes most of us hover around the equator without going so much as touching the poles!


[3/30, 9:18 AM]  XHcg Bangalore: Cancer , and Capricon line types


[3/30, 9:32 AM] Rakesh Biswas: Critical feedback and provocative thought for the day:

Do any of the current LLMs have any critical appraisal skills when it comes to evaluating scientific evidence or does it like the average busy doctor, simply reads the author's conclusions in the abstract and returns linguistically rehashed opinion masquareding as scientific wisdom?

Have tested and discussed this earlier with @Ra⁨⁩ and his opinion could be different from mine around what we found most of the time! Is it possible to make our LLMs more intelligent when it comes to appraising the literature?


[3/30, 9:45 AM] Ra: You are right.. it's at best rehashing the text of what's already said, not (yet) cross referencing and critically analyzing knowledge across texts. 

I use Scispace (typeset) often when analyzing papers.. it has LLMs giving one line summaries or one line limitations but yet need to read the paper to really know what the gaps or insights are. Great tool though.. 



[3/30, 9:51 AM] Ko : Thanks for the feedback...this is exactly what we need..would love to circle back and showcase what Cosmo does with scientific literature as a critical appraiser...please stay tuned for a demo soon


[3/30, 9:51 AM]Ko: This also warrants a detailed understanding of what the physician community expects from LLMs..if you could share more such opinions, that will be very helpful


[3/30, 9:53 AM] Rakesh Biswas: Yes we expect a scientific study data such as that of an RCT to be represented in terms of absolute values when sharing numerical data in the PICO format


[3/30, 9:56 AM] Rakesh Biswas: The references cited here are not RCTs but filtered wisdom/opinion


[3/30, 9:57 AM] Ko : This was a non clinical reference citation..Cosmo will allow users to transition from generic to clinical grade citation smoothly..will speak soon


[3/30, 9:57 AM] Ko : Please keep the feedback flowing ..much needed..


[3/30, 9:59 AM] Rakesh Biswas: Just a provocative (hopefully constructive) query more related to my area of interest (medical cognition) :

If it's available as well defined criteria then even an LLM can diagnose it albeit with a human professional seconding it? 

Is there a difference between how individual human professionals look at defined criterias v how a machine processes the same?


[3/30, 10:03 AM] : LLM will never aim to pursue that sir...it's there to just help augment your cognition and potentially eliminate biases ..that's about it.


[3/30, 10:06 AM] AIM : Good questions. To answer these thoughts and understand the core differences, we need more human validation of LLMs in a structured way. Dr. Piyush Mathur from Cleveland clinic has developed Humanely for validation on similar grounds. We need more stronger approved validation protocols. It is pertinent that all LLMs undergo human expert validation before launched in the market, specifically if used for CDSS. We are working on similar collaborative efforts from DoctorsAI.


[3/30, 10:07 AM] AIM : Going ahead, more tech expert - medico collaboration will produce stronger and useful LLMs for addressing the ground zero need gaps


[3/30, 10:09 AM] Rakesh Biswas: All the more reason for it to learn critical appraisal in terms of evidence based PICO formats to help understand clinical significance of the data rather than just the statistical significance and augment human capacity and competence to sort the hype from the meat


[3/30, 10:12 AM] Rakesh Biswas: As mentioned here earlier that kind of collaboration is a bit difficult currently in India as unlike the West as in Harvard, most medical colleges and engineering institutions in India are designed to be stand alone monoliths with no cross fertilization between the two due to huge distances between their workflows! 

Medical colleges getting set up in IISC and IIT's aim to address that but who knows how much time they would take to scale


[3/30, 10:17 AM]  XHcg Bangalore: Nice to hear this as a stand alone analysis..
Am working to address this gap..sharing my prg details. 
DM me if you would like to be a part of this movement


[3/30, 10:17 AM]  XHcg Bangalore: https://youtu.be/8RYURpBCu-I


[3/30, 10:17 AM] Ko : Thankfully things are changing fast in India...always optimistic for a better collaboration between academia and industry..here's to a better future..we have 1.4 billon human to take care...1000s of Cosmo, Doctors ai and what not are required to make a decent impact


[3/30, 10:20 AM] AIM : Nothing is impossible for India. Infact we are better placed than the west at fostering such collaborations in the domain of healthtech and AI. This group itself has a good potential for the same. Structure it and start somewhere. We are already doing it at some community. I'm hopeful.


[3/30, 11:01 AM]Sivaram Rajagopalan: Also I'd like to add from my general observations and experiences... Also and maybe as I come from Singapore... I feel its not happening (understandably) fast enough

1) Unlike in the developed markets, where Medical doctors  are given time and responsibilities backed by resources and funding to explore synergies around innovation. In India it's often personal interest, and has to be done on largely personal time. 

2) Furthermore due to the tradition (which is reducing) doctors are often on a pedastal and difficult to access and can end up dictating rather than collaborate. Often leading to solution that fits him but not the market. 

3) The way our universities have traditionally been run... Is to generate undergraduates... Not innovate or collaborate with commercial players. I was part of a group that studied this way back in 2010/11 for DBT. Things are improving, but maybe a bit slow. In USA many of my Engineering friends also went to become doctors... I think a near impossibility in Indian Universities. 

4) Our engineers need to build up their capability and capacity in knowledge of the Medical niches they work in to enable better communication and discussions. They need to bring deeper engineering principles to support the collaboration.

End of the day... I feel we have lots of room to improve the conversation between these domains and move to synergistic collaboration.


[3/30, 11:57 AM] AJ : That is not how LLMs are constructed !!



[3/30, 2:28 PM] +91: My thoughts are that few criteria can be identified by both machine and human - but there can be few which can be picked up by a human ( example to identify if it’s a delusion one needs to be aware of the cultural beliefs of that region to understand if it’s a delusion or a superstitious belief)
I am not well versed with what exactly LLM means or what it can do but seems technology can go beyond what we can imagine in future.


[3/30, 4:15 PM] Rakesh Biswas: From what I gather it appears to be an issue of data capture, where humans still capture a lot of data that the machine cannot see unless someone feeds that too to it?



[3/30, 4:51 PM] Ko: Not necessarily sir..computer vision tech can capture data on its own..but supervision is recommended although not necessary..



[3/30, 4:59 PM] Rakesh Biswas: It's not about vision as in seeing alone but the human mind still scores when it comes to understanding what are the cultural nuances and belief systems of the humans that generate the data as also perhaps hinted at by Dr J. We need AGI for that I guess and that's no where in our near vision at the moment?



[3/30, 5:23 PM] +91 9: As part of a course on Systems Thinking that I am teaching, I stumbled on this fascinating book The Logic of Care by Annemarie Mol, an ethnographic enquiry into the idea of patient choice vs collaborative care practice that is no doubt a daily frustration that medical professional deal with in their situations. I am wondering if some of you have read this and what are your own views? The author argues that creating more choice for patients is not necessarily wise. (currently still reading...which is more fascinating the Perplexity summary)


[3/31, 8:36 AM] Rakesh Biswas: Here are some very important marketing (or anti marketing) quotes from the book :

" ‘Nobody ever said that care would be easy’ (p. 76). It is not a well-delineated product but is an open-ended process involving knowledge, skill and experience."



[3/31, 8:40 AM] Rakesh Biswas: "The case she makes begs the question as to how we can better attend to care in a healthcare world that is driven by what can be measured via targets and outcomes."




[3/31, 8:43 AM] Rakesh Biswas: Cross readings :

"Comparative information seems to have a relatively limited influence on the choices made by many patients and patients base their decisions on a variety of provider characteristics instead of solely on outcome characteristics. The assumptions made in health policy about patient choice may therefore be an oversimplification of reality."

Unquote 




[3/31, 8:54 AM] Rakesh Biswas: In this context the scoping review linked below may be important to understand this emerging area :

"Patients’ choices are determined by a complex interplay between a variety of patient and provider characteristics. There is no such thing as the typical patient: different patients make different choices in different situations. Patients often attach greater importance to their own previous healthcare experiences or to GP recommendations than to comparative information. Additionally, patients base their decisions not only on outcome indicators but on a variety of provider characteristics. It can thus be argued that the choice process is much more complex than is often assumed."

Unquote 




[3/31, 9:17 AM] Rakesh Biswas: Opportunity to deconstruct and reconstruct? 



[3/31, 9:36 AM] Rakesh Biswas: Nice insights 

Although the pivotal title mars them a bit (perhaps just me). Intelligence is not just a human life form but is distributed across all life forms exhibiting cognition, some in a very subtle manner. 

Their allusion to read write technology is apt as this subset of intelligence (the one that is a human life form) is also called "asynchronous intelligence," which developed from a specific human need that we have previously expanded upon here : https://medicinedepartment.blogspot.com/2021/06/draft-3a-scholarship-of-integration-and.html?m=0


[3/31, 11:10 AM] +91 98: The question I have is to frame how 'patients' will adapt to AI technologies and how will it alter patient-care giver/provider relation? When Google got popular, such secondary knowledge was seen as friction in the dialogue. With AI how will this change? What would be a responsible framework for AI products in healthcare that assist well...not just end with caveats such as you are warned.



[3/31, 11:32 AM] Dr Sridhar DHIA IAMI: In current state, AI would have to say ,"This is for information only,please check with a trained medical professional".


[3/31, 12:08 PM] +91: I would imagine that medical professionals can use AI 

1. as second opinion tool. 

2.  Can be used as a guide or reference 

3. Can also be used is there anything we are missing (isn’t this second opinion ?)


[3/31, 12:18 PM] NPC: We have discussed a lot in the past, "non-invasive" AI listening in background with zero impact of clinician-patient interaction can create great summaries. The summaries can be enriched using a high quality medical LLM containing the vast universe of medical knowledge. The summaries can contain recommendations too. All of this as an assistant, not substitute for what a clinician does. Entire workflow before being surfaced back to doctor can have a human in loop approval (with reinforcement  learning so summaries and recommendations become higher accuracy) to ensure some value etc


[3/31, 12:55 PM] AIM: If any tech is working on this, or is looking to collaborate please reach out over DM. We are planning on a similar project on HL7 base  that can easily integrate with existing EMR.   You may also email your portfolio / project ideas officially to doctors.ai2024@gmail.com


[3/31, 1:13 PM] NPC: Many examples discussed in past. Here's one https://www.suki.ai/


[3/31, 2:40 PM] Rakesh Biswas: Excellent question that has also been in our minds since the early days of "user driven healthcare" 


Patients and healthprofessional users will rapidly adapt and gain as they did with google despite the negative flaK. 

Complexities of clinical care is compounded by "challenges regarding (a) untimely information, (b) irrelevant information, (c) confusing information, (d) missing information, (e) information overload, and (f) information multiplicity. Artificial intelligence could address these by (i) identifying and verifying low-quality information, (ii) targeting information for different user groups, (iii) visually summarizing relevant information, and (iv) jointly presenting multiple versions."


And hence the need for EBM trained LLM agents to get into our multiple- stakeholder-user driven healthcare ecosystems  and begin interacting with other users providing verifiable EBM info! Long way to go?
[3/31, 2:56 PM] Rakesh Biswas: AI can do way better than second opinion if it can be trained in critical appraisal of healthcare data driven studies that it can search and fetch and then use XAI to even explain around questions arising from individual patient issues. 

As someone rightly mentioned above that current AI doesn't work that way but perhaps that too wouldn't be an impossibility



No comments: