SI – Subtracted Intelligence

In our newfound enthusiasm of all things AI, some might take issue with the word “Artificial” and prefer “Augmented”, or the less pretentious “Added” to reflect a “man+machine” future, rather “man or machine” one. Just as there is a rule that scuba divers operate in pairs to look out for each other, maybe machines should be paired with a human minder/beneficiary.

Trusting them is an issue for Ms. Melanie Mitchell, Professor of Computer Science at Portland State University, according to the article titled “Artificial Intelligence Hits the Barrier of Meaning” in the New York Times:

“While some people are worried about “superintelligent” A.I., the most dangerous aspect of A.I. systems is that we will trust them too much and give them too much autonomy while not being fully aware of their limitations. As the A.I. researcher Pedro Domingos noted in his book “The Master Algorithm,” “People worry that computers will get too smart and take over the world, but the real problem is that they’re too stupid and they’ve already taken over the world.”

Around the same time, Andrew Fentem wrote a piece in The Register called “Mything the point: The AI renaissance is simply expensive hardware and PR thrown at an old idea”.

He introduced it with:

“For the last few years the media has been awash with hyperbole about artificial intelligence (AI) and machine learning technologies. It could be said that never, in the field of computer science, have so many ridiculous things been said by so many people in possession of so little relevant expertise.”

A lot of the comments agreed with him.

The articles above targeted the lofty goals of AI. Meanwhile, longstanding problems with more down to earth tech has been exposed in an article in The New Yorker, by Atul Gawande that had been “in gestation for a couple of years now”: “Why Doctors Hate Their Computers”.

Atul is a particularly interesting contributor. Apart from being a surgeon, he has written “The Checklist Manifesto” and in 2018, was named the CEO of an independent health-care venture formed by Amazon, Berkshire Hathaway, and JPMorgan Chase. They intend to “deliver solutions for better outcomes, satisfaction, and costs of care”.

The appearance of the A-word (Amazon) should make any industry nervous, let alone the developers of a software product that was the subject of much of Atul’s attention in the article.

The hospital system profiled reminded me of some of my longstanding gripes with legal practice management software (PMS). While the article title could inspire me to write a similar article about lawyers, I would prefer to write about the many who do love their computers, or rather the software they have chosen, rather than the systems enterprises impose on us. I certainly love the tools I selected, and developed, as they empower me. Unfortunately, many work situations do not provide a choice.

Having considerable experience with an intuitive PMS, and seen the alternatives, I certainly appreciated this:

“Ordering a mammogram used to be one click,” she said. “Now I spend three extra clicks to put in a diagnosis. When I do a Pap smear, I have eleven clicks. It’s ‘Oh, who did it?’ Why not, by default, think that I did it?” She was almost shouting now. “I’m the one putting the order in. Why is it asking me what date, if the patient is in the office today? When do you think this actually happened? It is incredible! …

She continued rattling off examples like these. “Most days, I will have done only around thirty to sixty per cent of my notes by the end of the day,” she said. The rest came after hours. Spending the extra time didn’t anger her. The pointlessness of it did.”

A better legal PMS might handle timesheets, for example by filling in date (today), who you are, who you are speaking to, and how long it took. It then presents you with your filenote to edit down to a time entry summary. All this default data saves us say 95% of the time, and is so much quicker than doing it manually, because the system designers had thought ahead.

Lawyers are not alone when it comes to information overload. Atul goes on to say:

“Sadoughi told me of her own struggles—including a daily battle with her Epic “In Basket,” which had become, she said, clogged to the point of dysfunction. There are messages from patients, messages containing lab and radiology results, messages from colleagues, messages from administrators, automated messages about not responding to previous messages. “All the letters that come from the subspecialists, I can’t read ninety per cent of them. So I glance at the patient’s name, and, if it’s someone that I was worried about, I’ll read that,” she said. The rest she deletes, unread. “If it’s just a routine follow-up with an endocrinologist, I hope to God that if there was something going on that they needed my attention on, they would send me an e-mail.” In short, she hopes they’ll try to reach her at yet another in-box.”

Ironically, she is unlikely to find time to keep up with the often AI-fueled medical breakthroughs that seem to be a weekly event. I suspect that her only hope is an AI that monitors her “In Basket” for messages that are critical.

Meanwhile, here is a system that subtracts, not adds intelligence and capability to doctors.

While the rest of the world was revelling in tech-enabled collaborations, the inflexibility of the hospital system was isolating doctors from support staff:

“Jessica Jacobs, a longtime office assistant in my practice … said that each new software system reduced her role and shifted more of her responsibilities onto the doctors. Previously, she sorted the patient records before clinic, drafted letters to patients, prepped routine prescriptions—all tasks that lightened the doctors’ load. None of this was possible anymore. The doctors had to do it all themselves. She called it “a ‘stay in your lane’ thing.” She couldn’t even help the doctors navigate and streamline their computer systems: office assistants have different screens and are not trained or authorized to use the ones doctors have.”

It seemed none of the medical staff liked the system, yet management try to justify the inconvenience:

“… we think of this as a system for us and it’s not,” he said. “It is for the patients.” While some sixty thousand staff members use the system, almost ten times as many patients log into it to look up their lab results, remind themselves of the medications they are supposed to take, read the office notes that their doctor wrote in order to better understand what they’ve been told. Today, patients are the fastest-growing user group for electronic medical records.”

Management also demonstrated they totally missed the point: it should be designed to make all users tasks easier, and quicker, not add to their workload. More importantly, it should not detract from their primary purpose of optimising the healthcare delivered by doctors. Could patients need to use online systems more because they don’t get enough information when with their doctors?

An earlier Slaw item explores some variations on these ideas.

Getting the balance right between the time spent staring at screens vs interacting with patients can benefit from the use of “scribes“. They type the notes thereby allowing the doctor more face-time with the patients. These scribes might be in the room in real time, or in India transcribing from a recording. Atul notes: “We replaced paper with computers because paper was inefficient. Now computers have become inefficient, so we’re hiring more humans. And it sort of works.”

His article was well received on Twitter, particularly by doctors, with the suggestion that it was not a new problem, and that workarounds were evolving.

Atul’s concluding suggestion was to share a screen and discuss it with the patient. A tool I use extensively as a patient, and otherwise, is Notability which both records the audio, and allows you to type notes on a Mac or iPhone. I get away with typing just brief notes and when I review them later, I click on the text of any point that needs clarification, and recording from when I typed it can be heard. This allows me to jump straight to the part of the recording that interests me without wading through a linear recording trying to find that part. For Windows users, OneNote has a similar feature.

Consumer apps which helped raise our usability expectations, will hopefully rescue us from the complexity of poorly designed legacy systems. It is all very well getting excited about the intelligence of machines, but when those who designed them aren’t so smart interacting with them will be challenging. Fortunately, Apple and co will apply AI to the “pointless” aspects of tasks. I touched upon how software has helped with tasks in an earlier Slaw item.

In view of Atul’s new CEO role, and with tech biggies like Apple, Microsoft and Google moving into healthcare, this will be a space worth watching. Meanwhile, it seems that we still have a long way to go before we can truly love Big Brother.