The Devil is in the Data

James K. Waldsmith, DVM
President, Vetel Diagnostics; Owner, The Equine Practice

Education: Michigan State University

AI Mission: Working heavily to improve patient outcomes and quality of life for veterinarians by helping veterinarians implement proper use of artificial intelligence and related tools.

Dr. James Waldsmith says he's interested in technology. That's an understatement. Dr. Waldsmith has been working in technology – veterinary digital imaging specifically – for decades. To say he's “been working” is another understatement. “Pioneering” is a better word. Dr. Waldsmith was selected as an equine imaging specialist for the 1996 Olympic Games and the 1999 Pan American Games. In 1999, he started Vetel Diagnostics, a company that specializes in diagnostic imaging hardware and software for the veterinary community. As Dr. Waldsmith delved deeper and deeper into technology, he continued to think about how software could be used to make veterinarians more precise and more prompt in their diagnoses. His hope is that better software will translate into improved quality of life for veterinarians while at the same time producing better patient outcomes.

He has channeled his technological interest and experience into a quest for good data that can improve veterinarians' lives. Join him on his journey with this first-person account.

— Bowman Report


Waldsmith's words: First the bad news

There seems to be a rush to market for artificial intelligence (AI) platforms related to diagnostics. This rush has resulted in platforms that don't perform correctly because of their data sets – or lack thereof – and inappropriately built networks.

Veterinarians can post a radiograph on certain websites, and the site will return a diagnostic tree or rule-out list. The problem with these sites is that if you take the same radiograph, adjust the brightness or contrast then resubmit it, the site will return a different diagnostic rule-out list. That's evidence of a database that's improperly curated and too small.

My team and I at Vetel Diagnostics have been working for more than five years to build a radiographic database to enable AI to accurately identify an object and determine that it's normal. Making that “normal” identification takes about 5,000 radiographs of each anatomic area. We need 5,000 lateral radiographic views and 5,000 vertical radiographic views, not to mention needing 5,000 radiographs each for the left, front and back views. That's a total of 25,000 radiographs needed simply to define normal for one anatomic region. The next step is adding thousands more images so the system can detect anomalies.

Technically speaking

AI systems like this work by analyzing images – radiographs, in this case – and determining how close to normal they are. Let's say AI is looking at a radiograph of a dog's lateral thorax. With a complete image database, AI can identify the radiograph as 88 percent normal. It also identifies the specific areas in the image that don't match “normal.” This information tells the veterinarian which areas of the radiograph to examine to determine whether the dog's thorax has an anomaly.

This technology is called machine learning, a component of AI that makes predictions based off patterns. The machine isn't really learning, it's simply been given algorithms that allow it to recognize what should be done in certain circumstances. These algorithms, also known as neural networks, are a sophisticated and growing segment of AI that will impact medicine more and more.

People tend to think computers learn like we do. They don't. We learn in three dimensions, whereas computers learn in 16,000. This is important, because it illustrates how the computer can find information that we don't even realize we don't know.

For example, there's a thorough, curated database of human retinal scans. With this database, the computer can look at a scan and correctly determine with 97 percent accuracy if the person is male or female. An ophthalmologist can't do that, but an ophthalmologist is needed to interpret the information.

The good news

This all points to the fact that AI will not be displacing the veterinarian or diagnostician at all. Let's go back to the radiograph of the dog's lateral thorax. A properly working neural network with a thorough database can analyze the image and report that it's 88 percent normal with abnormalities occurring at specific coordinates in the image. The system provides a rule-out list of the pathology most likely to be identified.

Even with this information, a diagnostician still needs to look at the radiograph and say, “I agree with the rule-out risk. This is the most likely scenario.” Then the diagnostician makes a final diagnosis.

Rather than posing a threat to veterinarians, AI is going to help answer diagnostic questions much faster, giving veterinarians time back to care for patients. I see AI as a tool that will significantly increase quality of life for the veterinary practitioner, plus produce better patient outcomes faster.

Practically speaking

The key use of AI for today's veterinarian is image identification. It's being used primarily in radiography, like I talked about earlier. It's also been used for deciphering urine sediments and cytology samples.

Image processing allows veterinarians to do orthopedic templating and plan orthopedic surgeries ahead of time and add hardware, for example, in orthopedic surgeries. If AI calibrates an image, then it can provide precise measurements.

Another major application for AI in veterinary medicine is in measuring heart sizes. This AI tool can measure the size of a patient's heart relative to body size, assessing the degree of normality and, therefore, disease risks associated with either an enlarged or smaller heart.

Another evolving use of AI is the Norberg hip angle, which measures socket depth of hip joints in medium- and large-breed dogs as an early predictor of hip dysplasia. Automated AI processes mark up the joint and measure socket depth.

In the future, we'll see methods of measuring relative bone density for a predictive index of the age of a skeleton. If the skeleton and chronological age are different, we'll be able to look at why the body is wearing more heavily.

Be skeptical, especially at first

Additional advances won't happen at the speed most people expect. For AI to be effective, we need good data and that's hard to find in veterinary medicine right now. If a platform includes just 3 percent improperly curated or inaccurate data, the whole purpose of AI is defeated.

This means veterinarians need to be cautious about the platforms they adopt. Technologists will be creating a marketing tsunami of messages that appeal to veterinarians' desire for more time, better patient outcomes and perhaps more money. But the veterinary industry has no regulation to ensure AI platforms have good data that deliver appropriately on these promises.

"Any student of Game Theory knows that whenever there's a new game, cheating is rewarded until everyone understands the game and the rules. The veterinary industry doesn't understand it yet."

Any student of Game Theory knows that whenever there's a new game, cheating is rewarded until everyone understands the game and the rules. The veterinary industry doesn't understand it yet. There's a lot of opportunity for people to come into our market with a product that wouldn't see the light of day in the human market because of its level of scrutiny.

When veterinarians evaluate an AI platform, I suggest a few watch-outs.

  • If an AI platform sounds too good to be true, it might be at this juncture.
  • Test the platform to see if it will make you more precise and save time. If it can do these two things, it's worthwhile. If it can't, it's probably best to wait.
  • Validate the data set provided by the vendor, comparing what the neural network says to your own opinion or diagnosis. Start with the basics of recognizing an object in an image. Does the platform recognize the image shows a right front foot? If it does, then look at some measurements the platform provides. Take the same measurements yourself and see whether you agree.
  • Ask the vendor questions. How many thousands of data points does the platform use for each item it's capable of identifying? How many layers is the neural network? What's its back-propagation process or how does it teach itself when it makes a mistake?

As veterinary practice continues to evolve, it becomes more and more associated with business acumen and technology. Veterinarians especially need to cultivate an understanding of technology, because that's what's running the world.

Good Read

Deep Learning by Eric Topol, MD

Dr. Waldsmith's review:
In his book Deep Learning, Eric Topol, a cardiologist at Scripps Research, has done an elegant job of presenting a vast amount of information distilled into the patterns and directions AI is likely to go in medicine. He has a great body of work that provides a better appreciation of what's available in AI today.

Leave a comment