fbpx
Connect with us

MedTech

Shortfalls arise in machine learning’s COVID-19 detection, researchers find

Inside Telecom Staff

Published

 on

machine learning

A recent paper published in Nature Machine Intelligence has found several common yet serious flaws with machine learning (ML) made for COVID-19 diagnosis and prognosis.

At the start of the worldwide pandemic, companies such as DarwinAI, as well as major U.S. chipmaker Nvidia and groups such as the American College of Radiology kickstarted efforts to produce technology that can discover the COVID-19 virus from simple CT scans, X-rays, and other forms of medical imaging.

The technology is meant to help health workers on the frontlines to differentiate between COVID and pneumonia, while presenting a clearer patient diagnosis; some of these technologies were said to predict if a person will die or needed a ventilator based on a CT scan. 

However, a consortium of artificial intelligence (AI) researchers along with an assorted healthcare professional from the universities of Cambridge and Manchester specialized in infectious diseases, radiology, and ontology have uncovered a number of shortfalls within the tech; claiming that major changes are needed before this form of machine learning can be used in a clinical setting.

Authors of the scientific paper examined more than 2,200 papers, which was later narrowed down to 320, given the focus group full text review for quality, following the removal of duplicates and irrelevant titles.

At the end, only 62 papers were considered fit to be part of what the authors called a systematic review of published research and preprints shared on open research paper repositories such as arXiv, bioRxiv, and medRxiv.

During the analysis, roughly half of the 62 papers made no attempt to perform external validation of training data, did not assess model sensitivity or robustness, and did not report the demographics of people represented in training data.

According to the scientific paper, “Frankenstein” datasets – which are made with duplicate images assembled from other datasets and redistributed under a new name – were a cause for concern since only one in five COVID-19 diagnosis or prognosis models shared their code so others can reproduce results claimed in literature.

“This repackaging of datasets, although pragmatic, inevitably leads to problems with algorithms being trained and tested on identical or overlapping datasets while believing them to be from distinct sources,” the authors noted.

Authors later highlighted that in their current reported form, none of the machine learning models included in this review are likely candidates for clinical translation for the diagnosis/prognosis of COVID-19.

“Despite the huge efforts of researchers to develop machine learning models for COVID-19 diagnosis and prognosis, we found methodological flaws and many biases throughout the literature, leading to highly optimistic reported performance,” the research paper read.

Researchers also found that with ML models developed via medical imaging data was virtually no assessment for bias and generally being trained with sufficient images. It is important to note that nearly every paper reviewed was revealed to be at high or uncertain risk of bias, with only six considered at low risk of bias.

In parallel, publicly available datasets are known to suffer from low quality image formats, while not being large enough to train reliable AI models. According to the paper’s methodology, researchers used the checklist for artificial intelligence in medical imaging (CLAIM) and radiomics quality score (RQS) to help assess the datasets and models.

“The urgency of the pandemic led to many studies using datasets that contain obvious biases or are not representative of the target population. Before evaluating a model, it is crucial that authors report the demographic statistics for their datasets, including age and sex distributions,” the paper said.

The authors stressed that higher-quality datasets, manuscripts with sufficient documentation to be reproducible and external validation are required to increase the likelihood of models being taken forward and integrated into future clinical trials to establish independent technical and clinical validation as well as cost-effectiveness.

Advertisement

We’re a diverse group of industry professionals from all corners of the world. Our desire is to provide a high-quality telecoms publication that caters to an international market, offering the latest and most relevant telecoms information to businesses, entrepreneurs and enthusiasts.

MedTech

EU throws weight behind Pfizer-BioNTech and new technology

Associated Press

Published

 on

EU throws weight behind Pfizer-BioNTech and new technology

In a stinging rebuke to pharma giant AstraZeneca Wednesday, the European Union announced plans to negotiate a massive contract extension for Pfizer-BioNTech’s COVID-19 vaccine insisting the 27-nation bloc had to go with companies that had shown their value in the pandemic.

“We need to focus on technologies that have proven their worth,” said EU Commission President Ursula von der Leyen. She also announced that America’s Pfizer and Germany’s BioNTech would provide the EU with an extra 50 million doses in the 2nd quarter of this year, making up for faltering deliveries of AstraZeneca.

In contrast to the oft-criticized Anglo-Swedish company, von der Leyen said Pfizer-BioNTech “has proven to be a reliable partner. It has delivered on its commitments, and it is responsive to our needs. This is to the immediate benefit of EU citizens.”

Exacerbating the problems for AstraZeneca, Denmark decided Wednesday not to resume use of its vaccine, after putting it on hold last month following reports of rare blood clots in some recipients. The bulk of the shots given in the Scandinavian country so far have been the Pfizer-BioNTech vaccine.

The Johnson & Johnson jab, which uses the same base technology as AstraZeneca, hit a snag this week when U.S. regulators recommended a “pause” in administering Johnson & Johnson shots. Deliveries in the EU have been suspended.

AstraZeneca was supposed to be the workhorse of the EU’s vaccine drive this year — a cheap and easy-to-transport shot to break the pandemic’s back. Yet, the EU said that out of 120 million doses promised for the 1st quarter, only 30 million were delivered, and, of the 180 million expected, now there are only 70 million set for delivery in the 2nd quarter.

Because of that shortfall, the EU has come under crushing pressure as, even though it it is a major producer and exporter of vaccines, it cannot get its vaccinations even close to the levels of the United Kingdom and the United States.

The Our World in Data site said 47.5% of people in the U.K. have received at least one dose of COVID-19 vaccine, compared to 36.6% in the U.S. and 16.4% in the EU.

Now, Pfizer-BioNTech could well become the key to beat the pandemic on the continent.

With 200 million doses already earmarked for the bloc this quarter from Pfizer-BioNTech. the 50 million additional deliveries will be especially welcomed by EU nations dealing with supply delays and concerns over rare blood clots potentially linked to the Oxford-AstraZeneca vaccine.

Von der Leyen said the EU will start negotiating to buy 1.8 billion doses of the Pfizer-BioNTech vaccine through 2023.

“It will entail that not only the production of the vaccines, but also all essential components, will be based in the EU,” von der Leyen said.

The European Commission currently has a portfolio of 2.3 billion doses from half a dozen companies and is negotiating more contracts.

Von der Leyen expressed full confidence in the technology used for the Pfizer-BioNTech vaccine, which is different from that behind the Oxford-AstraZeneca vaccine.

The active ingredient in the Pfizer-BioNTech shot is messenger RNA, or mRNA, which contains the instructions for human cells to construct a harmless piece of the coronavirus called the spike protein. The human immune system recognizes the spike protein as foreign, allowing it to mount a response against the virus upon infection.

Astra Zeneca’s is made with a cold virus that sneaks the spike protein gene into the body. It’s a very different form of manufacturing: Living cells in giant bioreactors grow that cold virus, which is extracted and purified.

Von der Leyen said Europe needs to have a technology that can boost immunity, tackle new variants and produce shots quickly and massively. “mRNA vaccines are a clear case in point,” she said.

The planned negotiations with Pfizer left in the middle what the EU would do about any new contracts with AstraZeneca. “Other contracts, with other companies, may follow,” said von der Leyen.


BRUSSELS (AP) — By RAF CASERT Associated Press
Jan M. Olsen contributed from Copenhagen.

Continue Reading

MedTech

UK NHS Cancer backlog treated by digital tools, says report

Karim Hussami

Published

 on

digital app

A report from ORCHA (The Organisation for the Review of Care and Health Apps), an NHS partner and a leading authority on health app trends and usage, says that MedTech digital tools can be part of the solution to the backlog in cancer services.

The report adds that patients must be supported by healthcare staff in their choice of apps and be extremely wary of poor-quality tools which could damage their health.

“There are excellent apps or digital tools supporting cancer patients. These have been developed with clinicians, rigorously reviewed and frequently updated. Apps such as these can be embedded into cancer services to provide tremendous support to patients and ease the healthcare system at a time of tremendous backlog,” former NHS clinician and founding CEO of ORCHA, Liz Ashall-Payne, said.

“For example, BELONG, Beating Cancer Together gives users access to oncologists, radiologists and doctors to answer questions and notifies users of available clinical trials around the world. Vinehealth Cancer Companion helps patients monitor their symptoms and track their medication,” he noted.

Around 40,000 fewer people started cancer treatment in 2020 due to COVID-19, putting potential pressure on services for years to come.

“We believe there is massive potential for intelligent apps such as these both to help patients and provide excellent returns on investment to the NHS,” Liz added.

According to ORCHA research, 3,603 digital tools in forms of apps to support cancer can be found in app stores. Worryingly, 74 percent of these have not been updated in the last 18 months. This means the vast majority have not kept pace with medical, data or usability guidelines.

Amongst the apps updated within 18 months, ORCHA reviewed 190 of the most downloaded, testing them against more than 350 health standards and measures including elements of the NICE framework. This diligence revealed that only 24.7 percent of the apps reviewed meet minimum quality thresholds.

Liz Ashall-Payne, said: “These statistics are deeply concerning, given how easy it is for vulnerable patients and care providers to search app stores and stumble across apps which may give poor or outdated advice or blatantly misuse their private data.”

Continue Reading

MedTech

Wearable tech start-up aims to tackle head injuries in sport

Karim Hussami

Published

 on

Wearable tech start-up

An Edinburgh-based business recently launched a Kickstarter campaign to help move into production of a technology to safeguard athletes against head injuries.

Based at the Edinburgh Business School (EBS) Incubator within Heriot-Watt University, the technology has been developed by start-up company HIT. The concept is wearable tech which measures and tracks head impact force in sport and recreational activities and is set to aid research and support informed decisions on the risk of brain injury.

Founder Euan Bowen, an avid rugby player, was inspired to develop the technology a teammate was injured. With brain injuries rarely reported, Bowen spotted a gap in the market for sportspeople to track brain health.

Bowen explained: “I found little technology available to monitor head impact, despite the severity of the issue across different sports.

“As a member of a rugby club in Edinburgh, I began researching and developing a project, working closely with the team to develop an initial prototype.”

Featuring a unique impact sensor, wearable across multiple sporting and activity applications, the device universally clips onto any helmet or halo headband, detecting g-force and recording impact via a companion app.

Using a traffic light system, the app records data and acts as an early warning notification for the user regarding the level of impact force recorded – highlighting the caution required in continued exercise.

“High impact sports are focusing increasingly on concussion mitigation with the Field – ‘Football’s Influence on Lifelong Health and Dementia Risk’ – study recently finding that former professionals are three and a half times more likely to die of dementia than the general population,” Bowen noted.

Kallum Russell, manager of the EBS Incubator said: “HIT Impact is a much-needed technology to track and support the current efforts to increase sports safety at a time when governing bodies across high impact sports are increasingly focused on minimizing head trauma.”

The current parliamentary inquiry into concussion recently heard evidence about the long-term implications of repetitive head trauma on sports people with MPs asking how sports could be made safer.

Continue Reading

Trending