Medical Imaging

In this post, we will be looking at what is medical imaging, the different applications and use-cases of medical imaging, how artificial intelligence and deep learning are aiding the healthcare industry towards early and more accurate diagnosis. We will review literature about how machine learning is being applied in different spheres of medical imaging and the end implement a binary classifier to diagnose diabetic retinopathy.

What is Medical Imaging?

Medical imaging consists of a set of processes or techniques to create visual representations of the interior parts of the body such as organs or tissues for clinical purposes to monitor health, diagnose and treat diseases and injuries. Moreover, it also helps in creating a database of anatomy and physiology.

Owing to the advancements in the field today medical imaging can achieve information of the human body for many useful clinical applications. Different types of medical imaging technology give different information about the area of the body to be studied or medically treated.  

Organizations incorporating medical imaging devices include freestanding radiology and pathology facilities as well as clinics and hospitals. Major manufacturers of these medical imaging devices include Fujifilm, GE, Siemens Healthineers, Philips, Toshiba, Hitachi, and Samsung. With the advancement and increase in the use of medical imaging, the global market for these manufactured devices for medical imaging is estimated to generate around $48.6 billion by 2025 which was estimated to be $34 billion in 2018.

• 

Why is it so important?

The use of medical imaging for diagnostic services is regarded as a significant confirmation of assessment and documentation of many diseases and ailments. High-quality imaging improves medical decision-making and can reduce unnecessary medical procedures. For example, surgical interventions can be avoided if medical imaging technology like ultrasound and MRI are available.

Earlier diagnosis included exploratory procedures to figure out issues of an aging person, children with chronic pain, detection of early diabetes and cancer. With the advent of medical imaging, the vital information of health can be made available from time to time easily which can help diagnose illnesses like pneumonia, cancer, internal bleeding, brain injuries, and many more.

A study by the National Bureau of Economic Research shows an increment in human life expectancy with incremental use of medical imaging. Therefore, a basic inference can be made that diagnosis and treatment via medical imaging can avoid invasive and life-threatening procedures. Therefore, minimizing the risk caused by these procedures and also help in reducing the cost incurred and time taken by those procedures. A study done by Harvard researchers concluded that $385 spent on medical imaging saves approximately $3000 i.e. a hospital day stay.

Moreover, breast cancer diagnostics through medical imaging have helped medical professionals to prescribe medications that have reduced breast cancer mortality by 22% to 34% (click here). Apart from that, the early medication to stop blood clotting has resulted in a 20% reduction in the death rates owing to colon cancer (click here). Therefore, early detection via effective medical imaging has empowered both the doctors with the opportunity to diagnose ailments early and the patients with the opportunity to fight to live longer.

Medical imaging is an ever-changing technology. With the advancement in the field of computer vision, medical imaging is improving day by day. This means that the benefits of it will keep on improving in the coming time as more and more computer vision researchers and medical professionals are coming together for the advancement of medical imaging

Who does it and for whom?

Doctors perform medical imaging to determine the status of the organ and what treatments would be required for the recovery. The choice of imaging depends on the body being examined and the health concern of the patient. Therefore, patients are tested before if their body reacts affirmatively to the radiation used for medical imaging and making the sure least possible amount of radiation is used for the process. Moreover, proper shielding is done to avoid other body parts from getting affected.

The end-users of medical imaging are patients, doctors, and computer vision researchers as explained below:

• Doctors use it for the organ study and suggest required treatment schedules and also keep the visual data in their library for future reference in other medical cases too.
• Patients are the end-users of treatments received owing to the conclusion derived from the images captured.
• Computer vision researchers along with doctors can label the image dataset as the severity of the medical condition and type of condition post which the using traditional image processing or modern deep learning-based approaches underlying patterns can be captured have a high potential to speed up the inference process from medical images.

Medical Image Analysis

Different kinds and their corresponding approaches

Medical imaging is a part of biological imaging and incorporates radiology which includes the following technologies:

Radiography: One of the first imaging techniques used in modern medicine. It uses a wide beam of X-rays to view non-uniformly composed material. These images help in the assessment of the presence or absence of disease, damage, or foreign object.

• 

Two forms of radiographic images are used in medical imaging which are:

1. Fluoroscopy: Produces real-time internal body part images but requires a constant input of lower dose rate of X-rays. Thus, mainly used in image-guiding procedures where continuous feedback is demanded by the procedure.
2. Projectional Radiography: More commonly used form of X-rays to determine the type and extent of the fracture and pathological changes in the lung. They are used to visualize internal areas around the stomach and intestine and therefore, can help in diagnosing ulcers and a certain type of colon cancer.

MRI – Magnetic Resonance Imaging: MRI scanner uses powerful magnets thereby emitting radiofrequency pulse at the resonant frequency pulse of the hydrogen atoms to polarise and excite hydrogen nuclei of water molecules in human tissue. MRI doesn’t involve X-rays nor ionizing radiation. MRI is widely used in hospitals and seen as a better choice than a CT scan since MRI helps in medical diagnosis without exposing the body to radiation. MRI scans take a long time and are louder. Moreover, people with medical implants or non-removable metal inside the body can’t undergo MRI scans safely.

Ultrasound: Ultrasound uses high-frequency broadband MH range sound waves that are reflected by tissue to varying degrees to produce sort of 3D images. It is most commonly associated with fetus imaging in a pregnant woman. Ultrasound is also used for the imaging of abdominal organs, heart, breast, muscles, tendons, arteries, and veins. It provides less anatomical detail relative to CT or MRI scans. The major advantage is ultrasound imaging helps to study the function of moving structures in real-time without emitting any ionizing radiation. Very safe to use, can be quickly performed without any adverse effects, and relatively inexpensive.

Endoscopy: Endoscopy uses an endoscope that is inserted directly into the organ to examine the hollow organ or cavity of the body. The type of endoscope differs depending upon the site to be examined in the body and can be performed by a doctor or a surgeon. Endoscopy is used to examine the gastrointestinal tract, respiratory tract, ear, urinary tract, etc. The main risks involved with this procedure are infection, over-sedation, perforation, tear lining, and bleeding.

Thermography: Thermographic cameras detect long infrared radiations emitted by the body which create thermal images based on the radiations received. The amount of radiation increases with an increase in temperature. Therefore, thermography helps in checking temperature variations. It is capable of capturing moving objects in real-time. Thermographic cameras are quite expensive. Images of the objects having varying temperatures might not result in accurate thermal imaging of themselves.

Nuclear Medicine Imaging: This type of medical imaging is done by taking radio-pharmaceuticals internally. Then, external gamma detectors capture and form images of the radiations which are emitted by the radio-pharmaceuticals. This is the opposite of X-rays where radiations are through the body from outside but in this case, the gamma rays are emitted from inside the body. However, the radiation dosage ar small still there’s a potential risk.

Tomography: Single-photon emission computed tomography (SPECT) also known as tomography uses gamma rays for medical imaging. The gamma-emitting radioisotope is injected into the bloodstream. SPECT is used for any gamma imaging study which is helpful in treatment, especially for tumors, leukocytes, thyroids, and bones.

We have discussed the important ones above but there are many more medical imaging techniques helping and providing solutions during various medical cases. Techniques such as electroencephalogram(EEG), magnetoencephalogy(MEG), electrocardiography(ECG) which produce data in form of a graph concerning time contain important information of the human body part but can’t be considered as a part of medical imaging directly.

Deep Learning for Medical Imaging

Why Deep Learning over traditional approaches

The Healthcare industry is a high-priority sector where the majority of the interpretations of medical data are done by medical experts. Interpretation of medical images is quite limited to specific experts owing to its complexity, variety of parameters, and most important core knowledge of the subject.

As mentioned in the above section about different medical imaging techniques, the advancement of image acquisition devices has reduced the challenge of data collection with time. Therefore, we are in an age where there has been rapid growth in medical image acquisition as well as running challenging and interesting analysis on them. With the increase in data the burden on medical experts examining that data increases. Therefore, the probability of human error might increase.

Moreover, traditional machine learning can’t comprehend the complexity of such healthcare-oriented problem statements owing to the complexity and importance of the subject.

• The best we had to date, was traditional machine learning applications in computer vision which relied heavily on features crafted by medical experts who are the subject matter people of the concerned field.
• This is a labor-intensive process, as data varies from patient to patient and data comprehension varies with the experience of the medical expert too. Therefore, traditional learning methods were not reliable.
• These earlier machine learning algorithms of Logistic Regression, Support Vector Machines(SVMs), K-Nearest Neighbours(KNNs), Decision Trees, etc. used to take raw image data into account without any learning of hidden representations.
• Moreover, the preprocessing was based on the knowledge provided by the medical expert which was very time-consuming.
• As mentioned above, image acquisition devices like X-Ray, CT and MRI scans, etc. have improved over time and can fetch internal images of high resolution. But automated image interpretation is a tough ordeal to achieve.

On the other hand, deep learning in computer vision has shown great progress in capturing hidden representations and extract features from them. These feature extractions improve with better data and supervision so much that they can help diagnose a physician efficiently. The deep learning techniques are composed of algorithms like Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM) Networks, Generative Adversarial Networks (GANs), etc which don’t require manual preprocessing on raw data.

Diabetic Retinopathy  

There are two types of disorders owing to diabetes. Diabetes Mellitus is a metabolic disorder where Type-1 being the case in which the pancreas can’t produce insulin and Type-2 in which the body doesn’t respond to the insulin, both of which lead to high blood sugar. Diabetic Retinopathy is an eye disorder owing to diabetes resulting in permanent blindness with the severity of the diabetic stage.

According to World Health Organisation(WHO)

• The number of people suffering from diabetes has increased from 108 million in 1980 to 422 million in 2014.
• The disease is increasing in low and medium-income countries.
• Diabetes is the major cause of blindness, kidney failure, heart attacks, stroke, and lower limb amputation.
• In 2016, approximately 1.6 million deaths were due to diabetes and this approximation is estimated to rise to 2.2 million for the year 2022 due to high blood glucose levels.
• Diabetic retinopathy is an important cause of blindness and occurs as a result of long-term accumulated damage to the small blood vessels in the retina. 2.6% of global blindness can be attributed to diabetes.

Diabetic retinopathy can be controlled and cured if diagnosed at an early stage by the retinal screening test. Manual processes to detect diabetic retinopathy are time-consuming owing to equipment unavailability and expertise required for the test. The issue being the disease doesn’t show any symptoms at an early stage owing to which ophthalmologists need a good amount of time to analyze the fundus images which in turn causes a delay in treatment.

• Deep learning-based automated detection of diabetic retinopathy has shown promising results. Gulshan et al. research where they applied Deep Convolutional Neural Network (DCNN) on the following two datasets for the classification between moderate and worse Referable Diabetic Retinopathy (RDR): Eye Picture Archive Communication System (EyePACS-1) dataset: 9963 images from 4997 patients and the Messidor-2 dataset: 1748 images from 874 patients. The algorithm devised by Gushan et al. claimed to have achieved 97.5% sensitivity and 93.4% specificity on EyePACS-1 data, and 96.1% sensitivity and 93.9% specificity on Messidor-1. The specific neural network used in this work is the Inception-v3 architecture proposed by Szegedy et al.

Google AI team worked closely with doctors both in India and the US to create a dataset of 128,000. Each of which was evaluated by 3-7 ophthalmologists from a panel of 54 ophthalmologists. This dataset was used to train a deep neural network to detect RDR. The algorithm’s performance was further tested on two separate validation sets (totaling approx. 12000 images), with the decision of a panel of 7 or 8 U.S. board-certified ophthalmologists serving as the reference standard. The performance was mind-blowing and matched that of the ophthalmologists. On validation set the algorithm performed with an F1-score of 0.95, better than the median F1-score of the 8 ophthalmologists (measured at 0.91) who were consulted for the research.

Examples of retinal fundus photographs that are taken to screen for DR. The image on the left is of a healthy retina (A), whereas the image on the right is a retina with referable diabetic retinopathy (B) due to several hemorrhages (red spots) present.

Kathirvel et al trained a DCNN with dropouts on publicly available Kaggle, DRIVE, and STARE dataset to classify affected and healthy fundus which reported an accuracy of 94%. Kaggle dataset includes clinician labeled images across 5 classes namely: No DR, Mild, Moderate, Severe, and Proliferative DR.

The advancements in the methods of automated diabetic retinopathy screening methods with high metrics pose a strong potential to assist doctors in evaluating more patients and speed up the diagnostic process which in turn can reduce the time gap for treatments. Google is trying hard to work with doctors and researchers to streamline the screening process across the world with the hope that these methods can benefit maximally both patients as well as doctors. Moreover working with the FDA and other regulatory agencies to further evaluate these technologies in clinical studies to make this a standard part of the procedure.

Histological and Microscopial Elements Detection

Histological analysis is the study of the cell, group of cells, and tissues. Microscopic imaging technology and stains are used to detect the microscopic changes occurring at cellular and tissue level. It involves steps that include fixation, sectioning, staining, and optical microscopic imaging.

Microscopical imaging is used for diseases like squamous cell carcinoma, melanoma, gastric carcinoma, gastric epithelial metaplasia, breast carcinoma, malaria, intestinal parasites, etc. The Genus Plasmodium parasite is the main cause of malaria and microscopical imaging is the standard method for parasite detection in blood smear samples. Mycobacteria in sputum are the main cause of Tuberculosis. Smear microscopy and fluorescent auramine-rhodamin stain or Ziehl-Neelsen stain are standard methods for Tuberculosis diagnosis.

• In 2016, the Department of Computer Science of the University of Warwick opened the CRCHistoPhenotypes – Labeled Cell Nuclei Data. The data includes 100 H&E stained histology images of colorectal adenocarcinomas. Out of which a total of 29,756 nuclei were labeled for detection and 24,444 nuclei among them had class labels associated with them. These class labels were epithelial, inflammatory, fibroblast, and miscellaneous.
• Sirinukunwattana et al show the research where they used the CRCHistoPhenotypes datasets to train a deep learning model for the detection and classification of colon cancer. The research went in the favour of the proposed spatially-constrained CNN for nucleus detection and the softmax CNN with the proposed neighboring ensemble predictor for nucleus classification. The combination of the two has the potential to benefit the analysis of tissue morphology, and tissue constituents eventually can become a useful tool for a better understanding of the tumor microenvironment.

Example patches of different types of nuclei found in the dataset used by Sirinukunwattana et al, 1st Row – Epithelial nuclei, 2nd Row – Inflammatory nuclei (from left to right, lymphocyte, plasma nucleus, neutrophil, and eosinophil), 3rd Row – Fibroblasts, 4th Row – Miscellaneous nuclei (from left to right, adipocyte, endothelial nucleus, mitotic figure, and necrotic nucleus).

• Bayramoglu et al aimed at the issue of limited availability of training data. This research checks the possibility of using transfer learning to minimize the hurdle owing to fewer data and still obtain a well-hidden representation with limited data availability.
• Quinn et al research employed a setup to capture blood smear images, sputum samples images, and stool samples images. The experts identified bounding boxes around each object of interest in every image. In thick blood smear images, plasmodium was annotated (7245 objects in 1182 images); in sputum samples, tuberculosis bacilli were annotated (3734 objects in 928 images), and in stool samples, the eggs of hookworm, Taenia, and Hymenolepsis nana were annotated (162 objects in 1217 images). Automatic microscopic diagnostic analysis was done by training Deep Convolutional Neural Networks (DCNNs) separately for each which resulted in a high AUC of 1.00 for plasmodium images to detect Malaria, 0.99 for tuberculosis bacilli, and 0.99 for hookworm detection. In this work, DCNN used comprised of four hidden layers:
• Convolution layer: 7 filters of size 3 × 3.
• Pooling layer: max-pooling, factor 2.
• Convolution layer: 12 filters of size 2 × 2.
• Fully connected layer, with 500 hidden units.

ROC and precision-recall for malaria, tuberculosis, and intestinal parasites detection, showing Area Under Curve (AUC) and Average Precision (AP)

Malaria detection is highly crucial and important. According to 2018 reports by World Health Organisation(WHO), in 2018, an estimated 228 million cases of malaria occurred worldwide out of which there were an estimated 405,000 deaths from malaria globally. Children aged under 5 years are the most vulnerable group affected by malaria. In 2018, they accounted for 67% (272,000) of all malaria deaths worldwide.

Gastrointestinal Diseases Detection

The gastrointestinal tract consists of all the organs which are involved in the digestion of food and nutrient absorption from them starting from mouth to anus. The organs included are the esophagus, stomach, duodenum, large intestine(colon), and small intestine(small bowel). Oesophagus, stomach, and duodenum constitute the upper gastrointestinal tract while the large and small intestine forms the lower gastrointestinal tract.

Digestion and absorption get affected by the disorders like inflammation, bleeding, infections and cancer in the gastrointestinal tract. Ulcers cause bleeding in the upper gastrointestinal tract. Polyps, cancer, or diverticulitis cause bleeding from the large intestine. Celiac, Crohn, tumors, ulcers, and bleeding owing to abnormal blood vessels are the issues concerned with the small intestine.

Current imaging technologies play a vital role in diagnosing these disorders concerned with the gastrointestinal tract which include endoscopy, enteroscopy, wireless capsule endoscopy, tomography, and MRI.

• Jia et al trained a DCNN to detect bleeding in wireless capsule endoscopy(WCE) images. The domain of computer-aided diagnosis for gastrointestinal bleeding detection is an active area of research. Traditional methods were based on handcrafted features which are insufficient for blood detection with higher accuracy owing to very less capture of features. Therefore, the deep convolutional neural network was applied to an expanded dataset of 10,000 WCE images. With the resultant F1-score of 0.9955, the DCNN based method outperformed the state-of-the-art approaches in WCE bleeding detection.

Examples of WCE images in our dataset. (a) A normal WCE image. (b) An active bleeding WCE image (c) An inactive bleeding WCE image.

DCNN architecture used by Jia et al to detect bleeding in wireless capsule endoscopy

• Pampering et al employ DCNN to detect intestinal hemorrhage in WCE images. WCE can painlessly capture a large number of internal intestinal images. However, only a small portion of these WCE images contain hemorrhage. Therefore, to perform automated detection of intestinal hemorrhage data augmentation methods were employed to create a balanced dataset. As a result, the DCNN model achieved an F1-score of 0.9887.
• Leenhardt et al employed DCNN to detect gastrointestinal telangiectasia(GIA) in small bowel capsule endoscopy(SBCE) images of the small intestine. GIA is the most common small intestinal vascular lesion with an inherent risk of bleeding. SBCE is the currently accepted procedure. Still, frames of SBCE were annotated with typical GIA and normal which were given to a DCNN to do semantic segmentation and classification. The algorithm devised received a sensitivity of 100% and a specificity of 96%. The success of this deep learning-based approach paved the way for the future development of software for automated detection of GIA from SBCE images.
• Urban et al employed DCNN to localize and detect polyps in screening colonoscopies. Colonoscopy is done for colorectal cancer prevention depends on the adenoma detection rate (ADR). The ADR by colonoscopists varies from 7% to 53%. It is estimated that with a 1% increase in ADR risk of interval colorectal cancers decreases by 3%-6%. To develop new methods to increase the ADR a DCNN was trained on a diverse and representative set containing 8,641 images from screening colonoscopies of more than 2000 patients labeled by expert colonoscopists. The resulted algorithm gave an AUC of 0.991 and an accuracy of 96.4%. This success has given hope to create an automated diagnostic system to increase the ADR to decrease interval colorectal cancers but requires validation in large multicenter trials.

Examples of the dataset used by Urban et al. (Top row) Images containing a polyp with a superimposed bounding box. (Bottom row) Nonpolyp images. Three pictures on the left were taken using NBI and 3 pictures on the right include tools (eg, biopsy forceps, cuff devices, etc) that are commonly used in screening colonoscopy procedures.

Cardiac Imaging

CT and MRI scans are the most widely used technology for cardiac imaging. The uphill task being the manual identification of the coronary artery calcium (CAC) scoring in cardiac CT scans which incorporates a good amount of effort. Therefore, making it to be a time-consuming task for epidemiological studies.

• Litjens et al research paper discusses different deep learning frameworks which can be used in different ways in cardiovascular analysis which has been shown in the diagrammatic representation below.

This flowchart highlights how certain applications can be realized by using a specific algorithm. The arrows indicate for which applies an algorithm is typically used. Note that this does not mean that, for example, a fully connected network cannot be used for segmentation, but it is not the most appropriate choice.

Tumor Detection

Abnormal growth of cells of any body part creating a separate mass of tissue. This is called tumor or neoplasm. Generally, cells in our body undergo a cycle of developing, aging, dying, and finally replaced by new cells. This cycle gets disrupted in the case of tumors and other forms of cancer. There are two types of tumor: Benign (non-cancerous) and Malignant (cancerous). A benign tumor is not that dangerous and sticks to one part of the body and does not spread to other parts. On the other hand, a malignant tumor is extremely harmful spreading to other body parts. Spreading of the malignant tumor makes both treatment and prognosis difficult.

• Wang et al are one of the initial research to detect breast cancer in digital mammography using machine learning. They used 482 mammographic images out of which 246 had tumors. Their patients were women from North-East China between the ages of 32 and 74. This image database was built using a sonographer 2000D full digital breast X-ray camera and confirmed by radiologists at Tumor Hospital of Liao Ning Province. The machine learning approaches involved two models one being the single-layered neural network which was named extreme machine learning (ELM) and the second being the traditional support vector machine (SVM). The images underwent through series of preprocessing techniques like noise reduction, edge enhancement, edge segmentation which were followed by geometrical and textural feature extraction. These features were fed into the model and trained via a cross-validation approach. The results obtained had 84 error responses for ELM and 96 error responses for SVM out of 482 images. This research done in 2012 didn’t use CNN based approach but paved the way for the idea to use deep learning to do automated breast cancer detection in digital mammography.

Shen et al in 2019 presented a research report where they applied DCNN on mammographic images to improve breast cancer detection. The DDSM contains digitized film mammograms in a lossless JPEG format now being outdated. Therefore an updated version of the database containing the images in standard DICOM format called CBIS-DDSM was used. The dataset used consisted of 2478 mammography images from 1249 women who had both craniocaudal (CC) and mediolateral oblique (MLO). Each view was treated as a separate image in this study and was randomly split in the ratio of 85:15 at the patient level to create independent training and test sets. The training data was further split 90:10 to create a validation set. The splits maintained a similar proportion of cancer cases in all the training, validation, and test sets. The total numbers of images in the training, validation, and testing sets were 1903, 199, and 376, respectively. Two patch datasets were created by sampling image patches from ROIs and background regions. All patches had the same size of 224 × 224, which were large enough to cover most of the ROIs annotated. The first dataset (S1) consisted of sets of patches in which one was centered on the ROI and one is a random background patch from the same image. The second dataset (S10) consisted of 10 patches randomly sampled from around each ROI, with a minimum overlapping ratio of 0.9 with the ROI and inclusion of some background, to more completely capture the potentially informative region; and an equal number of background patches from the same image. All patches were classified into one of the five categories: background, malignant mass, benign mass, malignant calcification, and benign calcification. The training consisted of two steps. The first step is to train the patch classifier to detect the region of interest(ROI) from the background using a DCNN. The second step included training an image classifier which would run on the ROI images obtained from the first step. The second step classifies the ROI image into one of the five categories which are background, malignant mass, benign mass, malignant calcification, and benign calcification. The training for step two was done using Resnet-50 and VGG-16, confusion matrix of the results of which are shown in the figure below.

Architecture used by Shen et al. Converting a patch classifier to an end-to-end trainable whole image classifier using an all convolutional design. The function f was first trained on patches and then refined on whole images.

Confusion matrix analysis of 5-class patch classification for Resnet50 (a) and VGG16 (b) in the S10 test set. The matrices are normalized so that each row sums to one.

Alzheimer’s and Parkinson’s Detection

Alzheimer’s disease(AD) is a brain disorder that is irreversible and slow progresses to destroy memory and thinking skills hampering the ability to carry out simple tasks. Accurate diagnosis of AD plays an important role in patient care particularly in the early phase of the disease.

Parkinson’s disease is a neurological disorder causing the progressing decline in the motor system due to the disorder of basal ganglia in the brain. The symptoms start with tremors in the hand followed by slow movement, stiffness, and loss of balance.

• Sarraf et al employed LeNet-5(a basic CNN model) to detect Alzheimer’s disease in fMRI data. The model was trained on 270900 images and tested on 90300 images in fMRI which finally obtained a mean accuracy of 96.8588%.

Jha et al employed sparse autoencoders for the detection of Alzheimer’s disease. The technique required fewer labeled training examples and minimal prior knowledge. In this study, the deep learning model used, consisted of sparse autoencoders, scale conjugate gradient (SCG), stacked autoencoder, and a softmax output layer to classify the condition being in the prodromal or mild stage. The algorithm devised resulted in the inaccuracy of 91.6% with a sensitivity of 98.09% and a specificity of 84.09%.

Ortiz et al employed DCNN to detect Parkinson’s disease.  A total of 269 DaTSCAN images from the PPMI (Parkinson’s Progression Markers Initiative) database were used in the preparation of the article. Out of 269, 158 were suffering from Parkinson’s disease and 111 were normal. The approach undertook was to create isosurfaces from DaTSCAN images as the raw images are too complex for CNN architectures like LeNet-5 and AlexNet. This approach resulted in an average accuracy of 95.1% and an AUC of 0.97. This research was able to conclude that computation of isosurfaces helped in better convergence of CNN which in turn resulted in better and faster-automated detection of Parkinson’s disease in DaTSCAN images.

Current Scenario and Challenges

Deep learning implementation in medical imaging makes it a more disruptive technology in the field of radiology. Medical fields which have shown promises to be revolutionized using deep learning are:

• Ophthalmology
• Pathology
• Cancer diagnosis

Google DeepMind Health and National Health Service, UK have signed an agreement to process the medical data of 1 million patients.

IBM Watson has entered the imaging domain after their successful acquisition of Merge Healthcare

The application of deep learning algorithms to medical imaging is fascinating and disruptive but many challenges are pulling down the progress. Some of the major challenges are as follows:

Limited Datasets

The first and the major prerequisite to use deep learning is a massive amount of training dataset as the quality and evaluation of deep learning-based classifier relies heavily on the quality and amount of the data. Limited availability of medical imaging data is the biggest challenge for the success of deep learning in medical imaging.

The development of a massive training dataset is itself a laborious time-consuming task that requires extensive time from medical experts. Therefore, more qualified experts are needed to create quality data at a massive scale, especially for rare diseases. Moreover, a balanced dataset is necessary for deep learning algorithms to learn the underground representations appropriately. In healthcare majority of the available dataset is unbalanced leading to class imbalance.

Privacy and Legal Issues

Sharing of medical data is severely complex and difficult compared to other datasets. Data privacy is both sociological as well as a technical issue, which needs to be addressed from both angles.

HIPAA (Health Insurance Portability and Accountability Act of 1996) provides legal rights to patients to protect their medical records, personal and other health-related information provided to hospitals, health plans, doctors, and other healthcare providers.

Therefore, with the increase in healthcare data anonymity of the patient information is a big challenge for data science researchers because discarding the core personal information makes the mapping of the data severely complex but still a data expert hacker can map through a combination of data associations.

Differential privacy approaches can be undertaken which restricts the data to the organization on a requirement basis. Sharing sensitive data with limited disclosure is a real challenge. Therefore, it leads to a lot of restrictions. Limited data access owing to restriction reduces the amount of valuable information. Apart from that, the data is increasing day by day adding incremental threats to data security.

Process Standardization (Data and Models)

1. Data Standards: Standardization of data is the need of the hour for deep learning in any domain especially for healthcare. The reason behind this is that variability of the data increases from hardware to hardware which loses consistency causing variations in the data(here, images) captured. Healthcare is a domain where aggregations of data from different sources are required for improved learning and better accuracy. Health data must be standardized and shared between providers. HIPAA, HL7(Health Level 7), HITECH(Health Information Technology for Economic and Clinical Health Act of 2009), and other health standardization and monitoring bodies have defined some standard maintaining guidelines. Authorized Testing and Certifying Body (ATCB) provides third-party opinion on EHR(Electronic Health Records).
2. Uninterpretable Black Box Model: Deep learning opened new avenues in the domain of medical imaging leading to new opportunities. Deep learning solved the complexity which wasn’t possible with traditional machine learning-based approaches. One of the biggest roadblocks being the black-box model. The maths behind neural networks is crystal clear but weight matrices created with the increase in layer depth make the model uninterpretable.