Collaborating Authors


Artificial Intelligence Colonoscopy System Shows Promise


Laird Harrison writes about science, health and culture. His work has appeared in national magazines, in newspapers, on public radio and on websites. He is at work on a novel about alternate realities in physics. Harrison teaches writing at the Writers Grotto.

Drowning in Data


In 1945 the volume of human knowledge doubled every 25 years. Now, that number is 12 hours [1]. With our collective computational power rapidly increasing, vast amounts of data and our ability to assimilate it, has seeded unprecedented fertile ground for innovation. Healthtech companies are rapidly sprouting from data ridden soil at exponential rates. Cell free DNA companies, once a rarity, are becoming ubiquitous. The genomics landscape, once dominated by the few, are being inundated by a slew of competitors. Grandiose claims of being able to diagnose 50 different cancers from a single blood sample, or use AI to best dermatologists, radiologists, pathologists, etc., are being made at alarming rates. Accordingly, it's imperative to know how to assess these claims as fact or fiction, particularly when such claimants may employ "statistical misdirection". In this addition to "The Insider's Guide to Translational Medicine" we disarm perpetrators of statistical warfare of their greatest ...

Artificial intelligence excels at catching pre-cancerous cells


Authors of a new study on detecting precancerous polyps in colorectal cancer screening came to the conclusion that "Artificial Intelligence (AI) may detect colorectal polyps that have been missed due to perceptual pitfalls." They go on to say "By reducing such miss rate, Artificial Intelligence may increase the detection of colorectal neoplasia leading to a higher degree of Colorectal Cancer (CRC) prevention." According to a news release, a team of international researchers led by Mayo Clinic reported that AI "reduced by twofold the rate at which pre-cancerous polyps were missed in colorectal cancer screening." The Mayo Clinic defines a colon polyp as "a small clump of cells that forms on the lining of the colon" and says most are harmless. Yet, it cautions, "over time, some colon polyps can develop into colon cancer, which may be fatal when found in its later stages."

AI reduces miss rate of precancerous polyps in colorectal cancer screening


Most colon polyps are harmless, but some over time develop into colon or rectal cancer, which can be fatal if found in its later stages. Colorectal cancer is the second most deadly cancer in the world, with an estimated 1.9 million cases and 916,000 deaths worldwide in 2020, according to the World Health Organization. A colonoscopy is an exam used to detect changes or abnormalities in the large intestine (colon) and rectum. Between February 2020 and May 2021, 230 study participants each underwent two back-to-back colonoscopies on the same day at eight hospitals and community clinics in the U.S., U.K. and Italy. One colonoscopy used AI; the other, a standard colonoscopy, did not. The rate at which precancerous colorectal polyps is missed has been estimated to be 25%.

Confusion Matrix


In some of my previous blogs I have discussed different machine learning algorithms. Using those algorithms we can build our models. We do data cleansing, pre-processing and then pass the data into our model. The model does the prediction. How to know if the model is good or bad.

Emerging Applications of Artificial Intelligence in Cancer Care - American Association for Cancer Research (AACR)


Now, we trust the complex processes underlying artificial intelligence (AI) with everything from navigation to movie recommendations to targeted advertising. Can we also trust machine learning with our health care? The integration of AI and cancer care was a popular topic in 2021, as evidenced by prominent sessions at two of last year's AACR conferences: the 14th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved, held virtually October 6-8, 2021, and the San Antonio Breast Cancer Symposium (SABCS), held in a hybrid format December 7-10, 2021. During these sessions, experts gave an overview of how machine learning works, shared data on new applications of AI technologies, and emphasized important considerations for making algorithms equitable. Recognizing that a diverse audience of breast cancer clinicians and researchers may have questions about the fundamentals of AI, the SABCS session "Artificial Intelligence: Beyond the Soundbites" opened with a talk titled, "Everything You Always Wanted to Know About AI But Were Afraid to Ask," presented by Regina Barzilay, PhD, the AI faculty lead at the Jameel Clinic of the Massachusetts Institute of Technology.

How to Implement and Evaluate Decision Tree classifiers from scikit-learn


A Decision Tree follows a tree-like structure (hence the name) whereby a node represents a specific attribute, a branch represents a decision rule, and leaf nodes represent an outcome. We will show this structure later so you can see what we mean but you can imagine it is like one of the decision trees you used to draw in high school maths, just on a far more complicated scale. The algorithm itself works by splitting the data according to different attributes at each node while attempting to reduce a selection measure (often the Gini index). In essence, the aim of a Decision Tree classifier is to split the data according to attributes while being able to classify the data accurately into predefined groups (our target variable). For this decision tree implementation we will use the iris dataset from sklearn which is relatively simple to understand and is easy to implement.

Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods Machine Learning

Tree-based models such as decision trees and random forests (RF) are a cornerstone of modern machine-learning practice. To mitigate overfitting, trees are typically regularized by a variety of techniques that modify their structure (e.g. pruning). We introduce Hierarchical Shrinkage (HS), a post-hoc algorithm that does not modify the tree structure, and instead regularizes the tree by shrinking the prediction over each node towards the sample means of its ancestors. The amount of shrinkage is controlled by a single regularization parameter and the number of data points in each ancestor. Since HS is a post-hoc method, it is extremely fast, compatible with any tree growing algorithm, and can be used synergistically with other regularization techniques. Extensive experiments over a wide variety of real-world datasets show that HS substantially increases the predictive performance of decision trees, even when used in conjunction with other regularization techniques. Moreover, we find that applying HS to each tree in an RF often improves accuracy, as well as its interpretability by simplifying and stabilizing its decision boundaries and SHAP values. We further explain the success of HS in improving prediction performance by showing its equivalence to ridge regression on a (supervised) basis constructed of decision stumps associated with the internal nodes of a tree. All code and models are released in a full-fledged package available on Github (

Improving Specificity in Mammography Using Cross-correlation between Wavelet and Fourier Transform Artificial Intelligence

Breast cancer is in the most common malignant tumor in women. It accounted for 30% of new malignant tumor cases. Although the incidence of breast cancer remains high around the world, the mortality rate has been continuously reduced. This is mainly due to recent developments in molecular biology technology and improved level of comprehensive diagnosis and standard treatment. Early detection by mammography is an integral part of that. The most common breast abnormalities that may indicate breast cancer are masses and calcifications. Previous detection approaches usually obtain relatively high sensitivity but unsatisfactory specificity. We will investigate an approach that applies the discrete wavelet transform and Fourier transform to parse the images and extracts statistical features that characterize an image's content, such as the mean intensity and the skewness of the intensity. A naive Bayesian classifier uses these features to classify the images. We expect to achieve an optimal high specificity.

Beyond Visual Image: Automated Diagnosis of Pigmented Skin Lesions Combining Clinical Image Features with Patient Data Artificial Intelligence

Among the most common types of skin cancer are basal cell carcinoma, squamous cell carcinoma and melanoma. According to the who (2018), currently, between 2 and 3 million non-melanoma skin cancers and 132.000 melanoma skin cancer occur every year in the world. Melanoma is by far the most dangerous form of skin cancer, causing more than 75% of all skin cancer deaths (Allen, 2016). Early diagnosis of the disease plays an important role in reducing the mortality rate with a chance of cure greater than 90% (SBD, 2018). The diagnosis of pigmented skin lesions (PSLs) can be made by invasive and non-invasive methods. One of the most common non-invasive methods was presented by Soyer et al. (1987). The method allows the visualization of morphological structures not visible to the naked eye with the use of an instrument called dermatoscope. When compared to the clinical diagnosis, the use of dermatoscope by experts makes the diagnosis of PSLs easier, increasing by 10-27% the diagnostic sensitivity (Mayer et al., 1997).