Validating Artificial Intelligence in Clinical Dermatology: From Proof of Concept to Clinical Tool

This article describes peer-reviewed academic research. Dermi Atlas is a clinical documentation platform; the methods described here are not part of it.

When the term "artificial intelligence" was coined for the 1956 Dartmouth workshop, few would have pictured machines estimating the extent of skin disease. Modern AI, built on neural networks whose interconnected nodes loosely mimic biological synapses, has proven remarkably adept at learning complex, nonlinear patterns in visual data. And few medical specialties are as visual as dermatology.

Dermatologists diagnose and monitor skin diseases largely by looking at them. That reliance on visual pattern recognition is both the specialty's strength and its vulnerability: how accurately a clinician assesses a rash depends heavily on their training, experience, and even how tired they are that day. Two dermatologists examining the same patient can arrive at meaningfully different severity estimates, and this inter-rater variability has real consequences for treatment decisions.

Nowhere is this problem more apparent than in Body Surface Area (BSA) estimation, the percentage of a patient's skin affected by disease. BSA underpins virtually every major severity index in dermatology: the Psoriasis Area and Severity Index (PASI), Eczema Area and Severity Index (EASI), SCORing Atopic Dermatitis (SCORAD), and Vitiligo Area Scoring Index (VASI). The standard clinical approach, the "palm method," assumes that one patient palm equals roughly 1% of BSA. But planimetric studies have shown the actual figure is closer to 0.78%, a built-in error that cascades through every calculation that depends on it.

Various computer-based alternatives have been tried: automated total-body imaging (ATBI), Computer-measured Psoriatic Area (ComPA), and the optical pencil (OP) method, among others. Each moved the needle toward objectivity, but none struck the right balance of accuracy, cost, and clinical practicality to gain wide adoption.

It was in this context that Breslavets and colleagues posed a simple question: could an artificial neural network estimate psoriasis BSA more accurately than a trained dermatologist?

The Validation Study

The study design was deliberately straightforward. Breslavets et al. assembled 130 de-identified photographs of psoriasis from a community-based dermatology clinic, then asked four evaluators to estimate the affected skin percentage in each image: a board-certified dermatologist, two undergraduate science students with no specialized dermatology training, and a trained artificial neural network (ANN).

The ANN was a 17-layer convolutional neural network (CNN) built in Python with TensorFlow, accepting input images at 224x224 or 448x448 pixels. It had been trained on over 2,000 unaltered psoriasis photographs, each paired with a manually annotated mask outlining healthy versus affected skin. The network's job was conceptually simple: count the pixels classified as psoriasis, count the pixels classified as healthy skin, and compute the ratio. Despite this apparent simplicity, the task directly mirrors what clinicians attempt during physical examination, only with pixel-level granularity instead of gestalt estimation.

An important practical detail: the team deployed the tool both as a desktop application and as an Android mobile app. If AI assessment tools are going to matter clinically, they need to work where clinicians actually are: in exam rooms, at bedsides, on the go. Statistical comparisons used the Mann-Whitney-Wilcoxon test, a non-parametric method well-suited to the likely skewed distribution of estimation errors.

Results

The numbers were unambiguous. The ANN achieved a mean percentage error (MPE) of 8.95% (SD 13.94%, 95% CI [6.55, 11.34]). The board-certified dermatologist came in at 27.84% (SD 15.89%, 95% CI [25.11, 30.57]), roughly three times the neural network's error. Participant 1, an undergraduate student, was close behind the dermatologist at 29.23% MPE. Participant 2 recorded an MPE of 162.1%, suggesting a fundamental misunderstanding of the task or a systematic calibration problem.

All pairwise comparisons between the ANN and human evaluators were statistically significant (p < 0.05). The mobile application produced identical results to the desktop version, confirming that the deployment platform did not compromise performance.

What stands out is not just that the ANN was better, but how much better. This was not a marginal improvement requiring large samples to detect; it was a clear, clinically meaningful gap. A machine learning model trained on a few thousand labeled images could consistently beat an experienced physician at a core clinical measurement task.

Broader Context

These results did not emerge in a vacuum. By 2022, AI had already made inroads across several areas of dermatology. Phillips et al. and Esteva et al. had reported AI classification of skin lesions from dermoscopic and clinical images.

The smartphone app landscape told a more mixed story. Published evaluations of AI-powered dermatology apps reported sensitivity and specificity that varied widely from app to app, a spread that said more about the inconsistency of the field than about the technology itself.

On the adoption side, though, clinicians were broadly receptive: surveys of dermatologists reported mostly favorable attitudes toward AI, and few respondents expected to be replaced.

Significance of the Study

First, training did not close the gap: the students' poor performance was expected, but the board-certified dermatologist also trailed the network, which pointed to the limits of human visual estimation for quantitative tasks rather than to a lack of training.

Second, the study targeted quantification rather than diagnosis. Much of the AI-in-dermatology literature had focused on classification (is this lesion malignant or benign?), which puts AI in direct competition with a dermatologist's core expertise. BSA estimation, by contrast, is a quantitative task that even experienced clinicians find tedious and imprecise. Framing AI as a measurement tool rather than a diagnostic rival made the work easier to accept and easier to integrate into existing workflows.

Limitations

The study had clear constraints. All 130 images came from a single clinic, so center-specific factors (camera equipment, lighting, photographic technique) could have influenced results. The patients represented Fitzpatrick skin types 1 through 4 only; how the algorithm would perform on darker skin tones remained an open question with significant equity implications. Difficult anatomical sites like the scalp and genital areas were not systematically assessed. The training set of 2,000 images, reasonable for 2021, would be considered modest by current deep learning standards. And comparing against a single dermatologist offered limited insight into the range of expert performance.

Follow-Up Research

The next phase of the research program, the development of the SPREAD (Skin Patch-based Regional Extent Assessment for Dermatoses) Framework and the subsequent AI PASS comparative study, would build on the foundation laid by this initial validation work.

References

Breslavets, M., Shear, N. H., Lapa, T., Breslavets, D., & Breslavets, A. (2022). Validation of artificial intelligence application in clinical dermatology. Journal of the American Academy of Dermatology, 86(1), 201-203. https://doi.org/10.1016/j.jaad.2021.01.064

Breslavets, M., Shear, N. H., Breslavets, D., & Lapa, T. (2023). Use of artificial intelligence as an objective assessment tool in clinical dermatology. EADV Symposium, Seville, Spain.

Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115-118.

Phillips, M., Greenhalgh, J., Marsden, H., & Palamaras, I. (2020). Detection of malignant melanoma using artificial intelligence: An observational study of diagnostic accuracy. Dermatology Practical & Conceptual, 10(1), e2020011.