The Computer-aided Diagnostics of Gastric Lesions

by using High Definition Narrow-band Imaging Endoscopy

and Real-time Pattern Recognition System

K. Yu. Erendzhenova

, O. A. Kulagina

, R. M. Kadushnikov

and T. V. Zarubina

Pirogov Russian National Research Medical University, Ostrovitianov str. 1, 117997, Moscow, Russian Federation

Medical Research and Education Center of Lomonosov Moscow State University,

Lomonosovsky prospect 27, korp. 10, 119991, Moscow, Russian Federation

LLC “SIAMS”, Kominterna str. 16, 620078, Yekaterinburg, Russian Federation

Keywords: High Definition (HD) Endoscopy, Narrow-Band Imaging (NBI) Endoscopy, Early Gastric Cancer

Diagnostics, Decision Support, Pattern Recognition, Endoscopic Image Processing.

Abstract: High Definition (HD) and Magnified Narrow band imaging endoscopy (ME-NBI) allowed to

recognizetypes of gastric lesions according modified VS-classification by professor Yao K., becausethe

parameters to describe regular or irregularvascular or microsurface pattern and demarcation line in

lesionswere formalized. In this work endoscopic differential criteria of benign and neoplastic epithelial

lesions of stomach were obtained. Based on them classification algorithm for the real-time processing of

narrow–band endoscopic images with a highly productive distributed intellectual analytic decision support

system for multiscale endoscopic diagnostics is presented. We also created the electronic atlas and database

to collect high resolution endoscopic images, applied and proved the differential diagnosis of gastric lesions

through the computer analysis. The algorithm consistentlyused scale– invariant feature transform detector,

computation of gastric mucosa pit–pattern skeletons, “Bag of visual words” method, and K–means method

for key pointsclustering. Resulting classification algorithm is completely automated, performed real-time

analysis, and did not require preliminary selection of interest area. Image classification accuracy was 85%.

1 INTRODUCTION

In Russia stomach cancer takes the first position in the

structure of cancerdiseases (Kaprin, 2017). Every year

tens of thousands of people fall ill with stomach

cancer, and almost half the deaths occur (Savelyev,

2009). Gastric cancer is more often detected in the

late stages of the tumor process. Treatment is

expensive and despite all the efforts of surgeons and

oncologists it often does not prolong the life of such

patients for more than five years. The solution of the

social and economic problem in this direction is the

diagnosis of early gastric cancer (Japanese Gastric

Cancer Association, 1998). Among the instrumental

methods only upper gastrointestinal endoscopy aims

to identify this disorder, as well as benign epithelial

neoplasia and non-neoplastic lesions that are inclined

to atypia.Gastroscopy allows directly visually

evaluate the condition of the gastric mucosasurface.

Correct differential diagnosis of such formations

contributes to the selection of correct treatment tactic

and 95% of patients' survival.

Routine white light endoscopy helps detect

pathology focuses, whereas analysis of a pit and

vascular pattern in mucosal structure, derived from

enhancement techniques such as magnified narrow-

band imaging (ME-NBI) endoscopy with video

endoscopic high-resolution systems, allows to

determine the types of lesions. The variety of these

mucosal changes, the difficulties of their visual

interpretation cause insufficient accuracy of

recognition of pathological processes (Buntseva,

2014). Promising is the creation of an automated

images processing during endoscopic exploration.

Currently endoscopic computer-aided decision–

making systems are created in different countries.

Japanese specialists have achieved the most

advanced results of analysis of thin structure for the

revealed endoscopic lesions. Research by H.

Osawa., H. Yamamoto, Y. Miura, H. Ajibe, H.

Shinhata, M. Yoshizawa, K. Sunada, S. Toma, K.

Satoh, and K. Sugano have shown the efficiency of

computer–aided analysis of endoscopic images using

digital chromoscopy in ascertainment of the

Erendzhenova, K., Kulagina, O., Kadushnikov, R. and Zarubina, T.

The Computer-aided Diagnostics of Gastric Lesions by using High Deﬁnition Narrow-band Imaging Endoscopy and Real-time Pattern Recognition System.

DOI: 10.5220/0006724906150620

In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 5: HEALTHINF, pages 615-620

ISBN: 978-989-758-281-3

615

boundaries between neoplasms and surrounding

mucosa, thereby allowing determination of stomach

cancer flattened forms (Osawa, 2012). R. Miyaki, S.

Yoshida, S. Tanaka, Y. Kominami, Y. Sanomura, T.

Matsuo, S. Oka, B. Raytchev, T. Tamaki, T. Koide,

K. Kaneda, M. Yoshihara, and K. Chayama

examined application of computerized image

processing to endoscopy images obtained using

digital chromoscopy for the purpose of detecting

benign neoplasms and intramucosal early gastric

cancer (Miyaki, 2013). T.C. Lee, Y.H. Lin, N. Uedo,

and H.P Wang presented the previously results of

computer–aided analysis of suspicious gastric

carcinoma images, obtained with narrow band

imaging in (Lee, 2013).Such computer–aided

medical decision-support systems for endoscopy

allow to keep labor costs down for endoscopy

investigation. Nevertheless, theyrequire high level of

endoscopist qualification to adjust analysis settings

for each image. Besides, existing decision–making

systems are not unified; they determine the

particulartype of pathology, do not allow decision-

making during the endoscopic exploration.

Thedisadvantages of the existing decision–support

systems mentioned abovemean that endoscopy

services are very time taking.

With that in mind, the aim of this work was to

develop a decision support system for endoscopic

departments to diagnose precancerous and early

neoplastic changes in the stomach mucosabased on

the intelligent analysis of endoscopic images using

computational methods, which include computer

vision. We propose the method for processing

narrow–band with or without magnification

endoscopic images using a highly productive “Smart

Endoscope” distributed intellectual analytic

decision–support system for multiscale endoscopic

diagnostics and surgery. This system exhibits the

following advantages:ability to learn towards

classify images obtained with various endoscopic

methods; real–time operations that allow to make

decisions on-the-fly, rather than post-test;

completely automated analysis algorithms, which do

not require prior selection of interest areas or

additional operator training.

2 MATERIALS AND METHODS

2.1 Materials

We prospectively selected164 patients with 192

focal superficial epithelial gastric lesions. We

performed 220 HD-NBI and ME-NBI endoscopic

images of lesions surface. In this work we included

prospectively both protruded and flat or depressed

sites of damage. 220 images included 141 photos of

benign lesions (hyperplastic polyps, erosions, ulcers,

focuses of intestinal metaplasia) and 79 photos of

neoplasia (low and high grade intraepithelial

neoplasia, early gastric cancer, invasive and

advanced gastric cancer).So, all images were divided

into two groups according to the tactic of treatment:

First group - non-neoplastic lesions (tactic – is

observation); Second group - epithelial neoplasms of

the stomach (tactic – is minimally invasive

endoscopic,laparoscopic or open surgical treatment)

(Dixon, 2002). The structure of the lesions was

verified by histological examination of biopsy

samples or resected portions of the mucosa.

2.2 Methods

The methods of investigation were routine white

light endoscopy, high definition enhanced narrow

band imaging endoscopy (with magnification from

50 to 115 times).

Statistical analysis included the application of

Fisher's exact test, Cramer's criterion, the

inhomogeneous sequential diagnostic Bayesian

procedure (Gubler, 1978). The significance p-level

was assumed to be 0.05.

On 182 (out of 220) endoscopic images the

method of machine vision "Bag of visual words"

was used to classify benign lesions (n=111) and

epithelial neoplasms of the stomach (n=71)

(Liedlgruber, 2011).

3 RESULTS

3.1 Statistical Analysis

All lesions and images we characterized with 34

criteria. Four criteria were clinical (age, gender,

presence or absence of Helicobacter pylori infection,

primary or residual lesion), 14 – were got during

traditional endoscopy (localization, size, macrotype,

number of lesions,presence or absence of fibrin,

erosionof lesion surface, signs of inflammation,

atrophy or intestinal metaplasia in surrounding

mucosa, consistency, mobility, etc.), 16 – were got

in time enhanced endoscopic explorations. These

sixteen microendoscopic criteria of microsurface and

microvascular pattern included specific features

(such as size, shape) and features by most spreading

classifications (VS-classifications by Yao K. and

Kato, Kaise triad) and combined features.

HEALTHINF 2018 - 11th International Conference on Health Informatics

616

We compared benign and neoplastic lesions by

34 parameters with Fisher's exact test and Cramer's

conjugation coefficient.

Statistical analysis allowed to define 6 significant

endoscopic criteria. On these parameters using the

Bayes procedure, the probabilities of assigning

images to each of the two groups were obtained. The

group classified according to this decision rule was

determined by the maximal probability. For

differentiation benign (n=141) and neoplastic lesions

(n=79) obtained sensitivity was 92%, specificity was

96% and accuracy was 94%. The parameterswere

adopted for clinical use (most significant, easy and

objective endoscopic signs). By comparing groups

of epithelial lesions of the stomach with the exact

Fisher test and the Cramer test, six statistically

significant parameters were identified (table 1).

The first criterion is the thickness of vascular

component in the lesion compared with

surrounding mucosa. In neoplastic lesions thickness

of vascular component is less or heterogenous than

in surrounding mucosa. And in benign lesions it is

more similar.

The second criterion is the thickness ratio of

glandular and vascular component in the lesion. In

neoplasia thickness of glandular white component is

more than dark vascular component. For benign

lesions thickness of glandular is less or similar to

vascular component.

Next criteria are the thickness and contours of

vascular componentin the lesion. For benign lesions

thickness of vascular component is uniform and

contours are relatively smooth. For neoplasia – they

are highly unequal and uneven.

In neoplastic lesions we can frequently see the

thick findings like blackbright sticks

(individualvessels). In benign lesions, usually, there

are notblack individualvessels.

And the last criterion is the demarcation line.

Similar to VS-classification it is specific for

neoplastic lesions.

The practical check of these criteria in our clinic

showed high accuracy and interobserver agreement.

We checked these 6 microendoscopic features

for differentiation 40 endoscopic images (21 benign,

19 neoplastic lesions) by 3 experts (accuracy was

100%), 2 low experienceddoctors (accuracy was 92-

95%, interobserver agreement (IA) coefficient was

equal 0,75) and 2 inexperienced in HD-NBI

endoscopy doctors (accuracy was 95-98%, IA

coefficient was equal 0,85).

Table 1: Endoscopic differential parameters.

Endoscopic

parameter

Benign lesions

Epithelial

neoplasia

The thickness of

vascular component

(area between the

glands) as compared

with surrounding

mucosa

More or similar

Less or

heterogeneous

The thickness ratio of

glandular (G) and

vascular (V)

component

G is less or

similar than V

G is more than

The thickness of

vascular component

Relatively

uniform

Highly unequal

The contours of

vascular component

Relatively

smooth

Highly uneven,

jagged, wavy

The thickened

individual vessels as

bright sticks

Yes

The demarcation line

Yes

3.2 Real-Time Pattern Recognition

Analysis

The electronic atlas includes white light and

magnified NBI endoscopic images of benign lesions

(hyperplasia, inflammation, atrophy, intestinal

metaplasia), low and high grade intraepithelial

neoplasia, early gastric cancerand advanced stomach

adenocarcinoma. All images with delineated regions

of interest are accompanied by expert’s description

of the clinical parameters, macroscopic and

microscopic structure features of the lesion,

including the vascular and surface patterns, and

histological structure of the lesion.

With the help of the algorithm of computer

vision it became possible to divide endoscopic

images into groups of non-neoplastic lesions of the

mucosa and epithelial gastric neoplasms. The use of

the "Bag of visual words" method for the

mathematical representation of images of focal

superficial epithelial stomach lesions included the

steps of detecting key points (SNoL-detector),

mathematical description (SIFT, Scale-invariant

feature transform descriptor) and clustering of local

characteristics in the key points area (hierarchical k-

means method) and constructing visual words

dictionary (Canny, 1986;Liedlgruber, 2011).

At the first stage, the endoscopic image was

delivered to the automated endoscopist’sworkplace,

where it was transformed in gray levels. Gaussian

blurring was then applied using values of blur radius

The Computer-aided Diagnostics of Gastric Lesions by using High Deﬁnition Narrow-band Imaging Endoscopy and Real-time Pattern

Recognition System

617

from the certain range with the present step value.

That produced the “pyramid” of Gaussians.

There are two object types present on endoscopic

images (photos and videos) of stomach mucosa

microstructure – glands, which are the bright areas,

and vessels – dark areas surrounded by glands. Use

of FAGF (Fast Anisotropic Gauss Filtering) to

source images allowedto select of vessels and

glandsbetter (Geusebroek, 2003). At the next

processing step, the image was binarized.

After applying a median filter with a 3x3

window, the pit skeleton wasoutput using the FFPT

(Fast fully parallel thinning) algorithm(Guo, 1992).

The elementary unit of a skeleton is its branch or

rib, which characterizes a pit or a vessel on ainitial

image. Intersections and endpoints of skeleton ribs

become key points of the analyzed image.

Further image processing was performed by

selection and classification of local image features

using the SIFT descriptors(Lowe, 2004). SIFT

descriptors of local image features were built.

Produced local feature descriptors were invariant for

scaling, shifting, rotation and changing illumination

direction. The local feature area relevant to point

blur radius is covered by a 4x4 grid (16 cells in

total). For each grid cell, Histograms of Oriented

Gradients (HOGs) were built for eight directions.

After that, vectors for the cells combined into a

single 128–dimensional vector (8х16), which was

used to describe a key point (Lingua, 2009).

The set of key point vectors was processed using

the “Bag of visual words” algorithm (Liedlgruber,

2011). Histogram of key points distribution by

defined groups was built for each image. Key points

selected by a computer on images were placed into

clustersin a 128–dimensional space using clustering

cloud of points by applying the hierarchical k–

means method. It was found that, for the endoscopic

examinations relevant, the number of clusters should

be equal to 1000. The number of key points that

were placed into each cluster was determined and a

histogram illustrated distribution of the points for

analyzed image by clusters and characterized image

as a wholewas built. The histogram was represented

by a vector in a 1000–dimensional space (vector

coordinate values showed the number of key points

falling into relevant clusters). Image classification

was performed by placing a multidimensional vector

into a classification space produced from the training

set by the Support Vector Machines Method (SVM).

Aimed to test the suggested method, a set of 182

endoscopic images was selected. Images featured

different lesions of gastric mucosa, validated by

means of histological examination. Images were also

obtained using high-resolution narrow–band

endoscopy. There were 111 images of non-

neoplastic gastric epithelial lesions, and 71 images

of different epithelial neoplasms of high and low

grade, of early and advanced gastric cancer. The

scientific and clinical goal for computer-aided

analytic system was to evaluate and differentiate

focal lesions of gastric mucosa as non-neoplastic or

stomach epithelial neoplasms (figure 1).

Recognition algorithm applying results

demonstrate that in 85% of the cases it did perform

correct image classification, placing image into First

orSecond group. Average image analysis time was

less than 30 seconds, and,in fact, that allows using

the algorithm for real-time analysis of endoscopic

images(Stepanov, 2016). A positive assessment of

the current results was approved by practicing

endoscopists.

Figure 1: Classification algorithm for morphological

analysis of capillary and gland microstructure of stomach.

4 CONCLUSIONS

In accordance with the statistical analysis six

microendoscopic features of gastric lesions were

proven most significant for effective differential

diagnosis between benign and neoplastic epithelial

gastric lesions. These features can be successfully

used by nonexperience doctors and also for creating

the decision support system, including through the

computer analysis.

Also, the electronic database will be useful for e-

learning of specialists in gastrointestinal endoscopy

thanks to function of similar endoscopic images

search. However, the main goal of this atlas and

database in future is to provide the direct computer-

aided image analysis during endoscopic

investigation for predicting the histological structure

of the epithelial lesions and choosing the correct

treatment strategy

HEALTHINF 2018 - 11th International Conference on Health Informatics

618

Effective results of application of the decisive

rule and high accuracy of the mathematical

algorithm for the classification of epithelial

neoplasms and non-tumor lesions of the stomach

show the fundamental possibility of formalizing the

microendoscopic structure of the formations, and

hence the possibility of developing their objective

clinical classification and decision support system

for the doctor.

The decision support system with automatic

image identification soon will become an

indispensable part of endoscopic video systems

A method for processing narrow–band

endoscopic images using a highly productive

distributed intellectual analytic decision–making

system was presented. This method allows

improving accuracy and helps avoiding subjectivity

in real-time classification of endoscopic images. It

possesses the following distinctive features:

1. Key points are selected in real time due to

application of FFPT algorithm.

2. Use of SIFT descriptors allows real-time

selection and vectorization of local image

features invariant to scale, shift, rotation and

illumination.

3. Application of the “Bag of visual words”

approach enables processing the whole image

of a mucosal neoplasm (and not just the part

of it examined using histological methods).

4. Use of SVM method for building

classification space.

Examination of a suggested computer algorithm

using 182 endoscopic images todetermine the

neoplasia demonstrated that the accuracy of correct

recognition reaches 85%.

That allows formalization of pit and vessel

pattern descriptors for gastric epithelial neoplasms,

and, in turn, development of a precise and objective

clinical classification. Algorithm efficiency can be

improved by breaking image sets into a larger

number of subgroups according to histological data.

That requires processing wide image set for each

subgroup. It is important that the algorithm is fast

and efficient. That allows using it for processing

video streams and endoscopic images in real time.

ACKNOWLEDGEMENTS

The work was done within the framework of the

project performed by SIAMS Ltd, and supported by

the Ministry of Education and Science of the

Russian Federation (Grant agreement 14.576.21.

0018 dated June 27, 2014. Applied research (project)

UID: RFMEFI57614X0018).

REFERENCES

Buntseva, O. A., Plakhov, R. V., Galkova, Z. V., Fedorov,

E. D., 2014. Modern endoscopic methods of diagnosis

and treatment of precancerous changes and early

gastric cancer.Polyclinic, 2(2), pp. 56-64.

Canny, J., 1986. A computational approach to edge

detection. IEEE Transactions on pattern analysis and

machine intelligence, 6, pp. 679-698.

Dixon, M. F., 2002. Gastrointestinal epithelial neoplasia:

Vienna revisited. Gut, 51(1), pp. 130-131.

Geusebroek, J–M., Smeulders, A. W. M., van de Weijer,

J., 2003. Fast anisotropic Gauss Filtering. IEEE

Transactions on image processing (TIP 2003),

pp. 938–943.

Gubler, E. V., 1978. Computational methods of analysis

and recognition of pathological processes, Medicine.

Leningrad.

Guo, Z., Hall, R. W., 1992. Fast fully parallel thinning

algorithms.CVGIP: Image Understanding, 55(3),

pp. 317–328.

Japanese Gastric Cancer Association, 1998. Japanese

classification of gastric carcinoma - 2nd English

edition. Gastric Cancer, 1, pp.10-24.

Kaprin, A. D., Starinsky, V. V., Petrova, G. V. (ed.), 2017.

Malignant neoplasms in Russia in 2015 (morbidity

and mortality), MSRI named after P.A. Herzen - FGBI

branch of NMRRC of Russian Health Ministry,

Moscow.

Lee, T-C., Lin Y-H., Uedo, N., Wang, H-P., Chang, H-T.,

Hung C-W., 2013. Computer-aided diagnosis in

endoscopy: A novel application toward automatic

detection ofabnormal lesions on magnifying narrow-

band imagingendoscopy in the stomach.In Proc. 35th

IEEE AnnualInt. Conf. of the Engineering in Medicine

and Biology Society (EMBC), pp. 4430–4433.

Liedlgruber, M., Uhl, A., 2011. Computer-aided decision

support systems for endoscopy in the gastrointestinal

tract: a review. IEEE reviews in biomedical

engineering, 4, pp 73-88.

Lingua, A., Marenchino, D., Nex, F., 2009.Performance

Analysis of the SIFT Operator for Automatic Feature

Extractionand Matching in Photogrammetric

Applications. Sensors, 9(5), pp. 3745–3766.

Lowe, D. G. 2004. Distinctive image features from scale-

invariant keypoints. International journal of computer

vision, 60(2), pp. 91-110.

Miyaki, R., Yoshida, S., Tanaka, S., Kominami, Y.,

Sanomura, Y., Matsuo, T., Oka, S., Raytchev, B.,

Tamaki, T., Koide, T., Kaneda, K., Yoshihara, M.,

Chayama, K., 2013. Quantitative identification of

mucosal gastric cancer under magnifying endoscopy

withflexible spectral imaging color enhancement.J.

Gastroenterol. Hepatology, 28(5), pp. 841–847.

Osawa, H., Yamamoto, H., Miura, Y., Ajibe, H., Shinhata,

The Computer-aided Diagnostics of Gastric Lesions by using High Deﬁnition Narrow-band Imaging Endoscopy and Real-time Pattern

Recognition System

619

H., Yoshizawa, M., Sunada, K., Satoh, K., Sugano, K.,

2012. Diagnosis of depressed-type early gastric cancer

using small-caliber endoscopy with flexible spectral

imaging color enhancement. Digestive Endoscopy,

24(4), pp 231-236.

Savelyev, V. S., Kirienko, A. I. (ed.), 2009. Clinical

surgery: National guidelines, GEOTAR-Media.

Moscow.

Stepanov, D. M., Mizgulin,V. V., Kosulnikov, V. V.,

Kadushnikov, R. M., Fedorov, E. D., Buntseva, O. A.,

2016. Detector of interest point within region of

interest on NBI endoscopyimages.In Proc. Analysis of

Images, Social Networks, and Texts.

HEALTHINF 2018 - 11th International Conference on Health Informatics

620