SOLO-MEDICINE IN OPTICAL BIOPSIES

A Way to Practice Telemedicine

Olga Ferrer, Francesco Ettorre, Xiomara Santos

University of La Laguna, La Laguna, Canary Islands, Spain

Thomas Zinkl, Ruben Tous, Jaime Delgado

Department of Computer Architecture, Universitat Politecnica de Catalunya (UPC-BARCELONATECH), Barcelona, Spain

Keywords: Solo-medicine, Optical biopsy, Confocal laser endomicroscopy, CLE, CBIR, Medical image retrieval,

Query by image, ISO-15938-12, MPEG query format, MPQF, ISO 24800-3, JPSearch, JPEG query format,

JPQF, Artificial intelligence, Multimedia standard.

Abstract: A way to practice Telemedicine is to access a data-base capable to assist you in medical procedures

(diagnosis, treatment and prognosis), similarly to consult a book or to ask a college. In many countries the

lack of specialists and training capabilities demand to practice solo-medicine, that in the case of surgery

require robots capable to induce anesthesia or help in vision or handling instruments. A relevant case is the

diagnostic self-training requirements for optical biopsies (OBs) obtained with confocal laser

endomicroscopy (CLE) or the assistance in the diagnosis of pathology slides. In both cases it is required a

training set of digital images against which to compare the question case by means of image-queryformat.

The present paper present a content-based image retrieval system (CBIR) based on the MPEG Query

Format Standard in order to provide a set of similar pictures and the corresponding diagnosis to help on

diagnosis or just to train the doctor. The paper defined the Image Solo-Medicine Paradigm (ISMP)

architecture merging medical image standards and MPEG and JPEG standards. It tested the solution with

normal, and benign colon OBs with 90% congruency. The ISMP is of particular interest viewed the

proliferation of iPhone medical applications aiming to train doctors and support medical decisions.

1 INTRODUCTION

Nowadays the number of medical applications for

iPhone proliferate (CATAI, 2010), attracting the

interest of relevant medical Journals such as the

British Medical Journal (BMJ Group, n.d.) to build

applications that help doctor to make their decisions

and auto-train themselves. The time of training

books, with periodical updates for new diagnosis or

treatments, is arriving to an end; doctors will have

on-line and on mobile phones that information, and

will use mobile phones for a variety of medical

applications (Ferrer-Roca and Marcano, 2009;

Ferrer-Roca and Marcano, 2010; Ferrer-Roca,

2010).

We have been working in a diagnostic medical

application based on images and on which “gold-

standards” are still on the way (Hersch et al., 2005;

Kiesslich et al., 2008; Ferrer-Roca et al., 2010). This

is the so-called optical biopsy (OB) (Wang and

VanDam 2004). A non-intrusive optic diagnostic

method, capable to analyze the tissue in surface and

in deepness with one of the following techniques:

laser, OCT, infrared, fluorescence, spectroscopy etc.

This means, that it is not necessary to extract the

tissue from the body. Tissue is accessed through the

surface of the body through the skin or by

endoscopy.

In OBs images are obtained in real time together

with complementary information that allows

evaluating the disease in vivo, but “gold-standards”

are still lacking while in surgical pathology

standards lay on the histology of the normal fixed

tissue (Ferrer-Roca, 2009). To provide training and

self-confidence on OB diagnosis, two possibilities

are open: (a) Tele-consult to a pathologist or (b)

Train themselves with a non-supervised search for a

“similar image” on the Net using multimedia query

and image mining techniques (Chen et al., 2006).

441

Ferrer O., Ettorre F., Santos X., Zinkl T., Tous R. and Delgado J..

SOLO-MEDICINE IN OPTICAL BIOPSIES - A Way to Practice Telemedicine.

DOI: 10.5220/0003090804410445

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2011), pages 441-445

ISBN: 978-989-8425-34-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

The present proposes a standardized ISMP

(Image Solo-Medicine Paradigm) architecture based

in the usage of two novel standards, the MPEG

Query Format (MPQF) and the JPEG’s JPSearch

project (Tous, 2006). While MPQF provides a

uniform language for querying multimedia

databases, JPSearch provides an interoperable

architecture for images’ metadata management.

Preliminary results on Internet image search and

discovery system for diagnostic medical purpose are

showed. Results were based on a training-set of

CLE-OB images annotated with specific CLE

semantics.

2 MATERIAL AND METHODS

The proposed system to allow users to navigate

searching similar images considered to be golden

standards (due to the pathology confirmation and

availability of pathology image) in a pair image

database integrated by OB-CLE images together

with the histological counterpart.

Images used in the present paper were provided

by one of the authors (OFR) or taken from data

published in Internet. All were JPEG images.

2.1 ISMP (Image Solo Medicine

Paradigm) Architecture

The ISMP system provided tools to annotate an

unknown OB-CLE image with key-words and image

structural information for content based image

retrieval (CBIR). Figure 1 summarizes the overall

architecture.



CBIR CBIR

image

GUI ClientGUI Client

HTT P

GUI

Server

GUI

Server

Image

Analysis

Image

Analysis

MPEG

Query

Format

Interpreter

MPEG

Query

Format

Interpreter

Metadata

Image

Processing

Steps

Image

Processing

Steps

Feature

Extraction

Algorithms

Feature

Extraction

Algorithms

Similarity

Functions

Similarity

Functions

image

imageimage

Image indexing

Engi ne

Framework

Engi ne

Framework

metadata

indexing

metadata

index

cbir index

MPQFMPQF

Figure 1: Overall architecture of Image Solo-medicine

Paradigm(ISMP).

Image Solo-Medicine Paradigm (ISMP) architecture

was integrated by four main modules:

1) Image processing: Offline extraction of medium-

level and high-level metadata from the images in the

database, and also to the on-the-fly extraction of the

same metadata from an example image submitted by

a user as a query. We used the ImageJ (ImageJ, n.d.)

Java library to implement an adhoc algorithm.

2) CBIR index construction: We generated an index

for query-by-example search by means of selection

of a feature vector and a similarity function.

3) Search Engine Framework: We built a query

processor capable of solving text-based queries,

CBIR queries and combinations of both.

4) MPEG Query Format Interpreter: In order to

effectively ensure interoperability with potential

third-party applications we built a standard interface

based on ISO/IEC 15938-12:2008 (MPEG Query

Format, MPQF).

3 RESULTS

3.1 ISMP Training Set

It is composed by 25 OB-CLE images obtained with

a PENTAX CLE with their re-sulting histological

images (50 images in total).

3.1.1 ISMP Preprocessing

ISMP preprocessing was done in two steps: 1-

Normalization (to minimize light in homogeneities

caused by laser light source) that included several

image processing steps (enhanced contrast,

equalization, etc.). 2- Grey level reduction using

pixel value range reduction and region merging

algorithms as seen in Figure 2.

Figure 2: ISMP pre-processing of original images (top).

Results in the bottom line.

3.1.2 ISMP Feature Extraction

ISMP feature extraction was done in two steps:

1) The Local Binary Pattern (LBP) (Pietikainen et

HEALTHINF 2011 - International Conference on Health Informatics

442

al., 2000) operator (A gray-scale invariant texture

measure derived from a general definition of texture

in a local neighbor-hood). The process included (a)

Integration: On each pixel, we calculated an array of

bits of 0 and 1 comparing the original pixel value

and its neighbors in a certain radius. (b) Decision

maker: The array values are summed up. The higher

lbpSum for a pixel indicated more likely to be the

center of one of the big black areas (Figure 2).

2) The modified density-based DBSCAN

algorithm, highlighted the various crypts and their

boundaries.

With the lbpSum value for every pixel, we apply a

clustering algorithm to cluster to a certain crypt. In

the clustering process we used a modified density-

based DBSCAN algorithm originally proposed in

(Sander et al., 2004). See Figure 3.

Figure 3: Clustering for gland identification. Normal (left)

and hiperplastic glands (right).

3.1.3 ISMP Feature Measurement

The results of this process allow us to extract and

measure certain features such as the silhouette

coefficient, the crypt compactness, the crypt

roundness or the inter-crypt distance.

3.1.4 ISMP-indexing and ISMP-retrieval in

Two Steps

1) We defined a feature vector and normalized it

applying linear scaling unit range normalization.

2) We retrieved similar images to a given one,

using the similarity function. The selected function

operated over the vector of selected features, whose

composition determines which is the nature of the

similarity being considered (similarity is relative in a

multidimensional space).

The test set demonstrated that the manhattan and the

euclidian distance, in combination with the linear

scaling unit range normalization, provide better

performance.

3.1.5 ISMP Data-base Query

MPQF queries were evaluated against one or more

multimedia databases which were unordered set of

Multimedia Contents-MC (combination of

multimedia data and its associated metadata).

1) Data-base: It was a dual database model (Figure

4) by content and by metadata. The MPQF operated

over sequences of evaluation items.

2) Condition Tree

: It was dual condition tree since

it (a) Combined filtering ele-ments (conditions) from

the BooleanExpressionType and (b) Interconnected

them with Boolean operators (AND, OR, NOT and

XOR).

The (image retrieve) IR-like condition used

QueryByExample query type and in-cluded the

Base64 encoding of the binary contents of a JPEG

image. The (data retrieve) DR-like condition

specified, in the present case, that the metadata field

FileSize must be less than 1000 bytes. Each

condition acted over a sequence of evaluation items

and, for each one, re-turned a value. For IR-like

conditions, returned any value in the range of [0.1].

For DR-like conditions returned 1 or 0 (true /false).

A threshold value within a condition was used to

indicate the minimum value the score to be

processed in the training set.

3.1.6 ISMP Image Retrieval

The ISMP retrieval system over the web interface

present de problem image for query and retrieve a

list of similar images (as many as possible) from the

data base.

3.2 Test Set

The web user interface used both 1) the query-by-

image in combination with 2) classic XML

metadata-based criteria.



Figure 4: ISMP retrieval in the Test set. Original OB

image (left), preprocessing (middle) and retrieved image

(right), in this case a histological image.

SOLO-MEDICINE IN OPTICAL BIOPSIES - A Way to Practice Telemedicine

443



Figure 5: ISMP retrieval in the Test set . Original OB

image (left) and retrieved image (right) only histological

images were retrieved.

The rate of adequate image retrieval from

normal, benign and hyperplasic images using the

threshold values indicated in the Section 3.1.5 was

90%.

4 DISCUSSION

The use of solo-medicine (Ferrer-Roca and

Marcano, 2009) will be a common practice in a near

future, and there-fore professional will require

support by any media, including mobile phone. Two

are the characteristics of this support: 1) is going to

be on line (books will be soon obsolete) and 2) it

will be required during patient interventions,

therefore the possibility to get access to mobile

phones is of paramount importance. The latter is

even more relevant considering that the majority of

this solo-medicine will be carried out in remote,

isolated or developing countries where satellite

mobile phones will be, probably, the only available

technology.

The solution brought in this paper merge the

standardization process of mass-used multimedia

standards with the medical image standardization

process, and built a ISMP (Image solo medicine

paradigm) architecture, that in the present paper was

applied to colon OBs (Optical Biopsies).

One of the main functionalities of the ISMP

system architecture is the ability to combine

conventional search criteria (keywords, metadata

ranges) with the direct usage of an example image

(query-by-example paradigm) to retrieve similar

precedent cases. In the data retrieve DR-like

conditions, the MPQF standard acts as a

conventional Boolean-based filtering language,

while with respect to (Information retrieve) IR-like

conditions MPQF acts preserving scores as a fuzzy-

logic system. The standard specifies the behaviour

of the provided Boolean operators in presence of

non-Boolean values.

The result showed that automatic feature-

extraction by image analysis on Black & White

images coming from the CLE, as well as colour

images from surgical specimens, reached the 90%

congruency. Thus indicating that the image-query

solution proposed in the ISMP architecture is an

adequate one to give professional support in

medicine at least in the normal and benign cases.

Nevertheless we have to test the borderline and

malignant ones to detect the sensitivity and

specificity of the proposed solution.

ACKNOWLEDGEMENTS

This work has been partly supported by the Spanish

government (TEC2008-06692-C02-01) and the

CATAI association.

REFERENCES

BMJ Group. Differential Diagnosis iPhone application.

<http://bestpractice.bmj.com/best-practice/marketing/

differentials.html>

CATAI 2010, En India: El hospital mas eficiente del

mundo. < http://catai.net/blog//2010/06/en-india-el-

hospital-mas-eficiente-del-mundo/>.

CATAI 2010, Anatomia Patologica en iPhone.

<http://catai.net/blog//2010/06/anatomia-patologica-

en-iphone/>.

Chen, N., Shatka, H., Blostein, D., 2006. Use of Figures in

Literature Mining for Biomedical Digital Libraries.

Dial, pp.180-197, Second International Conference on

Document Image Analysis for Libraries (DIAL'06).

Ferrer-Roca, O., 2009. Telepathology and optical biopsy.

Int J Telemed Appl Vol: 2009 Pages: 740712

doi:10.1155/2009/740712.

Ferrer-Roca, O., Marcano F., 2009. Anatomia Patologica

Digital. Control de calidad y patoinformatica.,

Rev.Esp.Patol 42(2): 85-85.

Ferrer-Roca, O., Marcano F., 2010. Computed assisted

microscopy. The era of the Small Size Virtual Slides

and the 4M microscopes. HEALTHINF 2010 Vol:

2010 Pp: 517-522. Portugal, ISBN: 978-989-674-018-

Ferrer Roca, O., Duval V., Delgado J., Rolim C., Tous R,

2010 Query by image medical training. Optical Biopsy

with confocal endoscopy (OB-CEM). HEALTHINF

2010 Vol: 2010 Pp: 166-172. Portugal, ISBN: 978-

989-674-016-0.

Ferrer-Roca, O., 2010. Mobile phones in pathology.

J.Telemed & Telecare 16(3): 165.

Hersch W. R., Bhuptiraju R. T., Ross L., Johnson P.,

Cohen A. M., Kraemer D. F., 2005. TREC 2004

Genomics Track overview. Proc of TREC 2004 NIST

Special Publication <http://ir.ohsu.edu/genomics>.

HEALTHINF 2011 - International Conference on Health Informatics

444

ImageJ. <http://rsbweb.nih.gov/ij/>.

Ishii, N., Koike, A., Yamamoto, Y., Takagi T., 2008.

Figure Classification in Biomedical Literature

towards Figure Mining. bibm, pp.263-269, IEEE

InternationalConference on Bioinformatics and

Biomedicine, 2008.

Kiesslich R., Galle P. R., Neurath M. F., 2008. Atlas of

endomicroscopy. Springer-Verlag Heidelberg. ISBN

978-3-540-34757-6.

Pietikainen M., Ojala T. and Maenpaa T., 2000. Gray

Scale and Rotation Invariant Texture Classification

with Local Binary Patterns. In Proceedings of the

Sixth European Conference on Computer Vision

(ECCV2000, pages 404–420.

Sander, et al., 2000. Density-based Clustering In Spatial

Databases: The Algorithm Gdbscan and its

Applications. Data Min. Knowl. Discov., 2(2):169–

194.

Tous, R., 2008. Query formats for multimedia applications

ISO/IEC 15938-12 (MPEG Query Format) & ISO/IEC

24800 (JPSearch). In CATAI 2009: Super-resolution

and optical Biopsy. CATAI editions. Tenerife. Pp25-

32.

Wang, T. D., VanDam, J., 2004. Optical Biopsy: A New

Frontier in Endoscopic Detection and Diagnosis.. Clin

Gastroenterol Hepatol 2(9): 744–753.

SOLO-MEDICINE IN OPTICAL BIOPSIES - A Way to Practice Telemedicine

445