Automatic Recognition of the Hepatocellular Carcinoma from
Ultrasound Images using Complex Textural Microstructure
Co-Occurrence Matrices (CTMCM)
Delia Mitrea
1
, Sergiu Nedevschi
1
and Radu Badea
2
1
Computer Science Department, Technical University of Cluj-Napoca, Baritiu Str., No. 26-28, Cluj-Napoca, Romania
2
I. Hatieganu University of Medicine and Pharmacy, V. Babes Str., No. 8, Cluj-Napoca, Romania
Keywords: Complex Textural Microstructure Co-occurrence Matrices (CTMCM), Textural Model, Hepatocellular
Carcinoma, Ultrasound Images, Classification Performance.
Abstract: The hepatocellular carcinoma is one of the most frequent malignant liver tumours. The golden standard for
HCC detection is the needle biopsy, but this is a dangerous technique. We aim to perform the non-invasive
recognition of this tumour, using computerized methods within ultrasound images. For this purpose, we
defined the textural model of HCC, consisting of the relevant textural features that separate this tumour
from other visually similar tissues and of the specific values that correspond to these relevant features:
arithmetic mean, standard deviation, probability distribution. In this paper, we demonstrate the role that the
Complex Textural Microstructure Co-occurrence Matrices have in the improvement of the textural model of
HCC and in the increase of the recognition performance. During the experiments, we considered the
following classes: cirrhosis, HCC, cirrhotic parenchyma on which HCC evolved and hemangioma, a
frequent benign liver tumour. The resulted recognition accuracy for HCC was towards 90%.
1 INTRODUCTION
The hepatocellular carcinoma (HCC) is the most
frequent malignant liver tumour, occurring in 70%
of the liver cancer cases. It evolves from cirrhosis,
after a phase of liver parenchyma restructuring at the
end of which dysplastic nodules (future malignant
tumours) result (American Liver Foundation, 2015).
The most reliable method for HCC diagnosis is the
needle biopsy, but this technique is invasive,
dangerous, as it could lead to the spread of the
tumour inside the human body. We develop non-
invasive, computerized methods for the automatic
and computer assisted diagnosis of this tumour,
based on ultrasound images. Ultrasonography is a
reliable method for patient examination and
monitoring, being safe, non-invasive, inexpensive
and repeatable. Other examination techniques that
involve medical imaging, such as the computer
tomography, the magnetic resonance imaging, the
endoscopy, even the contrast enhanced
ultrasonography or elastography, are irradiating
and/or expensive. Concerning the aspect of the HCC
tumour in ultrasound images, in incipient phase, it
appears as a small lesion (2-3 cm), usually having a
hypoechogenic and homogeneous aspect. In more
advanced phases, mostly often, its aspect becomes
hyperechogenic and heterogeneous, due to the
interleaving of many tissue types (fibrosis, necrosis,
active growth tissue, fatty cells) and to the complex
vessel structure (Sherman, 2005). However, it is
difficult to distinguish this tumour from the
surrounding cirrhotic parenchyma, and also it
sometimes resembles the benign tumours. Texture is
an important property in this context, being able to
reveal subtle characteristics of the tissue, beyond the
perception of the human eye. In nowadays’ research,
there are many approaches involving the
combination between the texture analysis methods
and various classifiers, aiming to perform the
recognition of some severe pathologies, based on
ultrasound images (Sujana et al., 1996), (Yoshida et
al., 2003), (Madabhushi, 2005), (Chikui, 2005),
(Duda et al., 2013), (Masood, 2006). In our former
research, we defined the textural model of HCC,
consisting of the complete set of relevant textural
features that best characterize this tumour,
respectively of the specific values associated to the
178
Mitrea, D., Nedevschi, S. and Badea, R.
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstr ucture Co-Occurrence Matrices (CTMCM).
DOI: 10.5220/0006652101780189
In Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2018), pages 178-189
ISBN: 978-989-758-276-9
Copyright © 2018 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
relevant textural features: mean, standard deviation
and probability distribution. For texture analysis, we
previously employed classical methods, such as the
first order statistics of the grey levels (arithmetic
mean, maximum and minimum values), the
autocorrelation index, the grey level co-occurrence
matrix of order two and the associated Haralick
features, edge-based statistics, the frequency of the
textural microstructures derived by using the Laws’
convolution filters, the Hurst fractal index, the
Shannon entropy computed after applying the
Wavelet transform at two resolution levels (Meyer-
Base, 2009). We also developed more accurate
texture analysis techniques, such as the GLCM
matrix of superior order (Mitrea, D., et al., 2012),
(Mitrea, D., Mitrea, P. et al., 2012), the edge
orientation co-occurrence matrix of second and third
order (Mitrea, D., Mitrea, P. et al., 2012),
respectively the Textural Microstructure Co-
occurrence Matrices (TMCM) of second and third
order, all these techniques being experimented in the
context of abdominal tumour recognition based on
ultrasound images (Mitrea et al., 2014). In this work,
we highlighted the role that our newly defined
Complex Textural Microstructure Co-occurrence
Matrices (CTMCM) of order two and three, which
involved a set of Laws’ features, as well as gradient
features, had in the supervised recognition of the
HCC tumour.
(a.)
(b.)
(c.)
Figure 1: Representative US images for (a.) Cirrho.sis; (b.)
HCC; (c.) Hemangioma.
The Laws’ based CTMC was experimented
before in the context of the liver tumour recognition,
in the unsupervised detection of the HCC evolution
phases (Mitrea et al., 2016), and also for colo-rectal
tumour recognition, providing satisfying results
(Mitrea et al., 2015). In this paper, we compare the
role that the CTMCM based on Laws’ textural
microstructures, or gradient features, have in
increasing the supervised recognition accuracy of
the HCC tumour and liver cancer.
We assessed the improvement of the textural
model and the classification performance increase
by providing the new set of relevant textural features
at the inputs of some powerful classifiers and meta-
classifiers such as: Support Vector Machines
(SVM), Multilayer Perceptron (MLP), decision trees
(C4.5), AdaBoost combined with C4.5. We
compared the performance due to the newly defined
textural features with that provided by the formerly
existing textural features. The experimental dataset
consisted of B-mode ultrasound images representing
liver tissue affected by cirrhosis, hepatocellular
carcinoma, cirrhotic parenchyma surrounding the
HCC tumour, and also the hemangioma benign
tumour. Relevant images belonging to each of these
classes are illustrated in Figure 1. When taking into
account the newly defined textural features, the
obtained accuracy for HCC recognition was towards
90%.
2 THE STATE OF THE ART
The texture-based methods were widely used during
nowadays’ research, in combination with classifiers,
with the purpose of the automatic recognition of the
tumours within medical images (Yoshida et al.,
2003), (Madabhushi, 2005), (Chikui, 2005), (Duda
et al., 2013), (Masood, 2006). Thus, methods like
the Wavelet and Gabor transforms were used in
combination with Artificial Neural Networks (ANN)
and Bayesian classifiers for the recognition of the
liver and prostate tumours from ultrasound (Sujana
et al., 1996), (Yoshida et al., 2003) and magnetic
resonance (MRI) images (Madabhushi, 2005), the
fractal-based methods were used for the recognition
of the salivary gland tumours in (Chikui, 2005),
while the run-length matrix parameters, in
combination with the Haralick features derived from
the GLCM matrix were used in conjunction with
ANN classifiers, Support Vector Machines and
Fisher Linear Discriminants, for the automatic
recognition of the liver lesions (Sujana et al., 1996).
In a more recent approach, the authors performed the
recognition of the liver tumours based on the
computation of the textural parameters from typical
and contrast enhanced computer tomographic (CT)
images. Feature selection was performed and then a
C4.5 classifier was applied, which provided a
recognition rate above 90% (Duda et al., 2013).
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
179
The technique of Grey Level Co-occurrence
Matrix (GLCM) was frequently used with the
purpose of tumour characterization and recognition
within ultrasound images and proved its efficiency
in many situations. A representative approach is
described in (Khuzir et al., 2009), where the GLCM
method was implemented in conjunction with a
decision trees based classifier, in order to perform
the recognition of the breast lesions from
mammographic images. This matrix was computed
for the following directions: 0°, 45°, 90°, 135° and
the corresponding Haralick features were derived.
The accuracy, measured in terms of the area under
the ROC curve, was 84%. The generalized co-
occurrence matrices, defined by Davis (Davis,
1979), were recently used in the form of the Local
Binary Pattern (LBP) Co-occurrence Matrix (Sujatha
et al., 2012), respectively in the form of the texton
and texture orientation co-occurrence matrix
(Sujatha et al., 2013). In the context of the research
described in (Sujatha et al., 2012), the authors firstly
performed texton detection, by applying specific
convolution filters, then the Local Binary Pattern
(LBP) method was applied; the logically compact
LBP features were also determined, after applying
the OR operator among the neighbours of the current
pixel. At the end, the Logical Compact Local Binary
Pattern Co-occurrence Matrix (LCLBPM) was
determined and experimented on standard textures
from the VisTex database, providing a satisfying
classification accuracy, situated between 80%-98%,
better than that provided by the Gabor features
applied in the same context. In the work described in
(Sujatha et al., 2013), the authors applied
convolution filters for texton detection, then they
determined the texton co-occurrence matrix for the
following directions of the displacement vectors: 0º,
45º, 90º, 135º. The texture orientation co-occurrence
matrix was also determined, after computing the
orientation of the edges resulted after the application
of the Sobel and Canny specific methods. These
methods were experimented on the VisTex database
and provided recognition rates situated between 89%
- 98.8%. As it results from the above-described
approaches, there does not exist any significant
study involving a co-occurrence matrix based on
textural microstructures determined after applying
the Laws’ convolution filters, or multiple edge
detection filters. Also, there is no systematic study
of the relevant textural features that best characterize
HCC, in the context of the automatic recognition of
this tumour. In our previous research, we defined the
textural model of the malignant tumours (Mitrea et
al., 2012), useful in both automatic and computer
aided diagnosis. In this work, we analyse the
efficiency of the Complex Textural Microstructure
Co-occurrence Matrices (CTMCM) in the context of
HCC recognition from ultrasound images.
3 THE PROPOSED SOLUTION
3.1 The Imagistic Textural Model of
the Hepatocellular Carcinoma
The imagistic textural model of the hepatocellular
carcinoma (HCC), also defined in (Mitrea et al.,
2012), consists of:
The complete set of the relevant textural
features able to differentiate this tumour from
visually similar classes
The specific values associated to the relevant
textural features: arithmetic mean, standard
deviation, probability distribution.
In order to build the imagistic textural model, the
following phases are necessary: first, a preliminary
phase is performed, when the training set is built,
consisting of regions of interest selected inside the
tissue of interest, within the ultrasound images.
Then, an image analysis phase is due, when the
textural features are computed by applying specific,
texture analysis methods. The learning phase
consists of selecting the relevant textural features,
respectively of computing the specific values of the
relevant textural features (arithmetic mean, standard
deviation, probability distribution). At the end, the
validation phase is performed, when the values of
the relevant textural features are provided to
classifier inputs and supervised classification
methods are applied in order to assess the
classification performance due to the generated
imagistic textural model.
3.2 The Image Analysis Phase
During the image analysis phase, classical textural
features, as well as newly defined textural features,
derived from advanced texture analysis methods,
were computed. Thus, first order statistics of the
grey levels (the arithmetic mean, the standard
deviation and the probability distribution), edge-
based statistics (the edge frequency, edge contrast,
the average edge orientation), the density and
frequency of the textural microstructures obtained
after applying the Laws convolution filters, second
order statistics of the grey levels (the autocorrelation
index, the Gray Level Co-occurrence Matrix -
ICPRAM 2018 - 7th International Conference on Pattern Recognition Applications and Methods
180
GLCM of order two), the Hurst fractal index, as
well as the Shannon entropy computed after
applying the Wavelet transform, recursively, twice,
were determined in the first phase. Then, the GLCM
of order 3, 5 and 7, as well as the Edge Orientation
Co-occurrence Matrix (EOCM) of order 2 and 3 and
the associated Haralick features were also computed.
In addition, a new type of superior order generalized
co-occurrence matrix, the Complex Textural
Microstructure Co-occurrence Matrix (CTMCM) of
order two and three was employed and analysed in
the current work, as described below.
3.2.1 The Complex Textural Microstructure
Co-Occurrence Matrix (CTMCM) of
Second and Superior Order
The Complex Textural Microstructure Co-
occurrence Matrix (CTMCM) was determined
through the methodology described below,
consisting of the following steps:
(1) First, we associated feature vectors to the pixels
in the region of interest, consisting of:
the results obtained after applying the 2D
Laws’ convolution filters for detecting levels,
edges, spots, waves, ripples and also combined
microstructures (L5L5, E5E5, S5S5, W5W5,
R5R5, S5R5, R5S5), in the case of the Laws’
based CTMCM (Laws, 1980).
the results obtained after applying edge detection
filters: the Sobel filters for identifying horizontal
and vertical edges (Meyer-Base, 2009), the
Kirsch Compass filters for finding edges with
different orientations (Kirsch, 1971), as well as
the Laplacian convolution filter, in the case of
the edge-based CTMCM (Meyer-Base, 2009).
(2) Then, we applied an improved k-means
clustering method, in the following manner: we
started from a minimum number of centres (k=50);
this number was increased by splitting the
corresponding centres; a centre was split in two
other centres, if the standard deviation of the items
within the corresponding class (cluster) overpassed
the threshold equal with ¾ of the average standard
deviation of all the existing classes. The newly
resulted centres were computed as being 1/2 of the
old centre, respectively 3/2 of the old centre.
(3) All the labels of the pixels from the region of
interest (ROI) were re-assigned after splitting the old
centres, then the step (2) was performed again. The
condition for the algorithm to finish was the
convergence, the maximum number of centres being
also established to 200. The solution of the
algorithm (the optimal solution) corresponded to the
minimum value of WCSS (Within Cluster Sum of
Squared Errors) (Duda, 2003). Thus, the definition
of the Complex Textural Microstructure Co-
occurrence Matrix (CTMCM) was provided in (1):
)}sgn()))(sgn((,
),..sgn()))(sgn((
|,||||,..,||||,|||
|,||||,..,||||,|||
,),(,..,),(,),(
:)),(),..,,(),,(),,{((#),..,,(
1111
111212
11213112
11213112
222111
33221121
nnnn
nn
nn
nnn
nnnD
ydxdyyxx
ydxdyyxx
ydyyydyyydyy
xdxxxdxxxdxx
tyxAtyxAtyxA
yxyxyxyxtttC
(1)
In (1), the # symbol represents the number of
elements of the set specified between the braces and
(2)
are the displacement vectors. A stands for the
attribute associated to each pixel, while t
1
, t
2
,..., t
n
are the values of the textons (cluster labels) obtained
after the application of the improved k-means
clustering algorithm. Thus, each element of the
CTMCM matrix represents the number of the n-
tuples of pixels, with the coordinates (x
1
, y
1
), (x
2
,
y
2
),…, (x
n
, y
n
) having the cluster labels t
1
, t
2
,..., t
n
and
being in a spatial relationship defined by the
displacement vectors. We computed the CTMCM
matrix of order two and three and we determined the
corresponding Haralick parameters, in a similar way
as described in (Mitrea et al., 2015). For the
CTMCM of order two, the following directions were
considered: 0°, 90°, 180°, and 270°. For the
CTMCM of order three, the current pixel was
considered in the central position and together with
the other two pixels, they were either collinear, or
formed a right angle triangle (the current pixel being
in the position of the right angle). We considered the
following orientations for the two displacement
vectors: (0°, 180°), (90°, 270°), (45°, 225°), (135°,
315°) for the case of collinear pixels; (0°, 90°), (90°,
180°), (180°, 270°), (0°, 270°), (45°, 135°), (135°,
225°), (225°, 315°), (45°, 315°), for the right angle
triangle case. The displacement vectors had the
absolute value 2, in both cases. We determined the
CTMCM matrices for all the considered direction
combinations of the displacement vectors, the final
features resulting as an average between the
Haralick features of the individual matrices, just as
in the case of the superior order GLCM, respectively
EOCM. We also extended the cluster shade and
cluster prominence features at order n, and we
computed the maximum area of the intersection of
the corresponding histogram with a horizontal plane,
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
181
as described in (Mitrea et al., 2016); (Nanni et al.,
2013).
3.3 The Learning Phase
3.3.1 Feature Selection Methods
In order to derive the set of the relevant textural
features, the following methods were considered,
which provided the best results in our former
experiments: Correlation based Feature Selection
(CFS) combined with genetic search, respectively
Gain Ratio Attribute Evaluation combined with the
Ranker method. The method of Correlation-based
Feature Selection (CFS) assigned a merit to each
group of features with respect to the class parameter
(Hall, 2003), as described by the formula (3). In (3)
__
cf
r
represents the average correlation of the features
to the class,
__
ff
r
is the average correlation between
the features, while k is the number of the elements in
the subset.
(3)
The second technique assessed the individual
features by assigning them a gain ratio with respect
to the class (Hall, 2003), as provided in (4). In (4),
H(C) is the entropy of the class parameter, H(C|A
i
)
is the entropy of the class after observing the
attribute A
i
, while H(A
i
) is the entropy of the
attribute A
i
.
GainR(A
i
) = (H(C) - H(C|A
i
)) / H(A
i
)
(4)
The final relevance score for each feature was
obtained by computing the arithmetic mean of the
individual relevance values provided by each
method. Only those features that had a significant
value for the final relevance score (above a
threshold) were taken into account.
3.3.2 The Specific Values for the Relevant
Textural Features
The specific values of the relevant textural features
(the arithmetic mean, the standard deviation,
respectively the probability distribution) were
computed using the Weka library (Weka, 2017). In
order to determine the probability distribution for
each feature, with respect to the class, the method of
Bayesian Belief Networks (Duda, 2003) was
adopted. The technique of Bayesian Belief Networks
detects influences between the features, by
generating a dependency network, which is
represented as a directed, acyclic graph (DAG). In
this graph, the nodes represent the features, while
the edges stand for the causal influences between
these features, having associated the values of the
corresponding conditional probabilities. Every node
X in this graph has a set of parents, P and a set of
children, C. Within a Bayesian Belief Network, each
node has associated a probability distribution table,
indicating the specific intervals of values for that
node, given the values of its parents.
3.3.3 Textural Model Assessment through
Supervised Classification
The following supervised classifiers and classifier
combinations, well known for their performance,
which provided the best results in the context of our
experiments, were taken into account: the Multilayer
Perceptron (MLP), the Support Vector Machines
(SVM), the Random Forest (RF) classifier, as well
as the AdaBoost meta-classifier combined with the
C4.5 method for decision trees. For the Multilayer
Perceptron, multiple architectures were
experimented and the best one was adopted in each
case.For classification performance assessment, we
considered the following parameters: the
classification accuracy (recognition rate), the TP rate
(sensitivity), the TN rate (specificity), as well as the
area under the ROC curve (AUC) (Duda, 2003).
4 EXPERIMENTS AND
DISCUSSIONS
For the experiments, we used 100 cases of cirrhosis,
300 cases of HCC, together with the cirrhotic
parenchyma on which HCC had evolved,
respectively 100 cases of hemangioma. All these
patients underwent biopsy, for diagnostic
confirmation. For each patient, 3 images were
considered, for various orientations of the
transducer. The images were acquired by our
research collaborators, the medical specialists from
the 3
rd
Medical Clinic of Cluj-Napoca, using a Logiq
7 ultrasound machine, at the same settings:
frequency of 5.5 MHz, the gain of 78 and the depth
of 16 cm. On each image, a region of interest,
having 50x50 pixels in size, was selected manually
inside the corresponding class of tissue, this process
being supervised by the medical specialists in
ff
cf
s
rkkk
rk
Merit
)1(
ICPRAM 2018 - 7th International Conference on Pattern Recognition Applications and Methods
182
radiology. A number of 90 textural features were
computed on each ROI independently on orientation,
illumination and region of interest size, using our
Visual C++ software modules. The values of the
textural features followed by the class specification
represented an instance of the training set. In the
context of binary classification, the classes were
combined in equal proportion inside the training set
(the two considered classes always had the same
number of instances). The feature selection methods
and the classification techniques were implemented
on the dataset using the Weka library (Weka3,
2017). For feature selection, we used Correlation
based Feature Selection (CFS) in combination with
genetic search, respectively Gain Ratio Attribute
Evaluation in combination with the ranker method,
as all these techniques provided the best results in
our experiments. For classification, we used the
following algorithms of the Weka library: John
Platt’s Sequential Minimal Optimization (SMO) for
implementing SVM; in this case, we used one of the
second or third degree polynomial kernels, the input
data being normalized. The J48 method, standing for
the C4.5 algorithm, was adopted as well; also, the
AdaBoost meta-classifier with 100 iterations, in
conjunction with the J48 method was employed. For
the RF method, we used the Weka’s algorithm with
the same name, with 100 trees. The
MultilayerPerceptron (MLP) classifier was also
adopted. Multiple architectures of this classifier
were experimented, in the following manner: the
total number of nodes was always equal with the
arithmetic mean between the number of attributes
and the number of classes
(no_of_attributes+no_of_classes)/2, while the
number of layers was one, two or three, the nodes
being equally distributed among the layers. For the
MLP classifier, the learning rate was 0.2 and the
value of α parameter was fixed to 0.8, in order to
achieve both high speed and high accuracy for the
learning process (Weka, 2017). After all the
parameters of these classifiers were assigned, the
strategy of cross-validation with 5 folds was
implemented on the dataset, for classification
performance assessment, using the Weka library.
4.1 The Role of the CTMCM Matrices
in the Differentiation between
Cirrhosis and Cirrhotic
Parenchyma around HCC
The relevant textural features derived from
the Laws' based CTMCM matrix
The set of the textural features which were relevant
concerning the differentiation between cirrhosis and
the cirrhotic parenchyma on which HCC had
evolved are provided in (5):
{Laws_CTMCM3_Energy, Laws_CTMCM3
_Entropy, Laws_CTMCM_Entropy, Laws_
CTMCM_Homogeneity, Laws_CTMCM3
_MaxArea, Wavelet_Entropy1, Wavelet_
Entropy4, Laws_CTMCM_Correlation,
Laws_CTMCM3_ClusterPromminence,
Laws_CTMCM3_Homogeneity, Wavelet
_Entropy5_hl, Wavelet_Entropy5_ll,
Wavelet_Entropy5_lh, Wavelet_Entropy5 _hh,
Wavelet_Entropy6_ll, Wavelet_ Entropy6_hl,
Wavelet_Entropy6_hh, Wavelet_Entropy6_lh,
Wavelet_Entropy2, Wavelet_Entropy3,
Min_Grey_Level, GLCM_Contrast, Edge_
Orientation_Variability, Directional_Grad
_Variabiliy, Laws_CTMCM_ClShade,
Laws_Spot_Frequency, Laws_CTMCM
_ClPromminence, Laws_Spot_Mean,
Laws_CTMCM3_ClusterShade}
(5)
We can notice the importance of the textural
features derived from the second and third order
Laws' based CTMCM matrix. Most of these features
were situated on the first seven positions within the
feature ranking. From this set, the second and third
order CTMCM Energy, the second order CTMCM
Entropy, as well as the second order CTMCM
Homogeneity denoted differences in homogeneity
between liver cirrhosis and the cirrhotic parenchyma
which surrounds HCC and also the more accentuated
chaotic character of the cirrhotic parenchyma
structure, due to a more evolved restructuring
process. The CTMCM maximum area parameter
emphasizes the more increased structural complexity
of the cirrhotic parenchyma on which HCC had
evolved, in comparison with cirrhosis. We can also
notice the relevance of the second and third order
CTMCM Cluster Promminence parameter and of the
third order CTMCM Cluster Shade feature. Another
important relevant feature was the CTMCM
Correlation, denoting differences in granularity
between the two considered classes, due to the
evolving restructuring process. While the features
derived from the CTMCM matrices together with
the Laws Spot Mean and with the Laws Spot
Frequency express differences in homogeneity,
structural complexity and granularity in terms of
textural microstructures, the other relevant textural
features, derived from the GLCM matrix or referring
to edge-based statistics, emphasize these differences
in terms of grey levels or edges. The entropy
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
183
computed at multiple levels, after applying the
Wavelet transform twice, is also present at a great
extent among the relevant features, expressing the
fact that the increasing chaotic character of the tissue
structure, which is due to the restructuring process,
can be visualized at multiple resolutions.
The relevant textural features derived from
the edge based CTMCM matrix
In order to assess the role of the edge based
CTMCM matrix in the differentiation between
cirrhosis and the cirrhotic parenchyma on which
HCC had evolved, we considered the set formed by
the previously existing textural features, respectively
edge based CTMCM features. The relevant textural
features resulted after applying the feature selection
methods described in 3.3.1 are depicted in (6).
{Edge_CTMCM_Max_Area, Wavelet_
Entropy4, Edge_CTMCM3_Max_Area,
Min_Grey_Level,
Edge_CTMCM_Entropy,
Wavelet_Entropy1,Wavelet_entropy5_lh,
Wavelet_Entropy5_hl, Wavelet_Entropy5_
hh, Wavelet_entropy5_ll,
Wavelet_Entropy6 _hh,
Wavelet_Entropy6_ll,
Wavelet_Entropy6_hl,
Wavelet_Entropy2, Edge_CTMCM3
_Entropy, Edge_CTMCM_Homogeneity,
GLCM_Contrast, Edge_Contrast, GLCM_
Variance, Wavelet_Entropy_3, Edge_
CTMCM_Energy, Directional_gradient_
variability, Laws_Spot_Frequency, Laws
_Spot_Mean, Laws_Ripple_Mean}
(6)
From (6), it results the importance of the newly
defined features Maximum Area derived from the
second and third order edge based CTMCM matrix,
denoting differences in structural complexity
between the two considered classes (cirrhosis and
cirrhotic parenchyma on which HCC had evolved).
The entropy derived from the second and third
order CTMCM matrix, as well as the homogeneity
computed from the CTMCM matrix of order two
are also met among the relevant textural features,
emphasizing, in terms of edges, differences in
homogeneity between the liver affected by cirrhosis
and the cirrhotic parenchyma on which HCC had
evolved. The other part of the set containing the
relevant textural features, illustrated in (6), has an
extended intersection with the relevant textural
feature set described in (5), the entropy computed
on various components resulted after applying the
Wavelet transform at multiple resolutions being
met in this case as well.
Class differentiation accuracy, due to the
CTMCM matrices
Figure 2 illustrates the comparison among the
classification accuracies resulted when considering
various CTMCM features in combination with the
already existing (old) textural features. The
following feature sets were taken into account: the
old textural features, including classical textural
features, superior order GLCM features and EOCM
features; the old textural features combined with the
edge-based CTMCM features; the old textural
features combined with the Laws' based CTMCM
features. Thus, we notice that the feature sets
including the newly defined textural features,
derived from the Laws' based CTMCM matrix and
from the edge-based CTMCM matrix led to an
increase in accuracy, for each considered classifier,
in comparison with the set containing only the old
textural features. The highest recognition rate, of
86.71%, resulted in the case of the AdaBoost meta-
classifier combined with the J48 method
(AdaBoost+J48), when considering the set
containing the old textural features combined with
the edge based CTMCM features. In this case, the
best structure of the MLP classifier, that provided
the highest values for the performance parameters,
consisted of three layers.
Figure 2: The increase in accuracy due to the newly
defined textural features when differentiating between
cirrhosis and cirrhotic parenchyma on which HCC had
evolved.
Concerning the arithmetic mean of the recognition
rates, for all the classifiers, in the case of each
textural feature set, the highest mean value, of
81.83%, resulted for the set containing the old
textural features combined with the edge based
CTMCM features, followed by that resulted when
combining the Laws' based CTMCM features with
ICPRAM 2018 - 7th International Conference on Pattern Recognition Applications and Methods
184
the old textural features (80.075%), and then by the
mean accuracy value which corresponded to the old
textural feature set (68.89%). Concerning the other
classification performance parameters, the highest
sensitivity (TP Rate) of 87.8% and the highest AUC
of 88.2%, resulted in the case of the AdaBoost meta-
classifier combined with the J48 technique, while
the highest specificity (TP Rate) of 85.2% resulted
in the case of the SMO classifier.
4.2 The Role of the CTMCM Matrices
in the Differentiation between HCC
and the Cirrhotic Parenchyma
around HCC
The relevant textural features derived from
the Laws' based CTMCM matrix
The relevant textural features obtained in this
case, when considering the set formed by the
previously existing textural features and the Laws’
based CTMCM features, are illustrated in (7).
{EOCM3_Energy, EOCM3_ Entropy,
GLCM7_entropy, Laws_CTMCM3
_Contrast, Laws_CTMCM_Contrast,
Laws_CTMCM_Correlation, GLCM5
_Entropy, EOCM3_Correlation,
GLCM3_Energy, GLCM7_Energy,
Wavelet_Entropy4, EOCM3_Contrast,
EOCM3_Variance, Laws_CTMCM
_Variance, Laws_CTMCM3 _Variance,
Wavelet_Entropy6_lh, Wavelet_
Entropy6_hl, Wavelet_Entropy6_ll,
Wavelet_entropy6_hh, Wavelet
_entropy5_ll, Wavelet_entropy5_lh,
Wavelet_entropy5_hl, Wavelet
_entropy5_hh, Wavelet_Entropy1,
Wavelet_Entropy2, Wavelet_Entropy3,
EOCM_Variance, GLCM7_Correlation,
GLCM7_Variance, Min_grey_level,
Laws_CTMCM_Homogeneity,
GLCM_Variance}
(7)
We remark that the contrast derived from the
second and third order CTMCM matrices, as well as
the correlation derived from the second order
CTMCM matrix are situated on the top of the
relevant feature ranking in this case. The variance
derived from the second and third order CTMCM
and the homogeneity computed from the second
order CTMCM are relevant as well. These features
emphasize the complex structure of the tumour
tissue (the contrast and the variance computed from
the second and third order CTMCM), the differences
in granularity which exist between HCC and the
cirrhotic parenchyma on which HCC had evolved
(the CTMCM Correlation) and also the
heterogeneous structure of the HCC malignant
tumour (the CTMCM Homogeneity). Concerning
the classical textural features, we notice the
importance of the textural features derived from the
second and third order EOCM (energy, entropy,
correlation, contrast and variance), from the GLCM
of order two, three, five and seven and also of the
Shannon entropy computed after applying the
Wavelet transform, at multiple resolutions. All these
features confirm the echogenicity increase in the
cases of HCC, as well as the complex and chaotic
structure of this malignant tumour.
The relevant textural features derived from
the edge based CTMCM matrix
The set of the relevant textural features obtained in
this situation is depicted in (8).
{EOCM3_Energy, EOCM3_ Variance,
Wavelet_Entropy1, GLCM5_Entropy,
GLCM7_Energy, Edge_CTMCM_MaxArea,
Edge_ CTMCM_Entropy, GLCM3_Energy,
Wavelet_Entropy4, EOCM3_Contrast,
GLCM7_Entropy, Wavelet_Entropy6_ll,
Wavelet_Entropy6_hh, Wavelet_ Entropy6_hl,
Wavelet_Entropy6_lh,
Wavelet_Entropy5_lh,Wavelet_Entropy5_ll,
Wavelet_Entropy5_hh, Wavelet_Entropy1,
EOCM3_Correlation, EOCM_Entropy,
EOCM3_Entropy, Wavelet_Entropy2,
GLCM7_Correlation, Min_Grey_Level,
Wavelet_Entropy4, GLCM_Variance,
Wavelet_Entropy3, EOCM3_Homogeneity,
GLCM_Contrast, Edge_CTMCM _Correlation,
Edge_CTMCM_Contrast}
(8)
We notice that the Maximum Area and the
Entropy derived from the second order CTMCM
matrix are situated on the top of the corresponding
ranking, while the CTMCM Correlation and
Contrast are also present in the relevant feature set.
Thus, the chaotic structure (CTMCM Entropy),
respectively the structural complexity (CTMCM
Maximum Area and Contrast) of HCC, as well as
the differences in granularity between HCC and the
cirrhotic parenchyma on which HCC had evolved,
are emphasized in this case, as well. Concerning the
old textural features, we remark the minimum grey
level, expressing the hyperechogenic nature of the
HCC tissue in most of the cases; the Energy
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
185
computed based on the third and seventh order
GLCM, respectively the Entropy derived from the
fifth order GLCM, expressing the differences in
echogenicity between HCC and the surrounding
cirrhotic parenchyma, respectively the chaotic
structure of the malignant tumour; the variance
derived from the second and seventh order GLCM
characterizing the complexity of the HCC tissue, the
correlation derived from the seventh order GLCM
standing for the differences in granularity between
HCC and the neighbouring cirrhotic parenchyma.
The features derived from the third order EOCM
matrix and the Shannon entropy computed after
applying the Wavelet transform at multiple
resolutions are also met at a great extent within the
relevant feature set, denoting the complex, chaotic
structure of the malignant tumour, in comparison
with the cirrhotic parenchyma on which this tumour
had evolved.
Class differentiation accuracy due to the
CTMCM matrices
Figure 3 depicts the comparison of the classification
accuracies obtained for each considered classifier for
the same types of feature sets as in the case of the
cirrhosis and cirrhotic parenchyma comparison. We
can notice that the classification accuracy is always
better for the feature sets containing the newly
defined textural features than in the case of the old
textural feature set. The best classification accuracy,
of 84.09% was obtained in the case of the AdaBoost
metaclassifier combined with the J48 method for the
set of the old textural features combined with the
Laws’ based CTMCM features. Concerning the
arithmetic mean value for all the considered
classifiers, the highest value, 79.68%, resulted in the
case of the old textural features combined with the
edge based CTMCM features, followed by the value
of 79.25% obtained in the case of the combination
between the old textural features and the Laws’
based CTMCM features, respectively by the value of
77.72% resulted in the case when only the old
textural features were considered. The MLP
classifier provided the best result when the
corresponding architecture had one single hidden
layer, in all the cases. Regarding the other
classification performance parameters, the highest
specificity (TN Rate) of 82.7% and the highest AUC
of 87.3%, resulted in the case of the adaboost meta-
classifier combined with J48, while the highest
sensitivity (TP Rate) of 82.7% resulted in the case of
the RF classifier.
Figure 3: The increase in accuracy due to the newly
defined textural features when differentiating between
HCC and the cirrhotic parenchyma on which HCC had
evolved.
4.3 The Role of the CTMCM Matrices
Concerning the Differentiation
between HCC and Hemangioma
The relevant textural features derived from
the Laws' based CTMCM matrix
The set of the relevant textural features resulted in
this case is illustrated in (9):
{Laws_CTMCM_Homogeneity,
Laws_CTMCM_Correlation,
GLCM7_Entropy, GLCM5_Contrast,
Laws_CTMCM3_Energy,
Laws_CTMCM3_Contrast,
Directional_Gradient_Variability,
GLCM5_Entropy, GLCM3_Energy,
GLCM3_Homogeneity, Wavelet_
Entropy5_hl, Wavelet_Entropy6_hh,
Wavelet_entropy6_hl, GLCM_ Homogeneity,
Wavelet_Entropy7_hl,
GLCM5_Homogeneity}
(9)
The CTMCM Homogeneity and the CTMCM
Correlation are in the top of the relevant feature
ranking. They emphasize differences in
homogeneity (CTMCM Homogeneity) and in
granularity (CTMCM Correlation) between the
malignant and the benign tumour. The third order
CTMCM Energy and third order CTMCM Contrast
are also important, standing for the differences in
homogeneity, echogenicity (third order CTMCM
Energy) and tissue structure complexity (third order
CTMCM Contrast) between HCC and hemangioma.
Among the relevant textural features obtained in this
case we can also notice the features computed based
on the second and superior order GLCM, the
ICPRAM 2018 - 7th International Conference on Pattern Recognition Applications and Methods
186
directional gradient variability and the entropy
computed after applying the Wavelet transform at
the second level, on the third (high-low) and fourth
(high-high) components, containing the horizontal,
respectively the diagonal edges. All these features
denote the heterogeneous, chaotic, complex structure
of HCC, compared with the more homogeneous
structure of the benign tumour.
The relevant textural features derived from
the edge based CTMCM matrix
The set of the relevant textural features resulted in
this situation is provided in (10):
{EOCM3_Energy, GLCM7_Entropy,
EOCM3_Entropy, GLCM7_Energy,
GLCM3_Entropy, Edge_orientation
_Variability, Directional_Gradient
_Variability, GLCM5_Energy, GLCM3
_Homogeneity, Edge_CTMCM _MaxArea,
Edge_CTMCM3_Contrast,
GLCM_Homogeneity, Wavelet_Entropy7
_hl, GLCM5_Homogeneity,
GLCM7_Variance, Wavelet_Entropy3,
GLCM5_Contrast, Edge_CTMCM3
_Correlation, Directional_Gradient
_Variance, GLCM_Entropy, Laws
_Ripple_Mean, Edge_CTMCM_Contrast,
Autocorrelation_Index, Mean_Gray_Level}
(10)
We can notice, from (10), that the CTMCM
features are present within the set of the relevant
textural features in this case (the CTMCM
Maximum Area, the third order CTMCM Contrast,
respectively the second order CTMCM Contrast).
These features denote differences in structural
complexity between HCC and the hemangioma
benign tumor. The other features which are among
the most relevant textural parameters in this case are
those derived from the GLCM matrix of second and
superior order, which occupy a large portion of the
diagram, some features computed from the EOCM
matrix of order three (the Energy and Entropy), the
edge orientation variability, the autocorrellation
index, the mean grey level, the arithmetic mean of
the pixels values after applying the Laws'
convolution filter for ripple detection, respectively
the entropy computed on both first and second levels
after applying the Wavelet transform twice. All
these features confirm the chaotic, inhomogeneous,
complex character of the malignant tumor tissue.
The specific values of the edge based CTMCM
maximum area parameter, for each class (HCC and
hemangioma), obtained after applying the technique
of Bayesian Belief Networks in Weka 3.6, are
depicted within Table 1. It results that the
distribution of the values of the edge based textural
microstructures is more balanced in the case of
hemangioma, while in the case of HCC, these values
are probably grouped towards higher values, as it is
assumed that the edges are more emphasized in this
case, due to the more complex structure of the
malignant tumor.
Table 1: The probability distribution table for the
CTMCM Maximum Area.
(-∞, 64658.5]
(64658.5, ∞)
Hemangioma
0.38
0.62
HCC
0.77
0.22
Class differentiation accuracy due to the
CTMCM matrices
Figure 4 illustrates the comparison of the
classification accuracies obtained when considering
the feature sets mentioned before. We can notice a
classification accuracy increase in all cases, due to
the newly defined textural features. The best
classification accuracy, of 88.41%, resulted in the
case of the AdaBoost meta-classifier combined with
the J48 classifier, corresponding to the feature set
containing the old textural features and the Laws’
based CTMCM features. In the case of the
arithmetic mean (average) of the recognition rates,
the most increased value (84.32%) resulted for the
feature set that contained the Laws’ based CTMCM
features, followed by the value of 83.76% obtained
in the case when taking into account the edge based
CTMCM features, then by the value of 82.005%
resulted when taking into account only the old
textural feature set. Concerning the MLP classifier,
it provided the best results when a one layer
architecture was adopted, for the feature set
containing only the old features, as well as for the
feature set containing the old textural features and
the edge based CTMCM features, respectively,
when a two layers architecture was adopted, for the
feature set containing the old textural features and
the Laws’ based CTMCM features. The other
classification performance parameters resulted as
follows: the highest sensitivity (TP Rate), of 84.9%
was obtained in the case of the AdaBoost meta-
classifier combined with the J48 method; the highest
sensitivity (TN Rate) of 88.6% was obtained in the
case of SMO; the highest AUC of 89.9%, resulted in
the case of MLP.
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
187
Figure 4: The increase in accuracy due to the newly
defined textural features when differentiating between
HCC and Hemangioma.
4.4 Discussions
The newly defined CTMCM textural features always
resulted among the most relevant textural features in
our experiments, being part of the imagistic textural
model in the considered cases. These relevant
textural features highlighted the increase of the
echogenicity, heteorogeneity, chaotic structure and
structural complexity of the tissue during the
restructuring process, from cirrhosis to HCC and
also the differences between the malignant and the
benign tumoral tissue (HCC versus hemangioma).
Another aspect to be noticed is that the features
referring to the skewness and kurtosis derived from
the superior order histograms (the CTMCM Cluster
Shade and the CTMCM Cluster Promminece),
respectively the maximum intersection area with a
horizontal plane, usually occupied a place in the top
of the relevant feature ranking, highlighting, once
again, the complex structure of the malignant
tumour. We can notice that the edge based CTMCM
textural features led to a better classification
performance than the Laws’ based CTMCM textural
features, when differentiating between cirrhosis and
cirrhotic parenchyma on which HCC had evolved,
respectively between cirrhotic parenchyma and
HCC. It results the importance of the edges
concerning the refined differentiation between the
different phases of liver parenchyma restructuring.
In the case of differentiation between HCC and
hemangioma, the Laws’ based CTMCM textural
features overpassed the edge based CTMCM
textural feature from classification performance
point of view. Thus, in this case, the Laws’ based
textural microstructures better emphasized the
differences between the malignant and benign liver
tumours. The best obtained classification
performance was above 88% in the case of the
differentiation between the malignant and the benign
liver tumours, respectively above 84% when
differentiating between HCC and the cirrhotic
parenchyma. The classifier that provided the best
classification accuracy was AdaBoost combined
with the J48 technique, this combination scheme
being well known for its performance.
Comparison with the state of the art results
The classification accuracy due to the newly
defined textural features, in combination with the
formerly existing textural features always
overpassed the classification accuracy which was
due only to the formerly existing textural features, as
shown in the experiments above. Also, this accuracy
is comparable with that of the state of the art
algorithms, in the case of the differentiation between
malignant and benign liver tumors: 88.41% in the
case when the CTMCM matrices were employed;
above 80% when using classical textural features
and classifiers (Sujana et al., 1996); about 90%
when using hierarchical wavelet-based features
(Yoshida et al., 2003). In the case when
distinguishing between the malignant liver tumours
and the cirrhotic parenchyma, a recognition rate of
84.09% was obtained when employing the CTMCM
matrices within B-mode ultrasound images (in our
case), while in the case of employing textural
features derived from CEUS images, in combination
with classifiers, an accuracy of 90% resulted (Duda
et al., 2013). As it can be noticed, the accuracy is
lower in the case of differentiation between HCC
and the cirrhotic parenchyma, due to the fact that,
especially during the intermediate evolution phases,
HCC resembles sometimes the surrounding cirrhotic
parenchyma. Thus, our solution can be further
improved by using more advanced multi-resolution
techniques, as well as CEUS images, and also by
explicitly taking into account different HCC
evolution phases.
5 CONCLUSIONS
The CTMCM matrices demonstrated an obvious
contribution concerning the increase of the
classification performance and diagnosis accuracy in
the case of the HCC malignant tumour and of the
pathological phases that precede this form of liver
cancer (cirrhosis). The corresponding features were
always among the most relevant textural features,
considerably improving the imagistic textural model
of the considered pathologies. At the end, the
resulted classification accuracy was above 84%, in
ICPRAM 2018 - 7th International Conference on Pattern Recognition Applications and Methods
188
all cases. In our future research, we aim to further
increase the HCC automatic diagnosis accuracy by
employing the multi-resolution versions of the
CTMCM matrices and of the corresponding
Haralick features. We will also consider larger
datasets in order to improve the validation procedure
and deep learning methods in order to increase the
classification performance. We take into account the
possibility of using other types of ultrasound images,
as well, such as contrast enhanced ultrasound images
(CEUS), respectively elastographic images.
REFERENCES
American liver foundation, 2015. Online: http://www.
liverfoundation.org/abouttheliver/info/
Sherman, M. 2005. Approaches to the Diagnosis of
Hepatocellular Carcinoma. In Current
Gastroenterology Reports, vol. 7, no. 1, pp.11-18.
Sujana, H., Swarnamani, S., 1996. Application of
Artificial Neural Networks for the classification of
liver lesions by texture parameters. In Ultrasound in
Medicine & Biology, vol. 22, no. 9, pp.1177 1181.
Yoshida, H., Casalino, D., 2003. Wavelet packet based
texture analysis for diferentiation between benign and
malignant liver tumors in ultrasound images. In
Physics in Medicine & Biology, vol. 48, pp. 3735-
3753.
Madabhushi, A., 2005. Automated Detection of Prostatic
Adenocarcinoma from High-Resolution Ex Vivo MRI.
In IEEE Transactions on Medical Imaging, pp.1611-
1626.
Chikui, T., 2005. Sonographic texture characterization of
salivary gland tumors by fractal analysis, In
Ultrasound in Medicine & Biology, vol. 31, no. 10,
pp.1297- 1304.
Duda, D., Kretowski, M., 2013. Computer-Aided
Diagnosis of Liver Tumors Based on Multi-Image
Texture Analysis of Contrast-Enhanced CT. Selection
of the Most Appropriate Texture Features. In Studies
in Logic, Grammar and Rhetoric, vol. 35, no. 1, pp.
4970.
Khuzir, A., Besar, R., 2009. Identification of masses in
digital mammogram using gray level co-occurrence
matrices. In Biomedical Imaging Intervention Journal,
vol. 5, no. 3, pp. e17-e30.
Masood, K., 2006. Co-occurrence and morphological
analysis for colon tissue biopsy classification. In
Proceedings of the 4th International Workshop on
Information Technology, 2006, Pakistan, pp. 211-216.
Meyer-Base, A., 2009. Pattern recognition for medical
imaging, Elsevier, San Diego, California.
Mitrea, D., Mitrea, P., Nedevcshi, S., Lupsor, M.,
Socaciu, M, Golea, A., Hagiu, C., Ciobanu, L., 2012.
Abdominal tumor characterization and recognition
using superior order co-occurrence matrices, based on
ultrasound images. In Computational and
Mathematical Methods in Medicine, 17 pages.
Mitrea, D., Nedevcshi, S., Socaciu, M., Badea, R., 2012.
The Role of the Superior Order GLCM in the
Characterization and Recognition of the Liver Tumors
from Ultrasound Images. In Radioengineering, vol.
21, no. 1, pp. 79-85.
Mitrea, D., Nedevschi, S., Badea, R., 2014. The Role of
the Textural Microstructure Co-occurrence Matrices
in the Classification of the Abdominal Tumors. The
10th IEEE ICCP 2014 Conference, Technical
University of Cluj-Napoca , pp. 187-190.
Mitrea, D., Nedevschi, S., Abrudean, M., Badea, R., 2015.
Colorectal cancer recognition from ultrasound images,
using complex textural microstructure co-occurrence
matrices, based on Laws’ features, In proccedings of
the 38
th
IEEE International Conference on
Telecommunications and Signal Processing, 2015
(TSP 2015), Praga, pp. 458-462.
Mitrea, D., Nedevschi, S, Badea, R., 2016. The Role of the
Complex Extended Textural Microstructure Co-
occurrence Matrix in the Unsupervised Detection of
the HCC Evolution Phases, based on Ultrasound
Images. In Proceedings of International Conference on
Pattern Recognition Applications and Methods
(ICPRAM 2016), Roma, pp. 698-705.
Davis, L.S., Clearman, M., Aggarwal, J., 1979. A
Comparative Texture Classification Study Based on
Generalized Cooccurence Matrix. Proceedings of the
18-th IEEE Conference on Decision and Control, pp.
71-78.
Sujatha, B., Kumar, V., Harini, P., 2012. A new logical
compact LBP Co-Occurrence Matrix for Texture
Analysis. In International Journal of Scientific &
Engineering Research, vol. 3, no. 2, pp. 1-5, 2012.
Sujatha, B., Sekhar Reddi, C., 2013. Texture classification
using texton co-occurrence matrix derived from
texture orientation. In International J. Soft Computing
and Eng, vol. 2, no. 6, pp. 18-23.
Laws, K.I., 1980. Rapid texture identification. In SPIE
Vol. 238 Image Processing for Missile Guidance,
pp.76-380.
Nanni, L., Brahnam, S., Ghidoni, S., Menegatti, E.,
Barrier, T., 2013. “Different Approaches for
Extracting Information from the Co-Occurrence
Matrix”. In PloS One Journal, Vol.8, No. 12, 2013, pp.
1-9.
Kirsch, R., 1971. Computer determination of the
constituent structure of biological images. In
Computers and Biomedical Research, Vol.4, No.3, pp.
315328.
Hall, M, 2003. Benchmarking attribute selection
techniques for discrete class data mining. In IEEE
Trans On Knowledge and Data Eng., vol. 15, no. 3,
pp. 1-16.
Weka3, Data Mining Software in Java, 2017. Online:
http://www.cs.waikato.ac.nz/ml/weka.
Duda, R., 2003 Pattern Classification, Wiley Interscience,
Hoboken, New Jersey.
Automatic Recognition of the Hepatocellular Carcinoma from Ultrasound Images using Complex Textural Microstructure Co-Occurrence
Matrices (CTMCM)
189