Segmentation of Shoulder MRI Data for Musculoskeletal Model

Adaptation

Tom

s Ryba and Zden

ek Kr

noul

Department of Cybernetics, University of West Bohemia, Univerzitni 8, Pilsen, Czech Republic

Keywords:

Image Segmentation, MRI, Medical Imaging, Deltoid Muscle, Humerus, Clavicle, Scapula.

Abstract:

Applying image processing techniques to medical images has already brought many useful applications. This

work is focused on using these methods in the process of adapting a musculoskeletal model of the shoulder

joint. Comparing the model of healthy individuals and the patient with joint damage leads to a subject-

speciﬁc convalescence treatment. This work describes the procedure for segmentation of magnetic resonance

imaging (MRI) of the shoulder joint. Firstly three bones inside the shoulder area: humerus, clavicle, scapula

are identiﬁed and thereby it provides initial reference objects. A major step is the segmentation of the deltoid

muscle needed for the subsequent adaptation of the musculoskeletal model. This step is challenging in terms of

image processing due to the closeness of soft tissues, which are almost identical in intensity and the boundaries

between them are often barely visible. The approach to resolving this problem is described and possible

improvements and future work are described.

1 INTRODUCTION

Automated segmentation of medical image data is

still an unresolved task. There are computed to-

mography (CT) data that are characteristic of their

high image quality. At work (Kodym and

Span

el,

2018), the authors deal with a semi-automatic seg-

mentation method for general use. However, existing

applications or algorithms for segmentation of mag-

netic resonance imaging (MRI) data are always semi-

automatic and require some user intervention to set

initial parameter values that are often speciﬁc to the

task only. Therefore, for MRI data, we can ﬁnd Vas-

cular modeling toolkits (2016) or voice tract segmen-

tation capturing the articulation of healthy subjects

(Engwall and Badin, 1999), (Ojalammi and Malinen,

2017), etc.

There are also applications for MRI segmenta-

tion usable for general task but require very time-

consuming user inputs to deﬁne initial conﬁgurations.

These parameter values are often different for dif-

ferent anatomies, can not always be derived directly

from the data, and the user usually has to proceed

based on an attempt of error. Recently, there are

very successful general segmentation methods of the

supervised machine learning based on convolutional

neural networks (Xue et al., 2018), (Liu et al., 2018).

This solution, however, leads to an enormous amount

of manual work during the preparation of the large

training data needed as reference for their learning.

On the other hand, in (Ojalammi and Malinen, 2017),

the aim is to process extensive MRI data sets of upper

respiratory and oral routes with minimal user interfer-

ence.

The examination of MRI is useful to improve the

current understanding of the human body’s relation-

ships. The MRI imaging technique is an attractive

alternative to CT because it has no-ionizing radia-

tion. This aspect is particularly important for the pro-

cessing of healthy subjects. We are considering the

healthy subjects for an identiﬁcation of the muscu-

loskelatal model ((Havelkov

a et al., 2017)). On the

other hand, the disadvantage of MRI against CT, is the

worse spatial resolution given by the low raw voxel

data quality. This is caused by the imaging princi-

ple including motion artifacts due to a scanning time,

which can exceed 10 seconds for one high resolution

stationary 3D image.

The aim of this work is segmentation and sub-

sequent 3D reconstruction of the deltoid muscle of

the shoulder complex from MRI volume data. The

task is to separate the muscle from the background.

The background consist of other muscles and bones,

moreover some of them anatomically attached to the

segmented muscle. The 3D reconstruction transforms

segmented deltoid muscule to 3D surface model that

Ryba, T. and Kr

noul, Z.

Segmentation of Shoulder MRI Data for Musculoskeletal Model Adaptation.

DOI: 10.5220/0007580701550160

In Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019), pages 155-160

ISBN: 978-989-758-353-7

155

is the input to musculoskeletal simulation of the

shoulder complex.

We try to perform the segmentation with minimal

user interference. Our work is directed to subsequent

research needing processing of extensive data of both

healthy individuals and patients with a shoulder com-

plex disorder. Therefore the application is not speciﬁ-

cally designed for doctors to examine the patient, nor

is it an alternative to the software and user interface

provided by the MRI scanner manufacturer.

2 RELATED WORK

The hard tissue segmentation is a well-managed

technique in terms of their good visibility, in both

computed tomography (CT) and magnetic resonance

imaging (MRI). It is often used for diagnostic pur-

poses, such as assistance in preoperative planning

or initiation of downstream segmentation techniques

of other non-hard tissues. Precise and, at the same

time, fully automatic segmentation of hard tissue is

achieved when the patient’s anatomy does not devi-

ate signiﬁcantly from the standard. The results are

currently also used for example for 3D implant print-

ing (Tetsworth et al., 2017) or teaching aids (Ji

ık

et al., 2014).

The different non-hard tissues have low contrast

borders with each other in both CT and MRI data and

make its automatic segmentation very difﬁcult also

due to anatomical variability or various pathologies.

In this case, the semi-automatic approaches are cur-

rently successful. The user searches for speciﬁc al-

gorithm parameter values to achieve the required seg-

mentation result accuracy. Although some applica-

tions do not require very anatomically precise surface

models, for the precise segmentation, it must to be

done by clinical professionals processing volumetric

data one slice after another. This is very challenging

due to tedious processing of a large number of slices

in volume data.

The segmentation of the hard tissue is sufﬁcient by

adaptive thresholding techniques (Rathnayaka et al.,

2011) or area growth algorithm (Xi et al., 2014). The

segmentation failure is where the boundaries of an

object pass through areas of the low contrast. To

deal with this problem, there are approaches based

on models of active contours, (Pinheiro and Alves,

2015), optimizing the smoothness and a continuity

criteria.

Moreover, the methods of active or statistical

shapes (He et al., 2016) and (Yokota et al., 2013) as-

sume prior shape of a segmented object. However,

this must be obtained by learning from previous (often

manual) ideal segmentation of all potentially possi-

ble shapes. This can be a problem with unpredictable

anatomical pathologies. On the other hand, the user

input during their application degrades only on the

pre-positioning of the model in reference pose as near

as possible to the location (Virz

ı et al., 2017) or, in the

case of a fully automated method, this step is com-

pletely eliminated (Antong et al., 2010).

As another form of segmentation, the image reg-

istration can be considered (Hajnal and Hill, 2001).

This is not the pure segmentation technique because

needs a reference image in addition. It determines

the geometric relationship between each point of the

reference image and the processed image as a cost-

optimization function. However there are unregulated

registration algorithms for comparing deform-able or-

gans such as the brain, liver or lung (Rohlﬁng and

Maurer, 2003), (Ino et al., 2005). The failure of regis-

tration caused by dropping search algorithm into local

optima is prevented through generating a large train-

ing set for the deep-learning image registration (Ito

and Ino, 2018).

More recently, graph-based methods provide bi-

nary segmentation as the search for a global optima

separating object from the background. These meth-

ods are reliable if the user again provides a sufﬁcient

amount of accurate user inputs. For MRI or CT, it

has a form of seeds labeled in many slices of vol-

umetric images. Furthermore automation reducing

amount of manual inputs beneﬁts from a combina-

tion of graph-cut techniques with prior information

provided by some edge detection method (Keuster-

mans et al., 2012), (Kr

cah et al., 2011) or a classiﬁca-

tion technique based e.g. on random decision forests

(RDF) (Kodym and

Span

el, 2018). The last men-

tioned method searches for optimal binary segmenta-

tion of volumetric data with respect to the probability

ﬁeld obtained from RDF classiﬁers online trained on

only a few expertly annotated sections.

In general, the convolutional networks are ma-

chine powerful learning techniques and are currently

a successful segmentation technique for the medical

imaging data (Ghosal and Ray, 2017), (Prasoon et al.,

2013). They overcome previously popular segmen-

tation techniques based on RDF classiﬁcation (Loh,

2011), which uses random subsets of available train-

ing data to build a set of binary decision trees. In

the context, these data-driven and supervised tech-

niques need training data that is very varied due to

a wide range of imaging techniques used in medicine.

There are methods increasing accuracy and robust-

ness by generating thousands of synthetic training

data from only a few input original images (Ito and

Ino, 2018) and-or often combining with an augmen-

BIOIMAGING 2019 - 6th International Conference on Bioimaging

156

tation method (Milletari et al., 2016), (Ronneberger

et al., 2015).

3 METHODS

3.1 Image Preprocessing

A common problem in image processing is the image

noise. In order to deal with it, several options may

be used. The commonly used method is the Gaussian

ﬁltering. Unfortunately, an obvious disadvantage of

this approach is blurring of image edges. As already

stated main goal of this work is to segment one cer-

tain muscle in the shoulder complex. The boundary

between this muscle and background is barely visi-

ble. Therefore, it’s not a reasonable approach to blur

them even more.

To reduce the image noise and leave the edges

untouched at the same time an edge preserving ﬁlter

must be used. In this work the bilateral ﬁlter (Tomasi

and Manduchi, 1998) is used. The weights in this ﬁl-

ter correspond not only to distance as in the Gaussian

ﬁlter but to the intensity difference of pixels as well.

This way, only pixels that are geometrically close and

have similar intensity are taken into account during

ﬁltering. And because edges are deﬁned as pixels

with high intensity gradient, they’re usually ﬁltered

very slightly.

Comparison of the Gaussian ﬁlter and the bilateral

ﬁlter is shown in Figure 1.

3.2 Image Segmentation

For the purpose of image segmentation, a semiauto-

matic tool was developed. It works on an established

system of placing seed points representing the object

of interest and another group of points representing

the background. For the algorithm processing the data

with deﬁned seed points, we tested three commonly

used methods - Graph Cut (GC) (Boykov and Jolly,

2000), Random Walker (RW) (Grady, 2006) and Wa-

tershed (WS) (Dobrin et al., 1994).

The GC method is based on creating intensity

models for each object. These models are calcu-

lated from given seed points. Using the GC often

leads to overtrained models. This is due to very

similar object densities when segmenting one spe-

ciﬁc muscle. In this case, points with similar den-

sity are marked as two different objects. Moreover, to

mark the background object properly the correspond-

ing model needs to describe image parts that signiﬁ-

cantly differ in intensity. The resulting model is not

descriptive enough and the algorithm gets confused.

Given the set of seed points, the RW algorithm

ﬁnds the closest path to one of the seed point for each

unlabeled pixel. There are no direct connections to an

intensity model like in the GC algorithm. Therefore,

the closest path depends strongly on the seed position,

which often yields to a signiﬁcantly higher amount

of needed seed points that spreads on all parts of the

object. The needed interactivity was overwhelming

especially when segmenting the muscle.

Using the simple WS algorithm provides us with

the best results regarding the precision of segmen-

tation and the amount of needed interaction. An-

other important advantage of this approach is its ef-

ﬁciency and computation speed. Using the WS al-

gorithm makes the whole segmentation process faster

and more ﬂuent.

To help the operator with orientation it’s possible

to switch between different views - coronal, sagittal

and axial. This way, it is possible to change the views

during the segmentation process to deﬁne the seed

points more accurately.

3.2.1 Extracting the Bones

For easier orientation and further data processing,

three bones are segmented: humerus, scapula, and

clavicle. The Humerus is a long bone of the arm that

forms the shoulder joint on the one end and the el-

bow joint on the second end. The Scapula or shoulder

blade is a triangular bone that lies on the upper back.

The Clavicle is an anterior bone of the shoulder. Its

main function is to support the shulder.

In the MRI the bones are well separated due to

their high intensity. The contrast between a bone and

near soft tissues is signiﬁcant. Therefore, the needed

amount of interactivity is much lower comparing to

the muscle segmentation.

Nevertheless, to use a fully autonomous approach,

e.g. thresholding, is not recommended. Despite the

bones, the’re different objects with the similar inten-

sity that would be segmented as well. Using such an

approach often yields to results where for example the

clavicle and the skin are connected into one big ob-

ject.

On the other hand, the WS algorithm is perfectly

suited for this task. The ﬁnal segmentation of the

humerus is shown in the second image in Figure 2.

3.2.2 Extracting the Deltoid

Segmentation of one speciﬁc muscle is a much more

challenging task. The reason for this is the intensity

similarity of the soft tissues and an unclear boundary

between individual muscles. The risk of overtraining

a segmentation algorithm based on intensity models is

Segmentation of Shoulder MRI Data for Musculoskeletal Model Adaptation

157

Figure 1: Input image (left) ﬁltered by Gaussian (middle) and by a bilateral ﬁlter (right).

signiﬁcantly higher than in the bones extraction case

and the amount of interactivity increases.

Moreover, this task is much more focus demand-

ing than the bone segmentation. A certain amount of

anatomy knowledge is also needed, especially in parts

with an unclear boundary between individual mus-

cles. Segmentation of bones prior to the muscle seg-

mentation is recommended for a better orientation.

The WS algorithm meets its limits here but still

provides reasonable results. Therefore, the segmenta-

tion reﬁnement that is described in the next section is

very important in these cases.

3.3 Segmentation Reﬁnement

The segmentation obtained by the WS algorithm is

sometimes very coarse and inaccurate. The reason

for that is the algorithm’s sensitivity to the noise and

unclear object boundaries. Despite the preprocessing

and proper noise ﬁltration, the ﬁltered data are still not

perfect. Is it, therefore, appropriate to use a segmenta-

tion method that will start at the coarse segmentation

and will reﬁne the result to better match the reality.

An information that could be used in this step is

that most of the objects in the human body tend to be

compact with no sudden changes in shape. A perfect

approach that respects such an information is the ac-

tive contours approach (Pinheiro and Alves, 2015).

Methods based on active contours need to be ini-

tialized by an initial curve. This initial curve should

be as close to the desired result as possible, which

minimizes the possibility of getting stuck in a local

energy minimum. In this work, this initial curve cor-

responds to the boundary of the coarse segmentation

achieved from the image segmentation. During the

iterative process the curve evolves and due to the cal-

culation tends to be smooth and compact.

The last step in the postprocessing procedure is

the morphological ﬁltering. This way the boundary

of the segmentation is smoothed even more, which

yields more reliable results.

4 RESULTS

In this work, we used the MRI of a 30 years old

healthy male subject. The ﬁnal segmentation of the

humerus and the deltoid muscle is shown in Figure 2.

The main result of this work is the software for

MRI processing. The developed software is used for

segmentation of speciﬁc bones and soft tissues in the

shoulder.

The segmentation is done with an interactive ver-

sion of the watershed algorithm. Using this approach

yields to fast responses of the algorithm as the user

draws seed points over the input data. This coarse

segmentation is then reﬁned using an active contours

method.

As more data are processed the software will learn

and the future processing should be faster.

5 CONCLUSIONS AND FUTURE

WORK

To develop a fully autonomous segmentation pro-

cess is a very challenging task especially regarding

muscle segmentation. We faced this challenge using

an interactive method based on the watershed algo-

rithm. Probably the biggest disadvantage of this ap-

proach is the amount of needed interactivity in some

cases. This interactivity could be reduced using sev-

eral ways.

For example, a thresholding algorithm could be

used for segmentation of the bones. As already men-

BIOIMAGING 2019 - 6th International Conference on Bioimaging

158

Figure 2: Segmentation of bones (the humerus on the ﬁrst two images) and segmentation of soft tissues (deltoid muscle on

the last two images).

tioned, using such an approach could yield to a big

object consisting of a bone and other objects. We be-

lieve that with a proper postprocessing this problem

could be solved. Therefore, at least some steps could

be automized in the future work.

At this moment, the segmentation process is cal-

culated over the whole image. To improve the pro-

cessing speed deﬁning a region of interest could be

implemented.

Another goal of our work is to create a statisti-

cal representation of the position of the deltoid mus-

cle. This means that for each point we would like to

calculate a probability that this point belongs to the

deltoid muscle. Segmented humerus, scapula, and

clavicle will be taken as the reference objects. Cre-

ating such an atlas could autonomously propose seed

points deﬁning the deltoid thus decreasing the needed

amount if interactivity.

The problem of this approach is the uniqueness

and difference between subjects - factors such as

height, weight, musculature, sex etc. plays a signif-

icant role regarding the position and shape of the cor-

responding deltoid muscle. Having a large number of

data containing all of these factors, it could be possi-

ble to create more statistical atlases and then use the

one that best ﬁts the given subject.

ACKNOWLEDGEMENTS

This research was supported by the project ”38 Vir-

tual human body model for prevention, therapy and

rehabilitation of shoulder diseases” realized within

the frame of the Program INTERREG V-A: Cross-

border Cooperation between the Czech Republic and

the Federal State of Germany Bavaria, Aim European

Cross-border Cooperation 2014 - 2020. The realiza-

tion is supported by ﬁnancial means of the European

Regional Development Fund and the state budget of

the Czech Republic.

REFERENCES

Antong, C., Deeley, M. A., Niermann, K. J., Moretti, L.,

Dawant, B. M., Department of Radiation Oncology,

Vanderbilt-Ingram Cancer Center, . n. A. S. N. T. .,

of Electrical Engineering, D., and Computer Science,

Vanderbilt University, N.-T. . (2010). Combining reg-

istration and active shape models for the automatic

segmentation of the lymph node regions in head and

neck CT images. Medical Physics, 37(12).

Boykov, Y. and Jolly, M.-P. (2000). Interactive organ seg-

mentation using graph cuts. In International con-

ference on medical image computing and computer-

assisted intervention, pages 276–286. Springer.

Dobrin, B. P., Viero, T. J., and Gabbouj, M. (1994). Fast wa-

tershed algorithms: analysis and extensions. In Non-

linear Image Processing V, volume 2180, pages 209–

221. International Society for Optics and Photonics.

Engwall, O. and Badin, P. (1999). Collecting and analysing

two- and three-dimensional MRI data for swedish.

Quarterly Progress and Status Report - Royal Insti-

tute of Technology, Department of Speech, Music and

Hearing, 3.

Ghosal, S. and Ray, N. (2017). Deep deformable registra-

tion: Enhancing accuracy by fully convolutional neu-

ral net. Pattern Recognition Letters, 94:81 – 86.

Grady, L. (2006). Random walks for image segmentation.

IEEE transactions on pattern analysis and machine

intelligence, 28(11):1768–1783.

Hajnal, J. and Hill, D. (2001). Medical Image Registration.

Biomedical Engineering. Taylor & Francis.

Havelkov

a, L.,

Spi

cka, J., and Hyn

ık, L. (2017). The new

torus obstacle method used for musculoskeletal mod-

eling. In 23st CONGRESS OF EUROPEAN SOCIETY

OF BIOMECHANICS.

He, B., Huang, C., Zhou, S., Hu, Q., and Jia, F. (2016). Fast

automatic 3D liver segmentation based on a three-

Segmentation of Shoulder MRI Data for Musculoskeletal Model Adaptation

159

level adaboost-guided active shape model. Medical

Physics, 43(5).

Ino, F., Ooyama, K., and Hagihara, K. (2005). A data dis-

tributed parallel algorithm for nonrigid image regis-

tration. Parallel Computing, 31:19–43.

Ito, M. and Ino, F. (2018). An automated method for

generating training sets for deep learning based im-

age registration. In Proceedings of the 11th Inter-

national Joint Conference on Biomedical Engineering

Systems and Technologies (BIOSTEC 2018) - Volume

2: BIOIMAGING, Funchal, Madeira, Portugal, Jan-

uary 19-21, 2018., pages 140–147.

ık, M., Ryba, T., Svobodov

a, M., M

ırka, H., and

ska, V. (2014). LISA - liver surgery ana-

lyzer software development. In Proceedings of the

11th World Congress on Computational Mechanics

(WCCM 2014), Barcelona.

Keustermans, J., Vandermeulen, D., and Suetens, P. (2012).

Integrating statistical shape models into a graph cut

framework for tooth segmentation. In Wang, F., Shen,

D., Yan, P., and Suzuki, K., editors, Machine Learn-

ing in Medical Imaging, pages 242–249, Berlin, Hei-

delberg. Springer Berlin Heidelberg.

Kodym, O. and

Span

el, M. (2018). Semi-automatic CT im-

age segmentation using random forests learned from

partial annotations. In Proceedings of the 11th In-

ternational Joint Conference on Biomedical Engineer-

ing Systems and Technologies - Volume 2: BIOIMAG-

ING,, pages 124–131. INSTICC, SciTePress.

cah, M., Sz

ekely, G., and Blanc, R. (2011). Fully auto-

matic and fast segmentation of the femur bone from

3D-CT images with no shape prior. In 2011 IEEE In-

ternational Symposium on Biomedical Imaging: From

Nano to Macro, pages 2087–2090.

Liu, F., Zhou, Z., Jang, H., Samsonov, A., Zhao, G., and

Kijowski, R. (2018). Deep convolutional neural net-

work and 3d deformable approach for tissue segmen-

tation in musculoskeletal magnetic resonance imag-

ing. Magnetic resonance in medicine, 79(4):2379–

2391.

Loh, W.-Y. (2011). Classiﬁcation and regression trees.

Wiley Interdisciplinary Reviews: Data Mining and

Knowledge Discovery, 1(1):14–23.

Milletari, F., Navab, N., and Ahmadi, S.-A. (2016). V-

net: Fully convolutional neural networks for volumet-

ric medical image segmentation. 2016 Fourth Inter-

national Conference on 3D Vision (3DV), pages 565–

571.

Ojalammi, A. and Malinen, J. (2017). Automated segmen-

tation of upper airways from MRI: Vocal tract geom-

etry extraction. In Proceedings of the 10th Interna-

tional Joint Conference on Biomedical Engineering

Systems and Technologies, volume 2, pages 77–84.

SciTePress.

Pinheiro, M. and Alves, J. L. (2015). A new level-set-

based protocol for accurate bone segmentation from

CT imaging. IEEE Access, 3:1894–1906.

Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., and

Nielsen, M. (2013). Deep feature learning for knee

cartilage segmentation using a triplanar convolutional

neural network. In International conference on medi-

cal image computing and computer-assisted interven-

tion, pages 246–253. Springer.

Rathnayaka, K., Sahama, T., Schuetz, M. A., and Schmutz,

B. (2011). Effects of CT image segmentation meth-

ods on the accuracy of long bone 3D reconstructions.

Medical engineering & physics, 33(2):226–233.

Rohlﬁng, T. and Maurer, C. R. (2003). Nonrigid im-

age registration in shared-memory multiprocessor en-

vironments with application to brains, breasts, and

bees. IEEE Transactions on Information Technology

in Biomedicine, 7(1):16–25.

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net:

Convolutional networks for biomedical image seg-

mentation. In International Conference on Medical

image computing and computer-assisted intervention,

pages 234–241. Springer.

Tetsworth, K. D., Block, S., and Glatt, V. (2017). Putting

3D modelling and 3D printing into practice: virtual

surgery and preoperative planning to reconstruct com-

plex post-traumatic skeletal deformities and defects.

In SICOT-J.

Tomasi, C. and Manduchi, R. (1998). Bilateral ﬁltering for

gray and color images. In Computer Vision, 1998.

Sixth International Conference on, pages 839–846.

IEEE.

Virz

ı, A., Marret, J. ., Muller, C. O., Berteloot, L., Boddaert,

N., Sarnacki, S., and Bloch, I. (2017). A new method

based on template registration and deformable models

for pelvic bones semi-automatic segmentation in pe-

diatric MRI. In 2017 IEEE 14th International Sympo-

sium on Biomedical Imaging (ISBI 2017), pages 323–

326.

Xi, T., Schreurs, R., Heerink, W. J., Berg

e, S. J.,

and Maal, T. J. J. (2014). A novel region-

growing based semi-automatic segmentation protocol

for three-dimensional condylar reconstruction using

cone beam computed tomography (CBCT). In PloS

one.

Xue, Y., Xu, T., Zhang, H., Long, L. R., and Huang, X.

(2018). Segan: Adversarial network with multi-scale

l 1 loss for medical image segmentation. Neuroinfor-

matics, pages 1–10.

Yokota, F., Okada, T., Takao, M., Sugano, N., Tada, Y.,

Tomiyama, N., and Sato, Y. (2013). Automated CT

segmentation of diseased hip using hierarchical and

conditional statistical shape models. In Mori, K.,

Sakuma, I., Sato, Y., Barillot, C., and Navab, N.,

editors, Medical Image Computing and Computer-

Assisted Intervention – MICCAI 2013, pages 190–

197, Berlin, Heidelberg. Springer Berlin Heidelberg.

BIOIMAGING 2019 - 6th International Conference on Bioimaging

160