![](bg7.png)
Table 2: Four patient examples whose reports are generated by pair2txt two attention based on the input of keyword set and
historical report. The first example illustrates the association between the current report with the keyword set, historical report
and generated report. The second and third examples have the same keyword set, but different historical reports. The fourth
example shows the trend of nodules over time.
Historical report Keyword set Generated report Ground-truth report
The left and right lobes of the thyroid gland are
normal in shape, the thickness of the isthmus is
normal, the border is clear, the surface is smooth,
the envelope is complete, and the internal echo
is a dense medium echo, and the echo distribu-
tion is heterogenous. CDFI: No obvious abnor-
mal blood flow signal. Several hypoechogenicity
can be seen in the left thyroid gland, the shape
is elliptical, the internal echo is heterogenous,
the border is still clear, and there is no obvious
point-like strong echo inside. CDFI: an abun-
dant blood flow signal is visible. . . . .
Envelope com-
plete, medium
echo, echo dis-
tribution even,
several hypoe-
chogenicity,
elliptical, echo
uneven
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface is
smooth, the envelope is complete, and the inside
echo is a dense medium echo, the echoes distribu-
tion is even. CDFI: No obvious abnormal blood
flow signal. Several hypoechogenicity in the left
thyroid gland, the shape is elliptical, the inter-
nal echo is uneven, the boundary is still clear,
and there is no obvious point-like strong echo
inside. CDFI: an abundant blood flow signal is
visible. . . . .
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface is
smooth, the envelope is complete, and the inside
echo is a dense medium echo, the echo distribu-
tion is even. CDFI: No obvious abnormal blood
flow signal. Several hypoechogenicity in the left
thyroid gland, the shape is elliptical, the inter-
nal echo is uneven, the boundary is still clear,
and there is no obvious point-like strong echo
inside. CDFI: an abundant blood flow signal is
visible. . . . .
Postoperation, right lobe: anteroposterior di-
ameter, left lobe (resected), isthmus: resected.
In the residual thyroid, the internal echo is a dense
medium echo, and echo distribution is even. . . . .
Envelope com-
plete, medium
echo, echo
distribution
even
Postoperation, left lobe (resected), the residual
right thyroid gland is normal in size and shape,
the boundary is clear, the surface is smooth, the
envelope is complete, and the inside echo is a
dense medium echo, and the echo distribution is
even. . . . .
In the residual right lobe of the thyroid gland
is normal in size and shape, the boundary is clear,
the surface is smooth, the envelope is complete,
and the inside echo is a dense medium echo, and
echo distribution is even. . . . .
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface
is smooth, the envelope is complete, and the in-
side echo is a dense medium echo, and the echo
distribution is even. . . .
Envelope com-
plete, medium
echo, echo
distribution
even
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface
is smooth, the envelope is complete, and the in-
side echo is a dense medium echo, and the echo
distribution is even. . . .
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface
is smooth, the envelope is complete, and the in-
side echo is a dense medium echo, and the echo
distribution is even. . . .
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface
is smooth, the envelope is complete, and the in-
side echo is a dense medium echo, and the echo
distribution is even. CDFI: No obvious abnormal
blood flow signal. A mixed echo in the left thy-
roid gland. The shape is elliptical, the internal
echo is uneven, the boundary is still clear, and
there is no obvious point-like strong echo inside.
. . . .
Envelope com-
plete, medium
echo, echo dis-
tribution even,
a liquid mixed
echo, elliptical,
echo distribu-
tion uneven
point-like strong
echo
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface
is smooth, the envelope is complete, and the in-
side echo is a dense medium echo, and the echo
distribution is even. CDFI: No obvious abnormal
blood flow signal. A liquid mainly mixed echo
in the left thyroid gland. The shape is ellipti-
cal, the internal echo is uneven, the boundary
is still clear, and there is a spot-like strong echo
(colloid agglutination). . . . .
The left and right lobes of the thyroid gland are
normal in size and shape, the thickness of the isth-
mus is normal, the boundary is clear, the surface is
smooth, the envelope is complete, and the inside
echo is a dense medium echo, and the echo distri-
bution is even. CDFI: No obvious abnormal blood
flow signal. A liquid mainly mixed echo in the
left thyroid. The shape is elliptical, the internal
echo is uneven, the boundary is still clear, and
there is a spot-like strong echo (colloid aggluti-
nation). . . . .
(a) Matrix Keywords-record1 (b) Matrix Report-record1
Figure 3: The attention weight matrix for the first example in Table 2. The row lists the generated report, the column of Figure
(a) is the keyword set, and the column of Figure (b) is the historical report.
the nodule, such as ”a liquid mixed echo”, but also
learn information from ”a mixed echo left thyroid
gland” in the historical report. Then the model can
infer that this is a description of the same nodule
at different times. The echo type of the nodule
changes over time, from ”mixed echo” to ”liquid
mainly mixed echo”. In the pair2text two attention
model, the progress of the disease is learned from the
historical report and the current report in the training
set.
4.2.3 Attention Weight Matrix
To further understand how the historical report and
keyword set to help generate the current report, we
provide the attention weight matrix when the patient
examples’ reports in Table 2 are generated. Since
there are two attention layers in the proposed method,
HEALTHINF 2021 - 14th International Conference on Health Informatics
172