A New Technique for Education Process Optimization via the Dual
Control Approach
Nicholas A. Nechval
1
, Gundars Berzins
1
, Vadims Danovics
1
and Konstantin N. Nechval
2
1
BVEF Research Institute, University of Latvia, Raina Blvd 19, Riga 1050, Latvia
2
Department of Aviation, Transport and Telecommunication Institute, Riga 1019, Latvia
Keywords: Education Process, Performance Index, Optimization, Dual Control, Dynamic Programming.
Abstract: In this paper, examples arising from a problem in education process control are considered. The provision of
the number of information items in a specified discipline for education to optimize a suitable performance
index is treated as a “dual control” process. According to the theory of dual control, the control signal has
two purposes which might be in conflict with each other: 1) to help learn about any unknown parameters
and/or state of the system (estimation); 2) to contro1. In view of this, one can see that the open-loop
feedback control is, from the estimation point of view, passive, since it does not take into account that
learning is possible in the future. In contrast to this, a dual control is active, not only for the control purpose
but also for the estimation purpose because the performance depends also on the “quality” of the estimates.
Therefore, the dua1 control can be called actively adaptive since it regulates the speed and amount of
learning as required by the performance index. To determine the sequence of optimal decisions, dynamic
programming is used. It will be noted that the optimal policy can be compared with the non-optimal policy
of optimizing stage by stage. To illustrate a new technique for education process optimization via the dual
control approach, numerical examples are given.
1 INTRODUCTION
In all control problems there are certain degrees of
uncertainty with respect to the process to be
controlled. The structure of the process and/or the
parameters of the process may vary in an unknown
way. To obtain good process information it is
necessary to perturb the process. Normally, the
information about the process will increase with the
level of perturbation. On the other hand the
specifications of the closed loop system are such that
the output normally should vary as little as possible.
There is thus a conflict between information
gathering and control quality. This problem was
introduced and discussed by A. A. Feldbaum in a
sequence of four seminal papers from 1960 and
1961, see (Feldbaum, 1960-61). Feldbaum’s main
idea is that in controlling the unknown process it is
necessary that the controller has dual goals. First the
controller must control the process as well as
possible. Second, the controller must inject a
probing signal or perturbation to get more
information about the process. By gaining more
process information better control can be achieved in
future time. The compromise between probing and
control or in Feldbaum’s terminology investigating
and directing leads to the concept of dual control.
Feldbaum showed that a functional equation gives
the solution to the dual control problem. The
derivation is based on dynamic programming and
the resulting functional equation is often called the
Bellman equation. Alper and Smith (1967) use
Feldbaum’s idea to provide places in a sector of
education to satisfy an unknown social demand.
This paper is an endeavour to clarify some
concepts of underlying decision processes relating to
education process control by utilizing the theory of
automatic control. The purpose of this paper then is
to indicate how some of the concepts of control
theory may be utilized in order to bring about better
performance of the education process. One of the
most important features of control theory is its great
generality, enabling one to analyze diverse systems
within one unified framework.
In order to illustrate the Feldbaum’s main idea, a
simple example relevant to education process
control is given below. The example has two
outstanding virtues: the performance index is not the
Nechval, N., Berzins, G., Danovics, V. and Nechval, K.
A New Technique for Education Process Optimization via the Dual Control Approach.
DOI: 10.5220/0006799304490456
In Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), pages 449-456
ISBN: 978-989-758-291-2
Copyright
c
2019 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
449
usual quadratic one, and analytic solutions without
approximations are possible, thus affording the
opportunity of comparison with reasonable but not
optimal policies.
2 PROBLEM STATEMENT
Let us assume that one has to decide on the number
of information items to be made available in a
specified discipline (course of lectures). Let the
number of information items (discipline information
quantity) made available be designated by u.
However, the number of information items, x,
successfully acquired by students, may be less than
u, because the above number is limited to an
unknown level H (as a function of the number of
education hours and a fundamental knowledge level
of students).
Thus, one may write
X = min (u, H). (1)
The outcome of the decision is assessed by
considering that the cost of providing an information
item is the same whether it is understood or not and
that the benefit derived from a student is to be given
by a factor q times as great. Therefore, the total
"education effect" may be written as
V =q X u = q min (u, H) u. (2)
Clearly q > 1, otherwise one should not provide any
information items at all. If H were known, V is
maximized by choosing u=H.
It is further assumed that
h
1
H h
2
(3)
and the decision maker's a priori knowledge is
expressed by means of a probability density f(h).
The question is then: what value should be chosen
for u so as to maximize the expected education
effect averaged over the a priori distribution of H?
There are two possibilities:
H < u, with probability
1
()
u
h
f
hdh
and
H u, with probability
2
() .
h
u
f
hdh
(4)
In the first case, X = H, with an expected H given by
11
1
{} () () .
uu
hh
EH hf hdh f hdh
(5)
In the second case, X = u. The expected education
effect is thus
{} { | }Pr( ) { | }Pr( )EV EVhu hu EVhu hu
2
1
()()(1)().
h
u
hu
qh u f hdh q u f hdh
(6)
Therefore, the optimum value of u, designated u
is
given by
2
1/ ( )
h
u
qfhdh
(7)
and the corresponding extremized effect is
1
{} ().
u
h
EV q hf hdh
(8)
3 MAXIMIZATION OF THE
EXPECTED EDUCATION
EFFECT OVER N STAGES
The education effect of any particular stage (say, jth)
of the education process may be written as
V
j
=q min (u
j
, H) u
j
. (9)
Thus, the problem is to maximize the expected
education effect over N stages,
11
[min( , ) ]
NN
Nj jj
jj
GE V E q uHu
2
1
`1
() ()
j
j
u
h
N
j
j
j
hu
qhfhdh ufhdhu
2
1
(1) ()
h
h
Nq hf hdh
2
1
1
()()(1)()().
j
j
u
h
N
jj
j
hu
uhfhdhq hufhdh
(10)
Now, consider a sequence of decisions maximizing
(10). Let
1
()
N
Gh
be the optimum total future
education effect when there are N stages to go, and
note that it is expressed as a function of the lower
limit on H. The expected cost at the first of N stages
is given by (6).
If H < u, the expected effect of the succeeding
(N
1) stages will be given by
CSEDU 2018 - 10th International Conference on Computer Supported Education
450
11
1
( 1)( 1) () () .
uu
hh
N q hf hdh f hdh
(11)
If H
u, then H is not known precisely, but the
lower limit will have been raised from h
1
to u. The
optimum expected education effect of the
succeeding N stages can thus be denoted by
()
N
Gu
.
Consequently, by analogy with the single-stage
optimization
1
1
() max[ ( 1)( 1)] ()
N
N
u
N
u
h
Gh q N q hfhdh
2
1
1
() [( 1) ()] () .
N
N
u
h
NNNN
hu
u f hdh u q G u f hdh
(12)
The results of the optimal policy can be compared
with the outcome of repeated applications of the
single-stage policy. The expected education effect
over N+1 stages, Q
N+1
, may then be interpreted in
this instance as the ''non-dual" solution of the
problem and may be derived as
1
11
1
() [ ( 1)( 1)] () ( ),
u
NN
h
Qh q N q hfhdh Q u
q
(13)
where u
is given by (7).
3.1 Procedure for the Optimal Policy
The procedure for the optimal policy is then, for the
appropriate f(h), as follows:
1) Determine
N
u
using (12) and the current
values of h
1
and h
2
.
2) Either H
<
N
u
in which case u
= H for the
remaining N
1stages or H
N
u
in which case the
above step is repeated again with N
reduced by one
and h
1
replaced by
N
u
.
3.2 Modification of the Optimization
Equations
If H were known to have its expected value, then it
is convenient to express (12) as follows:
2
1
11
() () ( 1) () .
h
NN
h
Gh Rh Nq hfhdh
(14)
Then (12) may be rewritten as
1
11
() max ( ) ( )()
N
N
u
NNNN
u
h
Rh R u u hfhdh
2
(1)( )() .
N
h
N
u
qhufhdh
(15)
As for
1
()
N
Gh
above, we can write
2
1
11
() () ( 1) () ,
h
NN
h
Qh Sh Nq hfhdh
(16)
so that (13) can be rewritten as
22
1
11
( ) ( ) () () .
hh
NN
h
u
S h S u q hf h dh hf h dh
(17)
3.3 Illustrative Example 1
(Rectangular probability density function of a priori
knowledge). If only an upper and lower bound on H
is known, then a priori knowledge about H is
expressed as
12
21
1
(; , ) ,fhhh
hh
h
1
h h
2
. (18)
It follows from the above that in this case we
have:
121
( 1)( ),
NN
uhaq hh
(19)
where
1
0
1
1
, 0;
(1)
N
N
N
a
aa
qa q
(20)
11221
1
( ) [ ( ) ( )];
2
NN
q
Gh Nh h ah h
(21)
11221
(1)
() [ ( )],
2
NN
Nq
Qh h h bh h
(22)
where
2
21 2
11
.
1
N
N
N
q
b
Nq q
(23)
The optimum policy when the number of stages to
go is very large is, surprisingly enough, given by
121
3
lim ( 1) 1 ( )
1
N
N
q
uuhq hh
q
(24)
With regard to a two-stage problem when for
example, h
1
=10, q=1.2, h
2
=40, then
2
G
= 5.976 and
Q
2
= 5.764, an improvement of 3.6 percent for the
optimum policy as compared to merely treating the
problem as a non-dual, repeated single-stage
problem.
A New Technique for Education Process Optimization via the Dual Control Approach
451
3.4 Illustrative Example 2
(Exponential probability density function of a priori
knowledge). It is assumed that a priori knowledge is
such that H is given as
1
11
1
(| ,) exp , .
hh
fhh h h
(25)
In this situation,
1
,
NN
uh
(26)
where
1
ln( ),
NN
q
0
= 0; (27)
11
() ( 1)( ) ;
NN
Gh Nq h
(28)
11
1
1
() ( 1)( ) ln.
(1)
N
N
N
q
Qh Nq h q
qq
(29)
Then
1
,uh
(30)
where
ln( ).vqv (31)
The comparison for the dual solution versus the non-
dual solution for N=2, h
1
=0, q = e,
= l shows that
2
G
= 2.123 and Q
2
= 2.068, an improvement of 2.56
percent.
3.5 Illustrative Example 3
(Polynomial probability density function of a priori
knowledge). It is assumed that a priori knowledge is
such that H is given as
1
11 1
(| , ) ( 1) , 2, .
mm
fhhm m h h m h h
(32)
In this case,
1
,
NN
uh
(33)
where
1/( 1)
1
(1) ,
m
NN
qm
0
=0; (34)
11
1
() ( 1)( 1) ;
2
NN
m
Gh h q N q
m
(35)
1/( 1)
11
1
() ( 1)( 1)
2
m
NN
m
Qh h q N q q r
m
(36)
where
1/( 1)
1
1
1,
m
NN
rqqr
q
r
1
=1. (37)
The optimum policy for the infinite-stage case is to
set
1
,uhw
(38)
where w is greater than one and a root of
1
(1)(1).
m
wqmw
(39)
For a two-stage process and q=1.2, h
1
=10, m=3, we
have that
2
G
=4.42 and Q
2
=4.36, an improvement of
1.35 percent for the dual solution over the non-dual.
4 PREDICTIVE INFERENCES
FOR EDUCATION PROCESS
CONTROL
Various solutions have been proposed for the
prediction problems, that is, the problems of making
inferences on a random sample {Y
i
; i=1, …, m}
given independent observations {X
j
; j=1, …, n}
drawn from the same distribution. The Y
i
’s and the
X
j
’s are commonly featured as “future outcomes”
and “past outcomes” respectively. Inferences usually
bear on some reduction Z of the Y
i
’s possibly a
minimal sufficient statistic and consist of either
frequentist prediction intervals or likelihood or
predictive distribution of Z, depending on different
authors. The following result is not new, but the
example illustrates the procedure.
4.1 Illustrative Example 1
(Predictive exponential distribution). Suppose X
1
,
…, X
n
, Y are independent random variables each
having density function
( | ) (1 / ) exp( / ), 0, 0.fs s s
(40)
The joint density is given by
1
1
1
1
( , ..., , | ) exp
n
i
i
n
n
x
y
fx x y
(41)
and the factorization theorem gives
1
n
i
i
TXY
(42)
sufficient for
. The conditional density of Y given T
= t is
1
1 , for 0 ,
(|)
0, otherwise.
n
ny
yt
gyt
tt
(43)
CSEDU 2018 - 10th International Conference on Computer Supported Education
452
This is obtained by finding the joint distribution of
i
X
and Y and transforming to
i
txy
and y.
Then the two-sided predictive interval for Y will be
(a, b) where a and b satisfy
(|)
b
a
gytdy
(44)
or
11 .
nn
ab
tt
(45)
For example, if we take an interval of the form (0,
b), which will be the interval of shortest width, we
have
1/
(0, ) {0, [1 (1 ) ]}.
n
bt
(46)
Using the fact that
i
txy
,
1/
[1 (1 ) ]
n
yt
(47)
implies
1/
1
[(1 ) 1] .
n
n
i
i
y
x
(48)
Thus,
1/
1
(0, ) 0, [(1 ) 1] .
n
n
i
i
bX
(49)
This is, of course, the same result given by the
standard approach of using the fact that the pivotal
statistic
/
i
nY X
is distributed F(2, 2n).
5 SAMPLING DISTRIBUTIONS
FOR EDUCATION PROCESS
CONTROL
The sampling truncated distributions have found
many applications, including education process
control. It is known that a sampling distribution for
truncated law may be derived using, namely, the
method based on characteristic functions (Bain and
Weeks, 1964), the method based on generating
functions (Charalambides, 1974), or the
combinatorial method (Cacoullos, 1961). In this
section, a much simpler technique than the above
ones is proposed. It allows one to obtain the results
for truncated laws more easily (Nechval et al., 2002,
2008).
Suppose an experiment yields data sample X
n
=
(X
1
, … , X
n
) relevant to the value of a parameter
(in general, vector). Let L
X
(x
n
|) denote the
probability or probability density of X
n
when the
parameter assumes the value . Considered as a
function of for given X
n
=x
n
, L
X
(x
n
|) is the
likelihood function. If the data sample X
n
can be
summarized by a sufficient statistic
S, one can write
L
S
(s|) L
X
(x
n
|). Further, for any non-negative
function
(s),
(s)L
S
(s|) is also a likelihood
function equivalent to L
X
(x
n
|). Suppose we
recognize a function
(s) such that
(s)L
S
(s|),
regarded as a function of
s for a given , is a density
function. It can be shown that this is the sampling
density of
S if the family of recognized densities is
complete.
The technique for finding sampling distributions
of truncated laws is based on the use of the
unbiasedness equivalence principle (UEP) and
consists in the following. If
L
X
(x
n
|,
)=[w(,
,
)]
n
L
X
(x
n
|), (50)
represents the likelihood function for the truncated
law, where w(,
) is some function of a parameter
(,
) associated with truncation,
is a known
truncation point (in general, vector), then a sampling
density for the truncated law is determined by
() ()() (),
n
gwwg
s
|
ss
|
sS
, (51)
where
() ( ) ( )
n
ww g
ss
|
=
(s)L
S
(s|,
) L
X
(x
n
|,
), (52)
g(
s|) is a sampling density of a sufficient statistic
s(X
n
) (for a family of densities {f(x|)}) determined
on the basis of L
X
(X
n
|), ()w S
is an unbiased
estimator of 1/[w(,
)]
n
with respect to g(s|), sS
(a sample space of a non-truncated sufficient statistic
S),
(S) is a function of S for a given , which is
equivalent to unbiased estimator
)(Sw
of
1/[w(,
)]
n
, i.e.,
(S) ()w S
(53)
or
(S) =
() (, ) ( )/ ( , )
n
ww g L
S
SS|S|
, (54)
g
(s|) is the sampling density of a sufficient
statistic
S (for a family of densities {f
(x|)}) when
the truncation parameter
is known, S
is a sample
space of a truncated sufficient statistic
S.
To illustrate the above technique, we present the
following illustrative examples.
A New Technique for Education Process Optimization via the Dual Control Approach
453
5.1 Illustrative Example 1
(Sampling distribution for the left-truncated Poisson
law). Let the Poisson probability function be
denoted by
( | ) , 0, 1, 2, ... .
!
x
fx e x
x
(55)
The probability function of the restricted random
variable, which is truncated away from some
0, is
then
( | ) ( , ) ( | ), 1, 2, ... ,fx w fx x
(56)
where
11
10
(, ) 1 .
! !
jj
jj
we e
jj
(57)
Consider a sample of n independent observations X
1
,
X
2
, …, X
n
, each with probability function f
(x|
),
where the likelihood function is defined as
1
(|,) (|)
n
n
Xi
i
Lx fx
1
(, ) ( | ) (, ) ( | )
n
nn
n
Xi
i
wLxw fx
1
1
(, ) ,
!
n
i
i
x
n
n
n
i
i
we
x
(58)
and let
1
, ( 1), ( 1) 1, ... .
n
i
i
SXsn n
(59)
It is well known that
1
, 0, 1, ...
n
i
i
SXs
. (60)
is a complete sufficient statistic for the family
{f(x|
)}. A result of Tukey (1949) states that
sufficiency is preserved under truncation away from
any Borel set in the range of X.
For the sake of simplicity but without loss of
generality, consider the case
=0. It follows from
(51) that
(| ) () (,) (| )
n
gs wsw gs
!
, , 1, ... ,
(1) !
s
n
s
n
n
Csnn
es
(61)
where
()
( | ) , 0, 1, ... ,
!
s
n
n
gs e s
s
(62)
1
[w( , )] ,
(1 )
n
n
e
(63)
!
() ,
n
s
s
n
ws C
n
(64)
n
s
C
denotes the Stirling number of the second kind
(Jordan, 1950) defined by
0
1
( 1) , , 1, ... ,
!
0, ,
n
nj s
n
j
s
n
jsnn
C
j
n
sn
(65)
0
() () ( | )
s
Ews wsgs
0
! ( )
!
s
nn
s
s
s
nn
Ce
s
n
()
00
()
(1) (1 ),
!
s
n
nj nj j n
js
n
j
eee
j
s
(66)
This is the same result that of Tate and Goen (1958).
Their proof was based on characteristic functions.
5.2 Illustrative Example 2
(Sampling distribution for the right-truncated
exponential law).
Let the probability density
function of the right-truncated exponential
distribution be denoted by
( | ) ( , ) ( | ), 0 ,fx w fx x
(67)
where
/
1
(, ) ,
1
w
e
(68)
/
( | ) (1 / ) , [0, ).
x
fx e x
(69)
Consider a sample of n independent observations X
1
,
X
2
, …, X
n
, each with density f
(x|
), where the
likelihood function is determined as
CSEDU 2018 - 10th International Conference on Computer Supported Education
454
1
(|,) (|) (,) (|)
n
n
nn
XiX
i
Lx fx w Lx
1
/
1
1
(, ) ( | ) (, ) .
n
i
i
n
x
nn
i
n
i
wfxwe
(70)
It is well known that
1
, [0, ),
n
i
i
SX s
(71)
is a complete sufficient statistic for the family
{f(x|
)}. It follows from (51) that
(| ) () (,) (| )
n
gs wsw gs
/
1
/
0
(1) [( ( ))] ,
() (1 )
s
n
nj n
nn
j
n
e
snj
j
ne
[0, ],
s
n
n 1, (72)
where a
+
=max(0,a),
1
/
n
( | ) , [0, ) ,
()
n
s
s
gs e s
n
(73)
/
1
[ ( , )] ,
(1 )
n
n
w
e
(74)
1
1
0
1
(s) ( 1) [( ( ) ) ] ,
n
nj n
n
j
n
wsnj
j
s
(75)
0
() () ( | )Ews wsgs ds
1
()/
0
0
[( ( ) ) ]
(1)
()
n
n
nj nj
n
j
n
snj
e
j
n
(( ))/
(())
snj
edsnj
.)1(
/ n
e
(76)
This is the same result that of Bain and Weeks
(1964). Their proof was based on characteristic
functions.
5.3 Illustrative Example 3
(Sampling distribution for the doubly truncated
exponential law).
Consider an exponential
distribution (69) that is doubly truncated at a lower
truncation point (
1
) and an upper truncation point
(
2
). The probability density function of the doubly
truncated exponential distribution is defined as
12
( | ) ( , ) ( | ), ,fx w fx x
(77)
where =(
1
,
2
),
12
//
1
(, ) .w
ee
(78)
Consider a sample of n independent observations X
1
,
X
2
, …, X
n
, each with density f
(x|
), where the
likelihood function is determined as
1
(|,) (|) (,) (|)
n
n
nn
XiX
i
Lx fx w Lx
1
/
1
1
(, ) ( | ) (, ) .
n
i
i
n
x
nn
i
n
i
wfxwe
(79)
It is well known that
1
, [0, ),
n
i
i
SX s
(80)
is a complete sufficient statistic for the family
{f(x|
)}. It follows from (51) that
(| ) () (,) (| )
n
gs wsw gs
12
/
//
() ( )
s
nn
e
ne e
1
121
1
(1) [( ( )( )] ,
n
nj n
j
n
sn n j
j
12
[, ],snn
n 1, (81)
where a
+
=max(0,a), g(s|
) is given by (73),
12
//
1
[(,)] ,
()
n
n
w
ee
(82)
A New Technique for Education Process Optimization via the Dual Control Approach
455
1
0
1
(s) ( 1)
n
nj
n
j
n
w
j
s
1
121
[( ( )( )) ] ,
n
sn n j
(83)
0
() () ( | )Ews wsgs ds
12
//
().
n
ee
(84)
6 CONCLUSIONS
This work has presented the new technique for
education process optimization via the dual control
approach. The main feature of this technique is the
fact that it is actively adaptive, i.e., the control plans
the future learning of the system parameters as
needed by the overall performance. This contro1 is
obtained by using the dynamic programming
equation in which the dual effect of the control
appears explicitly. The technique yields a closed-
loop control that takes into account not only the past
observations but also the future observation program
and the associated statistics. A detailed description
of the technique is given and illustrative examples
are presented.
Although the examples discussed in this paper
are highly simplified and have orders of magnitude
simpler than the complex situation faced by the
education decision-maker, it does indicate the way
to some very interesting points. The optimum
procedure is to consider the situation as a dual
control problem where information and action are
interrelated.
The authors hope that this work will stimulate
further investigation using the approach on specific
applications to see whether obtained results with it
are feasible for realistic applications.
ACKNOWLEDGEMENTS
The authors wish to acknowledge partial support of
this research via Grant No. 06.1936 and Grant No.
07.2036.
REFERENCES
Alper, P., Smith, C., 1967. An application of control
theory to a problem in educational planning. IEEE
Transactions on Automatic Control, 12, 176-178.
Bain, L.J., Weeks, D.L., 1964. A Note on the truncated
exponential distribution. Ann. Math. Statist. 35, 1366-
1367.
Bain, L.J., Weeks, D.L., 1964. A Note on the truncated
exponential distribution. Ann. Math. Statist. 35, 1366-
1367.
Cacoullos, T.A., 1961. Combinatorial derivation of the
distribution of the truncated Poisson sufficient
statistic. Ann. Math. Statist. 32, 904-905.
Charalambides, C.A., 1974. Minimum variance unbiased
estimation for a class of left-truncated discrete
distributions. Sankhyā 36, 397-418.
Feldbaum, A.A., 1960-61. Dual control theory, I–IV.
Automation Remote Control, 21, 22, pp. 874-880,
1033-1039, 1-12, 109-121.
Feldbaum, A.A., 1965. Optimal Control Systems,
Academic Press. New York.
Jordan, C., 1950. Calculus of Finite Differences, Chelsea.
New York.
Nechval, N.A., Nechval, K.N., Vasermanis, E.K., 2002.
Finding sampling distributions and reliability
estimation for truncated laws. In Proceedings of the
30
th
International Conference on Computers and
Industrial Engineering. Tinos Island, Greece, June 29
– July 1, Vol. II, pp. 653-658.
Nechval, N.A., Nechval, K.N., Berzins, G., Purgailis, M.,
2008. A new approach to finding sampling
distributions for truncated laws. Journal of the
Egyptian Mathematical Society 16, 93-105.
Tate, R.F., Goen, R.L., 1958. Minimum variance unbiased
estimation for the truncated Poisson distribution. Ann.
Math. Statist. 29, 755-765.
Tukey, J.W., 1949. Sufficiency, truncation and selection.
Ann. Math. Statist. 20, 309-311.
CSEDU 2018 - 10th International Conference on Computer Supported Education
456
