ADAPTIVE CONTROL BY NEURO-FUZZY SYSTEM OF AN

OMNI-DIRECTIONAL WHEELCHAIR USING A TOUCH PANEL AS

HUMAN-FRIENDLY INTERFACE

Kazuhiko Terashima, Yoshiyuki Noda, Juan Urbano, Sou Kitamura and Takanori Miyoshi

Department of Production Systems Engineering, Toyohashi University of Technology

Hibarigaoka 1-1, Toyohashi, 441-8580, Japan

Hideo Kitagawa

Department of Electronic Control Engineering, Gifu National College of Technology

Kamimakuwa, Motosu, Gifu, 501-0495, Japan

Keywords:

Omni-directional wheelchair, power assistant, operability, human-machine interface.

Abstract:

For improving the operability of an omni-directional wheelchair provided with a power assist system, the

system must be able to adapt to the individual characteristics of the many different attendants that will use it.

For achieving this purpose, an innovative human-interface using a touch panel that provides easy input and

feedback information in real time of the operation of a power-assisted wheelchair was developed. The system

was tested experimentally with many different attendants and the results show that in addition to providing

a human friendly interface by using the touch panel system with monitor it can adapt successfully to the

particular habits of the attendants.

1 INTRODUCTION

In order to satisfy the demand for higher mobility,

designers have created new driving concepts such as

omni-directional movement which allows any com-

bination of forward, sideways, and rotational move-

ment, thus ensuring users much more freedom and

safety in wide or narrow spaces. A variety of

wheelchairs with different options and special add-on

features have been developed to meet a wide range

of needs (Wada and Asada, 1999)-(West and Asada,

1992).

In the author’s laboratory, a holonomic Omni-

directional Wheelchair (OMW) which can act as

an autonomous (Kitagawa et al., 2002) or semi-

autonomous (Kitagawa et al., 2001) omni-directional

wheelchair has been developed. Comfort has been a

subject of study in the case with and without the joy-

stick (Kitagawa et al., 2002), (Terashima et al., 2004).

For handicapped people or elderly people that

can use their arms freely, many power assisted

wheelchairs have been developed such as Seki (Seki

et al., 2005) and Frank Mobility Systems (FrankMo-

bilitySystems, 2002), for example. However, it is nec-

essary to consider that some elderly people or hand-

icapped people can not use their arms because they

are damaged or they are so weak. These people

needs the help of an attendant. Considering this back-

ground, a power assist system that helps attendants

to move a heavy load has been designed and devel-

oped in author’s laboratory (Kitagawa et al., 2004).

Application of power assist for supporting the atten-

dant of an omni-directional wheelchair is one of a

novel research. To the authors knowledge, no other

report about this topic has appeared yet. However,

there is some research about power system for omni-

directional vehicles, but it is related to carts (Maeda

et al., 2000), not to wheelchairs. Moreover, it still has

some problems in rotation and in occupant’s comfort

since this system was developed for a food tray carry

vehicle in a hospital.

However, there is a problem related to the operabil-

ity of the OMW. Due to the application of the power

assist system, operability of the OMW degrades, es-

pecially when the attendant tries to rotate in clockwise

(CW), or counter-clockwise (CCW) direction around

the center of gravity (CG) of the OMW. This problem

is generated from the fact that it is difﬁcult to give the

human force exactly towards the target direction by

means of the handle attached to the wheelchair, hence

the movement of the OMW using conventional power

assist does not provide to the target exactly. Further-

Terashima K., Noda Y., Urbano J., Kitamura S., Miyoshi T. and Kitagawa H. (2007).

ADAPTIVE CONTROL BY NEURO-FUZZY SYSTEM OF AN OMNI-DIRECTIONAL WHEELCHAIR USING A TOUCH PANEL AS HUMAN-FRIENDLY

INTERFACE.

In Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics, pages 25-32

DOI: 10.5220/0001648700250032

 SciTePress

Figure 1: Omni-directional wheelchair (OMW).

more, the sensor position to measure the force added

by human for power assist is different from the posi-

tion of the gravity center of the OMW, and therefore

the force generated by its difference must be compen-

sated.

It was impossible to ﬁnd general rules to solve the

both problems stated in the above, but the relation-

ship was found by authors between lateral and ro-

tational movements. These relationships were used

as the base for constructing a fuzzy reasoning system

that helped to improve the operability of the OMW.

Nevertheless, when the system was tested by dif-

ferent attendants, it was found that a complete satis-

factory result was not obtained by every attendant. It

is because each person has its own tendencies and the

fuzzy inference system must be tuned to respond to

them. Tuning of the fuzzy inference system by trial

and error thus has been tried by authors’ group. How-

ever it is a time consuming and needs a lot of trials of

the attendants, then these can become tired and bored.

Thus, a better tuning method, a method that allows

tuning of the fuzzy inference system, is needed. It

can be obtained by adding Neural Networks (NN) to

the fuzzy inference system, obtaining what is known

as a neuro-fuzzy system. There is a lot of research

in this topic (Jang, 1993)-(Lin and Lee, 1991), being

the basic difference the kind of NN that is used in

combination with the fuzzy inference system.

Jang (Jang, 1993) developed ANFIS: Adaptive-

Network-based Fuzzy Inference Systems, a neuro-

fuzzy system in which the fuzzy inference system is

tuned by using the input data of the system.

The desired direction of motion of the attendant as

the teaching reference for the learning could be input

by just using the keyboard of the computer. However

keyboard input is not user-freindly. Furthermore, this

method does not provide feedback information to the

attendant in order to know how well he is accomplish-

ing the desired motion. Then, a human interface that

provides information to the attendant is desired. This

can be achieved by using a touch panel system with

monitor, which is a device that can be used as an input

and at the same time can show the resultant motion of

the OMW. The desired motion and the real motion

of the OMW are compared in order to obtain the dif-

ference, or error, that will be used for the training of

ANFIS, as explained in a previous paper (Terashima

et al., 2006).

In a previous paper (Terashima et al., 2006) by

the authors, the forwards-bacwards velocity was not

included in the ANFIS system of the OMW and

then a Reduction Multiplicative Factor (RMF) was

used for the improvement of the rotational motion

of the OMW when there was some interference of

the forwards-backwards velocity. By using the RMF

it was possible to achieve good operability in the

forwards-backwards motion, lateral motion and rota-

tional motion. However the results were not satisfac-

tory in the case of slanting motion. By including the

forwards-backwards velocity in the ANFIS system as

shown in Fig. 6, and with the use of the touch panel

for providing teaching reference for the learning, it

was possible to acomplish a general omni-directional

motion. Simulation and experimental results in the

case of diagonal motion are shown in Fig. 13 and Fig.

14.

Hence, in this paper, an innovative method for

improving the operability of a power assist omni-

directional wheelchair by using a touch panel with

neuro-fuzzy controller as a human interface is pro-

posed.

2 CONSTRUCTION OF OMW

USING A TOUCH PANEL AS

HUMAN INTERFACE

A holonomic omni-directional wheelchair (OMW)

using omni-wheels has been built by authors’ gropup,

as is described in (Kitagawa et al., 2002)-(Kitagawa

et al., 2001). Figure 1 shows an overview of the

OMW developein by authors’ group.

The OMW is able to move in any arbitrary direc-

tion without changing the direction of the wheels.

In this system, four omni-directional wheels are in-

dividually and simply driven by four motors. Each

wheel has passively driven free rollers at their circum-

ference. The wheel that rolls perpendicularly to the

direction of movement does not stop its movement

because of the passively driven free rollers. These

wheels thus allow movement that is holonomic and

omni-directional.

The OMW is also equipped with a handle and a six-

axis force sensor, as shown in Fig. 1, that allows the

OMW’s use in power-assist mode. The force that the

ICINCO 2007 - International Conference on Informatics in Control, Automation and Robotics

Figure 2: Touch panel used for the OMW.

Figure 3: GUI developed for the touch panel.

attendant inputs to the grips of the handle is measured

by this force sensor. Second order lag ﬁlter is used for

the transformation from force to velocity (Terashima

et al., 2006).

A touch panel is a display device that accepts user

input by means of a touch sensitive screen. Because

of their compact nature and ease-of-use, touch panels

are typically deployed for user interfaces in automa-

tion systems, such as high-end residential and indus-

trial control. Touch panels are also becoming com-

mon on portable computers such as Tablet PCs, Ultra-

Mobile PCs and consumer devices such as VOIP

phones. In this research, a touch panel as shown in

Fig. 2 is used as an input device in which the atten-

dant of the OMW draws the desired direction of mo-

tion. As shown in Fig. 2, the touch panel is mounted

in the rear part of the OMW such as the attendant can

reach to it easily. The touch panel used in this re-

search is a TFT Touch Monitor HV-141T produced by

ULTEC Corporation, Japan. A GUI (Graphical User

Interface) was developed for making easy the interac-

tion with the attendant, as show in Fig. 3. In this GUI

the attendant can draw any kind of motion, be it an

Figure 4: Working force.

Table 1: Fuzzy reasoning rules for lateral motion and rota-

tional motion.

R Antecedent Consequent

1 If V y < 0 and ω < 0, then ω < 0

2 If V y ≈ 0 and ω < 0, then ω < 0

3 If V y > 0 and ω < 0, then V y > 0

4 If V y < 0 and ω ≈ 0, then V y < 0

5 If V y ≈ 0 and ω ≈ 0, then 0

6 If V y > 0 and ω ≈ 0, then V y > 0

7 If V y < 0 and ω > 0, then V y < 0

8 If V y ≈ 0 and ω > 0, then ω > 0

9 If V y > 0 and ω > 0, then ω > 0

slanting motion, or a rotational movement.

3 NEURO-FUZZY SYSTEM FOR

IMPROVING OPERABILITY

When the user tries to rotate OMW around its grav-

ity center, OMW begins to slide and the radius of ro-

tation sometimes becomes very big. Then, rotation

around the center is very difﬁcult (Kitagawa et al.,

2004). A survey was conducted among various at-

tendants trying to discover some relationships in the

way they realized forwards-backwards, lateral and ro-

tational movements. The goal of the survey was to

ﬁnd general rules that related the three mentioned

motions. Even when it was impossible to ﬁnd gen-

eral rules that explained all cases, a relationship was

found between lateral and rotational movements by

authors. These relationships were used as the base for

constructing a fuzzy reasoning system (MathWorks,

2002)-(Harris et al., 1993) that helped to improve the

operability of the OMW. In order to establish the rules

of direction inference, ﬁrst, the force applied to the

grips of the force sensor are changed to the center of

OMW, as shown in Fig. 4. It is easy to derive a ba-

ADAPTIVE CONTROL BY NEURO-FUZZY SYSTEM OF AN OMNI-DIRECTIONAL WHEELCHAIR USING A

TOUCH PANEL AS HUMAN-FRIENDLY INTERFACE

Table 2: Fuzzy rules for the change of V x in order to im-

prove operability.

R Antecedent Consequent

1 If V x < 0 and V y < 0, then V x < 0

2 If V x ≈ 0 and V y < 0, then V x ≈ 0

3 If V x > 0 and V y < 0, then V x > 0

4 If V x < 0 and V y ≈ 0, then V x < 0

5 If V x ≈ 0 and V y ≈ 0, then 0

6 If V x > 0 and V y ≈ 0, then V x > 0

7 If V x < 0 and V y > 0, then V x < 0

8 If V x ≈ 0 and V y > 0, then V x ≈ 0

9 If V x > 0 and V y > 0, then V x > 0

6axis force

sensor

2nd-order

lag filter

directional

reasoning

Figure 5: Block diagram of the power assist system.

Figure 6: Contents of the block ”directional reasoning”.

sic equation to compensate the difference between the

sensor and the actuators allocation (Kitagawa et al.,

2004). However, it is difﬁcult to exactly give the force

for the sensor to the target direction. Further, how to

give the force for the gripper sensor is slightly dif-

ferent depending on persons even for the same target

motion of the OMW. The rules of direction inference,

in which just lateral motion and rotational motion are

considered, are shown in Table 1. In Table 1, V y rep-

resents the lateral velocity of the OMW, and ω repre-

sents the angular velocity of the OMW. The forwards

and backwards velocity of the OMW is given by V x.

The system in which fuzzy reasoning was applied

just to the lateral and rotational velocity was tested,

and it was found that even when the operability in

lateral direction was improved, there were still some

problems with the rotational movement because of a

component V x that did not allowed to achieved a per-

fect rotation over the center of gravity of the OMW.

A Reduction Multiplicative Factor (RMF) which de-

creases the value of V x in the case of rotational mo-

Figure 7: Results when fuzzy reasoning is not applied for

improving operability.

Figure 8: Results when fuzzy reasoning is used by

”Attendant 1”.

tion, and keeps it unchanged in the case of forwards-

backwards movement was the solution provided by

authors in previous research (Terashima et al., 2006).

By using the RMF it was possible to improve the

forwards-backwards motion, lateral motion and rota-

tional motion over the gravity center of the OMW.

However, as V y was subjected to fuzzy reasoning

and V x was not, it was not possible to achieve good

operability for slanting motions, like diagonal motion.

In the case of diagonal motion, for example, the at-

tendant tries to move the OMW in such a way that

the inputs of V x and V y are almost the same in the

beginnig. Nevertheless, as V y is subjected to direc-

tional reasoning, its value changes. V x is not sub-

jected to directional reasoning, then its value remains

always the same. As a consequence, it is not possible

to achieve good operability in diagonal motion.

For solving this problem, V x was subjected to di-

rectional reasoning too using the fuzzy rules shown

in Table 2. This rules make it possible to include V x

in the fuzzy reasoning system without disturbing the

ICINCO 2007 - International Conference on Informatics in Control, Automation and Robotics

Figure 9: Results when fuzzy reasoning is used by

”Attendant 2”.

Figure 10: Results when fuzzy reasoning is used by

”Attendant 3”.

values of V y or ω. The block diagram of the sys-

tem that considers power assist and fuzzy reasoning

is shown in Fig. 5, and the contents of the block la-

beled as ”directional reasoning” are shown in Fig. 6.

By including V x in the ANFIS system it was possible

to acomplish a general omni-directional motion.

Fig. 7 shows the results in the case of a counter-

clockwise rotational over the center of gravity of the

OMW when no fuzzy reasoning is used. It is possi-

ble to see that there is a deviation in the lateral di-

rection as well as in the forwards-backwards direc-

tion. For solving this problem, the fuzzy system was

used. It was tuned by trial and error, as explained

in (Kitagawa et al., 2004), for an attendant that will

be called ”Attendant 1”, and the results, presented in

Fig. 8 shows that the rotational movement was im-

proved considerably. However, when the same system

was tested with two more different attendants, called

”Attendant 2” and ”Attendant 3”, the results were not

as good as in the case of ”Attendant 1”, as shown in

Fig. 9 and Fig. 10. It means that the system must be

tuned in order to respond to the individual character-

istics of the different attendants. However, the tuning

by trial and error is time consuming and boring for

the attendant. For that reason, the automatic tuning

of the system by using a neuro-fuzzy system, ANFIS

(Adaptive-Neural Fuzzy Inference System) was pro-

posed and developed as described in (Terashima et al.,

2006). The ANFIS system of the OMW provided in

this paper is shown in Fig. 11.

4 ADAPTIVE CONTROL WITH

HUMAN INTERFACE AND

RESULTS

In previous research (Terashima et al., 2006), the de-

sired direction of motion of the attendant was input

by using the keyboard of the computer of the OMW.

However, the attendant could not get a clear idea of

the direction in which he wanted to move, neither ver-

ify if the real motion of the OMW really corresponded

to his desire. In order to provide the attendant with an

easy way for inputing the desired direction of motion

and for verifying the direction of motion, a human in-

terface consisting of touch panel, as shown in Fig. 2

is used. A GUI (Graphical User Interface) was de-

veloped for making easy the interaction with the at-

tendant, as shown in Fig. 3. In this GUI the atten-

dant can draw any kind of motion, like, for example,

an slanting motion, or a rotational movement. More-

over, it allows the attendat to follow the motion of the

OMW in the screen of the touch panel, and compare

the difference between the desired motion and the real

motion of the OMW. The complete system, when the

touch panel is included, is shown in Fig. 12.

The procedure for applying the touch panel is as

follows:

1. First, the attendant draws in the touch panel the

kind of movement that he desires to accomplish,

as teaching signal for the learning of Neural Net-

works.

2. Then, the attendant moves the OMW trying to ac-

complish the desired motion.

3. However, in the general case, there as a differ-

ence between the desired motion and the real mo-

tion. This difference is used for the training of

the ANFIS system of the OMW, as explained in

(Terashima et al., 2006).

This system was used for supporting the operation

of the attendant in many kinds of movements. Like

for example forwards-backwards motion, lateral mo-

tion, rotational over the gravity center of the OMW in

clockwise and counter-clockwise direction, and many

cases of slanting motion. In Fig. 13 it is possible

ADAPTIVE CONTROL BY NEURO-FUZZY SYSTEM OF AN OMNI-DIRECTIONAL WHEELCHAIR USING A

TOUCH PANEL AS HUMAN-FRIENDLY INTERFACE

to observe the simulation results of one attendant for

the case of diagonal movement to the upper right cor-

ner of the XY system shown. Fig. 13 (a) shows the

diagonal trajectory obtained before tuning is accom-

plished. It is possible to see that it is more an arc than

an straight diagonal line. By using the same input

data used in Fig. 13 (a), the system is tuned by using

ANFIS, and the trajectory obtained after the tuning

is shown in Fig. 13 (b). It can be observed that the

trajectory has been improved, as expected. The num-

ber of data used for the training of the ANFIS was in

the range of 3500 ∼ 4000 data, and the learning time

was around 30 [s] ∼ 40 [s] in a Pentimum III 1 [GHz]

personal computer. The system was tested by exper-

iment, for the same attendant, with the results shown

in Fig. 14 (a) for the case before tuning, and Fig. 14

(b) for the case after tuning. As in the case of the sim-

ulation, the trajectory obtained in the experiments is

not so good before tuning, but it was improved after

the tuning of the ANFIS system of the OMW.

5 CONCLUSIONS

An innovative human-interface using a touch panel

that provides easy input and feedback information

in real time of the operation of a power-assisted

wheelchair was developed. Furthermore, adaptive

control using a neuro-fuzzy system was proposed in a

human friendly fashion by means of a touch panel as

a human-interface for improving the operability of the

wheelchair. The system was tested by simulation and

experiments, and its efectiveness was demonstrated.

ACKNOWLEDGEMENTS

This work was partially supported by The 21

Cen-

tury COE (Center of Excellence) Program ”Intelligent

Human Sensing”

REFERENCES

FrankMobilitySystems (2002). Frank mobility systems inc.

In http://www.frankmobility.com/.

Harris, C. J. et al. (1993). Intelligent control. World Scien-

tiﬁc.

Jang, J. (1993). Anﬁs: Adaptive-network-based fuzzy infer-

ence system. In IEEE Transactions on Systems, Man,

and Cybernetics, Vol. 23(3), pp. 665-685.

Kitagawa, H. et al. (2001). Semi-autonomous obstacle

avoidance of omnidirectional wheelchair by joystick

impedance control. In Proc. IEEE/RSJ Int. Symp. on

Intelligent Robots and Systems, pp. 2148-2153.

Kitagawa, H. et al. (2002). Motion control of omnidi-

rectional wheelchair considering patient comfort. In

Proc. IFAC World Congress, T-Tu-E20.

Kitagawa, H. et al. (2004). Fuzzy power assist control

system for omni-directional transport wheelchair. In

Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and

SystemsC1580-1585.

Lin, C. T. and Lee, C. S. G. (1991). Neural-network-based

fuzzy logic control and decision system. In IEEE

Trans. on Computers, Vol. 40(12), pp. 1321-1336.

Maeda, H. et al. (2000). Development of omni-directional

cart with power assist system (in japanese). In Proc.

18th Annual Conf. of Robotics Society of Japan, 15,

pp.1155-1156.

MathWorks (2002). Fuzzy logic toolbox user’s guide ver-

sion 2, pp. 3-18. The Matworks Inc.

Seki, H. et al. (2005). Novel driving control of power as-

sisted wheelchair based on minimum jerk trajectory.

In IEEJ Trans. EIS Vol. 125(7) (in Japanese), pp. 1133

- 1139.

Terashima, K. et al. (2004). Frequency shape control of

omni-directional wheelchair to increase user’s com-

fort. In Proceedings of the 2004 IEEE International

Conference on Robotics and Automation (ICRA), pp.

3119-3124.

Terashima, K. et al. (2006). Enhancement of maneuver-

ability of a power assist omni-directional wheelchair

by application of neuro-fuzzy control. In Proceedings

of the 3

International Conference on Informatics in

Control Robotics and Automation (ICINCO 2006), pp.

67-75.

Wada, M. and Asada, H. (1999). Design and control of a

variable footprint mechanism for holonomic omnidi-

rectional vehicles and its application to wheelchairs.

In Proc. IEEE Trans. Robot. Automat, 15, pp. 978-

989.

West, M. and Asada, H. (1992). Design of a holonomic om-

nidirectional vehicle. In Proc. IEEE Int. Conf. Robot.

Automat., pp. 97-103.

ICINCO 2007 - International Conference on Informatics in Control, Automation and Robotics

Figure 11: ANFIS systems of the OMW.

Figure 12: Complete system when the touch panel is in-

cluded.

Figure 13: Simulation results for one attendant in the case

of diagonal movemement to the right.

ADAPTIVE CONTROL BY NEURO-FUZZY SYSTEM OF AN OMNI-DIRECTIONAL WHEELCHAIR USING A

TOUCH PANEL AS HUMAN-FRIENDLY INTERFACE

Figure 14: Experimental results for one attendant in the case

of diagonal movemement to the right.

ICINCO 2007 - International Conference on Informatics in Control, Automation and Robotics