TEACHING MATH AND PHYSICS BY DECONSTRUCTING

GRAPHICS

Amit Shesh

School of Information Technology, Illinois State University, Normal, IL, U.S.A.

Keywords:

Graphics and Math, High School Math and Science, Interactive Virtual Environments.

Abstract:

The foundations of computer graphics theory lie in mathematics, whereas most effects that add realism to a

virtual graphical world are a direct simulation of the rules of physics. Complementary to the role of math

and physics in learning about computer graphics, this paper proposes using graphics which has widespread

appeal to directly encourage learning math and physics. We aim to use graphics to kindle interest in learning

basic math, physics and by extension, computer science at the high school and college level, by directly and

immediately matching theoretical concepts to how they are practically used in familiar graphics effects. We

propose to achieve this by creating a virtual navigable environment of arbitrary complexity that, upon simple

interaction, automatically de-constructs its constituent objects and various visual effects to reveal how they

were created as well as relevant physics such as light and object behavior. We concentrate on two technical

problems (a) setting up a visually appealing environment with ease for users like instructors who are not

necessarily familiar with technical details of graphics and (b) de-constructing, illustrating and viewing various

effects as automatically as possible so that users like students and teachers concentrate on the concepts and

not how best to illustrate them.

1 INTRODUCTION

Almost every aspect of computer graphics is based on

sound mathematical principles, with judicious com-

promise for engineering purposes. Any manifestation

of graphics, like movies and games, illustrates a va-

riety of math converging on a set of visual effects.

However widespread appeal of such visual effects

does not seem to translate into an equally widespread

interest in math, despite such a strong dependence on

it. Can computer graphics, as a “celebrity offspring”

of math, encourage its learning to a wider audience?

Most college-level computer graphics courses use

high-school level math. If graphics is used in a peda-

gogical role in such math courses, students may bene-

ﬁt from seeing a direct application of what they learn.

For example preliminary lighting from a game can be

used to illustrate the basic vector algebra that it uses,

moving characters could illustrate matrix operations

like multiplication, inversion, etc. Even college-level

courses can use these metaphors by using water/ﬁre

in games to illustrate applications of differential equa-

tions. Besides math, computer graphics can be used to

kindle interest in STEM ﬁelds like computer science

where math is a strong requirement. At our university

we see many students opt out of the computer sci-

ence major because of its math and science require-

ments. Even computer science students taking their

ﬁrst course in graphics are often put off by the signif-

icant math content. Using graphics effects that stu-

dents have seen in games and other graphics appli-

cations could persuade them to like and specialize in

computer science and other STEM ﬁelds.

Like math, physics plays a signiﬁcant role in many

graphics applications. Achieving “realism” in com-

puter games rests on the accuracy with which physi-

cal phenomena can be simulated. Exploiting the nat-

ural “pull” of such applications, can graphics be an-

other reason why a student ought to be interested in

physics? Even a simple game like Angry Birds(Angry

Birds, 2011) uses object dynamics and projectile mo-

tion. Can dynamics be taught with the aim of imple-

menting or at least explaining Angry Birds?

Interactive visualization has been used in recent

years to teach math concepts (The Open Directory

Project, 2011; CalcSee3D, 2011; Calculus.org, 2011;

Paul Seeburger’s Dynamic Calculus Site, 2011), but

strictly as an enhanced illustrator. Similarly animated

simulations have been used to teach dynamics(e.g.

Magic Paper(Magic Paper, 2011)). We believe com-

523

Shesh A..

TEACHING MATH AND PHYSICS BY DECONSTRUCTING GRAPHICS.

DOI: 10.5220/0003930905230526

In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP-2012), pages 523-526

ISBN: 978-989-8565-02-0

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

puter graphics can be promoted from a role of illus-

trator to that of a motivator, moving from visual de-

piction of math and physics to being the reason they

should be studied. Furthermore to achieve this we be-

lieve that the pedagogical role of graphics should be

more implicit, in that applications written with a dif-

ferent and entertaining goal can be used to illustrate

and even teach underlying math and physics. In this

paper we present a proof-of-concept idea of a navi-

gable environment that may outwardly act as a game,

but is able to de-construct itself upon interaction.

2 RELATED WORK

The mathematics community has investigated using

visualization to teach mathematics: we refer to a sum-

mary by Gutierrez et al. (Gutierrez and Boero, 2006).

Many online visualizations that illustrate Math con-

cepts are available (Java applets (The Open Direc-

tory Project, 2011; CalcSee3D, 2011), tools to learn

calculus (Calculus.org, 2011; Paul Seeburger’s Dy-

namic Calculus Site, 2011), etc.). All of the above

use graphics as an illustration tool: in contrast, we

propose a more direct role of using graphics as the

reason to learn Math by showing how it is practically

used.

Relevant graphics research includes tools such as

JHAVE (Jhave, 2011) for algorithm visualization and

Mathpad (Jr. and Zeleznik, 2004) to create sketch-

based tools for mathematical simulations. The Graph-

ics Teaching Tool (Spalter and Tenneson, 2006) in-

vestigates teaching graphics to non-computer science

majors. Both use graphics more intimately as a teach-

ing tool in education. Also a popular approach in

using computer graphics for math and science learn-

ing has been to use virtual reality (Geitz, 1991; Tax´en

and Naeve, 2002; Kaufmann and Schmalstieg, 2003;

Moustakas et al., 2005; Kaufmann and Schmalstieg,

2006; Kaufmann and Meyer, 2008), all of which have

an explicit learning metaphor. In contrast we pro-

pose an application that makes learning more implicit.

Similar to using using computer games as implicit

learning tools (Games for Learning Institute, 2011),

we focus on motivating high school and college stu-

dents towards enjoying math.

3 OVERALL VISION

Our overall vision comprises a 3D virtual environ-

ment that allows two types of user interaction: ex-

ploration/navigation and inspection. Although the ex-

ploratory experience would greatly beneﬁt from more

sophisticated interaction such as hand-held devices,

etc. we do not regard these technologies as being crit-

ical to the teaching power of the environment.

Exploration of the 3D environment is not geared

towards learning, and thus the “apparent” purpose of

the environment is different. The rationale for this

choice is to keep the user as interested as he/she would

be when interacting with a regular game, by minimiz-

ing any negative effects of it seeming like a learning

or pedagogical tool. Interesting possibilities include

a ﬁrst-person or a maze-based game, simulation of

an existing physical environment or interactive explo-

ration of an “alternate” virtual universe. During such

navigation and exploration, the user may move or in-

spect objects to which the system responds by cre-

ating and augmenting to the virtual world 3D illustra-

tions that de-construct various visual effects related to

the objects. At any time the user may switch to other

explanations about the same object, select another ob-

ject or return to the original, “un-annotated” 3D en-

vironment. Thus we envision the entire user experi-

ence as similar to but more interactive than watching a

movie on DVD, where the actual movie is augmented

with “behind-the-scenes” footage detailing how cer-

tain scenes were created.

3.1 Intended Audience

We target three types of users for our system:

High-school Students. High-school students learn

the basic math and physics that likely persuade or dis-

suade them from future careers in science and tech-

nology. Typical users would be students enrolled in

required and advanced math and physics courses that

teach and apply linear algebra and calculus.

College Students. We target college students who are

interested in a major that uses math in a signiﬁcant

way (e.g. STEM). We believe that such a system may

show such students more effectively about the “fun”

and practical use of the math that they learn in the

class, thereby steering them towards related careers

that are in high demand.

Teachers for all above Student Groups. For such a

system to be used in an academic environment, it must

be easily customizable by teachers to speciﬁc content.

3.2 Objectives

Ease of Environment Creation. As it is difﬁcult

to create a single environment that illustrates a large

number of examples on diverse types of math and

physics, we envision that a customized environment

would need to be created by teachers. For the system

to be effective, this should be neither cumbersome nor

GRAPP 2012 - International Conference on Computer Graphics Theory and Applications

524

should it expect the user to be knowledgeable in tech-

nical computer graphics. This could be accomplished

by allowing teachers to select concepts to be illus-

trated and using that to select only a relevant subset of

possible de-constructions in an existing environment,

or customizing a toolbox which the user can then use

to create a 3D environment.

As-Automatic-as-Possible Illustrations. The sys-

tem should not force theteacher to create details of the

3D illustrations, as this will make the process tedious

and time-consuming. This will also require 3D place-

ment and annotation tasks that are not suitable for

the identiﬁed user groups. The system should create

3D illustrations, choose appropriate vantage points to

view them and create navigationto view them as auto-

matically as possible, without needing extensive “set-

up interaction”. We envision achieving this by cre-

ating a toolbox of simple 3D entities and effects that

are “pre-programmed” to de-construct on command.

A complete 3D environment can then be assembled

using these tools.

Easy InteractionandUsability. Since learning is not

an explicit goal of the environment we expect users to

not have speciﬁc topics in mind when they explore it.

Thus the interaction should be easy, and non-speciﬁc

with respect to the actual math concepts or physical

phenomena that it is capable of describing. Since the

original purpose of the system would be explore or

play, the metaphor for inspection would match that

of its original application. Any customization due to

speciﬁc topics would be handled by correspondingly

changing the environment as discussed above.

4 A PROOF-OF-CONCEPT

SYSTEM

We illustrate our above idea using a simple 3D maze-

based game. The environment of this game has been

created using simple implicit shapes such as spheres,

boxes, cylinders and cones. The application pro-

gram uses scene graphs created from XML-based in-

put. The scene graph representation facilitates eas-

ier hierarchical modeling (i.e. creating the environ-

ment) as well as object-speciﬁc and object-relative

transformations and object placement for animated

3D scenes. Using XML ﬁles provides the ﬂexibility

to create the 3D environment part-by-part and possi-

bly programmatically by using maze-generation algo-

rithms (Weiss, 2006). Figure 1 shows some prelimi-

nary results from this system.

The user interacts with the 3D environment by ei-

ther moving in it (camera navigation), moving a spe-

ciﬁc object in it or inspecting the visual details of an

object. Camera movement could be either from a 1

person perspective, simulating a more immersive ex-

perience or a ﬂy-by perspective. Any object can be se-

lected by clicking on it. Although our current proof-

of-concept system uses manual selections to inspect

various properties of an object, we envision either a

gesture-based or 3D-menu-based interface that would

more closely match the interaction metaphor of the

game. Upon selecting an object, the following prop-

erties can be inspected.

Lighting. When any point on the selected object

is clicked, the scene automatically “augments” that

point with vectors relevant to its lighting (i.e. the view

vector towards the original camera, the local normal

vector and vectors representing the light directions for

each light source from that point) (Figure 1(b)). The

user can then move the vectors which causes lights

to move around the environment (Figure 1(c-d). In

this way the user can experiment more with how these

vectors affect the lighting of the scene.

Geometry. The geometry of a composite object

is shown by automatically creating an exploded

view and moving the camera to a suitable vantage

point. Thus each part is automatically de-constructed

to show its composition using simple shapes (Fig-

ure 1(e-g)).

Additional operations that can be supported in fu-

ture include:

Image Pasting. If the object is textured to provide a

certain look, this would be de-constructed by showing

the image and how it is pasted onto the object. Al-

though this is a speciﬁc operation related to graphics,

texture coordinates are often related to the geometry

of the object.

Object Animation. The user could select an object

and force it to move in a certain straight line. In

this case it would react appropriately with the envi-

ronment in an animation, showing object-dynamics.

Text-based Annotation. Using graphical elements

alone may not create self-explanatory annotations. In

future it would be possible to annotate the environ-

ment with text information. However such text would

be automatically placed in a “3D-aware” manner to

complement the overall scene. For example, the text

may appear as writing on a white board or easel that

is part of the scene.

5 FUTURE WORK

We propose creating a 3D world with hidden peda-

gogical capabilities, by which we hope to channel ex-

isting widespread interest in graphics and games into

one in math and science. We believe using a 3D en-

TEACHING MATH AND PHYSICS BY DECONSTRUCTING GRAPHICS

525

(a) (b) (c)

(d) (e) (f)

(g)

Figure 1: Representative results from our proof-of-concept system. (a) A view of a virtual room in our navigable environment.

(b) When the user switches to “Vector” mode and clicks on a point on the translucent sphere, the system adds relevant vectors

to illustrate lighting at that point and animates the camera smoothly to a close-up vantage point. The blue quad shows the

local orientation of the surface of the clicked object, with the green arrow showing the normal at that point. The pink arrow

points to the original camera position. The yellow arrows point to the three sources of light. (c) Upon clicking and dragging

any row, the arrow and the corresponding light move to observe the relationship of the vector to the illumination of the point.

One of the room lights and the table lamp are changed in this way. (d) The ﬁrst view point with the changed lighting. (e)

When the user switches to “Geometry” mode and clicks on the car on the table, its exploded view is automatically generated.

The camera smoothly animates to show the exploded view suitably. (f) Clicking on the stack of boxes from the new view

shows it in exploded view (g) same result for the lamp, showing its different parts along with the light source. Please see the

accompanying video for a real-time capture of this interaction.

vironment that de-constructs itself on command is an

effective way to achieve this.

The proposed framework can be customized to il-

lustrate speciﬁc effects and geared towards speciﬁc

audiences. There are many research challenges in

this proposed idea: how to create and customize such

an environment as automatically as possible while re-

taining the appeal of a game or other well-known ap-

plications and how to gear the underlying system to-

wards users who do not possess technical knowledge

of computer graphics.

REFERENCES

Angry Birds (2011). http://www.rovio.com/en/our-

work/games/view/1/angry-birds.

CalcSee3D (2011). http://web.rollins.edu/∼dchild/.

Calculus.org (2011). http://www.calculus.org.

Games for Learning Institute (2011). http://research.micros

oft.com/en-us/collaboration/institutes/gamesinstitute.

aspx.

Geitz, R. (1991). Algorithms and images: computer graph-

ics as an introduction to science. In Proc. SIGCSE,

pages 82–86.

Gutierrez, A. and Boero, P. (2006). Handbook of Research

on the Psychology of Mathematics Education. Sense

Publishers.

Jhave (2011). http://jhave.org/.

Jr., J. L. and Zeleznik, R. (2004). Mathpad2: a system for

the creation and exploration of mathematical sketches.

ACM Trans. Graph., 23(3):432–440.

Kaufmann, H. and Meyer, B. (2008). Simulating educa-

tional physical experiments in augmented reality. In

Proc. SIGGRAPH ASIA 2008 educators programme,

pages 3:1–3:8.

Kaufmann, H. and Schmalstieg, D. (2003). Mathematics

and geometry education with collaborative augmented

reality. Computers & Graphics, 27:339–345.

Kaufmann, H. and Schmalstieg, D. (2006). Designing im-

mersive virtual reality for geometry education. In

Proc. Virtual Reality Conference, pages 51–58.

Magic Paper (2011). http://icampus.mit.edu/projects/Natura

lInteraction.shtml.

Moustakas, K., Nikolakis, G., and Tzovaras, D. (2005). A

geometry education haptic vr application based on a

new virtual hand representation. In Proc. IEEE Con-

ference on Virtual Reality, pages 245–248.

Paul Seeburger’s Dynamic Calculus Site (2011).

http://web.monroecc.edu/pseeburger/.

Spalter, A. M. and Tenneson, D. (2006). The graphics

teaching tool. In Proc. SIGGRAPH Educators pro-

gram, page 41.

Tax´en, G. and Naeve, A. (2002). A system for explor-

ing open issues in vr-based education. Computers &

Graphics, 26(4):593–598.

The Open Directory Project (2011). http://www.dmoz.org/

Science/Math/Education/Java

Applets/.

Weiss, M. A. (2006). Data Structures and Algorithm Anal-

ysis in C++, pages 331–334. Addison Wesley, third

edition.

GRAPP 2012 - International Conference on Computer Graphics Theory and Applications

526