Table of Contents


Computers, Graphics, & Learning

Copyright 2000 Lloyd P. Rieber

Chapter 9

Multimedia

OVERVIEW

This final chapter briefly considers the relationship among computers, graphics, and learning to the budding area of multimedia. Multimedia, and the closely related area of interactive video, is described in relation to design issues surrounding "hyper-" environments, such as hypertext and hypermedia. Some design considerations based on constructivism are also revisited.

OBJECTIVES

Comprehension

  1. After reading this chapter, you should be able to:
  2. Define multimedia, hypermedia, and interactive video.
  3. Describe how arguments and issues related to constructivism, instructional design, and learner control may extend to multimedia learning environments.
  4. Describe five levels of interactivity associated with interactive video.
  5. Summarize some of the research associated with multimedia, interactive video, and hypermedia.

Application

After reading this chapter, you should be able to:

  1. Apply the theory, research, and design principles considered in previous chapters to multimedia.

It should be clear by now that, if used appropriately, graphics can be an important part of learning and instruction. We have critically analyzed functions served by graphics in learning and instruction, especially when designed, developed, and delivered by computer. As computer technologies continue to flourish in education and as computers increase in graphical ability, there will be a strong need for designers to remain in control of how these technologies will be applied to learning environments. History has shown that the temptation to have machine technologies drive instructional design is difficult to resist. There will always be the tendency by some to believe that they have the right to abandon and ignore established knowledge bases as they apply new technologies. There will also be others who feel a need to maintain the status quo. This book has presented arguments suggesting that current knowledge bases of instructional design derived by theory, research, and experience are crucial elements to consider. Yet, there should be no mistaking the need to extend and elaborate these knowledge bases to take into account new philosophies and techniques. Additionally, none of these knowledge bases negates the need for designers to be inventive, creative, and willing to take risks. Increasing instructional wisdom in light of emerging technologies has been slow and gradual. One foot should remain in what is known and understood (e.g., available theory and research), while the other foot carefully explores uncharted areas of design.

The purpose of this last chapter is to briefly discuss several computer applications that should be particularly relevant for those interested in instructional visualization in the years to come. The future of visually based computer technologies is at once exciting, inspiring, and intimidating. Fortunately, the case can be made that the principles covered in this book can offer the best offense and defense for tapping the potential of future technologies while avoiding being swept away by the flood of options and considerations. One of the areas we will consider is the broadly defined area of multimedia. First, however, we will revisit and put relative closure on a topic from the last chapter -- constructivism -- which has much to do with many issues related to multimedia, hypermedia, and interactive video.

CONSTRUCTIVISM REVISITED

As described and discussed in the previous chapter, constructivism asserts that learning is a continuous and never-ending process of building and reshaping mental structures. Knowledge cannot be imposed on an individual; rather, knowledge is itself constructed by each person. The inherent antagonism between direct instruction and constructivism was not meant to be resolved in the previous chapter. Instead, a working compromise was offered, which may allow instructional designers to tap the strengths of direct instructional methods and constructivism in designing learning environments. These learning environments share attributes of gaming, simulations, and microworlds.

The previous chapter asked designers to consider how to merge features of gaming, simulations, and microworlds as they construct learning environments for students. Of course, these learning environments should, at the very least, be flexible enough for learners to be able to appropriately alter the environment to match their abilities and interests. Space Shuttle Commander was offered as one example of how one might design computer software according to this confluence of instructivist and constructivist views. Radical constructivists, on the other hand, seek ways to have learners construct their own learning environments. Rather than consider if SSC is suitable for a group of learners, a radical constructivist would probably prefer to focus on how SSC's designer learned physics as a result of building it. Similarly, LOGO was not meant for designers to develop structured lessons, but rather for students to use as a kind of mathematical "erector set."

The same debate can be initiated for any authoring tool, graphical or otherwise. On one hand, graphical software packages are viewed as tools or resources for instructional designers to use as they develop instruction for learners. On the other hand, these same packages should be considered as graphical tools and resources for students to construct their own learning materials and experiences. These issues are not foreign to instructional designers and developers. Cognitive orientations to instructional design frequently call for generative learning strategies (Wittrock, 1974, 1978), where learners are asked to deliberately take action to create meaning from material. Rather than viewing students as passive agents who "receive" instruction, generative learning assumes and requires learners to be active participants in their own learning. The generative learning hypothesis creates a learning "partnership" between the instruction and the learner. Learners are given much authority and responsibility for their learning, but are guided by and through instruction.

Simple examples of generative learning activities include underlining meaningful parts of text, note-taking strategies, paraphrasing, and outlining textual passages. Other activities can be more elaborate, such as student- generated questions and the creation of mnemonic learning aids and concept maps. Any activity can promote generative learning in which learners are required to "consciously and intentionally . . . relate new information to their existing knowledge rather than responding to material without using personal, contextual knowledge" (Jonassen, 1988b, p. 154). Instead of presenting students with ready-made representational, analogical, or arbitrary graphics, a generative approach would ask students to create their own graphics, with guidance, for the same purpose: to clarify relationships, and to facilitate understanding, establish meaning, and promote motivation.

Experience shows that there are times when learners need (and perhaps want) some imposed structure and times when learners should be given more freedom and responsibility to direct and design their own learning paths. Research on learner control in CBI is inconsistent (Milheim & Martin, 1991). One pool of research generally indicates that total learner control of computer-based instruction is usually not advisable unless paired with some sort of coaching or advisement strategy (see review by Steinberg, 1989). Yet, other research indicates that learner control is an important characteristic of successful instruction (Kinzie, Sullivan, & Berdel, 1988). It seems that a full understanding of learner control must simultaneously take into account performance and motivation variables. Related theories of motivation and attribution suggest that learners should be provided with some level of control over the selection, sequence, and pacing of content in order to reinforce the belief that they personally control their own success (Milheim & Martin, 1991). Obviously, the issue of learner control must be based on a combination of perspectives -- some cognitive and some motivational.

Figure 9.1 illustrates an example of an instructional computer activity that attempts to balance these issues. The activity's main cognitive objective is to help a learning-disabled student develop a working sight-word vocabulary. A secondary cognitive objective is to help the student with shape recognition. Both cognitive objectives are set in a highly motivating graphical context. When any of the words on the right is selected, the computer pronounces the word and displays a simple graphic of the word comprised of a combination of the three basic shapes. The student can then fill in the graphic by moving shapes onto the graphic's outline. The student is free to choose any of the words on the right at any time or to just doodle. Words can be added to or deleted from the list. This activity provides a variety of sensory inputs, including tactile, to help the student associate the written and spoken word. The activity, by nature of the available words, is guided, yet the student is free to explore and make individual choices. Also, the student can choose at any time to "clean up" the shapes and start over. The computer animates all the shapes back to their starting positions, which has proven to be a very motivating feature.

Figure 9.1

An example of a computer activity that balances the guidance of instruction with a student's purposeful construction of ideas and concepts.

The decision of when to use a more instructivistic or constructivistic orientation will depend largely on the interplay of the learner's experience or background in a particular domain and the learner's ability to self-regulate his or her learning in the domain. Novices will be especially prone to disorientation and confusion if left without guidance. Conversely, as students become more experienced and confident in a domain, they may become more resistant to imposed control on what they learn, how they learn, and when they learn. These same arguments are currently being played out in the case of multimedia (Locatis, Letourneau, & Banvard, 1989).

MULTIMEDIA

Multimedia is one of the latest buzzwords in educational technology. As such, its meaning has been stretched to fit almost any situation in which a variety of educational media are used. In its most general sense, it refers to any instructional delivery system that includes two or more media components, such as print-based, computer-based, and video-based. A traditional instructional setting combining lecture with a slide/tape presentation could be considered multimedia, for example. In its more common usage, it refers to integrated instructional systems that deliver a wide range of visual and verbal stimuli, usually through or in tandem with computer-based technologies. Although the computer is not necessarily a prerequisite component of multimedia, it is usually the focus of the instructional system. The most common multimedia systems are highly interactive computer-managed video/audio systems. Figure 9.2 illustrates a high-end work station to produce multimedia, and Figure 9.3 shows a system design for the delivery of multimedia. Ambron and Hooper (1990) define interactive multimedia as "a collection of computer-centered technologies that give a user the capability to access and manipulate text, sounds, and images" (p. xi). Although this book is not about multimedia, per se, it can be argued that the concepts and principles that have been discussed are directly relevant to multimedia. The principles of instructional visualization on which we have focused must be seriously considered in educational applications of multimedia.

Multimedia is not a new concept. However, enthusiasm for multimedia has grown as manufacturers rapidly expand computer hardware to use, integrate, and standardize video and audio formats in their systems. But the greatest enthusiasm for multimedia is probably due to the ease with which text, graphics, sounds, and video can be incorporated and accessed in instructional systems. As discussed briefly in chapter 3, the range of video and audio capabilities of desktop computers is evolving at a tremendous rate. For better or for worse, there seems to be a commitment among hardware manufacturers to conquer the sizable memory and processing hurdles inherent with making video and audio an integrated part of the desktop computer. The current proliferation of compact disc (CD) technology is a case in point.

Much of the enthusiasm for multimedia is centered on hardware. Proponents of multimedia in education usually refer to the old argument that merely increasing the external modes of delivery will result in increases in learning. Other proponents use surface-level arguments that students learn best when given a great variety of stimuli and instructional strategies. While there has been some initial work done to shift multimedia research from media to psychology (see Nix & Spiro, 1990, for example), rarely do the most popular media arguments extend beyond novelty effects. Although there is cause for enthusiasm, given the increasing number of options available to the instructional developer by computer-based multimedia advances, many of these arguments are in danger of falling into the "technocentric design" traps discussed in chapter 1. Enthusiasts also risk the unfortunate mindset that the past 50 years of experience (both successes and failures) with educational media do not apply to multimedia. There is also the curious dilemma of the hardware evolving faster than instructional designers, developers, and researchers are able to test and apply the resulting applications. Unfortunately, this pattern of forgetting the old while not being able to keep up with the new has been often repeated. There is also the continual danger (and paradox) that software designers will not be able to have direct influence on hardware advances.

 

Figure 9.2

A sample computer configuration to produce instructional multimedia material. Copyright 1992 by R.D. Zellner and reprinted with permission.

 

Figure 9.3

A sample computer configuration needed to deliver instructional multimedia material. Copyright 1992 by R.D. Zellner and reprinted with permission.

Multimedia and Hypermedia

Interestingly, most current discussions of multimedia are linked with the development of hypertext tools, such as Apple's HyperCard and IBM's Linkway and ToolBook. The term "hyper" translates simply as "link" and has been extended to include hypermedia environments, or systems that link various media, such as computer and video (Locatis, Letourneau, & Banvard, 1989). The origin of hypertext is usually traced to an article by Vannevar Bush (1945), then president of the Massachusetts Institute of Technology, entitled As We May Think (the term "hypertext" was actually coined by Ted Nelson in the 1960s). Bush described a hypothetical device, called the Memex, which would allow people to explore ideas in nonlinear ways. The technology has only recently been able to catch up with Bush's original ideas.

The philosophy behind hypertext and hypermedia environments is that informational and instructional systems can be built to allow users to explore knowledge bases in ways that may mirror how people actually think. In contrast to CBI, founded on some external instructional design model, hypertext is meant to allow users to create their own knowledge representations in a particular domain. Hypermedia proponents argue that human thinking is not linear, so, therefore, users should be able to explore informational systems by selecting and sequencing their own paths in a domain.

There is relative support for hypertext from the field of cognitive science. Hypertext environments seem to closely conform to the idea of propositional networks discussed in chapter 4, which suggest that cognition involves an ever-transforming network of nodes and links. Nodes represent one of many different kinds of informational units, and links represent how the nodes are related or associated. As the basic unit of information, nodes can be represented verbally or visually. Therefore, a hypermedia environment would allow for the knowledge base to be represented through a variety of stimuli, including text, graphics, video, and sound. In addition, users would be able to add or reconfigure the hypermedia environment, thus promoting an interactive and dynamic system. However, the cognitive power of hypermedia is derived much more from the links than the nodes. Higher-order applications of human memory, such as problem-solving, are believed to be a function of the strength (in terms of meaning) of the association between informational units.

Hypermedia and CBI go in seemingly opposite design directions. In CBI, authors and designers make decisions on how information will be related, and that representation is subsequently imposed on learners. Proponents of hypermedia argue that since there may be as many representations of a knowledge base as there are learners, one interpretation given by the author is more or less arbitrary. Therefore, they suggest it is better to allow users to make their own associations. However, cognitive interpretations of instructional design show many similarities with principles of hypermedia (Jonassen, 1991b).

The serious research on hypertext has only begun. Considering the high expectations, most current reports have been discouraging for proponents (an often-repeated pattern for new educational media innovations). It appears that without guidance, novices have a difficult time knowing how to explore a hypertext environment and often become disoriented (Tripp & Roby, 1990; Jonassen, 1988c). Novices have limited cues or strategies for how to allocate their limited attentional resources. Also, as users allocate cognitive processing to certain tasks, there is the risk that their performance on other potentially rewarding tasks will deteriorate. Based on related research on human cognition, it is likely that hypertext environments would be more facilitative as a user becomes more familiar with a content area or domain. Hypertext environments are probably not good instructional systems for introducing novices to an area, but may be good environments in which users can subsequently organize and integrate information. It may be that users simply are not accustomed to the nature (and responsibility) of having total navigational control within a knowledge base. Simply providing an environment that allows users to customize knowledge for themselves does not necessarily mean that they will be able to do so.

Applied properly, hypermedia environments have much potential in education. However, it is clear that significant learning will not occur simply by haphazardly planting these environments in educational settings. There is surely a need for both structured and unstructured learning environments in training and education. The potentials of hypermedia closely parallel the arguments calling for a balance between deductive and inductive learning strategies that were presented in the previous chapter in the context of simulations, games, and microworlds.

Multimedia and Interactive Video

Interactive video is the most longstanding application area that most closely resembles current interpretations of multimedia. Interactive video is best thought of as the marriage of computer and video technologies. It is typical for users to emphasize either the video or computer component of interactive video, such as considering it as computer-managed video or as CBI with a video component. However, many point to the increased visualization capabilities as the chief advantage of interactive video systems (Locatis, Charuhas, & Banvard, 1990), especially in regard to increasing a learner's control over the video material (Hannafin & Peck, 1988). In most systems, a computer is directly cabled, or interfaced, with a video play-back unit, most frequently a laser disc player. One side of a standard 12-inch videodisc typically contains 54,000 individual frames that can be referenced directly, allowing for 30 minutes of linear video footage (figuring 30 frames per second). However, interactive video can also include computer-controlled videotape players. Despite the utility of interactive videotape, laser disc systems are far more popular for several reasons. Laser discs allow for the random access of any frame with a typical delay of no more than about two seconds, whereas tape systems take relatively large amounts of time while the system either fast-forwards or rewinds to the proper location. Laser discs also provide unlimited playbacks, including freeze frames, with no deterioration of quality since there is no physical contact with the medium -- the digital information on the disc is accessed by the reflection of a laser beam. In contrast, the read/write heads of a videotape system are in continuous contact with the magnetically charged tape, resulting in a relatively quick drop in signal quality after a series of plays. Although the physical differences between tape and disc may be offset so that no instructional differences may be experienced (e.g., Hannafin & Phillips, 1987), videodisc applications are far more common. The growth and improvement of disc technology, as evidenced by compact discs, seem to indicate that disc technology will ride the crest of applications in the immediate future.

A taxonomy of the levels of interactive video has been offered (Daynes & Butler, 1984; Gayeski & Williams, 1985; Parsloe, 1983). Although this taxonomy is largely hardware-based, it speaks to the interactive opportunities made possible through different hardware configurations. The first two levels of the taxonomy, levels 0 and 1, includes only video technology -- no computer technology is involved. At level 0 there is no overt interactivity between the video materials and students, such as in linear video presentations or broadcast television. There is no opportunity to interrupt the video presentation once it has begun. Level 1 interactive video includes manual interruptions of a stand-alone videodisc or videotape, usually stop/start, by either the teacher or student. Level 1 also includes manual branching and searching of segments by the teacher or student through the manual controls of the video unit.

Level 2 interactive video is the first level at which computer technology begins to play a role. Through an onboard microprocessor, a videodisc player is able to run a program encoded onto the videodisc itself. Such programs would allow for simple conditional branching based on a student's input to the videodisc's keypad. However, level 3 is the level at which the video player is interfaced to a separate computer. This is the level which is usually considered for mainstream interactive video or multimedia applications. Most level 3 systems have one monitor for the computer and one for the video, although some more expensive systems allow one monitor to be switched between the video and computer signal. The latest technology allows a computer monitor to include video "windows," in which a small portion of the monitor displays video material. It is hoped that such video windows will be as easily accessed and manipulated in a graphical user interface (GUI) as text and graphic windows are now.

Level 4 usually defines the last level of interactive video (although some taxonomies include several more levels). One might refer to this as the "dare to dream" level. Typically, level 4 systems include a creative assortment of hardware, such as multiple video units, sound synthesizers, voice recognition, touch screens, etc. In light of new technologies, such as virtual reality (see chapter 8), interactive video takes on a completely new definition. One might learn about the relationship between enzymes and proteins not by simply interacting with a computer-based multimedia station, but by reaching out, grabbing, and manipulating a particular molecule or even perhaps by "becoming" the molecule.

As you can see, these levels are heavily hardware-oriented. While such a taxonomy makes it easy for newcomers to understand the system configurations, it also unfortunately promotes technocentric design, as discussed in chapter 1. While advances in hardware technologies provide wonderful opportunities for instruction and learning, it is only through the software or idea technologies, such as instructional design, that the potentials can be realized. For this reason, most of the serious research and developmental work in interactive video has carefully concerned the research and theory most related to the parent technologies of instructional design, CBI, and video instruction. Although there are those who consider interactive video and multimedia as completely unique technologies, a more realistic view is that interactive video and multimedia will be best appropriated for learning when based on careful analysis of learner and instructional design attributes.

Views on the effectiveness of interactive video range from highly enthusiastic (Debloois, 1982) to cautious (Hannafin, Garhart, Rieber, & Phillips, 1985). Most of the evidence for interactive video has come from developmental projects that have been largely atheoretical (Cronin & Cronin, 1992). The most credible research has investigated instructional design issues with interactive video, rather than studying the medium itself (see for example, Hannafin, 1985; Hannafin, 1992; and Hannafin & Hughes, 1986).

Arguments for interactive video and multimedia, apart from the interactive components of CBI, are best understood as times when video provides the best source of instructional delivery. Some rationales for video are rooted in the cognitive domain, such as the use of high-fidelity video images to demonstrate what a particular chemical reaction will look like without exposing students to highly volatile chemicals (Smith & Jones, 1991) or medical education where real-life situations can be better represented with video than text and graphics (Nashel & Martin, 1991).

One of the most compelling justifications for video may be its dramatic and immediate ability to elicit an emotional response from an individual. Such a reaction can provide a strong motivational incentive to choose and persist in a task. For example, compare the differences between hearing or reading an account of a bridge collapse and actually watching the video footage of the bridge oscillating wildly before disintegrating and crashing into the water below. (See Footnote 1)

Other examples combine cognitive and motivational elements by using video to provide a meaningful context for learning, called "anchored instruction" (Cognition and Technology Group at Vanderbilt, 1990). An example would be showing students a video segment where Indiana Jones carefully replaces a golden idol with a sandbag to prevent the booby traps from being triggered as a context for understanding the relationship between volume and weight (Bransford, Sherwood, & Hasselbring, 1986; also see descriptions of the Jasper Woodbury Problem Solving Series by the Cognition and Technology Group at Vanderbilt, 1992). Other applications point to social and language applications, such as using interactive video in bilingual training (Reed, 1991). Still others have a more constructivistic flavor, where children build their own interactive video materials to learn about science and social studies (Gerrish, 1991).

Despite the allure of interactive video and other multimedia environments, there is every reason to believe that the instructional design of these systems should be based on a careful analysis of the many interrelated and interdependent elements discussed throughout this book. These include psychological foundations of the individual, especially those related to visual learning, and the instructional design of interactive learning systems.

A FINAL WORD

This book has been but a beginning in the effort to tie together the theory, research, and practice of instructional visualization in the computer age. We have considered information from many different areas and points of view. This book was written to summarize, organize, and synthesize a wide spectrum of ideas related to computers, graphics, and learning. It is hoped that you now feel empowered, not overwhelmed. Much is written and known about these three topics when they are considered individually or in pairs, but little when taken collectively. Designers who have searched the literature for guidance have probably found themselves either with the feeling that no integrated literature is available or swamped with the idea that everything they read seems to apply. A fundamental principle of learning is that when people have too much or too little to do or consider, they seek either to "turn off" the task or look for another. In either case, the result is invariably the same -- they stop trying the task at hand. It is the goal of this book to provide a compromise in both cases. For some, this book may have opened up an understanding of how many areas are relevant that were not previously considered. For others, this book may have organized the flood of available information, thereby increasing its potential to be understood and used. Ultimately, it is up to you to decide how to best apply what we know about computers, graphics, and learning in the design of instructional systems and learning environments.

Despite this book's frequently cautious tone, there is much cause for enthusiasm. Desktop computer technology gives designers and developers access to impressive graphical power. This does not diminish the role of graphic designers, artists, programmers, and technicians. Instead, it places more power into the hands of people who more directly influence the construction of learning environments. Although this book was written for instructional designers and developers, it is hoped that this book has also provided a window for other interested professionals to glimpse at the task of putting computers and graphics to use in instructional settings. We all come to instructional design with our own strengths, interests, and experiences. The challenge is to take advantage of all these diverse abilities to achieve the common goal of enhancing an individual's learning in a particular domain.

This is a good time to repeat that this book is not meant to be a substitute for a thorough introduction to the many ideas and areas that have been included here. Probably the most important of these are learning theory, instructional design, computer-based instruction, and computer graphics. Each of these areas can be considered as distinct and sophisticated fields of inquiry requiring years of study to adequately understand. However, this book has tried to bring these areas together without demanding that readers be experts in any one. An obvious next step is for you to further explore these areas.

Finally, it is recognized that many areas have not been adequately explored in this book. State-of-the-art computer visualization is a particular case in point where highly realistic (though perhaps imaginary) three-dimensional graphics are modeled, rendered, and animated on high-end computer graphics work stations (see Figure 9.4 for an intriguing example of 3-D graphics and Figure 9.5 on the next page for the solution). The development of virtual reality is another example. These have been deliberate omissions for two main reasons. First, most of these areas have not, as yet, been sufficiently applied to educational settings. For example, computer visualization has been applied most frequently in fields such as architecture, medicine, art, and commercial television. Second, few of these areas

 

Figure 9.4

This figure contains a 3-D message. To see it, you need to stare through the figure as though it were a window. As you relax your eyes, the two dots on top will blend into three dots (you may need to adjust your viewing distance). Be patient because it will take some time and practice for the image to appear. Beware, not all people report seeing the image. (See Figure 9.5 at the end of the chapter for the solution.)

have migrated to desktop computer applications. As educational practice catches up to the potentials of computer graphics technology, this will surely change.

It is fitting that this book end with an image of what it represents. We choose to compare an instructional designer applying knowledge about computers, graphics, and learning to an explorer who wants to verify which reports of a faraway land are accurate and which are pretend, imagined, or lies. The motivation to go will be part decreed, part economic, part curiosity, and part self-satisfaction. Theory and research related to learning and graphics are like basic training about many fundamental ideas and skills, such as knowledge about survival, first aid, navigation, map reading, and how to use a compass.

Instructional design is like plotting a course and setting out on the journey. The computer and other instructional materials are like the explorer's gear, which has been deliberately chosen for the trip. Important decisions must be made, and the consequences of good and bad choices made before and during the trip must be recognized and dealt with as they are encountered. Along the way, oceans are crossed, rivers are forded, and mountains are climbed; however, each step is meant to be taken in the charted direction. At times, the explorer travels already established routes and other times must blaze a new trail. There comes the time when the explorer must determine if the intended destination or another land has been reached. It is even possible that an entirely new world will be discovered. The journey is a success only when knowledge gained from it is shared, understood, and used by others. However, unlike some actual historical examples of this metaphor, our explorer celebrates the journey, as well as the destination, without exploiting or destroying that which is encountered along the way.

REVIEW

NOTES

  1. Such a disaster, the collapse of the Tacoma Narrows Bridge in Washington in the 1940s, was actually captured on film. The Nebraska Videodisc Production/Design Group designed an early interactive videodisc project that described the collapse and the physical science principles explaining why it occurred.

Figure 9.5

This is the solution to Figure 9.4.



Table of Contents