Sunday, November 16, 2008

Museum Remix; Further Consideration of 3D Virtual Surrogates for Real Museum Objects

The term "remix" has come to identify the phenomenon whereby a body of knowledge is understood through technology that rearranges and re-contextualizes it "in order to construct an original narrative," as would any multimedia emboldened web-based version of a museum object. Adopting such tools undoubtedly re-emphasizes the role of collections for public use and interest, while making their ability to inspire learning more participatory (Fisher 2007). Perhaps the most exciting innovation regarding the creation of surrogate virtual objects, and exhibitions to contextualize them within, is 3D imaging and modeling. Jim Forrest, Web Creative Director of the Peabody Essex Museum (PEM) in Salem, Massachusetts, sees the three-dimensional as the "next big phase" of web-development for online museum resources. His own recent projects include work on the virtual exhibition Joseph Cornell; Navigating the imagination (2007), which allows for interactive play and study with some of the artist's shadow boxes in simulated three-dimensions through the use of Flash animation and video, specifically Macromedia Flash Player 8.

One choice within this ongoing exhibit is to visit Cornell's "Little Dream Gum Machine," where the viewer can drag and drop a virtual penny into a slot to activate it. The penny cascades down the machine's revealed innards activating a link to other 2D digital versions of shadow boxes. this interactive play allows you to navigate through the collection and learn by seeing. Online web pages also serve adults and young learners at onsite computer kiosks like PEM's Art and Nature Center. Instruction on how to fold paper animals is among the hands on activities offered through Flash video in the Origami Now Folding Station 2007 web page for the exhibit Origami Now! (2007). Viewers learn by seeing and doing.

In addition to the digital images of origami works of art in Origami Now! , the computer-based interactive experience of this exhibit puts art in the context of not only something that is made by the artist, but something that you can make. Children and adults are enabled to learn what one can in the museum seemingly independent of educators, and this too may lend itself to unique experiences with objects and the hows and whys of their creation. The code of behavior required wthat barred the visitor from fully exploring the intricacies and the suggestive play of the real objects of Joseph Cornell's shadow boxes in the PEM exhibition are abandoned through the interactive play and study provided by these offerings in 3D animation and video. 3D virtual experiences with these objects are experientially more real than the more conventional single, frontal digital stills of objects contained in a digital collections database or the real object itself maintained within glass vitrines and motion detecting security fields whih put them out of reach of the visitor.

A collections database comprised totally of 3D images is for the time being unrealized at any museum, but there have been some, like PEM, that have taken the lead in creating examples of objects and environments that make use of a number of technologies which maked interactivity and simulated multi-dimensional study possible. The evolution of simple virtual representation of objects as exemplified by 360 degree rotational views and magnification options of detail have already been superseded by "modeled reconstruction and deconstruction" of virtual objects as exemplified in IBM's Digital Pieta project as well as by "virtual replacement of artifacts in situ
at their point of creation or discovery as experienced in the Eternal Egypt website (Tolva 2005). Although identified as 3D , such images are in reality 2D, but through various techniques like graphic modeling, digital video, and synching multiple digital stills of a real object using multimedia software like Flash (action software) and QuickTime to run it, three-dimensionality can essentially be simulated.

Any sampling of some of the most innovative uses of 3D imaging and modeling would include the Webby award-winning design of Second Story Interactive, the Monticello Explorer website, which includes the design and home of Thomas Jefferson. Visitors can navigate through a black and white 3D model of this famous home. Entering, one pans each room's contents clicking on architectural details or furnishings to reveal their detailed construction. Among other details, a parquet floor panel reveals the full schematics of its intricate weave of multiple pieces of wood though animation offered by Flash software. Clicking a camera icon reveals a photographic image of specific details. IBM Research's State Hermitage Museum website is another that includes numerous "virtual viewings" of objects. A virtual model of a "mechanical orchestra," an elaborate clock, rotates 360 degrees to reveal side panel doors and the mechanisms within. IBM's Visual Technological Department has developed "graphics, visualization, and image technologies for extracting, conveying, and visually communicating information" about art and historical objects (IBM Research).

Another IBM project was the aforementioned , detailed 3D model of Michelangelo's Pieta which made use of a scanner that measured the shape and appearance of every portion of the original sculpture. This scanned information was merged together to create an amazing virtual object that can be manipulated and examined unlike the real. Moreover, this website allows the viewer to imbed the Pieta in a number of virtual environments that virtually re-create it in its in situ circumstances. Other applications might include an object's point of creation or discovery. This evidences possibilities for new and interesting ways to re-contextualize objects and "help audiences see them and experience them differently" for possibly greater educational impact (Tolva 2005; Freedman 2003). The interactive viewer cannot peruse the whole but virtually dissect it and isolate each of its two figures for study. The more interactive features of this site, and others, only provide simplified virtual versions of the object whereas elements of lmited interaction are of higher resolution. This is also true of magnifying tools extant. Greater magnification of details results in decreased resolution. In the website Eternal Egypt , four directional arrows allow easy 360 degree rotational views of objects included, but virtual objects like the "Throne and Footrest of Tutankhamen" become increasingly fuzzy with use of the "zoom."

Simulated 3D has been around for quite some time. The National Research Council of Canada (NRC) experimented early with what 3D models could contribute to virtual display in Canadian museums (1995). Seven figurines were among the first objects scanned at high resolution in color for the purpose of interactive 3D virtual display. The intention was to allow visitors both a "microscopic" and "stereoscopic" view of the figurines using "active shutter glasses." This technique, intended for in-house computer kiosks, allowed enhancement of details, which were difficult to see on the real object. This use of 3D technology emphasized a potential use as a complementary element to traditional exhibitions and not as a substitute. In addition, the NRC has "designed and tested...3D laser imaging systems and processing algorithms with the aim of improving high-resolution modeling of complex objects and environments" for use in museum environments (NRC).

Earlier than any applications to museums, CAD (Computer-Aided Design) software assisted in the 3D digital rendering of structures and objects. CAT scanners and MRI machines provided 3D modeling of the human body and are well known for the purposes of medical diagnosis. Computer modeling is characterized by a digital wire frame or "mesh" that can be manipulated or allow one to render images, diagrams, or animations. One can "add, subtract [from], and sketch on this mesh. The models can be viewed from a variety of angles, simultaneously, as well as be rotated and viewed from different magnifications. More complicated modeling software, with the desired goal of close virtual reproduction of the real, requires the entering of mathematical coordinates in order to achieve exactitude. The relevancy of such multimedia tools to public interest is evidenced by Google's own purchase of the rights to and offering of the free downloadable software Sketch-Up, an "easy-to-learn" program that allows one to achieve the "conceptual stages of design" through 3D modeling. One of the popular, purchasable programs also available is 3DS MAX 2009 offering a menu of basic forms that can be pulled, prodded, stretched, among other possibilities, to achieve a desired form. The templates are all based on a polygon mesh that once shaped serves as a type of armature for additional characteristics, including texturing. Such a modeling program allows for full motion video, i.e. animation, of the rendered object or environment (Autodesk 2008).

For research, modeling may focus on the "shape and curvature" of real objects like "ceramic vessels, bones, and lithic [tools]." By plotting the measurements of height, width, surface area, and the like of the real on a malleable digital mesh, spatial definitions can be achieved. Such measurements can include even the smallest irregularities of curvature that succeed in creating an exact virtual rendering. This geometric type of modeling can be achieved through scanning the real object, with hardware like Cyberware Scanners, M15 & 3030. Multiple images of the scanned object are superimposed upon each other creating an on-screen three dimensional mesh version of it. Accuracy varies from one scanner to the next, and there is always another better model on the horizon (Rowe 2003;Voltoni 2007). Such accuracy could include "high resolution digital images" not unlike analog photos from the same vantage point," but interactivity is usually only possible at "20 frames per second." The limitation is due to the human brain's ability to detect "latency or jitter" with a greater number of frames as well as "loss of interactivity." Photorealism and smooth navigation are still not plausible "without some compromises" (Voltoni 2007). Not yet anyway.

These developments promise greater safety and preservation of real objects because they offer an alternative to the repeated handling necessitated by conventional exhibition and educational purposes. There is the possibility of effecting reduced labor in the composition of detailed condition reports and descriptions with the existence of 360 degree rotational views of objects through mouse clicks. Such a record would detail more about the object than is usually included in the several paragraphs of written description and the requisite one or two digital stills that comprise many digital collections databases. For the researcher, connoisseur, and the curious alike the visual totality of this virtual version could be made available for examination online. The museum space itself has from its conception made views of objects in their totality prohibitive, if not impossible.

There is urgency for greater consideration of developing technologies like 3D imaging and modeling objects in museum collections. Realizing virtual surrogates for real museum objects is a remedy for the prohibitions necessitated by the need to preserve and protect. Preserving and protecting has indirectly inhibited the learning styles of many in the museum and limited contexts by which objects can be enjoyed and appreciated. 3D imaging and modeling offers not only the best virtual surrogate to the real for the purpose of protecting and preserving it, but it allows a greater offering to the "seven ways of learning," specifically allowing tactile, kinesthetic, spatial experiences not usually options at most museums (Millicent Rogers). In addition, there are endless connections to be made by learners through the tools of multimedia interactivity that would allow greater possibilities for thematic programming and addressing every learning style.

One afternoon not long ago a student viewing the exhibit The Artist's Book as Volume of Knowledge by artist Angela Lorenz at the Addison Gallery of American Art in Andover, MA asked whether it was possible for a "board game" entitled South American Trading Game depicting a world map and the lasting effects of colonialism as its theme, could be removed from the glass case so he could play it. The museum educator leading this student's class held back a snicker and asked his peers why that wasn't possible. One dutifully answered that this was art and that it had been placed in the glass case so that people couldn't handle it, soil it, damage it. The fact you couldn't play it seemed the most obvious restriction, but that, strangely, wasn't verbalized. The fact is that this work was designed as a game with an important lesson to be learned from it. Games by their very definition involve play, but this particular one and others in the exhibit allowed no such interaction. It occurred to me that beyond merely looking at, hearing about, and verbally responding to this art and other works within a traditional museum experience, there are tactile and kinesthetic cues embodied that remain unanswered and are essentially stifled by the need to preserve and protect.

A consolation might be that these works by Lorenz are being seen and are inspiring discussion whereas many museum holdings and artist's creations are rarely, if ever, seeing the light of day, for they are packed away in archival boxes or stored on shelving in undisclosed storage facilities. A case in point is the Boston Children's Museum (BCM)'s greatest kept secret, its own teeming collection of cultural artifacts which largely dates from a time when museums for young visitors sought to create holdings identical to the ones for adults. This museum, and others like it, have dramatically changed their attitude about using museum objects for hands-on learning experiences off-site in area schools as they once did in the 1930s-1950s; subsequently, collections policies have changed regarding the acquisition of new objects because they are still wresting with what to do with what they have. One solution has been to de-accession allowing other institutions the opportunity to use these objections to serve their missions. Much remains though at the Children's Museum with no forseeable plans to get objects out of storage and exhibit them.

Not only has museum practice changed regarding these sometimes extremely valuable objects in some cases, but a revitalized emphasis on hands-on activities and interactive play with expendable and safe facsimiles may keep the sometimes dangerous and precious real McCoys once similarly used for education behind closed doors forever (Schwarzer 2001). Object cataloging, never of high priority, is frustratingly incomplete and sometimes nonexistent, and an effort to create a digital representation of each object is slowly underway, and this labor intensive and time consuming process may offer hope for bringing these objects, at least as a digital image, back into public view again.

What can be done? When asked this question, BCM Collections Manager Lindsay Richardson confided that the resources are not available for creating a complete web-based digital collections database any time soon. She dreams that one day, possibly during her tenure, this large eclectic treasure trove of samurai armor, cigar store Indians, doll houses, stuffed birds, and the like will benefit children through some type of digital interactive play. Collections that contain such artifacts may rarely if ever exhibit much of it. Their care and preservation come at great cost. Their fragility and value make public handling of them again out of the question. Exhibiting and loaning objects, no matter what preventative measures are taken, result in deterioration as well. Given this fact, many museums have become paralyzed by their obligation to preserve falling short of their one-time professed role of educating the public through these objects, but new purpose and function may correct this through the continued development and adoption of digital imaging and multimedia interactive software.

Museums may have their cake and eat it too, for those that have created a sampling of their holdings for the public with virtual objects and exhibitions have consequently limited future handling of the real thing. Digital versions can be viewed, researched, and be part of interactive play through the click of a mouse. Implementation of a variety of 3D imaging and modeling techniques have made these virtual objects publically accessible via website and in-house computer kiosks and an even more appealing surrogate for those who need to touch and manipulate to learn, to appreciate.

While some institutions have embraced the latest 3D imaging and modeling techniques in their offerings, many others have been slow to realize even digital collections databases which provide conventional two dimensional frontal views of each object. Creating 3D is possible with little expertise. It involves mounting a digital camera on a tripod for a series of stills. An object is placed in front of the camera and rotated at regular intervals between exposures. These stills could later be synched together and manipulated using Flash software to achieve the desired effect (Kumar 2007). Creating a virtual tour entails digitally photographing a 360 degree panoramic view of a museum space. A camera is positioned on a tripod in the center of the space and camera stills are taken at 30 degree intervals. This is repeated for "negative and positive pitch angles of 45 degrees" which can be synched together and manipulated relatively easily with satisfactory results. A scenario involving objects and an environment is achieved through a layering of elements.

Cost is often touted as the major obstacle to fully embracing new digital technologies. It is also pointed out that adopting such technology will mean that it will have to be replaced once the next innovation comes along. Given this limitation money could be better spent on something else. There is no denying that online software products and the hiring independent designers for implementing them with state of the art interactives is expensive proposition, but museums with little financial resources could seek sponsorship, pro-bono designers, and programmers who could be compensated through promotional and advertizing benefits. The process of developing and researching a virtual exhibition is equivalent to that of a physical exhibition, but "digital information is cheaper, faster, and more flexible" in the long run. Once the appropriate software has been secured and an expertise with the software has been gained, actual production of the virtual is relatively low in cost and expenditure of
man hours by comparison. It seems the "ideal medium for small museums, allowing them to diversify outreach efforts, experiment with innovative display, and develop a variety of advertizing campaigns." Christian Schickelgruber, Museum fur Volkerkunde, Vienna, claims he was able to develop a virtual exhibit for "approximately a seventh of the cost" of a physical one (Muller 2002).

Some institutions have made agreements with institutions of higher education whereby the digital images of their collection are licensed for educational use and made available through these institution's own databases. This has been a means by which smaller institutions have offset the cost of creating and maintaining digital assets. Such arrangements would likely also fall under the aegis of "fair use."

3D virtual objects and exhibition scenarios would necessitate the maintenance required of any asset, digital or real. Data entry and upkeep are an expense. Ensuring that your system is maintaining your digital assets and their functionality would be of especial concern with 3D interactive offerings. Without functionality and easy navigation such sources are useless. Specialized knowledge of these issues and the ability to diagnose existing problems would be important to collections management staff. When licensing multimedia software for the purpose of creating 3D images and/or offering interactivity, technical support would be a necessity. Depending on how technically savvy or willing your staff is to take on the responsibility of learning to create these assets and maintain them, would determine what portion of such a project would be realized by independent designers.

PEM, for example, contracts designers to help them realize their ideas by telling them what they can do and can't do with the particular content for a planned future virtual exhibition, usually to accompany one in the museum. Much of the construction process of writing code and testing functionality is left to PEM's own staff. This arrangement has evolved over the course of realizing many projects (Forrest 2007).

Some larger institutions have taken the lead in forming partnerships with other institutions to realize such projects, not to mention the sponsorship that often accompanies exhibitions themselves and provides funding for accompanying digital offerings. A partnership between London's Tate and New York's MET is among those that have realized the potential revenues generated by sites which include offerings from their collections virtually realized through cutting-edge technology. Museum websites often have twice the number of visitors as the physical museum, so "retail sales of museum merchandise on these sites is likely to "generate income, possibly substantial shared revenues." Such partnerships have reportedly reaped 6 billion dollars in earnings in recent years (Muller 2002:Reilly 2001).

A digital collections database augmented with 3D images promises quicker retrieval of collections information that would otherwise involve manually locating and removing objects from archival storage materials and subsequent hands-on physical examination. In addition, real objects need to be returned to their previous circumstances after answering inquiries, doing condition reports, or providing details to another institution that wishes to borrow the object for their future exhibition. Limiting handling of the real might be possible in many situations with accessibility to 3D images of them (Buck 18).

Virtual exhibitions place less strain on museum staffs because they are "restricted by time." They usually don't have the urgencies associated with real exhibitions. The virtual exhibition can remain online for extended periods with little maintenance. It can be archived online as well for public access seemingly in perpetuity, and, ideally, "such accessibility could lead to a density of information that might change people's ways of seeing, interpreting, and researching" (Muller 2002). Given the "cardinal rule of a good database" to not enter information more than once, if avoidable, adopting 3D images would ideally entail adding this information to already existing fields within your current database. The labor involved in adopting such a technology would therefore be minimized. This would be dependent on whether your current database can handle such large amounts of digital information.

In this age of heightened concern over carbon emissions and the consumption that fuels it, collections managers can take a leadership role in reducing their museum's environmental impact. Greater emphasis on virtual objects and exhibition may result in a reduction in the number of physical exhibitions which would subsequently result in the use of less packing materials for object exhibition, loans, and travel. It would also reduce the fuel use that comes with such transport (Loiko 2008).

With any type of digital or analog photographic reproduction of a work there may be issues of copyright. Knowing what your institution can and can't do with the objects in your collection is integral to any plan for either creating an online collections database or virtual exhibition. Given that educational purpose is often a museum's chief incentive for both virtual object development and interactvity, it is likely that the obstacles that for-profit institutions regularly face may be avoided through allowances for "fair use" for educational purpose granted to the non-profit museum. Researching the existence of copyright restrictions for your specific objects is always prudent.

Copyright may effect the digital assets themselves. Virtual objects may be considered original creative work, although it is derivative of something real within the collection. These would likely fall under separate copyright. This is evidenced by the precedence of distinctions between rights to real objects within the public domain and a museum's exclusive right to digital reproductions that they make publically available (Malaro 180). Museums should seek copyrights to all its digital assets as well as insure various safeguards, like watermarking, restricted downloading, and an inability to print such assets without license. These safeguards would be more applicable to specifically 3D images because reducing pixilation to insure low resolution images as a deterrent to theft would defeat the quality and therefore desirability of virtual surrogacy ultimately achieved through high resolution and interactivity (183).

The fear by some that museum visitors will eventually forgo the real for the virtual is unfounded. Evidence shows that "virtual versions of works of art increase people's desires to see the real thing" (Muller 2002). There is also no denying the existence of many who are still in denial about the role of digital media to extend amd enrich "the total museum experience." High resolution imagery has its flaws just like the physical museum. One can only concede that one won't replace the other but rather serve to complement each other (Cooper 2006). Some argue virtual objects, or what they call "digital-originals," have "the maximum possible likelihood of retaining all meaningful and relevant aspects of the original"; therefore, they will function "in just the same way as the original" (Smith 2000). The virtual does allow interaction that is prohibitive with the original , but the virtual also has a dependency on functioning, hardware, a network, and expertise that becomes an additional responsibility of collections management. The virtual isn't simply better than the real. A collection of appealing real objects is always needed to serve as the basis for any type of virtual reality a museum hopes to offer.


Autodesk 3ds Max. 9 April 2008. Accessed 4.11.08

Buck, Rebecca A. and jean Allman Gilmore. The New Museum Registration Methods. Washington, DC: American Association of Museums, 1998.

Cooper, Jonathan. Archives & Museum Informatics. Museums and the Web 2006 Papers: Beyond the Online Museum: Participatory Virtual Exhibitions. March 22-25, 2006, Albuquerque, NM. Accessed 3.28.08

Eternal Egypt. Accessed 3.28.08

Forrest, Jim, Web Creative Director, Peabody Essex Museum, Salem, MA, Guest Class Lecturer, Museums and New Media, Tufts University, 18 September 2007.

Free Library of Philadelphia. "Medieval Manuscripts." Digital Collection of Medieval and Renaissance Manuscripts. Accessed 3.28.08

Freedman, Michael. Archives & Museum Informatics. Papers: Museums and the Web 2003. "Think Different: Combining Online Exhibitions and Offline Components to Gain New Understanding of Museum Permanent Collections." Accessed 3.28.08

IBM Research. "Visual Technologies Department." 2004. Accessed 3.22.08

IBM Research. "Pieta." 2004 Accessed 3.22.08

Image Dtabases. Medieval Studies Program Library Resources. "Digital Scriptorium." http://www.library.u/

Johnson, Brad. Archives & Museum Informatics. Papers: Museums and the Web 2003. "Disintegration and the Museum Web experience: Database or Documentary-Which Way Should We Go? Accessed 3.20.08

Kumar, Vipan. "Making of a Virtualized Museum." (May 2007) Available at SSRN: Accessed 3.30.08

Lahanier, Christian, Genevieve Aitken, and Ruven Pillay. "Two dimensional multi-spectral digitization and three-dimensional modeling of easel paintings, pp. 30-42. ICOM-CC Triennial Preprints Documentation, ICOM-CC International Council of Museums/Committee of Conservation. Accessed 3.20.08

Lahanier, Christian, Denis Pitzalis, Oliver Feihl, Micheline Jeanlin, Francis Schmit. "Three-Dimensional Modeling for Archeological Objects for Conservation, Visualization, Color and Shape Characterization; Comparison of Details," pp. 43-51. ICOM-CC Triennial Preprints Documentation, ICOM-CC International Council of Museums/Committee of Conservation. Accessed 4.8.08

Loiko, Pat, Head Registrar, Museum of Fine Arts, Boston, MA, Guest Lecturer, Tufts University, 21 February 2008.

Malaro, Marie C. A Legal Primer on Managing Museum Collections. Washington, DC: Smithsonian Books, 1998.

Millicent Rogers Museum of Northern New Mexico. "Seven Ways of Learning." Squash Blossom & Stars. Accessed 4.20.08

Explorer Monticello. Accessed 4.10.08

Muller, Klaus. "Going Global: Reaching Out for the Online Visitor." Museum News, Sept./Oct. 2002, Vol.81, No.5.

Museum of Fine Arts. Accessed 4.8.08

National Research Council/Canada. "3D Virtual Museum Display and Applications." Accessed 3.28.08

Peabody Essex Museum. Joseph Cornell; Navigating the Imagination. Accessed 4.2.08

Peabody Essex Museum. Art and Culture Center. Origami Now! Accessed 4.2.08

Reilly Jr., Bernard F. "Merging and Diverging: New International Business Models from the Web." Museum News, Jan./Feb. 2001, Vol.80, No.1.

Rowe, Jeremy, and Anshuman Razdan. Archives & Museum Informatics. Papers/Museums and the Web 2003; A Prototype Digital Library for 3D Collections: Tools to capture, Model, Analyze, Query Complex 3D Data. Accessed 4.13.08

Schwarzer. Marjorie. "Art & Gadgetry: the Future of the Museum." Museum News, July/Aug. 2001, Vol. 80, No.4.

Smith, Abby. "Authenticity in Perspective." Council on Library & Information resources (CLIR). 25 May 2000. Accessed 4.13.08

van Dijk, Dick. Archives & Museum Informatics. Papers/Museums and the Web 2007: Operation Sigismund; Bringing An Archive Into Play.

Voltoni, F., and A. Beraldin, S. El-Hakim, and L.Gonzo. "Photorealistic 3D Modeling Applied to Cultural Heritage." Stock, Oliver, and Massimo Zancanaro, Eds. PEACH-Intelligent interfaces for Museum Visits, NY: Springer, 2007.