SKETCHING IN SPACE

Sketching in Space is an ongoing, interdisciplinary project. It started in 2007 as part of a doctoral thesis by Johann Habakuk Israel at the Fraunhofer IPK Berlin, Division Virtual Product Creation, and the graduate school prometei of the TU Berlin. The thesis focussed on usability aspects, the empirical investigation and possible applications of immersive sketching in product design. Christian Zöllner developed physical tools (i.e. tangible interfaces) for interacting with the immersive sketching system. A wireless version of the tools was recently developed by the designers Sebastian Piatza and Alexander Müller, and the computer scientist and human factors expert Achim Meyer. Various artists, scientists and students from the fields of computer science, product design, human factors, media art, psychology, and engineering have also worked in the project. This project contributed to the large body of research on immersive modelling which was previously conducted in the fields of computer science, engineering and psychology.

The aim of Sketching in Space is to explore the possibilities of free-hand sketching and modelling in three-dimensional space. The developed system allows for creating strokes, sketches, drawings and shapes in a virtual environment, using physical tools (i.e. tangible interfaces) such as pens, bi-manual Bezier-modelling tools and pliers. The sketching system runs in a Virtual Reality-Cave, a cubical installation with five projection planes of 2.5 meter edge length each. Using stereoscopic projection, the systems allows for perceiving a sketch as floating in space, directly where it was created.

Immersive sketching can be regarded as a new design tool which affords new creative and aesthetic possibilities. It allows users to perceive themselves and their sketches in the same physical space and to experience the sketch in relation to their own body. Furthermore by creating large sketches in one-to-one scale and walking around once own models, the whole body is involved and the process is turned into an interactive experience. Users can also apply existing motor skills for free-hand sketching; they are not bound to modelling functions and interaction devices of desktop-based CAD-tools.

During the project, various studies were conducted in order to understand the benefits and drawbacks of immersive modelling for design. The initial system and interaction tools were developed following an intensive requirement analysis (Israel et al., 2009; Israel, 2010) which revealed that potential users expect improvements in the design process due to the one-to-one-scale of the visualisation, the three-dimensionality and the possibility to interact with once own sketches and ideas. Corresponding results were found in further comparative studies on 2D- and 3D-sketching . The studies also revealed that drawing of fine details and connecting sketch elements is difficult and requires training, even though the learning curve is steep.

Special attention was given to the design of the physical modelling tools as it turned out to be of great importance for the acceptance of the entire system. Beside aesthetical requirements the tools had to fit into hands of various sizes and to allow for fine and voluminous modelling movements which in turn require precision and power grips. The revised versions of the tools also integrated sensors for changing the digital function applied to the virtual tool, e.g. to change the function of the pen from a pencil to an eraser etc.

Since the first system was implemented, various case studies were conducted which embedded immersive sketching into exemplary product development process (cf. Wiese et al., 2009). In a joint project with the Berlin University of the Arts in 2009 students developed ceiling lamps using immersive sketching, desktop CAD systems and rapid prototyping for the physical creation of the models. It turned out that the participants kept parts of their original free-hand in the final design, smoothening, modifying or replicating others and creating additional elements in CAD. Other joint case studies e.g. with the Muthesius Academy of Fine Arts and Design Kiel in 2011 demonstrated the unique properties and stimulating effects of immersive sketching for creative design processes The studies also revealed that immersive sketching is not a stand-alone tool but needs a balanced interplay with common design tools such as pen-and-paper, CAD and physical prototypes in order to improve the design process substantially.

Israel, J. H. (2010) Hybride Interaktionstechniken des immersiven Skizzierens in frühen Phasen der Produktentwicklung. Dissertation, Berlin: Technische Universität Berlin.

Israel, J. H., Wiese, E., Mateescu, M. & Stark, R. (2009) Investigating three-dimensional sketching for early conceptual design — Results from expert discussions and user studies. Computers & Graphics 33(4), 462-473.

Stark, R., Israel, J. H., and Wöhler, T. (2010). Towards hybrid modelling environments — Merging desktop-CAD and virtual reality-technologiess. CIRP Annals – Manufacturing Technology 2010(59), 179-182.

Wiese, E., Israel, J. H., Meyer, A., and Bongartz, S. (2010). Investigating the Learnability of Immersive Free-Hand Sketching. In: Proc. ACM SIGGRAPH/Eurographics Symposium on Sketch-Based Interfaces and Modeling SBIM’10, Annecy, France, ACM SIGGRAPH and the Eurographics Association. 135-142.

Wiese, E., Israel, J. H., Zöllner, C., Pohlmeyer, A. E. & Stark, R. (2009) The Potential of Immersive 3D-Sketching Environments for Design Problem-Solving. In: Proc. 13th International Conference on Human-Computer Interaction HCI 2009. San Diego, CA, USA.

The system was set up in a VR-Cave, an immersive virtual environment with five rear-projected walls of 2.5m edge length each, employing active stereo projection and shutter glasses. Both the user and the interaction devices were tracked using optic, formerly magnetic tracking methods. The tracking data were smoothened in order to reduce noise and to allow the drawing of straight lines by hand.

The developed system allowed for creating strokes, sketches, drawings and shapes in a virtual environment, using physical tools (i.e. tangible interfaces) such as pens, bi-manual Bezier-modelling tools and pliers. Drawing with virtual ink with the pen was possible by gently pressing at the upper side of the pen. Virtual ink was drawn from the tip of the pen directly into the virtual environment, following the movements of the tool (and the user’s hand). The width of the virtual ink drawn in the form of a plain band was scaled from 1 to 8mm according to the force by which the user pressed the upper component. Releasing the upper side of the pen stopped the drawing. The Bezier-tool allowed for creating surfaces from Bezier-curve-profiles. The Bezier-curves were controlled by two physical handles hold in the user’s hand. Moving the handles changed the control points and the shape of the Bezier-curve. Pressing and moving the handles let to the creation of shapes along the movement path. By touching virtual elements of the sketch with the physical pliers it was possible to freely move and re-arrange them in the virtual scene. A simple physical slider device allowed both sequentially undoing strokes and replaying the creation process of the sketch.

For the rapid prototyping of the final models, e.g. lamps and sculptures, Selective Laser Sintering SLS and ABS printing was employed.