Motion capture, motion tracking, or ‘mocap’ are terms used to describe the process of recording movement and translating that movement into a digital model. Our project explores the use of analogue sensors as control input data for a digital, animated model. The aim is to define a connection between the virtual world and the digital world by investigating new and intuitive methods of communication between user and computer.
TECHNOLOGY
Maya was used as a tool to create a virtual environment. The MEL scripting language has allowed the development of complex and even unpredictable animation output. Arduino was used as an interface between computer software and the physical environment. It is both an input and an output device simultaneously. It has the ability to capture both analogue and digital input readings from a series of sensors and can be used to drive motors, lights, and other actuators. Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It is widely used by artists, designers, hobbyists, and those interested in the creation of interactive objects and environments. Arduino’s micro controller is programmed using the Arduino programming language (based on Wiring) and the Arduino development environment (based on Processing). Arduino projects can be stand-alone or they can communicate directly with software such as Maya, Flash, Max/MSP and Processing.
As the means to extract analogue data from the physical world, the project uses Sharp proximity sensors. These sensors consist of two LEDs, one which emits infrared light and another which receives the reflected beam. Using triangulation, the sensor is then able to calculate the distance an object is from itself. Their operational range is 4cm — 30cm.
TESTING THE SENSORS
Initial sensor testing began to ensure proper communication with Maya. A series of simple animations were created in which we mapped sensor inputs to cluster points, allowing us to manipulate a lofted surface through physical interaction with the sensors.
BUILDING THE DEVICE
After refining our initial experimentation with the simple animation, we decided to place the sensors in a more controllable environment. This would allow us to have greater control over the animation, giving us a more accurate set of outputs. For our first prototype we decided to create a box that controlled our three proximity sensors with greater accuracy. The box consists of two sheets of plywood fixed to 4 batons, forming 3 chambers. The proximity sensors are housed at the end of each chamber. The length of the chambers correspond with the range of the sensors (40mm-300mm). Each chamber is fitted with a plunger. Each plunger has an end pad with dimensions just smaller than that of the inside of the chambers. The aim is to remove as much interference from the sensor readings as possible by providing constant, flat surfaces for the infrared beams to reflect off. The box needed to be adapted to allow for the wiring from the sensors to connect to the Arduino board more easily. This meant that modification of the original design needed to be implemented immediately to allow for testing.
TESTING THE DEVICE
Once communication was established between the sensors and Maya, the box underwent initial testing. A series of short videos were made demonstrating the sensor effects upon a digital model when the values were mapped to various attributes.
REFINING THE DEVICE AND CONCEPT
During discussions and testing of the initial device, it became evident that an underlying theme of modularity was rooted in the project. To increase usability and adaptability it was decided that each sensor should be encased in its own individual piston. Since the device uses only one type of sensor, its potential use could be somewhat restricted, but by allowing each individual sensor a higher degree of versatility in its own, independent operation, the device would be opened up to an array of uses. The refined version of the device is a series of individual piston tubes, containing sensors. The tubes are designed to the dimensions of a grid. They are transformable and free to adopt the most intuitive arrangement of operation. This means that firstly, the device is non-specific to an animation, it can be rearranged to suit any, and secondly, that the device is extremely user friendly. The user decides on the arrangement of the pistons that suits them for the specific animation they are operating. The piston casings have a series of male components along one of the longer sides and a series of female components along the other three. This allows pistons to be locked together into a certain arrangement when in use. There is also a baseplate containing female components so as to allow all pistons to be securely fixed down. The device was designed using AutoCAD and the individual components laid out as a net. The design was then ran through the cutter onto a series of plywood boards. The laser cutter can cut and etch material up to 4mm in thickness. The plywood used in the second device is 4mm thick to improve strength, sturdiness and durability. The device took around an hour in total to cut. The laser cutter was used to enhance accuracy in the device, for speed of manufacture, and for its general aesthetic appeal.
ASSEMBLING THE REFINED DEVICE
Once the components were cut out, the piston casings had to be assembled. Each piece of ply was cut with a series of teeth running along its edges, meaning that pieces could be easily and securely slotted together. Wood glue was used to hold the boxes together and to secure the sensors to the end plate. The sensors had to be modified to fit securely into the casings, and a small hole was cut into the end plate to allow the sensor wires to exit the boxes. The male components were fitted into the etched square sockets to create the interlocking modular system. As part of the interaction between the user and the animation, we needed to develop the physical interaction. The first part of this development of the physical experience of our animation involved creating a board in which to mount each box. This allowed for a great deal of flexibility, as it meant that each box could be reconfigured to suit the specific animation and provide the user with the most intuitive method of interaction.
The decision to use the laser cutter proved very beneficial, the device being constructed quickly and easily. Certain initial problems did occur however, possibly due to the laser cutter being too accurate. For instance, the end plates at times were becoming stuck in the inside of the casing and detaching from the piston handles. The solution was to simply sand down the offending edges, but in retrospect there should have been 1mm or so of tolerance allowed for the internal components. The general look and feel of the pistons proved very successful. Operation was smooth and easy after some slight modification and the device felt sturdy when in use.
BASIC ANIMATIONS
The next stage was to trial our new device. This began with the manipulation of series of simple, 2-dimensional surfaces by mapping a series of cluster points to the sensor inputs. Early animations involved simply attaching a cluster point to each corner of a lofted surface before mapping them to sensor input value. The next stage saw a surface divided into a 5×5 grid. A cluster point was assigned to each intersection point on the grid, and this cluster had its Y-translation randomly mapped to one of six sensor input pistons.
RE-CALIBRATING THE SENSORS
It quickly became obvious that simply containing the sensors within a casing did not give us the results we had hoped for. The initial idea was that a cluster point was selected depending on how much the piston was pushed in. The obvious way of interacting would be that pushing the piston half way in would select the cluster point half way along an edge. The problem, however, was that the sensors do not give a reading proportional to the distance that an object is from it. That is to say that if when the piston is at its fully open position, the value reads 0, and if when it is in its fully closed position, the value reads 100, the reading will not necessarily read 50 when the piston is in the half way position. If mapped onto a graph, plotting sensor reading values against the distance that the piston is pushed into the casing would result in an exponential curve. This proved a major problem and led to a radical rethink of the method of scripting we used to achieve what we wanted
In order to recalibrate each sensor, we needed to adjust the corresponding sensor value graph in Maya. The graphs on the left are attempts to adjust how the sensor input can be remapped to the output value across the range. This required a lot of experimentation with the graphs to try to counteract any anomalies from the sensors, to produce a constant set of results that were proportional to the actual distances of the plunger along the piston. Each sensor produced different anomalies and so a different graph was needed for each.
After much experimenting with the sensor value graphs, we achieved the best set of results possible, the selection of cluster points relating fairly well to the amount the piston was pushed. The graphs to the left show a series of graphs that produced a reasonably proportional set of input and output values. It became apparent that the main problems were with the lower end of the sensors’ range. This is evident in the graphs, as they all needed to be significantly lowered at one end to counteract any inaccuracies with the sensors.
ADVANCED ANIMATIONS
By recalibrating the sensors we now had far greater control over our animations. This allowed us to begin developing a series of animations which better reflected the full potential of our device. We introduced a digital representation of our piston into an animation to better understand the relationship between the physical and the digital models. We continued to use the theme of the lofted surface, which we deformed with cluster points. The pistons allow the user to interact with and deform a surface in a virtual world via physical instruments in an intuitive manner. This aims to replicate how a physical surface could be deformed in the real world.
In order for us to progress our animation we had to take a different approach. Moving away from the sensor graph we introduced ‘If’ statements. Unforeseen problems began to develop with the calibration of the sensors and we eventually had to settle on a revised method of mapping the device to our digital model. We split up the range of the sensors into slots which could be mapped to certain points of selection. The script consisted of a series of ‘if’ statements, for example ‘if sensor reading is between X and Y, select point Z’. If statements allowed us to define where on the surface the point of manipulation takes place. By assigning a sensor to each of the axis in the animation we were able to control which cluster was controlled. This allowed for greater flexibility in subsequent animations. We continued to use the theme of the lofted surface, which we deformed with cluster points. The pistons allow the user to interact with and deform a surface in a virtual world via physical instruments in an intuitive manner.
FINAL ANIMATIONS
For our final animation we took what we had learnt from the previous work and applied it to a more refined method of interaction. Initially, we worked hard to harness the accuracy of our device into an animation which could be precisely and intuitively controlled.
1.
A modular physical interface which is adaptable and changeable in order to suit the requirements of the user and the nature of the animation. The natural tendency for this type of project would be to develop an animation and then design the physical device to operate and interact with this animation. However we considered this approach to be quite restrictive and therefore decided to create a device that would be able to adapt to various animations. This means that the physical interface is not confined to one particular animation type. The practical approach to this concept was to devise a modular system that allowed for maximum flexibility through simple physical transformation. To allow for uniform results across all the physical devices, we opted to use the same sensor type, proximity sensors. Due to the nature of the range of reading of the sensors, it was essential to be able to control each input. This is why we made the box and plunger devices. As the next logical step in our development of the cluster deformed surfaces and in keeping with the form of a modular physical device, we developed an animation based on a series of cubes. These cubes are restricted to a 3D grid. The initial state of the animation is a 4 x 4 grid, all with ‘0’ Y translation. A plunger is assigned to select a cube in both the X and Z axis and a further plunger is used to translate the selected cube in the Y axis. A change of colour from blue to red denotes which cube is active. After testing we introduced a further plunger to prevent unwanted readings being assigned to cubes whilst navigating the space.
2.
Experimentation with the intuitiveness of the device and introducing randomness. The initial animation proved to be very effective. The blocks could be accurately controlled and translated, with the red indicator cube allowing the user to easily see the specific cube they were modifying. The problem was that the animation was rather mundane and monotonous. Our device, although incredibly accurate and intuitive to operate, was also rather complex, and there was an obvious interest in this complex system being used to control a relatively simple animation. We decided to remove some of the predictability of the device. To create this 2nd animation, which is clearly more abstract than the 1st, we removed the red selector box and randomly positioned the pistons, making it very hard for the operator to know which box they were manipulating with each piston. The result sees the operator either naturally struggle to learn which piston is associated to which manipulation by a process of trial and error, or else become completely content in randomly pushing pistons and enjoying the unexpected results that occur. We also began to introduce transparency and reflection into our animation, as well as a panning camera, all of which allow a greater understanding of how the animation is developing in 3D, and increase the general aesthetic appeal. Giving others the chance to try our device alerted us to the joy of its use as a tool to create, rather than simply a tool to control.
3.
Turning our concept on its head — removing the predictability of the device. This new line of thinking proved to be exciting and we explored the idea of turning our concept on its head yet further. Our initial animations were incredibly predictable, we had calibrated and smoothed all the life out of our sensors. We were basically using our analogue sensors as digital ones, giving us 1’s or 0’s, yes or no. We were essentially restricting our device and not exploring its full potential. The theme began to move towards abstraction. Not about how precisely our device could control an animation, but about how the device could be used within a seemingly random and counter-intuitive system. In animation 3, we have introduced further modification options such as rotate and scale. These modifications, as well as the translate function, are mapped to various boxes in a complex system which is almost impossible to understand when the animation is in use. Modifications of boxes are now depended on different modifications of different boxes. For instance, pushing piston 1 may rotate box A by X amount, and in turn trigger the scale of box B by Y amout and the translation of box C by Z amount. The results are wholly more abstract and certainly more interesting. Embracing the analogue nature of our sensors down to removing their value-smoothing capacitors results in a more entropic outcome to our previous attempts, where we enforced digital precision onto our device.
4.
Enhancing the animation and further liberating the sensors. Further experimentation was done with the animation, the scale factor was reduced to help add some visual density and the rotational factor was also increased. The camera angle was reduced to enhance the abstract nature of the animation.
5.
Introducing colour. A colour mapping has been introduced to the animation so that when each piston generates a certain value, the respective box has a blinn shading applied in one of the primary colours [100% red, 100% blue or 100% green]. Whereas in previous animations, each modification [scale, rotate, translate] only applied to a single axis of each cube, here the cubes can be manipulated in all 3 axis.
6.
The device becomes a tool to create and enjoy rather than to control. In the final incarnation of the animation our device is entirely free to create. The user still has precise control over the device, yet there is almost no sense of predictability, and so every action generates a new, unexpected result. The exact role of each piston is now not important, all 6 operate as part of a coherent system, although different results are achieved depending on the number of pistons in use. The transparency of each cube has been reduced to 50%, so there is an added bonus of the introduction of colour theory. At the points that the primary colours overlap, secondary colours are generated. Where secondary colours overlap with primary colours, tertiary colours are generated.
The project was a series of ups and downs, whereas the process of building and refining a physical device was extremely rewarding, the process of interaction with Maya was slow and painfully laborious. We found Maya to be an unintuitive, difficult piece of software to operate, with the most completely ridiculous naming conventions for tools! Despite the numerous difficulties we experienced throughout the project, we are very happy with the results.
Further documentation of the project is available on my Flickr account, as well as my Vimeo account.
Posted in MA Applied Animation | No Comments | Bookmark or Share
Well the Flash animation is finally finished, take a look here – Glimpses of the Spectacular
Below is a recording of the animation.
Glimpses of the Spectacular from Richard Almond on Vimeo.
Recording of the final Flash animation.
http://blog.rafolio.co.uk
Posted in MA Site + Motion | 4 Comments | Bookmark or Share
PLAN
DETAIL SECTIONS
LONG SECTION
Posted in MA Site + Motion | No Comments | Bookmark or Share
I’ve added a little texture and detail to the initial drawings. Hopefully these further demonstrate the intended atmosphere generated by the installation.
PLAN
ELEVATION
Posted in MA Site + Motion | No Comments | Bookmark or Share
The installation comprises an elevated viewing platform, pentagonal in plan, with projections of the 5 main timeline images located on screens along the perimeter edges. The screens are positioned over head height and their top edges are angled inward at 45 degrees. The space will be enclosed around its perimeter, with the projection screens acting like an angled ceiling. This, and the long journey to the installation via a stair represent the studio space and in particular the stairwell. Users feel confined and even oppressed beneath the screens which represent the looming rooflights. In its physical incarnation, the navigational aspect of the main images will not be required, since the user will be free to move between screens as they desire. The triggered animations, however need more thought. The screens will be hung on counter-weighted pulley systems similar to those found above theatre stages. Certain actions by the user will trigger the movement of the main screens, allowing glimpses of those special images which are located on screens behind the main screens. The counter weight system means that these special images, the ‘glimpses of the spectacular’ are only visible briefly, and the method to trigger these images depends on the screen that is being viewed.
Image 1 – the primary method of physical interaction between the user and the animation will be a series of pull cords, the use of which is reminiscent of operating blinds. The first image, like most others, has passive, non-triggered animation, but there are also aspects which will require human interaction. The shadow animations emanating from the lower window panes will be activated by a series of these pull cords.
Image 2/ Image 7 – there will be a section of the screen cut away which corresponds to the gap in the lower window panes. The user will be aware that there is something behind the screen and so move closer to the gap, from which they will be able to peer through to the next image, located on another screen behind the first. As the user moves forward towards the screen containing image 2, they will stand on a pressure sensor which will trigger the upward movement of the screen via the pulley system. This will reveal image 7, albeit only for a short time before the screen containing image 2 returns back to its original position, denying the user more than a glimpse.
Image 3 – both of the fluorescent light buttons will again be activated by pull cords, but image 3 requires a more complex, user-defined positioning method of interaction to control the smoke/stars effect. The device to achieve this effect will be a laser pointer embedded into a torch, which the user will pick up and point at the screen. Using a method similar to that used in the Laser Tagging project by Graffiti Research Lab, the user will be able to draw clouds or stars behind the glass, depending on whether the image is in day or night mode.
Image 4/Image 6 – a series of four pull cords will be connected to the slabs of snow which cover the central 4 rooflights. These cords will be looped, similar to those found on Venetian blinds, and the amount of snow seen on the glass will be directly related to the amount that the cords are pulled. When the user has pulled all 4 cords enough so that the glass is clear, the pulley system is triggered and the screen containing image 4 is lifter upwards, revealing image 6. Image 6 will be the opposite to the web-based version of the animation, and will initially be fully saturated. Drawn towards the vibrant colour, the user will be encouraged to move forward to explore the image in more detail, but as they move closer to the screen the image will become de-saturated, proximity sensors controlling the saturation based on the user’s distance from the screen. Again, the user is denied much more than a glimpse of this special image, and again after a short while the screen containing image 4 drops back into place.
Image 5/Image 8 – pull cords will trigger the opening and closing of the blinds within the animation, when fully closed, the screen containing image 5 will lift, temporarily revealing image 8.
Posted in MA Site + Motion | No Comments | Bookmark or Share
Proposal drawings of the physical incarnation of the animation.
PLAN
ELEVATION
Posted in MA Site + Motion | No Comments | Bookmark or Share
Now that the animation is beginning to come together, I felt it necessary to define a graphical style for any text, instructions, descriptions, etc. Below is the splash page for the animation, which will be embedded into a customized html page upon completion. The splash page contains a set of isolated greyscale window panes which again relate back to the Constructivist imagery, and the script typeface is in keeping with the poetical atmosphere of the piece.
Posted in MA Site + Motion | No Comments | Bookmark or Share
Image 8 is the last ‘Glimpse of the Spectacular’ and accessed through Image 5. Another very simple yet quite effective animation, cool rays of light from a winter sunset glisten between low clouds as the day sinks into dusk.
The effect is achieved by creating a star-like white vector shape which is blurred and has a blend mode applied to it. This vector is then tweened to simulate the subtle flickering of the rays of sunlight. Again the scene reveals itself only briefly, before again disappearing.
Below is the final version. Please note that the transition buttons to other images have been disabled for preview purposes.
Posted in MA Site + Motion | No Comments | Bookmark or Share
Image 6 is the 2nd of the ‘Glimpse’ images, and is located through Image 4. When the glass in Image 4 is clear of snow, the transition to Image 6 is triggered. The scene is the simplest of all, yet probably the most effective. A dull, grey view of a rooflight appears, there is no interaction from the user and the scene seems to have little interest. The user soon notices however that the image has become vibrant and colourful, with bright yellows and blues glowing through the dirty glass. A burst of sunlight has illuminated the rooflight. The panes appear like thin, opaque slabs of brilliant marble, just for a second, before again the image blends back to its initial gloomy grey.
The method for creating this scene was very simple, consisting of two versions of the same image, one full saturation, and one with a very low saturation. The saturated image is arranged above the de-saturated version, and a very slow alpha tween is applied, from 0% to 100% and back to 0%. Because of the pace of the tween, the user often does not realize the transition is occurring until the image is at full saturation, and they remember how grey it was initially. As with all of these “Glimpses of the Spectacular”, the view disappears as suddenly as it appeared. The user is limited to a brief interaction, a brief pleasure.
The final version is below. Please note that the transition buttons to other images have been disabled for preview purposes.
Posted in MA Site + Motion | No Comments | Bookmark or Share
Image 7 is the first of the feature images, the first ‘Glimpse of the Spectacular’. The image is reached by clicking the small gap between the lower window panes in Image 2. It is immediately apparent that this image is different to those the user has viewed up to this point. Intrigued by the glow piercing the gap in Image 2, the user wonders what is behind this gap, and when clicked they are offered a glimpse through. A vibrant, warm sunset squeezes between the misaligned panes. The user’s mouse becomes their eyes any they are able to pan around the image of the sunset, exploring.
However close the user feels to the sunset, streaks across the glass remind them that there is still a layer separating them from what they desire, and as quickly as this ‘Glimpse of the Spectacular’ appears to the user, it is taken away from them again.
To create this animation, a zoomed-in image of the gap in the window panes was blown up to fill the scene. This was blurred to achieve the effect of being in the foreground and to simulate depth. Behind this, an over-sized image of a sunset was taken, and ActionScript applied to allows it to move in the opposite direction to the position of the mouse, relative to the centre point.
Below is the final version. Please note that the transition buttons to other images have been disabled for preview purposes.
Posted in MA Site + Motion | No Comments | Bookmark or Share
