Rafolio.co.uk

I put together a detailed diagram to help clarify how the MaxMSP patch for the installation will operate. The key to success will be the timeframe on which the patch operates. I plan to have various options for the total running time of the patch [6 days, 6 hours, 60 minutes], to allow for the different circumstances in which the installation may be used. Each of these timeframes will be divided into 6, equal parts, 6 x 1 day, 6 x 1 hour, 6 x 10 minutes, etc.

The HSFlow patch will run throughout the duration of the installation, data moshing the live camera feed, but during each of the 6 sections of the installation its effect will be different, due to a different range of presets being used. The effect initially will be subtle, becoming more and more abstract as each section of the installation passes.

A jitter matrix patch will also run throughout, sensing the presence of visitors by registering a disturbance in the pixel field. This will trigger a recording [the time of which depends on the total length of the installation and available storage space], which will be stored under a numerical name.

These recordings are subsequently overlaid back into the live feed at random in progressively increasing quantities. In the first section, one part in 6 will see a recording overlaid onto the live feed, in the second section, 2 parts in 6 will be previous recordings, in the 3rd section 3 parts in 6, etc.

A similar system will apply to images, which be grabbed from a harddrive or an internet server/flickr page and overlaid into the live feed.

Posted in MA Thesis | No Comments | Bookmark or Share

This MaxMSP Jitter patch by Andrew Benson simulates the effect of Data Moshing in real time. I have been doing a fair bit of experimentation with the patch and feel aspects of it can definitely be used in my animation. There are a series of variables which can be modified and with the simple shifting of certain sliders, the aesthetic of the effect completely changes. The patch can modify the live camera feed very subtly or it can morph it completely to the point of total abstraction. MaxMSP has a preset function which allows the storage of a range of variables, so the effect could be made to progressively decay the live feed, a simple counter triggering each range of presets as required.

Posted in MA Thesis | No Comments | Bookmark or Share

Motion capture, motion tracking, or ‘mocap’ are terms used to describe the process of recording movement and translating that movement into a digital model. Our project explores the use of analogue sensors as control input data for a digital, animated model. The aim is to define a connection between the virtual world and the digital world by investigating new and intuitive methods of communication between user and computer.

TECHNOLOGY

Maya was used as a tool to create a virtual environment. The MEL scripting language has allowed the development of complex and even unpredictable animation output. Arduino was used as an interface between computer software and the physical environment. It is both an input and an output device simultaneously. It has the ability to capture both analogue and digital input readings from a series of sensors and can be used to drive motors, lights, and other actuators. Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It is widely used by artists, designers, hobbyists, and those interested in the creation of interactive objects and environments. Arduino’s micro controller is programmed using the Arduino programming language (based on Wiring) and the Arduino development environment (based on Processing). Arduino projects can be stand-alone or they can communicate directly with software such as Maya, Flash, Max/MSP and Processing.

As the means to extract analogue data from the physical world, the project uses Sharp proximity sensors. These sensors consist of two LEDs, one which emits infrared light and another which receives the reflected beam. Using triangulation, the sensor is then able to calculate the distance an object is from itself. Their operational range is 4cm — 30cm.

TESTING THE SENSORS

Initial sensor testing began to ensure proper communication with Maya. A series of simple animations were created in which we mapped sensor inputs to cluster points, allowing us to manipulate a lofted surface through physical interaction with the sensors.

BUILDING THE DEVICE

After refining our initial experimentation with the simple animation, we decided to place the sensors in a more controllable environment. This would allow us to have greater control over the animation, giving us a more accurate set of outputs. For our first prototype we decided to create a box that controlled our three proximity sensors with greater accuracy. The box consists of two sheets of plywood fixed to 4 batons, forming 3 chambers. The proximity sensors are housed at the end of each chamber. The length of the chambers correspond with the range of the sensors (40mm-300mm). Each chamber is fitted with a plunger. Each plunger has an end pad with dimensions just smaller than that of the inside of the chambers. The aim is to remove as much interference from the sensor readings as possible by providing constant, flat surfaces for the infrared beams to reflect off. The box needed to be adapted to allow for the wiring from the sensors to connect to the Arduino board more easily. This meant that modification of the original design needed to be implemented immediately to allow for testing.

TESTING THE DEVICE

Once communication was established between the sensors and Maya, the box underwent initial testing. A series of short videos were made demonstrating the sensor effects upon a digital model when the values were mapped to various attributes.

REFINING THE DEVICE AND CONCEPT

During discussions and testing of the initial device, it became evident that an underlying theme of modularity was rooted in the project. To increase usability and adaptability it was decided that each sensor should be encased in its own individual piston. Since the device uses only one type of sensor, its potential use could be somewhat restricted, but by allowing each individual sensor a higher degree of versatility in its own, independent operation, the device would be opened up to an array of uses. The refined version of the device is a series of individual piston tubes, containing sensors. The tubes are designed to the dimensions of a grid. They are transformable and free to adopt the most intuitive arrangement of operation. This means that firstly, the device is non-specific to an animation, it can be rearranged to suit any, and secondly, that the device is extremely user friendly. The user decides on the arrangement of the pistons that suits them for the specific animation they are operating. The piston casings have a series of male components along one of the longer sides and a series of female components along the other three. This allows pistons to be locked together into a certain arrangement when in use. There is also a baseplate containing female components so as to allow all pistons to be securely fixed down. The device was designed using AutoCAD and the individual components laid out as a net. The design was then ran through the cutter onto a series of plywood boards. The laser cutter can cut and etch material up to 4mm in thickness. The plywood used in the second device is 4mm thick to improve strength, sturdiness and durability. The device took around an hour in total to cut. The laser cutter was used to enhance accuracy in the device, for speed of manufacture, and for its general aesthetic appeal.

ASSEMBLING THE REFINED DEVICE

Once the components were cut out, the piston casings had to be assembled. Each piece of ply was cut with a series of teeth running along its edges, meaning that pieces could be easily and securely slotted together. Wood glue was used to hold the boxes together and to secure the sensors to the end plate. The sensors had to be modified to fit securely into the casings, and a small hole was cut into the end plate to allow the sensor wires to exit the boxes. The male components were fitted into the etched square sockets to create the interlocking modular system. As part of the interaction between the user and the animation, we needed to develop the physical interaction. The first part of this development of the physical experience of our animation involved creating a board in which to mount each box. This allowed for a great deal of flexibility, as it meant that each box could be reconfigured to suit the specific animation and provide the user with the most intuitive method of interaction.

The decision to use the laser cutter proved very beneficial, the device being constructed quickly and easily. Certain initial problems did occur however, possibly due to the laser cutter being too accurate. For instance, the end plates at times were becoming stuck in the inside of the casing and detaching from the piston handles. The solution was to simply sand down the offending edges, but in retrospect there should have been 1mm or so of tolerance allowed for the internal components. The general look and feel of the pistons proved very successful. Operation was smooth and easy after some slight modification and the device felt sturdy when in use.

BASIC ANIMATIONS

The next stage was to trial our new device. This began with the manipulation of series of simple, 2-dimensional surfaces by mapping a series of cluster points to the sensor inputs. Early animations involved simply attaching a cluster point to each corner of a lofted surface before mapping them to sensor input value. The next stage saw a surface divided into a 5×5 grid. A cluster point was assigned to each intersection point on the grid, and this cluster had its Y-translation randomly mapped to one of six sensor input pistons.

RE-CALIBRATING THE SENSORS

It quickly became obvious that simply containing the sensors within a casing did not give us the results we had hoped for. The initial idea was that a cluster point was selected depending on how much the piston was pushed in. The obvious way of interacting would be that pushing the piston half way in would select the cluster point half way along an edge. The problem, however, was that the sensors do not give a reading proportional to the distance that an object is from it. That is to say that if when the piston is at its fully open position, the value reads 0, and if when it is in its fully closed position, the value reads 100, the reading will not necessarily read 50 when the piston is in the half way position. If mapped onto a graph, plotting sensor reading values against the distance that the piston is pushed into the casing would result in an exponential curve. This proved a major problem and led to a radical rethink of the method of scripting we used to achieve what we wanted

In order to recalibrate each sensor, we needed to adjust the corresponding sensor value graph in Maya. The graphs on the left are attempts to adjust how the sensor input can be remapped to the output value across the range. This required a lot of experimentation with the graphs to try to counteract any anomalies from the sensors, to produce a constant set of results that were proportional to the actual distances of the plunger along the piston. Each sensor produced different anomalies and so a different graph was needed for each.

After much experimenting with the sensor value graphs, we achieved the best set of results possible, the selection of cluster points relating fairly well to the amount the piston was pushed. The graphs to the left show a series of graphs that produced a reasonably proportional set of input and output values. It became apparent that the main problems were with the lower end of the sensors’ range. This is evident in the graphs, as they all needed to be significantly lowered at one end to counteract any inaccuracies with the sensors.

ADVANCED ANIMATIONS

By recalibrating the sensors we now had far greater control over our animations. This allowed us to begin developing a series of animations which better reflected the full potential of our device. We introduced a digital representation of our piston into an animation to better understand the relationship between the physical and the digital models. We continued to use the theme of the lofted surface, which we deformed with cluster points. The pistons allow the user to interact with and deform a surface in a virtual world via physical instruments in an intuitive manner. This aims to replicate how a physical surface could be deformed in the real world.

In order for us to progress our animation we had to take a different approach. Moving away from the sensor graph we introduced ‘If’ statements. Unforeseen problems began to develop with the calibration of the sensors and we eventually had to settle on a revised method of mapping the device to our digital model. We split up the range of the sensors into slots which could be mapped to certain points of selection. The script consisted of a series of ‘if’ statements, for example ‘if sensor reading is between X and Y, select point Z’. If statements allowed us to define where on the surface the point of manipulation takes place. By assigning a sensor to each of the axis in the animation we were able to control which cluster was controlled. This allowed for greater flexibility in subsequent animations. We continued to use the theme of the lofted surface, which we deformed with cluster points. The pistons allow the user to interact with and deform a surface in a virtual world via physical instruments in an intuitive manner.

FINAL ANIMATIONS

For our final animation we took what we had learnt from the previous work and applied it to a more refined method of interaction. Initially, we worked hard to harness the accuracy of our device into an animation which could be precisely and intuitively controlled.

1.

A modular physical interface which is adaptable and changeable in order to suit the requirements of the user and the nature of the animation. The natural tendency for this type of project would be to develop an animation and then design the physical device to operate and interact with this animation. However we considered this approach to be quite restrictive and therefore decided to create a device that would be able to adapt to various animations. This means that the physical interface is not confined to one particular animation type. The practical approach to this concept was to devise a modular system that allowed for maximum flexibility through simple physical transformation. To allow for uniform results across all the physical devices, we opted to use the same sensor type, proximity sensors. Due to the nature of the range of reading of the sensors, it was essential to be able to control each input. This is why we made the box and plunger devices. As the next logical step in our development of the cluster deformed surfaces and in keeping with the form of a modular physical device, we developed an animation based on a series of cubes. These cubes are restricted to a 3D grid. The initial state of the animation is a 4 x 4 grid, all with ‘0’ Y translation. A plunger is assigned to select a cube in both the X and Z axis and a further plunger is used to translate the selected cube in the Y axis. A change of colour from blue to red denotes which cube is active. After testing we introduced a further plunger to prevent unwanted readings being assigned to cubes whilst navigating the space.

2.

Experimentation with the intuitiveness of the device and introducing randomness. The initial animation proved to be very effective. The blocks could be accurately controlled and translated, with the red indicator cube allowing the user to easily see the specific cube they were modifying. The problem was that the animation was rather mundane and monotonous. Our device, although incredibly accurate and intuitive to operate, was also rather complex, and there was an obvious interest in this complex system being used to control a relatively simple animation. We decided to remove some of the predictability of the device. To create this 2nd animation, which is clearly more abstract than the 1st, we removed the red selector box and randomly positioned the pistons, making it very hard for the operator to know which box they were manipulating with each piston. The result sees the operator either naturally struggle to learn which piston is associated to which manipulation by a process of trial and error, or else become completely content in randomly pushing pistons and enjoying the unexpected results that occur. We also began to introduce transparency and reflection into our animation, as well as a panning camera, all of which allow a greater understanding of how the animation is developing in 3D, and increase the general aesthetic appeal. Giving others the chance to try our device alerted us to the joy of its use as a tool to create, rather than simply a tool to control.

3.

Turning our concept on its head — removing the predictability of the device. This new line of thinking proved to be exciting and we explored the idea of turning our concept on its head yet further. Our initial animations were incredibly predictable, we had calibrated and smoothed all the life out of our sensors. We were basically using our analogue sensors as digital ones, giving us 1’s or 0’s, yes or no. We were essentially restricting our device and not exploring its full potential. The theme began to move towards abstraction. Not about how precisely our device could control an animation, but about how the device could be used within a seemingly random and counter-intuitive system. In animation 3, we have introduced further modification options such as rotate and scale. These modifications, as well as the translate function, are mapped to various boxes in a complex system which is almost impossible to understand when the animation is in use. Modifications of boxes are now depended on different modifications of different boxes. For instance, pushing piston 1 may rotate box A by X amount, and in turn trigger the scale of box B by Y amout and the translation of box C by Z amount. The results are wholly more abstract and certainly more interesting. Embracing the analogue nature of our sensors down to removing their value-smoothing capacitors results in a more entropic outcome to our previous attempts, where we enforced digital precision onto our device.

4.

Enhancing the animation and further liberating the sensors. Further experimentation was done with the animation, the scale factor was reduced to help add some visual density and the rotational factor was also increased. The camera angle was reduced to enhance the abstract nature of the animation.

5.

Introducing colour. A colour mapping has been introduced to the animation so that when each piston generates a certain value, the respective box has a blinn shading applied in one of the primary colours [100% red, 100% blue or 100% green]. Whereas in previous animations, each modification [scale, rotate, translate] only applied to a single axis of each cube, here the cubes can be manipulated in all 3 axis.

6.

The device becomes a tool to create and enjoy rather than to control. In the final incarnation of the animation our device is entirely free to create. The user still has precise control over the device, yet there is almost no sense of predictability, and so every action generates a new, unexpected result. The exact role of each piston is now not important, all 6 operate as part of a coherent system, although different results are achieved depending on the number of pistons in use. The transparency of each cube has been reduced to 50%, so there is an added bonus of the introduction of colour theory. At the points that the primary colours overlap, secondary colours are generated. Where secondary colours overlap with primary colours, tertiary colours are generated.

Posted in MA Applied Animation | No Comments | Bookmark or Share

Computers and man are polar opposite. Computers were created by humans, and now humans need computers to survive. The relationship is imperative. Analogy and metaphor have engulfed computers and software since they came into existence. We have enforced human analogy into our software and expect human traits in return, yet ‘folders’, ‘files’ and ‘desktops’ all essentially refer to the same thing, bits and bytes of pure, apathetic, binary data. There are two metaphors which stand above all others in their prominence as symbols of human characteristics with which we refer to our digital world. These are ‘Memory’ and ‘Decay’.

Memory is the metaphor most widely and commonly used in the digital world. Since the dawn of man, the word memory has denoted the human ability to recall past experiences and learnings, however with the rapid shift of our culture into the digital realm, if one was to now say the word ‘memory’ to a teenager, their thoughts would likely turn to a hard drive as much as they would to a past event they had experienced. Such is the power and pertinence of this metaphor.

Decay is a far newer and lesser-explored metaphor than memory, but one of equal importance. We are all aware of the decay that exists in our physical world, we are exposed to degradation, death and destruction frequently. We still struggle however with the concept of decay in the digital sense. We forget what we have been taught, but we do not expect our computer to forget what we have used it to create. Our physical possessions decay over time, but we do not expect this of our digital possessions. Are the little software glitches and the operating system crashes decay in the digital sense?

It is imperative that we begin to understand decay from a digital point of view, living as we do in a world that could in theory have the record of its recent history completely wiped out by a rogue Trojan horse (yet another metaphor).

In “The Art of Memory”, Frances A. Yates refers to the book “Ad Herennium”, a document on rhetoric dating to circa 86-82 BC by an unknown Roman teacher.

“When he comes to memory as an essential part of the orator’s equipment, he opens his treatment of it with the words ‘Now let us turn to the treasure-house of inventions, the custodian of all parts of the rhetoric, memory.’ There are two kinds of memory, he continues, one natural, the other artificial. The natural memory is that which is engrafted in our minds, born simultaneously with thought. The artificial memory is a memory strengthened or confirmed by training.”

One could ask whether there are now three forms of memory, with the third being ‘Digitally Artificial’. Furthermore, one could question whether the human artificial memory exists at all now, and whether it in fact serves any purpose in the modern world. This would result in there remaining only two forms of memory, yet with one of the two being very different.

In the times of antiquity, long before the invention of computers or even of literacy, scholars would employ complex techniques in order to memorize stories, speeches and events. The Art of Memory was said to have been invented by poet Simonides of Ceos whilst performing at the banquet of Scopas, a nobleman of Thessaly. The poet dedicated the first half of his chant to his host, and the latter part to the twin gods Castor and Pollux. Scopas then paid the poet only half of the agreed fee, and suggested that he demanded the rest of the payment from the gods. A little later a message was sent to Simonides telling him that two young men were waiting outside to speak to him. Upon leaving the banquet, SImonides could find no one, but during his absence, the roof of the building collapsed, crushing everyone inside to their deaths. The bodies were so heavily mangled that the relatives could not tell them apart, but Simonides was able to remember every single guest from the position at which they were seated around the table. The twin gods had paid their share handsomely. Simonides realized how essential orderly arrangement was for a good memory.

This association of memory with physical space became imperative, and techniques were established to allow one to memorize vast amounts by mentally placing images in physical spaces. The artificial memory the unknown teacher refers to consists of images and places. A ‘locus’ is a place which is easily memorable, such as a family home. The method involves pacing this locus, either mentally or physically, room by room and placing an image which is to be remembered in each room. Later, for instance when delivering a speech, the individual takes a mental journey through this locus, demanding from each room what he has mentally stored there. The images placed in each room must be as explicit and powerful as possible. Scenes that shock are most likely to be remembered, and so relating an image to, say, a scene of bloody battle works particularly well. Cicero defined two types of memory, not memory as in the style of memorizing, but memory as in the material that needs to be memorized. He defined the first as the memory of ‘things’, and the second the memory of ‘words’. Both ‘things’ and ‘words’ could be memorized using the spatial association techniques. Yet another question is posed when this idea is related to today’s society, is there now room for another form of data to be memorized? We now, surely, need to remember the bit and the byte as much as we need to remember things and words.

If we ponder memory and its modern meanings we reach a crossroads. Memory now refers as much to a computer’s storage capacity as it does to that of our own brains, but the two memories are very dissimilar. Human memory is poetic, it is nostalgic and entropic. Patches come, patches go, we forget both intentionally and unintentionally and we often are unable to erase an unwanted memory. Over time our memories are altered, they develop with the individual, they become amplified and even fabricated. Each individual memorizes an event in his or her own, often very different way. Digital memory is the polar opposite. It is clean and pristine, it is simple in its binary form. It is a series of 1’s and 0’s which either exists or does not. Digital memory can be copied, replicated and backed up, time after time, each version as perfect as the very first. The bytes that comprise an image can never decay in the physical sense, they do not age. As long as they have the media to exist upon, and there is no enforced destruction (deletion, viruses, etc) they will exist in their original form. But maybe the unexpected software glitches are a metaphor for aging. Maybe a virus is to a byte of data what Cancer is to a human cell. The differences in decay in the physical sense and in the digital sense mirror the differences between memory in the human sense and memory in the digital sense. Physical decay is slow but certain, it is expected and often encouraged. Like human memory, it is nostalgic and entropic. Conversely, the decay of digital data is sudden and abrupt. Like its binary polarity, it either works or it does not.

Memory and decay exist in an inextricable relationship. Without memory, it is questionable as to whether decay exists, as there is no recollection that an object has ever existed in any other form than its current. Without decay, can memory exist? If nothing ever changes then one can never have a record of something in a previous state.

Posted in MA Thesis | 1 Comment | Bookmark or Share

Messing about with an old webcam. It doesn’t work quite as well as it did…

Flickr set

Posted in MA Thesis | No Comments | Bookmark or Share

Further experimentation with the Jitter Mean patch. As well as manipulation of a live stream, the patch can also work on a pre-recorded video file. Here I ran a rendering from our Motion Capture Device through the patch. Ghostly forms of the boxes’ previous states fade to a pale mist. A trail of a past incarnation is captured. Is this digital memory acting in a poetic, human way?

Click for the full Flickr set

Posted in MA Applied Animation, MA Thesis | No Comments | Bookmark or Share

Took a day off on Saturday and had a trip down to Brighton. It was the first time I’ve been and I was very impressed with the Royal Pavilion, but particularly intrigued by the West Pier, which was victim to a huge fire in 2002. The piers’ walkway has subsequently collapsed but the charred frame of the main platform still exists a few hundred meters out at sea. I can’t help but imagine the burnt shell as the site for a student architectural project, or in an ideal world even a site for my installation, possessing obvious decay as well as vivid memories for locals and holiday makers. We can all dream…

Posted in General, MA Thesis | No Comments | Bookmark or Share

MaxMSP Debris + Mean from Richard Almond on Vimeo.

Managed to combine the ‘Mean’ patch with the ‘Debris’ patch to begin to mash digital decay with digital memory. The results are subtle yet hint at something exciting. This could be quite a brake-through in terms of the technology behind my installation. The effect does need some fine tuning, at the moment it seems a little heavily orientated towards the ‘Mean’ patch – need more of the ‘Debris’ decay.

Would also like to get a little more interest into the clips – possibly some colour, although there isn’t exactly much of it in the studio…

Posted in MA Thesis | No Comments | Bookmark or Share

MaxMSP Mean from Richard Almond on Vimeo.

This Jitter patch calculates the mean matrix value of the pixels it records over time. Perfect for achieving the effect of capturing memory.

This could be further successful if combined with the previous debris patch, allowing these ‘digital memories’ to be decayed in some fashion.

Posted in MA Thesis | No Comments | Bookmark or Share

MaxMSP Debris from Richard Almond on Vimeo.

Exploring MaxMSP as an alternative to Processing. There are many free patches available which can be edited and tweaked, and this is the recipe 27 – ‘Debris’ patch.

This particular effect is pretty successful and very similar to the data moshing I have previously explored, albeit a real time version.

Need to do further experimentation into the possibility of overlaying this effect onto others to create an effect which continues to decay over time

Posted in MA Thesis | No Comments | Bookmark or Share