Abegayle Elmes

VII

Walter Murch… We gestate in Sound, and are born into Sight
Cinema gestated in Sight, and was born into Sound.

P8

Empathetic and Anempathetic Effect

‘On one hand, music can directly expresses participation in the feeling of the scene, by taking on the scenes rhythm, tone, and phrasing…One the other hand, music can also exhibit conspicuous indifference to the situation, by progressing in a steady, undaunted and includable manner:’

Side note: examples Hitchcock’s Psycho (the shower) and Antonioni’s The Passenger (an electric fan)

P10

Sound perception and visual perception have their own average pace by their very nature; basically, the ear analyzes, processes, sythensizes faster than the eye.

P25

…there are at least three modes of listening, each of which addresses different objects. We shall call them casual listening, semantic listening, and reduced listening.

Casual listening, the most common, consist of listening or a sound in order to gather information about its cause (or source). 

Side note: Sound which indicates something has happened e.g. like a closed door.

P28

I call semantic listening that which refers to a code or a language to interpret a message: spoken language, of course, as well as Morse and other such codes. 

P29

Pierre Schaeffer gave the name reduced listening to the the listening more that focuses on the traits of the sound itself, independent of its cause and of its meaning. 

…every individual hears something differently, and the sound perceived remains forever unknowable. But perceptions not a purely individual phenomenon, since it partakes in a particular kind of objectivity, that subjectively that reduced listening, as Schaeffer defined it, should be situated.

P30

Everybody practices at least rudimentary forms of reduced listening. When we identify the pitch of a tone or figure out an interval between two notes, we are doing reduced listening; for pitch is an inherent characteristic of sound, independent of the sounds cause or the comprehension of its meaning. 

Side note: look at Traite des objets musicaux by Schaeffer – system classification

P31

However, reduced listening has the enormous advantage of opening up our ears and sharpening out power of listening. 

P33 

But we must also remember that these three listening nosed overlap and combine in the complex and varied context of the film soundtrack.

…we must take into account that conscious and active perception is only one part of a wider perceptual field in operation. 

Surely, out conscious perception can valiantly work at submitting everything to its control, but, in the present cultural state of things, sound more than image has the ability to saturate and short-circuit our perception.

P34

…sound has an influence on perception through the phenomenon of added value, it’s interprets the meaning of the image, and makes us see in the image what we would not otherwise see, or would see differently. And so we see that sound is not at all invested and localised in the same way as the image. 

p47 

Unification

The most widespread function of film sound consists of unifying or binding the flow of images. 

P48

The function of punctuation inits widest grammatical sense…has long been a central concern of theatre directing. 

P49

So synchronous sound brought to the cinema not the principle of punctuation but increasingly subtle means of punctuating senses without putting a strain on the acting and editing.

Naturally, music can play a major punctuative role. It certainly did in the silent era, but in a less precise, more appropriate way, owing to the much looser methods of synchronising music with image. 

P58

Point of Synchronization and Synchresis

A point of synchronization, or synch point, is a salient moment of an audiovisual sequence during which a sound event and a visual event in synchrony. 

P71

Acoustic, a word of Greek origin discovered by Jerome Peignot and theorized by Pierre Schaeffer, describes “sounds one hears without seeing their originating cause.”

P74 

(Pie Chart)

P75

Why reject a valuable distinction simply because it isn’t absolute? It is a mistake to see things in a binary, all-or-nothing logic. 

We must add new categories—not claiming thereby to exhaust all possibilities, but at least to enlarge the scope, to recognize, define, and develop new areas. 

P114

The materialization indices and the sound’s details that cause us to “feel” the material conditions of the sound source, and refer to the concrete process of the sound”s production. They give us information about the substance causing the sound—wood, metal, paper, cloth—as well as the sound is produced —friction, impact, uneven oscillations, periodic movement back and forth, and so on. 

P120

Sound in Animation: Putting to Movements

Delande and Celeste determined that sometimes these vocal productions “partake in a code of expression of feelings”

Side note: look into the study

P121

The sound here conveys movement and its trajectory rather than the timbre of the noise that supposedly issues from a car. “The substance of the sound has nothing to do with resemblance, it’s the sound trajectory that does.”

P122

[Mickey Mousing]…has been criticised for being redundant, but it has an obvious function nonetheless.

The various gags and actions of the film are accompanied and punctuated by the musical figures you would expect; when the flea jumps, a musical cue jumps with it, as in the circus.

P149

It can be said that sound’s greatest influence on film is manifested at the heart of the image itself. The clearer treble you hear, the fast your perception of sound and the keener your sensation of presentness.

Side note : what does ontologically mean?

P166

Never is television as visual as during some moments in music videos, even when the image is conspicuously attacking itself to some music that was sufficient to itself. 

Of course not everyone aggress. Cinephiles especially attack music videos as eye-assaulting; they dislike the stroboscopic effect of the rapid editing. 

Audio Visual Analysis

P185

For reasons we have already examined, sound seems to remain much more difficult to categorize than images, and there remains the risk of seeing the audiovisual relationship as a reptile of illusions, even tricks—all the more contemptible for being so. 

P186

Audiovisual analysis must rely on words, and so we must take words seriously — whether they are words that already exist, or ones being invented or reinvented to designate objects that begin to take shape as we observe and understand.

P187

[masking method] Screen a given sequence several times, sometimes watching sound and image together, sometimes masking the image, sometimes cutting out the sound. 

P189

[Forced Marriage] …we become conscious of yer fundamental strangeness of the audiovisual relationships; we become aware of the incompatible character of these elements we called sound and image. 

P192

Side note: look at experiment called La Dolce Vita

P 327

A considerable amount of time elapsed before the “talkies” were accepted as constituting a medium in their own right.

Both sound and noise are audible vibrations (oscillations)

Sound has purpose; it is organised. Noise is essentially random. 

P 333

Descriptive Nonliteral Sounds

P 342

The sound rhythm acts like a clothesline on which you can “hang” shots of various lengths without sacrificing rhythmic continuity.

P350

Timber…describes the tone quality or tone color. 

P 353

The attack or decay of a sound is part of its dynamics and duration. Attack refers to how fast a sound reaches a certain loudness level.

P355

When you combine picture and sound vectors, you can match the attack and decay variables of both vector types.

P 359

In musical terminology polyphony refers to two or more melodic lines that, when played together, form a harmonic whole.Unlike homophonic structures, where a single dominating melody is accompanied by supporting chords, polyphonic structures are composed of multiple dominating voices (melody or horizontal vectors)

P 363

One of the most common elements of contrapuntal structure is imitation. Not only is the theme or subject is stated in one voice and then repeated verbatim or in a slightly changed form in the other voices or voices while the first voices continues on its way, providing the counterpoint to the imitated theme.

The canon, or round, is the purest and most obvious form of imitation. Not only is the theme repeated verbatim by the other voices but also the entire melody. 

P366

You must learn to combine the video and audio vector fields so that they form a synergistic structure.

Polyphonic Structures

In a polyphonic picture/sound structure, pictures and sound seem to develop independently as “melodic” lines yet combine vertically into an intensified audiovisual experience.

1. Art in Motion, AnimationAesthetics by Maureen Furniss

P85 

One of the biggest differences between amateur and professional animation often lies in the way that sound is employed.

…most people will find that the secret success for many award-awarding films is the care with which aural elements- voices, sound effects and music – have been handled.

In Film Art, Bordwell and Thompson identify the acoustic properties of motion picture sound as loudness, pitch and timber.

P.86

Bordwell and Thompson write that:

… the harmonic components of a sound give a certain ‘color’ or tone quality – what musicians call timbre. 

Side note – Look at ‘The Sound of the Early Warner Bros. Cartoons’ Scott Curtis

At the Fliescher Studio, a manual given to employees spelled out the technicalities of synchronising lip movement to record dialogue.

… in animating a bit of dialogue or singing, the mouth actions are really secondary. 

…They must been the ‘nose’ (accurate) and convincing, but perfect mouth-actions mean nothing if the actions itself is not convincing. 

P.89

Sound Effects

Some of the most famous animated characters of studio animation- Tom and Jerry, the Pink Panther, the Roadrunner… – have had no voices at all )at least for most of their careers). 

Sound effects in animation often are applied in an exaggerated manner, with no realistic congruity between an action and the noise created from it. 

… a relatively simple mix of sounds can serve to heighten the drama or suspense of a scenario.

P 90

Musical scores

Of all the components of sound, music is the category that has attracted the most critical attention, perhaps because it is also the aspect that has most concerned major producers of motion pictures, both live-action and animated. 

‘A Sound Idea: Music for Animated Films’ Jon Newsom

P92 

Low pitches I relate to colour of low-light values, irrespective of hue and chroma; high pitches to high-light values, irrespective of hue and chroma. In other words, I let dark browns, olives, reds, purples, blues predominate on the screen during a low pitch passage, and use yellows, pale greens, pale oranges, very pale reds, purples, and blues during a high- pitched passage.     (Norman Mclaren)

Side note: Look into people who can see colours and experiences one psychedelics. 

He was intrigued by the idea that different shapes produce particular sounds when placed on an optical soundtrack – another aspect of his interest in synaesthesia. As a result he conducted experiments with ‘animated sound’, ti see how sound elements could be created by drawing (or otherwise fabricating) synthetic soundtracks.

‘Technical Notes on the Card Method of Optical Animated Sound’ and ‘Handmade Sound Track for Beginners’

‘Neighbours‘ film(1952)

… it would be a film that would marry animated images with classical symphonic works into a concert of visual music. A stereo-sound system, Fantasound, was even developed to create a dimensional sound environment, breaking new ground in the use of multiple audio tracks for film. 

Music and the Animated Cartoon’1940s Chuck Jones

… all cartoons use music as an integral element in their format. 

Steamboat Willie (1928)

In 1928, close synchronisation was an incredible novelty to be capitalised upon – so it is no wonder that sound is linked so closely with the film’s visuals. 

After the coming of sound, the Disney Studio began producing both the character-driven ‘Mickey Mousing” shorts and music driven “Silly Symphony’ shorts.

‘Who’s afraid of the big bad wolf?’

P 95

The Fleischer Studios (and later the Famous Studios) employed a musical director named Sammy Timberg, who said reworked with the artists from the first drawings to the end.

‘The Ink and Paint of Music.’ 1997 article Amin Bhatia MIDI

P 96

In animation the picture is usually the last to show up!

P 97

Most of these productions employ a narrative structure; that is, they tell stories with a beginning, middle and end.

Gag structure

‘Narrative threat’

P 99

Story mood chart image – Mr Bug Goes to Town story mood chart courtesy William Mortiz

Fleischer Studio created a ‘Story Mood Chart’ to illustrate the degree ‘degree of emotional pace’ in its feature,

A sort of rise and fall in emotion throughout the film, 

Norman McLaren used a different kind of visual chart to illustrate the development of action and emotion in A Chairy Tale (1957)

https://vimeo.com/215870921

So I am looking now into collecting primary research. I have made an account on ArtStation website where there are loads of artists around the world I can access and message. I have favourited my favourite audio drive animation work. I have also messaged them individually to ask if they were interest answer a few simple questions. I constructed a simple questionnaire to get the ball rolling. I think from this I can then fashion up a second questionnaire once I am a bit more assertive of where I will be taking my Thesis paper.

Audio Driven Animation Questionnaire 1

This is the first questionnaire of my thesis research. I am looking for some perspectives and ideas while I research further into the topic. Once I have formulated my work more I will have more questions to send you later on this month. Thank you for taking part. 

Name :
Date :
Job Title/Position (optional): 

1. What is your position and role in the VFX/3D computer animation industry?
2. Where about are you based?
3. In film, what is the driving force of an amazing story? What techniques/directional decisions are the most effective to creating a master piece? (Use examples to illustrate your point if needed)
4. In your creative process, how does audio plays its role?
5. How do you go about choosing your audio music when accompanying your animation?
6. What is audio driven animation mean to you?
7. Can you give me an example of audio driven animation that has been powerful and/or influential to your experience as an artist?
8. When creating audio driven work, how do you achieve this?
9. Why is audio driven animation important and/or interesting to you as a designer and as a viewer?

I have been fortunate enough to have 3 responses and one artist has already completed the questionnaire. I think that I need to broaden my target population so that I get more responses but from all kinds of people from different areas of animation, music, artists and non creative roles as all of these opinions are valid.

My next plan of action is to ask more artists and animators on ArtStation and also to set up a monkey survey and just see what comes back. I have been emailing animators who made the Walt Disney Fantasia 2000 but haven’t had a response as I assume, they are very busy…

The tutors on this course are extremely valuable, so once I get a bit more of an idea I will pick their brain as I am sure they have a lot to say on this topic.

3. Walt Disney’s Fantasia: The Cosmos and the Mind in Two Hours by William L. Benzon

P. 1 … the cumulated techniques of 3000 years of art history, Western and Eastern, and a large swath of the cosmos and of life on Earth.

P. 3 The fact that Fantasia is not even a narrative – it tells no connected story from the beginning to end — is a more serious matter.

P. 13 A light in sound, a sound-like power in light, Rhythm in all thought, and joyance every where – S. T. Coleridge, The Aeolian Harp

…— oh, it’s just things and stuff that come to mind as your mind wanders while listing to the music, just things and stuff, not to worry.

4. Music-Driven Animation Generation of Expressive Musical Gestures by Alsysha Bogaers, Zerrin Yumak and Anja Volk

p. 1 gestures animation, neutral networks, music driven animation

…to model these personal choices such that a performance of a musical piece can be generated in a certain performer’s  style of a musical piece can be generated in a certain performer’s style. The movement made by the performance are also used to express emotion in music. 

However, human perception is vital in perceiving these expressed emotions, which makes the simulation of emotion-based expressive gestures very difficult. 

Data-driven approaches are promising to produce more natural results than procedural animation.

Look up these references:

• Anders Friberg. 2004. A fuzzy analyzer of emotional expression in music performance and body motion.
• Danielle Sauer and Yee-Hong Yang. 2009. Music-driven character animation. 
• Herwin Van Wellbergen, Ben JH Van Basten, Arjan Egges, Zs M Rutttkay, and Mark H Overmars. 2010. Real time animation of virtual humans: a trade-off between naturalness and control.

Research Papers Quotes:

1. Playing in ‘Toon:Walt Disney’s Fantasia (1940) and imagineering of Classical Music by Mark Clague. (2004)

p91 – “music is a spontaneous impulsive expression – that its range is without limit” Leopard Stokowski, Music for All of US, 318

Check out Walt Disny quotedin Culhane

p92- That is why great music associated with motion pictures is so important, because motion pictures reach millions all over our country and all over the world.

(fantasia critics) … audiovisual alignment can articulate form, how gestures of melody and motion reinforce one another, and how musical rhetoric and image fuse to create meaning.

p96 … the combination of science and creativity, engineering and imagination.

p97 – … image does not simply explicate sound, it introduces a host of associations, ideas, and references to music that brings new meanings …

… audio-visual synthesis that transformed both sounds and image into a “new form of entertainment.” (Roadshow program, (18). The Term “audio-visual” is borrowed from Michel Chion.

It appealed to middlebrow culture, puncturing social barriers between high and low, education and entertainment, to nurture a middle-class America culture by making classical music accessible.

p98 – The movie this connects music to life through image … new vision … ideological imagineering…

The host of expressions had to be organized and expressed in harmony with the rhythm and structure of the music itself.

… artists have not only remain faithful to the spirit of the various compositions, but they have also… translated the very phrases and measures and even individual notes into just the right colours and actions.

p99 – It pairs music with “just the right” imagery, giving the whole ideological construction the imprimaturs of Art.

2. Animation and Music: Principles for Effective Combination

p1 Music and film both emotionally rich mediums.

… limitless range of human emotion.

… create a deeper, more emotionally rich experience.

… animators using music as both the driving force for an animated film as well as

a means of aiding in the understanding of the action.

… Norman McLean, Oskar Fischinger, Jaques Drowin and John and Faith Hubley who were also experiencing with the connections between musicians animation. 

Disney’s technique of synchronising every single animated movement to the beat or pulse of music become so popular and recognisable that it was known in the industry as “Mickey Mousing”

P. 2 Animation sequences were planned out in strict adherence to the number of beats in the music. 

Fig. 1. From the Illusion of Life by Frank Thomas and Ollie Johnston

Out of thisProcess came a new form of music and animation notation called a bar sheet, also referred to as a dope sheet (fig. 1)

… animation were forced tofu their movements and storytelling techniques to the confines of music.

“There is a special feeling in work that is done this way that is not found in other methods, but it is more expensive because it demands it makes in all the creative department” Disney animator Frank Thomas

… animator were challenged to limit their expressions of motion the absolute minimum.

Side thought – emotion has motion within the word, is there a link in meaning?

p. 3 Arguably, animation that is built around music soundtrack creates the strongest from of audio/visual combination.

My thoughts – is animation better when made to fit music?

The process of animating to music is extremely laborious and involves highly details analysis of music before creating a single animation drawing. 

My thoughts – How does it differ in production then and now? Less difficult?

… studies of a more scientific nature have been conducted George Washington University in which computer animated objects are used to drive synthesised sound production.

Researchers are, in a sense, creating virtual objects that can programmatically generate their own sounds when they collide with other objects. 

…shows a physics based approach to sound generation. It attempts to create a sense of realism virtual environment by having objects appear to create realistic and believable sounds. Emotional considerations do not seem to be considered.

My thoughts – emotional vs physics

There have been several computer programs written that create visualization of music such as Microsoft’s Media Player, Nullsoft’s Winamp and RealNetwork’s RealOnePlayer.

The colors and shapes can change with changesinthe intensity and frequency (low sounds verses high sounds) of music.

My thoughts – frequency driven

P. 4 … direct relationship between two elements of music, pitch and loudness, which are integral factors in music’s ability to express emotion.

… visualization hanse way of really differentiating between these two emotions.

… hypnotic effect for the viewer.

What these visualizations lack, however any deeper meaning or connotations implications that are derived from emotional elements.

… this emotional connection between visual and audio would be purely coincidental.

P. 5 In the early 1980s, a new electronic protocol was developed for communicating between music synthesizer … MIDI (Musical InstrumenT Digital Interface)

My thoughts – Coyote Ugly and incredibox

https://www.incredibox.com/

… “Butterflies in the Rain”, was developed at Ohio State University as part of a graduate thesis… mapped that data to the animation of a 3d animated piano,

Animusic that produces little mini “music-videos” based on pre-compsed MIDI music.

… animation appears to be “playing” the music.

My thoughts – illusion of reality and interactive. – touch / tactical animation. 

P. 6 … the musical mood and the visual mood are not what is being driven automatically. The animated motion is complex and synchronized very closely with the music, but the qualities of music that help to enhance the level of emotion expressed in the films appear to be created manually by the animator.

Disney animators Frank Thomas and Ollie Johnson wrote, “Music is undoubtably themes important addition that will be made the picture. It can do morello bring a production to life, to give it integrity, style, emphasis, meaning, and unity than any other single ingredient. 

Music in combination with film can take two forms, diagetic or nondiagetic. 

P. 7 Nondiagetic music refers to music that does not emanate from within the environment of the film.

… Herbert Zettle calls “visual vectors.” In his book Sight Sound Motion: Applied Media Aesthetics he defines these as forces that are leading the viewer’s eye from one point to another.

My thoughts – all animation is audio driven?

The most basic elements of music are pitch, timber (pronounced “timber”), duration, loudness and attack/decay.

p. 8 Structure of music…melody, rhythm, harmony, homophony, polyphony.

Melody – series musical tones grouped in succession

Rhythm – formed by varying the duration of notes based on a “felt” pulse or beat

Harmony – Major, minor

Combination Principles

Herbert Zettle