In sound design, one of the most used processes consists in modifying the pitch of a sound, generally by changing its playback speed. This effect is called pitch shifting. This fairly simple manipulation has important consequences on the timbre of the sound. Let’s see this in detail.
What is the pitch of a sound ?
Pitch is a fundamental characteristic of sound that corresponds to the frequency at which air vibrates. We express the frequency in Hertz, 1 Hz is one oscillation per second. The human ear can hear frequencies between 20 Hz and 20,000 Hz. Frequencies between 20 Hz and 200 Hz are called bass, 200 Hz to 2000 Hz midrange, and 2000 Hz to 20,000 Hz treble. Note: These values, which are easy to remember, are approximate and there is no consensus on this.
What is pitch shifting ?
Pitch shifting consists in modifying the pitch of a sound. The easiest way to achieve this is to speed up or slow down a sound, this is called resampling. When we speed up a sound, we increase its frequency and reduce its duration: the sound becomes higher pitched. Conversely, if we slow down a sound, we lower its frequency and increase its duration: the sound becomes lower pitched.
You have certainly heard dozens of examples of pitch shifting in the movies or on the radio. The effect is often used to give funny voices to characters: Cinderella’s mice, Minions or Rabbids.
The question is: since we are capable of modulating the pitch of our voice at will, why do computer pitched voices sound artificial ? The answer lies in a notion known to acousticians: the formants of sound.
What are the formants of sound?
Simply put, formants are a kind of unique acoustic imprint of a voice or an instrument. Formants are frequency zones that always contain more energy than the others. They are are related to the physiognomy of a person or an instrument and do not vary with pitch.
Below is the spectrogram of a sung phrase. The lighter areas materialize the formants and the different lines materialize the harmonics of the sound. The formants act as filters inside which the harmonics are amplified.
Pitch shifting modifies acoustic formants
When we artificially modify the pitch of a sound, we transpose all of its components: the fundamental note, the harmonics, but also the formants. However, for the timbre of a sound to be recognizable, the formants must be unchanged.
In the following example, we successively hear a sung note, then the same note sung up an octave, followed by a note artificially raised by an octave.
The following spectrogram corresponds to the three preceding sounds. It is clear that the first two notes, although separated by an octave, have the same formants (light areas). The formants of the pitched third note are all shifted, which is why the timbre sounds artificial.
Here is a second example, with a portamento (a note that slides to another note) performed vocally, then with a pitch shifting tool. Listen to how quickly the sound becomes artificial in the second case.
On the left, we notice that the formants do not change place. On the right spectrogram, the formants follow the pitch variation.
History of pitch shifting
1 The beginnings of cinema: analogue media
The tape recorder is arguably the first real pitch shifting tool in history. With magnetic tape, it is quite easy to change the playback or recording speed to alter the timbre and pitch of sound. The first pitched sound effects and voices appear in the 1940s in cartoons.
Cinderella, produced by Disney studios in 1950, is one of the finest examples. Indeed, the majority of the characters in the film have a pitched voice.
2 80s: samplers
The first electronic samplers spread in the 80s. (Read also What is a virtual instrument). They allow sound samples to be played at different pitches using a keyboard.Their simplicity paves the way for new uses of pitch shifting, in particular for musical composition.
In Terminator 2, the musical theme of the T1000 robot is built around samples of trumpets pitched several octaves down. They turn into a kind of artificial respiration that evokes a robot heart.
3 Years 2000: computer tools
Progress in digital signal processing gradually overcomes constraint of resampling. It is no longer necessary to change the playback speed of a sound to modify its pitch, which opens up many new perspectives.
Pitch shifting becomes accessible in the form of plugins and natively in all sequencers. Some algorithms are designed to preserve timbre, rhythm, only modify formants, etc. Each parameter: pitch, duration and timbre, becomes independent.
In music, you can now change the tempo of a track without changing its key (or vice versa), which offers endless possibilities for DJs.
The famous Autotune tool, designed to correct voice pitch in real time, was first exploited artistically in 1998 by Cher, thus inventing the “Autotune effect”.
4 Ultrasonic microphones: the future of pitch shifting?
Microphones are usually designed to record the same frequency range as the human ear. But in recent years, we have seen the appearance of new professional microphones capable of recording ultrasonic frequencies up to 100,000 Hz. What’s the use of this?
When you lower the pitch of a sound, you transpose all of its components down. If the highest frequencies picked up by the microphone are the same as the highest audible frequencies, then pitch shifting will cause an audible loss of high frequencies in the sound. If the ultrasonic frequencies are also picked up, they take the place of the high frequencies which have been lowered. The sound then remains defined over the entire frequency range of hearing, even when lowering its pitch by two octaves.
Here is an example: a simple vocalization recorded with a standard microphone and an ultrasonic microphone. The two sounds don’t sound very different, but when you lower them by 2 octaves, the difference is obvious: the second sound is much clearer and more defined.
Conclusion: Pitch shifting, an essential sound design tool
Today, we can use pitch shifting thanks to a huge palette of tools, to manipulate sound effects and its pitch in extreme proportions. Pitch shifting allows us to modify the emotion conveyed by a sound and make it completely unrecognizable. To transform sound effects, adjust sounds to the key of music or even create funny or monstrous character voices, the only limit is our imagination.
I will end with an example that I find brilliant in its simplicity and effectiveness. In The Fellowship of the Ring, by Peter Jackson, there is a scene in which the elf Galadriel suddenly falls prey to the power of the ring. Sound designer Mike Hopkins asked Cate Blanchett, who plays Galadriel, to record her text twice, once at normal speed and once while speaking slightly faster. The second take was then slowed down (thus pitched down) by one tone and mixed with the first take to create a demonic double voice.
Photos by Nicolas Titeux and Nam Anh on Unsplash.
