Apple is proud of the cameras it places in the iPhone and the resulting photos have been used in multiple marketing campaigns highlighting the technological advances in imaging. Now it’s the turn of audio.
A newly published Apple patent (“Spatially biased sound pickup for binaural video recording”) describes a number of techniques to process recorded audio to give the viewer a greater sense of ‘being in the scene’. Jack Purcher reports:
“[the patent] …covers Binaural recording of audio that facilitates a means for full 3D sound capture—in other words, being able to reproduce the exact sound scene and giving the user a sensation of ‘being there.’
“Apple states that full 3D sound capture can be accomplished through spatial rendering of audio inputs using Head Related Transfer Functions (HRTF), which modifies a sound signal in order to induce the perception in a listener that the sound signal is originating from any point in space.”
Traditionally, binaural recordings have used dual microphones specifically placed to reproduce where human ears would be, with some recording units using physical models of the ear to increase the realism of the sounds being captures. This pulls the brain into a scene, and it is magnified when coupled with visuals.
Here’s a sample video which lets you hear the difference between a mono recording, and a binaural recording as you look around a musical practice room.
Apple is not planning to put a pair of comedy ears on either side of your iPhone to capture the sounds. Instead the patent discusses software methods that use a dual camera system and four microphones to build up a digital picture of the surroundings, before using software processing to create the binaural soundtrack.
AR provides an obvious practical use of this technique, especially when paired with equipment that tracks head movements and sculpt the sound around the viewer, but it could also be used to enhance video recordings made on the iPhone to create a better ‘you are there’ sensation when watching personal videos. After all the iPhone knows its place in time and space.
As always, the publishing of a patent does not guarantee that any hardware or technique will make it to the public, but with Apple’s continued exploration of AR, I suspect this one is going to appear in some form.