So, I'm finally all settled in back at university, and getting a start on my new project for the year; 3D Auditory Pixels. The brief for the project goes a little something like this:
3D Auditory Pixels
The Schools of Computer Science and Psychology have just purchased an audio system capable of
generating as if it is emanating from a specific point in 3D space.
This is achieved via control of an array of 20 small loudspeakers.
This project will look at developing methods to generate sounds that can be placed in the 3D soundscape.
Some simple interface and methods for controlling the sound location will need to be implemented.
Scope exists to also utilise 3D motion capture equipment as input to control the sound location.
For example, a virtual 3D "theremin" musical instrument (www.thereminworld.com) could be developed where the 3D position of
the hand, for example, controls parameters of a simple "theremin" sound generator.
The loudspeakers are actually located behind an acoustically transparent video screen. So scope exists to us the
screen for video output. Some 3D display technologies including a 3D head mounted display or stereo viewers
could also be utilised.
The first part of the project will be to devise suitable sound generation and control of the sound. The second part,
which could take one of many paths, would look to utilise some of the I/O technology mentioned above.
Prerequisites: Good level of programming and algorithms. C/C++ or Java programming knowledge.
Some knowledge of audio and basic maths an advantage.
So, in layman's terms, i get to mess around with a kick ass sound system, a WiiMote, and a sha-weet
Polhemus motion capture rig.
I've made a little progres so far, mostly just experimentation;
- Capturing motion from the WiiYourself! library
- Playing randomised noise through sequencial/random or selected speakers
- Trying to figure out what the hell is going on with the library written by the summer student, and figuring out where to start on a re-write
- Had a look at the libsndfile library and attempted to read a WAVE file to an array
- Tried to figure out Visual Studio 2008s (eugh..) idiosyncrasies
- Talked to an Audio-Psychophysicist from the psychology department about what he needs from an API I'll be working on, and what they'll be doing
Though it's only two weeks in, there seems to be some half decent progress being made nonetheless.
The psychology aspect seems really interesting, involving the ways the mind interprets the senses, weights them according to their accuracy and compiles them into what you see/hear etc. For example, vision is usually the more heavier weighted, so if there's a dot on the screen, and a sound coming from near by, your mind will tell you the sound is coming from the spot. I believe this is the
ventriloquism effect. However, if you blur the spot, or make it much bigger, your mind lessens the weight on your vision, and instead sound takes the primary role.
It's all crazy-crazy stuff.
Posts
