The Virtual Mixer
The Future of
“David Gibson’s Groundbreaking work is a real eye-opener for all of us, music professionals and casual listeners alike. He has single-handedly rethought the whole metaphor for the visual representation of recorded music and conjured up a brand new way to interact with it. It’s high time we took a new look at the antique user interface employed by typical MIDI and sample editing tools”
– Thomas “Dolby” Roberson
THE VIRTUAL MIXER INTERFACE
The Virtual Mixer is a Graphical User Interface that displays the amplitude and frequency range of sounds on a multitrack, and then allows you to use these visuals to control various types of mixing consoles and effect units.
MIXING IN VIRTUAL REALITY
Sitting in the middle of the Surround Mix below wearing a Dataglove.
You could then reach out and grab a sound
and place it wherever you want!
You could even throw sounds and bounce them off the walls.
With a full Virtual Reality Head Tracking Helmet,
you could then turn around and see the sounds behind you.
You can also mix in normal stereo in using 3D images.
3D helps you to see around sounds in a mix in order to control them.
3D visualization allows you to see the sounds precisely where they are between the speakers
in the world of Imaging.
This is the ideal interface for Mixing in Surround Sound
and it will be so much fun!
CLICK HERE to see examples of Surround Sound Mixes from “The Art Of Mixing, 2nd Edition.
The next step is to place images and sounds inside the body
for more info
The mapping of audio parameters to visuals is based on our common perception of imaging between the speakers. The primary design consideration has been to show masking (where one sound hides another sound) visually. Of course, the second design consideration was to create an intuitive interface that is calibrated to the settings on a mixer so that a Professional Engineer will still have precise control.
Fader level (volume) is mapped out as a function of front to back, so that louder sounds appear closer (and therefore, a little larger), and softer sounds appear more distant. The grid on the floor is calibrated to the faders on the mixing console. Panning is naturally mapped out as left to right placement. The average pitch of the sound over the entire song is mapped out as a function of up and down. Higher pitch sounds appear higher; lower pitch sounds appear lower (just as they do in a mix). Also, hi frequency sounds are smaller because they take up less space in a mix. Low frequency sounds are larger because they mask other sounds more in a mix. You can also call up windows that show equalization and auxiliary send levels visually.
The next step is to put on 3D glasses and have the images floating in 3D right where the sounds seem to be between the speakers. In such a virtual environment the mixer uses a dataglove to move the sounds around in the mix. If you want a sound louder pull it toward you. If you want it in the left speaker put it over there. You can even toss sounds back and forth (and bounce them off the walls). Watch out, here comes an electric guitar sound. In 5.1 surround sound mixing, the visuals would be floating throughout the room.
FORMATS AND VERSIONS
The Interface is being developed for a wide range of mixing equipment — hardware and software products including:
Digital Audio Workstations: The program is being developed to work on PC and Macintosh computer formats as a plugin for TDM, RTAS, and VST systems. This includes Cakewalk, Cubase, Logic Audio, Ableton Live and ProTools. The Virtual Mixer will show up as an alternative to pictures of a mixing console and may be used to control the volume, panning and all of the automation parameters of the other plugins on each of the channels. Cost will be approximately $400.
Digital Consoles – Yamaha, Roland, Tascam, Soundcraft, Mackie, Panasonic (Ramsa), and all future digital consoles. A software program will be developed to run on a separate computer. It will interface to the Digital Console by way of MIDI and Digital Audio. Sold for $400. Ultimately, it is assumed these companies will license the interface and incorporate it as an alternative to pictures of a mixing console on the computer screen.
Major 100% Automated Consoles: Euphonix, Sony, AMS NEVE, AMEC, and SSL. Price range will vary depending on the manufacturer.
3D DISPLAYS AND CONTROLLERS
– Touch Sensitive Computer Screen – Makes mixing as easy as dragging the sound to where you want it with your finger. Approximately $700 additional.
– 3D Glasses for the Computer Monitor – Extra $200.
– Full Virtual Reality with Head Tracking Helmet – So you can turn around and see sounds floating behind you when mixing in Surround Sound or using a 3D Sound Processor. Extra $300.
– Big Screen Projection in full 3D (similar to IMAX 3D). Images float between the speakers right where you hear them. Approximately $3,000 additional.
The software will work in normal stereo with volume as front to back, or in Surround with volume as a function of size of the sound visual.
We will also put out versions that will control display various waveforms and audio synthesis parameters. These parameters will also be able to be controlled visually, and in full 3D Virtual Reality. Specific textures of spheres correspond to the waveform.
One of the most revolutionary aspects of the program is the way in which automation is visualized. Time is shown as the Z-axis (back to front).
The past is in the distance and time moves forward until you get to the the present moment in the middle of the screen.
The major advantage is that you can see the relationships of all of the settings at any one moment. Current “line” automation does not show you any usable relationships between the information. The other big advantage is that you use this window to create automation in real time by dragging spheres around while the mix is playing.
When mixing in Virtual Reality this timeline will appear as a tunnel of time. You will be able to go to the past or future to change any parameter of the mix.
More Engrossing – When you see the mix visually you are incorporating more of your senses in the musical process. The more senses you use (especially in a fleeting artistic endeavor such as mixing sound) the more your entire being is engulfed in the experience. And, the more you are engrossed in all of the details the easier it is to come up with a great mix.
More Intuitive – Pictures of sounds are one logical step closer to the music you are mixing than faders and knobs on a console. Manipulating visuals of the sounds themselves is much more intuitive than pictures of a console. Pictures of sounds are more like music than knobs.
More Conducive to the Creative Process – Studio equipment is notorious for getting in the way of the creative process. Faders and knobs on a console distract recording engineers from the music they are trying to mix. Images that are flashing to the music help the user to focus more clearly on the invisible sounds they are mixing.
More Information about the Mix – The visuals provide additional information helpful for the mix. The primary goal of the visual system has been to show “audio masking” visually. Engineers can use this information to discover hidden problems, and best of all, to be able to explain the problem to the band and producer. For example, a bad arrangement becomes clearly evident.
More Perspective – The visual framework provides the engineer with a perspective on all of the possibilities available to the engineer. The framework shows each parameter within each piece of equipment in the studio and how each parameter contributes to an overall mix. With this visual perspective of all that is possible in a mix, a humongous number of possibilities are displayed in a way that puts an array of creative ideas at an engineer’s fingertips.
More Relationships – The interface shows more than just all of the settings of each piece of equipment in the studio. The interface shows the relationships of all the settings and how they work together to create a mix. After all, it is the relationships of all the settings that really count.
Better Communication – The interface enhances and simplifies the communication process between recording engineer and client (band or producer). The interface is so intuitive that even inexperienced clients will be able to follow the development of the mix and communicate effectively with the engineer.
More Fun – And of course, we mustn’t forget — Flashing 3D visuals are a blast to watch and work with.
“The nature of the medium affects the art you create.”
• What’s it for and what does it do? The Virtual Mixer is an interface that replaces visuals of mixing consoles on the computer screen. Instead of using pictures of faders, we are using pictures of the sounds between the mix.
• Don’t visuals distract you from listening? Actually, they help you focus more on the music you are mixing. Pictures of a mixing board on a computer screen and physical faders, knobs and buttons on a console distract a listener way more. They look nothing like music. Since you have to have some type of interface, why not have one that looks more like the music that you are actually mixing. Also, when you have more different types of stimulus involving all of the senses, you are actually less distracted.
• What’s imaging? Imaging is the apparent placement of sounds between the speakers. It is where we imagine the sounds to be between the speakers.
• How does it aid in better communication? It makes it so that people who know very little about mixing can at least see what is being done every step of the way. It also helps you to focus better on the sounds in the mix — and often helps you to notice sounds that you might be missing or have forgotten.
• How’s it a teaching tool? The visuals are currently being used in the classes at Globe Institute of Recording and Production to explain and show all of the styles of mixes in the world — and to explain how to create all types of mixes with the available studio equipment.
• Why the spheres — why not pictures of instruments? Spheres best represent the reality of the dynamics that actually go on in mixing. They show the exact amount of space that each sound takes up in a mix between the speakers. This is important because of the problem of masking (where one sound hides another). Ultimately, however, you will be able to import any type of visual that you like — from instrument icons, to clouds, to crystals.
Virtual Mixer Company Information
David A. Gibson has been teaching recording in Colleges, Institutes and major recording studios for twenty years. He developed the concept of using visuals as an aid to teaching mixing theory and has refined it over the past fifteen years. Ten years ago Gibson realized that the images could be used as an interface for a computer controlled automated mixing board and began developing specifications for the current interface. In October, 1991 Gibson hired a programmer to create a basic prototype of the system on the Macintosh computer controlling a Yamaha automated mixing console. In April, 1992 Gibson applied for the patent on the use of these visuals as an interface for a complete computerized recording studio. Gibson’s book, The Art of Mixing, uses the visuals to explain and show how to mix every style of music in the world. The book is published by Mix Books and is distributed by Hal Leonard Publishing. It is currently being marketed in most of the music stores across the nation and is used in over 30 Colleges of recording around the world. Gibson has also produced a series of three educational videos which also use the visuals to show various styles of mixes. The videos are now on DVD and are also published by Mix Books. An educational CD Rom is being developed which interactively displays visuals of the functions of all the equipment in the studio, and how to use them together to make a great mix. The use of these visuals has received broad news media coverage including major news periodicals and television broadcast. Richard Hart of the show the “The Next Step” aired a spot on the use of the visuals for teaching recording. This whole visual concept is revolutionizing the field of recording education. It is similar to the advent of music theory — it opens up a whole world of aesthetics and values that have never been explored in the Industry.
The Company’s Mission
We have created a highly sophisticated yet simple tool to make mixing boards more friendly and accessible. Our goal is to establish a new standard in the industry in order to make the mixing process easier and more intuitive for recording engineers. We are committed to making the recording/mixing process less mysterious and more intuitive for the common musician and more efficient and creative for the professional recording engineer. We believe that by making the mixing process more intuitive it will make it easier for musicians and engineers to communicate better and to make better mixes more often. It is our belief that current recording technology is not only abhorrently difficult and time consuming to learn, but even when understood it still impedes the creative process. It is a classic case of the technology getting in the way of the creative process.
The Management and Location
David A. Gibson is the primary owner and creative director of the corporation. The office is located at 351 9th St. #202, San Francisco, California.
Development to Date
A working prototype using StereoGraphics’ 3D shutter glasses is currently working on the Silicon Graphics Computer is about 1/3 completed in C++.
Patents and Trademarks
Gibson has three patents for “Method and Apparatus for Using Visual Images to Mix Sound.” The patents are based on Gibson’s specific system for visually representing sound imaging. Currently, the patent office has allowed two of the eleven claims.
Besides the book and educational videos, we will also be releasing an educational CD ROM that shows various styles of mixes. But we are also planning on the following products:
A) Additional Professional Tools:
1) An Interface for mixing audio for video…
Each sphere is “attached” to an image on the screen (such as an actor’s head). As the image moves across the video stage it is automatically panned left/right. As the image moves closer, the volume is automatically raised. As the image moves into the distance, the volume is automatically lowered. An audio for video engineer commonly spends hours simply performing these moves as actors walk around on the video stage. Therefore, it will save a tremendous amount of expensive studio time.
The Interface will be licensed to current manufacturers of Audio for Video Digital Audio Workstations.
B) Consumer Interfaces:
1) Interface for controlling surround sound systems and sound cards in computers…
The consumer will use the Virtual Mixer interface on their television screen to control and visualize all controls on the surround sound system or sound card. On surround systems the consumer will also be able to put on 3D glasses to see the sound mix of 5.1 surround audio mixes, and surround sound television programs and videos
The Virtual Mixer interface will be licensed to manufacturers of surround sound systems and sound cards. The system will be bundled with current sound systems and sound cards.
2) A new standard for creating and mixing sound on the Internet …
As mixing and recording on the Internet become more commonplace, the Virtual Mixer will become the standard interface because it so intuitive and simple to use. Currently, the primary manufacturer that we would interface to is Rocket. Rocket currently creates the only interface for recording digital audio over the Internet.
C) Entertainment Products:
1) 3D Videos for home surround sound systems…
People will be able to see and hear mixes throughout the living room. Videos of different styles of music will be offered on DVD and Video. Specialty television programs will be programmed that display visuals correlating to sounds throughout the surround sound room.
2) The Virtual Mixer 3D Juke Box…
For a buck or more, you can listen to, watch, and manipulate hit songs in 3D. The 3D Juke Boxes will be displayed in arcades, theme parks (such as Universal Studios), and specialty restaurants (such as Hard Rock cafe).
3) Interactive displays for Virtual Reality type theme parks and festivals…
Similar to the Virtual Mixer 3D Juke Box, but with a full 3D projection system with interactive control. In the past year, we would have been able to have the technology at 4 major events if it was ready (and that’s without any solicitation).
4) 3D visual display of mixes for live concerts…
The Virtual Mixer interface would be marketed through lighting companies as a new and unique visual display for live concerts. The audience would don 3D glasses in order to see the mix in 3D. Images other than spheres could, of course, be utilized. All sounds would be synchronized to specific instruments being played on the stage.
5) Installations of VR Caves, Lazerium shows, etc….
The audience will be able to see and manipulate the sounds with cheap datagloves in 3D while a live band is playing. The same technology that is used for the 3D visual display of mixes for live concerts would be expanded to include dataglove technology and ultimately full Virtual Reality display. The dataglove technology will already be implemented in the Recording Studio Virtual Mixer technology.
6) Children’s Interactive Programs…
An interactive story program is currently under development based on public domain stories by Carl Sandburg. In the musical sections of the stories, each character plays an instrument in the mix that can be manipulated by the kid in 3D. We have scripts and talent ready to go.
7) Entertainment CD Roms and Games…
Visuals in 3D correlating to sound will be used in a wide range of CD Roms and Games to be developed.
8) Television shows for MTV, VH1, and other music…
Video Programs Entertaining programs on sound will be produced similar to the current Video series now on the market.
9) Interactive Television Home Mixing…
Once the bandwidth arrives, consumers will be able to actually mix the sound at home using the Virtual Mixer interface. They can, of course, have it in 3D or not.
D) Healing Products
1) Sound Healing in Virtual Reality – Sounds and visuals inside the body in Virtual Reality. MORE INFO
E) Educational Tools:
1) Educational television shows for public TV and Interactive TV…
Educational Programs on audio using the visuals as a tool will be developed. Once the bandwidth arrives these programs will be interactive.
2) Installations in High Tech Museums…
We currently have 4 invitations to place the technology in High Tech Museums: The Franklin Institute in Philadelphia, Paul Allen’s Music Experience Museum in Seattle, the Tech Museum in San Jose, and a Tech Museum in Calgary.