Course Outline of Record

Item	Value
Curriculum Committee Approval Date	12/02/2020
Top Code	061410 - Multimedia
Units	2 Total Units
Hours	54 Total Hours (Lecture Hours 27; Lab Hours 27)
Total Outside of Class Hours	0
Course Credit Status	Credit: Degree Applicable (D)
Material Fee	No
Basic Skills	Not Basic Skills (N)
Repeatable	No
Grading Policy	Standard Letter (S)

Course Description

Fundamentals and techniques of recording and post-production processing of 360-degree spatial audio specific to creating Immersive Media, such as VR/AR, Video Game Design, Sound Design for 360 Film & Television, and immersive music composing. Students will learn to use a variety of spatial microphones and recording techniques, including mono point-source, binaural and ambisonic, as well as industry-specific post-production software and plug-ins to handle 360 spatial audio mixdown in sync with immersive visuals. The techniques for spatial audio differs greatly from traditional stereo recording and mixing, making these skills absolutely necessary for creating 360 content or pursuing careers in Immersive Media. ADVISORY: FILM A193 or MUS A261. Transfer Credit: CSU.

Course Level Student Learning Outcome(s)

1. Discriminate between a variety of 360-degree spatial audio formats, content delivery methods, and use cases, from initial recording through post-production, for cinematic, Immersive Media (VR/AR), and video game applications.
2. Demonstrate a basic level of proficiency in: a. 360-degree audio recording techniques using mono, binaural, and ambisonic microphones b. 360-degree spatial audio processing and post-production within a Digital Audio Workstation (DAW) c. Appropriate placement and 'gamification' of mono, stereo, binaural, and ambisonic audio files within a Game Engine application.
3. Apply foundational knowledge of the 360-degree spatial audio workflow pipeline to enhance cinematic 360 video, virtual reality (VR), augmented reality (AR), and video game experiences.

Course Objectives

• 1. Understand the current industry trends, history, and future direction of 360 Spatial Audio.
• 2. List the basic techniques used in recording Spatial Audio.
• 3. Understand and utilize the unique recording and post-production workflow of 360 Spatial Audio for a variety of immersive projects.
• 4. Understand essential terminology used in the 360 Spatial Audio industry.
• 5. Experience and critique current Immersive Media content, including VR/AR and Games, which effectively use 360 Spatial Audio.
• 6. Demonstrate effective microphone selection and placement techniques, including mono, binaural, and ambisonic, to capture audio intended to be spatialized in 360-degrees.
• 7. Demonstrate proficiency in recording and outputting 360-degree audio using a variety of digital recording devices.
• 8. Demonstrate proficiency in 360 Spatial Audio processing and post-production techniques using a Digital Audio Workstation.
• 9. Recognize important technologies related to Spatial Audio, including middleware, plug-ins, and interactive music.
• 10. Demonstrate proficiency in quality control while recording, editing, manipulating, and mixing spatial audio to ensure a clean signal.
• 11. Demonstrate the ability to use a sound effects library in conjunction with a spatial audio project.
• 12. Discuss the chronological process and decision making tree when working as a spatial audio sound designer in the Immersive Media industry.

---

Lecture Content

I. Introduction to 360 Spatial Audio A. Brief history of audio in Radio, TV, Film, and Games B. Early Holophonics, Ambisonics, and 360 audio inventions C. The rise of Virtual and Augmented Reality, and how 360 Audio fits in D. 360 Spatial Audio in contrast to traditional Stereo and Surround Sound E. How the ears and brain processes 360 Audio vs. Stereo in a headphone   II. Basic 360 Audio Workflow for a Cinematic 360 Video A. Essential terminology B. Workflow for location recording C. Microphone and recording device selection and placement D. Mono, Stereo, Binaural, and Ambisonic microphone differences E. Workflow for output/ingestion of sound recordings into a DAW F.  Workflow for setting up a DAW for a 360 Spatial Audio session G. Spotting the script for addition sound effects, VO, and music needs H. Basic theory of creating a sound scape from start to finish I.  Saving, backing up, and delivering a final mix in multiple formats   III. Basic 360 Audio Workflow for a Gamified Immersive Experience  A. Essential terminology B. Workflow for game-engine-created projects  C. Branching narratives vs. linear storytelling, and how it effects audio D. Spotting a branching script for audio needs E. Pipeline of audio from a DAW, through middleware, to a Game Engine F.  Workflow for setting up a DAW for a Gamified 360 Spatial Audio session G. Recording/collecting sound effects and foley vs. using a sound library H. The benefits of decision trees and node-based triggering and randomization I.  Saving, backing up, and delivering the final files in multiple formats   IV. Advanced recording and processing A. Higher-order Ambisonics, quad-binaural, and future advancements B. Mixing with regular vs. HRTF-optimized headphones C. Mixing with stereo speakers vs. dome-mounted speaker arrays D. Advanced plug-ins and signal processing for 360 Audio   V. Digital Audio Workstation Breakdown A. Startup procedure B. Name and location of essential windows C. Name and location of essential menus and buttons D. Setting up a new project, naming and formatting conventions E. Personalization of windows and settings, industry-standard norms F. Importing audio content from various sources G. Working with traditional linear timelines and gamified branching timelines H. Signal processing of audio sources I. Mixing and mastering techniques J. Finalizing a project and saving to the appropriate file types for delivery   VI. Game Engine Breakdown for Audio Integration A. Startup procedure B. Name and location of essential windows C. Name and location of essential menus and buttons D. Setting up a new project, naming and formatting conventions E. Personalization of windows and settings, industry-standard norms F. Importing audio content from various sources G. Attaching both mono, stereo, and ambisonic audio files to visual 3D objects H. Spacializing a soundscape within a Game Engine I. Mixing and processing techniques within a Game Engine J. Finalizing a project and saving to the appropriate file types for delivery   VII. Related Software for Immersive Audio A. Middleware B. Plug-ins and Filters C. HRTF optimization for headphones   VIII. Understanding Auxilliary Hardware and Dev. Kits Related to Audio A. Haptic devices (gloves, vests) - Triggered by audio frequencies B. Simulators - including speaker arrays and bass vibration C. Performance Capture - Face a nd Voice data D. Alternative Display and Projection for VR E. Domes, Planetariums, Amusement Parks, Arcades F. New Developments in Immersive Media and Gaming

Lab Content

I. Recording Audio on Location for 360-degree Spatialization A. Microphone types B. Audio Recorders C. Proper placement of audio equipment and wiring D. Practice recordings with various microphone types, recorders and formats   II. Experiencing Commercially Available Immersive Content - Listening Lab A. 360-video on YouTube B. 360-video in room-scale C. VR games in room-scale D. 360 Spatial Music   III. 360-degree Camera Demo Incorporating Audio   IV. 360 Spacial Audio Post-Production A. 360 video + audio editing in a non-linear editing system B. 360 Spacial Audio using a DAW C. 360 Spatial Audio using a Game Engine D. Middleware, plug-ins, filters, and auxilliary software   VII. Haptics, Simulators, and Auxilliary Equipment Related to Audio VIII. Audio in Performance Capture A. Face Capture data B. Reference Audio from Face Capture sessions   IX. DAW and Game Engine Projects - QA Lab Time   X. FIELD TRIP  (if available)

Method(s) of Instruction

• Lecture (02)
• Lab (04)

Instructional Techniques

1. Lecture 2. Demonstration 3. Video examples 4. One-on-one instruction 5. Individual assignments 6. Group assignments 7. Assignment critique 8. Examinations

Reading Assignments

Students will Read 1-2 hours per week from assigned from handouts, equipment and software manuals, and online sources.

Writing Assignments

(none)

Out-of-class Assignments

Students will spend 3-4 hours creating 360-degree audio recordings, listening to and selecting sounds from effects libraries, building basic 360 soundscapes, and compositing 360 audio and visuals together into immersive experiences. Students will also watch online tutorials and listen to professinal 360 audio examples for reference.

Demonstration of Critical Thinking

A. Assigned individual projects B. Assigned group projects C. Examinations

Required Writing, Problem Solving, Skills Demonstration

Proficiency will be demonstrated by satisfactorily completing assignments and by incorporating the elements of 360 audio techniques they have learned. Students will demonstrate critical thinking and problem solving skills through the utilization of unique 360 spatial audio techniques, and by working through production and postproduction challenges.

Eligible Disciplines

Broadcasting technology (film making/video, media production, radio/TV): Any bachelors degree and two years of professional experience, or any associate degree and six years of professional experience. Media production (also see broadcasting technology): Any bachelors degree and two years of professional experience, or any associate degree and six years of professional experience. Multimedia: Any bachelors degree and two years of professional experience, or any associate degree and six years of professional experience.

Other Resources

1. Weekly podcasts pertaining to Immersive Media, Game Design, and Spatial Audio, as well as internet research on the topic.