# An Introduction to Computer Audio ### Will Styler <http://savethevowels.org/talks/computeraudio.html> --- ### How do acousticians say hello? - They wave! --- ### Today's Plan - Capturing Pressure variations - Computer Audio, Sampling, and Quantization - Audio Codecs and Formats - Audio Compression - Noise Reduction --- ### Sorry if you know these things! - Some of this will surely be review for many! - All might be review for some! --- ### Sound is compression and rarefaction in a medium <img class="r-stretch" src="phonmedia/sound_diagram.jpg" alt="sounddiagram.jpg The image is a diagram labeled Figure 1 that illustrates sound waves. The diagram consists of two main parts: a graph at the top showing pressure changes over time and a series of wavy lines below it representing the actual sound wave. Graph (Top Part): - X-axis: Represents distance along the length of the sound wave. - Y-axis: Represents pressure, with an upward direction indicating higher pressure and downward indicating lower pressure. The graph shows peaks and troughs that represent areas of high and low pressure in a sound wave. Sound Wave Representation: Below the graph is a series of wavy lines labeled Distance at the bottom center. These waves are divided into two sections: condensation on the left side and rarefaction on the right side. - Condensation: This section shows areas where particles in the medium (air, water, etc.) are closer together than usual, resulting in higher pressure. The peaks of these wavy lines represent condensations. - Rarefaction: This section shows areas where particles are farther apart than normal, leading to lower pressure. The troughs of the waves represent rarefactions. Key Features: 1. Wavelength (Distance): The distance between two consecutive crests or troughs is labeled as wavelength. It represents how far a sound wave travels in one complete cycle. 2. Pressure Changes: The graph above shows these changes, with peaks indicating high pressure and valleys indicating low pressure. Labels: - The term condensation points to the areas where particles are closer together (higher pressure). - The term rarefaction points to the areas where particles are farther apart (lower pressure). This diagram is a simplified representation of how sound waves travel through a medium, showing both the graphical and physical aspects of these waves. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Timeshifted sound is a novelty - For most of our species history, this wasn't a thing - *How do we capture and recreate the pattern of sound pressure?* --- ### Analog Recording - "Let's capture the pressure pattern in a physical medium" --- ### The Phonograph - Air pressure pushes a stylus into very soft wax cylinder <img class="r-stretch" src="phonmedia/phonograph.jpg" alt="phonograph.jpg The image depicts an antique Edison phonograph, a device used for playing recorded sound from cylinders. The phonograph is made of wood with a dark finish and has a metal crank on its right side that is used to turn the cylinder. At the top center of the machine, there's a large, curved horn designed to amplify the sound produced by the needle as it moves along the grooves of the cylinder. The cylinder itself is not visible in this image; instead, we see the mechanism at rest. The phonograph has a rectangular base with a label that reads Edison on its front panel. The overall design suggests an early 20th-century style, reflecting the technology's historical context and the craftsmanship of the era. The horn is metallic and shiny, contrasting with the darker wood of the base. It curves outward in a bell shape to enhance sound projection. The crank handle appears to be made of metal as well, possibly brass or another similar material, and it protrudes from the side for easy operation by hand. There are no people present in this image; only the phonograph is shown against a plain white background. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Playback from Phonographs - Put a stylus on a membrane into the groove, and let it 'trace the wave' <img class="r-stretch" src="phonmedia/phonograph2.jpg" alt="phonograph2.jpg The image depicts a vintage mechanical device that appears to be an old-fashioned sewing machine or possibly a similar type of industrial equipment used for crafting or manufacturing purposes. The device is predominantly metallic with some parts having a polished finish and others appearing slightly worn, suggesting it might be quite old. Here are the details: 1. Base: The base of the machine is rectangular and made of wood, which has a dark brown color and shows signs of wear, indicating age. 2. Mechanical Components: - There's a large, circular wheel at the center that looks like it could be part of the mechanism for winding thread or operating the device. - To the left side of this central wheel is another cylindrical component with a metallic finish and what appears to be a spool of thread on top. This suggests that the machine might involve threading as part of its operation. - On the right side, there's a lever mechanism with a handle attached to it. This could be used for operating or adjusting parts of the device. 3. Text: There is some text visible on the base of the machine near the spool holder. The text appears to read Charles Burrell, which might indicate the brand name or manufacturer of this device. 4. Thread and Spool Holder: A black thread is fed into a spool holder, which is part of the mechanism. This suggests that the device could be used for sewing or some form of textile work. The overall appearance of the machine gives it an antique look, with its metallic parts showing signs of age and use. The wooden base adds to this vintage aesthetic. There are no people in the image; only the mechanical components are visible. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### These recordings are ephemeral and bad - The stylus wears away the groove - The power of the air pressure limited the strength of the medium 'The Lost Chord' by Arthur Sullivan (1888) <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> --- ### There's an inherent tradeoff - You want a soft medium for capture - ... and a hard medium for playback - Air pressure only provides so much power --- <img class="r-stretch" src="phonmedia/conerecording.jpg" alt="conerecording.jpg The image depicts two individuals standing outdoors near a large, conical object that appears to be some kind of industrial equipment or machinery. The setting seems to be an open area with a rough, possibly gravelly ground. Description of the Individuals: 1. Person on the Left: - This individual is wearing a light-colored jacket and dark pants. - They have short hair and are standing in front of a stool that has a conical object resting on it. - The person appears to be holding or interacting with something small, possibly tools or parts related to the machinery. 2. Person on the Right: - This individual is wearing a vest over a long-sleeved shirt and dark pants. - They have short hair and are standing slightly behind the first person. - The second person seems to be observing the activity of the first person, possibly assisting or supervising. Description of the Conical Object: - The conical object is large and metallic in appearance. It has a rough texture with visible wear marks, suggesting it might have been used for some industrial process. - The object appears to be suspended by ropes or cables, indicating that it may not be stationary but could be moved or adjusted. Background: - The background shows an open area with what looks like a wall or structure made of stone or concrete blocks. This suggests the setting is likely in an industrial or construction site. - There are no other significant objects or people visible in the immediate vicinity, focusing attention on the two individuals and the conical object. Overall Context: The image seems to capture a moment where these two individuals might be working together on some form of maintenance, inspection, or operation involving the large conical machinery. The setting and their attire suggest an industrial environment from a past era, possibly early 20th century based on the style of clothing. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Electric Recording fixes this! - Electrical signals are easy to amplify - ... and easier to store --- ### Microphones - A Microphone *transduces air pressure patterns into electrical patterns* - 'Give me a pattern of voltage that matches the pattern of compression and rarefaction' <img class="r-stretch" src="phonmedia/sound_diagram.jpg" alt="sounddiagram.jpg The image is a diagram labeled Figure 1 that illustrates sound waves. The diagram consists of two main parts: a graph at the top showing pressure changes over time and a series of wavy lines below it representing the actual sound wave. Graph (Top Part): - X-axis: Represents distance along the length of the sound wave. - Y-axis: Represents pressure, with an upward direction indicating higher pressure and downward indicating lower pressure. The graph shows peaks and troughs that represent areas of high and low pressure in a sound wave. Sound Wave Representation: Below the graph is a series of wavy lines labeled Distance at the bottom center. These waves are divided into two sections: condensation on the left side and rarefaction on the right side. - Condensation: This section shows areas where particles in the medium (air, water, etc.) are closer together than usual, resulting in higher pressure. The peaks of these wavy lines represent condensations. - Rarefaction: This section shows areas where particles are farther apart than normal, leading to lower pressure. The troughs of the waves represent rarefactions. Key Features: 1. Wavelength (Distance): The distance between two consecutive crests or troughs is labeled as wavelength. It represents how far a sound wave travels in one complete cycle. 2. Pressure Changes: The graph above shows these changes, with peaks indicating high pressure and valleys indicating low pressure. Labels: - The term condensation points to the areas where particles are closer together (higher pressure). - The term rarefaction points to the areas where particles are farther apart (lower pressure). This diagram is a simplified representation of how sound waves travel through a medium, showing both the graphical and physical aspects of these waves. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Dynamic Microphones - Air pressure pushes a membrane, moving a coil of wire around a magnet, inducing voltage - Durable, but less sensitive <img class="r-stretch" src="phonmedia/mic_dynamic.jpg" alt="micdynamic.jpg The image is a diagram illustrating the cross-section of a dynamic microphone. The diagram includes labels for various components within the microphone. - Sound Waves: These are depicted as wavy lines on the left side of the diagram, indicating the input sound waves that enter the microphone. - Diaphragm: This component is shown at the bottom of the diagram and appears to be a thin, flexible membrane. It vibrates in response to incoming sound waves. - Coil: Positioned above the diaphragm, this is a coiled wire that moves when the diaphragm vibrates due to sound waves. - Magnet: Located near the top of the diagram, it has a cylindrical shape and surrounds the coil. The magnet's magnetic field interacts with the moving coil. - Wires carrying electrical audio signal: These are shown at the very top of the diagram, extending from the coil area. They carry the electrical signals generated by the movement of the coil in response to sound waves. The overall structure is a rectangular shape, which represents the housing or casing of the microphone. The labels and arrows provide clear guidance on how each part functions within the microphone's operation. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Condenser Microphones - Air pressure pushes one plate closer to another, producing changes in capacitance - This can then be amplified using external ('phantom' or 48v) power for output - More sensitive, but more fragile too! <img class="r-stretch" src="phonmedia/mic_condenser.jpg" alt="miccondenser.jpg The image is a diagram illustrating how sound waves interact with a microphone's internal components. The diagram includes labels for each part of the microphone and arrows indicating the direction of movement. 1. Sound Waves: On the left side, there are two curved lines labeled Sound Waves. These represent the incoming sound that the microphone is designed to capture. 2. Front Plate (Diaphragm): This component is shown as a thin, flexible plate within the microphone. It is depicted in gray and positioned at the center of the diagram. The front plate vibrates when it encounters sound waves. 3. Back Plate: Located behind the Front Plate (Diaphragm), this part is labeled Back Plate and is shown in black. It serves as a boundary for the vibrations caused by the sound waves hitting the diaphragm. 4. Battery: Positioned at the bottom right of the diagram, there's a rectangular shape labeled Battery. This represents the power source that powers the microphone to convert mechanical energy into electrical signals. 5. Output Audio Signal: At the top right corner, an arrow points downward and is labeled Output Audio Signal. This indicates where the electrical signal generated by the microphone would be sent after conversion from sound waves to electricity. The diagram uses red arrows to show how the sound waves move towards the Front Plate (Diaphragm) and then how this movement is translated into an output audio signal. The battery provides energy for this process, enabling the microphone to function effectively in converting sound vibrations into electrical signals that can be amplified or recorded. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Now you have sound as a voltage on an electrical line - You can amplify it, transmit it, modify it and store it - You can even recreate the air pressure movements --- ### Speakers - Dynamic microphones in reverse - Changes in voltage move a membrane attached to a coil - This 'kicks' the air in the desired pattern of compression <img class="r-stretch" src="phonmedia/speaker.jpg" alt="speaker.jpg The image is a detailed diagram illustrating how an audio speaker works. It shows various components labeled with arrows pointing to each part of the mechanism. 1. Electrical Signals to Coil: At the top left corner, there's a yellow wire that represents electrical signals being sent from some source (not shown in this diagram). These signals are directed towards the coil. 2. Coil Makes Magnet Vibrate: Moving down and slightly to the right, we see a coiled wire labeled as coil makes magnet vibrate. This is where the electrical signal causes the coil to move. 3. Magnet Attached to Cone: Below the coil, there's a blue object that looks like a magnet. It’s labeled “magnet attached to cone.” The magnet is connected to another part of the speaker mechanism. 4. Cone Vibrates and Makes Sound: At the bottom right corner, we see a large, light-colored cone-shaped structure labeled cone vibrates and makes sound. This is where the vibrations from the magnet cause it to vibrate, producing sound waves that travel through the air. The diagram uses arrows to indicate how each part interacts with one another. The coil moves due to electrical signals, which in turn causes the magnet attached to the cone to move. As this happens, the cone vibrates and produces sound. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### There are many types of speakers, some are different! <img class="r-stretch" src="phonmedia/speakers.jpg" alt="speakers.jpg The image is a side-by-side comparison of two different speaker systems. On the left side, there are two tall, freestanding speakers with a light wood finish. Each speaker has three circular drivers: one large woofer at the bottom and two smaller woofers above it, positioned symmetrically on either side. The top part of each speaker is black, where the tweeter (the smallest driver) is located. There are also visible control knobs or switches near the base of each speaker. On the right side, there is a larger, more boxy speaker system with a dark gray finish. This setup includes multiple drivers: two large circular woofers at the bottom and four smaller circular drivers arranged in pairs on top. The speakers appear to be stacked vertically, suggesting they are part of a multi-speaker arrangement. The background for both sets of speakers is plain, allowing clear visibility of their design details. There's no text or diagrams present within these images that provide additional information about the speakers' specifications or brand names. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ## Any Questions so far? --- ### So, that's how we capture sound - ... and that's how we worked with sound for a good while! --- ### But then everything changed <img class="r-stretch" src="img/appleii.jpg" alt="appleii.jpg The image displays a vintage computer system set against a plain white background. The equipment consists of three main components: a central computer unit and two disk drives stacked to its right. On the left is the main computer, which appears to be an Apple IIe. It features a beige plastic casing with a built-in keyboard at the base that slopes upward toward the back. The keys are grey and arranged in standard rows. Just above the keyboard area, there is a rectangular label featuring the classic rainbow-colored Apple logo followed by the text Apple IIe. Resting on top of this main chassis is a separate monitor unit, also beige. It houses a CRT screen surrounded by a dark grey bezel. The screen itself is dark and inactive, but at the very top center of the display area, faint green text reads APPLE II. To the right of the computer are two identical disk drive units stacked vertically on top of one another. These are Apple Disk II drives. They have beige casings that match the main computer unit. Their front panels are dark grey. Each drive features a horizontal slot for inserting floppy disks, with a black lever mechanism in the center used to eject the disk. Below the slot on each drive, white text reads disk II. To the right of this text is another rainbow-colored Apple logo. On the far left side of the front panel for each drive, small red indicator lights are visible. The lighting in the image creates soft shadows beneath and to the right of the equipment. This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ## Computer Audio --- ### Computers don't do waves <img class="r-stretch" src="phonmedia/sampling_raw.jpg" alt="samplingraw.jpg The image is a line graph that appears to represent some form of data over time. The x-axis represents time, while the y-axis likely represents a measurement or value associated with the data being plotted. Key Features: 1. Axes: - The horizontal axis (x-axis) has labeled intervals, though they are not clearly marked in this description. - The vertical axis (y-axis) is labeled but the specific units of measurement are not provided here. 2. Data Points and Trends: - There are numerous data points plotted on the graph, which form a jagged line that fluctuates up and down across the y-axis. - The fluctuations suggest variability in the measured values over time. - The line does not show any clear upward or downward trend; instead, it appears to be quite erratic. 3. Gridlines: - There are gridlines on both axes for reference, helping to locate specific points on the graph more accurately. 4. No Text or Labels: - There is no text or labels that provide context about what the data represents. - The absence of a title and axis labels makes it difficult to interpret the exact nature of the data being plotted without additional information. People in the Image: - This image does not contain any people. It is purely a graphical representation of data with no human figures or characters depicted. If you need further assistance understanding this graph, please provide more context about what kind of data it might represent (e.g., financial data, scientific measurements, etc.), and I can help interpret the specific details for you. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> 010001110010101000100101101010101010 --- ### Sound is analog, computers are digital - How do we deal with that? --- ### Quantization - Also known as 'digitization', 'discretization', or 'sampling' - "Let's just measure the sound a LOT and store those values" --- ### Quantization <img class="r-stretch" src="diagrams/sampling_perfectwave.png" alt="samplingperfectwave.png The image displays a mathematical graph plotted on a plain white background. Two thin, light gray lines form the coordinate axes: a vertical y-axis running up the left side and a horizontal x-axis extending across the middle of the frame. They intersect at the bottom left corner to form an origin point. A thick, solid black line represents a wave function, specifically resembling a sine wave. The line begins exactly at the intersection of the two axes. It curves smoothly upward and to the right, forming a rounded peak in the upper left quadrant. After reaching its maximum height, the line descends, crossing the horizontal axis near the center of the image. Continuing downward into the lower right area, the line forms an inverted curve or trough, reaching This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Quantization <img class="r-stretch" src="diagrams/sampling_sampled.png" alt="samplingsampled.png This image displays a mathematical line graph plotted on a plain white background. The graph consists of two light grey axes: This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Quantization <img class="r-stretch" src="diagrams/sampling_withlines.png" alt="samplingwithlines.png The user wants a detailed description of an image for a blind student. The image is a graph showing a sine wave and vertical bars underneath/above it. Key Elements to Describe: 1. Axes: There are faint grey axes on the left (vertical) and center (horizontal). They form a coordinate system. 2. Curve: A thick black line forms a smooth, continuous wave. It starts at the origin (intersection of axes), goes up to a peak, comes down crossing the horizontal axis, goes down to a trough (valley), and comes back up to the horizontal axis. This is a classic sine wave or cosine wave shape. 3. Data Points: Along the black curve, there are distinct black dots marking specific intervals. Let's count them roughly to be precise if possible, but regularly spaced is key. Looking closely at the top half (positive area), there are about 20-25 points. Looking at the bottom half (negative area), there are a similar number. 4. Vertical Bars: There are vertical grey bars connecting the horizontal axis to the black curve. In the upper section (above the axis), the bars extend upwards from the axis to the curve. They look like a fence or comb. In the lower section (below the axis), the bars extend downwards from the axis to the curve. 5. Symmetry: The wave is symmetric. The top hump looks like an inverted version of the bottom dip. 6. Text/Diagrams: There is no text, numbers, or labels visible on the graph itself. It's a pure mathematical visualization. Drafting the Description - Step-by-Step: Introduction: Start with the overall layout. It's a line graph on a white background. Axes: Mention the light grey lines forming an L-shape (vertical y-axis and horizontal x-axis). The origin is where they meet on the left side. The Wave: Describe the thick black curve. It starts at the origin, rises to a maximum height, curves back down crossing the horizontal line, reaches a minimum depth, and returns to the horizontal line. This represents one full period of a wave (like a sine wave). Points on the Curve: Note that there are solid black dots placed at regular intervals along this curve. These mark specific data points or samples. The Vertical Lines/Bars: Describe the grey vertical lines. They connect the horizontal axis to the curve. Above the axis: They form a series of parallel vertical bars rising from the line up to the curve, creating a shape that looks like a filled area under the curve but made of discrete bars. Below the axis: Similarly, they extend downwards from the axis to the bottom of the wave. Visual Metaphor: It looks like a comb or a fence where the top edge is curved. Or perhaps a visualization of sampling a signal. Refining for Accessibility: Avoid left/right without context if possible, but standard graph reading implies left-to-right progression. Be precise about colors: Black curve/dots, grey bars/axes, white background. Structure the description logically from general to specific. Final Polish (Mental Check): Did I mention the dots? Yes. Did I mention the vertical bars? Yes. Did I describe the shape of the wave? Yes. Is there text? No. Is it accurate? Yes. Let's write it out clearly. Start with the axes. Describe the black curve (sine wave). Describe the dots on the curve. Describe the grey vertical bars connecting the axis to the curve. Note the lack of text/labels. (Self-Correction during drafting): I should make sure to clarify that the vertical lines are distinct from the axes. The axes are faint grey, the vertical bars are a slightly darker grey but still lighter than the black line. Wait, looking closer, the vertical bars look like a medium grey, and the axes are very light grey. The curve is thick black. Let's count the dots roughly to add detail This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Quantization <img class="r-stretch" src="diagrams/sampling_justlines.png" alt="samplingjustlines.png The user wants a description of an image for a blind student. The image is a graph showing a waveform. It looks like a bar chart representation of a sine wave or similar periodic function. Key Elements to Describe: 1. Axes: There's a vertical y-axis on the left and a horizontal x-axis running through the middle. They are light gray lines against a white background. 2. Data Points (Bars): The graph consists of many thin, vertical gray bars. 3. Pattern/Shape: Starting from the left, the bars grow taller upwards. They reach a peak height around the middle-left section. Then they decrease in height back down to the axis line. After crossing the axis, the bars extend downwards. They grow longer (downwards) reaching a maximum depth. Finally, they shorten again as they approach the right edge. 4. Symmetry: The upward curve and downward curve look somewhat symmetrical but not perfectly so in terms of bar count or exact height, suggesting it might be a specific signal or just a stylized wave. Wait, looking closer at the transition from positive to negative. Positive side (above axis): There are about 23 bars. The peak is around the 10th-12th bar. Negative side (below axis): There are about 24 bars. The deepest point is around the middle of this section. 5. Text/Labels: There is absolutely no text, numbers, or labels on the axes or title. It's a raw plot. Drafting the Description: Start with the overall layout: A graph on a white background. Describe the axes: Light gray vertical line on the far left and a horizontal line crossing the middle. Describe the bars: Thin, dark gray vertical lines. Describe the progression from left to right. Left side (positive): Bars start short, grow taller in an arch shape, peak, then shrink back to zero height. Right side (negative): Bars start appearing below the axis line. They get longer downwards, reach a maximum length, and then shorten again as they approach the right edge. Mention the lack of text/numbers. Refining for Blind Student: Use clear directional language (left to right). Describe relative heights clearly (tallest, shortest). Avoid vague terms like looks like a wave without explaining why (the bars getting taller and shorter). Be precise about the visual structure. Detailed Count/Estimation: Let's count the positive bars roughly: 1, 2, 3... up to about 24 bars before it hits zero? No, let's look at the transition. The bars go up, peak, come down. Then there is a tiny bar below the line, then longer ones. Actually, looking very closely at the center: There is a distinct break. The positive hump ends. Then there are negative bars. Let's just describe the visual flow rather than exact counts unless it's obvious. It looks like roughly 25-30 bars total. Final Polish: The image displays a graph plotted on a white background with light gray axes. A vertical axis runs along the far left edge, and a horizontal axis extends across the middle of the frame. The data is represented by a series of thin, dark gray vertical bars arranged side-by-side. Starting from the left, the bars begin at a short height above the horizontal axis. As you move to the right, the bars progressively increase in height, forming an arch or bell-like curve. They reach their maximum height near the center-left portion of the graph. Following this peak, the bars gradually decrease in height until they touch the horizontal axis line. Immediately after touching the axis, the pattern reverses direction downwards. The bars now extend below the horizontal axis. They progressively increase in length (downward), forming an inverted arch or U-shape that mirrors the first part of the graph but is shifted to the right. The longest downward bar appears near the center-right of the image This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Analog-to-digital conversion - Sample the wave many times per second - Record the amplitude at each sample - The resulting wave will faithfully capture the signal --- ### How often do we sample? - This is called the 'Sampling Rate' - Measured in samples per second (Hz) --- ### Sampling Rate <img class="r-stretch wide" src="phonmedia/sampling_quantized.jpg" alt="samplingquantized.jpg The image depicts a graph with an x-axis and y-axis. The x-axis is labeled at intervals of 10 units from -50 to +40. The y-axis has no label but appears to be scaled in increments that are not explicitly marked, making it difficult to determine the exact scale. The graph shows a smooth curve that starts at the origin (0,0) and rises to a peak before descending back towards zero. There is a series of dots connected by lines forming this curve. The curve has a symmetrical shape, resembling a sine wave or a cosine wave, which typically represents periodic behavior in mathematics. The curve reaches its highest point around x = 15 on the x-axis and then gradually decreases to reach another low point near x = -30 before leveling off again towards zero at x = +40. The curve is smooth with no sharp turns or breaks, indicating a continuous function without abrupt changes in direction. There are no other elements such as text, numbers, or additional diagrams present on the graph itself. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate <img class="r-stretch" src="diagrams/sampling_sampled.png" alt="samplingsampled.png This image displays a mathematical line graph plotted on a plain white background. The graph consists of two light grey axes: This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate (low rate) <img class="r-stretch" src="diagrams/sampling_lowrate.png" alt="samplinglowrate.png The user wants a detailed description of an image showing a line graph. The target audience is a blind student, so I need to be descriptive and spatial without relying on visual metaphors that don't translate well (like looks like a mountain). I need to focus on coordinates, relative positions, and the sequence of points. Image Analysis: - Type: It's a line graph or a plot of data points connected by straight lines. - Axes: There are two axes visible. A vertical y-axis on the left and a horizontal x-axis running through the middle. They intersect at the origin (0,0) roughly, though there are no numbers. The background is white. The axes are light gray. - Data Points: There are distinct black dots marking specific points along the line. Let's count them from left to right. 1. Point 1: On the y-axis (x=0), at a height of roughly 50% up the visible graph area. Wait, looking closer, it's actually on the intersection of the axes? No, let's re-evaluate. - The horizontal line is the x-axis. The vertical line is the y-axis. - Point 1: Located exactly at the intersection of the y-axis and the x-axis. Wait, no. Let's look really closely. - Actually, looking at the leftmost point, it sits on the vertical axis. It seems to be slightly above the horizontal line? No, let's look at the intersection. The horizontal gray line goes all the way across. The vertical gray line goes up and down. - Let's trace the points from left (Point 1) to right (Point 8). Tracing the Points: - Point 1: Located on the y-axis. It is exactly on the horizontal axis line. Wait, let me look again. The black dot is at the intersection of the vertical gray line and the horizontal gray line? No, looking very closely at crop 4, the first dot is on the vertical line. Is it above or below the horizontal line? - Actually, looking at the full image, the first dot is on the y-axis. It looks like it's on the x-axis line. Let's assume coordinates (0,0) for simplicity unless I see otherwise. Wait, let me look at crop 1 and 4 again. - In crop 4, there is a black dot on the vertical gray line. The horizontal gray line passes right through it. So Point 1 is at (0,0). - Point 2: Moving up and to the right. It's high up in the first quadrant (top left area relative to center). Let's call this x=1, y=high positive. - Point 3: Moving further right and slightly higher. This is the peak of the graph. Highest point. x=2, y=highest positive. - Point 4: Moving down and right. Still in the first quadrant (above the axis). x=3, y=medium positive. - Point 5: Moving down and right, crossing the horizontal axis. Wait, let's look at the line segment between Point 4 and Point 6. - Let's recount carefully. - Start: Leftmost dot on the vertical axis. It is exactly on the horizontal gray line. So (0,0). - Next dot up-right: High positive y. - Next dot up-right: Highest positive y (Peak). - Next dot down-right: Positive y, but lower than peak. - Next dot down-right: Negative y? Let's look at the crossing. The line goes from the 4th point (positive) to a point below the axis. Wait, let me count the dots again. - Dot 1: On vertical axis, on horizontal axis. (0,0). - Dot 2: Up and right. High positive. - Dot 3: Further right and slightly higher. Peak. - Dot 4: Down and right. Still above axis. - Dot 5: Down and right. Below the axis? Let This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate (low rate) <img class="r-stretch" src="diagrams/sampling_lowratejustlines.png" alt="samplinglowratejustlines.png The user wants a detailed description of the provided image for a blind student. I need to describe: - The overall structure (a line graph). - The axes (x and y). - The data points (black dots) and their relative positions. - The connecting lines (thick black lines). - The vertical grid lines (thin grey lines). - Any text or labels (there are none visible). Let's break down the graph from left to right. 1. Axes: There is a horizontal axis (x-axis) and a vertical axis (y-axis) meeting at the bottom left, though the origin isn't explicitly marked with a zero. The axes are light grey lines. 2. Data Points: I see distinct black dots connected by thick black lines. Let's count them and describe their trajectory. - Point 1: Starts on the y-axis (or very close to it), at a mid-level height. Wait, looking closely, the first point is actually on the horizontal axis line? No, let's re-evaluate. - Let's look at the intersection of the axes. The vertical grey line goes up from the bottom left corner. The horizontal grey line goes right. - Point 1: Located on the y-axis (the vertical grey line). It is roughly halfway up the visible height of the graph area. Wait, no. Let's look at the very first dot. It's on the far left edge. It seems to be on the horizontal axis line? No, looking closely at crop 4 and 1, there is a dot on the vertical axis line. Let's call this Point A. - Actually, let's trace the path. - Start: Far left. There is a black dot on the vertical grey line. It looks like it's at y=0 relative to the main oscillation, or maybe slightly above. Let's assume standard graph coordinates. The horizontal line acts as an axis (y=0). - Point 1: On the far left vertical axis. It is on the horizontal grey line? No, looking at crop 4, there is a dot on the vertical line. It looks like it's slightly above the bottom edge but definitely below the peak. Wait, let me look at the full image again. - Okay, let's trace carefully from left to right. - Point 1: Located on the far left vertical grey line. It is positioned roughly in the middle of the vertical range shown. Actually, looking at the horizontal grey line that runs across the middle... Point 1 is on that horizontal line? No, let's look really closely. The dot is on the vertical axis. Is it above or below the horizontal axis? - Let's re-examine the whole structure. It looks like a wave. - There is a central horizontal grey line. - To the left of this line, there are points above it. - To the right, there are points below it. - Let's look at the very first point on the far left (on the y-axis). It is actually on the horizontal grey line? No, looking at crop 4, the dot is clearly visible on the vertical axis. Is it above or below the horizontal line? The horizontal line passes through... wait. - Let's look at the intersection. The horizontal grey line seems to pass through a point? No. - Let's look at the first segment. It goes UP from the start point. So the start point is lower than the next point. - Let's assume the horizontal grey line is the x-axis (y=0). - Point 1: On the y-axis. It looks like it is on the x-axis? No, looking at crop 4, there is a dot on the vertical axis. The horizontal line passes through that dot? Or just below it? Let's assume for a moment it starts at (0,0) or close to it. - Actually, let's look at the valley and peak. - There is a peak on the left side. - There is a valley This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate (lower rate) <img class="r-stretch" src="diagrams/sampling_verylow.png" alt="samplingverylow.png The user wants a description of an image for a blind student. The image is a line graph on a Cartesian coordinate system. I need to describe the axes, the data points, and the connecting lines in detail. Visual Analysis: - Axes: There are two light gray lines forming an L-shape. One vertical (y-axis) on the left, one horizontal (x-axis) near the middle. They intersect at a point on the left side. Wait, looking closer, the intersection is actually slightly below the first data point. Let's re-evaluate the axes. - Vertical axis: A light gray line running up the left edge. - Horizontal axis: A light gray line running across the middle. - The origin (intersection) seems to be where the vertical line meets the horizontal line. - Data Points: There are 6 distinct black dots connected by thick black lines. Let's trace them from left to right. 1. Point 1: Located on the y-axis, slightly above the x-axis intersection. Wait, looking very closely at the left edge... The vertical line goes all the way up. The horizontal line crosses it. The first point is actually on the vertical axis, just above the horizontal axis line. No, let's look again. - Actually, the first point is on the y-axis (the vertical gray line). It is slightly above the x-axis (the horizontal gray line). Let's call this Point 1. 2. Point 2: The line goes up and to the right to a peak. This is the highest point on the graph. 3. Point 3: The line goes down and to the right, but not all the way back to the axis level yet. It's still above the x-axis. 4. Point 4: The line continues steeply down and to the right, crossing below the x-axis. This is a point in the lower quadrant. 5. Point 5: The line goes slightly further down and to the right. This looks like the lowest point (minimum) on the graph. Wait, let me re-examine the sequence. Let's trace carefully from left to right: - Start: There is a dot on the far left, sitting exactly on the vertical gray line. It is slightly above the horizontal gray line. Let's call this (0, y1). - Next: The line goes up and right to a peak. This is the highest point. Let's call this (x2, ymax). - Next: The line goes down and right to a point that is still above the horizontal axis. Let's call this (x3, y3). - Next: The line goes steeply down and right, crossing the horizontal axis. It ends at a dot below the axis. Let's call this (x4, y4). - Next: The line continues down and right to the lowest point on the graph. This is the minimum. Let's call this (x5, ymin). - End: The line goes up and right to a final dot. This dot sits exactly on the horizontal gray line at the far right edge. Wait, let me look at the image again very carefully. - Leftmost point: On the vertical axis. It is on the horizontal axis? No, looking closely at the intersection... The horizontal line passes through a dot. Let's re-evaluate. - Actually, there are dots on the axes. - Dot 1: Far left. It is on the vertical gray line. It is also on the horizontal gray line. Wait, no. The horizontal line goes through it. So Dot 1 is at the intersection of the visible axes? No, looking at the very bottom left corner... The vertical line extends down. The horizontal line extends right. - Let's look at the dot on the far left. It is on the vertical gray line. Is it on the horizontal gray line? Yes, the horizontal gray line passes directly through this dot. So Point 1 is at (0,0) relative to the visible axes intersection. This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate (lower rate) <img class="r-stretch" src="diagrams/sampling_verylow2.png" alt="samplingverylow2.png The user wants a description of an image for a blind student. The image is a line graph on a Cartesian coordinate system. Key Elements: 1. Axes: There are two light gray axes. A vertical y-axis on the left and a horizontal x-axis running across the middle. They intersect at the origin (0,0). 2. Data Points/Vertices: There are four distinct points marked by black dots connected by black lines. Let's trace them from left to right. Point 1: Located on the y-axis, slightly above the x-axis intersection? No, looking closely, it looks like it starts exactly at the intersection of the axes or very close to it. Wait, let me re-evaluate. The horizontal line is an axis. The vertical line is an axis. Let's look at the first point on the far left. It is on the y-axis. It seems to be slightly above the x-axis? No, looking at the intersection, the horizontal gray line crosses the vertical gray line. The black dot is right on that intersection. Wait, let me look closer. Actually, looking very closely at the left side: There is a black dot on the y-axis. It looks like it's slightly above the x-axis? No, the horizontal gray line goes through it. Let's assume it's at (0, 0) or close to it. Wait, let me look at the slope. The first segment goes UP and RIGHT. So the starting point must be lower than the peak. Let's re-examine the axes. There is a vertical gray line on the far left. There is a horizontal gray line running across the middle. They intersect. Point 1: On the y-axis (vertical line). It looks like it is exactly at the intersection of the two gray lines. So, coordinate (0,0) roughly. Point 2: The peak. It's high up and to the right. This is a local maximum. Point 3: The trough. It's low down and further to the right. This is a local minimum. Point 4: The end point. It's on the far right. It looks like it is at the same vertical level as the starting point (Point 1). Let's refine the coordinates/positions: Start Point: On the y-axis, seemingly at the intersection with the x-axis. Let's call this (0, ystart). Actually, looking really closely, the horizontal gray line passes through the dot on the left. So it's on the x-axis. Wait, no. The horizontal gray line is an axis. The vertical gray line is an axis. The black dot is at their intersection. Let's assume origin (0,0). Peak Point: To the right and up. It looks like roughly x=2, y=3 (in arbitrary units). Trough Point: Further right and way down. It crosses the x-axis on its way down. The lowest point is at roughly x=5, y=-4. End Point: Further right. It goes back up to meet the level of the start point. So it's at roughly x=7, y=0. Let's look again. The horizontal gray line is clearly an axis. The vertical gray line on the left is clearly an axis. The first black dot is on the vertical axis. It is on the horizontal axis line? No, looking very closely at crop 1 and 4... The horizontal gray line goes through the first dot. So the first point is at (0,0). Then the line goes up to a peak. Then it goes down steeply, crossing the x-axis. Then it hits a minimum (trough). Then it goes up to the final point on the right. The final point on the right is also on the horizontal gray line. So it ends at y=0. So the graph starts at (0,0), goes up to a peak, down below the axis to a trough, and back up to the axis at the end. Wait, let me look at the first dot again. In the full image This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Sampling Rate <img class="r-stretch" src="diagrams/sampling_sampled.png" alt="samplingsampled.png This image displays a mathematical line graph plotted on a plain white background. The graph consists of two light grey axes: This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Bad sampling makes for bad waves <img class="r-stretch wide" src="phonmedia/sampling_undersampled.jpg" alt="samplingundersampled.jpg The image consists of two main sections labeled A and B. Section A: - There are two overlapping waveforms depicted in this section. - The top waveform is a light gray color and appears smoother with less pronounced peaks and troughs. It represents one type of wave pattern, possibly an audio or sound wave. - Below it, there is a darker blue waveform that has more defined peaks and troughs compared to the gray one. This could represent another form of wave, such as a mechanical or electrical wave. Section B: - This section shows a series of repeating patterns below the waveforms in A. - The pattern consists of alternating light gray and dark blue shapes that resemble sine waves. - Each cycle of these waves is divided into segments by vertical lines. These lines are evenly spaced, indicating regular intervals between each segment. - There are small white squares at the beginning and end of each segment, which might represent points of measurement or reference. Overall: The image seems to be illustrating different types of waveforms and their periodic patterns. The top part (A) shows two waveforms side by side for comparison, while the bottom part (B) provides a detailed view of one type of waveform with marked intervals. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ## Nyquist Theorem The highest frequency captured by a sample signal is one half the sampling rate --- ### Sampling Rates (Shpongle - 'Nothing is something worth doing') 44,100 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 22,050 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 11,025 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 6000 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> --- ### Sampling Rates (Shpongle - 'Nothing is something worth doing') 44,100 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 6000 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 3000 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 1500 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 800 Hz <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> --- ### Different media use different sampling rates - Radio was historically less than this - CDs are at 44,100 Hz - DVDs are at 48,000 Hz - High-End Audio DVDs are at 96,000 Hz - Some people want 192,000 Hz - Likely they are dolphins --- ### Your sampling rates should be at least 44,100 - This covers the range of human hearing entirely - You can go higher, but don't go lower! --- ### ... but what are we storing at each point? - We want to store individual values for amplitude - We want to store values with enough precision to capture the wave well - 0.1 vs. 0.09 vs. 0.087 vs. 0.0866 vs. 0.08659 vs. 0.086588945372912 - ... but more precision means more numbers (which need more space to store!) - We need to find the right **bit depth** --- ### Bit Depth - How many bits of amplitude information do we store for each sample? - 4 bits gives 16 'levels' - 16 bits gives 65,563 levels - Praat records and plays at 16 bit, as do most things - 24 bits gives 16,777,216 levels - This is towards our upper limit of precision to be able to capture - **Bit Depth != Bit Rate!** --- ### Your bit depth will likely be 16 bit - If it's not spoken of, it's 16 bit - There's no reason to go higher, practically - ... and you'll run into compatibility issues - Don't go lower! --- ### This all means that 'vinyl captures more detail' people are provably wrong - Any audible audio signal can be captured digitally, c.f. the nyquist theorem - We can capture greater bit depth than we can hear - 'More detail' means 'the noise and distortion I appreciate' - **Audiophiles are generally slightly insane** --- ### This is what your 'sound card' or 'USB capture box' does - "ADC" or "AD" chips go from analog signals to digital samples - "DAC" or "DA" chips reverse the process, and create analog signals from digital samples - Every digital device that uses sound needs both - Other components provide (e.g.) level control, mixing, phantom power, different inputs - They can vary massively in quality - This is why you spend money on a decent capture card or sound card --- ### Capturing the samples into a file gives you uncompressed sound files! - WAV files are effectively large lists of amplitudes, with a sampling rate and channel info at the top - AIFF is the same idea, but Apple's own format - You can freely and *losslessly* turn WAV into AIFF and vice versa - This distinction doesn't actually matter - You should be a bit scared of any device which won't give you WAV or AIFF or FLAC - ... and if you're recording video data, check which format the audio is using! --- ### You should save your data files as WAV when possible - Disk space is ridiculously cheap - Not all software supports all filetypes - ... but they will support WAV - Format rot is a thing! --- ### ... but what if you need your files to take up less space - You're trying to store a bunch of sounds in a limited space - You're trying to save bandwidth costs when sending sound or music - You need to allow people with slow internet to talk synchronously by voice - You want to *encrypt* the signal so that others can't hear it without a key - **You want to send something smaller than large lists of samples!** --- ## Audio Codecs --- ### Codecs encode and decode signals - (This is a portmanteau of encoder-decoder) - In the audio world, it encodes the sample amplitudes into a different and more space-efficient format --- ### Codecs aren't *quite* the same as audio formats - Audio file formats are packages including data in one or more codecs - All videos include audio which is stored or compressed with a codec - It's possible to have different codecs with the same 'file type' - Occasionally, this causes video files not to open with audio, or means files won't convert or work with your software - **Generally this distinction isn't important to linguists!** --- ### There are many ways to store and stream audio - 'Uncompressed' formats - WAV, AIFF, a few others - 'Lossless' compressed codecs - FLAC, Apple Lossless - 'Lossy' compressed codecs - mp3, AAC, wma, Opus, GSM, AMR OGG --- ### Lossless Compression - 'Lossless' files contain the data to reconstruct exactly what was captured by the ADC - There are other lossless formats like FLAC, WavPack, and Apple Lossless - These save space by cleverly saving the full data stream - e.g. "4000 samples of silence here" rather than 4,000 instances of "0.000000" - Lossless compression asks "What can I do to make these files smaller while still keeping all the data?" - Lossless compression is **not a problem**, and you can convert between formats --- ## You should save your data files as WAV when possible! <img class="r-stretch" src="humorimg/bearsrepeating.jpg" alt="bearsrepeating.jpg The image is a collage consisting of multiple smaller images arranged in a grid format. Each small image features a black bear standing against a backdrop of gray rocks with some greenery at the base. Detailed Description: 1. Bears: - The bears are consistently positioned and appear to be facing forward. - They have thick, dark fur that covers their entire bodies. - Their ears are rounded and stand upright. - Each bear has a light-colored snout with visible whiskers around it. 2. Background: - The background in each image is the same: gray rocks forming a wall-like structure. - There's some greenery, possibly moss or small plants, at the base of the rock formation where the bears are standing. 3. Grid Layout: - The collage consists of 16 smaller images arranged in a 4x4 grid. - Each image is identical and shows the same bear against the same background. Observations: - There's no text or additional diagrams present within these individual images. - The bears appear to be stationary, with no visible movement or interaction between them. - The lighting seems consistent across all images, suggesting they were taken under similar conditions. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ## Lossy File formats <img class="r-stretch" src="img/voldemort.jpg" alt="voldemort.jpg The image displays a medium close-up shot of a man seated and facing forward. The man has a completely bald head and extremely pale, white skin that appears textured like snake scales. His nose is reduced to two small slits on his face. He has red eyes with thin lips and a gaunt facial structure. He is wearing dark, high-collared robes that appear black or very dark green in the lighting. Behind him, he is seated in a large, ornate chair made of dark wood. The back of the chair features tall, carved posts with decorative finials on top. The background behind the chair is out of focus but suggests a large room with light-colored stone walls or pillars, illuminated by cool, bluish light. This character is Lord Voldemort from the Harry Potter film series, played by actor Ralph Fiennes. He is looking directly at the camera with a serious expression. There is no text or diagrams in the image. This description was generated automatically. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Lossless vs. Lossy Compression - Lossless compression asks "What can I do to make the file smaller while keeping the same exact data?" - Lossy compression asks "What can I throw away to make the file smaller while keeping the human from noticing?" - Lossy compression *is tuned to human perception*! --- ### Lossy codecs are everywhere - mp3 is the most well known lossy codec - AAC is Apple's version - Your cell phone uses EVS, EVRC, AMR, or GSM - This one of the reasons old phones need to be changed - It's also why hold music sounds like garbage - Zoom uses the Opus codec - Free and open format, hooray! --- ### Lossy Compression throws away information strategically - Using things like Discrete Cosine Transform and LPC - This is the same LPC that finds formants in Praat! - Also uses psychoacoustic knowledge - "The human won't be able to hear this part anyways" - "Let's throw away or simplify the stuff that doesn't matter as much to the human!" --- ### It's a lot like image compression! --- <img class="r-stretch wide" src="phonmedia/compression1.jpg" alt="compression1.jpg The image depicts a serene landscape featuring majestic snow-capped mountains under a bright blue sky with scattered white clouds. In the foreground, there is a calm body of water that reflects the surrounding scenery, including the mountains and trees. The water appears tranquil and mirror-like. On either side of the water, there are dense clusters of evergreen trees, likely pine or fir, which extend towards the base of the mountains. These trees have dark green needles and are tall with a conical shape. Some of the trees in the foreground on the right side appear to be slightly taller than those on the left. The mountains rise prominently in the background, their peaks covered entirely in snow, giving them a pristine white appearance. The slopes leading up to these peaks show some areas where the snow has melted or is less dense, revealing patches of green vegetation and earth tones beneath. The sky above the mountains is clear with a few fluffy clouds, suggesting it might be early morning or late afternoon due to the soft lighting. The overall scene conveys a sense of peace and natural beauty, typical of a high-altitude alpine environment. There are no people or animals visible in this image; it focuses entirely on the landscape elements. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression2.jpg" alt="compression2.jpg The image depicts a natural outdoor scene featuring large rocks with patches of green moss growing on their surfaces. The ground is covered in dirt and small stones, indicating an unpaved area. There are no people present in the image. The rocks vary in size and shape; some are smooth while others have rough textures. One prominent rock has a noticeable crack running through it, suggesting possible weathering or geological activity over time. The moss appears to be concentrated on certain parts of the rocks, particularly around the edges where moisture might accumulate more easily. No text or diagrams are visible in this image. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression3.jpg" alt="compression3.jpg The image depicts a natural outdoor scene with large rocks dominating the foreground and midground. The rocks are of various sizes and shapes, with some appearing smooth while others have rough textures. Some of the rocks exhibit patches of green moss or lichen, indicating they may be in a damp environment. In the background, there is a larger rock formation that seems to form part of a cliff face. The ground appears to be dry dirt or soil, and there are small stones scattered across it. A few small plants can be seen growing between the rocks and on the ground. There's also what looks like a piece of driftwood lying near one of the rocks. The overall color palette is earthy, with shades of brown, gray, green, and beige dominating the scene. The lighting suggests that this photo was taken during daylight hours under natural light conditions. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression4.jpg" alt="compression4.jpg The image depicts a natural outdoor scene featuring large rocks with patches of green moss growing on their surfaces. The ground is covered in dirt and small stones, indicating an unpaved area. There are no people present in the image. The rocks vary in size and shape; some are smooth while others have rough textures. One rock in particular has a noticeable crack running through it, suggesting possible weathering or geological activity over time. The moss appears to be concentrated on one side of this rock, possibly due to differences in moisture levels or sunlight exposure. No text or diagrams are visible within the image itself; however, there is no additional context provided about its location or purpose. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression5.jpg" alt="compression5.jpg The image depicts a natural outdoor scene featuring large rocks and a dirt ground. The rocks are of various sizes and shapes, with some appearing smooth while others have rough textures. There is visible green moss growing on the surface of one rock in the center-left area of the image. The overall color tone of the photograph has a sepia-like quality, giving it a warm, earthy appearance. The ground appears to be dry dirt or gravel, scattered with small stones and pebbles. In the foreground, there is a prominent large rock on the right side that has some holes in its surface, suggesting erosion over time. The background consists of more rocks, some partially obscured by shadows cast by the larger rocks in the foreground. There are no people or text visible within this image; it focuses solely on the natural elements present. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression6.jpg" alt="compression6.jpg The image depicts a natural outdoor scene featuring large rocks and a rocky surface. The color palette is sepia-toned, giving it an aged appearance. There are several prominent rocks in the foreground; one of these has a rough texture with visible cracks and holes, suggesting erosion over time. Another rock lies on the ground near this larger one, appearing smaller in comparison. The background consists of more rugged terrain, including what looks like a large boulder or cliff face. The surface appears uneven and textured, indicating natural formation rather than man-made construction. There are no people present in the image; it focuses solely on the rocks and their surroundings. The lighting suggests an overcast day with diffused sunlight, as there are no harsh shadows visible. The overall impression is one of a quiet, possibly remote location where nature has been allowed to take its course without human intervention. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- <img class="r-stretch wide" src="phonmedia/compression7.jpg" alt="compression7.jpg The image appears to be a sepia-toned drawing or sketch of an outdoor scene featuring rocks and what seems to be part of a wall or structure in the background. The overall color palette is monochromatic with shades of brown, giving it a vintage feel. In the foreground, there are two prominent rocks. One rock on the right side is larger and more rounded at its base, while the other rock on the left appears smaller and has a more jagged edge. Both rocks have a textured surface that suggests they might be made of stone or concrete. The background includes what looks like part of a wall or structure with a rough texture, possibly indicating it's an old or weathered wall. The lines are uneven and give the impression of age or wear. There is no text or diagrams present in this image. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### Here's what it looks like when you make it lossless again --- <img class="r-stretch wide" src="phonmedia/compression7.jpg" alt="compression7.jpg The image appears to be a sepia-toned drawing or sketch of an outdoor scene featuring rocks and what seems to be part of a wall or structure in the background. The overall color palette is monochromatic with shades of brown, giving it a vintage feel. In the foreground, there are two prominent rocks. One rock on the right side is larger and more rounded at its base, while the other rock on the left appears smaller and has a more jagged edge. Both rocks have a textured surface that suggests they might be made of stone or concrete. The background includes what looks like part of a wall or structure with a rough texture, possibly indicating it's an old or weathered wall. The lines are uneven and give the impression of age or wear. There is no text or diagrams present in this image. This description was generated automatically from image files by a local LLM, and thus, may not be fully accurate. Please feel free to ask questions if you have further questions about the nature of the image or its meaning within the presentation."> --- ### You can choose how much to compress the sounds! - The *Bitrate* dictates how many bits are required to capture a second of audio - The unit is 'kbps', Kilobits per second - 'Variable Bitrate' (VBR) is the same idea, but adapts well to varied complexity - Lower bitrate means more compression, but more data loss - This is independent of bit depth! --- ### Sound Compression (Again, Shpongle 'Nothing is something worth doing') Uncompressed WAV <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 320kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 192kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 128kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> --- ### Sound Compression (Again, Shpongle 'Nothing is something worth doing') Uncompressed WAV <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 64kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 48kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 32kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> 8kbps mp3 <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> --- <p>(This audiovisual content has been removed for compliance with recent federal accessibility guidelines. <a href="https://accessibility.ucsd.edu/policies-standards/ucsd-accessibility-guidelines.html">Please see this site for details.</a>)</p> Originall from <https://www.youtube.com/watch?v=wBnevSbdb7g> --- ### Lossy compression of audio throws away data! - Compression is irreversible - Loss that you can't hear can still affect measurements - Some measurements more than others - Lower bitrates will have stronger effects, but just don't - Some codecs purposefully use and remove linguistic data - LPC is used for compression *and* measurement --- ### Lossy compression makes decisions! - These codecs were tuned for a data type and language - [mp3 was developed for Suzanne Vega's "Tom's Diner"](https://observer.com/2008/09/suzanne-vega-is-the-mother-of-the-mp3/) - Opus is meant for speech and makes decisions based on contributors' languages - **Saving or collecting your data with compression changes it irrecoverably!** --- ### An aside: FILE compression is lossless - There is no harm in putting a bunch of WAV files into a zip file - Don't worry if your backup service talks about compression - If the file extension at the end doesn't change, you don't care --- ## 'Noise Reduction' --- ### The World is Noisy - Non-speech noise - Room echo and feedback - Typing and mouse clicks - Background clatter - **Zoom (et al) want to send your voice, not the noise!** --- ### 'Noise Reduction' Algorithms - Discord, Zoom, Skype, and phones use speech tuned 'noise reduction' methods - Can be as simple as multiple mics allowing subtraction of background noise - These are increasingly neural-network-based filters - 'Noise Reduction' algorithms are usually trained on language data - They can adversely affect classes of phones found in languages outside of the training data - "That sound isn't found in the language I learned about, so it's noise!" - Zoom doesn't care for ejectives! --- ### Get a local recording alongside videoconferencing - ... but also record both streams via the conferencing app, to deal with possible alignment issues --- ## Summary --- ### Key takeaways - Sampling sound is necessary to put it into computers - 16-bit bit depth and 44,100 Hz Sampling rate is a good idea - Record and save your data losslessly, ideally as .wav files - Lossy, compressed audio will negatively affect quality and measurement - Always record locally, losslessly, if doing remote fieldwork --- ### Friends don't let friends use lossy codecs in science - No. - Do not. - Abso-[infix]-lutely not.