Speech and Language Processing

Reading · ASR

Summary

Ch. 14: Phonetics and Speech Feature Extraction
This chapter covers the basics of articulatory and acoustic phonetics at the outset and then goes into detail on how to extract feature vectors from a speech signal. As they note, the first step in speech processing is transforming a waveform (which measures changes in air pressure over time) to a series of acoustic feature vectors. There are four general steps to extracting these feature vectors.

1. **Convert the analog signal** (where information is represented using the continuously variable physical quantity of air pressure) **to a digital signal** (where information is represented as discrete digits). This involves sampling (where sampling rate (# of samples per second, e.g., 16k Hz) determines how many samples/sec are taken, and sampling size (8-bit vs. 16-bit samples) determines how precisely amplitude is represented) and quantization, where the physical pressure value is transformed to a discrete number, depending on the sampling size.

2. **Store quantized values using 3 parameters**: (1) sampling information, (2) number of channels (depending on # of speakers), and (3) linear vs. compressed storage.

**Windowing**, in which many frames over the length of the acoustic signal are chunked based on window size. These frames overlap, with the start time of the frame being determined by the *frame stride*, or distance (in ms) between initial frame start times. Finally, the shape of the window can be a rectangle cutoff (which causes issues when evaluating the spectral properties of the window) or be a Hamming window (which slowly smooths edges of the window to zero), which is more common.
**Extract spectral information for each frame of the signal**. This is done using *Discrete Fourier Transform (DFT)*, which takes a frame as input and outputs the associated spectrum. Mel cepstrum coefficients are then extracted and represented as an acoustic feature vector for that frame, which is a vector of values describing energy values at various frequency bands (using mel cepstrum coefficients to extract the most important information for identifying the frame's most likely phonetic value), overall energy of the frame (for identifying segment type - vowel, vs. stop, etc.), and changes to the cepstral features compared to previous frames in order to capture how the features are changing from frame to frame.