html5-img
1 / 30

Pitch Tracking ( 音高追蹤 )

Pitch Tracking ( 音高追蹤 ). Jyh-Shing Roger Jang ( 張智星 ) MIR Lab ( 多媒體資訊檢索實驗室 ) CS, NTHU ( 清華大學 資訊工程系 ) jang@mirlab.org , http://mirlab.org/jang. Pitch ( 音高 ）. Definition of pitch Fundamental frequency (FF, in Hz): Reciprocal of the fundamental period in a quasi-periodic waveform

kathie
Télécharger la présentation

Pitch Tracking ( 音高追蹤 )

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pitch Tracking (音高追蹤) Jyh-Shing Roger Jang (張智星) MIR Lab (多媒體資訊檢索實驗室) CS, NTHU (清華大學 資訊工程系) jang@mirlab.org, http://mirlab.org/jang

  2. Pitch (音高） • Definition of pitch • Fundamental frequency (FF, in Hz): Reciprocal of the fundamental period in a quasi-periodic waveform • Pitch (in semitone): Obtained from the fundamental frequency through a log-based transformation (to be detailed later) • Characteristics of pitch • Noise and unvoiced sound do not have pitch.

  3. Pitch Tracking (音高追蹤) • Pitch tracking: To compute the pitch vector of a give waveform (對整段音訊求取音高) • Applications • Query by singing/humming (哼唱選歌) • Tone recognition for Mandarin (華語的音調辨識) • Intonation scoring for English (英語的音調評分) • Prosody analysis for speech synthesis (語音合成中的韻律分析) • Pitch scaling and duration modification (音高調節與長度改變)

  4. Pitch Tracking Algorithms • Two categories for pitch tracking algorithms • Time domain (時域) • ACF (Autocorrelation function) • AMDF (Average magnitude difference function) • SIFT (Simple inverse filtering tracking) • Frequency domain (頻域) • Harmonic product spectrum method • Cepstrum method

  5. Typical Steps for Pitch Tracking • Chop signals into frames (aka frame blocking) • Compute pitch functions (ACF, AMDF, etc.) • Determine pitch for a frame • Max/min picking of the pitch function • Remove unreliable pitch • Via volume/clarity thresholding • Smooth the whole pitch vector • Via median filter, etc.

  6. Frame Blocking Overlap Zoom in Frame Frame size=256 points Overlap=84 points Frame rate = fs/(frameSize-overlap) = 11025/(256-84)=64 pitch/sec

  7. ACF: Auto-correlation Function 1 128 Frame s(i): Shifted frame s(i+t): t=30 acf(30) = inner product of overlap part Pitch period 30

  8. ACF Example 1 • sunday.wav • Sample rate = 16kHz • Frame size = 512 (starting from point 9000) • Fundamental frequency • Max of ACF occurs at index 132 • FF = 16000/(132-1) = 123.077 Hz

  9. ACF Example 2 • If the range of humans’ FF is [40, 1000], then we have the restriction for selecting pitch point: • Min FF=40Hz  acf(fs/40:end) is not considered. • Max FF=1000Hz  acf(1:fs/1000) is not considered.

  10. Pitch Tracking via ACF • Specs • Sampe rate = 11025 Hz • Frame size = 353 points = 32 ms • Overlap = 0 • Frame rate = 31.25 f/s • Playback • soo.wav • sooPitch.wav

  11. Variations of ACF to Avoid Tapering • Normalized version • Half-frame shifting:

  12. Variations of ACF to Normalize Range • To normalize ACF to the range [-1 1]: This is based on the inequality:

  13. AMDF: Average Magnitude Difference Function 1 128 Frame s(i): Shifted frame s(i+t): t=30 amdf(30) = sum of abs. difference Pitch period 30

  14. AMDF Example • sunday.wav • Sample rate = 16kHz • Frame size = 512 (starting from point 9000) • Fundamental frequency • Min of AMDF occurs at index 132 • FF = 16000/(132-1) = 123.077 Hz

  15. Variations of AMDF to Avoid Tapering • Normalized version • Half-frame shifting:

  16. Combining ACF and AMDF Frame ACF AMDF ACF/AMDF

  17. Example of Pitch Tracking

  18. UPDUDP (1/4) • UPDUDP: Unbroken Pitch Determination Using DP • Goal: To take pitch smoothness into consideration • : a given path in the AMDF matrix • : Number of frames • : Transition penalty • : Exponent of the transition difference

  19. UPDUDP (2/4) • Optimum-value function D(i, j): the minimum cost starting from frame 1 to position (i, j) • Recurrent formula: • Initial conditions : • Optimum cost :

  20. UPDUDP (3/4) • A typical example

  21. UPDUDP (4/4) • Insensitivity in

  22. Harmonic Product Spectrum • hps.m

  23. Frequency to Semitone Conversion • Semitone : A music scale based on A440 • Reasonable pitch range: • E2 - C6 • 82 Hz - 1047 Hz ( - )

  24. Unreliable Pitch Removal • Pitch removal via volume thresholding

  25. Unreliable Pitch Removal • Pitch removal via volume/clarity thresholding

  26. Rest Handling With rests Without rests

  27. Rest Handling Original pitch vectors with rests. Rests are replaced by previous nonzero pitch. Good for LS. Rests are removed. Good for DTW.

  28. Typical Result of Pitch Tracking Pitch tracking via autocorrelationfor茉莉花 (jasmine)

  29. Comparison of Pitch Vectors Yellow line : Target pitch vector

  30. Demo of Pitch Tracking • Real-time display of ACF for pitch tracking • toolbox/sap/goPtByAcf.mdl • Real-time pitch tracking for real-time mic input • toolbox/sap/goPtByAcf2.mdl • Pitch scaling • pitchShiftDemo/project1.exe • pitchShift-multirate/multirate.m • Intonation assessment • ap170/matlab/goDemo.m

More Related