Tone Analysis for CJK Languages: Domain Explainer#

What This Solves#

The Problem in Plain Language#

In tonal languages like Mandarin Chinese and Cantonese, the same syllable can mean completely different things depending on pitch contour. “Ma” spoken with a high-level pitch means “mother” (妈 mā), with a rising pitch means “hemp” (麻 má), with a dipping pitch means “horse” (马 mǎ), and with a falling pitch means “to scold” (骂 mà). Getting the tone wrong isn’t like having an accent—it’s like saying a different word entirely.

Tone analysis technology automatically detects and evaluates whether someone produced the correct pitch pattern. It’s like spell-check, but for the melody of speech instead of the letters.

Who Encounters This Problem#

Language learners: English speakers learning Mandarin struggle to hear and produce tone differences. Without feedback, they reinforce incorrect patterns for months.

Speech recognition systems: Voice assistants like Siri need to distinguish “买” (mǎi, “to buy”) from “卖” (mài, “to sell”) based solely on pitch. Getting tones wrong means misunderstanding user intent.

Content creators: Audiobook narrators and podcast hosts working in Mandarin need quality control. One mispronounced tone can make listeners think the narrator doesn’t speak the language fluently.

Speech therapists: Children with cochlear implants or adults recovering from stroke need assessment: Can they perceive and produce tones? Progress tracking requires objective measurements.

Linguistic researchers: Studying how tones change in connected speech (tone sandhi) or vary by dialect requires analyzing thousands of recordings. Manual analysis takes months; automatic tools reduce this to days.

Why It Matters#

Scale: Over 1.3 billion people speak Mandarin Chinese. The global language learning market is $4.4 billion and growing 17% annually. Mandarin is the #2 most-studied language worldwide.

Accuracy barrier: Current automatic tone analysis achieves 87-90% accuracy. This is “good enough” for language learning feedback but not sufficient for clinical diagnostics or high-stakes testing. The remaining 10-13% error rate is a persistent challenge.

Economic opportunity: The niche market for tone-specific training tools is $100-150 million (2026), but faces disruption risk. Tech giants like Google and ByteDance could commoditize basic tone analysis by 2028-2029 through foundation models (think “Whisper for tones”).

Accessible Analogies#

Pitch Detection: Finding the Melody in Speech#

Imagine trying to transcribe a melody while an orchestra is playing. You need to isolate the lead violin’s pitch from all the drums, horns, and background noise. That’s pitch detection—extracting the fundamental frequency (F0) from a complex audio signal.

The challenge: Human speech isn’t a pure tone like a tuning fork. It’s noisy, it starts and stops (consonants have no pitch), and everyone’s natural pitch range is different. A man saying “mā” might peak at 150 Hz, while a woman peaks at 300 Hz—same tone, different frequencies.

Established solutions: The Praat software (developed in phonetics labs, used for 25+ years) is the gold standard. It’s like the Adobe Photoshop of speech analysis—professional-grade, trusted by academics, but has a steep learning curve. Tools like Parselmouth bring Praat’s accuracy to Python with zero dependencies, making it accessible to software developers.

The trade-off: Accurate pitch detection takes 2-3 seconds per audio file. For batch processing (analyzing 1000 recordings for research), that’s acceptable. For real-time feedback (language learning app), that’s too slow—users need responses in under 200 milliseconds to feel “instant.”

Tone Classification: Pattern Recognition in Melodies#

Once you have the pitch contour (the melody), you need to classify it: Is this Tone 1 (high-level, like a sustained note), Tone 2 (rising, like asking a question), Tone 3 (dipping then rising, like a valley), or Tone 4 (falling, like a command)?

Analogy: Think of reading handwriting. An expert can glance at “Hello” written in cursive and know immediately what it says, even if the ‘o’ looks a bit like an ‘a’. They’ve seen thousands of examples and learned the pattern. Machine learning models do the same with pitch contours—trained on thousands of examples, they learn to recognize the characteristic shapes of each tone.

Accuracy levels:

• Rule-based (80-85% accurate): Like following explicit instructions (“If pitch rises more than 50 Hz, it’s Tone 2”). Fast and explainable, but brittle to edge cases.
• CNN neural networks (87-88% accurate): Like an expert who’s seen 10,000 examples. Can handle variations, but you can’t easily explain why it made a decision.
• State-of-the-art hybrids (90%+ accurate): Combining multiple techniques, but adds complexity and cost.

The persistent gap: That final 10-13% error rate is stubborn. It’s often Tone 3 (the dipping tone), which speakers sometimes produce incompletely in casual speech. Teaching a model to distinguish “sloppy but acceptable Tone 3” from “actually Tone 2” requires human-like contextual understanding—a current frontier.

Tone Sandhi: Rules That Change in Context#

In connected speech, tones don’t occur in isolation. Mandarin has “tone sandhi rules”—the tone of one syllable changes based on what comes next. It’s like how English speakers say “I’m gonna go” instead of “I am going to go”—the casual form follows implicit rules.

Example: The word 不 (bù, “not”) is normally Tone 4 (falling). But before another Tone 4, it changes to Tone 2 (rising). So “不是” (bù shì, “is not”) is pronounced “bú shì” with the first syllable rising instead of falling.

Detection challenge: A tone analysis system hearing “bú shì” needs to recognize: (1) the speaker produced Tone 2, but (2) this is the correct realization of an underlying Tone 4 due to a sandhi rule, not an error.

Solution landscape:

• Rule-based (88-97% accurate): Hard-code the known sandhi rules. Fast and transparent, but only handles documented patterns.
• Machine learning (97%+ accurate): Train models on thousands of examples of tone sandhi. Can discover patterns, but requires labeled data and careful validation.
• Hybrid approach (97%+, low false positives): Use rules to flag potential sandhi, then ML to verify. Combines explainability with accuracy.

When You Need This#

Clear Decision Criteria#

You NEED tone analysis if:

1. Language learning app for tonal languages: Your users are learning Mandarin, Cantonese, Vietnamese, or Thai and need automated feedback on pronunciation. Manual correction by tutors doesn’t scale.

2. Speech recognition for tonal languages: You’re building ASR (automatic speech recognition) and need to distinguish homophones. “Tone-deaf” ASR confuses “buy” (mǎi) with “sell” (mài), leading to frustrating user experiences.

3. Quality control for audio content: Your audiobook narrators or podcast hosts work in tonal languages, and you need to catch pronunciation errors before publication. Manual QC takes too long.

4. Phonetics research on tones: You’re studying tone variation, dialect differences, or tone sandhi, and manually annotating 1000+ recordings would take 6+ months.

5. Clinical assessment (future): You’re developing tools for speech-language pathologists to diagnose tone perception deficits in children with cochlear implants or patients recovering from stroke. Note: This use case requires 3-5 years of validation studies and regulatory clearance—technology is not yet production-ready.

When You DON’T Need This#

Skip tone analysis if:

1. Non-tonal languages only: If you’re working with English, Spanish, French, etc., pitch carries emotion and emphasis (prosody), not lexical meaning. Standard speech recognition handles this.

2. Casual accuracy is sufficient: If your learners just need to be “understandable, not perfect,” tone errors may be acceptable. Native speakers are forgiving—context often clarifies meaning. Focus budget elsewhere (vocabulary, grammar).

3. Small user base, high-touch: If you have 50 learners and 5 tutors, human feedback may be more cost-effective than building automated tools. Break-even is typically 500-1000+ learners.

4. Technology not mature for your use case: Clinical diagnostics requires 95%+ accuracy, test-retest reliability, and FDA clearance. Current technology is at 87-90% accuracy and lacks clinical validation. Wait 3-5 years or invest in validation studies yourself ($100K-500K).

Concrete Use Case Examples#

Duolingo-style language app:

• Use case: Give learners instant feedback on tone accuracy
• Stack: Real-time pitch detection (PESTO, <10ms latency) + lightweight neural network
• Cost: $50-60K for MVP (4-8 weeks)
• Success metric: 85%+ tone classification accuracy, <200ms latency

Baidu-style Mandarin ASR:

• Use case: Improve speech recognition accuracy by 2-5% relative WER
• Stack: Batch pitch extraction (Parselmouth) + tone features for acoustic model
• Cost: $17-37K per corpus (one-time)
• Success metric: Reduce tone-related ASR errors by 50%+

Audiobook QC tool:

• Use case: Flag potential tone errors for narrator re-recording
• Stack: ASR (Whisper) + dictionary lookup + tone verification (CNN)
• Cost: $62-68K Year 1
• Success metric: 80%+ error catch rate, <5% false positives

University phonetics lab:

• Use case: Analyze tone variation across 100 speakers, 10 hours of audio
• Stack: Praat/Parselmouth (batch F0 extraction) + manual verification
• Cost: $15-20K (including data collection)
• Success metric: Publication acceptance, reproducible results

Trade-offs#

Accuracy vs. Speed vs. Cost#

There’s no free lunch—you choose where to optimize:

The 87-90% plateau: Current technology hits a wall here. Exceeding 90% requires:

• More training data (10,000+ hours vs. 1,000 hours)
• Contextual understanding (what word was intended?)
• Speaker adaptation (learn individual’s pitch range)
• Foundation models (2028-2029 timeline, not available today)

Build vs. Buy vs. Wait#

Build custom (2-6 months, $50K-200K) if:

• Your use case has specific requirements (regulatory, integration, custom UX)
• You need differentiation (competitors use off-the-shelf tools)
• You have in-house ML expertise (don’t outsource your core competency)

Buy or integrate open-source (2-4 weeks, $0-30K) if:

• Standard use case (pronunciation practice, ASR features)
• Speed to market > customization
• Budget-constrained or testing market fit

Wait 2-3 years if:

• You need 95%+ accuracy (clinical, high-stakes testing)
• Foundation models may commoditize (2028-2029)
• Regulatory path unclear (FDA for medical devices)

Self-Hosted vs. Cloud Services#

Self-hosted (on-device or on-premise):

• ✅ Data privacy (HIPAA, GDPR compliant by default)
• ✅ Low latency (no network round-trip)
• ✅ Predictable costs (no per-API-call pricing)
• ❌ Deployment complexity (model updates, cross-platform)
• ❌ Upfront investment (optimize models for mobile)

Cloud API (SaaS):

• ✅ Easy deployment (just API calls)
• ✅ Always up-to-date (models improve automatically)
• ❌ Privacy concerns (voice data leaves device)
• ❌ Variable costs (scales with users, can balloon)
• ❌ Internet dependency (unusable offline)

Recommendation for tone analysis: Self-hosted preferred for consumer apps (privacy, latency) and clinical tools (HIPAA). Cloud acceptable for B2B enterprise if BAA (Business Associate Agreement) in place.

Language Coverage#

Mandarin (4 tones + neutral):

• Most mature technology (90% of research focuses here)
• Datasets: AISHELL-1 (170 hours), AISHELL-3 (85 hours)
• Production-ready (87-88% accuracy achievable)

Cantonese (6 tones):

• Less mature (fewer datasets, pre-trained models scarce)
• Requires custom training or fine-tuning
• Add 30-50% to timeline and budget

Vietnamese (6 tones):

• Similar maturity to Cantonese
• Research active but fewer production tools

Thai (5 tones):

• Less researched than Mandarin/Cantonese
• Expect to build from scratch or adapt Mandarin models

Trade-off: Start with Mandarin for fastest time-to-market. Expand to Cantonese/Vietnamese once validated. Avoid multi-language from day one (complexity explodes).

Implementation Reality#

Realistic Timeline Expectations#

Language Learning App (Pronunciation Practice):

• MVP (rule-based, 80-85% accuracy): 4-8 weeks
• Production (CNN, 87% accuracy): 3-4 months
• State-of-the-art (90%+): 6-9 months + validation

Speech Recognition (F0 Features):

• Integrate Parselmouth: 1-2 weeks
• Train ASR with tone features: 2-4 weeks (if corpus ready)
• Optimize and deploy: 1-2 weeks
• Total: 1-2 months

Linguistic Research Tool:

• Script Parselmouth pipeline: 1-2 weeks
• Test on pilot data: 1 week
• Full corpus analysis: Depends on size (100 hours = 1-2 weeks compute)
• Total: 1-2 months

Clinical Assessment Tool:

• Build core functionality: 3-6 months
• Validation study (reliability, accuracy): 6-12 months
• FDA 510(k) submission (if medical device): 12-24 months
• Total: 2-5 years to market

The rule of thumb: Consumer/research use cases = months. Clinical/regulated = years.

Team Skill Requirements#

Minimum viable team (for language learning app):

• 1 full-stack developer (mobile app, backend API)
• 1 ML engineer (pitch detection, tone classification)
• 1 linguist consultant (part-time, validate tone labels)
• Total: 2.5 FTE for 3-4 months

Ideal team (for production-grade product):

• 2 mobile developers (iOS + Android)
• 1 backend engineer (API, infrastructure)
• 1-2 ML engineers (pitch, tone classification, sandhi)
• 1 linguist (full-time, data annotation, validation)
• 1 UX designer (learner feedback is subtle, needs iteration)
• Total: 6-7 FTE

Key skills:

• Must have: Python, speech processing (Parselmouth/librosa), basic ML (scikit-learn)
• Nice to have: Deep learning (PyTorch/TensorFlow), Praat expertise, Mandarin fluency
• Can outsource: Data annotation (hire native speakers), UI/UX design

Talent availability: 50-100 PhD graduates per year specialize in tone analysis (globally). Concentrated in China, Taiwan, Singapore, and North America. Hiring is competitive—budget $120K-180K/year for experienced ML engineer with speech expertise.

Common Pitfalls and Misconceptions#

Pitfall 1: “Tone analysis is a solved problem.”

• Reality: 87-90% accuracy is state-of-the-art. The remaining 10-13% is hard. Tone 3 is especially tricky.
• Mitigation: Set realistic expectations with stakeholders. 85%+ is “good enough” for most consumer use cases.

Pitfall 2: “We’ll just use Praat.”

• Reality: Praat is powerful but has a steep learning curve. GUI-based workflows don’t scale. Researchers can use it; app developers need Parselmouth.
• Mitigation: Use Parselmouth (Praat algorithms, Python interface) for programmatic access.

Pitfall 3: “Real-time tone feedback is easy.”

• Reality: Real-time means <200ms latency. Most pitch detectors take 2-3s per file. You need specialized algorithms (PESTO) and lightweight models.
• Mitigation: Budget 2-3× more time for real-time vs. batch processing. Test on mid-range devices (not just your MacBook).

Pitfall 4: “87% accuracy sounds low.”

• Reality: Context matters. For language learning, 87% is sufficient—false positives are infrequent, learners improve despite imperfect feedback. For clinical diagnostics, 87% is unacceptable—misdiagnosis has consequences.
• Mitigation: Match accuracy requirements to use case. Don’t over-engineer.

Pitfall 5: “Big Tech will never care about tone analysis.”

• Reality: Mandarin is the #2 language. Google Translate, Duolingo, and ByteDance already use tone features in ASR. Foundation models may commoditize tone analysis by 2028-2029.
• Mitigation: Build data moat (collect learner pronunciation data 2026-2027) before commoditization. Differentiate on UX, personalization, or domain specificity.

Pitfall 6: “We’ll expand to Cantonese/Vietnamese later.”

• Reality: Multi-language adds 30-50% complexity per language (new datasets, models, validation). Design for it upfront or accept refactoring.
• Mitigation: If multi-language is core to your strategy, budget accordingly from day one. Otherwise, perfect Mandarin first.

First 90 Days: What to Expect#

Month 1: Setup and Prototyping

• Week 1-2: Evaluate open-source tools (Parselmouth, librosa, PESTO). Pick one.
• Week 3-4: Build proof-of-concept (record audio → extract pitch → classify tone → display result).
• Deliverable: Rule-based MVP (80% accuracy) that runs on your machine.

Month 2: Data and Training

• Week 5-6: Acquire dataset (AISHELL-1 or collect custom data from target users).
• Week 7-8: Train CNN tone classifier (TensorFlow or PyTorch).
• Deliverable: Model checkpoint (87% accuracy on test set).

Month 3: Integration and Validation

• Week 9-10: Integrate model into app (mobile or web).
• Week 11-12: User testing with 10-20 target users (language learners, narrators, etc.).
• Deliverable: Feedback report (accuracy, latency, UX issues).

Expect:

• Good surprises: Parselmouth works out-of-box. Pre-trained models (if available) save weeks.
• Bad surprises: Tone 3 classification is worse than expected (70-75% vs. 87% average). Real-world noise breaks pitch detection. Users find latency frustrating.
• Typical roadblocks: Dataset licensing (AISHELL requires citation, some corpora are restricted). Deployment (model too large for mobile, need quantization). User expectations (they expect 100% accuracy, need to set realistic expectations).

After 90 Days: Path to Production#

If MVP validates (users find it useful despite imperfections):

• Invest in CNN model (2-4 weeks training time)
• Optimize for production (model compression, latency)
• Scale infrastructure (handle 1000+ concurrent users)
• Launch beta (invite-only, collect feedback)

If MVP reveals issues:

• Pivot tone classification approach (try hybrid rule-based + ML)
• Reduce scope (focus on Tone 1 and 4 first, add Tone 2 and 3 later)
• Consider outsourcing (hire contractor with speech expertise)

If MVP fails (users don’t engage):

• Revisit use case (was tone feedback actually the pain point?)
• Check UX (is feedback too subtle? Too slow?)
• Assess accuracy (is 80-85% too low for your users?)

The litmus test: After 90 days, you should know whether tone analysis adds value to your product. Don’t over-invest until validated.

Summary: Making the Decision#

Decision Framework#

Choose tone analysis if:

• ✅ Working with tonal language (Mandarin, Cantonese, Vietnamese, Thai)
• ✅ User base large enough (500+ users or growing 50%+ annually)
• ✅ Acceptable accuracy exists (87-90% for consumer, 95%+ for clinical)
• ✅ Budget aligns ($50K-200K for custom, $0-30K for off-shelf)
• ✅ Timeline fits (3-4 months for MVP, 2-5 years for clinical)

Skip tone analysis if:

• ❌ Non-tonal language or prosody is “nice-to-have”
• ❌ Small user base (<500) with high-touch service model
• ❌ Accuracy insufficient for use case (clinical needs 95%+, current = 87-90%)
• ❌ Commoditization risk high (Big Tech may dominate 2028-2029)

Key Takeaway#

Tone analysis is production-ready for language learning and speech recognition (87-90% accuracy sufficient, technology mature). It’s emerging for content creation (QC tools being built, market validation in progress). It’s not yet ready for clinical diagnostics (requires validation studies, regulatory clearance, 3-5 year timeline).

The optimal stack varies by use case (see full research for details), but Parselmouth + CNN is the safe default for 80% of use cases. For real-time mobile apps, use PESTO + lightweight models. For clinical, wait or invest in validation.

Timeline to commoditization: Expect foundation models (“Whisper for tones”) by 2028-2029 to achieve 92-95% accuracy. If building a business, differentiate on data (user-specific models), UX (personalized feedback), or domain specificity (clinical workflows). Generic tone analysis APIs will be cheap or free by 2029.

Related Research#

• See 01-discovery/DISCOVERY_TOC.md for complete technical deep-dive (25+ documents, 400 KB)
• See 01-discovery/S3-need-driven/ for use case-specific implementation guides
• See 01-discovery/S4-strategic/ for market analysis and 3-5 year outlook

Research bead: research-bo34 (1.144.2 Tone Analysis) Date: January 2026 Researcher: Ivan (research/crew/ivan)

S1 Rapid Pass: Approach#

Objective#

Quick survey of available libraries for tone analysis and pitch detection in CJK languages, focusing on:

• Pitch/F0 detection capabilities
• Tone verification for pronunciation practice
• Tone sandhi rule implementation potential

Research Method#

• Web search for current (2026) documentation and examples
• Focus on two primary libraries: librosa and praatio
• Evaluate core capabilities, strengths, and weaknesses for CJK tone analysis

Scope#

• librosa: Pure Python audio analysis library
• praatio: Python wrapper for Praat TextGrid manipulation
• Bonus discovery: Parselmouth (direct Praat access from Python)

Key Questions#

1. Which library provides most accurate pitch detection?
2. What are integration requirements (dependencies, external tools)?
3. Can these libraries support tone sandhi rule detection?
4. Which approach is better for batch processing vs. interactive use?

Time Investment#

Initial research pass completed in single session.

librosa: Python Audio Analysis Library#

Overview#

Pure Python library for audio and music analysis with pitch detection capabilities suitable for tone analysis.

Version: 0.11.0 (current as of 2026)

Core Pitch Detection Functions#

librosa.pyin()#

Probabilistic YIN (pYIN) algorithm - recommended for F0 estimation

• Computes F0 candidates with probabilities
• Uses Viterbi decoding for optimal F0 sequence estimation
• Returns: f0, voiced_flag, voiced_probs

librosa.yin()#

Standard YIN algorithm for F0 estimation

librosa.piptrack()#

STFT-based pitch tracking (note: not a dedicated F0 estimator)

Basic Usage#

import librosa

# Load audio file
y, sr = librosa.load('audio.wav')

# Extract pitch using pYIN
f0, voiced_flag, voiced_probs = librosa.pyin(
    y,
    sr=sr,
    fmin=librosa.note_to_hz('C2'),  # ~65 Hz
    fmax=librosa.note_to_hz('C7')   # ~2093 Hz
)

# Get timestamps
times = librosa.times_like(f0, sr=sr)

Parameters for CJK Tones#

Mandarin (4 tones):

• Pitch range: 80-400 Hz (male), 120-500 Hz (female)
• Focus on F0 contour direction

Cantonese (6 tones):

• Similar pitch range
• Focus on F0 height and contour
• Requires precise height discrimination

General guidelines:

• fmin: ~65-80 Hz
• fmax: ~400-500 Hz (adjust for speaker)
• frame_length: 2048 default (~93ms @ 22050 Hz)
• Best practice: At least 2 periods of fmin should fit in frame

Strengths#

1. Pure Python - No external dependencies on Praat/other tools
2. Probabilistic approach - Uncertainty estimates useful for tone boundaries
3. Flexible and scriptable - Easy pipeline integration
4. Batch processing - Efficient for large datasets
5. Well-maintained - Active development in 2026
6. Additional features - Pitch shifting, tuning estimation, spectral analysis

Weaknesses#

1. Music-optimized - Designed for music information retrieval, not phonetics
2. Accuracy concerns - Research shows variability compared to Praat
   • F0 percentiles: strong correlation (r=0.993-0.999)
   • F0 mean: moderate correlation (r=0.730 or lower)
   • F0 std dev: poor correlation (negative in some cases)
3. Algorithm differences - Probabilistic methods vs. Praat’s cross-correlation
4. Voice onset/offset handling - Different behavior at transitions
5. No tone sandhi support - Requires custom implementation

Use Cases for CJK#

✅ Good for:

• Batch processing large pronunciation datasets
• Automated pipelines without Praat dependency
• Quick prototyping and experimentation
• Applications where pure Python is required

❌ Not ideal for:

• Research requiring Praat-level accuracy
• Clinical/diagnostic applications
• Situations where manual verification is impractical

Sources#

• librosa.pyin documentation
• Comparative evaluation of acoustic feature extraction

praatio: Python Library for Praat TextGrids#

Overview#

Pure Python library for working with Praat TextGrid files and running Praat scripts from Python.

Version: 6.2.0 (current as of 2026) Python Support: 3.7-3.12

Core Functionality#

Pitch Extraction#

pitch_and_intensity.extractPI() - Extracts F0 and intensity via Praat

TextGrid Operations#

• Reading/writing TextGrid files (short, long, JSON formats)
• Tier manipulation (union, difference, intersection)
• Time-aligned annotation management
• Hierarchical annotations (utterance > word > syllable > phone)

Basic Usage#

from praatio import pitch_and_intensity
from praatio.utilities import utils
from os.path import join

# Setup paths
wavPath = "path/to/wavfiles"
outputFolder = "path/to/output"
pitchPath = join(outputFolder, "pitch")

# Praat executable location
praatEXE = "/Applications/Praat.app/Contents/MacOS/Praat"  # Mac
# praatEXE = r"C:\Praat.exe"  # Windows

# Create output directories
utils.makeDir(outputFolder)
utils.makeDir(pitchPath)

# Extract pitch and intensity
# Male: 50-350 Hz, Female: 75-450 Hz
pitchData = pitch_and_intensity.extractPI(
    join(wavPath, "audio.wav"),
    join(pitchPath, "audio.txt"),
    praatEXE,
    50,   # minPitch
    350,  # maxPitch
    forceRegenerate=False
)

# Result: list of tuples (time, pitch, intensity)
pitchOnly = [(time, pitch) for time, pitch, _ in pitchData]

Strengths#

1. Leverages Praat accuracy - Uses proven Praat algorithms
2. TextGrid integration - Excellent for time-aligned annotations
3. Phonetics research standard - Praat is gold standard
4. Multi-tier support - Complex hierarchical annotations
5. Pure Python for files - No Praat scripting needed for TextGrid ops
6. Tutorial resources - IPython notebooks available

Weaknesses#

1. Requires Praat installation - Must have separate Praat executable
2. External process overhead - Slower than native Python
3. Maintenance concerns - May be inactive project (sources vary)
4. Limited functionality - Primarily file manipulation, not full Praat access
5. No real-time processing - External calls unsuitable for interactive use
6. Manual parameter tuning - Requires Praat expertise

Important Note: Parselmouth Alternative#

Parselmouth (v0.5.0.dev0, Jan 2026) may be superior for acoustic analysis:

• Direct C/C++ access - Accesses Praat internals (no external process)
• Identical results - Exact same algorithms as Praat GUI
• Full functionality - Complete Praat feature access
• Better performance - No external process overhead
• Active development - Recent 2026 release

Parselmouth Example#

import parselmouth

# Load sound
sound = parselmouth.Sound('audio.wav')

# Extract pitch (exactly like Praat's 'To Pitch (ac)...')
pitch = sound.to_pitch_ac(
    time_step=0.01,
    pitch_floor=75.0,
    pitch_ceiling=600.0
)

# Get pitch values
pitch_values = pitch.selected_array['frequency']

When to Use Which#

• Parselmouth: Acoustic analysis using Praat algorithms from Python
• praatio: TextGrid manipulation and annotation work only

Use Cases for CJK#

✅ Good for:

• Projects already using Praat workflows
• Time-aligned tone annotations
• Phonetics research requiring Praat-level accuracy
• Multi-tier annotation management

❌ Not ideal for:

• Pure Python environments (requires Praat install)
• Real-time or interactive applications
• High-throughput batch processing (external process overhead)

Sources#

• praatio GitHub repository
• praatio PyPI page
• Parselmouth documentation

S1 Rapid Pass: Recommendation#

Quick Summary#

For CJK tone analysis and pitch detection, the landscape has three viable options, not two:

1. Parselmouth - Direct Praat access from Python (discovered during research)
2. librosa - Pure Python audio analysis
3. praatio - Python wrapper for Praat TextGrid manipulation

Primary Recommendation: Parselmouth#

Winner: Parselmouth for most CJK tone analysis use cases.

Why Parselmouth?#

✅ Best of both worlds:

• Praat-level accuracy (identical algorithms, direct C/C++ access)
• Pythonic interface (no external process, no scripting)
• Full Praat functionality (acoustic analysis + TextGrid manipulation)
• Active development (v0.5.0.dev0 released Jan 2026)

✅ Ideal for:

• Pronunciation practice tools (accurate pitch feedback)
• Speech recognition tuning (F0 feature extraction)
• Tone sandhi research (accurate F0 contours)
• Production applications (reliable, fast)

Secondary Option: librosa#

When to use librosa:

✅ Choose librosa if:

• Pure Python environment required (no Praat installation possible)
• Praat-level accuracy not critical
• Batch processing at scale (millions of files)
• Experimentation/prototyping phase
• Integration with music/audio pipelines

⚠️ Be aware:

• Lower accuracy for F0 mean and std dev vs. Praat
• Different voice onset/offset behavior
• Manual verification recommended for critical applications

Tertiary Option: praatio#

When to use praatio:

✅ Choose praatio if:

• Only need TextGrid file manipulation (not acoustic analysis)
• Already using external Praat scripts
• Legacy workflow compatibility required

⚠️ Consider Parselmouth instead:

• Parselmouth handles TextGrids AND acoustic analysis
• Better performance (no external process)
• More Pythonic interface

Tone Sandhi Detection#

⚠️ None of these libraries provide built-in tone sandhi detection.

Current approaches:

1. Statistical modeling - Growth curve analysis, F0 target models
2. Neural networks - CNNs achieving 97%+ accuracy
3. Specialized tools - SPPAS, ProsodyPro
4. Custom implementation - Pitch tracking + rule-based or ML models

Recommendation: Use Parselmouth for accurate pitch extraction, then implement custom tone sandhi rules on top.

Implementation Path#

Phase 1: Pitch Detection#

# Install: pip install praat-parselmouth
import parselmouth

sound = parselmouth.Sound('audio.wav')
pitch = sound.to_pitch_ac(
    time_step=0.01,
    pitch_floor=80.0,   # Adjust for Mandarin/Cantonese
    pitch_ceiling=400.0
)

Phase 2: Tone Classification#

• Extract F0 contour from Parselmouth
• Normalize for speaker (z-score or min-max)
• Classify into tone categories (statistical or ML)

Phase 3: Tone Sandhi Rules#

• Implement rule-based system (e.g., 不 tone change before tone 4)
• Or train ML model on annotated tone sandhi examples

Next Steps for S2#

Deeper investigation needed:

1. Parselmouth performance benchmarks - Speed, memory, accuracy vs. Praat GUI
2. Feature comparison matrix - Parselmouth vs. librosa vs. praatio
3. Tone classification algorithms - HMM, GMM, CNN approaches
4. Tone sandhi detection - Existing research, implementation strategies
5. Real-world examples - Code samples for Mandarin/Cantonese

Decision Matrix#

Overall: Parselmouth wins for most use cases. Use librosa only if Praat installation is impossible.

S2 Comprehensive: Parselmouth Deep Dive#

Executive Summary#

Parselmouth is a Python library that provides direct access to Praat’s C/C++ code, offering identical accuracy to Praat with a Pythonic interface. For CJK tone analysis, it represents the gold standard for pitch extraction with minimal performance overhead.

Key Verdict:

• ✅ Identical accuracy to Praat (uses same underlying C/C++ code)
• ✅ Production-ready (v0.5.0.dev0, January 2026)
• ✅ Zero dependencies (standalone package)
• ✅ Full platform support (Windows, macOS, Linux with precompiled wheels)
• ✅ TextGrid support (via integration with TextGridTools)

1. Complete API Documentation#

1.1 Core Pitch Analysis Methods#

Basic Pitch Extraction#

import parselmouth

# Load audio
sound = parselmouth.Sound('audio.wav')

# Extract pitch using autocorrelation (recommended)
pitch = sound.to_pitch_ac(
    time_step=0.01,           # 10ms intervals
    pitch_floor=80.0,         # Minimum F0 (Hz)
    pitch_ceiling=400.0,      # Maximum F0 (Hz)
    max_number_of_candidates=15,
    very_accurate=False,
    silence_threshold=0.03,
    voicing_threshold=0.45,
    octave_cost=0.01,
    octave_jump_cost=0.35,
    voiced_unvoiced_cost=0.14
)

# Alternative: standard method
pitch = sound.to_pitch(
    time_step=0.01,
    pitch_floor=75.0,
    pitch_ceiling=600.0
)

Pitch Object Methods#

# Statistical measures
pitch_mean = pitch.get_mean()                    # Mean F0 (Hz)
pitch_std = pitch.get_standard_deviation()       # F0 std dev
pitch_min = pitch.get_minimum()                  # Minimum F0
pitch_max = pitch.get_maximum()                  # Maximum F0

# Time-based queries
f0_at_time = pitch.get_value_at_time(0.5)       # F0 at 0.5 seconds
f0_interpolated = pitch.selected_array['frequency']  # Full contour

# Contour analysis
slope = pitch.get_mean_absolute_slope()          # Mean F0 slope
slope_no_jumps = pitch.get_slope_without_octave_jumps()

# Manipulation
pitch.interpolate()                              # Fill unvoiced gaps
pitch.kill_octave_jumps()                        # Remove octave errors
pitch.smooth(bandwidth=10)                       # Smooth contour

1.2 Mandarin/Cantonese-Specific Parameters#

Recommended Settings for Mandarin#

# Male speakers
pitch_mandarin_male = sound.to_pitch_ac(
    time_step=0.01,
    pitch_floor=70.0,      # Lower bound for male voices
    pitch_ceiling=250.0,   # Upper bound for male voices
    voicing_threshold=0.45
)

# Female speakers
pitch_mandarin_female = sound.to_pitch_ac(
    time_step=0.01,
    pitch_floor=100.0,     # Higher floor for female voices
    pitch_ceiling=400.0,   # Higher ceiling for female voices
    voicing_threshold=0.45
)

# Adaptive approach (two-pass method)
# Pass 1: Wide range to find F0 distribution
pitch_initial = sound.to_pitch_ac(
    pitch_floor=50.0,
    pitch_ceiling=700.0
)

# Calculate quartiles
import numpy as np
f0_values = pitch_initial.selected_array['frequency']
f0_values = f0_values[f0_values > 0]  # Remove unvoiced frames
q1, q3 = np.percentile(f0_values, [25, 75])

# Pass 2: Refined range based on speaker's F0 distribution
pitch_refined = sound.to_pitch_ac(
    pitch_floor=0.75 * q1,
    pitch_ceiling=2.5 * q3
)

Recommended Settings for Cantonese#

# Cantonese has 6-9 tones (depending on classification)
# Wider pitch range needed due to more complex tone system

pitch_cantonese = sound.to_pitch_ac(
    time_step=0.01,
    pitch_floor=80.0,      # Adjust based on speaker gender
    pitch_ceiling=450.0,   # Higher ceiling for tone distinctions
    voicing_threshold=0.45,
    max_number_of_candidates=15  # More candidates for complex tones
)

1.3 TextGrid Integration#

# Load TextGrid
textgrid = parselmouth.TextGrid.read('annotations.TextGrid')

# Access tiers
tier = textgrid[0]  # First tier (0-indexed)
tier_by_name = textgrid['phones']  # Access by name

# Iterate through intervals
for interval in tier.intervals:
    print(f"Start: {interval.xmin}, End: {interval.xmax}, Label: {interval.text}")

# Query at specific time
interval_at_time = tier.get_interval_at_time(1.5)

# Integration with TextGridTools (since v0.4.0)
tgt_grid = textgrid.to_tgt()  # Convert to TextGridTools format

# Create new TextGrid
new_textgrid = parselmouth.TextGrid.create(
    xmin=0.0,
    xmax=sound.duration,
    tier_names=['words', 'phones'],
    point_tiers=None
)

2. Performance Benchmarks#

2.1 Accuracy vs. Praat GUI#

Key Finding: Parselmouth produces identical results to Praat because it uses the same underlying C/C++ code.

From the research:

“Parselmouth directly accesses Praat’s C/C++ code (which means the algorithms and their output are exactly the same as in Praat). Each released version of Parselmouth directly corresponds to a specific Praat version and produces the exact same numerical results.”

Accuracy Guarantee:

• F0 percentiles: r=0.999 correlation with Praat (perfect agreement)
• No algorithmic differences
• Numerically identical output for same parameters

2.2 Accuracy vs. librosa#

Recent comparative study (June 2025) on clinical speech data:

Key Issues with librosa:

• Different voice onset/offset behavior
• Inconsistent handling of unvoiced frames
• Lower accuracy for F0 mean and std dev vs. Praat
• Recommendation: Manual verification required for critical applications

2.3 Speed Benchmarks#

From research:

“When it comes to the execution of Praat’s functionality, Python scripts that access computationally expensive Praat algorithms are expected to take the same amount of time, but scripts with a high rate of interaction between Python code and Praat functionality show that Python and Parselmouth runs as fast or even faster than the equivalent script runs in the Praat interpreter.”

Performance Characteristics:

• Single-threaded: Comparable to Praat GUI
• Multi-threaded: Superior due to Python’s multiprocessing module
• Batch processing: Can run in parallel (impossible in Praat scripting)

Speed Comparison (relative):

• Parselmouth: 1x (baseline, same as Praat)
• librosa (pYIN): 0.8-1.2x (comparable)
• CREPE (CPU): 0.05-0.1x (20-50x slower, neural network overhead)
• CREPE (GPU): 2-5x (faster with GPU acceleration)

2.4 Memory Usage#

Parselmouth:

• Minimal overhead beyond audio data
• Pitch object memory: ~8 bytes per frame
• Typical 10-second audio (100 fps): ~8 KB pitch data

Comparison:

• Parselmouth: Low (C/C++ efficiency)
• librosa: Medium (Python NumPy arrays)
• CREPE: High (neural network model weights ~64 MB)

3. Installation & Compatibility#

3.1 Installation#

# Standard installation
pip install praat-parselmouth

# Verify installation
python -c "import parselmouth; print(parselmouth.__version__)"

3.2 System Requirements#

Python Versions:

• ✅ Python 2.7
• ✅ Python 3.5+
• ❌ Python 3.0-3.4 (not supported)

Platform Support:

• ✅ Windows (amd64) - Precompiled wheels
• ✅ macOS (x86-64, ARM64/M1/M2) - Universal2 wheels
• ✅ Linux (x86_64, i686) - Precompiled wheels

Dependencies:

• Zero external dependencies (standalone package)
• No need for Praat installation
• No NumPy/SciPy required (optional for data manipulation)

3.3 Windows-Specific Requirements#

Potential Issue: DLL error on import

Solution:

# Install Microsoft Visual C++ Redistributable for Visual Studio 2022
# Download from: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist

3.4 Version History#

• v0.5.0.dev0 (January 23, 2026) - Latest development version
• v0.4.7 (2025) - Stable release with TextGrid integration
• v0.4.6 (June 8, 2025) - Previous stable
• v0.4.0 - Added TextGridTools integration (to_tgt())

4. Code Examples for Tone Analysis#

4.1 Basic Mandarin Tone Extraction#

import parselmouth
import numpy as np
import matplotlib.pyplot as plt

def extract_mandarin_tone(audio_path, gender='male'):
    """Extract F0 contour for Mandarin tone analysis."""

    # Load audio
    sound = parselmouth.Sound(audio_path)

    # Set parameters based on gender
    if gender == 'male':
        pitch_floor, pitch_ceiling = 70, 250
    else:
        pitch_floor, pitch_ceiling = 100, 400

    # Extract pitch
    pitch = sound.to_pitch_ac(
        time_step=0.01,
        pitch_floor=pitch_floor,
        pitch_ceiling=pitch_ceiling,
        very_accurate=True  # More accurate for tone analysis
    )

    # Extract F0 contour
    f0_values = pitch.selected_array['frequency']
    time_points = pitch.xs()

    # Remove unvoiced frames (0 Hz)
    voiced_mask = f0_values > 0
    f0_voiced = f0_values[voiced_mask]
    time_voiced = time_points[voiced_mask]

    return time_voiced, f0_voiced, pitch

# Usage
time, f0, pitch_obj = extract_mandarin_tone('ma1.wav', gender='female')

# Plot
plt.figure(figsize=(10, 4))
plt.plot(time, f0, 'b-', linewidth=2)
plt.xlabel('Time (s)')
plt.ylabel('F0 (Hz)')
plt.title('Mandarin Tone Contour')
plt.grid(True, alpha=0.3)
plt.show()

4.2 Four-Tone Classification (Mandarin)#

import numpy as np
from scipy.interpolate import interp1d

def classify_mandarin_tone(f0_contour, normalize=True):
    """
    Classify Mandarin tone based on F0 contour shape.

    Mandarin tones:
    - Tone 1 (阴平): High-level (55)
    - Tone 2 (阳平): Rising (35)
    - Tone 3 (上声): Dipping (214)
    - Tone 4 (去声): Falling (51)
    """

    # Normalize to 0-1 scale
    if normalize:
        f0_norm = (f0_contour - f0_contour.min()) / (f0_contour.max() - f0_contour.min())
    else:
        f0_norm = f0_contour

    # Resample to 5 points for comparison
    time_original = np.linspace(0, 1, len(f0_norm))
    time_resampled = np.linspace(0, 1, 5)
    f = interp1d(time_original, f0_norm, kind='cubic')
    f0_5points = f(time_resampled)

    # Calculate features
    start_f0 = f0_5points[0]
    end_f0 = f0_5points[-1]
    mid_f0 = f0_5points[2]
    slope = end_f0 - start_f0

    # Classification rules (simplified)
    if slope < -0.2:
        tone = 4  # Falling
    elif slope > 0.2:
        tone = 2  # Rising
    elif mid_f0 < start_f0 and mid_f0 < end_f0:
        tone = 3  # Dipping
    else:
        tone = 1  # Level

    return tone, f0_5points

# Usage example
time, f0, _ = extract_mandarin_tone('syllable.wav')
tone_number, contour_5pt = classify_mandarin_tone(f0)
print(f"Detected tone: {tone_number}")

4.3 Batch Processing with TextGrid Alignment#

import parselmouth
from pathlib import Path

def batch_extract_tones(audio_path, textgrid_path, output_csv):
    """
    Extract F0 contours for each syllable in a TextGrid.
    """

    # Load audio and TextGrid
    sound = parselmouth.Sound(audio_path)
    textgrid = parselmouth.TextGrid.read(textgrid_path)

    # Extract pitch for entire utterance
    pitch = sound.to_pitch_ac(
        time_step=0.01,
        pitch_floor=80,
        pitch_ceiling=400
    )

    results = []

    # Get syllable tier (adjust tier name as needed)
    syllable_tier = textgrid['syllables']

    for interval in syllable_tier.intervals:
        if not interval.text.strip():
            continue  # Skip empty intervals

        # Get F0 values within interval
        f0_values = []
        time_points = []

        for i, t in enumerate(pitch.xs()):
            if interval.xmin <= t <= interval.xmax:
                f0 = pitch.get_value_at_time(t)
                if f0 > 0:  # Only voiced frames
                    f0_values.append(f0)
                    time_points.append(t)

        if len(f0_values) > 0:
            # Calculate statistics
            f0_mean = np.mean(f0_values)
            f0_std = np.std(f0_values)
            f0_range = max(f0_values) - min(f0_values)

            results.append({
                'syllable': interval.text,
                'start': interval.xmin,
                'end': interval.xmax,
                'duration': interval.xmax - interval.xmin,
                'f0_mean': f0_mean,
                'f0_std': f0_std,
                'f0_range': f0_range,
                'f0_contour': f0_values
            })

    # Save to CSV
    import pandas as pd
    df = pd.DataFrame(results)
    df.to_csv(output_csv, index=False)

    return results

# Usage
results = batch_extract_tones(
    'conversation.wav',
    'conversation.TextGrid',
    'tone_features.csv'
)

4.4 Speaker Normalization (z-score)#

def normalize_f0_zscore(f0_contour, speaker_f0_mean=None, speaker_f0_std=None):
    """
    Z-score normalization for speaker-independent tone analysis.

    Args:
        f0_contour: F0 values for current syllable
        speaker_f0_mean: Speaker's mean F0 (if None, computed from contour)
        speaker_f0_std: Speaker's F0 std dev (if None, computed from contour)

    Returns:
        Normalized F0 contour (z-scores)
    """

    if speaker_f0_mean is None:
        speaker_f0_mean = np.mean(f0_contour)
    if speaker_f0_std is None:
        speaker_f0_std = np.std(f0_contour)

    f0_normalized = (f0_contour - speaker_f0_mean) / speaker_f0_std

    return f0_normalized

# Usage: Compute speaker baseline from neutral tone 1 syllables
time, f0_tone1, _ = extract_mandarin_tone('speaker_baseline.wav')
speaker_mean = np.mean(f0_tone1)
speaker_std = np.std(f0_tone1)

# Normalize new syllable
time, f0_test, _ = extract_mandarin_tone('test_syllable.wav')
f0_normalized = normalize_f0_zscore(f0_test, speaker_mean, speaker_std)

4.5 Visualization with Plotting#

import parselmouth
import matplotlib.pyplot as plt
import numpy as np

def plot_pitch_spectrogram(audio_path):
    """
    Create publication-quality plot with spectrogram and pitch overlay.
    """

    sound = parselmouth.Sound(audio_path)
    pitch = sound.to_pitch_ac(time_step=0.01, pitch_floor=75, pitch_ceiling=500)

    # Create spectrogram
    spectrogram = sound.to_spectrogram(
        window_length=0.005,
        maximum_frequency=5000
    )

    # Plot
    fig, ax = plt.subplots(figsize=(12, 6))

    # Draw spectrogram
    X, Y = spectrogram.x_grid(), spectrogram.y_grid()
    sg_db = 10 * np.log10(spectrogram.values)

    ax.pcolormesh(X, Y, sg_db, shading='gouraud', cmap='gray_r', vmin=sg_db.max() - 70)

    # Overlay pitch
    pitch_values = pitch.selected_array['frequency']
    pitch_values[pitch_values == 0] = np.nan  # Hide unvoiced
    ax.plot(pitch.xs(), pitch_values, 'o', markersize=5, color='w')
    ax.plot(pitch.xs(), pitch_values, 'o', markersize=2, color='red')

    ax.set_xlabel('Time (s)')
    ax.set_ylabel('Frequency (Hz)')
    ax.set_title('Pitch Tracking on Spectrogram')
    ax.set_ylim(0, 500)

    plt.tight_layout()
    plt.show()

# Usage
plot_pitch_spectrogram('mandarin_utterance.wav')

5. Limitations & Considerations#

5.1 Current Limitations#

1. TextGrid API Incomplete

   • Basic read/write supported
   • Advanced manipulation via to_tgt() conversion to TextGridTools
   • Some Praat TextGrid functions not yet ported

2. No Built-in Tone Sandhi Detection

   • Parselmouth extracts pitch only
   • Tone sandhi rules must be implemented separately
   • No phonological rule engine included

3. Short Segments

   • Minimum duration: ~3 periods of pitch_floor
   • For 75 Hz floor: minimum ~40ms
   • Very short syllables may produce unreliable results

4. Unvoiced Consonants

   • No F0 during unvoiced segments
   • Requires interpolation or segmentation strategy

5.2 Best Practices#

Parameter Tuning:

• Start with wide pitch range, then refine
• Use very_accurate=True for tone analysis (slower but better)
• Adjust voicing_threshold for breathy/creaky voice

Quality Control:

• Always plot F0 contours for manual inspection
• Check for octave errors (kill_octave_jumps())
• Verify unvoiced frame handling

Performance Optimization:

• Use multiprocessing for batch jobs
• Cache pitch objects if analyzing multiple times
• Consider downsampling audio to 16 kHz for speed

6. Comparison Matrix#

7. Production Recommendations#

For Mandarin/Cantonese Tone Analysis:#

✅ Use Parselmouth if:

• Accuracy is critical (pronunciation training, speech therapy)
• You need TextGrid integration
• Working with phonetic research workflows
• Want Praat compatibility without external scripts
• Need batch processing with Python ecosystem

⚠️ Consider alternatives if:

• Pure Python environment required (use librosa)
• GPU acceleration needed (use CREPE)
• Integration with music/audio pipelines (use librosa)

Overall Verdict: Parselmouth is the recommended choice for serious CJK tone analysis work due to its proven accuracy, Python integration, and active development.

Sources#

• Parselmouth GitHub Repository
• Parselmouth Documentation (v0.5.0.dev0, Jan 2026)
• Parselmouth API Reference
• Introducing Parselmouth: A Python interface to Praat
• Parselmouth for bioacoustics
• Comparative Evaluation of Acoustic Feature Extraction Tools
• Parselmouth Installation Documentation
• VoiceLab: Automated Acoustic Analysis
• Standardization of pitch range settings

S2 Comprehensive: librosa Advanced Features#

Executive Summary#

librosa is a pure Python audio analysis library optimized for music and audio processing. For CJK tone analysis, it offers a lightweight alternative to Praat-based tools with good (but not Praat-level) accuracy.

Key Verdict:

• ✅ Pure Python (no external dependencies beyond NumPy/SciPy)
• ✅ Fast (comparable to Parselmouth for single-threaded work)
• ⚠️ Lower accuracy than Praat for F0 mean/std dev (voice onset/offset issues)
• ✅ Excellent documentation and active community
• ✅ Rich ecosystem (MIR features, spectral analysis, beat tracking)

Use Case: Choose librosa when Praat installation is impossible or when integrating with music/audio pipelines. Requires manual verification for critical tone analysis.

1. Pitch Detection Methods Comparison#

1.1 Overview of Three Methods#

1.2 pYIN (Probabilistic YIN) - RECOMMENDED#

What it is:

• Modification of the YIN algorithm for fundamental frequency (F0) estimation
• Two-step process:
  1. F0 candidates and probabilities computed using YIN
  2. Viterbi decoding estimates most likely F0 sequence and voicing flags

Advantages over YIN:

• Outperforms conventional YIN algorithm
• Reduction in pitch errors
• Better handling of uncertainty via probabilistic approach
• Computes multiple pitch candidates with associated probabilities

Code Example:

import librosa
import numpy as np

# Load audio
y, sr = librosa.load('mandarin_syllable.wav', sr=22050)

# Extract F0 using pYIN (RECOMMENDED)
f0, voiced_flag, voiced_probs = librosa.pyin(
    y,
    fmin=librosa.note_to_hz('C2'),  # ~65 Hz (male lower bound)
    fmax=librosa.note_to_hz('C7'),  # ~2093 Hz (female upper bound)
    sr=sr,
    frame_length=2048,              # Ideally >=2 periods of fmin
    hop_length=512,                 # Time resolution
    fill_na=None,                   # Best guess for unvoiced frames
    center=True,
    resolution=0.01,                # 0.01 = cents resolution
    max_transition_rate=35.92,      # Octaves per second
    switch_prob=0.01,               # Voiced/unvoiced transition prob
    no_trough_prob=0.01             # Probability of no trough
)

# voiced_flag: Boolean array indicating voiced frames
# voiced_probs: Confidence scores for voicing decisions

# Get time axis
times = librosa.times_like(f0, sr=sr, hop_length=512)

# Plot
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 4))
plt.plot(times, f0, 'b-', linewidth=2, label='F0 (pYIN)')
plt.fill_between(times, 0, 400, where=voiced_flag, alpha=0.2, label='Voiced')
plt.xlabel('Time (s)')
plt.ylabel('F0 (Hz)')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

1.3 YIN (Standard Autocorrelation)#

What it is:

• Autocorrelation-based method for F0 estimation
• Simpler than pYIN (no probabilistic modeling)
• Faster but less robust to noise

Code Example:

# Extract F0 using YIN
f0_yin = librosa.yin(
    y,
    fmin=65,
    fmax=2093,
    sr=sr,
    frame_length=2048,
    hop_length=512,
    trough_threshold=0.1,  # YIN threshold (default 0.1)
    center=True
)

# Note: YIN returns raw F0 values (no voicing flags)

1.4 piptrack (Spectral Peak Tracking) - NOT RECOMMENDED FOR F0#

What it is:

• Pitch tracking on thresholded parabolically-interpolated STFT
• Performs parabolic interpolation on spectrograms to infer local peaks
• NOT an F0 estimator - fundamentally different approach

Why not recommended:

“piptrack is for doing parabolic interpolation on spectrograms to infer local peaks, but it is not an f0 estimator. For f0 estimation, take a look at the yin and pyin functions added in librosa 0.8.”

Code Example (for completeness):

# piptrack returns multiple pitches per frame (not true F0)
pitches, magnitudes = librosa.piptrack(
    y=y,
    sr=sr,
    threshold=0.1,
    fmin=65,
    fmax=2093
)

# Extract dominant pitch (requires additional logic)
# Not recommended for speech F0 analysis

2. Parameter Tuning for Speech Analysis#

2.1 Frequency Range Parameters#

fmin (Minimum Frequency):

• Default: librosa.note_to_hz('C2') (~65 Hz)
• Mandarin male: 70-80 Hz
• Mandarin female: 100-120 Hz
• Cantonese: 80-100 Hz (wider range for tone distinctions)

fmax (Maximum Frequency):

• Default: librosa.note_to_hz('C7') (~2093 Hz)
• Mandarin male: 250-300 Hz
• Mandarin female: 400-500 Hz
• Cantonese: 400-600 Hz

Critical Rule:

“Ideally, at least two periods of fmin should fit into the frame (sr / fmin < frame_length / 2), otherwise it can cause inaccurate pitch detection.”

Example:

• fmin = 75 Hz
• period = 1/75 = 0.0133 s
• 2 periods = 0.0267 s
• sr = 22050 Hz
• Required frame_length >= sr * 0.0267 = 588 samples
• Use frame_length=2048 to be safe

2.2 Time Resolution Parameters#

frame_length:

• Controls frequency resolution
• Larger = better frequency resolution, worse time resolution
• Recommended for speech: 2048 samples @ 22050 Hz = ~93ms

hop_length:

• Controls time step between frames
• Smaller = better time resolution, more computation
• Recommended for speech: 512 samples @ 22050 Hz = ~23ms (4x oversampling)

Example calculation:

sr = 22050
frame_length = 2048
hop_length = 512

time_resolution = hop_length / sr  # 0.023 seconds = 23ms
freq_resolution = sr / frame_length  # 10.77 Hz

print(f"Time resolution: {time_resolution*1000:.1f} ms")
print(f"Frequency resolution: {freq_resolution:.2f} Hz")

2.3 pYIN-Specific Parameters#

max_transition_rate:

• Maximum pitch transition rate in octaves per second
• Default: 35.92 (allows rapid changes)
• For slow speech: 10-20
• For normal speech: 20-35
• For fast speech/singing: 35-50

switch_prob:

• Probability of switching from voiced to unvoiced or vice versa
• Default: 0.01 (1% probability)
• For clean recordings: 0.01
• For noisy recordings: 0.05-0.1

resolution:

• Resolution of pitch bins
• Default: 0.01 (corresponds to cents)
• Finer resolution = more candidates = slower computation

fill_na:

• Default value for unvoiced frames
• None: Use best guess (interpolation)
• np.nan: Mark as NaN
• 0.0: Mark as 0 Hz

2.4 Complete Parameter Guide#

def extract_f0_optimized(
    audio_path,
    gender='male',
    speech_rate='normal',
    recording_quality='clean'
):
    """
    Extract F0 with optimized parameters for Mandarin Chinese.
    """

    # Load audio
    y, sr = librosa.load(audio_path, sr=22050)

    # Gender-specific frequency ranges
    if gender == 'male':
        fmin, fmax = 70, 300
    elif gender == 'female':
        fmin, fmax = 100, 500
    else:
        fmin, fmax = 70, 500  # Wide range

    # Speech rate adjustments
    if speech_rate == 'slow':
        max_transition_rate = 15
    elif speech_rate == 'fast':
        max_transition_rate = 45
    else:
        max_transition_rate = 30

    # Recording quality adjustments
    if recording_quality == 'noisy':
        switch_prob = 0.1
        trough_threshold = 0.15
    else:
        switch_prob = 0.01
        trough_threshold = 0.1

    # Extract F0
    f0, voiced_flag, voiced_probs = librosa.pyin(
        y,
        fmin=fmin,
        fmax=fmax,
        sr=sr,
        frame_length=2048,
        hop_length=512,
        fill_na=None,
        resolution=0.01,
        max_transition_rate=max_transition_rate,
        switch_prob=switch_prob
    )

    return f0, voiced_flag, voiced_probs, sr

# Usage
f0, voiced, probs, sr = extract_f0_optimized(
    'mandarin_utterance.wav',
    gender='female',
    speech_rate='normal',
    recording_quality='clean'
)

3. Accuracy Studies & Limitations#

3.1 Comparative Study (June 2025)#

Study: “Comparative Evaluation of Acoustic Feature Extraction Tools for Clinical Speech Analysis”

Compared tools: OpenSMILE, Praat, librosa on clinical speech data

Results:

Key Findings:

1. F0 Percentiles: Strong agreement between all tools
2. F0 Mean: Algorithm-specific differences in handling unvoiced frames or edge conditions
3. F0 Std Dev: Poor correlation likely stems from fundamental differences in F0 extraction algorithms and how they handle voice onset/offset transitions

3.2 Known Limitations#

Voice Onset/Offset Issues:

• librosa handles transitions differently than Praat
• Can cause significant differences in F0 mean and std dev
• Impact: More pronounced for short syllables with rapid voicing changes

Unvoiced Frame Handling:

• Different algorithms for filling gaps in F0 contours
• Affects mean and variance calculations
• Impact: Tone sandhi detection may be affected

Octave Errors:

• Less robust than Praat at avoiding octave jumps
• No built-in kill_octave_jumps() function
• Impact: Manual post-processing required

Short Segment Performance:

• Requires minimum duration based on fmin
• Very short syllables (<100ms) may be unreliable
• Impact: Problematic for rapid speech

3.3 Comparison with Other Methods#

pYIN vs. YAAPT vs. CREPE (2022 study):

“A comparison study from 2022 evaluated pYIN alongside other algorithms (YAAPT and CREPE) for speech analysis, examining voicing decision errors and pitch errors on speech databases.”

Results:

• pYIN outperforms conventional YIN algorithm
• pYIN competitive with YAAPT for speech
• CREPE remains state-of-the-art for accuracy (but slower)

3.4 Recommendations for Critical Applications#

✅ Use librosa if:

• Pure Python environment required
• Praat installation impossible
• Integration with music/audio pipelines needed
• Batch processing at scale (millions of files)
• Prototyping phase

⚠️ Manual verification required:

• Always plot F0 contours for inspection
• Cross-validate with Praat/Parselmouth on sample data
• Use F0 percentiles (more reliable) over mean/std dev
• Implement octave jump detection

❌ Don’t use librosa if:

• Clinical/research-grade accuracy required
• Pronunciation training (user-facing feedback)
• Subtle tone distinctions critical (e.g., tone sandhi research)
• → Use Parselmouth instead

4. Integration with Tone Classification#

4.1 Feature Engineering Pipeline#

import librosa
import numpy as np
from scipy.interpolate import interp1d

def extract_tone_features(audio_path, gender='male'):
    """
    Extract features for Mandarin tone classification.
    """

    # Load audio
    y, sr = librosa.load(audio_path, sr=22050)

    # Extract F0
    fmin = 70 if gender == 'male' else 100
    fmax = 300 if gender == 'male' else 500

    f0, voiced_flag, voiced_probs = librosa.pyin(
        y,
        fmin=fmin,
        fmax=fmax,
        sr=sr,
        frame_length=2048,
        hop_length=512,
        fill_na=None
    )

    # Remove unvoiced frames
    f0_voiced = f0[voiced_flag]

    if len(f0_voiced) < 3:
        return None  # Insufficient voiced frames

    # Time-normalize to 5 points
    time_original = np.linspace(0, 1, len(f0_voiced))
    time_resampled = np.linspace(0, 1, 5)
    f = interp1d(time_original, f0_voiced, kind='cubic')
    f0_5points = f(time_resampled)

    # Extract features
    features = {
        # Statistical features
        'f0_mean': np.mean(f0_voiced),
        'f0_std': np.std(f0_voiced),
        'f0_min': np.min(f0_voiced),
        'f0_max': np.max(f0_voiced),
        'f0_range': np.max(f0_voiced) - np.min(f0_voiced),

        # Contour shape features
        'f0_start': f0_5points[0],
        'f0_mid': f0_5points[2],
        'f0_end': f0_5points[-1],
        'slope': f0_5points[-1] - f0_5points[0],

        # Velocity features
        'f0_velocity': np.diff(f0_5points),

        # Normalized contour
        'f0_5points': f0_5points,

        # Voicing features
        'voicing_ratio': np.sum(voiced_flag) / len(voiced_flag),
        'mean_voiced_prob': np.mean(voiced_probs[voiced_flag])
    }

    return features

# Usage
features = extract_tone_features('ma1.wav', gender='female')
print(f"F0 mean: {features['f0_mean']:.1f} Hz")
print(f"Slope: {features['slope']:.1f} Hz")
print(f"5-point contour: {features['f0_5points']}")

4.2 Speaker Normalization#

def normalize_f0_semitone(f0_contour, reference_f0=None):
    """
    Convert F0 to semitone scale relative to reference.

    Semitone normalization is more perceptually relevant than z-score.
    """

    if reference_f0 is None:
        reference_f0 = np.median(f0_contour)

    # Convert to semitones: 12 * log2(f0 / reference)
    semitones = 12 * np.log2(f0_contour / reference_f0)

    return semitones

def normalize_f0_zscore(f0_contour, speaker_mean=None, speaker_std=None):
    """
    Z-score normalization for speaker independence.
    """

    if speaker_mean is None:
        speaker_mean = np.mean(f0_contour)
    if speaker_std is None:
        speaker_std = np.std(f0_contour)

    f0_normalized = (f0_contour - speaker_mean) / speaker_std

    return f0_normalized

# Usage
y, sr = librosa.load('mandarin_syllable.wav')
f0, voiced, _ = librosa.pyin(y, fmin=70, fmax=400, sr=sr)
f0_voiced = f0[voiced]

# Semitone normalization (recommended for perception)
f0_semitones = normalize_f0_semitone(f0_voiced, reference_f0=np.median(f0_voiced))

# Z-score normalization (recommended for ML)
f0_zscore = normalize_f0_zscore(f0_voiced)

4.3 Tone Classification with librosa Features#

from sklearn.ensemble import RandomForestClassifier
import pandas as pd

def classify_tone_ml(features, model=None):
    """
    Classify Mandarin tone using machine learning.

    Features should include:
    - f0_mean, f0_std, f0_range
    - f0_start, f0_mid, f0_end, slope
    - f0_5points (normalized)
    """

    if model is None:
        # Load pre-trained model (placeholder)
        model = RandomForestClassifier()

    # Feature vector
    X = np.array([
        features['f0_mean'],
        features['f0_std'],
        features['f0_range'],
        features['slope'],
        features['f0_start'],
        features['f0_mid'],
        features['f0_end']
    ]).reshape(1, -1)

    # Predict
    tone = model.predict(X)[0]
    proba = model.predict_proba(X)[0]

    return tone, proba

# Example training workflow
def train_tone_classifier(audio_files, labels):
    """
    Train tone classifier on labeled data.
    """

    # Extract features for all files
    feature_list = []
    for audio_path in audio_files:
        features = extract_tone_features(audio_path)
        if features is not None:
            feature_list.append(features)

    # Convert to DataFrame
    df = pd.DataFrame(feature_list)

    # Feature matrix
    X = df[['f0_mean', 'f0_std', 'f0_range', 'slope',
            'f0_start', 'f0_mid', 'f0_end']].values

    # Train model
    model = RandomForestClassifier(n_estimators=100, random_state=42)
    model.fit(X, labels)

    return model

5. Advanced Usage Patterns#

5.1 Batch Processing Pipeline#

from pathlib import Path
from concurrent.futures import ProcessPoolExecutor
import pandas as pd

def process_single_file(audio_path):
    """Process single audio file."""
    try:
        features = extract_tone_features(audio_path)
        return {'file': audio_path.name, **features}
    except Exception as e:
        print(f"Error processing {audio_path}: {e}")
        return None

def batch_process_tones(audio_dir, output_csv, n_workers=4):
    """
    Batch process audio files in parallel.
    """

    audio_files = list(Path(audio_dir).glob('*.wav'))

    # Parallel processing
    with ProcessPoolExecutor(max_workers=n_workers) as executor:
        results = list(executor.map(process_single_file, audio_files))

    # Filter out failed files
    results = [r for r in results if r is not None]

    # Save to CSV
    df = pd.DataFrame(results)
    df.to_csv(output_csv, index=False)

    print(f"Processed {len(results)} files -> {output_csv}")

# Usage
batch_process_tones(
    audio_dir='mandarin_corpus/',
    output_csv='tone_features.csv',
    n_workers=8
)

5.2 Real-Time F0 Tracking (Streaming)#

import librosa
import numpy as np

class RealtimeF0Tracker:
    """
    Real-time F0 tracking with overlap-add buffering.
    """

    def __init__(self, sr=22050, frame_length=2048, hop_length=512):
        self.sr = sr
        self.frame_length = frame_length
        self.hop_length = hop_length
        self.buffer = np.array([])

    def process_chunk(self, audio_chunk):
        """
        Process incoming audio chunk.

        Args:
            audio_chunk: 1D numpy array of audio samples

        Returns:
            f0: Estimated F0 for this chunk (or None if insufficient data)
        """

        # Append to buffer
        self.buffer = np.concatenate([self.buffer, audio_chunk])

        # Check if we have enough samples
        if len(self.buffer) < self.frame_length:
            return None

        # Extract F0 for current frame
        frame = self.buffer[:self.frame_length]

        try:
            f0, _, _ = librosa.pyin(
                frame,
                fmin=70,
                fmax=400,
                sr=self.sr,
                frame_length=self.frame_length,
                hop_length=self.hop_length
            )

            # Advance buffer
            self.buffer = self.buffer[self.hop_length:]

            return f0[0] if len(f0) > 0 else None

        except Exception as e:
            print(f"Error in F0 extraction: {e}")
            return None

# Usage
tracker = RealtimeF0Tracker(sr=22050)

# Simulate real-time chunks (512 samples = ~23ms @ 22050 Hz)
y, sr = librosa.load('test.wav', sr=22050)

for i in range(0, len(y), 512):
    chunk = y[i:i+512]
    f0 = tracker.process_chunk(chunk)
    if f0 is not None:
        print(f"Time: {i/sr:.3f}s, F0: {f0:.1f} Hz")

5.3 Octave Jump Detection & Correction#

def detect_octave_jumps(f0_contour, threshold=0.7):
    """
    Detect and correct octave jumps in F0 contour.

    Args:
        f0_contour: F0 values (Hz)
        threshold: Ratio threshold for octave jump (default 0.7 = 70%)

    Returns:
        Corrected F0 contour
    """

    f0_corrected = f0_contour.copy()

    for i in range(1, len(f0_corrected)):
        if f0_corrected[i] == 0:
            continue  # Skip unvoiced frames

        ratio = f0_corrected[i] / f0_corrected[i-1]

        # Check for octave jump up (ratio ~2.0)
        if 1.7 < ratio < 2.3:
            f0_corrected[i] /= 2.0

        # Check for octave jump down (ratio ~0.5)
        elif 0.43 < ratio < 0.59:
            f0_corrected[i] *= 2.0

    return f0_corrected

# Usage
y, sr = librosa.load('audio.wav')
f0, voiced, _ = librosa.pyin(y, fmin=70, fmax=400, sr=sr)

# Correct octave jumps
f0_corrected = detect_octave_jumps(f0)

# Compare
import matplotlib.pyplot as plt
plt.figure(figsize=(12, 6))
plt.subplot(2, 1, 1)
plt.plot(f0, 'b-', label='Original')
plt.title('Original F0')
plt.ylabel('Hz')
plt.legend()

plt.subplot(2, 1, 2)
plt.plot(f0_corrected, 'r-', label='Corrected')
plt.title('Corrected F0 (Octave Jumps Removed)')
plt.ylabel('Hz')
plt.xlabel('Frame')
plt.legend()
plt.tight_layout()
plt.show()

6. Benchmarks & Performance#

6.1 Speed Comparison#

Single-threaded (1 minute of audio @ 22050 Hz):

• pYIN: ~2-3 seconds (0.03-0.05x real-time)
• YIN: ~1-2 seconds (0.02-0.03x real-time)
• piptrack: ~0.5-1 second (0.01-0.02x real-time)

Multi-threaded (100 files, 8 cores):

• Linear speedup with ProcessPoolExecutor
• ~8x faster than single-threaded

Comparison to alternatives:

• librosa pYIN: 1x (baseline)
• Parselmouth: ~1x (comparable)
• CREPE (CPU): ~0.05x (20x slower)
• CREPE (GPU): ~5x (5x faster with GPU)

6.2 Memory Usage#

Per audio file (1 minute @ 22050 Hz):

• Raw audio: ~5 MB (float32)
• F0 contour: ~4 KB (512 samples)
• Total: ~5 MB per file

Batch processing (1000 files):

• With multiprocessing: ~40-80 MB (8 workers × 5 MB)
• Sequential: ~5 MB (constant memory)

6.3 Accuracy Metrics#

From 2022 benchmark study:

• pYIN error rate: ~3x lower than conventional methods
• CREPE error rate: ~5x lower than pYIN (state-of-the-art)

Practical accuracy for Mandarin tones:

• Tone 1 (level): ✅ Excellent
• Tone 2 (rising): ✅ Good
• Tone 3 (dipping): ⚠️ Fair (onset/offset issues)
• Tone 4 (falling): ✅ Good

7. Use Case Recommendations#

✅ Use librosa for:#

1. Prototyping & Experimentation

   • Quick iteration on tone analysis algorithms
   • Testing different parameter configurations
   • Research without production requirements

2. Pure Python Environments

   • Docker containers without system dependencies
   • Cloud functions (AWS Lambda, Google Cloud Functions)
   • Jupyter notebooks for teaching

3. Music/Audio Pipeline Integration

   • Applications using librosa for other features (MFCCs, spectrograms)
   • Beat tracking + tone analysis hybrid systems
   • Audio augmentation pipelines

4. Large-Scale Batch Processing

   • Millions of files where manual verification impractical
   • F0 percentiles sufficient (more reliable than mean/std)
   • Non-critical applications (e.g., data exploration)

⚠️ Use with caution for:#

1. Tone Sandhi Research

   • Voice onset/offset issues may affect sandhi detection
   • Recommend Parselmouth for subtle distinctions

2. Clinical Applications

   • Speech therapy feedback
   • Pronunciation training (user-facing)
   • Medical diagnostics

3. Short Syllables

   • <100ms duration may produce unreliable results
   • Cross-validate with Praat/Parselmouth

❌ Don’t use librosa for:#

1. Production Pronunciation Training

   • Use Parselmouth for Praat-level accuracy

2. Research-Grade Publications

   • Reviewers expect Praat/Parselmouth validation
   • F0 mean/std differences may affect conclusions

3. Real-Time Critical Systems

   • Consider CREPE with GPU for better accuracy
   • Or Parselmouth for lower latency

8. Summary Comparison#

Sources#

• librosa Documentation (v0.11.0)
• librosa.pyin Documentation
• librosa.yin Documentation
• librosa.piptrack Documentation
• Comparing Conventional Pitch Detection (2022)
• Comparative Evaluation of Acoustic Feature Extraction Tools (June 2025)
• librosa: Audio and Music Signal Analysis in Python
• librosa GitHub - Pitch Detection Discussion
• Open-source packages for using speech data in ML

S2 Comprehensive: praatio Advanced Features#

Executive Summary#

praatio (formerly praatIO) is a Python library for working with Praat, TextGrids, time-aligned audio transcripts, and audio files. It’s primarily designed for feature extraction and manipulations on hierarchical time-aligned transcriptions.

Key Verdict:

• ✅ Specialized for TextGrid manipulation (robust API)
• ✅ Multiple output formats (short, long, JSON)
• ✅ Praat script integration (run Praat scripts from Python)
• ⚠️ External Praat dependency for acoustic analysis
• ⚠️ Limited maintenance (fewer updates than Parselmouth)

Use Case: Choose praatio when you need advanced TextGrid manipulation but don’t need acoustic analysis, OR when integrating existing Praat script workflows into Python.

Recommendation: Use Parselmouth instead for most use cases—it provides TextGrid support PLUS acoustic analysis in a more integrated package.

1. Complete API Overview#

1.1 Core Components#

praatio organizes data around three main classes:

1. Textgrid - Container for multiple annotation tiers
2. IntervalTier - Tier containing interval data (start, end, label)
3. PointTier - Tier containing point data (time, label)

Hierarchy:

Textgrid
├── IntervalTier (e.g., "words", "syllables", "phones")
│   └── Interval(xmin, xmax, text)
└── PointTier (e.g., "tones", "events")
    └── Point(time, text)

1.2 Reading TextGrids#

from praatio import textgrid

# Read TextGrid from file
tg = textgrid.openTextgrid('annotation.TextGrid', includeEmptyIntervals=False)

# Access tiers
tier = tg.getTier('words')  # By name
tier = tg.tiers[0]          # By index

# Get tier info
print(f"Tier name: {tier.name}")
print(f"Tier type: {tier.tierType}")  # 'IntervalTier' or 'PointTier'
print(f"Min time: {tier.minTimestamp}")
print(f"Max time: {tier.maxTimestamp}")
print(f"Number of entries: {len(tier.entries)}")

1.3 Creating TextGrids#

from praatio.data_classes import textgrid

# Create new TextGrid
tg = textgrid.Textgrid()
tg.minTimestamp = 0.0
tg.maxTimestamp = 10.0

# Create interval tier
from praatio.data_classes.interval_tier import IntervalTier

word_tier = IntervalTier(
    name='words',
    entries=[
        (0.0, 1.5, 'hello'),
        (1.5, 3.0, 'world'),
        (3.0, 10.0, '')
    ],
    minT=0.0,
    maxT=10.0
)

# Add tier to TextGrid
tg.addTier(word_tier)

# Create point tier (for tone markers)
from praatio.data_classes.point_tier import PointTier

tone_tier = PointTier(
    name='tones',
    entries=[
        (0.75, 'T1'),  # Tone 1 at midpoint of "hello"
        (2.25, 'T4')   # Tone 4 at midpoint of "world"
    ],
    minT=0.0,
    maxT=10.0
)

tg.addTier(tone_tier)

# Save TextGrid
tg.save('output.TextGrid', format='long_textgrid', includeBlankSpaces=True)

1.4 Modifying TextGrids#

# Insert new interval
tier.insertEntry((3.0, 4.5, 'new_word'), collisionMode='replace')

# Delete interval
tier.deleteEntry((1.5, 3.0, 'world'))

# Modify interval
tier.modifyEntries(
    entries=[(0.0, 1.5, 'hello')],
    newEntries=[(0.0, 1.5, 'HELLO')],  # Change label
    collisionMode='replace'
)

# Crop tier to time range
tier.crop(startTime=1.0, endTime=8.0, mode='truncated', rebaseToZero=True)

# Merge consecutive intervals with same label
from praatio import tgio

tier_merged = tgio.eraseRegion(tier, start=None, end=None, mode='strict')

2. File Format Support#

2.1 Four Output Formats#

praatio supports 4 TextGrid output file types:

1. Short TextGrid - Praat native, more concise
2. Long TextGrid - Praat native, more human-readable
3. JSON - Standard JSON format
4. TextGrid-like JSON - Custom JSON format

Comparison:

# Save in different formats
tg.save('output_short.TextGrid', format='short_textgrid')
tg.save('output_long.TextGrid', format='long_textgrid')
tg.save('output.json', format='json')
tg.save('output_tg.json', format='textgrid_json')

Format Details:

2.2 Format Conversion#

from praatio import textgrid

# Read in any format
tg = textgrid.openTextgrid('input.TextGrid')

# Convert format by saving
tg.save('output.json', format='json')
tg.save('output_long.TextGrid', format='long_textgrid')

3. Batch Processing Examples#

3.1 Basic Batch Processing#

from pathlib import Path
from praatio import textgrid

def batch_process_textgrids(input_dir, output_dir, operation):
    """
    Apply operation to all TextGrids in directory.

    Args:
        input_dir: Directory containing .TextGrid files
        output_dir: Directory for output files
        operation: Function that takes Textgrid and returns modified Textgrid
    """

    input_path = Path(input_dir)
    output_path = Path(output_dir)
    output_path.mkdir(exist_ok=True)

    for tg_file in input_path.glob('*.TextGrid'):
        print(f"Processing {tg_file.name}...")

        # Load TextGrid
        tg = textgrid.openTextgrid(str(tg_file))

        # Apply operation
        tg_modified = operation(tg)

        # Save
        output_file = output_path / tg_file.name
        tg_modified.save(str(output_file), format='long_textgrid')

# Example operation: Rename a tier
def rename_tier(tg):
    tg.renameTier('old_name', 'new_name')
    return tg

# Usage
batch_process_textgrids(
    input_dir='input_textgrids/',
    output_dir='output_textgrids/',
    operation=rename_tier
)

3.2 Extract Intervals to Individual Files#

from praatio import textgrid, audio

def extract_syllables_to_wav(audio_path, textgrid_path, output_dir, tier_name='syllables'):
    """
    Extract each syllable interval to separate WAV file.
    """

    output_path = Path(output_dir)
    output_path.mkdir(exist_ok=True)

    # Load TextGrid
    tg = textgrid.openTextgrid(textgrid_path)
    tier = tg.getTier(tier_name)

    # Load audio (requires pydub or audioread)
    from pydub import AudioSegment
    sound = AudioSegment.from_wav(audio_path)

    # Extract each interval
    for i, (start, end, label) in enumerate(tier.entries):
        if not label.strip():
            continue  # Skip empty intervals

        # Extract audio segment
        start_ms = int(start * 1000)
        end_ms = int(end * 1000)
        segment = sound[start_ms:end_ms]

        # Save
        output_file = output_path / f"{label}_{i:03d}.wav"
        segment.export(str(output_file), format='wav')

        print(f"Exported {output_file.name}")

# Usage
extract_syllables_to_wav(
    audio_path='recording.wav',
    textgrid_path='recording.TextGrid',
    output_dir='extracted_syllables/',
    tier_name='syllables'
)

3.3 Align Boundaries Across Tiers#

from praatio.utilities import utils

def align_boundaries_across_tiers(tg, reference_tier_name, tolerance=0.01):
    """
    Align boundaries across tiers to fix manual annotation errors.

    Args:
        tg: Textgrid object
        reference_tier_name: Name of tier to use as reference
        tolerance: Maximum time difference for alignment (seconds)
    """

    reference_tier = tg.getTier(reference_tier_name)

    # Get reference boundaries
    reference_boundaries = set()
    for start, end, _ in reference_tier.entries:
        reference_boundaries.add(start)
        reference_boundaries.add(end)

    # Align other tiers
    for tier in tg.tiers:
        if tier.name == reference_tier_name:
            continue

        new_entries = []
        for start, end, label in tier.entries:
            # Find closest reference boundary
            closest_start = min(reference_boundaries, key=lambda x: abs(x - start))
            closest_end = min(reference_boundaries, key=lambda x: abs(x - end))

            # Only align if within tolerance
            if abs(closest_start - start) <= tolerance:
                start = closest_start
            if abs(closest_end - end) <= tolerance:
                end = closest_end

            new_entries.append((start, end, label))

        # Update tier
        tier.entries = new_entries

    return tg

# Usage
tg = textgrid.openTextgrid('annotation.TextGrid')
tg_aligned = align_boundaries_across_tiers(tg, reference_tier_name='words', tolerance=0.01)
tg_aligned.save('annotation_aligned.TextGrid', format='long_textgrid')

3.4 Merge TextGrids from Multiple Annotators#

def merge_textgrids_multi_annotator(textgrid_paths, output_path):
    """
    Merge TextGrids from multiple annotators into single file.

    Each annotator's tiers get prefixed with annotator name.
    """

    # Load first TextGrid as base
    tg_merged = textgrid.openTextgrid(textgrid_paths[0])

    # Rename tiers with annotator prefix
    for tier in tg_merged.tiers:
        tier.name = f"annotator1_{tier.name}"

    # Add tiers from other annotators
    for i, tg_path in enumerate(textgrid_paths[1:], start=2):
        tg = textgrid.openTextgrid(tg_path)

        for tier in tg.tiers:
            tier_copy = tier.new(name=f"annotator{i}_{tier.name}")
            tg_merged.addTier(tier_copy)

    # Save merged TextGrid
    tg_merged.save(output_path, format='long_textgrid')

    return tg_merged

# Usage
merge_textgrids_multi_annotator(
    textgrid_paths=[
        'annotator1.TextGrid',
        'annotator2.TextGrid',
        'annotator3.TextGrid'
    ],
    output_path='merged_annotations.TextGrid'
)

4. Integration with Praat Scripts#

4.1 Running Praat Scripts from Python#

praatio includes praat_scripts.py for running Praat scripts from Python:

from praatio import praat_scripts

# Run Praat script
praat_scripts.runPraatScript(
    scriptFn='extract_f0.praat',
    argList=['input.wav', '75', '500'],  # Arguments to script
    outputFn='output.txt'
)

4.2 Extract Pitch and Intensity#

from praatio.utilities import pitch_and_intensity

# Extract pitch using Praat
pitch_data = pitch_and_intensity.extractPI(
    inputFN='audio.wav',
    outputFN='pitch.txt',
    praatEXE='/usr/bin/praat',  # Path to Praat executable
    minPitch=75,
    maxPitch=500,
    sampleStep=0.01,
    silenceThreshold=0.03,
    voiceThreshold=0.45
)

Note: This requires Praat to be installed separately.

4.3 Known Limitations#

Short segments issue (GitHub Issue #20):

“Short segments (word length or shorter) can cause errors from Praat even with fixes, as Praat needs a certain minimum window size to get good results, though phrase-length or longer segments work fine.”

PraatExecutionFailed errors:

• Occurs when optional arguments receive incorrect values
• Praat’s error messages may be cryptic
• Requires debugging Praat script directly

Workaround:

• Use Parselmouth for acoustic analysis (no external Praat needed)
• Use praatio only for TextGrid manipulation

5. Comparison: praatio vs. TextGridTools vs. Parselmouth#

5.1 Feature Comparison#

5.2 praatio Advantages#

✅ Choose praatio if:

1. Advanced TextGrid manipulation required
2. Multiple output formats needed (JSON export)
3. Existing Praat script workflows to integrate
4. Batch processing utilities helpful

5.3 Parselmouth Advantages#

✅ Choose Parselmouth if:

1. Acoustic analysis + TextGrid manipulation in one package
2. No external Praat installation possible
3. Active maintenance important
4. TextGridTools integration via to_tgt() sufficient

Verdict: For most tone analysis workflows, Parselmouth is superior because it combines TextGrid manipulation with acoustic analysis in a more integrated, actively maintained package.

6. Practical Workflow Example: Mandarin Tone Corpus#

6.1 Complete Pipeline#

from praatio import textgrid, audio
from pathlib import Path
import pandas as pd
import parselmouth  # Using Parselmouth for F0 extraction

def process_mandarin_corpus(
    audio_dir='corpus/audio/',
    textgrid_dir='corpus/textgrids/',
    output_csv='tone_features.csv'
):
    """
    Extract tone features from Mandarin corpus with TextGrid annotations.
    """

    results = []

    audio_files = Path(audio_dir).glob('*.wav')

    for audio_file in audio_files:
        # Find corresponding TextGrid
        tg_file = Path(textgrid_dir) / f"{audio_file.stem}.TextGrid"

        if not tg_file.exists():
            print(f"Warning: No TextGrid for {audio_file.name}")
            continue

        # Load TextGrid (using praatio)
        tg = textgrid.openTextgrid(str(tg_file))

        # Get syllable tier
        syllable_tier = tg.getTier('syllables')

        # Load audio (using Parselmouth for F0 extraction)
        sound = parselmouth.Sound(str(audio_file))
        pitch = sound.to_pitch_ac(pitch_floor=75, pitch_ceiling=500)

        # Process each syllable
        for i, (start, end, label) in enumerate(syllable_tier.entries):
            if not label.strip():
                continue

            # Extract F0 values in interval
            f0_values = []
            for t in pitch.xs():
                if start <= t <= end:
                    f0 = pitch.get_value_at_time(t)
                    if f0 > 0:
                        f0_values.append(f0)

            if len(f0_values) < 3:
                continue  # Insufficient data

            # Compute features
            import numpy as np
            results.append({
                'file': audio_file.name,
                'syllable_index': i,
                'syllable': label,
                'start': start,
                'end': end,
                'duration': end - start,
                'f0_mean': np.mean(f0_values),
                'f0_std': np.std(f0_values),
                'f0_min': np.min(f0_values),
                'f0_max': np.max(f0_values),
                'f0_range': np.max(f0_values) - np.min(f0_values)
            })

    # Save to CSV
    df = pd.DataFrame(results)
    df.to_csv(output_csv, index=False)

    print(f"Processed {len(results)} syllables -> {output_csv}")

    return df

# Usage
df = process_mandarin_corpus()
print(df.head())

6.2 Create TextGrid from Forced Alignment#

from praatio.data_classes import textgrid, interval_tier

def create_textgrid_from_alignment(
    audio_path,
    alignment_data,
    output_path
):
    """
    Create TextGrid from forced alignment output.

    Args:
        audio_path: Path to audio file
        alignment_data: List of (start, end, label) tuples
        output_path: Path for output TextGrid
    """

    # Get audio duration
    sound = parselmouth.Sound(audio_path)
    duration = sound.duration

    # Create TextGrid
    tg = textgrid.Textgrid()
    tg.minTimestamp = 0.0
    tg.maxTimestamp = duration

    # Create word tier
    word_tier = interval_tier.IntervalTier(
        name='words',
        entries=alignment_data,
        minT=0.0,
        maxT=duration
    )

    tg.addTier(word_tier)

    # Save
    tg.save(output_path, format='long_textgrid')

    return tg

# Example alignment data (from Montreal Forced Aligner, etc.)
alignment = [
    (0.0, 0.5, 'ni3'),
    (0.5, 1.0, 'hao3'),
    (1.0, 1.5, 'ma1'),
    (1.5, 2.0, '')
]

create_textgrid_from_alignment(
    audio_path='greeting.wav',
    alignment_data=alignment,
    output_path='greeting.TextGrid'
)

7. Limitations & Workarounds#

7.1 Known Limitations#

1. External Praat dependency for acoustic analysis

   • Workaround: Use Parselmouth for F0/formant extraction

2. Short segment issues with Praat scripts

   • Workaround: Process only phrase-length or longer segments

3. Limited maintenance compared to Parselmouth

   • Workaround: Use Parselmouth for new projects

4. No built-in interannotator agreement metrics

   • Workaround: Use TextGridTools for this feature

5. Manual error handling for Praat script failures

   • Workaround: Wrap in try-except with fallback logic

7.2 Best Practices#

File Management:

• Use consistent naming: audio.wav + audio.TextGrid
• Store TextGrids in separate directory from audio
• Use version control for TextGrid files

Annotation Guidelines:

• Enforce tier naming conventions across corpus
• Use empty intervals for pauses (don’t delete them)
• Document tier structure in README

Quality Control:

• Always validate TextGrid structure after modifications
• Check for overlapping intervals
• Verify tier boundaries align with audio duration

Performance:

• Cache loaded TextGrids if accessing multiple times
• Use batch processing for large corpora
• Consider parallel processing with multiprocessing

8. Migration Guide: praatio → Parselmouth#

If you’re using praatio primarily for TextGrid manipulation, consider migrating to Parselmouth:

8.1 Equivalent Operations#

8.2 Migration Example#

Before (praatio):

from praatio import textgrid

tg = textgrid.openTextgrid('annotation.TextGrid')
tier = tg.getTier('words')

for start, end, label in tier.entries:
    print(f"{label}: {start} - {end}")

After (Parselmouth):

import parselmouth

tg = parselmouth.TextGrid.read('annotation.TextGrid')
tier = tg['words']

for interval in tier.intervals:
    print(f"{interval.text}: {interval.xmin} - {interval.xmax}")

8.3 What You Gain#

• ✅ Acoustic analysis built-in (no external Praat)
• ✅ Active maintenance (v0.5.0.dev0, January 2026)
• ✅ Identical Praat accuracy for F0/formants
• ✅ Zero external dependencies

8.4 What You Lose#

• ⚠️ Multiple output formats (Parselmouth has fewer)
• ⚠️ Some batch processing utilities (need to rebuild)
• ⚠️ Specific praatio convenience functions

9. Summary Recommendations#

✅ Use praatio if:#

1. Legacy workflows with existing praatio code
2. JSON export required for TextGrids
3. Specialized TextGrid manipulation not available in Parselmouth
4. Already using Praat externally for acoustic analysis

⚠️ Consider alternatives:#

1. Parselmouth - For integrated TextGrid + acoustic analysis
2. TextGridTools - For interannotator agreement metrics
3. Custom scripts - For simple TextGrid parsing (standard text format)

Overall Verdict:#

For new projects involving CJK tone analysis, use Parselmouth instead of praatio. It provides:

• TextGrid manipulation (sufficient for most needs)
• Built-in acoustic analysis (no external Praat)
• Active development and maintenance
• Identical Praat accuracy

Use praatio only if you specifically need its advanced TextGrid manipulation features or JSON export capabilities.

Sources#

• praatio GitHub Repository
• praatio PyPI Package
• praatio API Documentation
• praatio TextGrid API
• praatio Issue #20: Wrong parameters with get_pitch_and_intensity
• TextGridTools Documentation
• Praat Scripting Tutorial
• Praat Batch Processing Examples

S2 Comprehensive: Tone Classification Algorithms#

Executive Summary#

Tone classification has evolved from traditional statistical methods (HMM, GMM) to modern deep learning approaches (CNN, RNN). For Mandarin Chinese, CNNs achieve 87-88% accuracy with end-to-end learning from raw features, while hybrid CNN-LSTM models with attention mechanisms represent the current state-of-the-art.

Key Findings:

• Traditional: HMM/GMM (84-85% accuracy)
• Deep Learning: CNN (87-88%), RNN (85-90%), CNN-LSTM (90%+)
• Feature Engineering: Critical for traditional methods, less important for deep learning
• Best Practices: Speaker normalization, time-normalization, data augmentation

1. Overview of Approaches#

1.1 Taxonomy of Methods#

Tone Classification Methods
├── Traditional Statistical
│   ├── Hidden Markov Models (HMM)
│   ├── Gaussian Mixture Models (GMM)
│   └── Support Vector Machines (SVM)
├── Classical Machine Learning
│   ├── Random Forest
│   ├── Decision Trees
│   └── k-Nearest Neighbors
└── Deep Learning
    ├── Convolutional Neural Networks (CNN)
    ├── Recurrent Neural Networks (RNN/LSTM)
    ├── Hybrid CNN-LSTM
    └── Attention-based Transformers

1.2 Performance Comparison (Mandarin)#

Trend: Deep learning approaches consistently outperform traditional statistical methods, with hybrid architectures achieving the best results.

2. Hidden Markov Models (HMM)#

2.1 Overview#

HMMs model tones as sequences of hidden states with observable F0 features. They capture temporal dynamics of tone contours.

Key Concept:

• Hidden states: Discrete tone categories (T1, T2, T3, T4)
• Observations: F0 features (mean, trajectory, derivatives)
• Transitions: Probability of tone sandhi or coarticulation

2.2 Architecture#

HMM Tone Model
┌─────────────────────────────────────┐
│  State 1 (T1)  →  State 2 (T2)     │  Hidden Layer
│      ↓               ↓               │
│  F0 Features    F0 Features         │  Observable Layer
│  [f0_1, Δf0]    [f0_2, Δf0]        │
└─────────────────────────────────────┘

2.3 Feature Extraction for HMM#

LDA-MLLT Method (Linear Discriminant Analysis + Maximum Likelihood Linear Transform):

“For GMM-HMM based acoustic model, utilization of spliced features is often achieved using LDA-MLLT method.”

Feature Splicing:

“Feature splicing has greatly improved tone classification performance, yielding 5.3% absolute improvement in RNN-based models.”

Common Features:

• F0 contour (sampled at fixed intervals)
• Δ F0 (first derivative)
• Δ² F0 (second derivative / acceleration)
• F0 from neighboring syllables (context)

2.4 Code Example#

import numpy as np
from hmmlearn import hmm

class ToneHMM:
    """
    Hidden Markov Model for Mandarin tone classification.
    """

    def __init__(self, n_tones=4, n_components=3):
        """
        Args:
            n_tones: Number of tone categories (4 for Mandarin)
            n_components: Number of hidden states per tone
        """
        self.n_tones = n_tones
        self.n_components = n_components
        self.models = []

        # Create one HMM per tone
        for i in range(n_tones):
            model = hmm.GaussianHMM(
                n_components=n_components,
                covariance_type='diag',
                n_iter=100
            )
            self.models.append(model)

    def extract_features(self, f0_contour):
        """
        Extract features: [f0, Δf0, Δ²f0]
        """
        # Normalize F0
        f0_norm = (f0_contour - np.mean(f0_contour)) / np.std(f0_contour)

        # First derivative
        delta_f0 = np.diff(f0_norm, prepend=f0_norm[0])

        # Second derivative
        delta2_f0 = np.diff(delta_f0, prepend=delta_f0[0])

        # Stack features
        features = np.column_stack([f0_norm, delta_f0, delta2_f0])

        return features

    def train(self, X_train, y_train):
        """
        Train one HMM per tone.

        Args:
            X_train: List of F0 contours
            y_train: Tone labels (0=T1, 1=T2, 2=T3, 3=T4)
        """
        for tone in range(self.n_tones):
            # Get training samples for this tone
            tone_samples = [X_train[i] for i in range(len(X_train)) if y_train[i] == tone]

            # Extract features
            tone_features = [self.extract_features(sample) for sample in tone_samples]

            # Concatenate with lengths
            lengths = [len(f) for f in tone_features]
            X_concat = np.vstack(tone_features)

            # Train HMM
            self.models[tone].fit(X_concat, lengths)

    def predict(self, f0_contour):
        """
        Classify tone using log-likelihood.
        """
        features = self.extract_features(f0_contour)

        # Compute log-likelihood for each tone
        scores = []
        for model in self.models:
            score = model.score(features)
            scores.append(score)

        # Return tone with highest likelihood
        tone = np.argmax(scores)
        return tone, scores

# Usage example
hmm_classifier = ToneHMM(n_tones=4, n_components=3)

# Training data (placeholder)
X_train = [np.random.randn(10) for _ in range(100)]  # F0 contours
y_train = np.random.randint(0, 4, 100)  # Tone labels

hmm_classifier.train(X_train, y_train)

# Predict
f0_test = np.array([0.5, 0.8, 1.2, 1.5, 1.8])  # Rising tone (T2)
tone, scores = hmm_classifier.predict(f0_test)
print(f"Predicted tone: T{tone+1}, Scores: {scores}")

2.5 Advantages & Limitations#

✅ Advantages:

• Models temporal dynamics naturally
• Handles variable-length sequences
• Interpretable (state transitions = linguistic rules)

❌ Limitations:

• Requires manual feature engineering
• Assumes Markov property (limited context)
• Outperformed by deep learning methods

3. Gaussian Mixture Models (GMM)#

3.1 Overview#

GMMs model tone F0 distributions as mixtures of Gaussian components. Each tone is represented by a unique probability distribution.

Key Concept:

• Each tone = mixture of K Gaussians
• F0 features → probability density
• Classification = maximum likelihood

3.2 Architecture#

GMM Tone Model (Tone 1)
┌────────────────────────────────┐
│  Gaussian 1   Gaussian 2   Gaussian 3  │
│   (μ₁, Σ₁)     (μ₂, Σ₂)     (μ₃, Σ₃)  │
│      π₁           π₂           π₃       │
└────────────────────────────────┘
         ↓
   F0 Features → P(X | Tone 1)

3.3 Overlapped Ditone Modeling (Cantonese)#

“Overlapped ditone modeling has been used for tone recognition in continuous Cantonese speech, incorporating contextual pitch features for GMM-based tone models.”

Ditone concept: Model two consecutive tones jointly to capture coarticulation effects.

3.4 Code Example#

from sklearn.mixture import GaussianMixture
import numpy as np

class ToneGMM:
    """
    Gaussian Mixture Model for tone classification.
    """

    def __init__(self, n_tones=4, n_components=3):
        """
        Args:
            n_tones: Number of tone categories
            n_components: Number of Gaussian components per tone
        """
        self.n_tones = n_tones
        self.models = []

        # Create one GMM per tone
        for i in range(n_tones):
            model = GaussianMixture(
                n_components=n_components,
                covariance_type='full',
                max_iter=100,
                random_state=42
            )
            self.models.append(model)

    def extract_features(self, f0_contour):
        """
        Extract statistical features from F0 contour.
        """
        # Time-normalize to 5 points
        from scipy.interpolate import interp1d
        time_orig = np.linspace(0, 1, len(f0_contour))
        time_new = np.linspace(0, 1, 5)
        f = interp1d(time_orig, f0_contour, kind='cubic')
        f0_5points = f(time_new)

        # Z-score normalization
        f0_norm = (f0_5points - np.mean(f0_5points)) / np.std(f0_5points)

        # Features: [f0_1, f0_2, f0_3, f0_4, f0_5, mean, std, range]
        features = np.concatenate([
            f0_norm,
            [np.mean(f0_5points), np.std(f0_5points), np.ptp(f0_5points)]
        ])

        return features

    def train(self, X_train, y_train):
        """
        Train one GMM per tone.
        """
        for tone in range(self.n_tones):
            # Get training samples for this tone
            tone_samples = [X_train[i] for i in range(len(X_train)) if y_train[i] == tone]

            # Extract features
            tone_features = np.array([self.extract_features(sample) for sample in tone_samples])

            # Train GMM
            self.models[tone].fit(tone_features)

    def predict(self, f0_contour):
        """
        Classify tone using log-likelihood.
        """
        features = self.extract_features(f0_contour).reshape(1, -1)

        # Compute log-likelihood for each tone
        scores = []
        for model in self.models:
            score = model.score(features)
            scores.append(score)

        # Return tone with highest likelihood
        tone = np.argmax(scores)
        return tone, scores

# Usage
gmm_classifier = ToneGMM(n_tones=4, n_components=3)

# Train (placeholder data)
X_train = [np.random.randn(10) for _ in range(100)]
y_train = np.random.randint(0, 4, 100)
gmm_classifier.train(X_train, y_train)

# Predict
f0_test = np.array([200, 210, 220, 230, 240])  # Rising tone
tone, scores = gmm_classifier.predict(f0_test)
print(f"Predicted tone: T{tone+1}")

3.5 Advantages & Limitations#

✅ Advantages:

• Simple, interpretable
• Fast training and inference
• Works well with limited data

❌ Limitations:

• Assumes fixed feature dimensionality
• Doesn’t model temporal dynamics well
• Lower accuracy than deep learning

4. Convolutional Neural Networks (CNN)#

4.1 Overview#

CNNs automatically learn hierarchical features from raw spectrograms or mel-spectrograms, eliminating manual feature engineering.

Key Innovation:

“CNN-based methods fully automate tone classification of syllables in Mandarin Chinese, taking raw tone data as input and achieving substantially higher accuracy compared with previous techniques based on manually edited F0.”

4.2 ToneNet Architecture#

ToneNet is a CNN model designed specifically for Mandarin tone classification: