Faculty - Dashboard

@inproceedings{bib_Telu_2025, AUTHOR = {Bhavana, Akkiraju V N S and Bandarupalli, Srihari and Sambangi, Swathi and Ravuri, Vasavi and Saraswathi, R Vijaya and Vuppala, Anil Kumar }, TITLE = {TeluguST-46: A Benchmark Corpus and Comprehensive Evaluation for Telugu-English Speech Translation}, BOOKTITLE = {International Joint Conference on Natural Language Processing}. YEAR = {2025}}

Despite Telugu being spoken by over 80 million people, speech translation research for this morphologically rich language remains severely underexplored. We address this gap by developing a high-quality Telugu–English speech translation benchmark from 46 hours of manually verified CSTD corpus data (30h/8h/8h train/dev/test split). Our systematic comparison of cascaded versus end-to-end architectures shows that while IndicWhisper + IndicMT achieves the highest performance due to extensive Telugu-specific training data, finetuned SeamlessM4T models demonstrate remarkable competitiveness despite using significantly less Telugu-specific training data. This finding suggests that with careful hyperparameter tuning and sufficient parallel data (potentially less than 100 hours), end-to-end systems can achieve performance comparable to cascaded approaches in low-resource settings. Our metric reliability study evaluating BLEU, METEOR, ChrF++, ROUGE-L, TER, and BERTScore against human judgments reveals that traditional metrics provide better quality discrimination than BERTScore for Telugu-English translation. The work delivers three key contributions: a reproducible Telugu– English benchmark, empirical evidence of competitive end-to-end performance potential in low-resource scenarios, and practical guidance for automatic evaluation in morphologically complex language pairs.

@inproceedings{bib_Effi_2025, AUTHOR = {Bandarupalli, Srihari and Bhavana, Akkiraju V N S and Charan, Devarakonda Sri and Simha, Narasinga Vamshi Raghu and Vuppala, Anil Kumar }, TITLE = {Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data}, BOOKTITLE = {International Joint Conference on Natural Language Processing}. YEAR = {2025}}

Automatic speech recognition for low-resource languages remains fundamentally constrained by the scarcity of labeled data and compu- tational resources required by state-of-the-art models. We present a systematic investigation into cross-lingual continuous pretraining for low-resource languages, using Perso-Arabic languages (Persian, Arabic, and Urdu) as our primary case study. Our approach demon- strates that strategic utilization of unlabeled speech data can effectively bridge the resource gap without sacrificing recognition accuracy. We construct a 3,000-hour multilingual corpus through a scalable unlabeled data collection pipeline and employ targeted continual pretrain- ing combined with morphologically-aware tok- enization to develop a 300M parameter model that achieves performance comparable to sys- tems 5 times larger. Our model outperforms Whisper Large v3 (1.5B parameters) on Persian and achieves competitive results on Arabic and Urdu despite using significantly fewer parame- ters and substantially less labeled data. These findings challenge the prevailing assumption that ASR quality scales primarily with model size, revealing instead that data relevance and strategic pretraining are more critical factors for low-resource scenarios. This work provides a practical pathway toward inclusive speech tech- nology, enabling effective ASR for underrepre- sented languages without dependence on mas- sive computational infrastructure or proprietary datasets.

@inproceedings{bib_Towa_2025, AUTHOR = {Sivaramasethu, Harinie and Bhavana, Akkiraju V N S and Simha, Narasinga Vamshi Raghu and Bandarupalli, Srihari and Kak, Aadil and Vuppala, Anil Kumar }, TITLE = {Towards a Common Phone Space for Arabic, Persian, and Urdu: Leveraging Shared Phonology for Low-Resource G2P}, BOOKTITLE = {Workshop on Speech and Language Technology in Education}. YEAR = {2025}}

Educational technologies like reading tutors and assistive applications rely on accurate pronunciation modeling. Low resource Perso-Arabic languages such as Persian and Urdu lack sufficient Grapheme-to-Phoneme resources compared to Arabic, limiting use in educational tools. We address this by exploiting phonological similarities between Arabic, Persian, and Urdu to construct a unified Common Phone Space (CPS-APU) that enables cross-lingual knowledge transfer. Using linguistic expertise, we develop a shared phoneme mapping across the three languages. To evaluate the hypothesis that lower-resource languages benefit from such shared representations, we test and enrich our G2P parser across three stages: (A) expert-authored rules, (B) corpus-mined rules from alignments, and (C) LLM-powered generation using lightweight instruction-tuned models. CPS framework consistently outperforms monolingual approaches for low-resource languages, with the LLM- augmented stage yielding the best results.

@inproceedings{bib_StuD_2025, AUTHOR = {Khanna, Pragya and Kommagouni, Parvathi Priyanka and Simha, Narasinga Vamshi Raghu and Vuppala, Anil Kumar }, TITLE = {StuD: A Multimodal Approach for Stuttering Detection with RAG and Fusion Strategies}, BOOKTITLE = {International Joint Conference on Natural Language Processing}. YEAR = {2025}}

Stuttering is a complex speech disorder that challenges both ASR systems and clinical assessment. We propose a multimodal stuttering detection and classification model that integrates acoustic and linguistic features through a two-stage fusion mechanism. Fine-tuned Wav2Vec 2.0 and HuBERT extract acoustic embeddings, which are early fused with MFCC features to capture fine-grained spectral and phonetic variations, while Llama-2 embeddings from Whisper ASR transcriptions provide linguistic context. To enhance robustness against out-of-distribution speech patterns, we incorporate Retrieval-Augmented Generation or adaptive classification. Our model achieves state-of-the-art performance on SEP-28k and FluencyBank, demonstrating significant improvements in detecting challenging stuttering events. Additionally, our analysis highlights the complementary nature of acoustic and linguistic modalities, reinforcing the need for multimodal approaches in speech disorder detection.

@inproceedings{bib_Dysa_2025, AUTHOR = {Anuprabha, and Gurugubelli, Krishna and Vuppala, Anil Kumar }, TITLE = {Dysarthric Speech Intelligibility Assessment by Custom Keyword Spotting}, BOOKTITLE = {Journal of Selected Topics in Signal Processing}. YEAR = {2025}}

Traditionally, dysarthric speech intelligibility assessment systems have focused on speech as the primary input, utilizing methods such as extraction of relevant speech features, classification models, alignment of Automatic Speech Recognition (ASR) outputs, and comparisons between speech representations of dysarthric and healthy speakers. However, to achieve an automated intelligibility assessment that closely mirrors the auditory-perceptual evaluations conducted by clinicians, a model that captures both the acoustic characteristics of dysarthric speech and the linguistic structure related to word pronunciation are needed. Inspired by the practices of clinicians, this study introduces a novel text-guided dysarthric speech intelligibility assessment framework that leverages custom keyword spotting (DySIA-CKWS). The model evaluates intelligibility by detecting specific keywords and is extensively tested using UA-Speech database for speaker-wise analysis and across word groups of varying complexity. To ensure robustness, the system's performance is further validated on TORGO database, demonstrating its adaptability in cross-database settings. Statistical analysis demonstrates strong alignment between predicted and subjective intelligibility scores, with a Pearson Correlation Coefficient (PCC) of 0.9588 and a Spearman's Correlation Coefficient (SCC) of 0.9141, achieved using the proposed system on the UA-Speech database. The findings emphasize the importance of word selection and showcase the model's effectiveness in diagnosing dysarthric speech, offering a significant advancement in intelligibility assessment methodologies.

@inproceedings{bib_Towa_2025, AUTHOR = {Kommagouni, Parvathi Priyanka and Khanna, Pragya and Simha, Narasinga Vamshi Raghu and Anirudh, Bocha Venkata Krishna Sai and Vuppala, Anil Kumar }, TITLE = {Towards Classification of Typical and Atypical Disfluencies: A Self Supervised Representation Approach}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2025}}

This paper investigates the nuanced distinctions between typical and atypical speech disfluencies, focusing on features captured by self-supervised models. Typical disfluencies are natural, non-pathological irregularities in speech, while atypical disfluencies are linked to speech disorders like stuttering, characterized by more frequent and severe disruptions. Despite progress in automatic disfluency detection, limited research addresses the direct classification of these two types. This study leverages intermediate representations from four pretrained models Wav2Vec2.0, HuBERT, WavLM, and TERA, to analyze and classify typical and stuttered disfluencies. The experiments utilize two novel Indian English datasets, IIITH-IEDE and IIITH-TISA, enabling a comprehensive analysis of disfluency patterns in a linguistically diverse context. Classification experiments with support vector machines (SVM) and convolutional neural networks (CNN) reveal that features from HuBERT’s 5th layer—balancing low-level acoustic and high-level semantic information—achieve a peak F1 score of 0.97. These findings highlight the importance of intermediate layer representations of self-supervised models in distinguishing nuanced speech variations and contribute to robust and interpretable systems for speech disfluency classification.

@inproceedings{bib_Fair_2025, AUTHOR = {Anuprabha, and Gurugubelli, Krishna and Vuppala, Anil Kumar }, TITLE = {Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTS}, BOOKTITLE = {Proceedings of Interspeech}. YEAR = {2025}}

Dysarthric speech poses significant challenges in developing as- sistive technologies, primarily due to the limited availability of data. Recent advances in neural speech synthesis, especially zero-shot voice cloning, facilitate synthetic speech generation for data augmentation; however, they may introduce biases to- wards dysarthric speech. In this paper, we investigate the effec- tiveness of state-of-the-art F5-TTS in cloning dysarthric speech using TORGO dataset, focusing on intelligibility, speaker sim- ilarity, and prosody preservation. We also analyze potential biases using fairness metrics like Disparate Impact and Parity Difference to assess disparities across dysarthric severity lev- els. Results show that F5-TTS exhibits a strong bias toward speech intelligibility over speaker and prosody preservation in dysarthric speech synthesis. Insights from this study can help integrate fairness-aware dysarthric speech synthesis, fostering the advancement of more inclusive speech technologies.

@inproceedings{bib_Spee_2025, AUTHOR = {Simha, Narasinga Vamshi Raghu and Khan, Hina Fathima Fazal and Sai, Motepalli Kowshik Siva and Mahesh, Sangeetha and Abraham, ajish K and Vuppala, Anil Kumar }, TITLE = {Speech Signal Processing based Stuttering Classification and Clinical Studies}, BOOKTITLE = {Circuits, Systems, and Signal Processing}. YEAR = {2025}}

Automatic detection and classification of stuttering events remain a challenging problem in speech processing, particularly due to the variability of disfluency types and the scarcity of large annotated datasets. This paper presents a signal processing-based approach for stuttering classification that avoids the data-intensive requirements of machine learning models. Our method focuses on syllable-level analysis using key acoustic parameters such as energy patterns, pitch contours, spectral stability, and temporal features, implemented through rule-based classification with predefined thresholds. The methodology involves automatic syllable segmentation followed by rule-based classification using thresholds on acoustic features. The dataset comprises annotated read and spontaneous speech from 106 Kannada-speaking adults, including 26 new participants evaluated during clinical trials. Three certified Speech-Language Pathologists (SLPs) conducted perceptual evaluations, and system outputs were validated using intraclass correlation (ICC), yielding strong agreement (textgreater 0.82) across stuttering types. The system achieved classification accuracies of 89% for blocks, 83% for repetitions, and 81% for prolongations. However, the system has several limitations: it relies on static threshold values, exhibits sensitivity to noise, and lacks automated severity grading. Additionally, generalizability across languages and speaking conditions requires further exploration. Machine learning models such as SVM, TDNN, and LSTM were trained and achieved a performance of around 30 to 60% accuracy. Our approach offers high interpretability, low computational cost, and real-time feasibility compared to machine learning methods. Future work will focus on adaptive thresholding, noise-robust processing, and hybrid approaches integrating signal processing with lightweight learning-based refinement.

@inproceedings{bib_IIIT_2025, AUTHOR = {Bhavana, Akkiraju V N S and Pothula, Aishwarya and Kesiraju, Santosh and Vuppala, Anil Kumar }, TITLE = {IIITH-BUT system for IWSLT 2025 low-resource Bhojpuri to Hindi speech translation}, BOOKTITLE = {International Conference on Spoken Language Translation}. YEAR = {2025}}

This paper presents the submission of IIITHBUT to the IWSLT 2025 shared task on speech translation for the low-resource Bhojpuri-Hindi language pair. We explored the impact of hyperparameter optimisation and data augmentation techniques on the performance of the SeamlessM4T model fine-tuned for this specific task. We systematically investigated a range of hyperparameters including learning rate schedules, number of update steps, warm-up steps, label smoothing, and batch sizes; and report their effect on translation quality. To address data scarcity, we applied speed perturbation and SpecAugment and studied their effect on translation quality. We also examined the use of cross-lingual signal through joint training with Marathi and Bhojpuri speech data. Our experiments reveal that careful selection of hyperparameters and the application of simple yet effective augmentation techniques significantly improve performance in low-resource settings. We also analysed the translation hypotheses to understand various kinds of errors that impacted the translation quality in terms of BLEU

@inproceedings{bib_End-_2025, AUTHOR = {Pothula, Aishwarya and Bhavana, Akkiraju V N S and Charan, Devarakonda Sri and Bandarupalli, Srihari and Kesiraju, Santosh and Vuppala, Anil Kumar }, TITLE = {End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2025}}

The scarcity of high-quality annotated data presents a sig- nificant challenge in developing effective end-to-end speech- to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly la- beled data can be used to build ST models for low-resource lan- guage pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for lan- guage pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance compa- rable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.

@inproceedings{bib_Exag_2025, AUTHOR = {Mondal, Anindita and Monica, Surtani and Vuppala, Anil Kumar and Krishnamurthy, Parameswari and Yarra, Chiranjeevi }, TITLE = {ExagTTS: An Approach Towards Controllable Word Stress Incorporated TTS for Exaggerated Synthesized Speech Aiding Second Language Learners}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2025}}

Computer-Assisted Language Learning systems provide speech exaggerating at mispronounced word locations as a feedback to the L2 learners. Traditionally, expert speakers recordings (which limit the scalability) are considered for this task though there are advancements in text-to-speech (TTS) that can generate native like natural sounding speech. To address these, this work proposes two novel controllable strategies for scalable speech exaggeration. One strategy is direct speech exaggeration that incorporates the proposed label conditioned tokenization in GlowTTS. Another strategy is cascading the state-of-the-art TTS to a WORLD vocoder with proposed energy and duration modifications. A subset of Tatoeba corpus, that we annotated with prominent words, is used for experimentation. Automatic and manual assessment reveals that the exaggerated speech quality from both direct and cascaded strategy with duration modification is closer to the prominent words in the native speaker's speech.

@inproceedings{bib_ProB_2025, AUTHOR = {Mondal, Anindita and Biju, Rahul and Vuppala, Anil Kumar and Cherian, Reni K and Yarra, Chiranjeevi }, TITLE = {ProBiEM: Acoustic and Lexical Correlates of Prosodic Prominence in English-Malayalam Bilingual Speech}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2025}}

Despite advancements in Speech-to-Speech Translation, maintaining expressiveness between the source and target speech, particularly between English and Indian languages, remains challenging. This study investigates prosodic similarities and variations between English and Malayalam, a language spoken in southern India. A set of 22 prompts from the IViE corpus, covering five categories—simple sentences, WH-questions, questions without morphosyntactic markers, inversion questions, and coordinations—was selected. These prompts, originally spoken by UK speakers, were translated into Malayalam and both language prompts were recorded by bilingual Malayalam speakers while preserving expressiveness. Word-level prominence was manually annotated, and comparisons were made across Indian English, Malayalam, and UK English. The analysis reveals that prominence is retained at key points in Indian English, whereas in Malayalam, it is low due to question specific diacritics and agglutinative nature.

@inproceedings{bib_Enha_2025, AUTHOR = {Khan, Mohammed Nasir Ali and Vuppala, Anil Kumar and Khan, Mohammed Zafar Ali and Azeemuddin, Syed }, TITLE = {Enhanced Identification of Transgenic Corn Using UV-Vis Spectroscopy}, BOOKTITLE = {Photonics and Electromagnetics Research Symposium}. YEAR = {2025}}

Abstract— Transgenic crops have become widespread but have associated health concerns. Testing for transgenic corn is slow and lengthy. In this paper, we used UV-Vis spectroscopy in combination with chemometrics and the PCA-SVM classifier to identify different types of corn samples from each other, including transgenic varieties. The presented methodology and the classifier can differentiate very well between the three different varieties of corn considered in this paper. Although there is a slight improvement in prediction accuracies compared to the state-of-the-art methods, our method is considerably more cost-effective and less time-consuming.

@inproceedings{bib_A_Pr_2025, AUTHOR = {Mondal, Anindita and Bharadwaj, Rangavajjala Sankara and Jhansi, Mallela and Vuppala, Anil Kumar and Yarra, Chiranjeevi }, TITLE = {A Preliminary Analysis of Automatic Word and Syllable Prominence Detection in Non-Native Speech With Text-to-Speech Prosody Embeddings}, BOOKTITLE = {Technical Report}. YEAR = {2025}}

Automatic detection of prominence at the word and syllable-levels is critical for building computer-assisted language learning systems. It has been shown that prosody embeddings learned by the current state-of-the-art (SOTA) text-to-speech (TTS) systems could generate word- and syllable-level prominence in the synthesized speech as natural as in native speech. To understand the effectiveness of prosody embeddings from TTS for prominence detection under nonnative context, a comparative analysis is conducted on the embeddings extracted from native and non-native speech considering the prominence-related embeddings: duration, energy, and pitch from a SOTA TTS named FastSpeech2. These embeddings are extracted under two conditions considering: 1) only text, 2) both speech and text. For the first condition, the embeddings are extracted directly from the TTS inference mode, whereas for the second condition, we propose to extract from the TTS under training mode. Experiments are conducted on native speech corpus: Tatoeba, and non-native speech corpus: ISLE. For experimentation, word-level prominence locations are manually annotated for both corpora. The highest relative improvement on word & syllable-level prominence detection accuracies with the TTS embeddings are found to be 13.7% & 5.9% and 16.2% & 6.9% compared to those with the heuristic-based features and self-supervised Wav2Vec-2.0 representations, respectively.

@inproceedings{bib_Typi_2025, AUTHOR = {Kommagouni, Parvathi Priyanka and Simha, Narasinga Vamshi Raghu and Barche, Purva and C, Sai Akarsh and Vuppala, Anil Kumar }, TITLE = {Typical vs. Atypical Disfluency Classification: Introducing the IIITH-TISA Corpus and Temporal Context-Based Feature Representations}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2025}}

—Speech disfluencies in spontaneous communication can be categorized as either typical or atypical. Typical disfluencies, such as hesitations and repetitions, are natural occurrences in everyday speech, while atypical disfluencies are indicative of pathological disorders like stuttering. Distinguishing between these categories is crucial for improving voice assistants (VAs) for Persons Who Stutter (PWS), who often face premature cutoffs due to misidentification of speech termination. Accurate classification also aids in detecting stuttering early in children, preventing misdiagnosis as language development disfluency. This research introduces the IIITH-TISA dataset, the first Indian English stammer corpus, capturing atypical disfluencies. Additionally, we extend the IIITH-IED dataset with detailed annotations for typical disfluencies. We propose Perceptually Enhanced Zero-Time Windowed Cepstral Coefficients (PE-ZTWCC) combined with Shifted Delta Cepstra (SDC) as input features to a shallow Time Delay Neural Network (TDNN) classifier, capturing both local and wider temporal contexts. Our method achieves an average F1 score of 85.01% for disfluency classification, outperforming traditional features.

@inproceedings{bib_Enha_2025, AUTHOR = {Simha, Narasinga Vamshi Raghu and Kommagouni, Parvathi Priyanka and Sridhar, Vanga and Sai, Motepalli Kowshik Siva and C, Sai Akarsh and Barche, Purva and Vuppala, Anil Kumar }, TITLE = {Enhancing Stutter Detection using Long-Term Average Spectrum Values}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2025}}

Stuttering is recognized as a prevalent speech disorder that significantly affects individuals worldwide. Identifying and diagnosing in the early stages enhances the quality of life for individuals experiencing atypical speech patterns. Traditional methods for classifying stuttering primarily depend on subjective assessments and short-term acoustic analysis. Although helpful, these methods face limitations in accurately capturing all stutter types due to their inherent subjectivity and temporal constraints. This study uses the long-term average spectrum (LTAS) values for stutter classification derived from various filter banks such as Constant Q, Gamma-tone, and Single-frequency filter banks. It also compares these LTAS-based methods with cepstral coefficients, such as MFCC and ZTWCC. Classifiers such as SVM, LSTM, and Bi-LSTM deep networks were used to study the effectiveness of these representations in accurately discerning stuttered speech from fluent speech and reported the results.

@inproceedings{bib_A_Mu_2025, AUTHOR = {Anuprabha, and Gurugubelli, Krishna and Vuppala, Anil Kumar and V, Kesavaraj }, TITLE = {A Multi-modal Approach to Dysarthria Detection and Severity Assessment Using Speech and Text Information}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2025}}

Automatic detection and severity assessment of dysarthria are crucial for delivering targeted therapeutic interventions to patients. While most existing research focuses primarily on speech modality, this study introduces a novel approach that leverages both speech and text modalities. By employing cross-attention mechanism, our method learns the acoustic and linguistic similarities between speech and text representations. This approach assesses specifically the pronunciation deviations across different severity levels, thereby enhancing the accuracy of dysarthric detection and severity assessment. All the experiments have been performed using UA-Speech dysarthric database. Improved accuracies of 99.53% and 93.20% in detection, and 98.12% and 51.97% for severity assessment have been achieved when speaker-dependent and speaker-independent, unseen and seen words settings are used. These findings suggest that by integrating text information, which provides a reference linguistic knowledge, a more robust framework has been developed for dysarthric detection and assessment, thereby potentially leading to more effective diagnoses.

@inproceedings{bib_Cust_2024, AUTHOR = {Vuppala, Anil Kumar }, TITLE = {Custom wake word detection}, BOOKTITLE = {Proceedings of Interspeech }. YEAR = {2024}}

n personalizing interactions with smart devices, identifying keywords in an open-vocabulary context is crucial. Previous methods for open-vocabulary keyword spotting relied on a shared embedding space created by audio and text encoders. However, they suffered from heterogeneous modality representations, causing audio-text mismatch. To tackle this issue, our proposed framework utilizes knowledge from a pre-trained text-to-speech (TTS) system. This knowledge transfer incorporates awareness of audio projections into text representations derived from the text encoder. Consequently, this approach aids in preventing false triggers in scenarios such as closely related pronunciations of audio-text pairs. Additionally, our proposed approach benefits from the keyword embedding calculation only during keyword enrollment phase. The proposed system gives consistent performance across all the word lengths.

@inproceedings{bib_Epoc_2024, AUTHOR = {Vuppala, Anil Kumar }, TITLE = {Epoch extraction in real-world scenario}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2024}}

Many real-world speech processing applications depend on the accurate estimation of epoch location. In a real-world scenario, speech may contain multiple speakers or multiple emotions in a single utterance. In the literature, the performance of epoch extraction algorithm was evaluated on speech utterance consisting of a single speaker or single emotion alone. In this study, speech utterance from different speakers, and emotions are combined to simulate multi-speaker and multi-emotion scenario, respectively. Five state-of-the-art epoch extraction methods were used to evaluate the performance for multi-speaker, multi-emotion, and voice disorder scenarios. We also analysed the performance of state-of-the-art epoch extraction methods after applying the region-wise approach on existing methods. CMU arctic, and Berlin emotional database is used in this study to simulate the multi-speaker, and multi-emotion scenarios, respectively for analysing the performance of epoch extraction algorithms. Saarbruecken voice disorder database is used to perform the experiments for voice disorder scenario. The result of this study indicates that performance of state-of-the art epoch extraction methods (which depend on pitch period) degrades in terms of false alarm rate (FAR) for these three scenarios. By applying the region-wise approach on these methods for extraction of epoch locations, FAR is reduced by 5% to 8% for the real-world scenario. It was also observed from this study, that the performance of dynamic programming based methods remains unchanged even after applying the region-wise approach on existing methods.

@inproceedings{bib_IIIT_2024, AUTHOR = {Mondal, Anindita and Vuppala, Anil Kumar and Yarra, Chiranjeevi }, TITLE = {IIIT-Speech Twins 1.0: An English-Hindi Parallel Speech Corpora for Speech-to-Speech Machine Translation and Automatic Dubbing}, BOOKTITLE = {Conference of the Oriental COCOSDA}. YEAR = {2024}}

The demand for high-quality parallel speech data has been increasing as deep learning-based Speech to Speech Machine Translation (SSMT) and automatic dubbing approaches gain popularity in speech applications. Traditional and well-established speech applications such as Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) heavily rely on large corpus of monolingual speech and the corresponding text. While there is a wealth of parallel text data available for both English and Indic languages, parallel speech data is available only for English and other European languages, yet it often lacks natural prosody and semantic alignment between the languages. For achieving cross-lingual prosody transfer, end-to-end SSMT models, and high-quality dubbing from English to Hindi, in this work, an English-Hindi parallel bilingual speech-text corpus named lIlT-Speech Twins 1.0 is created. This data contains twin-like English and Hindi speech-text pairs obtained from publicly available children's stories in both the languages, through manual and automatic processing. Starting with 8 stories in each language, totaling around 4 hours of audio, the final outcome was a 2-hour dataset. This was achieved through systematic segmentation, re-moval of non-speech background audio, and sentence-by-sentence alignment to ensure accurate meaning in both languages. In addition to ensuring proper alignment and transcription, this dataset offers a rich source of natural prosody, expressions, and emotions, due to the narrative diversity within the stories. The dataset also provides sig-nificant speaker variability, with different characters being voiced by various speakers, enhancing the richness of the lIlT-Speech Twins 1.0 corpus.

@inproceedings{bib_Enha_2024, AUTHOR = {Simha, Narasinga Vamshi Raghu and Fathima, Hina and Sai, Motepalli Kowshik Siva and Mahesh, Sangeetha and C, Sai Akarsh and Barche, Purva and Ganesh, Mirishkar Sai and Abraham, ajish K and Vuppala, Anil Kumar }, TITLE = {Enhancing Stuttering Detection: A Syllable-Level Stutter Dataset}, BOOKTITLE = {International Conference on Signal Processing and Communications}. YEAR = {2024}}

Stuttering constitutes a pervasive speech disorder with a global impact on individuals. Detecting stuttering in speech would aid speech pathologists in monitoring fluency over time and enhance the quality of life for those with atypical speech patterns. Presently, the available public datasets tailored for stuttering identification are confined to the utterance level, with labels ascribed to individual utterances. However, in real-time situations, a considerable proportion of individuals grappling with stuttering manifest dysfluency at the syllable level. This limitation markedly constrains the effectiveness and applicability of dysfluency detection systems. In an effort to address this lacuna in knowledge, this study introduces a novel dataset comprising audio recordings directly obtained from native Kannada-speaking individuals afflicted with the stuttering disorder at the syllable level and also provides detailed information on the dataset's creation, including patient selection criteria and clip labelling. Additionally, benchmark models trained on this distinctive dataset are presented as part of this research endeavor.

@inproceedings{bib_User_2024, AUTHOR = {V, Kesavaraj and Anuprabha, and Vuppala, Anil Kumar }, TITLE = {User-defined keyword spotting using shifted delta coefficients}, BOOKTITLE = {International conference on Pattern Recognition}. YEAR = {2024}}

Identifying user-defined keywords is crucial for personalizing interactions with smart devices. Previous approaches of user-defined keyword spotting (UDKWS) have relied on short-term spectral features such as mel frequency cepstral coefficients (MFCC) to detect the spoken keyword. However, these features may face challenges in accurately identifying closely related pronunciation of audio-text pairs, due to their limited capability in capturing the temporal dynamics of the speech signal. To address this challenge, we propose to use shifted delta coefficients (SDC) which help in capturing pronunciation variability (transition between connecting phonemes) by incorporating long-term temporal information. The performance of the SDC feature is compared with various baseline features across four different datasets using a cross-attention based end-to-end system. Additionally, various configurations of SDC are explored to find the suitable temporal context for the UDKWS task. The experimental results reveal that the SDC feature outperforms the MFCC baseline feature, exhibiting an improvement of 8.32% in area under the curve (AUC) and 8.69% in terms of equal error rate (EER) on the challenging Libriphrase-hard dataset. Moreover, the proposed approach demonstrated superior performance when compared to state-of-the-art UDKWS techniques.

@inproceedings{bib_Open_2024, AUTHOR = {V, Kesavaraj and Vuppala, Anil Kumar }, TITLE = {Open Vocabulary Keyword Spotting through Transfer Learning from Speech Synthesis}, BOOKTITLE = {International Conference on Signal Processing and Communications}. YEAR = {2024}}

Identifying keywords in an open-vocabulary context is crucial for personalizing interactions with smart devices. Previous approaches to open vocabulary keyword spotting dependon a shared embedding space created by audio and text encoders. However, these approaches suffer from heterogeneous modality representations (i.e., audio-text mismatch). To address this issue, our proposed framework leverages knowledge acquired from a pre-trained text-to-speech (TTS) system. This knowledge transfer allows for the incorporation of awareness of audio projections into the text representations derived from the text encoder. The performance of the proposed approach is compared with various baseline methods across four different datasets. The robustness of our proposed model is evaluated by assessing its performance across different word lengths and in an Out-of-Vocabulary (OOV) scenario. Additionally, the effectiveness of transfer learning from the TTS system is investigated by analyzing its different intermediate representations. The experimental results indicate that, in the challenging LibriPhrase Hard dataset, the proposed approach outperformed the cross-modality correspondence detector (CMCD) method by a significant improvement of 8.22% in area under the curve (AUC) and 12.56% in equal error rate (EER).

@inproceedings{bib_Stre_2024, AUTHOR = {C, Sai Akarsh and Simha, Narasinga Vamshi Raghu and Vuppala, Anil Kumar }, TITLE = {Stress Transfer in Speech-to-Speech Machine Translation Application Demo}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2024}}

India's education sector faces a significant challenge due to its linguistic diversity, hindering inclusivity. The dominance of English on the internet underscores the critical need for translating educational content into Indian languages to enhance accessibility. Although Speech-to-Speech Machine Translation (SSMT) technologies exist, their deficiency in reproducing intonation results in monotonous translations, diminishing audience engagement and interest in the content. To address this issue, this paper demonstrates an SSMT application with a Text-to-Speech (TTS) architecture capable of incorporating stress into synthesized speech to give a more engaging experience. The SSMT pipeline also has components like a stress classifier that captures the stress in the source speech and allows it to be utilized during speech generation. The application takes in a speech file and gives a translated speech file with stress transferred from the source.

@inproceedings{bib_Atte_2024, AUTHOR = {C, Sai Akarsh and Simha, Narasinga Vamshi Raghu and Mondal, Anindita and Vuppala, Anil Kumar }, TITLE = {Attempt Towards Stress Transfer in Speech-to-Speech Machine Translation}, BOOKTITLE = {International Conference on Signal Processing and Communications}. YEAR = {2024}}

The language diversity in India's education sector poses a significant challenge, hindering inclusivity. Despite the democratization of knowledge through online educational content, the dominance of English, as the internet's lingua franca, limits accessibility, emphasizing the crucial need for translation into Indian languages. Despite existing Speech-to-Speech Machine Translation (SSMT) technologies, the lack of intonation in these systems gives monotonous translations, leading to a loss of audience interest and disengagement from the content. To address this, our paper introduces a dataset with stress annotations in Indian English and also a Text-to-Speech (TTS) architecture capable of incorporating stress into synthesized speech. This dataset is used for training a stress detection model, which is then used in the SSMT system for detecting stress in the source speech and transferring it into the target language speech. The TTS architecture is based on FastPitch and can modify the variances based on stressed words given. We present an Indian English-to-Hindi SSMT system that can transfer stress and aim to enhance the overall quality and engagement of educational content.

@inproceedings{bib_An_I_2023, AUTHOR = {Jain, Shelly and Pal, Priyanshi and Vuppala, Anil Kumar and Ghosh, Prasanta Kumar and Yarra, Chiranjeevi }, TITLE = {An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2023}}

Speech systems are sensitive to accent variations. This is a challenge in India, which has numerous languages but few linguistic studies on pronunciation variation. The growing number of L2 English speakers reinforces the need to study accents and L1-L2 interactions. We investigate Indian English (IE) accents and report our observations on regional and shared features. Specifically, we observe phonemic variations and phonotactics in speakers’ native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing native language features with IE pronunciations obtained from literature studies and phonetically annotated speech. Hence, we validate Indian language influences on IE by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description of generalised and region-specific IE characteristics, which facilitates accent adaptation of existing speech systems.

@inproceedings{bib_Auto_2023, AUTHOR = {PONNAM, MONICA and Barche, Purva and Vuppala, Anil Kumar }, TITLE = {Automatic detection of Parkinsons disease Using Zero-Time Window based cepstral feature}, BOOKTITLE = {India Council International Conference}. YEAR = {2023}}

Parkinson’s disease (PD) is a neurological disorder that affects the nerves responsible for controlling movements. Automatic detection methods used for PD detection from the speech signal are important as they offer benefits such as reliability, repeatability, and cost-effectiveness. In this paper, cepstral features derived from zero-time windowing (ZTW) based time-frequency analysis method for automatic detection of PD were explored. This study used the PC-GITA database to evaluate the performance using support vector machine classifier and neural networks. The performance of the system is also compared with conventional mel-frequency cepstral coeffcients (MFCCs) features. Cepstral features derived from ZTW method shown better performance than baseline feature. The best performance of the PD detection system obtained using the cepstral features derived from ZTW method in terms of classification accuracy is 73.6% . Index Terms—Parkinson’s Disease, Zero-time windowing, Automatic detection methods using speech signal.

@inproceedings{bib_Enha_2023, AUTHOR = {Gupta, Shivang and Sai, Motepalli Kowshik Siva and Simha, Narasinga Vamshi Raghu and Kumar, Ravi and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Enhancing Language Identification in Indian Context Through Exploiting Learned Features with Wav2Vec2. 0}, BOOKTITLE = {International Conference on Speech and Computers}. YEAR = {2023}}

This work proposes the utilization of a self-supervised pre-trained network for developing a Language Identification (LID) system catering to low-resource Indian languages. The framework employed is Wav2vec2.0-XLSR-53, pre-trained on 53k hours of unlabeled speech data. The unsupervised training of the model enables it to learn the acoustic patterns specific to a language. Given that languages share phonetic space, multi-lingual pre-training is instrumental in learning cross-lingual information and building systems that cater to low-resource languages. Further fine-tuning with a limited amount of labeled data significantly boosts the model’s accuracy. The results showcase a relative improvement of 33.2% over the DNN-A (DNN with attention) model and 19.04% over Dense Resnets for the Language Identification task on the IIITH-ILSC database using the proposed features (Shivang Gupta and Kowshik Siva Sai Motepalli share first authorship).

@inproceedings{bib_Stud_2023, AUTHOR = {Pal, Priyanshi and Jain, Shelly and Yarra, Chiranjeevi and Ghosh, Prasanta Kumar and Vuppala, Anil Kumar }, TITLE = {Study of Indian English Pronunciation Variabilities Relative to Received Pronunciation}, BOOKTITLE = {International Conference on Speech and Computers}. YEAR = {2023}}

Analysis of Indian English (IE) pronunciation variabilities is useful in ASR and TTS modelling for the Indian context. Prior works characterised IE variabilities by reporting qualitative phonetic rules relative to Received Pronunciation (RP). However, such characterisations lack quantitative descriptors and data-driven analysis of diverse IE pronunciations, which could be due to the scarcity of phonetically labelled data. Furthermore, the versatility of IE stems from the influence of a large diversity of the speakers’ mother tongues and demographic region differences. To address these issues, we consider the corpus Indic TIMIT and manually obtain 13, 632 phonetic transcriptions in addition to those parts of the corpus. By performing a data-driven analysis on 15, 974 phonetic transcriptions of 80 speakers from diverse regions of India, we present a new set of phonetic rules and validate them against the existing phonetic rules to identify their relevance. Finally, we test the efficacy of Grapheme-to-Phoneme (G2P) conversion developed based on the obtained rules considering Phoneme Error Rate (PER) as the metric for performance.

@inproceedings{bib_Enha_2023, AUTHOR = {Simha, Narasinga Vamshi Raghu and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Enhancing Stutter Detection in Speech using Zero Time Windowing Cepstral Coefficients and Phase Information}, BOOKTITLE = {International Conference on Speech and Computers}. YEAR = {2023}}

Stuttering is a speech disorder that affects speech fluency and rhythm, with millions worldwide experiencing it. Early diagnosis and treatment can significantly improve speech fluency and the quality of life for individuals who stutter. Automatic detection of stuttering events can help diagnose, monitor, and develop effective interventions. Therefore, this paper aims to propose a feature space-based classifier for detecting stuttering events in speech. To achieve this, we have investigated Zero Time Windowing Cepstral Coefficients (ZTWCC) as a feature set for stutter detection using classifiers such as SVM, LSTM, and Bidirectional LSTM. We compared the performance of ZTWCC with the standard handcrafted features, such as MFCC, CQCC, and SFFCC, on the SEP- 28K dataset with and without including phase information. The results in both cases indicate that ZTWCC is giving a higher F-1 score than baseline MFCC features.

@inproceedings{bib_Stab_2023, AUTHOR = {Jabbar, Syed Abdul and Barche, Purva and KRISHNA, GURUGUBELLI and Syed, Azeemuddin and Vuppala, Anil Kumar }, TITLE = {Stable Implementation Of Voice Activity Detector using Zero-Phase Zero Frequency Resonator On FPGA}, BOOKTITLE = {IEEE International Conference and Expo on Real Time Communications}. YEAR = {2023}}

Voice activity detection (VAD), is a signal processing technique used to determine whether a given speech signal contains voiced or unvoiced segments. VAD is used in various applications such as Speech Coding, Voice Controlled Systems, speech feature extraction, etc. For example, in Adaptive multirate (AMR) speech coding, VAD is used as an efficient way of coding different speech frames at different bit rates. In this paper, we implemented the application of a Zero-Phase Zero Frequency Resonator (ZP-ZFR) as VAD on hardware. ZP-ZFR is an Infinite Impulse Response (IIR) filter that offers the advantage of requiring a lower filter order, making it suitable for hardware implementation. The proposed system is implemented on the TIMIT database using the Nexys Video Artix-7 FPGA board. The hardware design is carried out using Vivado 2021.1, a popular tool for FPGA development. The Hardware Description Language (HDL) used for implementation is Verilog. The proposed system achieves good performance with low complexity. Therefore this work is implemented on hardware, which can be used in various applications. Index Terms—Voice Activity Detection (VAD), Zero-Phase Zero Frequency Resonator (ZP-ZFR), Zero Frequency Filter (ZFF), MATLAB, Field Programmable Gate Array (FPGA).

@inproceedings{bib_IIIT_2023, AUTHOR = {Ganesh, Mirishkar Sai and V, VISHNU VIDYADHARA RAJU and Naroju, Meher Dinesh and Maity, Sudhamay and Yalla, Veera Prakash and Vuppala, Anil Kumar }, TITLE = {IIITH-CSTD Corpus: Crowdsourced Strategies for the Collection of a Large-scale Telugu Speech Corpus}, BOOKTITLE = {ACM Trasactions on Asian and Low Resource Language Information Processing}. YEAR = {2023}}

Due to the lack of a large annotated speech corpus, many low-resource Indian languages struggle to utilize recent advancements in deep neural network architectures for Automatic Speech Recognition (ASR) tasks. Collecting large-scale databases is an expensive and time-consuming task. Current approaches lack extensive traditional expert-based data acquisition guidelines, as they are tedious and complex. In this work, we present the International Institute of Information Technology Hyderabad-Crowd Sourced Telugu Database (IIITH-CSTD), a Telugu corpus collected through crowdsourcing

@inproceedings{bib_Enha_2023, AUTHOR = {Sai, Motepalli Kowshik Siva and Gupta, Shivang and Narasinga, Vamsi and Kumar, Ravi and Vuppala, Anil Kumar }, TITLE = {Enhancing Language Identification in Indian Context through Exploiting Learned Features with Wav2Vec2.0}, BOOKTITLE = {International Conference on Speech and Computers}. YEAR = {2023}}

This work proposes the utilization of a self-supervised pretrained network for developing a Language Identification (LID) system catering to low-resource Indian languages. The framework employed is Wav2vec2.0-XLSR-53, pre-trained on 53k hours of unlabeled speech data. The unsupervised training of the model enables it to learn the acoustic patterns specific to a language. Given that languages share phonetic space, multi-lingual pre-training is instrumental in learning crosslingual information and building systems that cater to low-resource languages. The results showcase a relative improvement of 33.2% over the DNN-A (DNN with attention) model and 19.04% over Dense Resnets for the Language Identification task on the IIITH-ILSC database using the proposed features.

@inproceedings{bib_Stut_2023, AUTHOR = {Sai, Motepalli Kowshik Siva and Simha, Narasinga Vamshi Raghu and Harsha, Pathuri Venkata Sri and Khan, Hina Fathima Fazal and Mahesh, Sangeetha and Abraham, Ajish and Vuppala, Anil Kumar }, TITLE = {Stuttering Detection Application}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2023}}

Stuttering is a prevalent speech disorder that affects millions of people worldwide. In this Show and Tell presentation, we demonstrate a novel platform that takes speech samples in English and Kannada to detect and analyze stuttering in patients. The user-friendly interface includes demographic details and speech samples, generating comprehensive reports for different stuttering disfluencies. The platform has four different user types, providing full read-only access for admins and full write access for super admins. Our platform provides valuable assistance for speechlanguage pathologists to evaluate speech samples. The proposed platform supports both live and recorded speech samples and presents a flexible approach to stuttering detection and analysis. Our research demonstrates the potential of technology to improve speech-language pathology for stuttering. Used F-score as a metric for evaluating the models for the stutter detection task.

@inproceedings{bib_Hard_2023, AUTHOR = {Yamini, Durgavajjula Shaarada and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar and Purini, Venkata Suresh Reddy }, TITLE = {Hardware Accelerator for Transformer based End-to-End Automatic Speech Recognition System}, BOOKTITLE = {Reconfigurable Architectures Workshop }. YEAR = {2023}}

Hardware accelerators are being designed to offload compute-intensive tasks such as deep neural networks from the CPU to improve the overall performance of an application, specifically on the performance-per-watt metric. Encoder-decoderbased sequence-to-sequence models such as the Transformer model have demonstrated state-of-the-art results in end-to-end automatic speech recognition systems (ASRs). The Transformer model being intensive on memory and computation poses a challenge for an FPGA implementation. This paper proposes an end-to-end architecture to accelerate a Transformer for an ASR system. The host CPU orchestrates the computations from different encoder and decoder stages of the Transformer architecture on the designed hardware accelerator with no necessity for intervening FPGA reconfiguration. The communication latency is hidden by prefetching the weights of the next encoder/decoder block while the current block is being processed. The computation is split across both the Super Logic Regions (SLRs) of the FPGA, mitigating the inter-SLR communication. The proposed design presents an optimal latency, exploiting the available resources. The accelerator design is realized using highlevel synthesis tools and evaluated on an Alveo U-50 FPGA card. The design demonstrates an average speed-up of 32× compared to an Intel Xeon E5-2640 CPU and 8.8× compared to NVIDIA GeForce RTX 3080 Ti Graphics card for a 32-bit floating point single precision model. Index Terms—Automatic speech recognition, Hardware accelerator, Transformer, FPGAs.

@inproceedings{bib_How__2023, AUTHOR = {Jain, Shelly and Yadavalli, Aditya and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {How do Phonological Properties Affect Bilingual Automatic Speech Recognition?}, BOOKTITLE = {IEEE Spoken Language Technology Workshop}. YEAR = {2023}}

Multilingual Automatic Speech Recognition (ASR) for Indian lan- guages is an obvious technique for leveraging their similarities. We present a detailed analysis of how phonological similarities and differences between languages affect Time Delay Neural Net- work (TDNN) and End-to-End (E2E) ASR. To study this, we select genealogically similar pairs from five Indian languages and train bilingual acoustic models. We compare these against corresponding monolingual acoustic models and find similar phoneme distribu- tions within speech to be the primary factor for improving model performance, with phoneme overlap being secondary. The influ- ence of phonological properties on performance is visible in both cases. Word Error Rate (WER) of E2E decreased by a median of 2.35%, and upto 8.5% when the phonological similarity was great- est. WER of TDNN increased by 11.69% when the similarity was lowest. Thus, it is clear that the choice of supplementary language is important for model performance. Index Terms— Phonetics and Phonology, Analysis, Automatic Speech Recognition, Low-Resource Languages, Multilingual Sys-

@inproceedings{bib_Mult_2022, AUTHOR = {Yadavalli, Aditya and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Multi-Task End-to-End Model for Telugu Dialect and Speech Recognition}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2022}}

Conventional Automatic Speech Recognition (ASR) systems are susceptible to dialect variations within a language, thereby adversely affecting the ASR. Therefore, the current practice is to use dialect-specific ASRs. However, dialectspecific information or data is hard to obtain making it difficult to build dialect-specific ASRs. Furthermore, it is cumbersome to maintain multiple dialect-specific ASR systems for each language. We build a unified multi-dialect End-to-End ASR that removes the need for a dialect recognition block and the need to maintain multiple dialect-specific ASRs for three Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema. We find that pooling the data and training a multi-dialect ASR benefits the low-resource dialect the most – an improvement of over 9.71% in relative Word Error Rate (WER). Subsequently, we experiment with multi-task ASRs where the primary task is to transcribe the audio and the secondary task is to predict the dialect. We do this by adding a Dialect ID to the output targets. Such a model outperforms naive multi-dialect ASRs by up to 8.24% in relative WER. Additionally, we test this model on a dialect recognition task and find that it outperforms strong baselines by 6.14% in accuracy. Index Terms: multi-dialect, speech recognition, sequence-tosequence, dialect recognition

@inproceedings{bib_Long_2022, AUTHOR = {LAHOTI, ANSHUL and KRISHNA, GURUGUBELLI and Arroyave, Juan Rafel Orozco and Vuppala, Anil Kumar }, TITLE = {Long-Term Average Spectral Feature-Based Parkinson’s Disease Detection from Speech}, BOOKTITLE = {Advanced Machine Intelligence and Signal Processing}. YEAR = {2022}}

Parkinson’s disease (PD) is a speech disorder caused by the effect of neurological damage on motor control activity of speech production. Due to the noninvasive nature, detection of PD from the speech is remarkable. This study proposes the use of Long-Term Average Spectral (LTAS) feature to detect PD from speech using different filterbank techniques and classifiers on the PC-GITA dataset. The fact that PD can be perceived from long-temporal information has motivated us to explore the LTAS features for PD detection. Performance of LTAS features is compared to baseline features, namely statistical and openSMILE features (ComParE, GeMAPS, and eGeMAPS). The performance of these features is validated using five different classifiers. The LTAS features, which are obtained with auditory filterbanks (Constant Q Transform and Gammatone), showed better performance as compared to the statistical features (MFCC_Stat and LPCC_Stat) and openSMILE features (ComParE, GeMAPS, and eGeMAPS). The combination of the Gammatone-based LTAS feature with eGeMAPS further improved the detection accuracy of the PD detection system on the PC-GITA database.

@inproceedings{bib_Expl_2022, AUTHOR = {Yadavalli, Aditya and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Exploring the Effect of Dialect Mismatched Language Models in Telugu Automatic Speech Recognition}, BOOKTITLE = {NAACL Student Research Workshop}. YEAR = {2022}}

Previous research has found that Acoustic Models (AM) of an Automatic Speech Recognition (ASR) system are susceptible to dialect variations within a language, thereby adversely affecting the ASR. To counter this, researchers have proposed to build a dialectspecific AM while keeping the Language Model (LM) constant for all the dialects. This study explores the effect of dialect mismatched LM by considering three different Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema. We show that dialect variations that surface in the form of a different lexicon, grammar, and occasionally semantics can significantly degrade the performance of the LM under mismatched conditions. Therefore, this degradation has an adverse effect on the ASR even when dialect-specific AM is used. We show a degradation of up to 13.13 perplexity points when LM is used under mismatched conditions. Furthermore, we show a degradation of over 9% and over 15% in Character Error Rate (CER) and Word Error Rate (WER), respectively, in the ASR systems when using mismatched LMs over matched LMs

@inproceedings{bib_Expl_2022, AUTHOR = {Vats, Nayan Anand and Barche, Purva and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Exploring High Spectro-Temporal Resolution for Alzheimer’s Dementia Detection}, BOOKTITLE = {International Conference on Signal Processing and Communications}. YEAR = {2022}}

Alzheimer’s Dementia is a progressive neurological disorder characterized by cognitive impairment. It affects mem- ory, thinking skills, language, and the ability to perform simple tasks. Detection of Alzheimer’s Dementia from the speech is considered a primitive task, as most speech cues are preserved in it. Studies in the literature focused mainly on the lexical features and few acoustic features for detecting Alzheimer’s disease. The present work explores the single frequency filtering cepstral coefficients (SFCC) for the automatic detection of Alzheimer’s disease. In contrast to STFTs, the proposed feature has better temporal and spectral resolution and captures the transient part more appropriately. This offers a very compact and efficient way to derive the formant structure in the speech signal. The experiments were conducted on the ADReSSo dataset, using the support vector machine classifier. The classification performance was compared with several baseline features like Mel-frequency cepstral coefficients (MFCC), perceptual linear prediction (PLP), linear prediction cepstral coefficient (LPCC), Mel frequency cepstral coefficients of LP-residual (MFCC-WR), ZFF signal (MFCC-ZF) and eGeMAPS (openSMILE). The experiments conducted on Alzheimer’s Dementia classification task show that the proposed feature performs better than conventional MFCCs. Among all the features, SFCC offers the best classification accuracy of 65.1% and 60.6% for dementia detection on cross- validation and test data, respectively. The combination of baseline features with SFCC features further improved the performance. Index Terms—Alzheimer’s disease, Cognitive impairment, Sin- gle frequency filtering cepstral coefficients.

@inproceedings{bib_Shif_2022, AUTHOR = {LAHOTI, ANSHUL and KRISHNA, GURUGUBELLI and Arroyave, Juan Rafel Orozco and Vuppala, Anil Kumar }, TITLE = {Shifted Delta Cepstral Coefficients with RNN to Improve the Detection of Parkinson’s Disease from the Speech}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2022}}

Parkinson’s disease (PD) is a progressive neurodegenerative dis- order of the central nervous system identified by motor and non- motor activities abnormalities. PD affects respiration, laryngeal, articulation, resonance, and prosodic aspects of speech production. Detection of PD from speech is a non-invasive approach useful for automatic screening. Perceptual attributes of speech due to PD are manifested as temporal variations in speech. In this regard, current work investigated the use of LSTM and BiLSTM networks with shifted delta cepstral (SDC) features to detect PD from speech. Further in BiLSTM networks, a multi-head attention mechanism is introduced, assuming that each head captures distinct information to detect PD. SDC features obtained from MFCCs, and SFFCCs are used for developing the PD detection system. The performance of the experiments is validated using the PC-GITA database. The experimental results revealed that BiLSTM networks give a rela- tive improvement of 4-5% over the LSTM networks. The use of a multi-head attention mechanism further improved the detection accuracy of the PD detection system, showing that it can capture various discriminative features.

@inproceedings{bib_Inve_2022, AUTHOR = {Yadavalli, Aditya and Jain, Shelly and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Investigation of Subword-Based Bilingual Automatic Speech Recognition for Indian Languages}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2022}}

Modelling Indian languages is difficult as the number of word forms they present is high. In such cases, prior work has proposed us- ing subwords instead of words as the basic units to model the language. This provides the potential to cover more word forms. Additionally, previous work revealed that all Indian languages have several phonetic similarities. Consequently, multilingual acoustic models have been proposed to counter the data scarcity issue most Indian language datasets have. However, these models use monolin- gual language models (LM). They do not exploit the common basic tokens that certain Indian languages have. This motivated us to im- plement a bilingual subword-based Automatic Speech Recognition (ASR) system for Hindi and Marathi. Further, we try a combination of word-based monolingual LM with bilingual acoustic model to examine the reason for degradation in word-based multilingual ASRs. Our experiments show that multilingual subword-based ASR models outperform their word-based counterparts by upto 9.77% and 26.35% relative Character Error Rate (CER) in the case of Hindi and Marathi respectively.

@inproceedings{bib_Impl_2022, AUTHOR = {Jabbar, Syed Abdul and Barche, Purva and KRISHNA, GURUGUBELLI and Syed, Azeemuddin and Vuppala, Anil Kumar and Vuppala, Anil Kumar }, TITLE = {Implementation of Zero-Phase Zero Frequency Resonator Algorithm on FPGA}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2022}}

Epoch location is an important parameter for analysis of excitation source information from the speech signals. From several years lot of research have been done to find accurate locations of epochs. Epochs are instants at which vocal tract system are excited signifi- cantly. The prominent location of epochs can be found during the production of speech signals. However, due to the time varying nature of excitation source and the vocal tract system it is difficult to find accurate location of epochs. From various epoch extraction methods, Zero Frequency Filtering (ZFF) is the simplest and most widely used method to find accurate locations of epochs due to its highest identification rate and lowest false alarm rate among other algorithms. ZFF uses Infinite Impulse Response (IIR) filter followed by trend removal blocks however, the filter used in ZFF method is unstable which makes it unsuitable for practical implementation. In the literature many stable implementations of ZFF algorithms have been proposed. Compared to other stable algorithms of ZFF, Zero-phase Zero Frequency Resonator has simple design and gives the highest identification rate among other epoch extraction algo- rithms including ZFF. In this paper, the implementation of ZFF and ZP-ZFR has been proposed on FPGA board using Verilog which is Hardware Description Language (HDL).

@inproceedings{bib_Towa_2022, AUTHOR = {Mehrotra, Utkarsh and SPARSH, and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Towards improving Disfluency Detection from Speech using Shifted Delta Cepstral Coefficients}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2022}}

In this paper, we investigate the use of shifted delta cepstral (SDC) coefficients for detecting disfluencies in two types of speech - stuttered speech and spontaneous lecture-mode speech. SDC features capture temporal variations in the speech signal effectively across several frames. The UCLASS stuttered speech dataset and IIITH-IED dataset are used here to develop frame-level automatic disfluency detection systems for four types of disfluencies and the effect of SDC features on the detection of each disfluency type is examined using MFCC and SFFCC cepstral representations. Overall, it is found that using MFCC+SDC features gives an absolute improvement of 2.98% and 6.02% for stuttered and spontaneous speech disfluency detection respectively over the static MFCC features, while SFFCC+SDC features give an absolute improvement of 4.62% for stutter disfluencies and 7.28% for spontaneous speech disfluencies over the static SFFCC features, showing the importance of considering temporal variations for disfluency detection.

@inproceedings{bib_An_I_2022, AUTHOR = {Jain, Shelly and Pa, Priyanshi and Vuppala, Anil Kumar and Ghosh, Prasanta and Yarra, Chiranjeevi }, TITLE = {An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations}, BOOKTITLE = {Technical Report}. YEAR = {2022}}

Speech systems are sensitive to accent variations. This is especially challenging in the Indian context, with an abundance of languages but a dearth of linguistic studies characterising pronunciation variations. The growing number of L2 English speakers in India reinforces the need to study accents and L1-L2 interactions. We investigate the accents of Indian English (IE) speakers and report in detail our observations, both specific and common to all regions. In particular, we observe the phonemic variations and phonotactics occurring in the speakers' native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing the native language pronunciations with IE pronunciations obtained jointly from existing literature studies and phonetically annotated speech of 80 speakers. Consequently, we are able to validate the intuitions of Indian language influences on IE pronunciations by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description in terms of universal and region-specific characteristics of IE, which facilitates accent conversion and adaptation of existing ASR and TTS systems to different Indian accents.

@inproceedings{bib_Deco_2022, AUTHOR = {Jain, Arhant and Gurgubelli, Krishna and Vemuri, kavitha and Vuppala, Anil Kumar }, TITLE = {Decoding self-automated and motivated finger movements using novel single-frequency filtering method–An EEG study}, BOOKTITLE = {Biomedical Signal Processing and Control}. YEAR = {2022}}

Electroencephalography (EEG) provides the temporal resolution required to map the neural activations for studying motor movements and control. This study aims to compare the power amplitude of electrodes covering the central and frontal regions of a 32-channel scalp EEG. The activations from a standard index finger-tapping and a game paradigm are analyzed. Twenty-five right-handed and five left-handed healthy subjects (range = 18–30 years; mean = 24.25 years; SD = 3.96 years) participated in this study. A novel single-frequency filter (SFF) bank was applied to identify the peak amplitude from the power spectral density plots. The results show that the gaming paradigm yields lower or comparable power amplitudes than the standard finger tapping. We observed that the right-hand finger tapping by the left-handed subjects shows lower between-subject dispersion in amplitude. Nonparametric Spearman correlation showed no association between game scores and power amplitude for the right-handed participants. However, for left-handers, both positive and negative associations were observed. This study demonstrates the efficacy of SFF for extracting power amplitudes with a better signalto-noise ratio, which has implications in BCI and motor rehabilitation applications. The findings support the role of game paradigms for motor movement research and in understanding bilateral hemispheric activations in cognitive tasks.

@inproceedings{bib_Nove_2022, AUTHOR = {Thirumurua, Ramakrishna and Gurbelli, Krishna and Vuppala, Anil Kumar }, TITLE = {Novel feature representation using single frequency ﬁltering and nonlinear energy operator for speech emotion recognition }, BOOKTITLE = {Digital Signal Processing}. YEAR = {2022}}

In this paper, the intrinsic characteristics of speech modulations are estimated to propose the instant modulation spectral features for eﬃcient emotion recognition. This feature representation is based on single frequency ﬁltering (SFF) technique and higher order nonlinear energy operator. The speech signal is decomposed into frequency sub-bands using SFF, and associated nonlinear energies are estimated with higher order nonlinear energy operator. Then, the feature vector is realized using cepstral analysis. The high-resolution property of SFF technique is exploited to extract the amplitude envelope of the speech signal at a selected frequency with good time-frequency resolution. The fourth order nonlinear energy operator provides noise robustness in estimating the modulation components. The proposed feature set is tested for the emotion recognition task using the i-vector model with the probabilistic linear discriminant scoring scheme, support vector machine and random forest classiﬁers. The results demonstrate that the performance of this feature representation is better than the widely used spectral and prosody features, achieving detection accuracy of 85.75%, 59.88%, and 65.78% on three emotional databases, EMODB, FAUAIBO, and IEMOCAP, respectively. Further, the proposed features are found to be robust in the presence of additive white Gaussian and vehicular noises.

@inproceedings{bib_Outc_2021, AUTHOR = {Vuppala, Anil Kumar and Yalla, Veera Prakash and Ganesh, Mirishkar Sai and V, VISHNU VIDYADHARA RAJU }, TITLE = {Outcomes of Speech to Speech Translation for Broadcast Speeches and Crowd Source Based Speech Data Collection Pilot Projects}, BOOKTITLE = {International Conference on Big Data Analytics}. YEAR = {2021}}

Speech-to-Speech Machine Translation (SSMT) applications and services use a three-step process. Speech recognition is the first step to obtain transcriptions. This is followed by text-to-text language translation and, finally, synthesis into text-speech. As data availability and computing power improved, these individual steps evolved. However, despite significant progress, there is always the error of the first stage in terms of speech recognition, accent, etc. Having traversed the speech recognition stage, the error becomes more prevalent and decreases very often. This chapter presents a complete pipeline for transferring speaker intent in SSMT involving humans in the loop. Initially, the SSMT pipeline has been discussed and analyzed for broadcast speeches and talks on a few sessions of Mann Ki Baat, where the source language is in Hindi, and the target language is in English and Telugu. To perform this task, industry-grade APIs from Google, Microsoft, CDAC, and IITM has been used for benchmarking. Later challenges faced while building the pipeline are discussed, and potential solutions have been introduced. Later this chapter introduces a framework developed to

@inproceedings{bib_CSTD_2021, AUTHOR = {Mirishkar, Ganesh S and V, VISHNU VIDYADHARA RAJU and Naroju, Meher Dinesh and Maity, Sudhamay and Yalla, Veera Prakash and Vuppala, Anil Kumar }, TITLE = {CSTD-Telugu Corpus: Crowd-Sourced Approach for Large-Scale Speech data collection}, BOOKTITLE = {Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)}. YEAR = {2021}}

Speech is a natural mode of communication among all beings. India is a densely populated country, and people are diverse throughout the globe. The spoken language is the medium of instruction to interact among the people. The majority of Indian languages are spoken globally. The unavailability of larger volumes of transcribed and annotated speech data is often a hurdle for building reliable speech recognition (ASR) systems for Indian languages. Crowdsourcing strategies are effective in collaboratively collecting speech data resources. This paper describes the experience of large-scale speech data collection for the Telugu language through mobile and web-based applications. With this crowd contributed speech, the performance of the baseline ASR system is shown for clean speech. ASR performance for pink and white noises is also compared for various deep neural network (DNN) based acoustic models. The details regarding the usage of frameworks and their challenges during their implementation are part of this paper. The framework adopted for collecting the speech data is rapid, cost-saving, and offers the advantage of extending it to all the other Indian languages. Index Terms—ASR, Crowd-sourced, TDNN, GMM, SGMM

@inproceedings{bib_Comp_2021, AUTHOR = {Barche, Purva and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Comparative Study of Filter Banks to Improve the Performance of Voice Disorder Assessment Systems using LTAS Features}, BOOKTITLE = {Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)}. YEAR = {2021}}

Objective assessment of voice disorders is widely explored as an early diagnosis tool for the classification of voice disorders. Voice disorders affect the pitch, loudness and voice quality, which are perceived at the suprasegmental-level in the speech signal. For the detection and assessment of voice disorders, this study explores the effectiveness of Long Term Average Spectral (LTAS) features using four state-of-the-art filter banks designed with critical-band, constant-Q, gammatone, and singlefrequency filtering approaches. Moreover, the performance of the systems is compared with state-of-the-art statistical-average and openSMILE features. Voice disorder detection experiment was carried out on SVD and HUPA database, while only SVD database is used for assessment task. Assessment task is performed in clinical way, in which four binary classifiers were trained in our study. Voice disorder detection and assessment tasks were carried out using the support vector machine classifier. From the results, it was observed that constant-Q filter bank based LTAS features performed better among all LTAS features with classification accuracy of 78% and 81.4% for voice disorder detection task on SVD and HUPA database, respectively. Further, the combination of LTAS features with OpenSMILE features improved (89.6% and 86.6% for SVD and HUPA database, respectively) the performance.

@inproceedings{bib_Dete_2021, AUTHOR = {Mehrotra, Utkarsh and SPARSH, and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Detecting Multiple Disfluencies from Speech using Pre-linguistic Automatic Syllabification withAcoustic and Prosody Features}, BOOKTITLE = {Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)}. YEAR = {2021}}

In this paper, a new method to detect disfluencies directly from speech is explored. The method makes use of pre-linguistic automatic syllabification - the process of segmenting input speech signals into perceptually distinct syllable-like regions, to develop syllable-level disfluency detection systems. Statistical prosody features related to fundamental frequency, energy and duration are extracted from each syllable-like region and used to train a DNN classifier for automatic detection of speech disfluencies. Further, complementary information useful for the task of disfluency detection is added to the pipeline with the help of acoustic features. A BiLSTM feature extractor is used to get complex acoustic representation from the baseline MFCC features for each syllable-like region. This acoustic representation is concatenated with the prosody features and used in the proposed system for detecting multiple speech disfluencies. Experiments are conducted for four types of disfluencies in the UCLASS and the IIITH-IED datasets to test the proposed disfluency detection system. Overall, it is found that the proposed system gives a detection accuracy of 88.75% for the disfluencies in the UCLASS dataset, whereas for the IIITH-IED dataset, the accuracy obtained is 91.24%, showing the effectiveness of considering perceptually distinct syllable-like regions as representational units for detecting disfluencies.

@inproceedings{bib_IE-C_2021, AUTHOR = {Jain, Shelly and Yadavalli, Aditya and Ganesh, Mirishkar Sai and Yarra, Chiranjeevi and Vuppala, Anil Kumar }, TITLE = {IE-CPS Lexicon: An Automatic Speech Recognition Oriented Indian English Pronunciation Dictionary}, BOOKTITLE = {International Conference on Natural Language Processing.}. YEAR = {2021}}

Indian English (IE), on the surface, seems quite similar to standard English. However, closer observation shows that it has actually been influenced by the surrounding vernacu- lar languages at several levels from phonology to vocabulary and syntax. Due to this, automatic speech recognition (ASR) systems developed for American or British varieties of English result in poor performance on Indian English data. The most prominent feature of Indian English is the characteristic pronunciation of the speakers. The systems are unable to learn these acoustic variations while modelling and cannot parse the non-standard articulation of non-native speakers. For this purpose, we propose a new phone dictionary de- veloped based on the Indian language Com- mon Phone Set (CPS). The dictionary maps the phone set of American English to existing Indian phones based on perceptual similarity. This dictionary is named Indian English Com- mon Phone Set (IE-CPS). Using this, we build an Indian English ASR system and compare its performance with an American English ASR system on speech data of both varieties of En- glish. Our experiments on the IE-CPS show that it is quite effective at modelling the pro- nunciation of the average speaker of Indian En- glish. ASR systems trained on Indian English data perform much better when modelled us- ing IE-CPS, achieving a reduction in the word error rate (WER) of upto 3.95% when used in place of CMUdict. This shows the need for a different lexicon for Indian English.

@inproceedings{bib_ACOU_2021, AUTHOR = {Vats, Nayan Anand and Yadavalli, Aditya and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {ACOUSTIC FEATURES, BERT Model AND THEIR COMPLEMENTARY NATURE FOR ALZHEIMER’S DEMENTIA DETECTION}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2021}}

Dementia is a syndrome chronic or progressive that usually affects the cognitive functioning of the subjects. Alzheimer’s, a neurodegenerative disorder, is the leading cause of dementia. One of the many symptoms of Alzheimer’s Dementia is the inability to speak and understand language clearly. The last decade has seen a surge in the research done in Alzheimer’s Dementia detection using Linguistics and acoustic features. This paper takes up the Alzheimer’s Dementia classification task of ADReSS INTERSPEECH-2020 challenge, "Alzheimer’s Dementia Recognition through Spontaneous Speech: The ADReSS Challenge". It uses eight different acoustic features to find the attributes in the human speech production system (vocal track and excitation source) affected by Alzheimer’s Dementia. In this study, the Alzheimer’s dementia classification is performed using five different Machine Learning models using ADReSS INTERSPEECH-2020 challenge dataset. Since most of the studies in the previous literature have used linguistic features successfully for Alzheimer’s dementia classification, the current study also demonstrates the performance of the BERT model for the dementia classification task. The maximum accuracy obtained by the acoustic feature is 64.5%, and the BERT Model provides a classification accuracy of 79.1% over the test dataset. Finally, the score-level fusion of the acoustic model with the BERT Model shows an improvement of 6.1% classification accuracy over the BERT Model, which indicates the complementary nature of acoustic features to linguistic features.

@inproceedings{bib_Towa_2021, AUTHOR = {SPARSH, and Mehrotra, utkarsh and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Towards a Database For Detection of Multiple Speech Disfluencies in Indian English}, BOOKTITLE = {National Conference on Communications}. YEAR = {2021}}

The detection and removal of disfluencies from speech is an important task since the presence of disfluencies can adversely affect the performance of speech-based applications such as Automatic Speech Recognition (ASR) systems and speech-to-speech translation systems. From the perspective of Indian languages, there is a lack of studies pertaining to speech disfluencies, their types and frequency of occurrence. Also, the resources available to perform such studies in an Indian context are limited. Through this paper, we attempt to address this issue by introducing the IIITH-Indian English Disfluency (IIITH-IED) Dataset. This dataset consists of 10-hours of lecture mode speech in Indian English. Five types of disfluencies - filled pause, prolongation, word repetition, part-word repetition and phrase repetition were identified in the speech signal and annotated in the corresponding transcription to prepare this dataset. The IIITH-IED dataset was then used to develop framelevel automatic disfluency detection systems. Two sets of features were extracted from the speech signal and then used to train classifiers for the task of disfluency detection. Amongst all the systems employed, Random Forest with MFCC features resulted in the highest average accuracy of 89.61% and F1-score of 0.89.

@inproceedings{bib_Reed_2021, AUTHOR = {Sen, Bipasha and Agarwal, Aditya and Ganesh, Mirishkar Sai and Vuppala, Anil Kumar }, TITLE = {Reed: An Approach Towards Quickly Bootstrapping Multilingual Acoustic Models}, BOOKTITLE = {IEEE Spoken Language Technology Workshop}. YEAR = {2021}}

Multilingual automatic speech recognition (ASR) system is a single entity capable of transcribing multiple languages sharing a common phone space. Performance of such a system is highly dependent on the compatibility of the languages. State of the art speech recognition systems are built using sequential architectures based on recurrent neural networks (RNN) limiting the computational parallelization in training. This poses a significant challenge in terms of time taken to bootstrap and validate the compatibility of multiple languages for building a robust multilingual system. Complex architectural choices based on self-attention networks are made to improve the parallelization thereby reducing the training time. In this work, we propose Reed, a simple system based on 1D convolutions which uses very short context to improve the training time. To improve the performance of our system, we use raw time-domain speech signals directly as input. This enables the convolutional layers to learn feature representations rather than relying on handcrafted features such as MFCC. We report improvement on training and inference times by atleast a factor of 4× and 7.4× respectively with comparable WERs against standard RNN based baseline systems on SpeechOcean's multilingual low resource dataset.

@inproceedings{bib_Comp_2021, AUTHOR = {Barche, Purva and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Comparative Study of Different Epoch Extraction Methods for Speech Associated with Voice Disorders}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2021}}

Accurate detection of epoch locations is important in extracting the features from the speech signal for automatic detection and assessment of voice disorders. Therefore, this study aimed to compare the various algorithms for detecting epoch locations from the speech associated with voice disorders. In this regard, nine state-of-the-art epoch extraction algorithms were considered, and their performance for different categories of voice disorders was evaluated on the SVD dataset. Experimental results indicate that most of the epoch extraction methods showed better performance for healthy speech; however, their performance was degraded for speech associated with voice disorders. Furthermore, the performance of epoch extraction methods was degraded for the speech of structural and neurogenic disorders compared to the speech of psychogenic and functional disorders. Among the different epoch extraction algorithms, zero phase-zero frequency filtering showed the best performance in terms identification rate (90.37%) and identification accuracy (0.34ms), for speech associated with voice disorders.

@inproceedings{bib_Towa_2020, AUTHOR = {Barche, Purva and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Towards Automatic Assessment of Voice Disorders: A Clinical Approach}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2020}}

Automatic detection and assessment of voice disorders is important in diagnosis and treatment planning of voice disorders. This work proposes an approach for automatic detection and assessment of voice disorders from a clinical perspective. To accomplish this, a multi-level classification approach was explored in which four binary classifiers were used for the assessment of voice disorders. The binary classifiers were trained using support vector machines with excitation source features, vocal-tract system features, and state-of-art OpenSMILE features. In this study source features namely, glottal parameters obtained from glottal flow waveform, perturbation measures obtained from epoch locations, and cepstral features obtained from linear prediction residual and zero frequency filtered signal were explored. The present study used the Saarbucken voice disorders database to evaluate the performance of proposed approach. The OpenSMILE features namely ComParE and eGEMAPS feature sets shown better performance in terms of classification accuracies of 82.8% and 76%, respectively for voice disorder detection. The combination of excitation source features with baseline feature sets further improved the performance of detection and assessment systems, that highlight the complimentary nature of exciting source features.

@inproceedings{bib_Towa_2020, AUTHOR = {KRISHNA, GURUGUBELLI and Javid, Mohammad Hashim and ALLURI, KNRK RAJU and Vuppala, Anil Kumar }, TITLE = {Toward Improving the Performance of Epoch Extraction from Telephonic Speech}, BOOKTITLE = {Circuits, Systems, and Signal Processing}. YEAR = {2020}}

Epoch is an abrupt closure event within a glottal cycle at which significant excitation to the vocal-tract system happens during the production of voiced speech. The state-of-the-art zero frequency filtering technique is a simple and efficient method that shows robustness in extracting the epochs from cleanspeech.However,this methodhasshownpoor performance for telephonic quality speech, due to the presence of spurious zero crossings in epoch evidence, which leads to a high false alarm rate. Recently,zero-phase zero frequency resonator(ZP-ZFR) an alternative tozero frequency filter isproposed for stable implementation of zero frequency filtering technique. In this study,higher-order ZP-ZFR is investigated to improve the performance of zero frequency filtering for epoch extraction from telephonic speech. The performance of the proposedZP-ZFR method is quantitatively evaluated on telephonic speech simulated usingsix standard databases having simultaneous electroglottograph recordings as ground truth. Experimental results suggest that the performance of the proposed method is significantly better than the state-of-the-art methods in terms of identification rate and false alarm rate.

@inproceedings{bib_Dete_2020, AUTHOR = {KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Detection of Fricative Landmarks Using Spectral Weighting: A Temporal Approach}, BOOKTITLE = {Circuits, Systems, and Signal Processing}. YEAR = {2020}}

Fricatives are characterized by two prime acoustic properties, i.e., having high-frequency spectral concentration and possessing noisy nature. Spectral domain approaches for detecting fricatives employ a time–frequency representation to compute acoustic cues such as band energy ratio, spectral centroid, and dominant resonant frequency. The detection accuracy of these approaches depends on the efficiency of the employed time–frequency representation. An approach that would not require any time–frequency representation for detecting fricatives from speech has been explored in this work. In this study, a time-domain operation is proposed which emphasizes the high-frequency spectral characteristics of fricatives implicitly. The proposed approach aims to scale the spectrum of the speech signal using a scaling function k2, where k is the discrete frequency. The spectral weighting function used in the proposed approach can be approximated as a cascaded temporal difference operation over speech signal. The emphasized regions in spectrally weighted speech signal are quantified to detect fricative regions. Contrasting the spectral domain approaches, the predictability measure-based approach in literature relies on capturing the noisy nature of fricatives. The proposed approach and the predictability measure-based approaches rely on two complementary properties for detecting fricatives, and a combination of these approaches is put forth in this work. The proposed approach has performed better than the state-of-the-art fricative detectors. To study the significance of the proposed evidence, an early fusion between the proposed evidence and the feature-space maximum log-likelihood transform features is explored for developing speech recognition systems.

@inproceedings{bib_Dura_2020, AUTHOR = {KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar and Narendra, NP and Alku, Paavo }, TITLE = {Duration of the rhotic approximant/ɹ/in spastic dysarthria of different severity levels}, BOOKTITLE = {Speech Communication}. YEAR = {2020}}

Dysarthria is a motor speech disorder leading to imprecise articulation of speech. Acoustic analysis capable of detecting and assessing articulation errors is useful in dysarthria diagnosis and therapy. Since speakers with dysarthria experience difficulty in producing rhotics due to complex articulatory gestures of these sounds, the hypothesis of the present study is that duration of the rhotic approximant /ɹ/ distinguishes dysarthric speech of different severity levels. Duration measurements were conducted using the third formant (F3) trajectories estimated from quasi-closed-phase (QCP) spectrograms. Results indicate that the severity level of spastic dysarthria has a significant effect on duration of /ɹ/. In addition, the phonetic context has a significant effect on duration of /ɹ/, the ɪ-r-ɛ context showing the largest difference in /ɹ/ duration between dysarthric speech of the highest severity levels and healthy speech. The results of this preliminary study can be used in the future to develop signal processing and machine learning methods to automatically predict the severity level of spastic dysarthria from speech signals.

@inproceedings{bib_Stud_2020, AUTHOR = {JAIN, PRIYAM and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Study on the Effect of Emotional Speech on Language Identification}, BOOKTITLE = {National Conference on Communications}. YEAR = {2020}}

Identifying language information from speech utterance is referred to as spoken language identification. Language Identification (LID) is essential in multilingual speech systems. The performance of LID systems have been studied for various adverse conditions such as background noise, telephonic channel, short utterances, so on. In contrast to these studies, for the first time in the literature, the present work investigated the impact of emotional speech on language identification. In this work, different emotional speech databases have been pooled to create the experimental setup. Additionally, state-of-art i-vectors, timedelay neural networks, long short term memory, and deep neural network x-vector systems have been considered to build the LID systems. Performance of the LID system has been evaluated for speech utterances of different emotions in terms of equal error rate and Cavg. The results of the study indicate that the speech utterances of anger and happy emotions degrades performance of LID systems more compared to the neutral and sad emotions. Index Terms—Language Identification, i-vector, TDNN, LSTM, DNN x-vector

@inproceedings{bib_Anal_2020, AUTHOR = {KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Analytic Phase Features for Dysarthric Speech Detection and Intelligibility Assessment}, BOOKTITLE = {Speech Communication}. YEAR = {2020}}

The objectives of the dysarthria assessment are to discriminate dysarthric speech from normal speech, to estimate the severity of dysarthria in terms of the dysarthric speech intelligibility, and to find the motor speech subsystem which causes defects in speech production. In this work, analytic phase features are investigated for the objective assessment of dysarthria. In this connection, the importance of analytic phase in speech intelligibility is studied by employing phase modification schemes. The current investigation on the analytic phase, proposed a novel approach to estimate the instantaneous frequency components from the speech signal, by using single frequency filtering technique. In this study, dysarthric speech detection and intelligibility assessment systems are developed by using UA-Speech database. The efficiency of the analytic phase features is compared with state-of-the-art spectral features. The proposed features outperformed the magnitude and group delay features and shown a classification accuracies of 95.61% and 64.47% for dysarthric speech detection and intelligibility assessment tasks, respectively. The fusion of the evidence obtained from the analytic phase and magnitude spectral features revealed the complementary nature of analytic phase features.

@inproceedings{bib_Sing_2020, AUTHOR = {KRISHNA, GURUGUBELLI and Javid, Mohammad Hashim and Vuppala, Anil Kumar }, TITLE = {Single Frequency Filter Bank Based Long-Term Average Spectra for Hypernasality Detection and Assessment in Cleft Lip and Palate Speech}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2020}}

Hypernasality is an abnormality in speech production observed in subjects with craniofacial anomalies like cleft lip and palate (CLP). Detection and assessment of hypernasality is a primary step in the clinical diagnosis of individuals with CLP. Existing methods explore the short-term spectral information from speech to assess hy-pernasality. The present work examines long-term average spectral (LTAS) features obtained from speech to detect and assess hyper-nasality. This work proposes single frequency filter bank based long-term average spectral (SFFB-LTAS) features for hypernasality detection and assessment. The SFFB is used to extract long-term average spectra with a good spectral resolution. The experiments are carried out using NMCPC-CLP database collected from 41 speakers with CLP and 32 speakers without CLP. The experimental results show that, SFFB-LTAS features performed better compared to state-of-art spectral and prosody features. The proposed systems for the detection and assessment of hypernasality have shown classification accuracy of 89% and 82.1%, respectively.

@inproceedings{bib_IIIT_2019, AUTHOR = {ALLURI, KNRK RAJU and Vuppala, Anil Kumar }, TITLE = {IIIT-H Spoofing Countermeasures for Automatic Speaker Verification Spoofing and Countermeasures Challenge 2019.}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2019}}

The ASVspoof 2019 challenge focuses on countermeasures for all major spoofing attacks, namely speech synthesis (SS), voice conversion (VC), and replay spoofing attacks. This paper describes the IIIT-H spoofing countermeasures developed for ASVspoof 2019 challenge. In this study, three instantaneous cepstral features namely, single frequency cepstral coefficients, zero time windowing cepstral coefficients, and instantaneous frequency cepstral coefficients are used as front-end features. A Gaussian mixture model is used as back-end classifier. The experimental results on ASVspoof 2019 dataset reveal that the proposed instantaneous features are efficient in detecting VC and SS based attacks. In detecting replay attacks, proposed features are comparable with baseline systems. Further analysis is carried out using metadata to assess the impact of proposed countermeasures on different synthetic speech generating algorithm/replay configurations

@inproceedings{bib_Soun_2019, AUTHOR = {Zarazaga, Pablo Perez and Das, Sneha and Backstrom, Tom and V, VISHNU VIDYADHARA RAJU and Vuppala, Anil Kumar }, TITLE = {Sound Privacy: A Conversational Speech Corpus for Quantifying the Experience of Privacy.}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2019}}

With the growing popularity of social networks, cloud services and online applications, people are becoming concerned about the way companies store their data and the ways in which the data can be applied. Privacy with devices and services operated by the voice are of particular interest. To enable studies in privacy, this paper presents a database which quantifies the experience of privacy users have in spoken communication. We focus on the effect of the acoustic environment on that perception of privacy. Speech signals are recorded in scenarios simulating real-life situations, where the acoustic environment has an effect on the experience of privacy. The acoustic data is complemented with measures of the speakers’ experience of privacy, recorded using a questionnaire. The presented corpus enables studies in how acoustic environments affect peoples’ experience of privacy, which in turn, can be used to develop speech operated applications which are respectful of their right to privacy. Index Terms: Experience of privacy, speech interfaces, speech corpus, acoustic environment, right to privacy

@inproceedings{bib_Appl_2019, AUTHOR = {V, VISHNU VIDYADHARA RAJU and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Application of emotion recognition and modification for emotional Telugu speech recognition}, BOOKTITLE = {Mobile Networks and Applications}. YEAR = {2019}}

Majority of the automatic speech recognition systems (ASR) are trained with neutral speech and the performance of these systems are affected due to the presence of emotional content in the speech. The recognition of these emotions in human speech is considered to be the crucial aspect of human-machine interaction. The combined spectral and differenced prosody features are considered for the task of the emotion recognition in the first stage. The task of emotion recognition does not serve the sole purpose of improvement in the performance of an ASR system. Based on the recognized emotions from the input speech, the corresponding adapted emotive ASR model is selected for the evaluation in the second stage. This adapted emotive ASR model is built using the existing neutral and synthetically generated emotive speech using prosody modification method. In this work, the importance of emotion recognition block at the front-end along with the emotive speech adaptation to the ASR system models were studied. The speech samples from IIIT-H Telugu speech corpus were considered for building the large vocabulary ASR systems. The emotional speech samples from IITKGP-SESC Telugu corpus were used for the evaluation. The adapted emotive speech models have yielded better performance over the existing neutral speech models.

@inproceedings{bib_Repl_2019, AUTHOR = {ALLURI, KNRK RAJU and Vuppala, Anil Kumar }, TITLE = {Replay spoofing countermeasures using high spectro-temporal resolution features}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2019}}

The easy implementation of replay attacks by a fraudster poses a severe threat to automatic speaker verification (ASV) technology than the other spoofing attacks like speech synthesis and voice conversion. Replay attacks refer to an attack by a fraudster to get illegitimate access to an ASV system by playing back the speech sample collected from genuine target speaker. The significant cues that can differentiate between genuine and replay recordings are channel characteristics. To capture these characteristics, one need to extract features from the spectrum, which should have high spectral and temporal resolutions. Zero time windowing (ZTW) analysis of speech is one such time-frequency analysis technique, which results in high spectral and temporal resolution spectrum at each sampling instant. In this study, new features are proposed by applying cepstral analysis to ZTW spectrum. Experiments are performed on two publicly available replay attack databases namely BTAS 2016 and ASVspoof 2017. The first set of experiments are conducted using Gaussian mixture models to evaluate the potential of proposed features. Performance of the proposed system in terms of half total error rate is 0.75% and in terms of equal error rate is 14.75% on BTAS 2016 and ASVspoof 2017 evaluation sets respectively. A score level fusion is performed by using proposed features with previously proposed single frequency filtering cepstral coefficients. This fused result outperformed the previously reported best results on these two datasets

@inproceedings{bib_Perc_2019, AUTHOR = {KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Perceptually enhanced single frequency filtering for dysarthric speech detection and intelligibility assessment}, BOOKTITLE = {International Conference on Acoustics, Speech, and Signal Processing}. YEAR = {2019}}

This paper proposes a new speech feature representation that improves the intelligibility assessment of dysarthric speech. The formulation of the feature set is motivated from the human auditory perception and high time-frequency resolution property of single frequency filtering (SFF) technique. The proposed features are named as perceptually enhanced single frequency cepstral coefficients (PESFCC). As a part of SFF technique implementation, speech signal passed through a single pole complex bandpass filter bank to obtain high-resolution time-frequency distribution. Then, the distribution is enhanced by using a set of auditory perceptual operators. Lastly, traditional homomorphic analysis has been carried out on the resulting signal to obtain PE-SFCC feature vector. The performance of proposed features in dysarthric speech detection and its intelligibility assessment has been reported on UASPEECH database. The PE-SFCC features outperformed the state-of-the-art features in dysarthric speech detection and intelligibility assessment.

@inproceedings{bib_Stab_2019, AUTHOR = {KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Stable implementation of zero frequency filtering of speech signals for efficient epoch extraction}, BOOKTITLE = {IEEE Signal Processing Letter}. YEAR = {2019}}

Epochs are the abrupt-closure events in vocal fold vibration during the production of voiced speech. Zero frequency filtering is a simple and effective technique used to estimate the glottal closure instants accurately from the speech signal. However, the zero frequency filter is an unstable system. Hence, it may not be suitable for practical implementation due to the requirements of high precision computation. In this letter, zero-phase zero frequency resonator is proposed as an alternative to zero frequency filter. The proposed approach provides a stable zero-phase response. The experimental results indicate that the performance of the proposed method outperformed the state-of-the-art methods in terms of identification rate 99.17% and provides comparable performance in terms of false alarm rate (0.41%), and identification accuracy (0.28 ms).

@inproceedings{bib_Mult_2019, AUTHOR = {KUMAR, VUDDAGIRI RAVI and Tirusha, Mandava and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Multi-Head Self-Attention Networks for Language Identification}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2019}}

Self-attention networks are being popularly employed in sequence classification and sequence summarization tasks. State-of-the-art models use sequential models to capture the high-level information, but these models are sensitive to length of utterance and do not equally generalize over variable length utterances. This work explores to study the efficiency of recent advancements in self-attentive networks for improving the performance of the LID system. In self-attentive network, variable length input sequence is converted to fixed dimensional vector which represents the whole sequence. The weighted mean of input sequence is considered as utterance level representation. Along with the mean, a standard deviation is employed to represent the whole input sequence. Experiments are performed using AP17-OLR database. Use of mean with standard deviation has reduced the equal error rate (EER) with an 8% relative improvement. A multi-head attention mechanism is introduced in self-attention networks with an assumption that each head captures the distinct information to discriminate languages. Use of multi-head self-attention has further reduced the EER with a 13% relative improvement. Best performance is achieved with multi-head self-attention network with residual connections. Shifted delta cepstral features (SDC) and stacked SDC features are used for developing LID systems.

@inproceedings{bib_Atte_2019, AUTHOR = {Tirusha, Mandava and Vuppala, Anil Kumar }, TITLE = {Attention based residual-time delay neural network for Indian language identification}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2019}}

—India is a multilingual society having more than 1600 languages. Most of these languages are having an overlapping set of phonemes. This makes developing language identification (LID) framework difficult for Indian languages. In this paper, the above challenge is addressed using phonetic features. To model the temporal variations in phonetic features, attention based residual-time delay neural network (RES-TDNN) is proposed. This network effectively captures long-range temporal dependencies through TDNN and attention mechanism. The proposed network has been evaluated on IIITH-ILSC database using phonetic and acoustic features. The database consists of 22 official Indian languages and Indian English. Attention based RES-TDNN outperformed the other state-of-the-art networks such as deep neural network, long short-term memory network and produced an equal error rate of 9.46%. Further, the fusion of shifted delta cepstral and phonetic features have improved the performance.

@inproceedings{bib_Towa_2019, AUTHOR = {JAIN, PRIYAM and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Towards Emotion Independent Language Identification System}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2019}}

—India is a multilingual society having more than 1600 languages. Most of these languages are having an overlapping set of phonemes. This makes developing language identification (LID) framework difficult for Indian languages. In this paper, the above challenge is addressed using phonetic features. To model the temporal variations in phonetic features, attention based residual-time delay neural network (RES-TDNN) is proposed. This network effectively captures long-range temporal dependencies through TDNN and attention mechanism. The proposed network has been evaluated on IIITH-ILSC database using phonetic and acoustic features. The database consists of 22 official Indian languages and Indian English. Attention based RES-TDNN outperformed the other state-of-the-art networks such as deep neural network, long short-term memory network and produced an equal error rate of 9.46%. Further, the fusion of shifted delta cepstral and phonetic features have improved the performance.

@inproceedings{bib_An_I_2019, AUTHOR = {Tirusha, Mandava and KUMAR, VUDDAGIRI RAVI and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {An Investigation of LSTM-CTC based Joint Acoustic Model for Indian Language Identification}, BOOKTITLE = {Workshop on Automatic Speech Recognition and Understanding}. YEAR = {2019}}

In this paper, phonetic features derived from the joint acoustic model (JAM) of a multilingual end to end automatic speech recognition system are proposed for Indian language identification (LID). These features utilize contextual information learned by the JAM through long short-term memoryconnectionist temporal classification (LSTM-CTC) framework. Hence, these features are referred to as CTC features. A multi-head self-attention network is trained using these features, which aggregates the frame-level features by selecting prominent frames through a parametrized attention layer. The proposed features have been tested on IIITH-ILSC database that consists of 22 official Indian languages and Indian English. Experimental results demonstrate that CTC features outperformed i-vector and phonetic temporal neural LID systems and produced an 8.70% equal error rate. The fusion of shifted delta cepstral and CTC feature-based LID systems at the model level and feature level further improved the performance.

@inproceedings{bib_An_E_2018, AUTHOR = {KRISHNA, VYDANA HARI and KRISHNA, GURUGUBELLI and V, VISHNU VIDYADHARA RAJU and Vuppala, Anil Kumar }, TITLE = {An Exploration towards Joint Acoustic Modeling for Indian Languages: IIIT-H Submission for Low Resource Speech Recognition Challenge for Indian Languages, INTERSPEECH 2018.}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2018}}

India being a multilingual society, a multilingual automatic speech recognition system (ASR) is widely appreciated. Despite different orthographies, Indian languages share same phonetic space. To exploit this property, a joint acoustic model has been trained for developing multilingual ASR system using a common phone-set. Three Indian languages namely Telugu, Tamil and, Gujarati are considered for the study. This work studies the amenability of two different acoustic modeling approaches for training a joint acoustic model using common phone-set. Sub-space Gaussian mixture models (SGMM), and recurrent neural networks (RNN) trained with connectionst temporal classification (CTC) objective function are explored for training joint acoustic models. From the experimental results, it can be observed that the joint acoustic models trained with RNN-CTC have performed better than SGMM system even on 120 hours of data (approx 40 hrs per language). The joint acoustic model trained with RNN-CTC has performed better than monolingual models, due to an efficient data sharing across the languages. Conditioning the joint model with language identity had a minimal advantage. Sub-sampling the features by a factor of 2 while training RNN-CTC models has reduced the training times and has performed better. Index Terms: Speech recognition, Joint acoustic model, lowresource, common phone set, Indian languages, RNN-CTC, SGMM

@inproceedings{bib_Impr_2018, AUTHOR = {KUMAR, VUDDAGIRI RAVI and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Improved Language Identification Using Stacked SDC Features and Residual Neural Network.}, BOOKTITLE = {Workshop on Spoken Language Technologies for Under-Resourced Languages}. YEAR = {2018}}

Language identification (LID) systems, which can model highlevel information such as phonotactics have exhibited superior performance. State-of-the-art models use sequential models to capture the high-level information, but these models are sensitive to the length of the utterance and do not equally generalize over variable length utterances. To effectively capture this information, a feature that can model the long-term temporal context is required. This study aims to capture the long-term temporal context by appending successive shifted delta cepstral (SDC) features. Deep neural networks have been explored for developing LID systems. Experiments have been performed using AP17-OLR database. LID systems developed by stacking SDC features have shown significant improvement compared to the system trained with SDC features. The proposed feature with residual connections in the feed-forward networks reduced the equal error rate from 21.04, 18.02, 16.45 to 14.42, 11.14 and 10.11 on the 1-second, 3-seconds and > 3-second test utterances respectively.

@inproceedings{bib_IIIT_2018, AUTHOR = {KUMAR, VUDDAGIRI RAVI and KRISHNA, GURUGUBELLI and JAIN, PRIYAM and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {IIITH-ILSC Speech Database for Indain Language Identification.}, BOOKTITLE = {Workshop on Spoken Language Technologies for Under-Resourced Languages}. YEAR = {2018}}

This work focuses on the development of speech data comprising 23 Indian languages for developing language identification (LID) systems. Large data is a pre-requisite for developing state-of-the-art LID systems. With this motivation, the task of developing multilingual speech corpus for Indian languages has been initiated. This paper describes the composition of the data and the performances of various LID systems developed using this data. In this paper, Mel frequency cepstral feature representation is used for language identification. In this work, various state-of-the-art LID systems are developed using i-vectors, deep neural network (DNN) and deep neural network with attention (DNN-WA) models. The performance of the LID system is observed in terms of the equal error rate for i-vector, DNN and DNN-WA is 17.77%, 17.95%, and 15.18% respectively. Deep neural network with attention model shows a better performance over i-vector and DNN models

@inproceedings{bib_Auto_2018, AUTHOR = {KRISHNA, THIRUMURU RAMA and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Automatic Detection of Palatalized Consonants in Kashmiri}, BOOKTITLE = {Workshop on Spoken Language Technologies for Under-Resourced Languages}. YEAR = {2018}}

In this study, the acoustic-phonetic attributes of palatalization in the Kashmiri speech is investigated. It is a unique phonetic feature of Kashmiri in the Indian context. An automated approach is proposed to detect this unique phonetic feature from the continuous Kashmiri speech. The i-matra vowel has the impact of palatalizing the consonant connected to it. Therefore, these consonants investigated in synchronous with vowel regions, which are spotted using the instantaneous energy computed from the envelope-derivative of the speech signal. The resonating characteristics of the vocal-tract system framework that reflect the formant dynamics are used to differentiate palatalized consonants from the other consonants. In this regard, the Hilbert envelope of the numerator of the group-delay function that provides good time-frequency resolution used to extract formants. The palatalization detection experimentation carried out in various vowel contexts using the acoustic cues, and it produced a promising result with a detection accuracy of 92.46 %.

@inproceedings{bib_Curr_2018, AUTHOR = {KUMAR, VUDDAGIRI RAVI and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Curriculum learning based approach for noise robust language identification using DNN with attention}, BOOKTITLE = {Expert Systems with Applications}. YEAR = {2018}}

Automatic language identification (LID) in practical environments is gaining a lot of scientific attention due to rapid developments in multilingual speech processing applications. When an LID is operated in noisy environments a degradation in the performance can be observed and it can be majorly attributed to mismatch between the training and operating environments. This work is aimed towards developing an LID system that can robustly operate in clean and noisy environments. Traditionally, to reduce the mismatch between training and operating environments, noise is synthetically induced to the training corpus and these models are termed as multi-SNR models. In this work, various curriculum learning strategies are explored to train multi-SNR models, such that the trained models have better generalization in performance over varying background environments. I-vector, Deep neural networks (DNN) and DNN With Attention (DNN-WA) architectures are used in this work for developing LID systems. Experimental verification of the proposed approach is carried out using IIIT-H Indian database and AP17-OLR database. The performance of LID system is tested at different signal-to-noise ratio (SNR) levels using white and vehicular noises from NOISEX dataset. In comparison to multi-SNR models, the LID systems trained with curriculum learning have performed better in terms of equal error rate (EER) and generalization in EER across varying background environments. The degradation in the performance of LID systems due to environmental noise has been effectively reduced by training multi-SNR models using curriculum learning.

@inproceedings{bib_Impr_2018, AUTHOR = {KRISHNA, THIRUMURU RAMA and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Improved vowel region detection from a continuous speech using post processing of vowel onset points and vowel end-points}, BOOKTITLE = {Journal on Multimedia Tools Applications}. YEAR = {2018}}

Vowels are produced with an open configuration of the vocal tract, without any audible friction. The acoustic signal is relatively loud with varying strength of impulselike excitation. Vowels possess significant energy content in the low-frequency bands of the speech signal. Acoustic events such as vowel onset point (VOP) and vowel end-point (VEP) can be used as landmarks to detect vowel regions in a speech signal. In this paper, two-stage algorithm is proposed to detect precise vowel regions. In the first level, the speech signal is processed using zero frequency filtering to emphasize energy content in low-frequency bands of speech. Zero frequency filtered signal predominantly contains lowfrequency content of the speech signal as it is filtered around 0 Hz. This process is followed by the extraction of dominant spectral peaks from the magnitude spectrum around glottal closure regions of the speech signal. The vowel onset points and vowel end-points are obtained by convolving the enhanced spectral contour of zero frequency filtered signal with first order Gaussian differentiator. In the next level, a post-processing is carried out in the regions around VOP and VEP to remove spurious vowel regions based on uniformity of epoch intervals. In addition, the positions of VOPs and VEPs are also corrected using the strength of the excitation of the speech signal. The performance of the proposed vowel region detection method is compared with the existing state of art methods on TIMIT acoustic-phonetic speech corpus. It is reported that this method produced significant improvement in vowel region detection in clean and noisy environments.

@inproceedings{bib_Diff_2018, AUTHOR = {KRISHNA, GURUGUBELLI and ALLURI, KNRK RAJU and Vuppala, Anil Kumar }, TITLE = {Differenced Prosody Features from Normal and Stressed Regions for Emotion Recognition}, BOOKTITLE = {International Conference on Signal Processing and Integrated Networks}. YEAR = {2018}}

The physiological constraints in the human speech production mechanism restrict the emotional state of the speaker for a short duration in an emotional speech. In this work, the emotionally stressed regions are detected using the strength of the excitation (SoE) and fundamental frequency (F0) of the speech signal. These emotionally stressed regions for the different emotions are further classified using a linear kernel support vector machine (SVM) with a twostage binary decision logic. This classifier is modelled using the differenced prosody features extracted by considering the relative difference of the prosody components between normal and emotionally stressed regions. This experimentation is carried out using Berlin emotion speech (EMO-DB) database. The proposed approach produced a better result compared to existing state-of-the-art emotion recognition systems. An accuracy of 80.83% is reported in this paper.

@inproceedings{bib_Sign_2018, AUTHOR = {KRISHNA, THIRUMURU RAMA and KUMAR, VUDDAGIRI RAVI and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Significance of Accuracy in Vowel Region Detection for Robust Language Identification}, BOOKTITLE = {International Conference on Signal Processing and Integrated Networks}. YEAR = {2018}}

In this study, a robust language identification (LID) system is investigated with vowel region detection scheme incorporated at the front-end. The background noise is known to severely reduce language identification performance. In this work, Mel-frequency cepstral features extracted from the vowel regions instead of voiced speech are utilized for language identification system modeling. Experimentation is done using different vowel region detection algorithms on deep neural networks (DNN) and deep neural networks with attention (DNN-WA) based LID systems on IIITH Indian language dataset. Experimental results show that the accurate vowel region detection mechanism at the front-end of a LID system improved the performance in noisy environment.

@inproceedings{bib_Appl_2018, AUTHOR = {KRISHNA, THIRUMURU RAMA and Vuppala, Anil Kumar }, TITLE = {Application of non-negative frequency-weighted energy operator for vowel region detection}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2018}}

In this paper, a novel technique has been proposed for the vowel region detection from the continuous speech using an envelope of the derivative of the speech signal, which is a non-negative, frequency-weighted energy operator. The proposed vowel region detection method is implemented using a two-stage algorithm. The first stage of vowel region detection consists of speech signal analysis to detect vowel onset points (VOP) and vowel end-points (VEP) using an instantaneous energy contour obtained from the envelope of the derivative of a speech signal. The VOPs and VEPs are spotted using the peak-finding algorithm based upon the first order Gaussian differentiator. The next stage consists of removal of spurious vowel regions and the correction of hypothesized VOP and VEP locations using combined cues obtained from the uniformity of epoch intervals and strength of the excitation of the speech signal. Performance of the proposed method for detecting vowel regions from the speech signal is evaluated using TIMIT acoustic-phonetic speech corpus. The proposed approach resulted in significantly high detection rate and less false alarm rate compared to the state-of-the-art methods in both clean and noisy environments

@inproceedings{bib_Auto_2018, AUTHOR = {KRISHNA, THIRUMURU RAMA and Vuppala, Anil Kumar }, TITLE = {Automatic detection of retroflex approximants in a continuous tamil speech}, BOOKTITLE = {Circuits, Systems, and Signal Processing}. YEAR = {2018}}

t Phonetic feature extraction is used in many types of speech applications. In this paper, we propose an approach for automatic detection of retroflex approximant /õ/ in a Tamil continuous speech using sonorant regions and formant structure of speech. Retroflex approximant is one of the distinctive phonetic features of Tamil speech. It occurs in V_V and V_ contexts. In the proposed approach, slopes of the formant trajectories and spectral dynamics of F2 and F3 in the region of retroflex approximant are considered as acoustic cues for detecting them in continuous speech. In this work, numerator of the group delay function-based formant extraction method is used to track formant trajectories. Based on experimental results, formants extracted using this approach produced better evidence compared to state-of-the-art formant extraction techniques for retroflex approximant detection.

@inproceedings{bib_Inco_2018, AUTHOR = {KRISHNA, VYDANA HARI and A, SIVANAND and Vuppala, Anil Kumar }, TITLE = {Incorporating Speaker Normalizing Capabilities to an End-to-End Speech Recognition System}, BOOKTITLE = {Workshop on Spoken Language Technologies for Under-Resourced Languages}. YEAR = {2018}}

Speaker normalization is one of the crucial aspects of an Automatic speech recognition system (ASR). Speaker normalization is employed to reduce the performance drop in ASR due to speaker variabilities. Traditional speaker normalization methods are mostly linear transforms over the input data estimated per speaker, such transforms would be efficient with sufficient data. In practical scenarios, only a single utterance from the test speaker is accessible. The present study explores speaker normalization methods for end-to-end speech recognition systems that could efficiently be performed even when single utterance from the unseen speaker is available. In this work, it is hypothesized that by suitably providing information about the speaker’s identity while training an end-to-end neural network, the capability to normalize the speaker variability could be incorporated into an ASR system. The efficiency of these normalization methods depends on the representation used for unseen speakers. In this work, the identity of the training speaker is represented in two different ways viz. i) by using a one-hot speaker code, ii) a weighted combination of all the training speakers identities. The unseen speakers from the test set are represented using a weighted combination of training speakers representations. Both the approaches have reduced the word error rate (WER) by 0.6, 1.3% WSJ corpus

@inproceedings{bib_Comb_2018, AUTHOR = {MOUNIKA, K V and KUMAR, VUDDAGIRI RAVI and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Combining evidences from excitation source and vocal tract system features for Indian language identification using deep neural networks}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2018}}

In this paper, a combination of excitation source information and vocal tract system information is explored for the task of language identification (LID). The excitation source information is represented by features extracted from linear prediction (LP) residual signal called the residual cepstral coefficients (RCC). Vocal tract system information is represented by the mel frequency cepstral coefficients (MFCC). In order to incorporate additional temporal information, shifted delta cepstra (SDC) are computed. An LID system is built using SDC over both MFCC and RCC features individually and evaluated based on their equal error rate (EER). Experiments have been performed on a dataset consisting of 13 Indian languages with about 115 h for training and 30 h for testing using a deep neural network (DNN), DNN with attention (DNN-WA) and a state-ofthe-art i-vector system. DNN-WA outperforms the baseline i-vector system. An EER of 9.93 and 6.25% are achieved using RCC and MFCC features respectively. By combining evidence from both features using a late fusion mechanism, an EER of 5.76% is obtained. This result indicates the complementary nature of the excitation source information to that of the widely used vocal tract system information for the task of LID.

@inproceedings{bib_Pros_2018, AUTHOR = {V, VISHNU VIDYADHARA RAJU and KRISHNA, GURUGUBELLI and Vuppala, Anil Kumar }, TITLE = {Prosody modification for speech recognition in emotionally mismatched conditions}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2018}}

A degradation in the performance of automatic speech recognition systems (ASR) is observed in mismatched training and testing conditions. One of the reasons for this degradation is due to the presence of emotions in the speech. The main objective of this work is to improve the performance of ASR in the presence of emotional conditions using prosody modification. The influence of different emotions on the prosody parameters is exploited in this work. Emotion conversion methods are employed to generate the word level non-uniform prosody modified speech. Modification factors for prosodic components such as pitch, duration and energy are used. The prosody modification is done in two ways. Firstly, emotion conversion is done at the testing stage to generate the neutral speech from the emotional speech. Secondly, the ASR is trained with the generated emotional speech from the neutral speech. In this work, the presence of emotions in speech is studied for the Telugu ASR systems. A new database of IIIT-H Telugu speech corpus is collected to build the large vocabulary neutral Telugu speech ASR system. The emotional speech samples from IITKGP-SESC Telugu corpus are used for testing it. The emotions of anger, happiness and compassion are considered during the evaluation. An improvement in the performance of ASR systems is observed in the prosody modified speech.

@inproceedings{bib_A_st_2017, AUTHOR = {GAMBHIR, MAHAK and Vuppala, Anil Kumar }, TITLE = {A study on the minimum dominating set problem approximation in parallel}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2017}}

A dominating set of a small size is useful in several settings including wireless networks, document summarization, secure system design, and the like. In this paper, we start by studying three distributed algorithms that produce a small sized dominating sets in a few rounds. We interpret these algorithms in the natural shared memory setting and experiment with these algorithms on a multi-core CPU. Based on the observations from these experimental results, we propose variations to the three algorithms and also show how the proposed variations offer interesting trade-offs with respect to the size of the dominating set produced and the time taken.

@inproceedings{bib_Resi_2017, AUTHOR = {KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Residual neural networks for speech recognition}, BOOKTITLE = {European Signal Processing Conference}. YEAR = {2017}}

Recent developments in deep learning methods have greatly influenced the performances of speech recognition systems. In a Hidden Markov model-Deep neural network (HMM-DNN) based speech recognition system, DNNs have been employed to model senones (context dependent states of HMM),where HMMs capture the temporal relations among senones. Due to the use of more deeper networks significant improvement in the performances has been observed and developing deep learning methods to train more deeper architectures has gained a lot of scientific interest. Optimizing a deeper network is more complex ask than to optimize a less deeper network, but recently residual network have exhibited a capability to train a very deep neural network architectures and are not prone to vanishing/exploding gradient problems. In this work, the effectiveness of residual networks have been explored for of speech recognition. Along with the depth of the residual network, the criticality of width of the residual network has also been studied. It has been observe dthat at higher depth, width of the networks is also a crucial parameter for attaining significant improvements. A 14-hoursubset of WSJ corpus is used for training the speech recognition systems, it has been observed that the residual networks have shown much ease in convergence even with a depth much higher than the deep neural network. In this work, using residual networks an absolute reduction of 0.4 in WER error rates (8%reduction in the relative error) is attained compared to the best performing deep neural network.

@inproceedings{bib_Inve_2017, AUTHOR = {KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Investigative Study of Various Activation functions for Speech Recognition}, BOOKTITLE = {National Conference on Communications}. YEAR = {2017}}

Significant developments in deep learning methods have been achieved with the capability to train more deeper networks. The performance of speech recognition system has been greatly improved by the use of deep learning techniques. Most of the developments in deep learning are associated with the development of new activation functions and the corresponding initializations. The development of Rectified linear units (ReLU)has revolutionized the use of supervised deep learning methods for speech recognition. Recently there has been a great deal of research interest in the development of activation functionsLeaky-ReLU (LReLU), Parametric-ReLU (PReLU), Exponentia lLinear units (ELU) and Parametric-ELU (PELU). This work isaimed at studying the influence of various activation functions onspeech recognition system. In this work, a hidden Markov model-Deep neural network (HMM-DNN) based speech recognitionis used, where deep neural networks with different activationfunctions have been employed to obtain the emission probabilitiesof hidden Markov model. In this work, two datasets i.e., TIMITand WSJ are employed to study the behavior of various speechrecognition systems with different sized datasets. During thestudy, it is observed that the performance of ReLU-networksis superior compared to the other networks for the smaller sizeddataset (i.e., TIMIT dataset). For the datasets of sufficientlylarger size (i.e., WSJ) performance of ELU-networks is superiorto the other networks.

@inproceedings{bib_Dete_2017, AUTHOR = {ALLURI, KNRK RAJU and A, SIVANAND and REDDY, KADIRI SUDARSANA and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Detection of Replay Attacks using Single Frequency Filtering Cepstral Coefficients}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2017}}

Automatic speaker verification systems are more vulnerable to spoofing attacks. Recently, various countermeasures have been developed for detecting high technology attacks such as speech synthesis and voice conversion. However, there is a wide gap in dealing with replay attacks. In this paper, we propose a new feature for replay attack detection based on single frequency filtering (SFF), which provides high temporal and spectral resolution at each instant. Single frequency filtering cepstral coefficients (SFFCC) with Gaussian mixture model classifier is used for the experimentation on the standard BTAS-2016 corpus. The previously reported best result, which is based on constant Q cepstral coefficients (CQCC) has achieved a half total error rate of 0.67% on this data-set. Our proposed method outperforms the state of the art (CQCC) with a half total error rate of 0.0002%.

@inproceedings{bib_SFF__2017, AUTHOR = {ALLURI, KNRK RAJU and A, SIVANAND and REDDY, KADIRI SUDARSANA and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {SFF Anti-Spoofer: IIIT-H Submission for Automatic Speaker Verification Spoofing and Countermeasures Challenge 2017.}, BOOKTITLE = {Annual Conference of the International Speech Communication Association}. YEAR = {2017}}

The ASVspoof 2017 challenge is about the detection of replayed speech from human speech. The proposed system makes use of the fact that when the speech signals are replayed, they pass through multiple channels as opposed to original recordings. This channel information is typically embedded in low signal to noise ratio regions. A speech signal processing method with high spectro-temporal resolution is required to extract robust features from such regions. The single frequency filtering (SFF) is one such technique, which we propose to use for replay attack detection. While SFF based feature representation was used at front-end, Gaussian mixture model and bi-directional long short-term memory models are investigated at the backend as classifiers. The experimental results on ASVspoof 2017 dataset reveal that, SFF based representation is very effective in detecting replay attacks. The score level fusion of back end classifiers further improved the performance of the system which indicates that both classifiers capture complimentary information

@inproceedings{bib_Sign_2017, AUTHOR = {SRIVASTAVA, BRIJ MOHAN LAL and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar and Srivastava, Manish }, TITLE = {Significance of neural phonotactic models for large-scale spoken language identification}, BOOKTITLE = {International Joint Conference on Neural Networks}. YEAR = {2017}}

Language identification (LID) is vital frontend for spoken dialogue systems operating in diverse linguistic settings to reduce recognition and understanding errors. Existing LID systems which use low-level signal information for classification do not scale well due to exponential growth of parameters as the classes increase. They also suffer performance degradation due to the inherent variabilities of speech signal. In the proposed approach, we model the language-specific phonotactic information in speech using recurrent neural network for developing an LID system. The input speech signal is tokenized to phone sequences by using a common language-independent phone recognizer with varying phonetic coverage. We establish a causal relationship between phonetic coverage and LID performance. The phonotactics in the observed phone sequences are modeled using statistical and recurrent neural network language models to predict language-specific symbol from a universal phonetic inventory. Proposed approach is robust, computationally light weight and highly scalable. Experiments show that the convex combination of statistical and recurrent neural network language model (RNNLM) based phonotactic models significantly outperform a strong baseline system of Deep Neural Network (DNN) which is shown to surpass the performance of i-vector based approach for LID. The proposed approach outperforms the baseline models in terms of mean F1 score over 176 languages. Further we provide significant information-theoretic evidence to analyze the mechanism of the proposed approach.

@inproceedings{bib_Sent_2017, AUTHOR = {ABBURI, HARIKA and ALLURI, KNRK RAJU and Vuppala, Anil Kumar and Srivastava, Manish and Gangashetty, Suryakanth }, TITLE = {Sentiment analysis using relative prosody features}, BOOKTITLE = {International Conference on Contemporary Computing}. YEAR = {2017}}

Recent improvement in usage of digital media has led people to share their opinions about specific entity through audio. In this paper, an approach to detect the sentiment of an online spoken reviews based on relative prosody features is presented. Most of the existing systems for audio based sentiment analysis use conventional audio features, but they are not problem specific features to extract the sentiment. In this work, relative prosody features are extracted from normal and stressed regions of audio signal to detect the sentiment. Stressed regions are identified using the strength of excitation. Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) classifiers are used to build the sentiment models. MOUD database is used for the proposed study. Experimental results show that, the rate of detecting the sentiment is improved with relative prosody features compared with the prosody and Mel Frequency Cepstral Coefficients (MFCC) because the relative prosody features has more sentiment specific discrimination compared to prosody features.

@inproceedings{bib_Impo_2017, AUTHOR = {V, VISHNU VIDYADHARA RAJU and KRISHNA, VYDANA HARI and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Importance of non-uniform prosody modification for speech recognition in emotion conditions}, BOOKTITLE = {Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)}. YEAR = {2017}}

A mismatch in training and operating environments causes a performance degradation in speech recognition systems (ASR). One major reason for this mismatch is due to the presence of expressive (emotive) speech in operational environments. Emotions in speech majorly inflict the changes in the prosody parameters of pitch, duration and energy. This work is aimed at improving the performance of speech recognition systems in the presence of emotive speech. This work focuses on improving the speech recognition performance without disturbing the existing ASR system. The prosody modification of pitch, duration and energy is achieved by tuning the modification factors values for the relative differences between the neutral and emotional data sets. The neutral version of emotive speech is generated using uniform and non-uniform prosody modification methods for speech recognition. During the study, IITKGP-SESC corpus is used for building the ASR system. The speech recognition system for the emotions (anger, happy and compassion) is evaluated. An improvement in the performance of ASR is observed when the prosody modified emotive utterance is used for speech recognition in place of original emotive utterance. An average improvement around 5% in accuracy is observed due to the use of non-uniform prosody modification methods.

@inproceedings{bib_DNN-_2017, AUTHOR = {V, VISHNU VIDYADHARA RAJU and KRISHNA, GURUGUBELLI and KRISHNA, VYDANA HARI and Pulugundla, Bhargav and Srivastava, Manish and Vuppala, Anil Kumar }, TITLE = {DNN-HMM Acoustic Modeling for Large Vocabulary Telugu Speech Recognition}, BOOKTITLE = {International Conference on Mining Intelligence and Knowledge Exploration}. YEAR = {2017}}

The main focus of this paper is towards the development of a large vocabulary Telugu speech database. Telugu is a low resource language where there exists no standardized database for building the speech recognition system (ASR). The database consists of neutral speech samples collected from 100 speakers for building the Telugu ASR system and it was named as IIIT-H Telugu speech corpus. The speech and text corpus design and the procedure followed for the collection of the database have been discussed in detail. The preliminary ASR system results for the models built in this database are reported. The architectural choices of deep neural networks (DNNs) play a crucial role in improving the performance of ASR systems. ASR trained with hybrid DNNs (DNNHMM) with more hidden layers have shown better performance over the conventional GMMs (GMM-HMM). Kaldi tool kit is used for building the acoustic models required for the ASR system.

@inproceedings{bib_Vowe_2016, AUTHOR = {KRISHNA, VYDANA HARI and REDDY, KADIRI SUDARSANA and Vuppala, Anil Kumar }, TITLE = {Vowel-Based Non-uniform Prosody Modification for Emotion Conversion}, BOOKTITLE = {Circuits, Systems, and Signal Processing}. YEAR = {2016}}

The objective of this work is to develop a rule-based emotion conversion method for a better emotional perception. In this work, performance of emotion conversion using the linear modification model is improved by using vowel-based non-uniform prosody modification. In the present approach, attempts were made to integrate features like position and identity for addressing the non-uniformity in prosody generated due to the emotional state of the speaker. We mainly concentrate on the parameters such as strength, duration and pitch contour of vowels at different parts of the sentence. The influence of emotions on the above parameters is exploited to convert the speech from neutral emotion to the target emotion. Non-uniform prosody modification factors for emotion conversion are based on the position of vowels in the word, and the position of the word in the sentence. This study is carried out by using Indian Institute of Technology-Simulated Emotion speech corpus. Evaluation of the proposed algorithm is carried out by a subjective listening test. From the listening tests, it is observed that the performance of the proposed approach is better than the existing approaches

@inproceedings{bib_Chan_2016, AUTHOR = {Mittal, Vinay Kumar and Vuppala, Anil Kumar }, TITLE = {Changes in shout features in automatically detected vowel regions}, BOOKTITLE = {International Conference on Signal Processing and Communications}. YEAR = {2016}}

Shouted speech signals have been studied mostlyfor utterances or word segments. In the production features derived over these segments, the differences between normal and shouted speech may sometimes be masked by variations due to pauses and unvoiced regions. Also, our recent study of electroglottograph signals has highlighted the usefulness of examining changes in the glottal excitation source characteristics during production of shouted speech. In this paper we examine changes in the shout production features in the vowel regions,that are found automatically by using the knowledge of accurate vowel onset points and epochs. Vowel regions are produced by periodic impulse-like excitation and contain relatively high signal energy. Hence the source features the instantaneous fundamental frequency and strength of excitation are examined, along with the filter feature the dominant frequency, for five different steady vowel regions. Changes in these features, that distinguish between normal and shouted speech, are more prominent in the automatically found vowel regions than for the entire word segments. This insight can help improving the automatic shout detection and other paralinguistic applications where acoustic cues need to be found in the vowel regions, e.g., detection of emotion categories.

@inproceedings{bib_Sign_2016, AUTHOR = {Mittal, Vinay Kumar and Vuppala, Anil Kumar }, TITLE = {Significance of automatic detection of vowel regions for automatic shout detection in continuous speech}, BOOKTITLE = {International Symposium on Chinese Spoken Language Processing}. YEAR = {2016}}

Automatic detection of shout prosody in continuous speech signal involves examining changes in its production characteristics. Our recent study of electroglottograph signals highlighted that significant changes occur in the glottal excitation source characteristics during production of shouted speech, especially in the vowel contexts. But the differences between normal and shouted speech, in the production features derived over utterances or word segments, may be masked sometimes by pauses or unvoiced regions related variations. Also, for such a real-time system, these vowel regions need to be found automatically. In this paper, changes in the shout production features are examined in the automatically detected vowel regions. Production of a vowel involves periodic impulse-like excitation and relatively high signal energy. Hence, the knowledge of epochs using zero-frequency filtering, and accurate vowel onset points can be used for detecting these regions. Changes in two excitation source features, the instantaneous fundamental frequency and strength of excitation, and in a vocal tract filter feature the dominant frequency, are examined for five steady vowel regions. Larger changes in these distinguishing features are observed in the automatically found vowel regions, than in word segments. This approach can help improving the systems for automatic detection of shout regions in continuous speech, and in paralinguistic applications that involve detection of prosody or emotions

@inproceedings{bib_Appl_2016, AUTHOR = {V, VISHNU VIDYADHARA RAJU and GANGAMOHAN, PAIDI and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Application of prosody modification for speech recognition in different emotion conditions}, BOOKTITLE = {IEEE Region 10 Conference}. YEAR = {2016}}

The main focus of this study is to analyze the performance of Automatic Speech Recognition (ASR) in different emotional environments using prosody modification. The majority of ASR systems are trained using neutral speech and the performance of such systems degrade when tested with the emotional speech. In this paper, the various components of speech that contribute to the emotion characteristics are studied. The prosody features of the source emotional utterancesare modified according to the target neutral utterances using Flexible Analysis Synthesis Tool (FAST). In the FAST, Dynamic Time Warping (DTW) is used to align the source emotional and target neutral utterances. Components of the prosody such as intonation, duration and excitation source are manipulated to incorporate the desired features into the source utterance. The modified (source emotional) utterances are then used for testing the ASR system which is trained using neutral speech. In this study, three emotions (compassion, happiness and anger) are considered for the analysis. Experimental results indicate an average improvement in the speech recognition system performance by considering prosody modified speech.

@inproceedings{bib_A_St_2016, AUTHOR = {ALLURI, KNRK RAJU and A, SIVANAND and Prasath, Rajendra and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {A Study on Text-Independent Speaker Recognition Systems in Emotional Conditions Using Different Pattern Recognition Models}, BOOKTITLE = {International Conference on Mining Intelligence and Knowledge Exploration}. YEAR = {2016}}

The present study focuses on the text-independent speaker recognition in emotional conditions. In this paper, both system and source features are considered to represent speaker specific information. At the model level, Gaussian Mixture Models (GMMs), Gaussian Mixture Model-Universal Background Model (GMM-UBM) and Deep Neural Networks (DNN) are explored. The experiments are performed using 3 emotional databases, i.e. German emotional speech database (EMO-DB), IITKGP-SESC: Hindi and IITKGP-SESC: Telugu databases. The emotions considered in the present study are neutral, anger, happy and sad. The results show that, the performance of a speaker recognition system trained with clean speech is degrading while testing with emotional data irrespective of feature used or model used to build the system. The best results are obtained for the score level fusion of system and source features based systems when speakers are modeled with DNNs.

@inproceedings{bib_Anal_2016, AUTHOR = {ALLURI, KNRK RAJU and V, VISHNU VIDYADHARA RAJU and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Analysis of source and system features for speaker recognition in emotional conditions}, BOOKTITLE = {IEEE Region 10 Conference}. YEAR = {2016}}

Source and system features are extensively used in building Speaker Recognition (SR) systems. In this paper, we investigate the influence of source and system features on the performance of the SR system in emotional conditions. The Linear Prediction Residual Cepstral Coefficients (LPRCC) which corresponds to source features, and Mel Frequency Cepstral Coefficients (MFCC) and Linear Prediction Cepstral Coefficients (LPCC) that correspond to system features are used for modeling SR system. A maximum-likelihood classifier based on Gaussian mixture density functions is used and experiments are carried out on 3 standard emotional speech databases (Indian Institute of Technology-Simulated Emotion speech corpus (IITKGP-SESC): Hindi, IITKGP-SESC: Telugu and German Emotional Speech Database (EMO-DB)). The speaker models are trained with neutral utterances and tested with 3 types (anger, happy and sad) of emotional speech utterances. Based on experimental results, the performance degradation was observed in mismatched (trained with one emotion and tested with other emotion) case and the average percentage of degradation in SR task using source features is approximately 16% more compared to system features. The performance degradation of SR system is nullified when trained and tested with the same emotion.

@inproceedings{bib_Star_2016, AUTHOR = {KRISHNA, VYDANA HARI and SRIVASTAVA, BRIJ MOHAN LAL and Srivastava, Manish and Vuppala, Anil Kumar }, TITLE = {Starting Small Learning Strategies for Speech Recognition}, BOOKTITLE = {India Council International Conference}. YEAR = {2016}}

Designing various learning strategies has been gaining a lot of scientific interest during the recent progress of deep learning methodologies. Curriculum learning is a learning strategy aimed at training the neural network model by presenting the samples in a specific meaningful order rather than randomly sampling the training examples from the data distribution. In this work, we have explored starting small paradigm of curriculum learning technique for speech recognition. The starting small paradigm of curriculum learning is performed by a two step learning strategy. Training dataset is re-organized as a set of easily classifiable examples followed by the actual training dataset and the model is trained on the re-organized dataset. We hypothesize that by following the starting small learning paradigm the learning gets initialized in a better way and progresses to attain a better convergence. We propose to rank the toughness of the training example based on the posterior probabilities obtained using a previously trained model. Apart from re-arranging the training corpus starting small paradigm of curriculum learning is applied at model level. We consider the broad manner-class classification objective function as the smoother version of the phone class classification objective function. The model initially trained for broad class classification is later adapted for phone classification.In this work, we have used TIMIT and a subset of WallStreet Journal (WSJ) corpus to validate the experiments, both the learning strategies have shown consistently better performances across the two datasets compared to the baseline system trained by randomly sampling the dataset.

@inproceedings{bib_Sign_2015, AUTHOR = {Vuppala, Anil Kumar and MOUNIKA, K V and KRISHNA, VYDANA HARI }, TITLE = {Significance of Speech Enhancement and Sonorant Regions of Speech for Robust Language Identification}, BOOKTITLE = {International Conference on Signal Processing, Informatics, Communication and Energy Systems}. YEAR = {2015}}

A high degree of robustness is a prerequisite to operate speech and language processing systems in practical environments. Performance of these systems is highly influenced by varying and mixed background environments. In this pa-per, we put forward a robust method for automatic language identification in various background environments. Combined temporal and spectral processing method is used as a preprocessing technique for enhancing the degraded speech. Language discriminative information in high sonority regions of speech is used for the task of language identification. Sonority regions are regions of speech whose signal energy is high and these regions are less influenced by background environments.Spectral energy of formants in the glottal closure regions is employed as an acoustic correlate for the detection of sonority regions of speech. In this paper performance of the LID system is studied in various background environments like clean room, car factory,high frequency,pink and white noise environments. In this work, Indian Institute of Technology Kharagpur - Multi LingualI ndian Language Speech Corpus (IITKGP-MLILSC) is used for building language identification system. Noise speech samples from the NOISEX database are employed in the present study.The performance of the proposed method is quite satisfactory compared to existing approaches.

@inproceedings{bib_Sign_2015, AUTHOR = {KUMAR, VUDDAGIRI RAVI and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar }, TITLE = {Significance of GMM-UBM based Modelling for Indian Language Identification}, BOOKTITLE = {Procedia Computer Science}. YEAR = {2015}}

Most of the Indian languages are originated from Devanagari, the script of the Sanskrit language. In-spite of similarity in phoneme sets, every language its own influence on the phonotactic constraints of speech in that language. A modelling technique that is capable of capturing the slightest variations imparted by the language is a prerequisite for developing a language identification system (LID). Use of Gaussian mixture modelling technique with a large number of mixture components demands a large training data for each language class, which is hard to collect and handle. In this work, phonotactic variations imparted by the different languages are modelled using Gaussian mixture modelling with a universal background model (GMM-UBM) technique.In GMM-UBM based modelling certain amount of data from all the language classes is pooled to develop a universal background model (UBM) and the model is adapted to each class. Spectral features (MFCC) are employed to represent the language specific phonotactic information of speech in different languages. During the present study, LID systems are developed using the speech samples from IITKGP-MLILSC. In this work, performance of the proposed GMM-UBM based LID system is compared with conventional GMM based LID system. An average improvement of 7–8% is observed due to the use of UBM-based modelling of developing a LID system.

@inproceedings{bib_Impr_2015, AUTHOR = {KRISHNA, VYDANA HARI and , P. Phani Kumar and Krishna, K. Sri Rama and Vuppala, Anil Kumar }, TITLE = {Improved Emotion Recognition using GMM-UBMs}, BOOKTITLE = {International Conference on Signal Processing and Communication Engineering Systems}. YEAR = {2015}}

In recent past a lot of scientific attention is paid on recognizing the emotional state of the speaker from his speech. Emotion recognition is a challenging task as human emotions are complex, subtle and emotive state in human speech does not persist long. So it is important to study the presence of emotion identifiable information in smaller segments of speech. This study is aimed at studying the presence of emotional specific information with relevance to the position of the word in the utterance. During the present study, spectral features are employed to represent emotion specific information in speech. Spectral features from smaller speech segments of speech based on their position in the utterance are employed to study the presence of emotion in speech. Due to the lack of adequate data in small speech segments to support conventional GMM during the course of present study Gaussian mixture modeling with a universal background model (GMM-UBM) is used for developing a emotion recognition system. Speech data from IITKGP-SESC is used during the course of the present study. During the present study 4 (Anger, Fear, Happy and Neutral) emotions are considered.

@inproceedings{bib_Dete_2015, AUTHOR = {KRISHNA, VYDANA HARI and Vikash, Peddakota and Vamsi, Tallam and Kumar, Kolla Pavan and Vuppala, Anil Kumar }, TITLE = {Detection of Emotionally Significant Regions of Speech For Emotion Recognition}, BOOKTITLE = {India Council International Conference}. YEAR = {2015}}

Emotions in human speech are short lived. In an emotive utterance, the emotive gestures produced due to the emotive state of the speaker persists only to a shorter duration. In this study, the regions of an utterance that are highly influenced by the emotive state of the speaker are detected. These regions are labeled as emotionally significant regions. Data from the detected emotionally significant regions is used for developing an emotion recognition system. Physiological constraints of human speech production system are explored for detecting the emotionally significant regions of an utterance. Spectral features from the detected emotionally significant regions are used to develop an emotion recognition system. A significant improvement in the performance of the emotion recognition system is observed inthe present approach. An average improvement of 11% is in noted owing to the use of data from emotionally significant regions while developing an emotion recognition system. Gaussian mixture modelling (GMM) technique is employed to develop an emotion recognition system. During the present study, speech samples from Berlin emotion speech database (EMO-DB) are used. Four basic emotions such as anger, happy, neutral and fear are considered for study

@inproceedings{bib_Anal_2015, AUTHOR = {DUMPALA, SRI HARSHA and ALLURI, KNRK RAJU and Gangashetty, Suryakanth and Vuppala, Anil Kumar }, TITLE = {Analysis of Constraints on Segmental DTW for the Task of Query-by-Example Spoken Term Detection}, BOOKTITLE = {India Council International Conference}. YEAR = {2015}}

Query-by-example spoken term detection (QbE-STD) refers to the task of determining the subsequence of a reference which matches with a query, where both the queryand the reference are in audio format. Dynamic time warping(DTW) based techniques are explored to match the two sequences with different lengths in an unsupervised manner. In this paper,a completely unsupervised approach based on Segmental DTW(SDTW), a variant of DTW, is considered for the task of QbE-STD where both reference and query utterances are represented using a sequence of Gaussian posteriorgram vectors. SDTW using two different types of bands i.e., Sakoe-Chiba band and Itakura parallelogram is considered to compare the Gaussian posterior-grams of the query and the reference sequence. The effect o fvarying different local constraints of the DTW algorithm on the performance of SDTW is also analyzed in this paper . Results obtained on MediaEval 2012 dataset indicate that SDTW using a band with variable speaking rate, as in Itakura parallelogram,performs better compared to that of using a band with fixed speaking rate, as in Sakoe-Chiba band, across all variations in local constraints.

@inproceedings{bib_A_LA_2015, AUTHOR = {SRIVASTAVA, BRIJ MOHAN LAL and KRISHNA, VYDANA HARI and Vuppala, Anil Kumar and Srivastava, Manish }, TITLE = {A LANGUAGE MODEL BASED APPROACH TOWARDS LARGE SCALE AND LIGHT WEIGHT LANGUAGE IDENTIFICATION SYSTEMS}, BOOKTITLE = {Technical Report}. YEAR = {2015}}

Multilingual spoken dialogue systems have gained prominence in the recent past necessitating the requirement for a front-end Language Identification (LID) system. Most of the existing LID systems rely on modeling the language discriminative information from low-level acoustic features. Due tothe variabilities of speech (speaker and emotional variabilities, etc.), large-scale LID systems developed using low-levelacoustic features suffer from a degradation in the performance. In this approach, we have attempted to model the higher level language discriminative phonotactic information for developing an LID system. In this paper, the input speech signal is tokenized to phone sequences by using a language independent phone recognizer. The language discriminative phonotactic information in the obtained phone sequences are modeled using statistical and recurrent neural network based language modeling approaches. As this approach, relies on higher level phonotactical information it is more robust to variabilities of speech. Proposed approach is computationally light weight, highly scalable and it can be used in complement with the existing LID systems

@inproceedings{bib_Vowe_2014, AUTHOR = {Vuppala, Anil Kumar and Rao, K. Sreenivasa }, TITLE = {Vowel Onset Point Detection for Noisy Speech using Spectral Energy at Formant Frequencies}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2014}}

Most of the existing vowel onset point (VOP) detection methods are developed for clean speech. In this chapter, we propose methods for detection of VOPs in the presence of speech coding and background noise conditions. VOP detection method for coded speech is based on the spectral energy between 500 and 2,500 Hz frequency band of the speech segments present in glottal closure region. In case of noisy speech, the proposed VOP detection method exploits the spectral energy at the formant locations of the speech segments present in glottal closure region. The proposed VOP detection methods are evaluated using objective measures and consonant vowel (CV) unit recognition experiments.

@inproceedings{bib_Auto_2014, AUTHOR = {Vuppala, Anil Kumar and Peri, Bhaskara Rao }, TITLE = {Automatic detection of breathy voiced vowels in Gujarati speech}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2014}}

This paper proposes a method for automatic detection of breathy voiced vowels in continuous Gujarati speech. As breathy voice is a specific phonetic feature predominantly present in Gujarati among Indian languages, it can be used for identifying Gujarati language. The objective of this paper is to differentiate breathy voiced vowels from modal voiced vowels based on loudness measure. Excitation source characteristics represented by loudness measure are used for differentiating the voice quality. In the proposed method, initially vowel regions in continuous speech are determined by using the knowledge of vowel onset points and epochs. Later, hypothesized vowel segments are classified by using loudness measure. Performance of the proposed method is evaluated on Gujarati speech utterances containing around 47 breathy and 192 modal vowels spoken by 5 male and 5 female speakers. Classification of vowels into breathy or modal voice is achieved with an accuracy of around 94 %.

@inproceedings{bib_Neut_2014, AUTHOR = {Vuppala, Anil Kumar and REDDY, KADIRI SUDARSANA }, TITLE = {Neutral to Anger Speech Conversion Using Non-Uniform Duration Modification}, BOOKTITLE = {International Conference on Industrial and Information Systems}. YEAR = {2014}}

In this paper, the non-uniform duration modification is exploited along with other prosody features for neutral speech to anger speech conversion. The non-uniform duration modification method modifies the durations of vowel and pause segments by different modification factors. Vowel segments are modified by factors based on their identities, and pause segments by uniform factors. Consonant and transition segments are not modified. These modification factors are derived from the analysis of neutral and anger speech. For this purpose, a well known Indian database named as the Indian Institute of Technology Kharagpur Simulated Emotion Speech Corpus (IITKGP-SESC) is chosen for analysis of emotions and synthesis of emotions from neutral speech. The prosodic features used in this study for emotion conversion are pitch contour, intensity contour, and duration contour. Subjective listening test results show that the effectiveness of perception of emotion is better in case of non-uniform duration modification than uniform duration modification

@inproceedings{bib_Neur_2012, AUTHOR = {·, Sreenivasa Rao Krothapalli and Yadav, Jaynath and Sarkar, Sourjya and Koolagudi, Shashidhar G and Vuppala, Anil Kumar }, TITLE = {Neural network based feature transformation for emotion independent speaker identification}, BOOKTITLE = {International Journal of Speech Technology}. YEAR = {2012}}

In this paper we are proposing neural network based feature transformation framework for developing emotion independent speaker identification system. Most of the present speaker recognition systems may not perform well during emotional environments. In real life, humans extensively express emotions during conversations for effectively conveying the messages. Therefore, in this work we propose the speaker recognition system, robust to variations in emotional moods of speakers. Neural network models are explored to transform the speaker specific spectral features from any specific emotion to neutral. In this work, we have considered eight emotions namely, Anger, Sad, Disgust, Fear, Happy, Neutral, Sarcastic and Surprise. The emotional databases developed in Hindi, Telugu and German are used in this work for analyzing the effect of proposed feature transformation on the performance of speaker identification system. In this work, spectral features are represented by mel-frequency cepstral coefficients, and speaker models are developed using Gaussian mixture models. Performance of the speaker identification system is analyzed with various feature mapping techniques. Results have demonstrated that the proposed neural network based feature transformation has improved the speaker identification performance by 20 %. Feature transformation at the syllable level has shown the better performance, compared to sentence level.

PhD Students (15)

MS Students (11)

Dual Degree Students (11)

Honours Students (6)