Podcast Link : Computational Linguist Podcast
In a multi-cultural society like India, language is the most important tool that can bridge the gap between technology and humans !
Arpanjyoti Gogoi, our next pathbreaker, leverages Machine Learning approaches on large language models (LLMs) to build natural language tools that generate content which is more inclusive and appropriate.
Arpanjyoti talks to Shyam Krishnamurthy from The Interview Portal about one of his internships, where he contributed to computational linguistics research for Assamese language in order to build a speech corpus, which deepened his interests in NLP and machine learning.
For students, language technologies are empowering our lives, through AI Agents that speak to us in our language and execute our tasks !
Arpanjyoti, Your background?
I grew up in a small tea estate in Assam, surrounded by the lush greenery of tea gardens where my father worked. Our home was right next to the estate, and my earliest adventures often began in our backyard. When I was about four, I’d join my father to clean the banana trees. One morning, we discovered a monkey had taken up residence overnight and eaten on half the bananas. As I stood there, processing the scene, I heard a peculiar sound—a loud call with a unique rhythm of consonants and vowels, unlike anything I’d heard before. My little brain buzzed with curiosity: “What was that? What did it mean?”
This was my first encounter with a minority language—a tea garden community language, a fascinating blend of Sadri and other Dravidian tongues, born from a diverse group of speakers. Even then, I was struck by how language could hold such mystery, reflecting the culture and identity of its speakers.
As I grew older, I developed a love for designing, storytelling, exploring cultures, and even creating imaginary languages and games. At one point, I dreamed of studying animation to bring these ideas to life, but then Linguistics happened. During my undergraduate studies in English, I discovered the magic of English linguistics, and soon, the entire world of speech patterns—both in text and audio—had me hooked. That passion led me to pursue a Master’s in Linguistics, where I spent countless hours in the university’s language lab, analyzing the acoustic features of languages.
It felt like all my childhood passions had converged in one place. Two projects, in particular, brought my early fascinations full circle. First, I designed a Hindi chatbot using Dialogflow, combining my love for building with my interest in conversational patterns. Then came my Indiana Jones Moment— a phonological documentation project in a village in Meghalaya for my master’s thesis. I immersed myself in the culture, analyzed the linguistic patterns of an undocumented language, and pieced together its unique story.
Later, during an internship, I contributed to computational linguistics research for Assamese, building a speech corpus that further deepened my dive into NLP and machine learning. It was here that I realized my love for languages, storytelling, and technology could come together in ways that felt both exciting and deeply personal.
What did you do for graduation/post graduation?
I did my BA (English) from Dibrugarh University.
I studied a Master’s in Linguistics program from The English and Foreign Languages University. During my PG program, my focus has been on developing understanding of speech processing and understanding of natural language.
What were some of the key influences that led you to such an offbeat, unconventional, and uncommon career in Linguistics?
During my Master’s program, I had the opportunity to intern at the Central Institute of Indian Languages (CIIL) as a resource person for developing language corpora for machine learning.
Language corpora are large collections of machine-readable texts that are used in a variety of applications such as speech processing, language teaching etc
A language corpus is a collection of authentic linguistic data, such as written texts or recorded speech, that is used for research, teaching, and scholarship. Corpora can be used to: Understand the meaning of words, Teach common patterns before rare ones, Use authentic examples of patterns, and verify hypotheses about a language.
This experience was a turning point for me—it was where I first ventured into the fascinating world of Computational Linguistics and NLP research. Working on building resources for language technologies opened my eyes to the vast possibilities of combining linguistic expertise with machine learning, sparking a new passion that would shape my future endeavors.
How did you plan the steps to get into the career you wanted? Tell us about your career path
As answered above, I also started learning about Deep Learning and different Linguistics of Indian languages. As a prerequisite, I practiced python for NLP tasks.
After completing my master’s degree, I intended to continue in academia but needed to secure a job. While searching online, I discovered Lionbridge, a company that provides outsourcing tasks. After creating a profile on their portal, a representative from Lionbridge, Finland contacted me about collaborating on a machine learning project to build Assamese language data. This led to multiple NLP projects related to the Assamese language, and I worked remotely with a team in Tampere as a freelancer. This was my first experience in the industry.
Subsequently, an opportunity at Cerence presented itself. Cerence is an AI company in the automotive space, where I contributed to building speech output systems for multiple languages, including French, Japanese, Spanish, and many more, for their conversational bots.
How did you get your first break?
After my graduation, I joined Cerence India, in their NLP TTS team as Speech Expert, developing conversation bots responses for multiple languages, including French, Mandarin, Spanish etc. and these are voice responses for those languages in a particular domain.
As I mentioned earlier, I created a profile on the Lionbridge platform and was later contacted by one of their team members for a machine learning project. This was followed by a couple more freelance projects. This initial industry experience in language technology provided me with the right experience needed at Cerence for the role, where I contributed to the development of similar language technology.
What were some of the challenges you faced? How did you address them?
While working on developing language technology, one of the most persistent challenges I’ve encountered is evaluating the quality of the generated content. Issues often arise in areas like grammar and syntax, and in the context of Generative AI, the challenges extend to managing hallucinations (false or fabricated information) and maintaining coherence in the output.
The evaluation process becomes particularly tricky because the generated data is unstructured, lacking a predefined format. Addressing this requires conducting multiple experiments to contextualize the outputs and assess their quality effectively. It’s a balancing act, as the evaluation process must blend both qualitative and quantitative methods.
Qualitative analysis provides insights into nuances like tone, relevance, and coherence, while quantitative metrics offer a structured way to measure aspects such as accuracy and consistency. This dual approach is critical for refining the system and ensuring that the generated language meets the desired standards.
Above all, the most challenging part, I feel, was being the first in my entire family generation to have the opportunity to complete school and pursue higher education. This did not come easily; there were financial constraints and significant risks to navigate. I didn’t have any family members to guide or even advise me in choosing a career path. However, I am fortunate that my college and university helped me select the right courses and subjects. I also sought advice and guidance from my social networks.
Where do you work now? What problems do you solve?
Currently, I work on Generative AI, and use multiple State of the art LLMs with prompt engineering techniques to achieve most of my tasks.
A large language model (LLM) is a type of artificial intelligence (AI) that uses machine learning to process natural language
I am conducting R&D in Generative AI using large language models (LLMs) to build natural language tools that generate content which is more inclusive and appropriate. Additionally, my work contributes to creating a robust ML data workflow by using generative AI to produce synthetic data. This approach helps reduce dependency on real data and decreases the time and effort required to create labeled datasets.
What are the skills required in your role?
Just like in any other role, important skills needed include being analytical and having the presence of mind to make actionable decisions; however, the most important is adaptability to acquire new skills. Some of the hard skills I have acquired for these roles are scripting in Python, machine learning, and currently, prompt engineering and LLM applications.
How does your work benefit society?
I see that language is the bridge that connects people and community. Most of the technologies that we build have been helpful towards easing user’s life, be it an AI Agent that speaks to you and executes your task or an AI powered tool that can go through payment bills and make an organized report.
Tell us an example of a specific memorable work you did that is very close to you!
During my early career, I contributed as a freelancer in the development of Assamese language resources, and being an L1 speaker of a low resource language, I am able to contribute a lot in development of it.
Your advice to students based on your experience?
Something that I wished I would have known is, focus your skills on upgrading via a ‘T’ learning style that covers all areas, see and explore what fits you and then sticking to one small area by trying to be a master of it.
Future Plans?
I do not have any concrete future plans but to learn and grow in the community, and be able to contribute to the society.