The real power of scientific research is realized through entrepreneurship that creates value driven innovations to make people’s lives easier!

Santosh Yadav (PhD), our next pathbreaker, AI Research Scientist at CeADAR (Ireland’s Centre for Applied AI), builds intelligent software for solving real-world and industrial challenges.

Santosh talks to Shyam Krishnamurthy from The Interview Portal about an intelligent system that he developed during his PhD for identifying human activities that were performed daily in order to predict the risk of fall, which he sold to a company in Tokyo.

For students, learn programming languages (at least one which is relevant) in-depth even if you are not from a computer science background. AI is going to be in almost every field, so make yourself aware of the applications of AI in your area of interest.

Santosh, tell us about your background?

I’m Dr. Santosh Kumar Yadav, born in a small village called Ganeshar in Block Jalalpur, District Ambedkar Nagar, Uttar Pradesh 224149, India. My childhood was marked by struggle as I lost my brother before I was born, and my father when I was only six months old. Raised by my mother, we lived with my uncles’ families, but none of us were financially stable. I studied in a government primary and nursery school in my village until the 8th grade while also working in the fields, caring for animals, and selling milk. I attended a government school in Jalalpur, where I studied science in high school and PCM (Physics, Chemistry, and Mathematics) in intermediate. Alongside my studies, I actively participated in sports such as cricket and kabaddi and excelled in Sanskrit and Mathematics. To aid my personal growth, I frequently read books on spirituality and self-improvement, which helped me stay focused on my goals.

What did you do for graduation/post-graduation?

After completing my intermediate (12th), I took a year-long break to prepare for the IIT JEE exams in Kanpur, Uttar Pradesh. During this time, I also attempted NDA, AIEEE, BITSAT, and UPTU exams but only managed to secure a good rank in UPTU. Subsequently, I did my BTech in Computer Science and Engineering (CSE) from University Institute of Engineering and Technology (UIET), CSJM University, Kanpur, which comes under Uttar Pradesh Technical University (UPTU). I studied here from 2011 to 2015.

I subsequently did my PhD in Computer Science (Multimodal Artificial Intelligence) from the Academy of Scientific and Innovative Research (AcSIR), at the CSIR-Central Electronics Engineering Research Institute (CEERI), Pilani, Rajasthan. I had one supervisor from CSIR-CEERI and one co-supervisor from BITS Pilani. I studied here from Aug 2016 to Aug 2021. I did a PhD without a Masters.

What made you choose AI as a career?

Since childhood, I have been fascinated by science and always dreamed of becoming a scientist. My BTech was my first introduction to computers, and it was not an easy task to learn programming languages and computer science subjects, due to their novelty. Financial constraints in my family meant that I had to tutor school students to earn fees and pocket money. Despite my best efforts, I struggled with many subjects and even ended up with a few backlogs. My BTech score was a mere 56%, but I firmly believe that one’s college score does not define him/her. 

In the final year of my BTech, I appeared for competitive exams such as GATE and CSIR-NET/JRF. Although I did not perform well in GATE, I secured a commendable rank of 107 in CSIR-NET/JRF. Prior to receiving the results, I moved to Bangalore to find a job, but my low percentage in BTech made it difficult to secure a good position. Eventually, I accepted a night shift job in an IT based mortgage processing company, but I resigned within six months as I did not like the job and my boss was very rude. I then began pursuing a PhD.

I was eligible for PhD admission despite not having a master’s degree, thanks to my good rank in CSIR-NET/JRF. After attending several interviews, I was selected for a PhD at CSIR-CECRI in Karaikudi, Tamilnadu, but the work there was not relevant to my area of expertise in computer science. Within six months, I transferred my PhD to CSIR-CEERI in Pilani, Rajasthan.

With the goal of becoming a scientist, I pursued a PhD in computer science. During my doctoral studies, I was particularly drawn to the field of artificial intelligence (AI), which I believe holds tremendous potential for making a positive impact on people’s lives. Here, I worked on Multimodal Human Activity Recognition, a cutting-edge topic in computer science. After completing my PhD, I actively pursued positions in the AI field, networking with professionals and attending industry events to stay up to date with the latest developments. I worked as a Postdoctoral Scientist to do in-depth research in AI. Recently, I received an offer to join CeADAR (Ireland’s Centre for Applied AI) in Dublin as an AI research scientist.

What were some of the key influences and major turning points that influenced your career path?

Throughout my journey, I have been fortunate to have the support and guidance of some incredible teachers and professors who have played pivotal roles in shaping my academic and professional career. I am grateful for their mentorship and guidance, and I would like to acknowledge some of their contributions.

Prof. Dr. Kamlesh Tiwari (BITS Pilani) has been an outstanding motivator in my life. He was my PhD Co-supervisor and helped me tremendously in developing research skills, guiding me through the research process, and providing valuable feedback on my work.

Dr. Jagdish Lal Raheja (CSIR-CEERI, Pilani) also deserves special mention for helping me transfer during the initial days of my PhD. Without his support, I would not have been able to pursue research in my area of interest.

Dr. SA Akbar (CSIR-CEERI, Pilani) was my PhD supervisor and played a critical role in shaping my research interests and providing me with valuable feedback and insights throughout my PhD journey.

Prof. Dr. Hari Mohan Pandey (Bournemouth University, UK) is another mentor who provided me with tremendous guidance and support in my research work. His expertise and knowledge in my field of interest were invaluable to me.

Mr. Mohit Mathur (Texas, USA) taught me essential lessons in entrepreneurship and helped me develop a business mindset.

Prof. Peter Corcoran (NUI Galway, Ireland) was my Postdoc supervisor, and I gained a great deal of inspiration from him. His expertise and guidance were instrumental in my research work during my time as a Postdoctoral Scientist.

Throughout my life, my mother has been my constant source of inspiration and guidance. From a young age, she instilled in me the importance of education and hard work. Despite facing numerous challenges in her own life, she always put my needs before her own and made sure that I had access to the best education possible. Her unrelenting belief in my abilities gave me the confidence to take on new challenges and strive for excellence in everything I do. I hope to continue to make her proud by achieving my goals and making a positive impact on the world, just as she has done throughout her life.

There have been several significant turning points in my life, both emotionally and professionally.

  • One of the most profound moments was during my BTech when I was struggling with financial hardships and poor academic performance. At one point, I was so broke that I couldn’t afford food for over a week and was too anxious and depressed to sit for my semester exams. One day, I went for a long walk by the river Ganga, and I sat there for nearly 13-14 hours contemplating my life’s purpose. I questioned why I had to suffer so much, losing my father and brother at an early age, being born into poverty, and what the meaning of my existence was. Eventually, I came to the realization that I had nothing to lose and everything to gain. From that day, I dedicated myself to helping others, to relieve their pain and struggles, and to make a positive impact on as many people’s lives as possible.
  • Another pivotal moment was when I started my PhD research work at CSIR-CEERI, Pilani. Instead of solely focusing on theoretical research, I decided to work on real-life problems and conduct applied research. During this time, I also began developing products with practical applications, with the aim of eventually building a startup. This decision was instrumental in shaping my future career path and instilling in me a passion for creating solutions that have a tangible impact on people’s lives. In my experience, the decisions we make at different stages of life have a significant impact on our destiny.

Tell us about your PhD research in the field of AI

I collaborated with my PhD supervisors on various machine learning and deep learning techniques such as 3D-CNNs, Pose Estimation Libraries, Single and Multi-Stream Networks, Transformer Networks, Temporal Attention Mechanisms, etc., and explored their applications in areas such as Yoga pose recognition, Group activity recognition, 3D human body modeling, Karate action recognition, Fall detection, Drone camera-based action recognition, Wi-Fi channel state information (CSI) based action recognition, Generative models, etc. During my PhD, I also collaborated with renowned professors from prestigious universities around the world, resulting in 13 high-impact publications in journals and international conferences, with over 200 citations. These can be accessed on my website: https://www.dr-santosh-yadav.com/home. 

During my final year of PhD, I developed an intelligent software for identifying human activities that were performed daily and predicting the risk of falls, which also analyzed gait scores and TUG tests for patients with walking anomalies. I sold this software to a company based in Tokyo, Japan for a good amount. 

Additionally, I secured a Postdoctoral Scientist position in Ireland, working on a project in collaboration with XPERI, SoapBox, and the National University of Ireland Galway. As a Postdoctoral Scientist, I led the development of a low-power, cost-effective AI processing platform that incorporated both sound and vision inputs and gained expertise in 3D computer vision and conversational AI. In addition, I am also working on my startup CogniX, which focuses on computer vision-based solutions for health, fitness, and sports using AI technology. The prospect of creating new job opportunities, introducing new products, and contributing to societal growth, while also fulfilling my responsibilities as a taxpayer, serves as a great motivation for my entrepreneurial journey. I am driven by the desire to constantly improve and grow, and to make a positive impact in the world through my work.

How did you get your first break?

After completing my BTech degree, I faced many challenges while searching for a job. Despite my efforts, it took me three months to find a job in Bangalore, and even then, it was not the job I had envisioned for myself. I had to work as a Process Associate in a US-based IT company for mortgage processing. However, I realized that I had to start somewhere to survive, and I took this job to gain experience and support myself until I could pursue my passion for research.

During the final year of my PhD, I built an intelligent software that could recognize daily living activities and predict the chances of a fall (through fall detection) using an android phone. It caught the attention of a company based in Tokyo, Japan. They were impressed with my work and paid me approx $38,000 to own that software and to assist their team in building a software to automatically identify symptoms of neurodegenerative diseases. Based on the software I helped create, the company raised ~2 million dollars in funding. This experience sparked my interest in entrepreneurship and motivated me to pursue more innovative projects.

After completing my PhD, I joined a Postdoctoral Scientist position in Ireland, where I gained valuable experience in my field. Recently, I joined a company as an AI Research Scientist in Dublin, where I am working on innovative projects to advance the field of AI. I am excited to use my skills and knowledge to make a positive impact on society and improve people’s lives through my work.

What were some of the challenges you faced? How did you address them?

Financial: 

Since childhood, I faced financial struggles with my mother as we lacked any source of income. I attended government schools and colleges, where fees were either minimal or waived due to being below the poverty line. To earn pocket money, I tutored school students.

Emotional: 

Losing my brother and father at a young age left a deep emotional scar that took time to heal. For a long time, I felt alone and helpless, and it was hard to see the purpose in my life. But gradually, I started to see that my own struggles gave me empathy and understanding for others who were going through tough times. I began volunteering, helping with after-school programs, and offering my skills in tutoring and mentoring. It was incredibly rewarding to see the difference I could make in someone else’s life, and it helped me to feel more connected to the world around me. Now, I try to live my life with the perspective that we all have our own challenges and hardships, and that it’s important to be there for one another. Whether it’s through volunteering, donating to charity, or simply being kind to others, I believe that we can all make a difference in someone’s life.

Career:

Career and financial stability are very important to me, after my family. I believe that having a good job is crucial for achieving this stability in life. I am currently working as an AI research scientist, which allows me to use my skills and knowledge to contribute to cutting-edge research and innovation. My long term goal is to establish my own AI and technology company that creates innovative solutions for real-world problems, positively impacting people’s lives. To achieve this, I continuously learn, network, research opportunities, develop a strong business plan, and seek investors/partners. I’m passionate about tech’s transformative power and excited to pursue my entrepreneurial dream while making a positive impact on society.

Where do you work now? 

I was working with the National University of Ireland Galway as a Postdoctoral Scientist for the last 1.5 years. Recently, I joined CeADAR (Ireland’s Centre for Applied AI) as an AI Research Scientist in Dublin.

What problems do you solve?

I build intelligent software for solving real-life problems. I have worked on various applications like Yoga pose recognition, Group activity recognition, 3D human body modeling, Karate action recognition, Fall detection & prediction, Neurodegenerative disease symptoms identification, Drone camera-based action recognition, Wi-Fi channel state information (CSI) based action recognition, conversational AI robots, Child face generation, etc.

What skills are needed for your role? How did you acquire the skills?

To become an AI research scientist, one needs strong technical and programming skills such as:

  • Computer science: Algorithms, data structures, computer architecture, databases, etc.
  • Mathematics: Linear algebra, calculus, probability theory, statistics, etc.
  • Programming: Python, C++, Java, R, etc.
  • Machine Learning & Deep Learning: Supervised and unsupervised learning, reinforcement learning, neural networks, etc.
  • Natural Language Processing: Text processing, sentiment analysis, machine translation, etc.
  • Computer Vision: Image processing, feature extraction, object detection, segmentation, etc.
  • Research Skills: Developing new algorithms, conducting experiments, publishing research papers, etc.
  • How to Acquire these skills: Online courses, attend classes at a university, work on research projects, read books, etc.

What’s a typical day like?

Being an AI researcher, I enjoy doing different things such as:

  • Reading research articles to keep myself updated with the latest research and developments.
  • Designing and implementing research experiments.
  • Understanding and analyzing the pattern in the data.
  • Collaborating with the team, writing research reports/articles, doing presentations.
  • In addition to that I make sure to keep the work life balance.

What is it you love about this job?

I love to solve the challenges, proposing innovative solutions, and making an impact with my research works.

How does your work benefit society? 

We make intelligent systems that solve real-life problems which are very useful for society. To know in detail please see https://www.dr-santosh-yadav.com/home. A few examples are:

  • Multimodal Conversational Artificial Intelligence: We made a conversational AI robot that can understand human behavior and talk to them.
  • Human Action Recognition in the Dark: In nighttime the cameras are almost blind while most of the crimes happen in the night or dark. We made intelligent software by which cameras can also see in the night.
  • WiFi based Human Activity Recognition: We proposed a system to count humans, track locations, and recognize their activities using WiFi signals.
  • Drone Camera Based Human Activity Recognition: We proposed a system by which drones can fly over a location and can count, track, and recognize people’s activities.
  • Video camera based human action recognition: We proposed camera-based people detection, pose estimation, tracking, activity recognition.
  • Multimodal Human Activity Recognition: Using this, the software can recognize activities using camera, wearable sensors, and/or Wi-Fi sensors.
  • We also proposed Yoga action recognition, posture correction, and Karate action recognition and evaluation systems.

Tell us an example of a specific memorable work you did that is very close to you!

During the early stages of my PhD program, I was looking for a unique research topic that could combine my interests in AI and yoga. As someone who regularly practices yoga and meditation, I came up with the idea of creating a system that could analyze yoga asanas and provide feedback on whether they were being performed correctly. I spent a considerable amount of time researching and developing this system, and I was excited to share my findings with others. In May 2019, I published my research and made it available to the public. Since then, my work has been widely recognized and has received thousands of reads and over 85 citations. I’m proud to say that I was the first person to combine the fields of yoga and AI in this way. It was a challenging task, but I’m glad I pursued it. Currently, I am building realistic 3D human virtual avatars and AI-assisted platforms for physical training and corporate collaborations, aiming to revolutionize the way people learn and work.

Your advice to students based on your experience?

  • Life is full of challenges, but it’s important to remember that these struggles can make you stronger and wiser. Reading spiritual and self-growth books can help you build emotional resilience and a deeper understanding of the world. When setting your goals, always aim high and don’t be afraid to fail. It’s important to embrace failure as a temporary setback and focus on the lessons you can learn from it. Stay humble and grounded, and never forget that you have the power to make a positive impact on the world. Take action every day to work towards your goals and make a difference in the lives of others. Remember that the journey is just as important as the destination, so enjoy the ride and keep learning and growing along the way.
  • Build a strong foundation on those subjects which interests you the most. Learn programming languages (at least one which is relevant) in depth even if you are not from a computer science background. AI is going to be in almost every field, so make yourself aware of the applications of AI in your area of interest. If you get a chance, pursue higher education to develop necessary skills and knowledge in your area of interest. Always stay curious and passionate about learning new things.

Future Plans?

I am more interested in scientific innovations and bringing these innovations to society. I always aim to solve problems and make people’s lives easier using the power of technology and innovations. In the future I see myself as a scientific entrepreneur. However, I am flexible with my plans.