Robots need to be able to evolve into highly sophisticated machines that can interact with their surroundings, especially in unstructured human-inhabited environments.
Arunabh Mishra, our next pathbreaker, Software Engineer Perception at Zoox (California), is part of a team that focuses on processing data from sensors like cameras and LIDARs to create representations that the robot uses for planning and action in the real world.
Arunabh talks to Shyam Krishnamurthy from The Interview Portal about building a harvesting robot for his undergrad thesis, capable of climbing tall arecanut trees and controlled remotely with a PlayStation controller, which shaped his career in Robotics !
For students, in a world that’s constantly evolving, staying curious and open to new knowledge is essential. It keeps you relevant and enriches both your professional and personal life.
Arunabh, what were your growing up years like?
I grew up in Jaipur, a city that shaped much of my early life. Tragically, I lost my father at the age of five, but my mother and elder sisters filled my life with warmth and love, helping me navigate through those early years. As an introverted child, I often turned to video games for comfort. My schooling was mostly in Jaipur, with a three-year stint in Kota preparing for the JEE, although I didn’t make it in. I ended up pursuing my undergrad studies in Manipal. After graduation, I spent a year back in Jaipur, volunteering and gaining some experience working in a friend’s startup. These experiences were very important to me as they helped me develop empathy and made me realize what I really wanted to focus on. Equipped with these insights, I embarked on my journey to the US for grad school. Since then, I’ve lived in five different states and now call Boston my home.
What did you do for graduation/post graduation?
I did my undergrad in Mechatronics Engineering at MIT, Manipal, and was a pretty average student. I had my likes and dislikes regarding the subjects, but it all came together in my final year. That’s when me and a couple of my friends built a harvesting robot for our thesis, capable of climbing tall arecanut trees. It was controlled remotely with a PlayStation controller and designed to climb and harvest using a cutter. In a bit of a comical twist, we broke the robot just before filming our final video. Resourcefully, we filmed it descending the tree, used video editing to reverse the footage, and submitted it as our final presentation.
I moved to the US in 2016 to pursue my Master’s in Robotics from the Colorado School of Mines. CSM was a relatively small school with a really tough program. I remember feeling super out of place for the first few months. But as time passed, I found my rhythm and started to really enjoy learning. I was particularly keen on coding, so I tailored my coursework to bolster these skills, ultimately graduating with a minor in Computer Science in addition to my Master’s. I was also fortunate to be accepted as a research assistant in the robotics lab at CSM. The lab became my second home, as I think I spent more time there than at my actual home.
Here’s a picture of me working on one of the robots in the lab.
What were some of the drivers that influenced you to pursue such an offbeat, unconventional and uncommon career?
Coming from a family with two generations of IPS officers, there was an unspoken expectation for me to uphold the legacy. However, this very expectation subtly pushed me away from following in their footsteps. Although I knew what I did not want to do, I didn’t really know what I wanted to pursue growing up. My journey, I believe, mirrors the principles of mimetic theory — the idea that our desires are imitations of what others want. I realized that many of my early ambitions were more influenced by my surroundings than by my own true passions.
Discovering robotics was a serendipitous event, but choosing to focus on autonomous driving technology was intentional. Before I joined my master’s program, I lost someone close to me in a completely avoidable car accident. This tragic event profoundly influenced my decision to specialize in autonomous vehicle technology, aiming to prevent such accidents in the future. It serves as a powerful reminder of the potential impact and importance of my work in robotics to this day.
How did you plan the steps to get into the career you wanted? Or how did you make a transition to a new career? Tell us about your career path
Grad school at CSM was a pivotal time where my career began to gain momentum. My research there involved programming a Clearpath Husky, a widely-used research robot, for autonomous navigation in GPS-denied environments using advanced optimal controllers. It was during this phase that I started to play with Lidars and cameras, gaining a deeper insight into the full stack necessary for autonomy, encompassing mapping, localization, perception, prediction, and planning.
After grad school, I joined DEKA as one of the initial engineers on a project developing autonomous robots for FedEx’s last-mile delivery. While DEKA is primarily recognized for its medical devices, its venture into robotics projects provided me with a unique opportunity. Here, I transitioned from planning to specializing in robot perception using machine learning. The small, agile team at DEKA was the perfect setting for broadening my skill set, allowing me to make substantial contributions to the perception stack. I developed expertise in ML tools like pytorch, TensorFlow and Tensorrt along with GPU programming languages like CUDA and OpenCL which paved the way to my dream job.
Looking back, my journey at DEKA was more than just a job; it was a crucial stepping stone that enriched my professional experience. The diversity of tasks and challenges I faced there played a key role in shaping my skills, preparing me for the future. I also made some great friends that I am in touch with till today.
Here’s a picture of me posing with our fleet.
Moving to Zoox, a leader in autonomous driving technology, was the realization of a long-held dream. At Zoox, I am fully immersed in computer vision research, continuously applying and expanding upon the knowledge and skills I’ve gathered throughout my academic and professional journey.
How did you get your first break?
Honestly, landing your first job out of grad school in the U.S. can be challenging. Career fairs are a viable option, though they often feature local companies that might not align with your desired role. My successful strategy involved applying to a wide array of job openings. Additionally, it’s beneficial to start your job search before completing your final semester, engaging in relevant research projects or interning during the summer. I recall dedicating about 10 hours each day for a month to practicing coding problems on LeetCode, an online platform designed to enhance coding skills and prepare for technical interviews. In essence, the more roles you apply for, the greater your chances of securing a job. It’s also crucial not to be overly selective about your first role. Having some experience can significantly ease the transition to more desirable opportunities later on.
What were some challenges you faced? How did you address them?
Challenges are important catalysts for fostering learning and growth. Here are some that come to mind from my journey:
Transitioning domains from my grad school research in optimal control to working on machine learning (ML) at DEKA was a significant early hurdle. My initial knowledge of ML was limited, necessitating significant effort to bridge the gap. I tackled this by dedicating extra hours after work, taking online courses in my own time, and asking for help from my peers who had a strong foundation in ML.
Another challenge I faced was during the rapid expansion of our team at DEKA. We went from a handful of engineers to about 150 which led to a shift from an individual contributor to a managerial role. Realizing my passion lay more in hands-on coding than in management, I strategically returned to a more technical role at Zoox.
Another experience that remains vivid in my memory is when I joined Zoox. Feeling out of place among my incredibly talented colleagues, I encountered a typical instance of imposter syndrome.To combat this, I focused on persevering and working even harder, aiming for and achieving small wins. These victories gradually built up my confidence, eventually providing me with the assurance needed to overcome the imposter syndrome. This issue is super common when you switch roles, and it’s important to recognize it as part of the professional growth process, not as a sign of incompetence.
Where do you work now? What problems do you solve?
I’m part of the Perception team at Zoox. Our team acts as the eyes of the robot, focusing on processing data from sensors like cameras and LIDARs to create representations that the robot uses for planning and action in the real world. These representations are typically machine-learned to generalize across different domains. My specific role involves building representations for scene semantics and agent intents, requiring a solid foundation in ML and software engineering. This includes proficiency in languages like C++, Python, and CUDA, as well as a deep understanding of the latest in computer vision research. I have picked up these skills along my journey through self-teaching, my time in grad school and my time at DEKA.
What’s a typical day like? What is it you love about this job?
My day starts pretty early. At 6 am, I take my dog for a run, followed by a 4-mile bike ride to work. Once I arrive at the office, I grab a coffee and begin a three-hour deep focus session to tackle the majority of the day’s tasks. As my West Coast colleagues log on, my schedule shifts to collaborative meetings, filled with engaging discussions and brainstorming. I head back home around 6 pm and spend my evenings with my partner.
What I treasure most about this job is the perfect mix of intellectual challenge and creativity. We have regular discussions about the latest academic papers, and the collaborative spirit of working and learning alongside my brilliant colleagues is unparalleled. It’s a constant reminder of the wondrous outcomes possible when technology is infused with imagination. Here’s my selfie with one of our beautiful robots.
How does your work benefit society?
If the industry is successful in deploying this technology at scale, transportation will be safer, more efficient, and accessible to everyone. By significantly reducing traffic accidents, often caused by human error, autonomous vehicles can save countless lives associated with road accidents. Moreover, this technology empowers those who currently face mobility challenges, such as the elderly and individuals with disabilities, by providing them with greater independence and social inclusion. The ripple effects of our work also extend to environmental benefits, like decreased traffic congestion and lower carbon emissions, through optimizing driving patterns and potentially reducing reliance on personal vehicle ownership. This vision isn’t just about advancing technology; it’s about creating a safer, more connected, and equitable world, where transportation barriers are a thing of the past and everyone has the freedom to move effortlessly and safely.
Tell us an example of a specific memorable work you did that is very close to you!
A particularly memorable instance for me was during the development of an ML model for encoding scene semantics. While the initial experimental results were quite promising, showing notable improvements across various metrics, these numbers only offered a partial view of the model’s potential. The real moment of awe occurred when I first saw the robot flawlessly perform a complex maneuver in the real world, a maneuver that was a direct outcome of the refinements I had made to the model. It was magical witnessing the robot execute a complex task in full autonomy, a task to which I had directly contributed. Moments like these are profoundly gratifying, underscoring the tangible impact and profound significance of our work.
Your advice to students based on your experience?
Firstly, embrace the ethos of continuous learning. In a world that’s constantly evolving, especially in fields like mine, staying curious and open to new knowledge is essential. It keeps you relevant and enriches both your professional and personal life. Whether it’s through formal education, online courses, or self-experimentation, never stop expanding your horizons.
Secondly, it’s crucial to optimize for happiness, but within a timeframe that aligns with your individual goals and life plan. Everyone’s journey is unique, with their own timelines and paths. This means making choices in your career and life that might not provide immediate gratification but will lead to long-term fulfillment. Recognize and honor your personal journey, understanding that what suits one person may not be right for another. Make choices true to your aspirations, remembering that enduring happiness often stems from pursuing goals that resonate with who you are.
Lastly, a quote by Naval Ravikant offers profound wisdom: ‘What seems like play to you but feels like work to others?’ This underscores the importance of finding your passion. When your work aligns with what you naturally enjoy, it doesn’t feel burdensome. Identify what excites you, what you can immerse yourself in for hours without feeling it as ‘work.’ When your profession mirrors your passion, success and satisfaction tend to follow naturally. Ultimately, it’s about creating a journey that resonates with your true self and what you love to do.
Future Plans?
I’m currently dedicated to a challenging yet immensely fulfilling personal goal: training to run a marathon. This journey demands dedication and perseverance, and I am fully committed to seeing it through. Alongside this, I’ve grown increasingly aware of the importance of family connections, particularly with my mom. Being far from her has reinforced my resolve to make more efforts to stay closely connected, cherishing these invaluable moments. Additionally, I am excited about fulfilling a long-held dream this year – experiencing the mesmerizing beauty of the northern lights.