Applying Computer Vision to address some of the most pressing real world problems like road safety still seems far fetched, due to the low margin for error in safety critical applications. But those are the challenges that inspire us !

Saptarshi Roy, our next pathbreaker, Lead Research Engineer at Hyundai MOBIS, manages the camera section in the ADAS (Advanced Driver Assistance Systems) department, working towards the larger goal of making roads safer for everyone.

Saptarshi talks to Shyam Krishnamurthy from The Interview Portal about being drawn to the field of Image Recognition and subsequently applying different algorithms to solve real problems in the automotive sector that solidified his goal to make a career in the field of automotive computer vision.

For students, nothing gets as personal as road accidents, because each one of us has been affected by accidents in one way on another. Bring safer technologies to Indian roads.

Saptarshi, tell us about Your background?

I was born and brought up in Kolkata where I stayed until my high school days. I come from a family of academicians. Both my father and grandfather were professors of physics and I grew up with an expectation that I should be contributing to science in some way. So, the idea of research was there from the very beginning.

My father also happened to be an amateur photographer in his college days. I discovered his boxes full of black and white negatives and photo prints when I was a teenager. They always used to fascinate me. I also started photography with his old OM-1 film camera in my college days and gradually shifted to digital later. In addition to photography, I also loved sketching and painting. Overall, the visual art forms always attracted me. 

Luckily, I am now working in the domain of computer vision that allows me to do research and deal with images at the same time. Maybe my early interest has helped me find my current career interest.

What did you do for graduation/ post graduation?

I did my BTech in ECE from Kalyani Government Engineering College and MS (by research) from IIT Kharagpur in Fault Tolerant Automotive Systems.

I secured an engineering entrance rank which was good enough to get me admitted into one of the few government colleges in my state. I did B.Tech in electronics and communication engineering from the Kalyani Government Engineering College. During my B. Tech, I did a project on hardware acceleration of basic image processing filters from Indian Statistical Institute, Kolkata. That was the first time I was exposed to serious problems in image processing, and I had to go through some technical papers to complete my thesis. 

Though I got a job from campus placements, I decided to go for higher studies. But I could not get into M.Tech in the reputed IITs with my GATE rank. At that time, a collaborative project between General Motors India Science Lab and IIT Kharagpur was going on. The project was a multi-disciplinary project having scope across many departments. They were recruiting for the post of research consultant and I joined the project after going through the written test and interview process. The scope of work in the electrical department, where I joined, was fault tolerance of automotive controllers (ECUs). After a few months, I enrolled for the MS by research program in the department of electrical engineering. The MS by research program is an interesting program particularly for those who are inclined towards research. The stipend for the MS program comes from the industrial project with which the student is attached and is generally higher than that of MTech. 

Though I was enrolled in the electrical department, my co-supervisor was from the computer science department and I had the opportunity to closely observe the professors and researchers from both departments. I observed how the research approaches of these two disciplines differed and how they complemented each other successfully. This helped me later in executing collaborative projects in the industry where multiple parties with different backgrounds work for a common project goal. 

My final thesis was on the design of a fault-tolerant automotive networked architecture where some of the real-time control tasks could work in a gracefully degraded fashion in case of an ECU failure (on the contrary to sudden and complete failures that generally occurs in electronic components) by optimizing the available computing resource through smart scheduling.

What made you choose such an offbeat, unconventional and unique career?

Towards the end of my MS journey, the General Motors India Science Lab suddenly stopped its operations. Generally, there is a good chance of getting placed in the companies where you do an internship. But that opportunity was gone and my dreams were shattered. I was briefly trying to get into other companies where I could contribute with my experience. At that point, I got a direct call from Renault Nissan Alliance in Chennai to join their R&D team. I was primarily recruited for Nissan projects (though I briefly worked for Renault projects also) and they wanted me to go to Nissan, Japan for engineering training for six months. I saw this as a good learning opportunity and accepted the offer.

How did you plan the steps to get into the career you wanted? Or how did you make a transition to a new career? Tell us about your career path

While working for Renault Nissan, I was exposed to various sides of the automotive industry. Due to my background, I was associated with autonomous driving and advanced driver assistance systems. As an individual contributor, I worked on a variety of projects starting from vehicle suspension system control, high accuracy localization by image processing, HMI system design for semi-autonomous vehicle to statistical driving data analysis for predicting bad drivers.

Apart from core algorithm development, I had to understand the communication protocols inside a vehicular network, the options of wireless communication, possibility of cloud computation, target hardware optimization and many more things. I worked on many of these different domains during my tenure at Nissan. But it was too much to be learnt by a single person end-to-end. I realized that I should focus on something that I really like and where I can contribute well. 

By this time, the artificial intelligence and the machine learning field was improving by leaps and bounds. The performance in object detection algorithms was improving every year. I decided to take an online course to check how these things were being done. I took the free Machine Learning course by Andrew Ng. in Coursera. I was so impressed with the concepts and possibilities of the field that I decided to move my career in that direction. Until that time, I was not directly associated with any of the standard machine learning workflows existing in the automotive industry. But our department being an R&D one, I had some freedom in terms of solving problems in different ways. I took up the task of pot-hole detection, which was tried in our department for some time. I collected accelerometer and GPS data by driving around our office campus in Chennai and labelled them against pothole or smooth roads. I got somewhat good results by using a simple neural network model on the collected data. After this work, the battery research team came to me with their battery materials data and we worked on optimizing the best combination of materials to improve the performance of batteries. After this I worked on an interesting task of clustering drivers based on their driving patterns and deciding which cluster of drivers might be more accident prone. 

From my early successes, I decided to learn more advanced techniques in this direction. This time I took up a paid course from deeplearning.ai which is also taught by Andrew Ng. I also learned the Python language to complete this course. This course gave me the confidence to solve more challenging problems in the industry.

How did you get your first break?

I consider there are two important breaks in my career. Admission into IIT and working for the General Motors project gave me an opportunity to work in the automotive research domain. I also got the much desired ‘IIT’ tag in the process. Getting into Renault Nissan gave me a wide range of experience in terms of new technology, multicultural work environment and the so-called ‘OEM’ work experience. 

What were the challenges? How did you address them?

After the initial deep learning experience on toy problems, I was eager to apply them to real world problems. But when I started working on real industrial problems, I realized how difficult it was to reach the desired level of accuracy. Particularly, for safety critical applications like pedestrian or vehicle detection, the permissible level of error is extremely low. While it is easy to go to a certain high level of accuracy using large data and standard models, it becomes incredibly challenging to increase accuracy even 1% beyond that. There are many ways to improve the performance like increasing datasets, using different model architectures, data balancing etc. From my experience of applying deep learning to computer vision problems, I realized that there can be no substitute for large, high quality data. But often it is either too costly or too late to get the right set of data. While working with a limited set of data, the best result I could extract is by use of innovative augmentation techniques. I also tend to use generative modeling to compensate for the lack of quality data.

Where do you work now? What problems do you solve

I currently work in the Hyundai Mobis Technical Center, India, as a Lead Research Engineer. I head the camera section in the ADAS (Advanced Driver Assistance Systems) department and manage multiple projects at different functional levels. Since all the big automotive manufacturers, and many non-automotive players like google and apple, are eyeing the autonomous driving breakthrough, the market demands a matured ADAS function as the first step. Our company, like many other tier 1 suppliers, is constantly working towards improving their products and technological foundations to achieve this goal. 

Apart from day-to-day technical management activities, I typically read a lot of research papers in the current computer vision domain to keep up with the state-of-the-art technologies. Also, I go through the latest news in the automotive sector to understand the market trend.   

How does your work benefit society? 

I have always been conscious about the societal impact of my work. At the end of the day, that is what satisfies me. I see myself working towards the bigger goal of autonomous driving. Road accidents are a major cause of deaths in the world. Even if the dream of a fully autonomous vehicle is not realized soon, the advanced driving assistance systems can still reduce road accidents. I hope to work particularly for Indian road scenarios in the future and make a change. 

Tell us an example of a specific memorable work you did that is very close to you!

I developed a lane detection algorithm early in my career. I travelled to Japan as the sole engineer to provide a demo in the Nissan test track. Unfortunately, the lanes on the test track were not as clear as the dataset based on which I wrote the algorithm. The algorithm failed instantly, and I was told to modify the algorithm within 2 days. I spent sleepless nights and developed the algorithm again from the basic principles of edge detection. It was not possible to simply go and test the newly developed algorithm in the test track because entry to the test track was highly regulated. On the next trial in the test track, the new algorithm was used and to everybody’s surprise it worked perfectly. It was a very memorable day for me as an engineer.

Your advice to students based on your experience?

The world is changing amazingly fast in the twenty first century. So, you must always be updated about what is going on around the world at all times. It is not possible to learn one technology and expect that it will last your entire career. Adaptability is important. So, do not be hasty in deciding your destination. Be ready to learn new stuff every year even after college is over.

You should have an overview of the subject or technology starting from the theoretical concepts to its final implementation in the industry. For this, try to talk to your seniors in the industry, follow news in line with your interests and try to form an idea in your mind. It is not possible to get the complete idea in the very beginning. But if you have an inclination of thinking in that direction, you can understand where your work is going in terms of the final business output. The sooner you understand that, the better you can steer your profile to the right direction. 

Future Plans?

I feel the ADAS functions in the Indian car market are still inadequate. I am particularly interested in creating ADAS products for the Indian car market.