The science of colors is highly interdisciplinary, encompassing physics, chemistry, physiology, statistics, computer science and psychology.

Abhijit Sarkar (PhD), our next pathbreaker, Imaging Architect of the Mixed Reality Architecture team at Microsoft, works on the overall display image quality of the next generation of HoloLens displays.

Abhijit talks to Shyam Krishnamurthy from The Interview Portal about his industrial PhD work on observer metamerism, a concept related to how individuals with normal color vision perceive colors differently.

For students, we are all unique individuals on a unique life trajectory carved out by our unique circumstances and personalities. Be inspired by others but create your own path !

Abhijit, can tell us a little bit about your background?

I was born and brought up in Kolkata. My father was a doctor (a consultant surgeon), and mother was a homemaker, both of whom passed away several years ago. I have an older sister, who is an anesthesiologist and settled in Kolkata with her family after living in the UK for over a decade. 

Like many traditional Bengali (and Indian) families in the seventies-eighties, we had a modest lifestyle – eating out or going away on vacation was relatively rare. My parents did everything and made countless sacrifices to ensure we focused on our studies and did well academically. I studied at Hindu School in north Kolkata, which was and still is quite reputed, with a glorious history starting from the pre-independence era. As a teenager, I was very serious about whatever I focused on, which was mostly my extra-curricular activities. I was not too motivated by the grades and ranks at school (well, until I got to my 8th grade, when, thanks to my mother, my priorities changed drastically). 

I had a unique hobby when I was growing up – I used to listen to the Bengali programs of foreign radio stations and used to send them correspondences. Radio Moscow, Radio Beijing (erstwhile Radio Peking), BBC, Deutsche Welle (German radio station) and several others had daily programs in local Indian languages to promote cultural exchange. I would write to the radio stations in Bengali or English, and share feedback on their programs. While I used to send letters to their Delhi address, the responses would almost always come from their respective country headquarters, often with little gifts like stamps, badges, postcards, calendars, or language books, which meant the world to me. I cannot describe the joy of waiting for days with bated breath for replies and then receiving letters from faraway lands with beautiful postage stamps, and with my name handwritten on them! It saddens me to think that the teenagers of our subsequent generation would never know the simple joy of writing and receiving letters or being glued to a radio. That was the era of B&W Television with one or two channels 😊.

From those teenage years I developed a great fascination for foreign countries, cultures, and languages, which stayed with me and helped imbibe within me a spirit of curiosity and receptivity, as my quest for higher education took me to new places far away from family and friends – as my address kept changing across seven different cities in the US and France in the span of a decade.

If there is one reverberating theme in my early interests and my career journey, then it is that I always levitated toward the idea of trying out something different and new, something that 95 out of 100 people would not choose to do. Many of my close relatives were in the medical profession, but biological sciences did not motivate me. I had a vague idea that I wanted to be in engineering, given the limited choices in the (regrettable) binary world of doctors and engineers, but my favorite subjects used to be languages.

What did you do for graduation/post graduation?

I completed my bachelor’s degree in electrical engineering from Jadavpur University, Kolkata. That was way back in 2000. 

I have two master’s degrees, the first from Pennsylvania State University, Pennsylvania, USA in Architectural Engineering (specializing in Lighting) in 2005, and the second from Rochester Institute of Technology (RIT), New York, USA, specializing in Color Science in 2008. Right after my 2nd MS, I moved to France for a three-year industrial PhD in applied computing (focused on fundamental aspects of color science). I was enrolled as a doctoral student at the Université de Nantes while working as a research engineer at Technicolor Research & Innovation, in Rennes (pronounced ren) where I was working full time toward my doctoral thesis, which I completed in 2011.

What made you choose such an offbeat, unconventional and unusual career?

My career has been somewhat of a meandering journey with its ebb and flow. As I moved from one stage to the other, I tried to leverage my background and accomplishments from the previous stages and made efforts to move toward a path that would lead to better career prospects and opportunities.

By the time I completed my bachelor’s, I completely lost interest in studying further. In fact, my self-confidence took a nosedive. My heart was in the topics of electronics or computer sciences rather than conventional core EE subjects that constituted the bulk of my undergraduate coursework. You might wonder then why I chose electrical engineering as my stream. Actually, I wanted to study at JU because of its reputation as a regional institute, but my state JEE ranking would have required me to go elsewhere to study ECE/CS. I would humbly admit today, I was miserable during the coursework since much of it did not resonate with or inspire me, which showed amply in my overall grades. My saving grace was seeking the mentorship of a couple of my professors and alumni, who gave me opportunities to engage in hands-on projects outside of my coursework. As it turned out, a software project that I started in my 2nd year would influence my career track much later. As part of that, I developed a fully functional software tool for graphical design of stadium lighting in C language and that arduous project made me quite proficient in programming and lighting design principles. That led me to choose Illumination Engineering as my elective and to continue my software project, even with some professional guidance from an alumnus at the Philips Lighting Center, Kolkata. I also took the initiative to solicit guidance from the founder of a reputed lighting design software firm in the US.

To my detriment, I never felt motivated by mainstream software jobs, which most of my classmates chose to go for after graduation. Thanks to the recommendation of one of my professors, I got a chance to work at a defense lab in Hyderabad, which builds electro-optical instruments for navigation systems. At that point, I was not seeing a solid path in front of me. Even though my first job convinced me that I wanted to be involved in research, I was not confident I could get into higher education and be successful in it, which I knew was imperative for a career in research. 

The turning point was when one of my cousins, who was studying at a US university after graduating from IIT Kharagpur, convinced me to give it a shot by taking the GRE and applying to several US universities. I took the GRE soon after 9/11. I applied to several US universities – as you will recall, that was a very uncertain and tense period for much of the world. The admission process at US universities is quite holistic, evaluating an applicant on a multitude of aspects. Despite having below average undergraduate grades, high GRE scores and a track record in initiating and executing hands-on projects landed me a break. The lighting program of Pennsylvania State University, one of the top lighting programs in the US, accepted me into their MS program and offered full financial aid (research assistantship). That was the start of a transformative phase for me both academically and in personal life. I started genuinely enjoying the graduate coursework, which was a lot more hands on and project oriented than my undergraduate studies, resulting in consistently good performance. My work and studies occupied most of my waking hours (as is common in most graduate coursework in the US). Moreover, being so far from family was emotionally taxing, particularly during the first year. Nevertheless, it seemed like my career was finally moving forward. 

How did you plan the steps to get into the career you wanted? Or how did you make a transition to a new career? Tell us about your career path

When I was finishing up my master’s at Penn State, my goal was to continue doing applied research in lighting control. My master’s thesis was a proof-of-concept work on a novel energy efficient lighting technology, where I was using a digital (CMOS) camera and image processing to integrate the functionalities of a photo sensor and occupancy sensor (An occupancy sensor detects the presence of movement within its given range). However, I started realizing career opportunities in lighting research were fairly limited and my US student visa was getting close to expiration, which meant I would have to return to India with uncertain employability. I started looking for a new avenue with better prospects where I could leverage my background in lighting, programming, and electrical engineering. While researching online, I stumbled upon the website of Munsell Color Science Lab (MCSL) at RIT in Rochester (it falls under the Imaging Science department), New York. I learnt there was a field called color science, which is highly interdisciplinary, encompassing physics, chemistry, physiology, statistics, computer science and psychology. Many color scientists would get employed by companies I knew of and revered, like Xerox, Canon, HP, Apple and such. I also learnt that RIT’s program was the only graduate program in the US focused on color science, arguably the most reputed program in the field worldwide, and hence quite competitive. By the way, Rochester, NY has been historically known as the imaging capital of the world – this is where two iconic imaging companies were founded: Xerox and Eastman Kodak. 

The application deadline for RIT’s graduate admission was within a week, so after a brief consultation with the professors in the program, I went with my hunch and rushed my application for a second master’s degree. To my huge relief, I got accepted with full funding (which included a scholarship and a graduate assistantship). I believe my interdisciplinary background and my first MS thesis, which included certain aspects of color processing, worked in my favor. I should also mention, I owe a lot to my advisor at Penn State who painstakingly guided me to become proficient at analysis and technical writing – two critical skills for any researcher. 

Anyway, that is how my career transition to color science started. I did two internships during my second MS, the first at HP in Vancouver, WA, and the second at Intel in Chandler, AZ. Intel was also the sponsor of my master’s thesis research, focused on perceptual color enhancement algorithms in still images and videos. Separately, my research proposal for an independent project earned a Kodak graduate student research grant from my department. This was to continue my research at Penn State. I also secured support from a lighting control company and received the required system hardware at a steep discount. With the support of my professors, I successfully completed the project and built a functional prototype system in our lab. To follow up, I organized a demonstration of my work to several lighting company representatives, secured more industry sponsorship and wrote a grant proposal to take my work to the next level – which ultimately did not materialize as I decided to leave RIT after completing my master’s thesis.

My plan was to continue on to a PhD at RIT. However, as luck would have it, there was some uncertainty around financial support, and around the same time I came across an opportunity to do a 3-year industrial PhD at Technicolor Research & Innovation in France, funded by the French government’s ANRT-CIFRE program for industry-university collaboration. By that time, I had already spent close to five years for graduate studies and was concerned about having to spend up to five more years as a doctoral student, even though I felt I needed to have a PhD under my belt to build a solid foundation for my career in color science. Hence, an option to do a shorter industrial PhD while earning a decent salary appealed to me. 

Moving to a new country not knowing anyone, and not knowing the language was an unnerving prospect, but I took solace in the fact that my thesis was going to be in English and at work I would be able to communicate without issues. Indeed, the first several months were difficult and often lonely outside of work. However, moving to France was one of the best decisions I ever took in my life. I started learning French earnestlyf and made rapid progress. In those three years I learnt to speak fluently, fell in love with the language and culture, met many friends, wonderful colleagues, and academic advisors, many of whom went out of their way to help me adapt in a new country. Moreover, I had an opportunity to visit seven different countries in Europe either as a tourist or for work – which I feel is a huge advantage of residing in the European continent. I have many heartwarming memories from my days in France that I will cherish forever. My supervisor’s family embraced and supported me during my stay, and essentially became my extended family – we are still close after a decade.

When I left the US, I was determined to produce an outstanding doctoral thesis even though it was a shorter industrial PhD. It worked out better than I expected. My thesis topic was about the fact that individuals perceive colors differently even within the human population with normal color vision, owing to the differences in the sensitivities of the photoreceptors in the retina, which are responsible for our color vision. So, two colors from disparate sources that seem to match for one person can be a complete mismatch for another person under certain conditions, a phenomenon known as “observer metamerism”. I came up with a cost-effective prototype device that would classify observers based on their color vision, which could then be used to create personalized visual experience. The work not only led to about ten conference and journal publications and multiple patent applications, but also helped me earn a prestigious scholarship from the main technical body in imaging science, Imaging Science & Technology (IS&T),as well as a best student paper in the most important conference in our field, where I demonstrated the little inexpensive contraption my colleagues and I built to classify people (with normal vision) into various categories based on their evaluations of how two colors matched. In an effort to start collaboration with other research teams, I traveled to two universities in Germany and Hungary to conduct visual experiments and published papers. Overall, it was quite a hectic 3-year period, but also quite exciting to come up with new ideas, take initiatives and collaborate with others in the field. I still feel grateful and immensely fortunate for all the unconditional support and encouragement I received from my colleagues and professors in France. My work led to follow-up doctoral research and other studies.

Recognition also came in terms of administrative and immigration processes. During my second year in France, I received a special residency permit called Carte de Séjour Competences et Talents, which was a highly selective, merit-based, multi-year permit that provided reprieve in terms of official paperwork during the remainder of my stay (here I must add that the French administrative system is typically extremely bureaucratic, which can be challenging for foreign students). Similarly, when my US work visa was expiring, I applied and received the equivalent US document called EB-1 green card. In both cases, I applied as I was advised, but never thought I would qualify and meet the stringent requirements. These were no less consequential in my life than receiving financial support. Before these breakthroughs, I had to deal with many life-altering challenges, which is quite common for someone on a student/work visa. The lesson here is that competitive applications like these are typically evaluated against a set of pre-requisites, and more often than not, you do not really need to have an extraordinary profile to qualify. Do not discount yourself without doing an objective evaluation of where your candidacy stands.

During the last year of my PhD, I started consulting for a New York based early-stage startup. It was moonlighting in its true sense since I would often work till late at night or early morning. While this increased my workload significantly, it was quite an interesting and rewarding technical challenge, which involved specialized processing of smart phone captured photos of a user to correctly identify skin, lip, hair, and eye colors, and recommend personalized beauty products based on color categorization. After completing my PhD in 2011, I returned to the US to join the startup as its first employee, to develop color imaging technology and help raise seed funding, which we accomplished the next year. The next phase of my life was truly tumultuous: leaving my startup job, losing my mother quite abruptly a month before my marriage, and starting my first corporate job at Microsoft – all these major life events happened within a span of six months, and in a most unexpected and overwhelming way.

How did you get your first break?

I think getting accepted at Penn State could be considered the first break in my career. But frankly, gathering enough courage and motivation to leave the job in Hyderabad to completely focus on preparing for GRE and the rest of the admission process despite all odds – I think that was what ultimately made a difference.

During the last year of my PhD, I started consulting for a New York based early-stage startup. It was moonlighting in its true sense since I would often work till late at night or early morning. While this increased my workload significantly, it was quite an interesting and rewarding technical challenge, which involved specialized processing of smart phone captured photos of a user to correctly identify skin, lip, hair, and eye colors, and recommend personalized beauty products based on color categorization. 

After completing my PhD in 2011, I had a full-time job offer from Technicolor to continue what I was doing, but the challenge and thrill of building something new attracted me, in spite of the inherent uncertainty of entrepreneurial ventures. So, I returned to the US to join the startup as its first employee, to help the CEO build on her vision of developing the underlying color imaging technology and help raise seed funding. The transatlantic move was less of a change this time since I had already lived in the east coast for five years, but the challenging part was that for the first six month or so, I did not have any income – relying entirely on my modest savings. During that period of constant anxiety and uncertainty over the future, my wife (fiancée at the time) became the key source of emotional support and encouragement for me. At that time, we were working round the clock to build a functional app as a proof-of-concept – there wasn’t any time to think about anything else. After pitching to several investors and going through their due diligence process, we finally achieved our goal – we received some seed funding around mid-2012, which we accomplished the next year. While it was intense, I feel that period taught me some vital life lessons and enriched me professionally.

The next phase of my life was truly tumultuous: leaving my startup job, losing my mother quite abruptly a month before my marriage, and starting my first corporate job at Microsoft – all these major life events happened within a span of about six months, and in a most unexpected and overwhelming way.

What were some of  the challenges you faced? How did you address them?

In hindsight, the biggest challenge was reconciling the chasm that existed between what I needed to thrive as a student and what, in my perspective, is typically needed to thrive in the traditional Indian educational system, particularly at the undergraduate level. In the latter, the focus is often on theories, tests, and grades, based on set coursework and little room for independent thinking or problem solving. The application part, which involves learning skills in building and constructing things, is often obscure for the student. Some do fare well in adjusting to that system of learning more than others. For those who don’t, this is potentially just a mismatch between the structure of the education system and the intellectual disposition of a student – neither the system is bad, nor is the student who is not able to fit in. I did not have this realization back then, and I left the university demoralized and with low self-esteem. For the same reasons, I did fairly well during my MS and PhD studies in the US and France, and thoroughly enjoyed being a postgraduate student for over eight years. 

The other challenge was adapting to all the curveballs life throws at you when you are an international student with little or no savings. Being away from home, friends and family and the resulting emotional toil, owning or sharing responsibilities for house chores that one never bothers about before leaving home, getting used to a lifestyle where not having or driving a car often means great hardship, taking care of all sorts of official matters (immigration, banking, tax, so on and so forth) while working 12-14 hours a day – all this takes a lot of adjustments and determination to persevere and succeed. Fortunately, I received full financial support throughout my MS and PhD studies, which does require a stroke of luck apart from active planning and efforts.

Where do you work now? 

I am a member of the Mixed Reality Architecture team working on HoloLens at Microsoft in Redmond, WA. HoloLens is Microsoft’s Augmented Reality (AR) headset, where a rendering of a synthetic (i.e., virtual or computer generated) world is overlaid onto the real world the user sees through the glass (a visor). In contrast, Virtual Reality (VR) headsets do not have a view of the real world, the display is opaque, so the user experiences an exclusively synthetic/virtual world. AR technology is still quite novel and is in the early stages of development. While it is not available for regular consumers quite as yet, it is finding its way into enterprise applications, for example in manufacturing of complex systems like car and aircrafts (for example: HoloLens-Volkswagen collaboration), as a training and education tool, as a supporting technology during surgical procedures and medical training (as an example, AIIMS Jodhpur announced a collaboration with HoloLens in May ’22), and even for defense applications. There are many potential consumer applications, which seems only a matter of time. Various companies like Microsoft, Meta, Apple and Google are aggressively investing in the technology development in this space, and many industry experts believe within the next 5-10 years mixed reality (MR) technologies will become as transformative in our daily lives as were the cell phones more than two decades ago.

What problems do you solve?

As an Imaging Architect and color scientist, I am responsible for overall display image quality of the next generation of HoloLens displays. During product design and manufacturing (whether it is a smart phone, a tablet or an AR headset), we need to think about several image quality aspects, for example how to make brightness and color look uniform throughout the display, how to reproduce accurate or preferred colors, how to make the display adapt to the user’s needs and ambient condition (e.g., bright outdoor vs dim indoor). These require software, hardware (electronics), and the factory calibration process to all work in concert.  So, the scope of my work includes developing and implementing the core algorithms and processes underlying various image quality features, defining, and implementing the display color calibration process at the factory, and work on computational models and metrics to objectively and subjectively assess display image quality.

What are the skills needed for your role? How did you acquire them?

My work involves quite a bit of programming (mainly in Matlab and C/C++) and various core concepts of color science, imaging, color vision and perception – much of which I learnt during my master’s and PhD studies. It also requires an understanding of display hardware, software, optics, and product assembly process that I learnt at work over the years. A career in technology entails constant learning. 

What’s a typical day like?

My day-to-day activities vary widely, depending on whether I am supporting the factory operations, or doing development work or analysis, or working on prototypes for demos. I often have to collaborate with colleagues and external partners in Europe and Asia across various time zones. During particularly busy periods during product development cycles, I have days that stretch up to 12-14 hours.  In a large company like Microsoft, collaboration/communication with a large number of people is typical, so apart from all the technical work, preparing for and participating in meetings can take up a significant part of weekly hours at work.

What do you love about your work? 

Well, in some respect I am at the frontier of technological innovation in the consumer electronics industry, and I believe my work, along with that of many of my colleagues and peers, would contribute in some way to advancing the technology that is still in its nascent stage. 

I really enjoy the interdisciplinary nature of my field where science and engineering come together, and I am happy that in my day-to-day work I am able to do a little bit of various things that I enjoy doing – coding, collaborative ideation, prototyping/invention, solving technical and project management related challenges and so on.

How does your work benefit society? 

The ultimate goal of the kind of work I do is to contribute to the development of new technologies and products that would one day improve the productivity of consumers and businesses alike, and maybe, would eventually help realize what is not feasible today. One example would be healthcare applications.

Tell us an example of a specific memorable work you did that is very close to you!

On the research front, as I described earlier, my PhD work on observer metamerism was quite rewarding. That topic has been close to my heart ever since. 

However, I would always cherish my initial years of work at Microsoft as I got to make significant impact in the formative years of a new category of a consumer electronics product described as “2-in-1 PCs”, which brought together the functionalities of a tablet and a laptop. I was one of the early members in the Surface display design team at Microsoft, responsible for improving the display image quality in multiple generations of Surface 2-in-1 PCs, laptops, and All-in-ones. I worked with a small team to introduce display color calibration on every product, eventually leading to more than a million-dollar investment to realize that goal. That was followed by three other product features that I conceived and helped implement. It brings a different kind of satisfaction when your work lights up on the devices of hundreds of thousands of users.

Your advice to students based on your experience?

One reason I took pains discussing my humbling experiences particularly during my early career phase is because I feel there might be a young student somewhere reading this, who has the same temperament and predicaments as I had, who could benefit from the insights I gained much later in life. Understanding one’s strengths and weaknesses vis-à-vis the system one belongs to, be it an educational or a professional setting, is key to self-improvement and career planning. Setbacks in life will come in different shapes and forms, but there is no setback big enough that you cannot overcome with perseverance and hard work. 

I have always believed that in order to go far, one must not fall prey to a herd mentality. We are all unique individuals on a unique life trajectory, carved out by our unique circumstances and personalities. As you plan your next career steps, be inspired by others’ success – this website by the way is an excellent resource for that – but do not try to imitate them, because the definition of success is subjective and conditional. In my opinion, one’s career goal should not be about earning a specific salary, or even about being in a specific role or a specific company. Career planning should ultimately be about self-discovery, about finding what it is you feel most passionate about, what gives you happiness and satisfaction, and most importantly, what you do demonstrably better than most others around you. That process of discovery takes time, and sometimes, as in my case, can take you along a winding road, but that is OK as long as you keep working hard toward better career prospects. Your passion by itself may or may not make you employable, but you must strive to make your passion a part of your vocation. As you start this journey, seek to grow your professional network on sites like LinkedIn and look for a mentor who you can respect and trust, who can guide you along.

I feel in India, starting right from primary school, we tend to put way too much emphasis on being naturally brilliant and intelligent as a precondition of being successful in life and career. My experience says those traits are rather overrated on their own – what matters most in the long term is perseverance and diligence. The good news is you can groom yourself to improve upon these traits. 

One final thing that I feel is not discussed often enough with regard to career planning: academic qualifications are not enough to succeed in professional life, particularly in a corporate environment. What is just as important is having social and emotional intelligence – developing interpersonal skills and a well-rounded, balanced personality. This starts with how you communicate with and treat your parents and loved ones, your classmates, your professors, and even strangers you come across in everyday life. Improving emotional intelligence should be part of everyone’s career preparation right from the beginning. Dale Carnegie’s book is a good starting reference on this topic. 

Future Plans?

I am actively looking for opportunities to assume a more impactful role in my organization and am currently considering various avenues available to me for the next stage of professional growth.