Senior Software Engineer, AI Infrastructure

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day.

Microsoft’s Bing Platform Engineering team is seeking a Senior Software Engineer, AI Infrastructure to join us in blazing the trail for developing the world-leading artificial intelligence (AI) infrastructure at datacenter scale. By leveraging the massive parallelism delivered by current and future computing hardware (e.g., GPUs and Brainwave FPGAs), the team accelerates many types of deep neural network (DNN) workloads, including large-scale inference and training. We are applying the accelerated computing to challenging applications important to Microsoft and our customers, including but not limited to Bing search. Our work includes developing, optimizing, deploying, and maintaining our machine learning (ML) accelerator software, infrastructure, and tools. Working with Microsoft model developers and data scientists to deploy their models gives us the unusual opportunity of working across the whole DNN stack, from cutting-edge DNN models, techniques, and frameworks to accelerator architecture. This cutting-edge development takes place in the context of a live, global-scale distributed system that touches millions of users daily.

Responsibilities

• Understand different DNN workloads (e.g., Transformers and LLMs) and map them efficiently to the underlying system and hardware architecture.
• Apply state-of-the-art AI and machine learning techniques to build and optimize the system to continuously achieve improved performance and user experience.
• Participate in/lead designs, and write production-level, quality code.
• Deploy and maintain software to meet significant reliability and service level requirements.
• Work in a team of software and hardware engineers to deploy working systems to a very large user base.

Qualifications

• Bachelor's Degree in Computer Science, or related technical discipline AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.
• OR equivalent experience.

• Familiarity with neural networks, ML frameworks (e.g., PyTorch and Tensorflow), and inference/training optimization techniques.
• Experience developing ML algorithms or system infrastructure in C/C++.
• Software development experience, particularly in distributed and/or fault-tolerant systems.
• Previous software development experience in all stages of a release cycle (design, implement, and test) with ability to cover multiple job roles.
• Experience in LLM finetuning is preferred.
• Proficiency in GPU programming (e.g., CUDA) and familiarity with CUDA libraries, OR in FPGA hardware and application development, is preferred.
• Experience with container and orchestration technologies (e.g., Kubernetes, Docker) is a plus.

Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $112,000 - $218,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $145,800 - $238,600 per year.

Microsoft will accept applications for the role until May 3, 2024.

Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.