Principal Data Engineer

Job Requisition ID #

24WD79024

Job Description

Position Overview
As a Principal Data Engineer, you will build platform capabilities to enhance conversion metrics such as Add to Cart, Trials, Search, and Downloads. Applying Kappa architecture and event stream processing, you will create scalable data solutions. We are looking for advanced technical expertise and 8+ years of experience in an enterprise environment.

Minimum Qualifications

1. Architect, design, and implement scalable data pipelines using Kappa architecture and event stream processing technologies.
2. Lead the development of data solutions that directly enhance conversion metrics and improve user experience across digital platforms.
3. Collaborate with data scientists, product managers, and other team members to define data requirements and ensure data quality and integrity.
4. Implement real-time data processing and analytics solutions using technologies like Apache Kafka, Apache Flink, Apache Spark, and similar tools.
5. Use big data frameworks such as Hadoop, HDFS, and Hive for large-scale data processing.
6. Improve data architecture and processes to support high availability, scalability, and performance.
7. Mentor senior and junior data engineers, promoting a culture of learning and improvement.
8. Ensure compliance with enterprise security, performance, and scalability standards in all data engineering projects.
9. Contribute to the execution of data projects aligned with team goals.
10. Collaborate with teams to lead enterprise-wide data projects.
11. Stay updated on the latest trends and best practices in data engineering and event stream processing.

Responsibilities

1. Successful implementation of scalable data pipelines using Kappa architecture, resulting in improvements in data processing performance.
2. Development and deployment of real-time data processing solutions that enhance conversion rates and user engagement.
3. Collaboration with teams to deliver data-driven insights and improvements.
4. Improve data architecture and processes to ensure high availability, scalability, and performance.
5. Leadership in guiding and mentoring data engineers, contributing to the team's overall growth.

The Ideal Candidate

1. Lead a team of data engineers, encouraging a culture of continuous improvement and new ideas.
2. Promote the use of best practices in data engineering and event stream processing.
3. Explore and implement new technologies and methodologies to enhance data processing effectiveness.
4. Collaborate to align data strategies with team goals and guide data-driven decision-making.
5. Learn and stay updated on the latest trends and advancements in data engineering.
6. Identify and address potential areas for improvement in data processes and architecture.
7. Inspire and guide junior engineers, providing mentorship and support to help them grow in their careers.
8. Contribute to Autodesk's culture of innovation, striving to shape a better world through high-quality data solutions.
9. Experience deploying code in AWS, Azure, or Google Cloud environments.

Job Title

Principal Data Engineer

Location

San Francisco, California

Department

Autodesk Digital Experiences (DXC)

Reports To

Engineering Manager