Principal Data Engineer

Job Summary

We are seeking a highly skilled and experienced Principal Data Engineer to join our team. As a Principal Data Engineer, you will be responsible for architecting, designing, and implementing scalable and robust data solutions that enable efficient data processing, storage, and retrieval. You will provide technical leadership, drive innovation, and ensure the integrity and reliability of our data infrastructure. This is a senior-level position that requires exceptional technical expertise, strong leadership capabilities, and a proven track record of successfully delivering complex data engineering projects.

Job Description

Key Accountabilities

• Lead the design and development of scalable and reliable data pipelines, including data ingestion, processing, storage, and retrieval.
• Develop data models and schemas that support efficient data storage, retrieval, and analytics, employing optimization techniques to enhance query performance and scalability.
• Leverage big data technologies and frameworks (e.g., Hadoop, Spark, Hive) to process and analyze large volumes of data, enabling advanced analytics and machine learning initiatives.
• Manage and optimize data infrastructure, including cloud-based platforms, containerization technologies, and distributed computing environments.
• Work closely with other teams, including Data Science, Analytics, and Product, to understand their data needs and requirements.
• Develop and implement best practices for data modeling, storage, and retrieval.
• Ensure the security and privacy of our data and compliance with relevant regulations.
• Develop and maintain documentation for all data processes and systems.
• Evaluate new technologies and tools for data processing, storage, and retrieval and recommend solutions to improve the efficiency and scalability of our data infrastructure.
• Propose and lead continuous improvement opportunities.
• Mentor team members to develop their technical and leadership skills.

Education, Experience, & Skill Requirements

• 9-11 years of experience in data engineering, including experience in designing and building data pipelines
• Strong proficiency in data engineering technologies, such as ETL frameworks, big data processing, and SQL and NoSQL databases.
• Deep understanding of database systems, data modeling, and data warehousing.
• Experience with cloud-based data storage and processing technologies, such as AWS, Azure, or Google Cloud.

Key Metrics

• Understanding of data privacy and data governance policies.
• Strong problem-solving and analytical skills.
• Ability to work collaboratively in a team environment.
• Strong communication and interpersonal skills.
• Ability to lead and manage projects.
• Experience in leading a team of data engineers and managing complex projects.
• Strong problem-solving skills and ability to work in a fast-paced environment.
• Excellent communication skills and ability to work collaboratively with cross-functional teams.
• Experience with agile software development methodologies.