Senior Data Engineer

As a Senior Data Engineer , you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support data analytics, business intelligence, and machine learning initiatives. You will lead the development of systems that ingest, process, and transform large volumes of data while ensuring data quality, governance, and security across various platforms. Your expertise will guide best practices in data architecture, ETL/ELT processes, and cloud-based data solutions.

• Design, develop, and maintain scalable and reliable data pipelines that extract, transform, and load (ETL/ELT) data from various data sources.
• Optimize and troubleshoot data workflows for performance, scalability, and cost efficiency.
• Automate and monitor data pipelines to ensure smooth, error-free data flows across environments.
• Design and implement cloud-based or on-premise data architectures to support large-scale data storage and processing.
• Collaborate with architects and engineering teams to ensure data systems are aligned with business objectives and technical requirements.
• Evaluate, select, and integrate new technologies and tools into the data stack (e.g., data lakes, data warehouses, real-time streaming technologies).
• Design and implement complex data models that enable efficient querying and support analytics and reporting needs.
• Build and maintain transformation logic to ensure data consistency, quality, and usability for data science and analytics teams.
• Partner with business stakeholders and analysts to gather requirements and develop high-quality, reusable data sets.
• Establish and maintain best practices for data governance, including data quality controls, privacy, and security.
• Implement data quality checks, validation routines, and monitoring mechanisms to ensure data accuracy and reliability.
• Ensure compliance with data policies and regulatory standards (e.g., GDPR, CCPA).
• Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to ensure data systems meet the organization's needs.
• Mentor and guide junior engineers in coding standards, data engineering best practices, and career development.
• Contribute to project management and development processes, including Agile methodologies and sprint planning.

Preferred Qualifications

• 5-7 years of experience in data engineering or software development, with a strong focus on building data pipelines and scalable infrastructure.
• Proficiency in programming languages like Python, Scala, Java, or SQL.
• Experience with modern data technologies such as Apache Spark, Kafka, Hadoop, or Flink.
• Strong knowledge of cloud platforms such as AWS, Azure, or Google Cloud, particularly their data services (e.g., AWS Redshift, Azure Data Factory, BigQuery).
• Experience with data warehousing, ETL/ELT processes, and distributed systems.