Data Engineer

*Must be a US Citizen or GC holder

*No C2C

As a Data Engineer with Databricks experience, you will play a critical role in designing and implementing data pipelines, building data solutions, and optimizing data workflows. You will collaborate closely with data scientists, analysts, and other engineering teams to ensure our data infrastructure supports business goals and analytics initiatives.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
• Integrate data from various sources, including databases, APIs, and third-party services, into a cohesive data ecosystem.
• Optimize ETL processes for performance and reliability, ensuring data is clean, accessible, and ready for analysis.
• Collaborate with data scientists and analysts to understand data requirements and support analytics initiatives.
• Implement data quality and governance practices to ensure data integrity and security.
• Monitor and troubleshoot data workflows, addressing any issues that arise in a timely manner.
• Document data architecture, processes, and workflows to facilitate knowledge sharing and maintenance.
• Stay updated on the latest Databricks features and best practices to continuously improve our data solutions.

Qualifications:

• Bachelor’s degree in Computer Science, Data Engineering, or a related field.
• Proven experience as a Data Engineer, with specific experience using Databricks and Apache Spark.
• Strong proficiency in programming languages such as Python, Scala, or SQL.
• Experience with cloud data platforms, preferably Azure or AWS.
• Familiarity with data warehousing concepts and tools (e.g., Snowflake, Redshift).
• Knowledge of data modeling and ETL tools (e.g., Azure Data Factory, Informatica).
• Excellent problem-solving skills and a detail-oriented mindset.
• Strong communication skills, with the ability to collaborate effectively in a team environment.