Senior DataBricks Developer

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Dew Softech Inc, is seeking the following. Apply via Dice today!

Senior DataBricks Developer

Location: Remote

Duration: 12+ Months

Job Summary

Preferably candidates within EST

Job Description

We are looking for a Senior DataBricks Engineer to develop robust code, understand and implement data pipelines and perform data transformations within the DataBricks platform. This role requires a combination of technical expertise, strong analytical skills, leadership skills, and a strong understanding of data engineering principles. This is a full-time , remote, contract opportunity.

What You'll Do:

1. Work closely with Software Designers to ensure best practices are being followed and offer suggestions for improvements to code proficiency and maintainability
2. Occasionally interface with customers to assist with analyzing user needs to determine technical requirements
3. Design, build, and maintain scalable and reliable data pipelines using DataBricks
4. Develop high-quality code with an emphasis on performance, scalability, and security
5. Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs
6. Implement data transformations and complex algorithms within the DataBricks environment
7. Optimize data processing and refine data architecture to improve system efficiency and data quality
8. Mentor junior engineers and contribute to the development of best practices within the team
9. Stay up-to-date with emerging trends and technologies in data engineering and cloud computing

Required Qualifications:

1. 7+ years of experience in data engineering or a related field
2. Strong hands-on experience with DataBricks, including development of code, pipelines, and data transformations
3. Proficient in at least one programming language (e.g., Python, Scala, Java)
4. In-depth knowledge of Apache Spark and its integration within DataBricks
5. Experience with cloud services (AWS, Azure, or Google Cloud Platform) and their data-related products
6. Familiarity with CI/CD practices, version control (Git), and automated testing
7. Excellent problem-solving skills and the ability to work independently or as part of a team
8. Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field

Physical Demands

1. Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state and local standards
2. Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
3. Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor