Data Engineer

Title: Data Engineer

Required Location: Dallas, TX (and in-office 5 days a week. Candidates must be local and available to interview in-person.

• TOP SKILLS:Although this role will be sitting on-site, they must also be able to use a personal workstation, or options are available for rent. They are a Microsoft shop.
• 5-8+ years of experience in data engineering or a related role.
• Seeking an individual with the ability to learn fast, eager to be in a collaborative team environment and work with an offshore team.
• Design, build, and manage ETL data pipelines using AWS services services, specifically Glue, Athena, and Lambda is a plus.
• Hands-on experience with Apache Hudi for real-time data management.
• Strong programming skills in Python and PySpark.
• Familiarity with SQL and NoSQL databases.
• Knowledge of data governance and data security best practices.
• Experience with workflow scheduling tools like AWS Step Functions or Control M.
• Collaborate with cross-functional teams to meet business objectives.
• Monitor performance and advise on any necessary infrastructure changes.
• Experience with Hadoop ecosystem is a must.
• Experience with RedShift is a must.
• Develop technical documentation including data dictionaries, metadata