AI Big Data Engineer

Job Title: AI Big Data Engineer

Location-Type: Hybrid - Troy, MI - T/W/Th on-site

Start Date Is: 10/21/24

Duration: 12+ month contract

Compensation Range: $60-$75/hr

Role Overview :

As a Data Engineer with our client, you will contribute to building a Unified Data Platform, focusing on designing, developing, and maintaining data pipelines, data lakes, and platforms to support analytics and business intelligence. You'll work with modern tools and technologies like Spark, Kafka, AWS, Azure, and Kubernetes to solve complex data challenges while collaborating with cross-functional teams to ensure data reliability and quality.

Day-to-Day Responsibilities :

• Build and automate data pipelines for legacy platforms (mainly SQL Server).
• Implement business logic on modern data platforms (e.g., AWS Glue, Azure, Databricks).
• Create vector databases, data marts, and the necessary data models.
• Optimize data systems for performance, reliability, and security.
• Integrate and transform data from various sources and formats (structured, unstructured, streaming).
• Maintain data quality checks, documentation, and tests.
• Support data analysis and visualization using tools like SQL, Python, Tableau, and Quicksight.
• Research and evaluate new data technologies to enhance current solutions.

Must-Haves :

• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field.
• 5+ years of experience in data engineering or a similar role.
• Proficiency with big data frameworks and tools (e.g., Spark, Kafka, Hadoop).
• Expert-level SQL skills, including query design, data modeling, and stored procedures.
• Experience with programming languages like Python, Go, or Java.
• Familiarity with CI/CD, containerization (Docker, Kubernetes), and orchestration (Airflow).
• Experience with cloud platforms (AWS, Azure, or GCP) and modern ETL/ELT tools like AWS Glue, Snowflake, and Databricks.

Soft Skills :

• Strong communication and collaboration abilities.
• Problem-solving mindset with attention to detail.
• Self-motivated and eager to explore new technologies.

Nice-to-Haves :

• Previous DBA experience.
• Knowledge of data governance, security, and quality principles.
• Experience with data visualization tools (e.g., Tableau, Quicksight).