AI/ML Data Engineer

Key Responsibilities

• Develop and maintain data pipelines and ETL processes using Python, Pandas, and Apache Spark.
• Design and implement scalable data processing solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Collaborate with data scientists and analysts to understand data requirements and ensure smooth data delivery and availability.
• Perform data cleaning, transformation, and normalization tasks to ensure data integrity and quality.
• Optimize data workflows and infrastructure for performance and efficiency.
• Implement data governance and security best practices to protect sensitive information.
• Proactively identify opportunities for data quality improvements and implement solutions.
  
  

• Proficiency in Python programming language with a strong emphasis on data manipulation using Pandas.
• Experience with Apache Spark for large-scale data processing and distributed computing.
• Hands-on experience with cloud computing platforms (AWS, Azure, GCP) and services such as S3, EC2, EMR, Databricks, or equivalent.
• Solid understanding of data cleaning, formatting, and transformation techniques.
• Familiarity with SQL and NoSQL databases for data storage and retrieval.
• Excellent problem-solving skills and the ability to work independently or as part of a team.
• Strong communication skills and the ability to effectively collaborate with cross-functional teams.
  
  

• Background or experience in finance or financial data analysis.
• Knowledge of machine learning frameworks and libraries (e.g., TensorFlow, scikit-learn) for data modeling and analysis.