Cloud engineer / Data engineer

Location: Mclean, VA (Hybrid)

Duration: 6 months with possible extension

Job Description

Must have: Spark, Python, Scala, and AWS (Glue & Lambda, S3, IAM)

1. Design and develop scalable data architectures and systems that extract, store, and process large amounts of data.
2. Develop and architect scalable data pipelines and ETL processes using programming languages such as Python, Scala, frameworks like Spark, and AWS services like Glue, Lambda, NoSQL databases, and serverless workflow orchestration using step functions.
3. Collaborate with Software Engineers, Business Analysts, Data Analysts, and/or Product Owners to understand their requirements and provide efficient solutions for data exploration, analysis, and visualization.
4. Automate integrations and data pipelines using data governance and various testing techniques.
5. Perform unit tests and conduct reviews with other team members to ensure your code is rigorously designed, elegantly coded, and effectively tuned for performance.
6. 4+ years of experience with a public cloud (AWS: Step functions, Glue, Lambda, DynamoDB).
7. 4+ years of experience with distributed data/computing tools (Spark).
8. 2+ years of experience with Agile engineering practices. Utilize programming languages like JavaScript, Java, HTML/CSS, TypeScript, SQL, Python, and Scala, NoSQL databases, and serverless workflow orchestration using step functions.