Data Engineer

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Spire Systems Inc, is seeking the following. Apply via Dice today!

Responsible for performing difficult or complex analysis, design and programming involving multi-project leadership and broad responsibility in support of new and existing Data Engineering Projects and production support activities.

Core Duties:

1. Collaborate with cross-functional teams to understand business needs and identify opportunities for applying machine learning solutions.
2. Experience with Natural Language Processing (NLP) and deploying/using Large Language Models.
3. Deep understanding and commitment to software engineering principles/processes (e.g. Lean, Agile, DevOps, CI/CD) and continuous improvement through measurement.
4. Thorough knowledge, expertise and practice Data Management Framework to design world class data stores. Best practices, Data Quality and security are critical.
5. Understand data endpoints, consumers and develop strategy.
6. Fluid end-to-end data vision, design pipelines for seamless data flow.
7. Lead and perform the design, development, implementation and maintenance of complex Data Store/ Data Lake/Lake house and Data warehousing systems and data-intensive solutions that are scalable, optimized, and fault-tolerant.
8. Design and implement Data Migration and Data Integration across cloud and hybrid environments.
9. Mastery and hands-on experience with Data Engineering technologies and scripting languages. Identify new technologies and provide recommendations to Management.
10. Solid understanding and experience in Cloud technologies and applications. Data Migration, Integration, API's development, Data Streaming (Batch and continuous) and scheduling.
11. Data Modeling skills. Able to come up with a Canonical Data Model and simplify data flow and interaction between different applications. Should be able to integrate new sources smoothly.
12. Ability to translate complex functional and technical requirements into detailed architecture, design and high performing software.
13. Design, build and optimize pipelines for data collection for storage, access and analytics.
14. Out of the box thinking to overcome engineering challenges with innovative design principles.
15. AWS data lake experience (Airflow, Redshift, S3, Glue Studio, Athena etc.).
16. Programming (SQL and Python/Pyspark).
17. AI/Client - Knowledge in AI/Client and using AWS Bedrock/SageMaker, LLMs and others. Nice to have some experience.