Description
We are seeking a Data Engineer to provide data engineering services and solutions to enable programs to scale with growth and complexity. Role requires deep expertise in the design, creation, management, and business use of various datasets, across a variety of data platforms. You will be responsible for designing, developing and operating a data service platform using Python, SQL, etc. to build the various ETL, analytics and data quality components. You will design and implement data models and build the end-to-end infrastructure for reports and dashboards to be created.
The ideal candidate will be an expert in sourcing semi-structured and complex data types and normalizing them into a consumable data models that are accessible for other users. You will develop data products, infrastructure and data pipelines leveraging AWS services (Glue, EMR, Lambda, Redshift, Quicksight) and internal Amazon tools.
About the team
Our vision is to build best-in-class analytics solution to support our internal stakeholders and partners for making data driven decisions org-wide. On our team, you will have the opportunity to dive deep into complex business and data problems, drive high-impact and large scale data solutions.
Basic Qualifications
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
- Experience with AWS database/ETL tools including Lambda, Glue, Redshift, DynamoDB, EMR
- Experience building data products incrementally and integrating and managing data sets from multiple sources
- Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
Preferred Qualifications
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience operating large data warehouses
- Proficiency in the DevOps style of software deployment (infrastructure-as-code)
- Proficiency with AWS technologies including SNS, SQS, SES, Route 53, Cloudwatch, VPC
- Background in Big Data, non-relational databases, and Data Mining is a plus