Senior Data Engineer / Developer
Strategic Data Systems (SDS) is a software development firm focused on strategy, technology, and business transformation for fortune 100 companies, mid-sized companies, and startups. We redefine what’s possible and create what’s next so our clients can ‘Accomplish More’ by trusting the excellent work and the technical professionals we provide to assist. As a consultant for Strategic Data Systems, you will have the opportunity to work with multiple clients, projects, and technologies.
About The Role
As a Developer /Data Engineer , you will collect, aggregate, store, and reconcile data in support of Customer's business decisions. You will design and build data pipelines, data streams, data service APIs, data generators and other end-user information portals and insight tools. Python Development Skills are an important requirement for this role.
Day-to-day You Will
- Plan, build and implement data solutions based on DataBricks
- Design and build Modern Data Pipelines and Data Streams
- Develop and Maintain Data Warehouse
- Design and create ETL processes supplying Data Warehouse
- Implement effective metrics and monitoring process
- Preprocess of structured and unstructured data, create queries in databases
- Experience working with Azure Databricks
- Familiarity with PySpark
- Knowledge of at least one component: Azure Data Factory, Azure Data Lake, Azure SQL DW, Azure SQL
- Strong software development skill using: Python
- Experience with Apache Nifi, Delta tables and Kafka a plus.
- Ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flows
- 5+ years of Python with experience in newer language features such as type hinting
- Experience with modern development tools such as PyCharm or VS Code
- "Unity Catalog" experience is a plus