Summary of Position:
The Data Engineer role will work closely with an agile development team to analyze and optimize ETL processes, tables, views, materialized views, and stored procedures to create a high-performance data warehouse solution for our client. This role will be working with a diverse set of technology including Amazon Redshift, S3, and Snowflake. The Data Engineer will focus on optimizing existing systems as well as developing designs and strategies for implementing new systems. This role will be responsible for requirements gathering, analysis, data categorization, interpreting requirements, and developing development tasks for sprint execution.
Essential Functions:
- Design, implement, test, and deploy data processing infrastructure
- Contribute to architecture of highly scalable and reliable data engineering solutions for moving large data efficiently across systems
- Perform work in an Agile team setting
- Break down, estimate, and provide just-in-time design for small increments of work
- Work in a complex data infrastructure environment
- Develop in-depth data pipeline using industry-standard data integration tools
- Full development life cycle management, including gathering, analysis, architecture, design, implementation, testing, deployment, and technical support
- Write test cases and test scripts for data quality assurance
- Responsible for creating stored procedures and functions
- Develop dimensional data model with the industry-standard tool
- Interpret reporting requirements into actionable development tasks
- Analyze and optimize SQL based stored procedures and jobs
- Analyze table indices for performance
- Design and implement Materialized Views and Views
- Develop high-performance programs and procedures for ETL Processes
- Analyze, categorize and document data sources and elements
- Write and optimize queries and provide guidance to other developers accessing data
- Agile development experience required, must be comfortable working with a distributed team
Competencies:
- Ensures Accountability
- Tech Savvy
- Communicates Effectively
- Values Differences
- Customer Focus
- Resourcefulness
- Drives Results
- Plans and Prioritizes
- Decision Quality
- Self-Development
Work Environment:
This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets, and fax machines.
Physical Demands:
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. While performing the duties of this job, the employee in this position frequently communicates with other co-workers/clients who have inquiries about the various projects and other needs. Must be able to exchange accurate information in these situations. The employee must be able to remain in a stationary position 75% of the time. The employee in this position needs to occasionally move about inside the office to access file cabinets, office machinery, etc. Constantly operate a computer and office machinery such as a calculator, keyboard, copy machine, and printer. Frequently moves boxes with equipment weighing up to 25lbs across the building and/or to other offsite buildings for various project needs.
Required Education and Experience:
- Bachelors Degree
- 5 + years of Data Engineering Experience or equivalent experience
Qualifications:
- Five + (5) years building out highly scalable, scaled-out architectures on large scale database platforms
- Five + (5) years of experience in a senior data engineering role
- Excellent SQL Skills
- Deep knowledge of systems such as Redshift, Snowflake, Postgres, Redis
- Experience with other programming languages such as Java and Ruby a plus
- Strong understanding of different data access standards including REST and SOAP data sources
- Advanced competency in SQL with the ability to optimize and mentor others to perform query optimization in large scale database environments
- Experience with any industry-standard tool for Source Control and Project Management
- Experience with data visualization and/or dashboard development
- Strong written and oral communication skills
- Demonstrates critical thinking, analytical and problem-solving skills, and ability to think creatively
- Exhibits a sense of ownership, urgency, accountability, and drive to learn new technologies
- Demonstrated ability to achieve stretch goals in a highly innovative and fast-paced environment
Preferred:
- Familiarity with AGILE and API development
- Familiarity with test-driven development methodology for analytic solutions
- Apache NiFi
- Redis
AAP/EEO Statement:
3MD Inc. is an equal opportunity employer and does not discriminate based on gender, sex, age, race and color, religion, marital status, national origin, disability, sexual orientation, gender identity or expression, veteran status, or any other category that is protected by applicable law.
Other Duties:
Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties, or responsibilities that are required of the employee for this job. Duties, responsibilities, and activities may change at any time with or without notice.
#J-18808-Ljbffr