Data Engineer - Active Secret Clearance

DATA ENGINEER – Active Secret Clearance

Python, Spark, API

Hybrid

ALTA IT Services has two direct hire openings for Data Engineers to join a respected systems integrator, in support of a leading DOD program. As a Data Connections Engineer specializing in Python, Spark, and Databricks, each candidate will be responsible for designing, implementing, and maintaining efficient data connections and integrations within the organization's data ecosystem. Their expertise in these technologies will be crucial in establishing seamless data flows, ensuring data accessibility, and facilitating data-driven decision-making. This role requires strong technical proficiency, problem-solving abilities, and collaboration with cross-functional teams.

This is a hybrid position, 2 to 3 days a week onsite in Crystal City, VA.

The role requires a minimum of an interim secret clearance.

Responsibilities

• Design and architect data integration solutions to connect various data sources and systems using Python, Spark, and Databricks.
• Define data connection requirements, data flow diagrams, and integration patterns.
• Develop and implement scalable data pipelines to extract, transform, and load data from source systems into target destinations.
• Utilize Python, Spark, and Databricks to orchestrate and automate data ingestion processes.
• Perform data transformation and enrichment operations to ensure data quality, consistency, and compatibility across different systems.
• Apply business rules and validation techniques to enhance data accuracy and reliability.
• Develop custom APIs (Application Programming Interfaces) to enable secure and efficient data exchange between internal and external systems.
• Integrate APIs with data pipelines to facilitate real-time data updates and synchronization.
• Implement data security measures and protocols to protect sensitive information during data connections and integrations.
• Ensure compliance with relevant data privacy regulations and industry standards.
• Identify and resolve performance bottlenecks in data connections and integration workflows.
• Optimize data processing, network utilization, and resource allocation to enhance overall system performance.
• Collaborate closely with cross-functional teams, including data engineers, software developers, and business analysts, to understand data integration requirements and deliver optimal solutions.
• Liaise with stakeholders to gather feedback, address concerns, and ensure alignment with business objectives.
• Identify and troubleshoot data connection issues, including connectivity problems, data synchronization discrepancies, and system errors.
• Develop effective resolutions and preventative measures to minimize future disruptions.
• Create comprehensive documentation, including technical specifications, integration guides, and operational procedures.
• Share knowledge and best practices with team members and stakeholders to facilitate effective use of data connections.

Requirements

• Experience in data engineering or integration roles with a focus on Python, Spark, and Databricks.
• Strong proficiency in Python programming and experience with relevant libraries and frameworks (e.g., requests, Flask, etc.).
• In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark DataFrame API.
• Proficiency in working with Databricks platform for large-scale data processing, data integration, and analytics.
• Familiarity with data integration patterns, ETL principles, and data governance practices.
• Experience with RESTful API development, implementation, and integration.
• Solid understanding of data formats and serialization techniques (e.g., JSON, XML, Avro, Parquet, etc.).
• Familiarity with data security principles, encryption, and access control mechanisms.
• Strong problem-solving abilities with the capability to analyze complex data integration challenges and provide effective solutions.
• Excellent communication and interpersonal skills, with the ability to collaborate effectively with technical and non-technical stakeholders.

COMPENSATION: $155,000 range annually, excellent benefits.

Interview Requirements:

• Video Interview – Yes (may include in-depth assessment).

Fully Remote Option:

• Fully remote work schedules are not currently available.

Clearance Sponsorship:

• We will only consider candidates that currently possess an active Secret security clearance or higher at this time.

For consideration, please contact Melissa McNally via

#M1

System One, and its subsidiaries including Joulé, ALTA IT Services, CM Access, and MOUNTAIN, LTD., are leaders in delivering outsourced services and workforce solutions across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan.

System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.