Data Engineer

Company:  Gilder Search Group
Location: Honolulu
Closing Date: 02/11/2024
Salary: £125 - £150 Per Annum
Hours: Full Time
Type: Permanent
Job Requirements / Description

Input Technology Solutions - Honolulu, HI, United States

What you will do:

  • Create and optimize data pipelines from scratch, ensuring scalability, reliability, and high-performance processing.
  • Perform data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets.
  • Leverage big data technologies to efficiently process and analyze large datasets, particularly those encountered in a federal agency.
  • Troubleshoot data-related problems and provide innovative solutions to address complex data challenges.
  • Implement and enforce data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices.
  • Work closely with cross-functional teams to understand data requirements and design optimal data models and architectures.
  • Collaborate with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision-making processes.
  • Maintain documentation for software applications, workflows, and processes.
  • Stay updated with emerging trends and advancements in data engineering and recommend suitable tools and technologies for continuous improvement.

Input Technology Solutions is seeking a Data Engineer to support a program in Honolulu, HI. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our Advana data infrastructure, pipelines, and systems. Proficiency in ETL, Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks is crucial for ensuring efficient data ingest, processing, and analysis. You will develop and manage ETL processes and data pipelines across various government platforms, working with a wide range of data types.

What you need to have:

  • Bachelor's Degree and 0 to 2 years of experience; 6 to 8 years with no degree
  • Clearance: TS/SCI clearance
  • 1+ years of Python experience including developing, running, packaging, and testing Python scripts
  • Experience with distributed version control systems (VCS)
  • Experience with the entire ETL/ELT pipeline, including data ingestion, data normalization, data preparation, and database design
  • Experience with conducting exploratory data analysis to communicate qualitative and quantitative findings to analysts
  • Experience processing and fusing structured and unstructured data
  • Experience with data engineering projects supporting data science and AI/ML workloads
  • Experience creating solutions within a collaborative, cross-functional team environment in team sprint cycles

What we'd like you to have:

  • Experience with using Palantir products for data manipulation, correlation, and visualization
  • Experience with AWS or other cloud computing services
  • Experience with Kafka and NiFi development
  • Experience with containerization tools, including Docker and Kubernetes
  • TS/SCI with Counterintelligence Polygraph

Posted: Thursday, August 29, 2024

Job # 6561

#J-18808-Ljbffr
Apply Now
Share this job
Gilder Search Group
An error has occurred. This application may no longer respond until reloaded. Reload 🗙