Company:
SARIAN Co
Location: Brooklyn
Closing Date: 04/11/2024
Hours: Full Time
Type: Permanent
Job Requirements / Description
Lead Data Engineer
Location: Brooklyn, OH
Fulltime
Skill: Lead Data Engineer
Location: Brooklyn, OH
Fulltime
Skill: Lead Data Engineer
- Proven big data experience, either from an implementation or a data science prospective.
- Knowledge of Scala and Hive which requires coding experience using parquet 4.
- Advanced SQL knowledge requiring writing SQL queries.
- Experience having worked with GCP Data Lake.
- Experience in working in an agile environment.
- Expert knowledge of at least one big data technology such as Spark, Hadoop, or Elasticsearch.
- A strong coding background in either Java, Python or Scala.
- Experience with testing libraries of common programming - for example scala test (or equivalent packages for other languages).
- Knows the difference between different test types (unit test vs integration test for example).
- Can explain unit testing and how they have been automated.
- Experience of building data processing pipelines for use in production "hands off" batch systems, including either traditional ETL pipelines and/or analytics pipelines.
Share this job
SARIAN Co
Useful Links