Company:
Diverse Lynx
Location: Weehawken
Closing Date: 22/10/2024
Salary: £125 - £150 Per Annum
Hours: Full Time
Type: Permanent
Job Requirements / Description
Job Title: Java + Databricks Developer
Location: Weehawken, NJ (Onsite)
Type: Contract
Job Description
Responsibilities:
- Must have skills: Java, PySpark, Azure Databricks, Azure PaaS experience.
- 6 to 8 years of IT experience and at least 4+ years in cloud native data engineering solution design/architecture and development.
- Strong hands-on expertise in building data engineering solutions using Java, Apache Spark, and Scala.
- Deep expertise in performance tuning and optimization of Java/Spark/Scala programs.
- Ability to design and build data engineering tasks using Databricks framework.
- Hands-on experience working with Azure Cloud services like Azure Data Factory, Azure Data Lake Storage, Azure Event Hub, Azure SQL, Azure Analytical services, Azure Cosmos DB.
- Expertise in designing and building efficient and scalable data engineering solutions using Massively Parallel Processing (MPP) architecture.
- Experience with micro-batch streaming and continuous streaming processes in Databricks for varying latency requirements using inbuilt Apache Spark modules.
- Data ingestion to one or more Azure Services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.
- Build data pipelines for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL.
- Experience in writing HQL queries in Hive Data Warehouse and performance tuning of HIVE scripts.
- Well-versed with relational and dimensional modeling techniques like Star, Snowflake Schema, OLTP, OLAP, normalization, fact and dimensional tables.
- Proficient in creating Technical Design Documents (TDD).
- Familiar with DevOps principles and experience in CI/CD implementations using Azure native services.
- Strong customer relationship and stakeholder management skills.
- Excellent communication and presentation skills.
Share this job
Diverse Lynx
Useful Links