Java Spark

Skills

Primary Skill Cluster: Spark, Big Data, Hive, Java

Skills outside Primary Skill Cluster:

Job Description:

1. 4+ years with hands-on experience in Java 1.8 and above is a must.
2. Database knowledge and ability to write queries is a must.
3. Hands-on experience with Unix Shell scripting is a must.
4. Knowledge in Spark SQL (Good to have).
5. Understanding of Regulatory reporting (Good to have).
6. Design and implement scalable, fault-tolerant ETL pipelines on Big Data platforms to store and process terabytes of contract information from upstream sources with high availability.
7. Actively work on performance tuning techniques through understanding Spark DAGs on data structures on both relational and Big Data platforms, ensuring high performance for both ETL and reporting components.

Responsibilities:

1. Perform ad-hoc data research and analysis; provide written summaries of results for non-technical business users.
2. Work with the onsite development team and provide necessary overlap coverage to ensure smooth transition and communication between offshore and onsite teams.
3. Collaborate closely with multiple teams (Business Analyst, ETL, DB team, Infra, Support, etc.).
4. Work collaboratively in a small, cross-functional global team.