Java Spark Engineer
The Judge Group Inc.

Location: Jersey City, NJ
Salary: $75.00 USD Hourly - $90.00 USD Hourly
Description:
Jersey City NJ

NO C2C

Must have hands on experience :

min. 10 year programing experience

Cloud - AWS

Spark - performance tuning

Machine learning ops

Key Responsibilities:

1. Design & Develop spark data processing pipelines that can apply complex transformations on large volumes of data

2. Help teams with Performance tuning of complex spark jobs processing huge volumes of data in limited time

3. Develop and enhance common data processing frameworks focusing on efficiency, scalability and code reusability

4. Implement and optimize streaming data processing pipelines using Kafka, Flink

5. Provide mentorship and technical guidance to junior team members, promoting best practices in data processing & Streaming

6. Stay current with the latest trends and technologies in Java, Spark, streaming and big data processing, AWS services

7. Lead code reviews, ensuring high coding standards and practices

Qualifications:

1. Bachelor's or Master's degree in computer science, engineering or related field

2. 10+ years of relevant professional experience in software development with a focus on Java, spark, streaming and batch high volume data processing

3. Proven experience in performance tuning of Spark jobs in large-scale data environments

4. Strong background in streaming data technologies and real-time data processing using kafka, spark, flink

5. Experience building common frameworks/libraries, particularly for data processing

6. Exceptional problem solving skills and algorithmic thinking

7. Experience with AWS cloud platform and services (including SNS, SQS, Event bridge, lambda, glue, lake formation etc.)

8. Knowledge of Docker, Kubernetes and other containerization and orchestration tools

9. Candidate little short on above experience can still apply if they have experience in working with MLOps with specific expertise in implementing and managing offline/online/in-line feature stores using vendor products like Sagemaker, Tecton, Feast, Databricks etc.

Contact: bmatey@judge.com

This job and many more are available through The Judge Group. Find us on the web at www.judge.com

This job has expired.