Engineer Jobs in Berkeley Heights, NJ

1 job found

Include remote jobs

15d ago

Data Engineer (Ab Initio, OzoneCH, Flink)

Next Gen Software Solutions Berkeley Heights, NJ

Berkeley Heights, NJ

15d ago

$50 - $53/hr Contract Ab Initio Big Data Data Engineer

Apply Now

Job Description

The contract Data Engineer role in Berkeley Heights, NJ onsite focuses on building high-performance, event-driven data pipelines with the Ab Initio ecosystem for a large credit card processing system.

Location: Berkeley Heights, NJ (onsite) • Salary: USD 50 - 53 per hour • Type: Contract

Responsibilities

Architect and develop intricate Ab Initio continuous flow graphs, incorporating stateful joins, sliding time windows, and aggregations.
Construct event-driven data pipelines leveraging Kafka, MQ, and file streams to push updates to dashboards or downstream data stores in near real time.
Guarantee the resilience of continuous flows through checkpointing and recovery to ensure exactly-once processing semantics.
Utilize advanced Ab Initio components such as Reformat, Rollup, Join, and Partition to attain low-latency data processing.
Proactively monitor live production streams to sustain 24/7 reliability and troubleshoot data issues as they arise.
Develop ETL pipelines for both batch and real-time data ingestion and transformation.
Implement and enforce data validation, security, integrity, and compliance across big data platforms.
Diagnose and resolve performance issues in large-scale clusters.
Collaborate with data scientists, analysts, and application teams to deliver robust data solutions.
Automate workflows and improve operational efficiency through scripting and orchestration tools.

Requirements

Strong understanding of credit card processing systems.
Expertise with GDE (Graphical Development Environment), EME (Enterprise Meta Environment), Conduct>It, and Continuous Flows.
Knowledge of Kafka, message queues, and real-time stateful services.
Proficiency in Unix/Linux shell scripting, SQL, and databases such as Oracle and Teradata.
Experience in Java, Scala, Python, or Kafka is a plus.
Familiarity with Linux/Unix environments and shell scripting.
Understanding of data security, governance, and compliance standards.
Experience with cloud-based big data platforms.
Exposure to containerization with Docker and Kubernetes for big data workloads.
Knowledge of CI/CD pipelines for data engineering projects.

Technologies

Ab Initio
GDE (Graphical Development Environment)
EME (Enterprise Meta>Environment)
Conduct>It
Kafka
MQ
Oracle
Teradata
Java
Scala
Python
SQL
Linux
Unix
Shell scripting
Docker
Kubernetes
CI/CD pipelines
Cloud-based big data platforms

Equal Employment Opportunity

Next Gen Software Solutions LLC is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all applicants and employees are treated with respect and dignity. We do not discriminate on the basis of race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, genetic information, veteran status, or any other characteristic protected by applicable federal, state, or local laws.

View full job page →

Get Job Alerts

New jobs delivered to your inbox.