Senior Big Data Engineer

Explore appropriate tools, platforms, and technologies aligned with organizational standards.

Evaluate and recommend improvements to existing data architecture and processes.

5+ years of experience in data architecture, data engineering, or a related field.

Proficient in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar.

Experience with data governance frameworks and tools (e.g., Collibra, Alation, OpenMetadata).

Strong knowledge of cloud platforms (Azure or Google Cloud), especially with cloud-native data services.

Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms.

Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming to handle high-velocity data streams.

Design /Orchestrate end-to-end automation using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies.

Design intelligent systems that can detect anomalies, trigger alerts, and automatically reroute or restart processes to maintain data integrity and availability.

Develop scalable data architecture strategies that support advanced analytics, machine learning, and real-time data processing.

Define and implement data governance, metadata management, and data quality standards.