Job Description
Lead Data Engineer | 9+ Years | Bangalore/Hyderabad
We're hiring a Lead Data Engineer to drive our enterprise Data Warehouse modernization to Google BigQuery & GCP!
Location: Bangalore / Hyderabad
Experience: 9+ years in Data Engineering/EDW, with 4+ years in a Lead/Principal role
⚡ Short joiners preferred
What you'll do:
Lead migration & modernization of legacy Oracle EDW/Exadata workloads to BigQuery and modern Lakehouse architectures
Design, build, and optimize large-scale batch & real-time ETL/ELT pipelines using Apache Spark (PySpark/Scala), Python, and Cloud Composer (Airflow)
Implement CDC and continuous replication from Oracle to BigQuery using Oracle GoldenGate, Kafka, and CDC frameworks
Establish end-to-end DataOps automation — CI/CD, data quality/reconciliation checks, and Infrastructure-as-Code (Terraform)
Champion AI/LLM-driven development — using tools like GitHub Copilot, Claude, ChatGPT, and Gemini for code generation, SQL refactoring, testing, and debugging
Set engineering standards, lead code/architecture reviews, and mentor teams on data engineering and AI-augmented workflows
What we're looking for:
✅ 9+ years in Data Engineering/Enterprise Data Warehousing, incl. 4+ years in a lead/principal capacity
✅ Deep expertise in BigQuery — architecture, partitioning, clustering, slot optimization, cost governance & security (policy tags, row/column access policies)
✅ Strong Oracle & SQL background — Exadata, advanced PL/SQL, complex data modeling (Star/Snowflake, Data Vault, SCDs), query tuning
✅ Advanced hands-on experience with Apache Spark (PySpark/Scala) and Python for distributed data processing
✅ Strong experience with Oracle GoldenGate, Kafka, or CDC streaming architectures
✅ Proven track record automating data workflows and CI/CD pipelines (GitHub Actions, Jenkins, Terraform)
✅ Demonstrated hands-on use of AI/LLMs in daily development — prompt engineering for code/SQL generation, automated testing, and code modernization