Senior Data Engineer

Job Description

Role: AWS Data Engineer

Experience: 8+ Years

Work location:Hyderabad

Technical Skills:

Proven experience as a Data Engineer or in a similar role. -

Strong proficiency in **SQL** scripting.

Proficiency in **Python** for data manipulation and automation

Experience with **Apache Airflow** for workflow orchestration

Hands-on experience with **AWS Cloud services**, including Lambda, DMS, and Glue.

Solid understanding of data warehousing concepts, with practical experience using **Snowflake**. Enhance Operational Efficiency: Reduce Job Monitoring Time, Decrease Manual Operations.

Strengthen Data and Infrastructure: Improve Data Quality, Accelerate Cloud Migration .

Deepen Architectural and Cross-Functional Understanding:Enhance System Architecture Knowledge, Map Cross-Functional Dependencies

Job Description:

* Design, develop, and maintain **ETL/ELT data pipelines** to ingest data from various sources, including databases and flat files, into our data warehouse.

* Utilize AWS services such as **AWS Lambda**, **AWS DMS**, and **AWS Glue** for data ingestion and transformation processes.

* Develop and optimize **Snowflake stored procedures** to implement complex business logic and data transformations.

* Manage and monitor scheduled data workflows using **Apache Airflow**.

* Work closely with stakeholders to understand project requirements and translate them into technical solutions, ensuring seamless integration with existing data pipelines.

* Implement and maintain robust **data quality checks** to ensure accuracy and reliability of all delivered datasets and reports.

* Adhere to the full project delivery lifecycle, including requirements gathering, design, implementation, documentation, and obtaining user acceptance testing (UAT) approvals.

* Deliver data sets to other AWS accounts and support the creation of Power BI reports.