Sr Data Engineer

Job Description

Roles and Responsibilities:

• · Need to detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness and business value.
• · Design, architect, solutioning, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.
• · You participate in early state design and feature definition activities, interacting with business stakeholders and TPO to gather requirements.
• · Reviewing team code and development to ensure on optimization of solution and code.
• · Responsible for implementing robust data pipeline using Microsoft, Databricks Stack
• · Responsible for creating reusable and scalable data pipelines
• · Act as Lead Data Engineer, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence.

Critical Skills to Possess:

• Knowledge/Experience with Azure cloud stack.
• · Advanced working knowledge and experience with relational and non-relational databases
• · Experience building and optimizing Big Data data pipelines, architectures and data sets
• · Strong analytic skills related to working with unstructured datasets
• · Strong experience in Azure Data factory, Databricks, Eventhub, Python, PySpark ,Azure Synapse and SQL
• · Azure Devops experience to deploy the ADF pipelines.

Preferred Qualifications:

• BS degree in Computer Science or Engineering or equivalent experience