Azure Data Engineering-Technical Lead

Job Description

Experience Requirements
Overall 8-10 years of IT experience preferably in Data and Analytics implementation.
4-5 years of Data engineering (Both On-prem and Azure) delivery experience including Architecture / analysis / development / configuration/ deployment.
4 years of implementation experience in Azure Data engineering projects
Mandatory to have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions on Azure Cloud.
Expert in Python/SQL/SparkSQL/PySpark
Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, Datra bricks, perview and Azure App Service is required.
Extensive hands-on experience implementing data migration and data processing using Azure services: Serverless Architecture, Azure Storage, Azure SQL DB/DW, Data Factory, Azure Stream Analytics, Azure Analysis Service, Databricks Azure Data Catalog, Cosmos Db, Azure Functions, ARM Templates, Azure DevOps, CI/CD etc.
Expert in at least one distributed data processing framework: Spark (Core, Streaming, SQL), Storm or Flink, etc.
Expert in distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc) and good to know traditional ETL tools like Informatica, SSIS, etc
Should have worked on any NoSQL solutions like Mongo DB, Cassandra, HBase, Cosmos DB etc, or Cloud-based NoSQL offerings like DynamoDB, Big Table, etc.
Good Exposure in development with CI / CD pipelines. Knowledge of containerization, orchestration, and Kubernetes engine would be an added advantage.
Designing and building of data pipelines using API ingestion and Streaming ingestion methods.
Knowledge in Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, Power BI is desirable.
Strong problem solving and troubleshooting skills.
Experience in working in a fast-paced agile environment.
Strong experience in common data warehouse modelling principles including Kimball, Inmon.
Experience with data visualization tools such as Power BI.
Experience in Streaming and batch architecture (e.g., Kafka, Kafka streams, Spark, Flink)

Knowledge Requirements
Excellent understanding of data technologies landscape/ecosystem (Both On-prem and Cloud)
Detailed knowledge of Software development essential.
Detailed knowledge of SQL (Structured Query Language) programming is essential.
Proven experience in business and technical problem-solving skills is essential.
Proven ability to build, manage and foster a team-oriented environment
Proven ability to work creatively and analytically in a problem-solving environment
Excellent communication (written and oral) and interpersonal skills
Excellent organizational, multi-tasking, and time-management skills