Data Science

Your Profile We are looking for experienced Data Engineers to support the design, development and deployment of scalable data pipelines. The ideal candidate will have strong hands-on experience with Azure Data Services and Databricks. They will be expected to: • Design and implement scalable data pipelines using Azure Data Factory, Data Lake, and Databricks ensuring they are optimized for data processing and analytics. • Develop and optimize PySpark and SQL workflows for data transformation and orchestration. • Create and refine processes for data modelling, mining and production to support operational and analytical needs. • Establish and enforce data quality checks and validation routines to maintain high standards of data accuracy and reliability. • Collaborate with AI/ML teams to support RAG-based LLM applications. • Ensure data security and governance using Azure Key Vault and related services. Your Role • Over 6 years of hands-on experience with Azure Data Services including Data Lake, Data Factory, Key Vault and Cognitive Search. • Proficient in Databricks ecosystem including cluster optimization, performance tuning with expertise in Delta Lake, PySpark and orchestrating workflows. • Experience with any relational SQL (SQL Server/Oracle) and NoSQL (MongoDB/DynamoDB) databases including Snowflake along with strong expertise in Python/PySpark for large‑scale data processing. • Experience with real-time systems like Event Hubs, Apache Kafka, Spark-Streaming, etc. • Experience with any Big Data frameworks like Spark/Kafka/ Hive/ Hadoop etc. • Strong programming skills in Python and SQL for data engineering and analytics. • Basic understanding of GenAI concepts including RAG and related AI/ML technologies and experience in generating embeddings for both structured and unstructured data sources would be preferred. • Familiarity with DevOps practices including CI/CD pipelines, automation strategies and experience in technologies like GitHub and Bitbucket will be good to have. • Working knowledge of BI tools (Tableau, Power BI) and data engineering platforms (Microsoft Fabric, Apache Storm, Apache NiFi) for reporting and pipeline setup will be beneficial.