Z2_Azure Data Engineer with DevOps

Req ID: 388542

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.

We are currently seeking a Z2_Azure Data Engineer with DevOps to join our team in Bangalore, Karnātaka (IN-KA), India (IN).

Key Responsibilities

BI FSL 2.0 Transformation & Solution Architecture

• Lead the end-to-end architecture and execution of BI FSL 2.0 transformation from SAP BW/BW4HANA to neXtollo/Azure Databricks.
• Translate existing SAP BW data models, transformations, process chains, extractors, and reporting logic into scalable Databricks Lakehouse patterns.
• Define the target-state data architecture covering ingestion, bronze/silver/gold layers, semantic consumption, Power BI integration, and reusable data products.
• Design SAP-to-Azure extraction and integration patterns using relevant approaches such as ODP, SLT, BW Bridge, APIs, CDC, or custom ETL frameworks.
• Drive fit-gap analysis between existing SAP BW capabilities and future-state neXtollo architecture.
• Establish architecture standards, design principles, reusable engineering patterns, and governance guidelines for the transformation program.
• Collaborate with business stakeholders, SAP functional teams, platform teams, data engineers, Power BI teams, and program leadership to ensure aligned execution.

Azure Data Engineering, DevOps & Platform Engineering

• Design, build, and optimize enterprise-scale data pipelines using Azure Databricks, PySpark, Delta Lake, Azure Data Factory, and ADLS Gen2.
• Implement CI/CD pipelines for notebooks, workflows, data pipelines, infrastructure, configuration, and deployment across development, test, and production environments.
• Automate infrastructure provisioning and environment setup using Infrastructure as Code, preferably Terraform or equivalent Azure-native tooling.
• Define release management, branching strategy, code review process, deployment gates, rollback approach, and environment promotion standards.
• Implement observability, monitoring, alerting, logging, data quality checks, reconciliation, and operational dashboards for production-grade data platforms.
• Embed DevSecOps practices including secure secret handling, service principals, managed identities, RBAC, Key Vault integration, policy compliance, and audit readiness.
• Troubleshoot migration defects, pipeline failures, data mismatches, performance bottlenecks, and production issues with structured root cause analysis.

Leadership & Delivery Ownership

• Provide technical leadership to data engineers, developers, analysts, and external partners throughout the migration lifecycle.
• Own architecture decisions and ensure alignment with enterprise cloud, security, data governance, and platform standards.
• Support roadmap planning, sprint execution, dependency management, technical risk mitigation, and delivery governance.
• Create solution documents, architecture decision records, migration playbooks, runbooks, and design review material for leadership and engineering teams.
• Act as a bridge between business outcomes and technical implementation, ensuring the platform enables reliable, faster, and future-ready analytics.

Required Technical Skills

Azure, Databricks & Lakehouse Architecture

• Azure Databricks and Databricks Workflows
• Delta Lake, Medallion Architecture, and Lakehouse design
• Azure Data Factory or equivalent orchestration tools
• Azure Data Lake Storage Gen2 and cloud object storage patterns
• Azure SQL Database, Synapse Serverless SQL, or Databricks SQL
• Unity Catalog, data lineage, access control, and data product governance
• Azure Key Vault, Managed Identity, Service Principals, and secure connectivity
• Power BI consumption patterns and semantic layer enablement

SAP BW/BW4HANA Migration & Integration

• Strong understanding of SAP BW/BW4HANA concepts, data models, InfoProviders, transformations, process chains, and reporting dependencies.
• Experience translating SAP BW logic into modern cloud data engineering patterns on Databricks.
• Knowledge of SAP data extraction approaches such as ODP, SLT, CDS Views, RFC/API-based extraction, BW Bridge, or third-party replication tools.
• Ability to assess migration complexity, data lineage, reconciliation needs, and functional equivalence between legacy and target platforms.
• Exposure to SAP functional domains in logistics, supply chain, warehouse, transportation, or aftersales analytics is highly preferred.

Data Technologies

• Python
• PySpark
• SQL/T-SQL
• Spark
• Delta Lake
• Data Modeling
• ETL/ELT Design
• Data Warehousing Concepts
• Performance Tuning

DevOps & Automation

• Azure DevOps, GitHub, or Git-based enterprise source control
• CI/CD pipeline design using YAML pipelines, GitHub Actions, or Azure DevOps pipelines
• Infrastructure as Code using Terraform, Bicep, ARM templates, or equivalent tooling
• Release management, environment promotion, approval gates, and rollback strategy
• Automated testing for data pipelines, data quality validation, and reconciliation frameworks
• DevSecOps controls for secrets, identity, policy, compliance, and audit traceability

Monitoring & Security

• Azure Monitor
• Log Analytics
• Application Insights
• Azure RBAC
• Azure Policy
• Key Vault Integration

Preferred Skills

• Experience with large-scale SAP BW, data warehouse, or BI platform modernization programs.
• Hands-on experience with Databricks Unity Catalog, Delta Sharing, Lakehouse Federation, or data marketplace patterns.
• Experience with near real-time streaming, Kafka, event-driven architecture, or CDC pipelines.
• Exposure to SAP Business Data Cloud, SAP Datasphere, or SAP Databricks concepts.
• Experience with Agile/Scrum delivery, PI planning, roadmap execution, and multi-vendor collaboration.
• Ability to work in a hybrid global team setup across India, Germany, platform teams, and business stakeholders.

Qualifications

• 10+ years of experience in data engineering, data architecture, BI modernization, or enterprise data platform delivery.
• Minimum 5+ years of hands-on experience with Microsoft Azure data services.
• Minimum 4+ years of hands-on experience with Azure Databricks, PySpark, Delta Lake, and Lakehouse implementation.
• Minimum 3+ years of practical DevOps experience for data platforms, including CI/CD and Infrastructure as Code.
• Proven experience delivering complex migration or modernization programs from legacy data warehouses to cl