Lead Data Engineer

Job Description We are seeking an experienced Databricks Architect / Lead Data Engineer to lead the design, development, and delivery of enterprise-scale data platforms on Databricks and AWS. The role combines architecture leadership with hands-on engineering responsibilities, including building scalable Lakehouse solutions, modern data pipelines, and AI/ML workloads.

Key Responsibilities Design and own Databricks Lakehouse architecture on AWS. Lead multiple Databricks initiatives and mentor data engineers. Build and optimize data pipelines using Python, PySpark, SQL, and Delta Lake. Implement Medallion Architecture (Bronze, Silver, Gold layers).

Develop batch, streaming, and near real-time data solutions. Enable AI/ML and Generative AI use cases on Databricks. Establish governance, security, Unity Catalog, RBAC, and operational controls. Integrate Databricks with AWS services including S3, IAM, networking, and data platforms.

Drive CI/CD, automated testing, deployment automation, and observability. Troubleshoot and optimize Spark workloads for performance, reliability, and cost. Required Experience In Data Engineering experience. And hands-on Databricks experience. as a Databricks Architect or equivalent architecture role. delivering AI/ML or Generative AI solutions on Databricks. and AWS experience.

Report this job