We are seeking a highly experienced Senior Data Engineer to design, build, and operate robust Databricks and Lakehouse data platforms. This role will focus on enabling analytics, AI, and Generative AI applications by delivering high-quality, governed, and scalable data assets. Working within product-aligned squads, you will collaborate with AI Engineers, Product Owners, and analytics teams to support cutting-edge Gen AI use cases, including Retrieval Augmented Generation (RAG), and contribute to the development of AI Engineers. Key Responsibilities
Design, build, and operate data solutions using Databricks components such as Delta Lake, Databricks Jobs and Workflows, and Unity Catalog. Develop production‑grade data pipelines using Python, SQL, and Apache Spark. Implement automated testing, CI/CD practices, and ensure data solutions are observable, resilient, performant, and cost‑efficient. Data Enablement for AI/Gen AI
Enable data consumption for Generative AI use cases (RAG, AI services, agent workflows), analytics, reporting tools, and downstream systems. Support feature‑style and curated data access patterns required for AI and Gen AI workloads. Build data pipelines to feed Generative AI applications, including curated knowledge datasets, structured/semi‑structured data, and metadata/lineage for AI consumption. Implement data patterns essential for Gen AI, such as RAG, context/prompt data preparation, and model input/output/feedback data flows. Collaboration and Governance
Work as a senior individual contributor within a cross‑functional product squad. Collaborate closely with Product Owners, AI/ML Engineers, Analytics teams, and platform/security teams. Provide engineering input into design discussions and delivery decisions, supporting peer reviews and shared engineering standards. Ensure data solutions comply with enterprise security, risk, and governance standards. Support operational stability, participate in incident resolution, root cause analysis, and maintain documentation. Required Skills and Qualifications
10-15 years of industry experience. Proven experience as a Senior/Lead Data Engineer (5+ years). Hands‑on experience with Databricks environments (2+ years). Strong understanding of enterprise data lake and lake house architecture (5+ years). Proficiency in Python, SQL, and Apache Spark (5+ years). Experience building and operating production‑grade data platforms (3+ years). Experience working in enterprise or regulated environments (5+ years). Preferred Qualifications
Experience enabling AI, ML, or Generative AI use cases from a data engineering perspective. Familiarity with RAG data patterns, feature‑style or AI‑serving datasets, and vector or embedding‑ready data workflows. Experience working in Agile, product‑aligned squads. Exposure to cloud‑native data platforms (AWS or Azure).
