Lead Data Architect / Data Engineer

We are seeking a hands-on Lead Data Architect/Data Engineer for a hybrid assignment with our Sacramento, California-based client, helping define architecture and deliver production-grade pipelines across multiple phases of a Databricks platform. Responsibilities include: Build and maintain automated ingestion and transformation pipelines that bring data from: o public portals and open data sources o REST APIs o internal departmental databases and systems into Delta tables in Databricks. Partner with the Program Manager to establish and maintain: o platform architecture o naming conventions and standards o technical documentation Collaborate with the Data Hub governance group to support decisions on: o platform and data architecture o data sharing patterns o data contracts and data ownership o data quality and reliability standards Design recommendations for protecting sensitive data, including: o row-level and column-level access controls o separation of confidential/internal data from public datasets Improve reliability through: o monitoring and alerting o data quality checks and validation frameworks Enable discoverability and usability by publishing datasets and documentation that support plain-language exploration and AI-enabled analysis tools. Write and maintain skill files (instruction files for AI coding assistants) to enforce consistent pipeline patterns and development standards. Support and enable other departments to build pipelines using shared templates and best practices. Requirements: Strong leadership experience, including executive-level stakeholder engagement 7+ years in data engineering, data architecture, or closely related roles Proven experience delivering large-scale, enterprise data engineering / architecture initiatives Strong hands-on expertise in Python and SQL Experience with Apache Spark or cloud data platforms such as: o Databricks (strongly preferred) o Snowflake, BigQuery, or similar Experience integrating data from REST APIs, databases, and other source systems Excellent communication skills with strong writing and documentation practices Preferred Hands-on Databricks experience with: o Delta Lake o Unity Catalog o Databricks Jobs Experience governing sensitive/confidential data (access control, masking, privacy requirements) Experience operating within a data governance program (stewardship, data contracts, cross-agency sharing agreements) Experience writing skill files, prompts, or instruction sets for AI coding assistants Experience with GIS / location-based datasets