Note : Please share USC/GC profiles only Job Role : Databricks Engineer Location : Houston, TX Long Term Contract Position Overview A Databricks Engineer is responsible for designing, developing, and optimizing large-scale data pipelines and analytics solutions using the Databricks Lakehouse Platform. This role involves working with big data technologies, ensuring data quality, enabling advanced analytics and machine learning workflows, and supporting cross-functional data teams. Key Responsibilities Develop workflows and jobs in Databricks notebooks, Jobs, Workflows, and Delta Live Tables. Strong and proficient in Databricks, DLT (Delta Live Tables) framework and Pyspark Implement real-time and batch data processing solutions. Integrate data from diverse sources (APIs, streaming platforms, databases, cloud storage). Manage and optimize Databricks clusters, pools, jobs, and workspace configurations. Apply best practices for cluster sizing, performance tuning, autoscaling, and cost optimization. Configure role-based access controls, audit logs, and Unity Catalog permissions. Ensure adherence to data privacy, compliance, and governance requirements. Implement secure data ingestion, encryption, and key management.
Read Less
