Lead Databricks Engineer

Lead Databricks Engineer Requisitos: NextEra Energy is embarking on a transformative initiative to modernize our IoT infrastructure for our diverse range of power generation assets, including solar, wind, and battery storage systems. This project replaces our current custom\-built CEWIS system with off\-the\-shelf solutions, integrating advanced platforms such as AWS, GE Proficy, Databricks, and ServiceNow. The objective is to create a cohesive and scalable ecosystem that enhances operational efficiency and data integrity. By adopting these new technologies, we aim to scale faster and benefit from improved support, enabling more timely and data\-driven decision making. Our architecture seamlessly combines edge data collection, cloud\-based processing, and advanced analytics, positioning us at the forefront of advancements in the renewable energy sector. We are seeking a highly skilled and motivated Business Analysis and Quality Assurance (BA/QA) Team on a contract basis. The team will be responsible for performing detailed business analysis to inform strategic decisions, ensuring the quality and reliability of software systems, and providing comprehensive testing and validation. This SOW outlines the key responsibilities, qualifications, and expectations for this team. Key Responsibilities * Develop and maintain data processing pipelines using Databricks. * Work with big data technologies to optimize data analysis and pipeline performance. * Collaborate with data science and engineering teams to integrate Databricks solutions into existing systems. * Support data engineers, data scientists, and analysts with advanced querying tools and data visualization services. * Implement and troubleshoot data governance and security best practices. Deliverables * Data processing pipelines in Databricks. * Performance optimization reports. * Documentation on data governance and security best practices. * Data visualization dashboards. General Qualifications * Strong experience in data engineering with data warehousing and big data technologies. * Proficiency in SQL, Python, and PySpark. * Experience with cloud\-based data services, particularly Databricks. * Strong problem\-solving skills and attention to detail.