Data Team Lead

OVERVIEW Acklen Avenue is a trusted partner in outsourced development. We help ambitious companies accelerate software delivery by providing complete agile development teams and staff augmentation. Acklen Avenue offers more than talent. Our dedicated teams offer a mature product development process, staffed with experienced, full\-time experts. Our staff augmentation offering delivers fully vetted and Acklen\-approved individual developers to add the right skills when and where you need them. WHO YOU ARE You’re a seasoned Lead Data Engineer with 5\+ years of experience designing and delivering real\-time streaming solutions in the Azure ecosystem, including deep expertise in Java and Apache Flink (DataStream API and SQL). You have hands\-on experience with stateful processing, checkpointing, watermark strategies, and complex IoT data patterns such as out\-of\-order and missing events, and you can build scalable, high\-performance pipelines using Flink 2\.x with Azure Event Hubs or Kafka. Proficient in Java and SQL, you understand Azure data services, data modeling, governance, and security best practices, and you thrive in Agile environments, leveraging CI/CD and DevOps tools to deliver reliable, cost\-efficient data solutions while promoting strong engineering standards and collaboration. POSITION ACCOUNTABILITIES * Architect, design, and implement scalable and efficient data solutions on Flink. * Design and implement the Flink 2\.x streaming pipeline for IoT. * Develop custom KeyedProcessFunctions to handle "Heartbeat Injection" (detecting sensor silence) and complex windowing. * Implement the Gatekeeper Pattern: A Flink job that applies Data Quality tags (Validity, Completeness) to the stream without dropping data. * Optimize State Backends for high\-cardinality sensor data (100k\+ concurrent sensors). * Work with the "Shared Core" team to integrate libraries into the Flink runtime. * Manage and optimize Azure resources (Flink clusters) for performance, reliability, and cost\-efficiency. * Transform, clean, and prepare data using SQL and Java. * Monitor and fine\-tune workloads and pipelines for optimal performance and reliability. * Maintain clear documentation of solutions, configurations, and workflows. * Actively participate in Agile team activities and continuous improvement initiatives. * Promote and enforce data engineering best practices, including data governance, security, and data quality. POSITION SKILLS \& REQUIREMENTS Must have * 5\+ years of hands\-on data engineering experience with deep expertise in the Azure ecosystem. * 5\+ years in Java (backend) with 3\+ years deep specialization in Apache Flink (DataStream API \& SQL). * Experience with Flink State Management, Checkpointing, and Watermark strategies. * Experience upgrading to or using Flink 2\.0/2\.x. * Experience with Azure Event Hubs or Kafka. * Understanding of "Sensor/IoT" data patterns (out\-of\-order and missing events). * Proficient in Java, SQL, and writing optimized data integration and processing code. * Strong understanding of SDLC and Agile methodologies with hands\-on experience in Azure DevOps, GitHub, CI/CD, and artifact management. * Deep expertise in Azure data services. * Skilled in data modeling, database design, and data warehousing solutions on Azure. * Knowledge of data quality, governance, and security best practices within Azure (AD, NSG, encryption, compliance). Nice to have * Certifications preferred: Azure Fundamentals, Azure Data Engineer Associate, and Azure Solutions Architect Expert (nice to have).