Lead Data Engineer with Python (Databricks, AWS)

We are seeking a motivated and self\-driven **Lead Data Engineer** with a strong technical background in data processing and manipulation. The ideal candidate will be responsible for working with data ingestion from various sources, transforming it into robust datasets, and managing it effectively — primarily in cloud storage environments. If you have expertise in Python\-based data processing, a keen interest in working with cutting\-edge tools like Databricks, and a passion for working with complex, large\-scale data, we want to hear from you! **Responsibilities** * Develop and maintain reliable data pipelines for ingesting, transforming, and storing data from multiple sources * Manipulate complex datasets and preprocess large\-scale logs and time\-series data using state\-of\-the\-art tools and frameworks * Leverage your knowledge of Python (Pandas, NumPy, etc.) to perform exploratory data analysis (EDA) and prepare data for business and analytical use cases * Work collaboratively with cloud\-based tools and infrastructure, focusing on scalability and performance optimization * Utilize tools like Databricks, if experienced, or demonstrate a willingness to quickly ramp up on Databricks’ Lakehouse, Workflow, and ETL functionalities **Requirements** * Proven experience of over 5 years as a Data Engineer or in a similar role, demonstrating a self\-driven and problem\-solving mindset * At least 1 year of relevant leadership experience * Expertise in Python for data processing, analysis, and manipulation (e.g., Pandas, NumPy, and advanced libraries) * Proven experience working with large\-scale logs, time\-series data, and structured/unstructured data in diverse formats, preferably hardware/device data logs * Knowledge of Databricks, including workflows, ETL processes, and working with Lakehouse architecture, is highly desirable * Experience working with device data manipulation, such as medical data, IoT device data, hardware data, or similar * Excellent command of written and spoken English (B2\+ level) **Nice to have** * Exposure to PyTorch or TensorFlow for advanced data processing and AI/ML tasks * Understanding of or experience with AI/ML models, along with the ability to apply predefined machine learning models to datasets