The Lead Assistant Manager - Data Engineering will be responsible for developing and maintaining data integration solutions, data pipelines, and cloud-based data platforms. The role requires strong expertise in data warehousing, ETL/ELT processes, SQL, Python, and cloud technologies. The individual will collaborate with business stakeholders, analysts, and data science teams to deliver robust data solutions that enable data-driven decision-making.
This position requires a blend of hands-on technical expertise, problem-solving skills, and the ability to lead projects while ensuring adherence to data governance, security, and quality standards.
- Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
- Build and optimize data models, data warehouses, data lakes, and modern data architectures.
- Develop data ingestion, transformation, and integration processes using cloud-native technologies.
- Ensure data quality, integrity, reliability, and governance across enterprise data assets.
- Collaborate with business, analytics, and data science teams to understand and deliver data requirements.
- Implement data pipeline monitoring, troubleshooting, and performance optimization procedures.
- Support real-time and batch data processing requirements.
- Automate data workflows and deployment processes using CI/CD best practices.
- Ensure compliance with security, privacy, and organizational data standards.
- Mentor junior team members and provide technical guidance on data engineering best practices.
- Participate in architecture discussions and recommend enhancements to improve scalability and efficiency.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
- 5-8 years of experience in Data Engineering, Data Warehousing, or related domains.
- Strong proficiency in SQL and Python for data processing and automation.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Hands-on experience with data integration and orchestration tools such as Azure Data Factory, Airflow, Informatica, or similar platforms.
- Knowledge of modern data platforms including Databricks, Snowflake, Redshift, BigQuery, or equivalent technologies.
- Experience with Spark/PySpark and large-scale data processing frameworks.
- Strong understanding of data modeling, ETL/ELT concepts, and database optimization techniques.
- Familiarity with DevOps, CI/CD practices, and version control systems.
- Excellent analytical, problem-solving, stakeholder management, and communication skills.
- Experience leading projects or mentoring team members will be preferred.
