Project description
Our client, a leading manufacturer of high-end household appliances, is expanding into new smart product lines. As part of this strategic initiative, a large-scale global program is being implemented across the company's IP portfolio. This includes the development of new embedded software, enhancements to cloud infrastructure, and the creation of innovative interfaces within the mobile application. As a data engineer you will be responsible for data analytics part for product insights including user data, telemetry. feedback and other sources.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for cloud-based data platforms. Build and support data integrations between Google Analytics and AWS environments. Work with GCP data sources, including user and CRM data, and process/analyze data using BigQuery. Design and maintain AWS data pipelines using Amazon Redshift, Amazon S3, and AWS Glue. Develop data ingestion, transformation, and loading processes for Redshift-based data warehouses. Implement and maintain DynamoDB backup, restore, and data extraction processes. Build reliable data flows between S3, Glue, DynamoDB, and Redshift. Optimize SQL queries, data models, ETL jobs, and Redshift performance for large datasets. Implement data quality checks, monitoring, logging, and error-handling mechanisms across data pipelines. Troubleshoot data pipeline failures, data inconsistencies, and performance issues. Ensure data security, integrity, availability, and appropriate access controls across AWS environment Collaborate with analytics, CRM, application, and business teams to understand data requirements and deliver reliable datasets for downstream consumption. Maintain technical documentation for data pipelines, transformations, data mappings, and operational procedures.
SKILLS
Must have
- Strong hands-on experience as a Data Engineer, with a primary focus on ETL/ELT development and data integration. Strong experience with AWS data services, particularly: Amazon Redshift Amazon S3 AWS Glue Hands-on experience designing, developing, and maintaining ETL/ELT pipelines. Strong SQL skills, including complex queries, data transformations, and performance optimization. Experience working with Amazon DynamoDB, including data extraction and backup/restore processes. Experience with BigQuery. Experience integrating and processing data from multiple sources, including CRM and user/customer data. Good understanding of data warehousing concepts, data modeling, and large-scale data processing. Experience with data quality, monitoring, troubleshooting, and optimization of production data pipelines. Proficiency in Python for data processing, automation, and ETL development. Good understanding of cloud data security, access management, and data protection principles. Strong troubleshooting and analytical skills. Good English communication skills and ability to work with distributed technical and business teams.
Nice to have
Working in an Agile environment and closely collaborating with various departments.
