Senior Data Engineer

Role Overview Alight Solutions is looking for a Senior Data Engineer to join their Chennai team and take ownership of building scalable, secure, and high-performance data pipelines across hybrid platforms. This role demands deep expertise in both traditional Hadoop ecosystems and modern AWS cloud-native architectures, supporting analytics, BI, ML, and AI initiatives at an enterprise scale. The ideal candidate is self-driven, capable of working independently, and able to provide technical leadership and mentorship to the broader team. Key Responsibilities - Design, build, and maintain high-volume ETL and ELT pipelines across Hadoop (HDFS, Hive, Spark, Kafka) and AWS services including Glue, EMR, Lambda, Step Functions, and Redshift - Develop distributed data processing solutions using PySpark, Spark SQL, and cloud serverless patterns - Implement reusable data ingestion frameworks for both batch and streaming workloads using Sqoop, Kafka, and Kinesis - Optimize data workflows through partitioning, bucketing, compression, and efficient file formats such as Parquet and ORC - Design and maintain hybrid data lake architectures combining S3 and HDFS with governance tools like Atlas, Ranger, and Lake Formation - Create data flow diagrams and perform data modelling including star and snowflake schema design - Manage job orchestration using Airflow, Control-M, Step Functions, or event-driven triggers - Lead architecture decisions, conduct code reviews, and provide technical guidance to peers and junior engineers - Own deployment, incident response, and post-incident reviews for production environments - Optimize AWS resource costs and recommend architecture improvements in line with security and compliance standards Required Qualifications - 5 to 8 years of hands-on experience with the Hadoop ecosystem including HDFS, Hive, Spark, YARN, and Kafka - Strong proficiency in Scala, PySpark, Spark optimization, HiveQL, and distributed computing - Solid SQL experience in Hive and Impala - Good working knowledge of the AWS data stack including S3, Glue, EMR, Lambda, Kinesis, Redshift, and Step Functions - Proficiency in Python and Shell scripting - Strong experience with CI/CD practices, GitHub, and Git workflows - Sound understanding of data modelling, partitioning strategies, and schema evolution - Hands-on experience with workflow orchestration tools such as Airflow or Control-M - Familiarity with AWS networking concepts including VPC, subnets, routing, and security groups - Experience with containerization technologies like Docker and ECS or EKS - Strong problem-solving, analytical, and stakeholder communication skills Why Join Us Alight offers a comprehensive total rewards package, ongoing learning opportunities, and a globally inclusive culture that champions diversity and employee growth. This is an excellent opportunity to work on enterprise-scale data platforms and make a meaningful impact on the lives of millions of people served by Alight's benefits ecosystem.

- 5 to 8 years of experience with Hadoop ecosystem (HDFS, Hive, Spark, YARN, Kafka) - Strong hands-on skills in PySpark, Scala, Spark optimization, and HiveQL - Proficiency in SQL using Hive and Impala - Working knowledge of AWS data services: S3, Glue, EMR, Lambda, Kinesis, Redshift, Step Functions - Scripting skills in Python and Shell - Experience with CI/CD pipelines, GitHub, and Git commands - Knowledge of data modelling techniques including star and snowflake schemas - Hands-on experience with Airflow, Control-M, or similar orchestration tools - Understanding of AWS networking (VPC, subnets, security groups, NACLs) - Familiarity with Docker and container orchestration (ECS/EKS) - Strong analytical, communication, and problem-solving abilities - AWS certifications (Data Engineer, Solutions Architect, or Developer) are a plus - Healthcare domain knowledge is an added advantage

- Comprehensive health coverage options - Wellbeing and employee support programs - Retirement and savings plans - Paid vacation and sick leave - Maternity, paternity, and adoption leave - Continuing education and professional training - Voluntary benefit options - Competitive total rewards package