Senior Data Engineer

Job Title: Senior Data Engineer

About Ganit Inc

At

Ganit

, we are not just data geeks – we are a family of scientists, refiners, and artisans. Our mission is to bridge the gap between intelligence and action, translating data into valuable insights and strategies. We work with businesses to create state-of-the-art solutions that push the boundaries of data science.

Incorporated in 2017, Ganit was founded by professionals with vast experience in consulting, analytics, marketing, merchandise, operations, and delivery.Our company culture revolves around maximizing decision velocity and minimizing decision risk for enterprises ambitious enough to make data their voice

• More about us: www.ganitinc.com

Roles & Responsibilities

• Data Engineering Excellence:

• Design and implement data pipelines using formats like JSON, Parquet, CSV, and ORC, utilizing batch and streaming ingestion.

• Cloud Data Migration Leadership:

• Lead cloud migration projects, developing scalable Spark pipelines.

• Medallion Architecture:

• Implement Bronze, Silver, and gold tables for scalable data systems.

• Spark Code Optimization:

• Optimize Spark code to ensure efficient cloud migration.

• Data Modeling:

• Develop and maintain data models with strong governance practices.

• Data Cataloging & Quality:

• Implement cataloging strategies with Unity Catalog to maintain high-quality data.

• Delta Live Table Leadership:

• Lead the design and implementation of Delta Live Tables (DLT) pipelines for secure, tamper-resistant data management.

• Customer Collaboration:

• Collaborate with clients to optimize cloud migrations and ensure best practices in design and governance.

Educational Qualifications

• Experience: Minimum 6 years of hands-on experience in data engineering, with a proven track record in complex pipeline development and cloud-based data migration projects.

• Education: Bachelor’s or higher degree in Computer Science, Data Engineering, or a related field.

• Skills: Proficiency in Spark, SQL, Python, and other relevant data processing technologies.

• Expertise in on-premises to cloud Spark code optimization and Medallion Architecture.
• Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations.
• Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus).

• Soft Skills:

• Excellent communication and collaboration skills, with the ability to work effectively with clients and internal teams.

Good to Have

• Certifications:

• AWS/GCP/Azure Data Engineer Certification.

• Minimum 6 years of hands-on experience in data engineering with complex pipeline development and cloud-based data migration projects
• Bachelor's or higher degree in Computer Science, Data Engineering, or a related field
• Proficiency in Spark, SQL, Python, and other relevant data processing technologies
• Expertise in on-premises to cloud Spark code optimization and Medallion Architecture
• Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations
• Familiarity with AWS services
• Experience with additional cloud platforms like GCP or Azure
• Excellent communication and collaboration skills, with ability to work effectively with clients and internal teams
• AWS/GCP/Azure Data Engineer Certification