Kirtan  ·  Data Engineer  ·  7+ yrs

Senior
India7+ years experienceremote
Available within 48 hrs

About Kirtan

Kirtan is a highly motivated Data Engineering leader with over 7 years of experience in data-driven organizations. He applies his leadership and technical skills to guide teams in building robust data pipelines and infrastructure. Kirtan has a proven track record of delivering scalable data solutions, optimizing data workflows, and ensuring data integrity. He collaborates effectively with cross-functional teams, translating business needs into technical solutions.

Core expertise

AWS
cloud
9/10
Python
language
9/10
AG
AWS Glue
devops
9/10
PySpark
language
8/10
Snowflake
database
8/10
DB
DBT
devops
8/10
AR
Amazon Redshift
database
8/10
AA
Apache Airflow
devops
8/10

Additional skills(24)

PySparkReactPythonSnowflakeOracleReact.jsAmazon RedshiftAWS GlueAWS IoT CoreAWS Kinesis

Why hire Kirtan?

Production deploy authorityMentored juniors

Successfully migrated 10 TB of data to Amazon Redshift with zero downtime.

Architected and implemented robust real-time data ingestion pipelines using AWS IoT Core and AWS Kinesis.

Led data processing initiatives using DBT and Airflow for end-to-end workflow optimization.

Integrated data from 50+ APIs and 5+ file storage locations, automating ingestion workflows.

Project highlights(7)

Modernizing Data Warehouse – Lead Data Engineer

Overview: Led a strategic migration from a legacy on-premises data warehouse to Amazon Redshift to enhance scalability, performance, and cost-efficiency. Responsibilities: Defined migration strategy and roadmap, assessed data compatibility and performance requirements, optimized ETL pipelines for cloud-native environment using AWS Glue.

Amazon RedshiftAWS GluePySparkReact

Key outcomes:

  • Successfully migrated 10 TB of data to Amazon Redshift with zero downtime.

  • Achieved significant performance improvements and cost reductions.

Real-Time Fuel Consumption Optimization Pipeline – Data Engineer/Architect

Overview: Developed a scalable real-time data pipeline to optimize fuel consumption within a glass factory by leveraging various sensors' data. Responsibilities: Architected and implemented a robust real-time data ingestion pipeline using AWS IoT Core and AWS Kinesis.

AWS IoT CoreAWS KinesisAWS GluePython

Key outcomes:

  • Optimized fuel consumption within a glass factory.

  • Successfully processed and stored high-volume data.

End-to-End Data Engineering Workflow Optimization – Lead Data Engineer

Overview: Led initiatives to optimize the end-to-end data engineering workflow, focusing on efficiency and automation. Responsibilities: Utilized DBT for data processing and testing, and Airflow for orchestration, ensuring seamless data flow from raw to aggregated layers.

SnowflakePySparkDBTAirflow

Key outcomes:

  • Optimized end-to-end data engineering workflow.

  • Enhanced workflow automation and data integration using Airflow and DBT.

Creation of MVP Framework for Data Migration using AWS Glue and Amazon EMR – Data Engineer/Architect

  • Developed an MVP framework for seamless data migration using AWS Glue for workflow orchestration and Amazon EMR for scalable data processing.
  • Leveraged Amazon Redshift for high performance and reliability, aiming for a scalable, efficient, and cost-effective data migration framework to modernize data infrastructure and optimize cloud-based analytics.
  • Designed and developed complex data pipelines using AWS Glue, ensuring efficient data extraction, transformation, and loading.
  • Implemented robust data migration strategies for minimal downtime and data integrity.
  • Developed, optimized, and managed ETL processes for loading data into Delta tables, focusing on performance and scalability using PySpark.
  • Leveraged Amazon Redshift for real-time data processing and continuous data validation.
  • Implemented and managed incremental and full load pipelines in AWS Glue, accommodating various source systems such as SQL DB, Oracle, and SFTP.
  • Implemented and monitored CDC (Change Data Capture) and SCD (Slowly Changing Dimensions) Type 1 and Type 2 to maintain historical accuracy and data integrity.
PySparkAmazon RDSAWS GlueAmazon EMRAmazon RedshiftOracleSFTPSQL

Key outcomes:

  • Developed a scalable, efficient, and cost-effective data migration framework.

  • Ensured minimal downtime and data integrity during transitions.

  • Maintained historical accuracy and data integrity using CDC and SCD.

Build a Scalable Data Pipeline for Customer Churn Prediction – Data Engineer/Architect

Scalable Data Pipeline for Customer Data — Salesforce + JIRA + AppInsights to data lake.

AWS Lake FormationSQLPythonPySparkSnowflakeReact.jsAWS GlueAmazon S3Amazon EMR

Key outcomes:

  • Successfully integrated data from multiple sources (Salesforce, JIRA, AppInsights).

  • Enhanced data quality through advanced cleaning and preprocessing.

  • Provided data for a customer churn prediction model.

7+ years of industry experience

Manufacturing & Industrial1 project
  • •Real-Time Fuel Consumption Optimization Pipeline— Data Engineer/ArchitectAWS IoT Core · AWS Kinesis · AWS Glue · Python
SaaS / B2B3 projects
  • •End-to-End Data Engineering Workflow Optimization— Lead Data EngineerSnowflake · PySpark · DBT · Airflow
  • •Creation of MVP Framework for Data Migration using AWS Glue and Amazon EMR— Data Engineer/ArchitectPySpark · Amazon RDS · AWS Glue · Amazon EMR +4
  • •Creating a Data Platform for Analytics— Data EngineerPython · PySpark · AWS EMR · Azure KMS +7
Logistics & Supply Chain1 project
  • •Modernizing Data Warehouse— Lead Data EngineerAmazon Redshift · AWS Glue · PySpark · React

Ready to work with Kirtan?

Onboard within 48 hours. No long hiring cycles, no recruiter middleman.

At a Glance

LocationIndia
Experience7+ years
Work moderemote
Direct hirePossible
Start within48 hours
From$2,228/ month

Single contract. Billed in USD.

Typically responds within 4 business hours.

5-day replacement guarantee
48-hour onboarding, single invoice
Direct chat — no recruiter middleman

Top Skills

AWS
9/10
Python
9/10
AWS Glue
9/10
PySpark
8/10
Snowflake
8/10
Seniority signals
Owns production deploysGreenfield architectSystem ownerCode reviewerMentor / leads juniors
VerifiedVetted by Witarist
Technical skills assessed & verified
Background & identity checked
English communication verified
Ready to onboard in 48 hours

Not sure if this is the right fit?

Tell us your requirements and we'll match you with the best candidates.

Kirtan

Data Engineer