Shivani is a skilled Python Developer with over 5 years of experience in the IT industry, specializing in AWS cloud services, SQL, and data engineering. She has a proven track record of designing and implementing robust data pipelines and ETL processes, significantly optimizing data processing times. Her expertise includes migrating complex systems to modern AWS-based architectures and managing databases for enterprise applications. Shivani excels in stakeholder communication and has successfully led data migration projects, demonstrating her ability to deliver scalable solutions.
Reduced processing time from 1 day to 5-6 hours for 15 million records by optimizing performance.
Successfully migrated a critical system to a modern AWS-based Python stack including RDS PostgreSQL and AWS Redshift.
Engineered and deployed scalable ETL pipelines processing millions of unstructured social media records.
Executed a strategic migration from Oracle to AWS Redshift, enhancing data management.
Designed and developed interactive reports and dashboards for real-time insights in the ERP system.
Achieved a significant reduction in file processing time from 1 day to 5-6 hours for 15 million records.
Executed a strategic data warehouse migration from an Oracle platform to AWS Redshift.
Overview: Developed a tool to find and generate email addresses using first name, last name, and company domain. Responsibilities: Engineered Python scripts for extracting, converting (CSV/JSON), and loading data into Redshift. Designed and implemented ETL scripts using AWS Lambda, Glue, and crawlers for processing millions of social media records. Developed a framework for cleaning source data and conducted data transformations based on specific scenarios. Deployed AWS services using CloudFormation, Ansible, and Terraform for infrastructure as code. Migrated the system to Python, RDS PostgreSQL, AWS Redshift, AWS Lambda, Glue, and implemented job scheduling with Step Functions. Implemented parallel file processing and optimized long-running queries for enhanced database performance.
Key outcomes:
Achieved a significant reduction in file processing time from 1 day to 5-6 hours for 15 million records by optimizing performance.
Successfully migrated the system to a modern AWS-based Python stack.
Implemented parallel file processing, enhancing throughput.
Overview: Implemented a Strategic ERP system leveraging SQL, Python, and Jaspersoft for advanced reporting and analytics. Responsibilities: Designed scalable and secure database schema and optimized SQL queries, conducting performance tuning. Developed backend logic, integrated ERP modules, and created ETL processes for data integration. Designed and developed interactive reports and dashboards using Tableau and PowerBI. Analyzed data for actionable insights, validated reports, and conducted testing to ensure system quality. Gathered requirements and facilitated stakeholder communication. Managed system deployment, security, and performance.
Key outcomes:
Conducted data analysis and refined data using SQL scripts, creating visualizations with PowerBI.
Maintained a regular ETL dashboard and addressed bugs as required.
Overview: Expertise in data transition from IBM Infosphere DataStage to next-gen products like Talend, Golden Gate, Python, and AWS. Responsibilities: Engineered Python scripts to extract, convert, and load data. Designed and implemented an ETL pipeline for processing data from diverse sources to AWS RDS and CRM systems, storing it in a data lake. Extracted data from multiple sources, applied transformations, and loaded it into the database. Developed and maintained Python scripts for full and incremental data processing. Implemented the entire back-end business logic and requisite database operations. Executed a strategic migration plan, transitioning the Data Warehouse from an Oracle platform to AWS Redshift. Implemented security best practices, safeguarding data and controlling access based on IAM policies.
Key outcomes:
Executed a strategic migration plan, transitioning a Data Warehouse from an Oracle platform to AWS Redshift.
Implemented security best practices for data safeguarding and access control.
Shivani
Python Developer