shivani  ·  Senior Azure Databricks Data Engineer  ·  7+ yrs

Senior
India7+ years experienceremote
Available within 48 hrs

About shivani

Shivani has extensive experience in data engineering, specializing in building enterprise-grade ETL pipelines and managing high-volume datasets. She is adept in cloud-native data architecture and CI/CD automation, with strong skills in stakeholder collaboration. Her technical expertise includes a variety of programming languages and frameworks, making her a versatile asset in any data-driven environment.

Core expertise

PySpark
backend
9/10
Azure Databricks
cloud
9/10
Python
language
9/10
Azure
cloud
8/10
SQ
SQL
language
8/10

Additional skills(21)

Azure DatabricksPySparkGitHub ActionsMSSQLFastAPIPostgreSQLDockerDjangoAzure Data FactoryDelta Lake

Why hire shivani?

Production deploy authorityBuilt scalable data platforms

Led cross-functional data engineering teams to deliver scalable solutions.

Designed and implemented ETL pipelines using Azure Databricks and PySpark.

Optimized data workflows for analytics and business intelligence.

Built Voice Bot + Langchain Chatbot + Recommendation + Text Image + Resume Builder

Project highlights(5)

Appointment Booking Voice BotData Engineer

Overview: Developed a voice bot designed to automate appointment bookings for device allocation and installation. Responsibilities: Developed and integrated the voice bot using the RASA framework to handle appointment booking interactions. Implemented voice integration using Murf AI to provide a natural voice experience for users. Deployed the solution on GCP for scalability.

Azure DatabricksPySparkAzure Data FactoryDelta LakeAzure Synapse AnalyticsAzure Blob StorageSQLGitHub Actions

Enterprise ChatbotData Engineer

Overview: Developed a chatbot for an enterprise that could interact with company documents and employee data. Responsibilities: Developed an LLM-powered chatbot integrated with Retrieval-Augmented Generation to query structured and unstructured data. Built backend APIs using FastAPI and deployed on GCP App Engine for scalable access.

Azure Data FactoryAzure DatabricksPySparkMSSQLLangchainFastAPIAzure Blob StorageMicrosoft Fabric

Personalized Recommendation SystemData Engineer

Overview: Built a recommendation system using hybrid filtering methods to suggest books, music, and movies. Responsibilities: Developed backend logic using Python and implemented model pipelines in AWS SageMaker. Deployed containerized solutions using Docker and integrated with CI/CD pipelines for production workflows.

Azure DatabricksPySparkAzure Data FactoryPostgreSQLAzure SynapseDockerGitHub Actions

Text Image Summarization ModelData Engineer

Overview: Developed a model for summarizing text extracted from images. Responsibilities: Developed an NLP pipeline to extract and summarize text from images using OCR and Hugging Face models. Created modular microservices for text extraction, processing, and result delivery.

Azure Data FactoryPySparkOCRHugging Face TransformersAzure FunctionsGCP Cloud RunGitHub Actions

Resume Builder Web ApplicationData Engineer

Overview: Developed a web application that allows users to create professional resumes and receive job suggestions. Responsibilities: Designed and developed a full-stack application using Python and Django, ensuring system security and user privacy.

Azure Data FactoryAzure DatabricksPySparkPostgreSQLAzure App ServiceDjangoGitHub Actions

7+ years of industry experience

  • Appointment Booking Voice BotData EngineerAzure Databricks · PySpark · Azure Data Factory · Delta Lake +4
  • Enterprise ChatbotData EngineerAzure Data Factory · Azure Databricks · PySpark · MSSQL +4
  • Personalized Recommendation SystemData EngineerAzure Databricks · PySpark · Azure Data Factory · PostgreSQL +3
  • Text Image Summarization ModelData EngineerAzure Data Factory · PySpark · OCR · Hugging Face Transformers +3
  • Appointment Booking Voice BotData EngineerAzure Databricks · PySpark · Azure Data Factory · Delta Lake +4
  • Enterprise ChatbotData EngineerAzure Data Factory · Azure Databricks · PySpark · MSSQL +4
  • Personalized Recommendation SystemData EngineerAzure Databricks · PySpark · Azure Data Factory · PostgreSQL +3
  • Text Image Summarization ModelData EngineerAzure Data Factory · PySpark · OCR · Hugging Face Transformers +3
  • Resume Builder Web ApplicationData EngineerAzure Data Factory · Azure Databricks · PySpark · PostgreSQL +3

Ready to work with shivani?

Onboard within 48 hours. No long hiring cycles, no recruiter middleman.

At a Glance

LocationIndia
Experience7+ years
Work moderemote
Direct hirePossible
Start within48 hours
From$1,581/ month

Single contract. Billed in USD.

Typically responds within 4 business hours.

5-day replacement guarantee
48-hour onboarding, single invoice
Direct chat — no recruiter middleman

Top Skills

PySpark
9/10
Azure Databricks
9/10
Python
9/10
Azure
8/10
SQL
8/10
Seniority signals
Owns production deploysGreenfield architectSystem owner
VerifiedVetted by Witarist
Technical skills assessed & verified
Background & identity checked
English communication verified
Ready to onboard in 48 hours

Not sure if this is the right fit?

Tell us your requirements and we'll match you with the best candidates.

shivani

Data Engineer