Anshuman  ·  Cloud Engineer  ·  5+ yrs

Overview: This project involved designing and implementing a scalable MLOps pipeline on Azure. Responsibilities: Designed and implemented a scalable MLOps pipeline on Azure, enabling seamless integration and deployment of ML models into production. Automated model training and deployment processes using Azure DevOps, reducing deployment time by 40%. Leveraged Databricks and PySpark for efficient data processing and model training, handling large datasets. Implemented CI/CD pipelines with Jenkins and Azure DevOps, ensuring continuous integration and robust deployment. Managed Kubernetes clusters (EKS) for containerized applications, optimizing resource allocation and ensuring high availability.

Overview: This project focused on developing a serverless architecture using AWS Lambda, S3, and Glue. Responsibilities: Developed a serverless architecture using AWS Lambda, S3, and Glue to support real-time data processing and ML model inference. Automated the deployment of ML models using AWS Lambda functions, reducing infrastructure costs by 30%. Integrated CI/CD pipelines with Jenkins for automated testing and deployment of ML models in a serverless environment.

Overview: This project involved designing and developing an end-to-end machine learning pipeline using Databricks and Apache Spark for a financial services client. Responsibilities: Designed and developed an end-to-end machine learning pipeline using Databricks and Apache Spark for a financial services client. Automated ETL processes to ingest and preprocess data from multiple sources, reducing data preparation time by 50%. Implemented a model training and deployment pipeline that integrates seamlessly with Azure services for monitoring and scaling. Set up GitLab CI/CD for continuous integration and deployment, enabling rapid iteration and minimizing downtime. Optimized PySpark jobs to run on Databricks clusters, improving model training speed by 30%.

Overview: This project focused on leading the deployment of AI models using Kubernetes on EKS for predictive analytics solutions. Responsibilities: Led the deployment of AI models using Kubernetes on EKS, ensuring scalable and reliable deployments for predictive analytics solutions. Automated the entire ML pipeline using Jenkins, Docker, and Terraform, reducing manual intervention and increasing deployment efficiency by 60%. Configured Kubernetes clusters to manage containerized workloads and ensure high availability and fault tolerance.

Overview: This project involved developing a hybrid cloud data pipeline integrating Azure and AWS to support large-scale predictive analytics models. Responsibilities: Developed a hybrid cloud data pipeline integrating Azure and AWS to support large-scale predictive analytics models. Automated ETL processes using Apache Airflow and Terraform to manage infrastructure as code, improving deployment speed by 50%. Deployed machine learning models in a multi-cloud environment to leverage the strengths of both Azure and AWS.