AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes is a scalable platform for deploying, serving, and managing machine learning models in production environments.
This instructor-led, live training (online or onsite) is aimed at intermediate-level professionals who wish to operationalize ML models reliably on Kubernetes.
Upon completing this training, participants will gain the skills to:
- Containerize and prepare ML models for efficient deployment.
- Serve models using modern inference frameworks.
- Optimize workloads with autoscaling, GPU configuration, and resource tuning.
- Implement model rollout strategies such as A/B testing and canary deployments.
Format of the Course
- Blended lecture, architectural analysis, and guided discussion.
- Extensive practical exercises with real-world deployment scenarios.
- Hands-on implementation using a live Kubernetes environment.
Course Customization Options
- If you need this course tailored to your environment or toolchain, please contact us to discuss customization options.
Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models with Docker
- Exporting models from TensorFlow and PyTorch
- Versioning and storage considerations
Model Serving on Kubernetes
- Overview of inference servers
- Deploying TensorFlow Serving and TorchServe
- Setting up model endpoints
Inference Optimization Techniques
- Batching strategies
- Concurrent request handling
- Latency and throughput tuning
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Configuring GPU nodes
- NVIDIA device plugin overview
- Resource requests and limits for ML workloads
Model Rollout and Release Strategies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for ML in Production
- Metrics for inference workloads
- Logging and tracing practices
- Dashboards and alerting
Security and Reliability Considerations
- Securing model endpoints
- Network policies and access control
- Ensuring high availability
Summary and Next Steps
Requirements
- An understanding of containerized application workflows
- Experience with Python-based machine learning models
- Familiarity with Kubernetes fundamentals
Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Need help picking the right course?
macao@nobleprog.com or +852 81990613
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale - Consultancy Enquiry
Testimonials (1)
I like the Kubernetes part of the training as I had some tasks with K8s pods and ports and it was nice to get a better understanding now. Thank you!
Laura Miron - Betfair
Course - Docker and Kubernetes
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at advanced-level platform engineers and DevOps professionals who wish to master scaling applications using microservices and Kubernetes.
By the end of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production.
- Apply best practices for security and compliance in a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursCertified Kubernetes Administrator (CKA) - exam preparation
21 HoursCertified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursCertified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at Kubernetes practitioners who wish to prepare for the CKS exam.
By the end of this training, participants will know how to secure Kubernetes environments and container-based applications throughout the different stages of an application's life cycle: build, deployment and runtime.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis 7-day programme provides a comprehensive, hands-on journey into designing, deploying, and operating cloud-native applications using modern DevOps practices.
Participants will explore how to design scalable microservices architectures, optimise container environments, and manage production workloads using Kubernetes. The course covers advanced deployment strategies, GitOps-based automation, and observability practices to ensure system reliability and performance.
A strong focus is placed on real-world operational challenges, including incident response, failure simulation, and root cause analysis. The programme concludes with the use of AI-powered tools to support troubleshooting and accelerate operational decision-making.
By the end of the training, participants will have a clear understanding of how to build, deploy, monitor, and maintain resilient distributed systems in a Kubernetes-based environment.
Introduction to Containers, Kubernetes & OpenShift
35 HoursLearn the fundamentals of containers, Kubernetes and OpenShift in a practical, hands-on training designed for developers, DevOps engineers and IT professionals. Participants will learn how to build containerized applications, deploy workloads, manage Kubernetes resources and use OpenShift to streamline modern application delivery in cloud and hybrid environments.
Container Platforms Administration with Docker, Kubernetes and OpenShift
35 HoursLearn how to build, deploy and administer containerized environments using Docker, Kubernetes and OpenShift. This five-day, hands-on training covers container images, Kubernetes workloads, cluster networking, storage, security, monitoring and practical OpenShift administration. Participants gain the skills needed to operate modern container platforms and troubleshoot applications across development and production environments.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led, live training in Macao (onsite or remote), participants will learn how to create and manage Docker containers, then deploy a sample application inside a container. Participants will also learn how to automate, scale, and manage their containerized applications within a Kubernetes cluster. Finally, the training goes on to more advanced topics, walking participants through the process of securing, scaling and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursTraining objectives: Acquire theoretical and operational skills in Docker and Kubernetes.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursDeploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at engineers who wish to use Helm to streamline the process of installing and managing Kubernetes applications.
By the end of this training, participants will be able to:
- Install and configure Helm.
- Create reproducible builds of Kubernetes applications.
- Share applications as Helm charts.
- Run third-party applications saved as Helm charts.
- Manage releases of Helm packages.
Creating a Service Mesh with Istio and Kubernetes
14 HoursIntroduction to Minikube and Kubernetes
21 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at beginner-level to intermediate-level software developers and DevOps professionals who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Understand the basic concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at intermediate-level developers and DevOps engineers who wish to use Minikube as a part of their development workflow.
By the end of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Understand how to deploy, manage, and debug applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Optimize their development process using Minikube's advanced features.
- Apply best practices for local Kubernetes development.