AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes is a scalable platform for deploying, serving, and managing machine learning models in production environments.
This instructor-led, live training (online or onsite) is aimed at intermediate-level professionals who wish to operationalize ML models reliably on Kubernetes.
Upon completing this training, participants will gain the skills to:
- Containerize and prepare ML models for efficient deployment.
- Serve models using modern inference frameworks.
- Optimize workloads with autoscaling, GPU configuration, and resource tuning.
- Implement model rollout strategies such as A/B testing and canary deployments.
Format of the Course
- Blended lecture, architectural analysis, and guided discussion.
- Extensive practical exercises with real-world deployment scenarios.
- Hands-on implementation using a live Kubernetes environment.
Course Customization Options
- If you need this course tailored to your environment or toolchain, please contact us to discuss customization options.
Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models with Docker
- Exporting models from TensorFlow and PyTorch
- Versioning and storage considerations
Model Serving on Kubernetes
- Overview of inference servers
- Deploying TensorFlow Serving and TorchServe
- Setting up model endpoints
Inference Optimization Techniques
- Batching strategies
- Concurrent request handling
- Latency and throughput tuning
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Configuring GPU nodes
- NVIDIA device plugin overview
- Resource requests and limits for ML workloads
Model Rollout and Release Strategies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for ML in Production
- Metrics for inference workloads
- Logging and tracing practices
- Dashboards and alerting
Security and Reliability Considerations
- Securing model endpoints
- Network policies and access control
- Ensuring high availability
Summary and Next Steps
Requirements
- An understanding of containerized application workflows
- Experience with Python-based machine learning models
- Familiarity with Kubernetes fundamentals
Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Need help picking the right course?
macao@nobleprog.com or +852 81990613
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale - Consultancy Enquiry
Testimonials (3)
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
How trainer deliver knowledge so effectively
Vu Thoai Le - Reply Polska sp. z o. o.
Course - Certified Kubernetes Administrator (CKA) - exam preparation
The knowledge and exchanges with Augustin
Laurent - L'Office national des vacances annuelles (ONVA)
Course - Docker and Kubernetes
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at advanced-level platform engineers and DevOps professionals who wish to master scaling applications using microservices and Kubernetes.
By the end of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production.
- Apply best practices for security and compliance in a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursKubernetes is a container orchestration platform widely used for managing distributed applications at scale.
This instructor-led, live training (online or onsite) is aimed at advanced-level practitioners who wish to apply AI and machine learning techniques to optimize Kubernetes resource usage, scheduling decisions, and autoscaling strategies.
At the completion of this program, participants will be able to:
- Apply AI/ML models to improve workload scheduling decisions in Kubernetes.
- Use predictive analytics to optimize CPU, GPU, and memory allocation.
- Implement intelligent autoscaling using reinforcement learning and metric forecasting.
- Reduce infrastructure cost and latency through automated resource optimization.
Format of the Course
- Instructor-guided technical presentations and deep-dive discussions.
- Hands-on lab work using real Kubernetes clusters.
- Practical exercises applying AI models to real operational scenarios.
Course Customization Options
- To tailor this course to your platform setup or operational requirements, please contact us for customization.
Certified Kubernetes Administrator (CKA) - exam preparation
21 HoursThe Certified Kubernetes Administrator (CKA) program was created by The Linux Foundation and the Cloud Native Computing Foundation (CNCF).
Kubernetes is nowadys a leading platform used for containers orchestration.
NobleProg have been delivering Docker & Kubernetes training from 2015. With more than 360 successfully completed training projects, we became one of the best known training companies worldwide in field of containerization.
Since 2019 we are also helping our customers to confirm their performance in k8s environment by preparing them and encouraging to pass CKA and CKAD exams.
This instructor-led, live training (online or onsite) is aimed at System Administrators, Kubernetes users who wish to confirm their knowledge by passing CKA exam.
On the other hand, training is focused also on gaining practical experience in Kubernetes Administration, so we recommend taking part in it, even if you don't intend to take CKA exam.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
- To learn more about CKA certification, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka
Certified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursThe Certified Kubernetes Application Developer (CKAD) program has been developed by The Linux Foundation and the Cloud Native Computing Foundation (CNCF), the host of Kubernetes.
This instructor-led, live training (online or onsite) is aimed at Developers who wish to confirm their skills in design, build, configure, and expose cloud native applications for Kubernetes.
On the other hand, training is also focused on gaining practical experience in Kubernetes application development, so we recommend taking part in it, even if you don't intend to take CKAD exam.
NobleProg have been delivering Docker & Kubernetes training from 2015. With more than 360 successfully completed training projects, we became one of the best known training company worldwide in field of containerization. Since 2019 we are also helping our customers to confirm their performance in k8s environment by preparing them and encouraging to pass CKA and CKAD exams.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
- To learn more about CKAD, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-application-developer-ckad/
Certified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at Kubernetes practitioners who wish to prepare for the CKS exam.
By the end of this training, participants will know how to secure Kubernetes environments and container-based applications throughout the different stages of an application's life cycle: build, deployment and runtime.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis 7-day programme provides a comprehensive, hands-on journey into designing, deploying, and operating cloud-native applications using modern DevOps practices.
Participants will explore how to design scalable microservices architectures, optimise container environments, and manage production workloads using Kubernetes. The course covers advanced deployment strategies, GitOps-based automation, and observability practices to ensure system reliability and performance.
A strong focus is placed on real-world operational challenges, including incident response, failure simulation, and root cause analysis. The programme concludes with the use of AI-powered tools to support troubleshooting and accelerate operational decision-making.
By the end of the training, participants will have a clear understanding of how to build, deploy, monitor, and maintain resilient distributed systems in a Kubernetes-based environment.
Introduction to Containers, Kubernetes & OpenShift
35 HoursLearn the fundamentals of containers, Kubernetes and OpenShift in a practical, hands-on training designed for developers, DevOps engineers and IT professionals. Participants will learn how to build containerized applications, deploy workloads, manage Kubernetes resources and use OpenShift to streamline modern application delivery in cloud and hybrid environments.
Docker, Kubernetes and OpenShift 3 for Administrators
35 HoursIn this instructor-led, live training in Macao, participants will learn how to manage Red Hat OpenShift Container Platform.
By the end of this training, participants will be able to:
- Create, configure, manage, and troubleshoot OpenShift clusters.
- Deploy containerized applications on-premise, in public cloud or on a hosted cloud.
- Secure OpenShift Container Platform
- Monitor and gather metrics.
- Manage storage.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led, live training in Macao (onsite or remote), participants will learn how to create and manage Docker containers, then deploy a sample application inside a container. Participants will also learn how to automate, scale, and manage their containerized applications within a Kubernetes cluster. Finally, the training goes on to more advanced topics, walking participants through the process of securing, scaling and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursTraining objectives: Acquire theoretical and operational skills in Docker and Kubernetes.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursEdge AI is a paradigm focused on running machine learning inference close to data sources to achieve low-latency, efficient, and scalable processing.
This instructor-led, live training (online or onsite) is aimed at intermediate-level to advanced-level practitioners who wish to deploy, orchestrate, and optimize AI workloads on Kubernetes-based edge environments.
By completing this course, participants will be able to:
- Set up lightweight Kubernetes distributions for edge deployments.
- Deploy AI inference workloads effectively across constrained edge nodes.
- Manage connectivity challenges and synchronization patterns.
- Optimize performance, storage, and networking for real-world edge scenarios.
Format of the Course
- Guided presentations supported by real-world examples.
- Scenario-based labs and practical edge deployment exercises.
- Hands-on experience with Kubernetes edge frameworks.
Course Customization Options
- To request a customized training tailored to your edge platform needs, please contact us to arrange.
Deploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at engineers who wish to use Helm to streamline the process of installing and managing Kubernetes applications.
By the end of this training, participants will be able to:
- Install and configure Helm.
- Create reproducible builds of Kubernetes applications.
- Share applications as Helm charts.
- Run third-party applications saved as Helm charts.
- Manage releases of Helm packages.
Creating a Service Mesh with Istio and Kubernetes
14 HoursIstio is an open-source service mesh that runs on Kubernetes to provide secure, observable, and manageable connectivity between microservices. By leveraging Istio’s Envoy-based sidecar proxies, teams can enforce policies, secure communications with mTLS, gain deep observability into traffic, and improve reliability at scale.
This instructor-led, live training (online or onsite) is aimed at intermediate-level engineers who wish to deploy, secure, and manage microservices applications using Istio on Kubernetes.
By the end of this training, participants will be able to:
- Install and configure Istio on Kubernetes clusters.
- Understand and apply service mesh concepts including traffic management, security, and observability.
- Deploy microservices applications within an Istio service mesh.
- Secure service-to-service communications with mutual TLS (mTLS) and Zero Trust principles.
- Monitor, trace, and troubleshoot microservices with Prometheus, Grafana, and Jaeger.
- Integrate Istio with Calico for advanced network policies and security.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Introduction to Minikube and Kubernetes
21 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at beginner-level to intermediate-level software developers and DevOps professionals who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Understand the basic concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Macao (online or onsite) is aimed at intermediate-level developers and DevOps engineers who wish to use Minikube as a part of their development workflow.
By the end of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Understand how to deploy, manage, and debug applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Optimize their development process using Minikube's advanced features.
- Apply best practices for local Kubernetes development.