Kubernetes Notes-I

What Kubernetes solve?

Container are perfect to host small applications like micro-services. Rise of technology now leads to 100 to thousands of containers 

Increased usage of containers in different environment is difficult to manage using scripts or some tools

That specific scenario caused to have a proper technology to maintain these containers

What feature orchestration tool offer?

High availability or no downtime of application

Scalability or high performance. Scale down or scale up application as per the user load. This provides more flexibility 

Disaster recovery - backup and restore. In case of break down of infrastructure there should be some mechanism to pick up the data, restore the data to the latest state.

So that application does not loose any data. Also, the containerised application can ran from the latest state after the recovery. 

All of above functionalities are offered by Kubernetes.

Kubernetes architecture

Kubernetes cluster is at least made of one master node and connected to couple of worker nodes.

Node can be a virtual or a physical machine

Each node has Kubelet process(node agent) running on it.

Kubelet is a kubernetes process that makes it possible for the cluster to communicate to each other and execute some tasks on those nodes like running some application process.

Each worker node has container of different applications deployed on it. So depending on how workload is distributed you will have different number of docker container running on worker nodes.

Worker nodes are the one where actual work is happening. On worker nodes your applications are running.

Master node runs several Kubernetes processes that are absolutely necessary to run manage the cluster properly

One of such process is an API server which also is a container. An API server is actually the entry point to the Kubernetes cluster so this is the process which the different Kubernetes clients will talk to

Like UI if you’re using Kubernetes dashboard an api if you’re using some scripts and automating technologies and command line tool.

So all of these will talk to API server. 

Another process that is running on master node is a controller manager which basically keeps an overview of what is happening in the cluster whether something needs to be repaired or maybe if a container died and it needs to be fixed.

Another process is Schedular basically responsible for scheduling containers on different nodes the workload and the available server resources on each nodes so its ’s an intelligent process that decides on which worker node the next container should be scheduled based on the available resources the worker nodes and the load that container needs 

And another very important component of whole cluster is an etcd key value storage which basically holds at any time the current state of the Kubernetes cluster so it has all the configuration data inside and all the status data of each node and each container inside of that node and the backup and restore that we mentioned earlier is actually made from these etcd snapshots because you can recover the whole cluster state using that etcd snapshot.

Another very important component which enables both master and worker nodes to talk to each other is virtual network that spans on all the nodes that are part of the cluster and in simple words virtual network actually turns all the nodes inside of the cluster into one powerful machine that has the sum of all the resources of individual nodes 

Actually worker nodes bear higher load as they are running applications on inside of it usually are much bigger and have more resources because they will be running hundreds of containers inside them whereas master node will be running just a handful of master processes like we  like we discussed earlier.

However master node is much more important than the individual nodes because for example if you lose a master node access you will not be able to access the cluster anymore and that means you should have backup of your master at any time so in production environments usually you would have at least two masters inside of your Kubernetes cluster but in more cases of course you’re going to have multiple masters where if one master node is down the cluster continues to function smoothly because you have other master available.

In summary:

API server: Entrypoint to K8s cluster

Controller Manager: Keeps track of what’s happening in the cluster

Scheduler: ensures Pods replacement

Etcd: Kubernetes backing store

Control Plane Nodes: handful of master process but of more important

Worker nodes: higher workload, much bigger and more resources

Main Kubernetes components:

Considering web app and data base as example.

Pod: 

smallest unit in Kubernetes

It is abstraction over container. Basically what pod does is it creates this running environment or a layer on top of the container and the reason is because Kubernetes wants to abstract away from the container runtime or container technologies so that you can replace them if you want to and also you don’t have to directly work with docker or whatever container technology you use in a Kubernetes so you can only interact with the Kubernetes layer so we have an application pod which is our own application and that will maybe use a database pod with its won container and this is also important concept here as pod is usually meant to run one application container inside it. You can also run multiple containers inside one pod but basically its only the case if you have one main application container and the helper container or some side service that has to run inside of that pod.Kubernetes offered out of the box virtual network which means that each pod gets its own IP address not the container.

And each pod can communicate with each other using that IP address which is an internal IP address not public so my application container can communicate with database using the IP address.

Summary:

Smallest unit in Kubernetes

Abstraction over container

Usually 1 application per pod

Each pod gets own its own IP address.

New IP address on re-creatio

Service and Ingress:

In case of pod die/crash, new one will get created in its place and when that happens it will get assigned a new ip address which is obviously inconvenient if you are communicating with the database using the IP address because you have to adjust it everytime pod restarts and  because of that another component of Kubernetes called service is used. So service is basically a static IP address or permanent IP address that can be attached so to say to each pod my app will have its own service and database pod will have its own service so even if pod dies the service and its IP address will stay so you don’t have to change that endpoint anymore.

Now to access the web app we have to create an external service. It is basically a service that opens the communication from external resources but again you wouldn’t want your database to be open to public requests and for that you would create something called internal service so this is a type of service that you specify when creating one 

There is another component of Kubernetes called ingress so instead of service the request first goes to ingress and it does the forwarding to service.

Summary:

Permanent ip address

Life cycle of Pod and service not connected

ConfigMap & Secret:

As we said pods communicate with each other using service so my application will have database endpoint say mongoldb service that it uses to communicate with the database but whether you configure usually this database url or endpoint in some application properties files or external environmental variable. Usually its inside of the built image of the application but problem is if endpoint service name or something change to mongoldb you would have to adjust that url in the application 

So usually you have to re-built the application with new version and have to push it to the repository and have to pull new image in pod and restart thing but this is tedious task for small change.

Like database url so for this purpose Kubernetes has a component called config map, it’s basically your external configuration to your application so configmap would usually contain configuration data like urls of a database or some other services that you use

And in Kubernetes you just connect it to the pod so that pod actually gets the data that config map contains.

It has another component called secret to store secret data credentials for example not in a plain text format but in base 64 in encoded format 

And to encrypt that there are tools deployed in Kubernetes that will make secrets secure.

Summary:

External configuration of your application

Configmap is for non-confidential data only

Secret is used to store secret data

Reference secret in deployment/pod. Use it as environmental variables or as a properties file.

Volume:

If database or the pod gets restarted the data would be gone and that’s problematic

So database data or log data to persisted reliably for long term we use component called volume.

So how it works that basically attaches a physical storage on a hard drive to your pod and that storage could be either on local machine meaning on the same server node where the pod is running or it could be on a remote storage ,meaning outside of the Kubernetes cluster can be in cloud storage or could be on-premise storage and just have reference to it.

Kubernetes cluster explicitly doesn’t manage any data persistence which means you as a Kubernetes user is responsible for backing up the data replicating and managing it and making sure it’s kept in a proper hardware.

Summary:

Storage on local machine

Or remote, outside of the K8s cluster

Deployment and StatefulSet:

What happens if my application pod die/crash? To avoid downtime we are replicating everything on multiple servers 

So we would have another node where a replica or clone of our application would run which will also be connected to the service.

As said earlier service is like persistent static IP address but also is a load balancer that will catch the request and forward to whichever part is least busy

In order to create second replica of application pod we don’t create second pod instead we define the blueprint for my application pod and specify how many replicas of that pod you would like to run 

And that component or that blueprint is called deployment. So in reality we will be creating deployment not pods. Deployment is like abstraction of Pods.

What if DB pod dies. We can’t replicate DB pod by deployment as they might be sharing different shared storage.

So this is done using Statefulset. All data base applications like MySQL, elastic and mongoDB are created using StatefulSet.

This makes sure database read and write are synchronised. As it is difficult to maintain stateful service, most of the time DB are often hosted outside of Kubernetes cluster.

Summary:

Deployment is for stateless apps

StatefulSet is for stateFUL Apps or database

Main Kubernetes Components summarized:

Pod: abstraction of containers

Service: communication

Ingress: route traffic into cluster

ConfigMap and Secret for external communication

Volume for data persistence

Deployment and Stateful for replication.

Kubernetes Configuration:

All the configuration in Kubernetes goes through the master node with the process called api server

Kubernetes client would be ui or an api which could be a script or a curl command or cmd tool like kubectl

And these requests have to be either in yaml or json format.

Every configuration file has 3 parts.

• Metadata

• Specification

• Status: Automatically generated and added by Kubernetes.

The way status works is that Kubernetes will always compare what is the desired state and what is the actual state. If no match then Kubernetes knows there’s something to be fixed and it will try to fix also knows as self-healing feature

Where does K8s get this status data?

From etcd. Etcd holds the current status of any K8s component

Format of the configuration file is yaml. Yaml is human friendly data serialisation standard for all programming languages

Store the config file with your code. It will be part of Infra or own git repository.

MiniKube and Kubectl:

=======

In a production cluster setup 

• we will have multiple worker nodes and master nodes

• Separate virtual or physical machines representing each node

To test something locally on such setup is very tedious or almost impossible.

So Minikube is basically one node cluster where the master processes and the worker processes both run on the one node and this node will have a docker container runtime pre-installed so you will be able to run the containers or pods with container on this node.

Kubectl is a command line tool for Kubernetes cluster. Kubectl is most powerful of all 3 clients(UI,API,CLI)

Kubectl is used to communicate with any type of Kubernetes cluster not only Minikube cluster

Minikube can run either as a container or Virtual Machine on your laptop.

Minikube has docker pre-installed to run the containers in the cluster.

Driver means we are hosting Minikube as a container on our local machine. 

We have 2 layers of Docker:

1. Minikube runs as a docker container

2. Docker inside Minikube to run our application containers.

Kubectl CLI is for configuring the Minikube cluster

Minikube CLI is for start up/deleting the cluster.

For testing web applications and DB app

Create 4 K8s config yaml file

1. ConfigMAP: MongoDB endpoint

2. Secret: MongoDB user & Pwd

3. Deployment and service: MongoDB application with internal service

4. Deployment and service: Our own WebAPP with external service

  

      

Kubernetes MCQ

_________ service is automatically created for you k8s cluster creation and takes care of the internal routing of the cluster?

1. NodePort

2. ClusterIP

3. Headless

4. Load Balancer

Which one of the following can be considered as the primary data store of Kubernetes?

1. Pod

2. Etcd

3. Kubelet

4. Kube-scheduler

How to run a POD on a particular node?

1. node Name

2. nodeSelector

3. node affinities

4. All of The Above

In Kubernetes PVC stands for?

1. Present Volume Claim

2. Persistent Volume Claim

3. Persistent Volume Class

4. None of the above

What are some important functionalities of a Namespace in Kubernetes?

1. Namespaces help pod-to-pod communication using the same namespace.

2. Namespaces provide logical separation between the teams and their environments.

3. Namespaces are virtual clusters that can sit on top of the same physical cluster.

4. All of the above

The command to create Kubernetes service is?

1. Kubectl expose

2. Kubectl set service

3. Kubectl run

4. Kubectl deploy

Which of the followings are the Kubernetes Node Components?

1. Docker

2. Kubelet Service

3. Kubernetes Proxy Service

4. All of the above

What are the types of multi-container pod patterns?

1. Sidecar

2. Adaptor

3. Ambassador

4. All of the above

An abstraction in kubernetes which defines a logical set of pods and a policy to access them?

1. Kubelet

2. Service

3. Node

4. Container

This volume type can be used to share contents within containers in a pod, but will not persist beyond the life of a pod?

1. Empty Dir

2. Flex Volume

3. ConfigMap

4. Local

 Which of the following process runs on Kubernetes non-master node?

1. Kube-proxy

2. Kube-apiserver

  Kubernetes supports which of the following container runtime(s)?

1. docker

2. rkt

3. Both of the above

4. None of the above

 Which of the following embeds the core control loop which is a non-terminating loop that regulates the state of the system.

1. kube-apiserver

2. kube-scheduler

3. Kube-controller-manager

 In Kubernetes, a node is:

1. A tool for starting a kubernetes cluster on a local machine

2. A worker machine

3. A machine that coordinates the scheduling and management of application containers on the cluster

    Which of the following services runs on a Kubernetes node?

1. Kubelet

2. Kube-proxy

3. Both of the above 

    Which of the following is a Kubernetes controllers?

1. Namespace

2. Replicaset

3. Deployment

4. Both Replicaset and Deployment

 What can you deploy on Kubernetes?

1. Containers

2. Virtual machines

3. System processes (like sshd, httpd)

 Which of the following validates and configures data for the api objects such as pods, services?

1. kube-apiserver

2. kube-scheduler

3. Kube-controller-manager

 Which of the following runs as a node agent and watches for the pods that have been assigned to its node?

1. Kubelet

2. Kube-proxy

3. Kubectl

4. Etcd

 Kubernetes cluster data is stored in which of the following?

1. kube-apiserver

2. etcd

3. Kubelet

  At its core, Kubernetes is a platform for:

1. Provisioning machines (similar to Puppet, Ansible)

2. Running and scheduling container applications on a cluster

3. Packaging software in containers

 Which of the following process runs on Kubernetes master node?

1. Kubelet

2. Kube-apiserver

3. Kube-proxy

    Which of the following forms the core Kubernetes objects?

1. Pod

2. Service

3. Volume

4. All of the above

 Which of the following is responsible for assigning a node to newly created pods?

1. kube-apiserver

2. kube-scheduler

3. Kube-controller-manager

 Which of the following represents Kubernetes command line interface?

1. Kubelet

2. Kube-proxy

3. Kubectl

 Which of the following is correct about Kubernetes Pods?

Pods can only contain a single VM

Pods can only contain a single container

Pods can contain one or more containers 

Pods must contain at least two containers

What is the smallest deployable object in Kubernetes?

Pod 

Replication Controller

Deployment

Container

How can you filter the output of a kubectl command to only show a Pod called “mypod”

kubectl get pods –filter mypod

kubctl get pods/mypod 

kubectl get mypod

Which of the following tools is good for installed Kubernetes in the AWS cloud?

kops 

docker-machine

kubeaws

kubectl

Which of the following is a best practice for running workloads on a Kubernetes cluster?

User workloads should only run on Nodes. 

User workloads should only run on Masters.

User workloads should be evenly balanced across Masters and Nodes.

Which of the following Master components are the front-end to the Kubernetes control plane?

The Controller Manager

The apiserver 

The Cluster Store

The Scheduler

Which of the following is the only stateful component on a Kubernetes Master?

The Cluster Store

The Controller Manager

The Scheduler

The apiserver

Which of the following is the most common container runtime on Kubernetes Nodes?

rkt

Docker 

systemd

Parallels

Which component of the Kubernetes worker stack implements networking?

Container runtime

Kubelet

kube-proxy 

Which of the following is the preferred method of issuing work to Kubernetes?

Via manifest files in a declarative fashion 

Via individual commands in an imperative fashion

Which of the following is the best option for two containers that need to share memory and access to a common volume?

Put each container in its own Pod and couple them over the network

Package them as a deployment

Rewrite the code so that both services run inside a single container

Put both container inside the same Pod 

Which of the following is a major reason for Kubernetes Services?

K8s Services allow Pods to scale

Services are highly available

Services allow for simple rollbacks

We cannot rely on Pod IPs 

How are Services linked to Pods?

TCP port bindings

Docker Links

Label Selectors 

Which of the following best describes the benefits of Kubernetes Deployment objects?

Declarative formatting, rolling updates, and simple rollback capabilty 

They provide High Availability to the Kubernetes control plane

Imperative formatting, rolling updates, and simple rollback capabilty

They hugely simplify the deployment and management of Masters and Nodes

Which component of the Kubernetes Master is responsible for issuing work to Nodes?

The Scheduler 

The controller Manager

The apiserver

The Cluster Store

Kubernetes Notes

What is Kubernetes?

Kubernetes (k8s) is a container orchestration system for deploying and managing containers. Its design is influenced by Google's internal cluster management system Borg.

A k8s cluster consists of worker machines called nodes that run containerized applications. Every cluster has at least one worker node that hosts pods - the components of the application workload. The control plane manages the nodes and pods. In production, the control plane usually runs across multiple computers for fault tolerance and high availability.

𝗖𝗼𝗻𝘁𝗿𝗼𝗹 𝗣𝗹𝗮𝗻𝗲 𝗖𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀

- API Server - Communicates with all k8s components and handles all pod operations

- Scheduler - Watches pod workloads and assigns them to nodes

- Controller Manager - Runs core control loops like the Node Controller and EndpointSlice Controller

- etcd - Key-value store that backs all cluster data

𝗪𝗼𝗿𝗸𝗲𝗿 𝗡𝗼𝗱𝗲 𝗖𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀

- Pods - The smallest unit deployed and managed by k8s. Pods group containers and give them a single IP address.

- kubelet - An agent on each node that ensures container runtimes are running in pods

- kube-proxy - A network proxy on each node that handles routing and load balancing for services and pods

Monolith: one giant legacy program/application  with all codes in it

This model is replaced by cloud native microservices app

Declarative model: Describe what you want(desired state)  in a manifest file

what is Declarative model in kubernetes

In Kubernetes, the declarative model refers to the approach of specifying the desired state of the system in configuration files, rather than imperatively instructing the system on how to achieve that state. In other words, instead of issuing commands to Kubernetes to perform specific actions (such as creating, updating, or deleting resources), you define the desired state of your applications and infrastructure using YAML or JSON configuration files. Kubernetes then takes care of reconciling the actual state of the system with the desired state specified in these configuration files. The key characteristics of the declarative model in Kubernetes include: 1. **Desired State**: You define the desired state of your Kubernetes resources (such as Pods, Deployments, Services, etc.) in configuration files. This includes specifications for things like the number of replicas, container images, resource limits, network policies, and more. 2. **Declarative Configuration**: Configuration files are used to declare the desired state of the system, rather than issuing imperative commands to make changes. These configuration files are typically stored in version control systems (such as Git) and treated as the source of truth for the desired state. 3. **Reconciliation**: Kubernetes continuously monitors the actual state of the system and compares it against the desired state specified in the configuration files. If there are any discrepancies, Kubernetes takes action to reconcile the actual state with the desired state, automatically creating, updating, or deleting resources as necessary. 4. **Idempotent Operations**: Declarative configuration enables idempotent operations, meaning that applying the same configuration multiple times will result in the same state, regardless of the current state of the system. This helps ensure consistency and reliability in managing infrastructure and applications. Overall, the declarative model in Kubernetes provides a powerful and flexible way to manage complex distributed systems, allowing you to define and maintain the desired state of your applications and infrastructure efficiently, while Kubernetes handles the details of implementation and reconciliation.

Vmware: VM

Docker: container

Kubernetes: Pod

Pod: Automatic unit of scheduling which can have one or more containers

Replicaset: how many pods to configure

Deploy: updates and rollbacks

SVC: stable network abstraction

what is SVC in kubernetes

In Kubernetes, SVC stands for "Service." A Service is an abstraction that defines a logical set of Pods and a policy by which to access them. Services enable decoupling between the frontend and backend components of an application, allowing clients to access the application without needing to know the IP addresses or specific details about the Pods that implement it. Here are some key points about Kubernetes Services (SVCs): 1. **Networking**: Services provide a stable IP address and DNS name that clients can use to access the Pods associated with the Service. They abstract away the details of individual Pod IP addresses, allowing Pods to be scaled up or down without affecting client connectivity. 2. **Load Balancing**: Services automatically distribute incoming traffic across the Pods that are part of the Service. This load balancing ensures that requests are evenly distributed and that individual Pods do not become overwhelmed with traffic. 3. **Service Discovery**: Services provide a consistent way for clients to discover and communicate with backend Pods. Clients can use the DNS name of the Service to access the Pods, regardless of their underlying IP addresses. 4. **Types of Services**: Kubernetes supports different types of Services, including ClusterIP, NodePort, and LoadBalancer, each with its own characteristics and use cases. For example, ClusterIP Services expose the Service on an internal IP within the Kubernetes cluster, while NodePort Services expose the Service on a port across all Nodes in the cluster. 5. **Labels and Selectors**: Services use labels and selectors to determine which Pods they should route traffic to. Pods that match the selector specified in the Service configuration are considered part of the Service, and traffic is load balanced across them. Overall, Kubernetes Services play a critical role in enabling communication and connectivity between different components of an application, providing a layer of abstraction and reliability for networking in Kubernetes clusters.

TKG cluster 

Tanzu Kubernetes Grid cluster. A fully upstream conformant Kubernetes cluster. a.k.a. TKC 

Supervisor Cluster a.k.a. SV or SC. 

This is the control plane running on vSphere that enables the deployment and management of TKG clusters. 

Load Balancer or LB

 A virtual machine used to load balance traffic between ingress networks and workloads. 

HAProxy is used in this configuration but more load balancers will be introduced. 

vDS vSphere Distributed Switch WCP Workload Control Plane 

kubectl The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters.

 What is kube-node-Lease?

This namespace holds Lease objects associated with each node. Node leases allow the kubelet to send heartbeats so that the control plane can detect node failure

Creating Objects using Imperative Method

Available Commands:

  clusterrole         Create a cluster role

  clusterrolebinding  Create a cluster role binding for a particular cluster role

  configmap           Create a config map from a local file, directory or literal value

  cronjob             Create a cron job with the specified name

  deployment          Create a deployment with the specified name

  ingress             Create an ingress with the specified name

  job                 Create a job with the specified name

  namespace           Create a namespace with the specified name

  poddisruptionbudget Create a pod disruption budget with the specified name

  priorityclass       Create a priority class with the specified name

  quota               Create a quota with the specified name

  role                Create a role with single rule

  rolebinding         Create a role binding for a particular role or cluster role

  secret              Create a secret using specified subcommand

  service             Create a service using a specified subcommand

  serviceaccount      Create a service account with the specified name

  

Creating Objects using Declarative Method

kubectl apply -f  app02.yml

Working with Services

- Load Balancer

- kube-proxy

• - Runs on every Node in the cluster

• - iptables

- Types of Services

• clusterip    Create a ClusterIP service

• nodeport     Create a NodePort service

• external name Create an ExternalName service

• loadbalancer Create a LoadBalancer service

externalip -  https://kubernetes.io/docs/concepts/services-networking/service/#external-ips

Working with the Pod

1. Pod with single Container

2. Pod with Multi Container

3. Init Container Pod

• containers:
• cnt01:
• cnt02:
• cnt03:
• initContainers:
• init01:
• init02:
• init03:

4. Sidecar Containers

5. Dynamic Scheduling

6. Manual Scheduling

• nodeName:

7. Scheduling based nodeSelector

8. Static Pods

10. ReplicaSet

• - Scaleup/down

11. Deployment

• - Rollout/Rollback

9. Working with Services

• - ClusterIP - Default

• - NodePort

• - LoadBalancer

- ExternalName

Various Types of Communications

- Between Containers in the same Pod - Loopback Interface or Localhost:Port or 127.0.0.1:Port

- Between Pod to Pod running on the same Node - Linux Bridge

- Between Pod to Pod running on the different Nodes - Overlay Network Plug-in (Weave-net)

- Between Service to a Pod - kube-proxy

- External to a Service

• - coreDNS

• - External Load Balancer

• - Ingress

• - ExternalIP

Pod Probes

- Readiness Probe - The kubelet uses readiness probes to know when a container is ready to start accepting traffic.

- Liveness Probe - The kubelet uses liveness probes to know when to restart a container.

- Startup Probe - The kubelet uses startup probes to know when a container application has started.

Cluster Upgrade

- Masters

• - Active

• - Passive

- Nodes

kubeadm - Tool

kubelet - Node Agent

kubectl - CLI

Kubernetes Storage

Docker - /var/lib/docker/containers/

Kubelet - /var/lib/kubelet/pods/

Various Storage Options

Ephemeral

• - emptyDir

Persistent Storage

• - hostPath

• - nfs

• - pv

- pvc

Static Storage Provisioning

- Storage Admin

• - Install and Configure Storage Solution - NFS

• - 192.168.100.22

• - /srv/nfs/storage

- Kubernetes Admin

• - Create a PV

• - Storage Device

- Developer

• - Create a PVC

• - Request for the Storage

- PV

• - A PersistentVolume (PV) is a piece of storage in the cluster

• - Static - Admin

• - Dynamic - StorageClass

- PVC

• - A PersistentVolumeClaim (PVC) is a request for storage by a user.

volumeMode:

• Filesystem

• Block

accessModes

• RWO - ReadWriteOnce

• ROX - ReadOnlyMany

• RWX - ReadWriteMany

RWOP - ReadWriteOncePod

Cluster -> Nodes - Pods -> Containers -> App

Reclaim Policy:

• - Retain

• - Delete

• - Recycle - Depricated

What is reclaim policy in kubernetes

In Kubernetes, the reclaim policy refers to the action that should be taken when a PersistentVolumeClaim (PVC) is deleted or released. PersistentVolumes (PVs) in Kubernetes can be dynamically provisioned or statically provisioned, and when a PVC is no longer needed, the reclaim policy determines whether the associated PV should be retained, recycled, or deleted.

There are three main reclaim policies in Kubernetes:

1. **Retain**: With the "Retain" reclaim policy, the PV associated with the PVC is not deleted when the PVC is deleted or released. Instead, the PV and its data are retained indefinitely, and it is the responsibility of the cluster administrator to manually reclaim or delete the PV if it is no longer needed.

2. **Delete**: With the "Delete" reclaim policy, the PV associated with the PVC is automatically deleted when the PVC is deleted or released. This results in the underlying storage resources being released and potentially reclaimed by the storage provider.

3. **Recycle**: The "Recycle" reclaim policy is deprecated as of Kubernetes v1.21 and will be removed in a future release. With the "Recycle" policy, the PV associated with the PVC is deleted when the PVC is deleted or released, and the underlying storage volume is formatted or scrubbed to remove any data. However, this policy has been deprecated due to security concerns and inconsistencies in implementation across storage providers.

By default, if no reclaim policy is specified in the PersistentVolumeClaim manifest, the default policy defined by the storage class is used. Additionally, the reclaim policy can be specified at the PV level or overridden at the PVC level. 

Choosing the appropriate reclaim policy depends on factors such as data retention requirements, storage provider capabilities, and organizational policies regarding resource cleanup and reclamation.

Dynamic Storage

#Storage Admin

- Backend Storage

• - NFS Service

• - 192.168.100.22

• - /srv/nfs/storage

#Kubernetes Admin

- Create a Dynamic PV

- Deploy Storage Provisioner

- Create a StorageClass and Refernce a Storage Provioner

#Developer

- Create a PVC and Reference StorageClass

- Refernce a PVC within a Pod

ConfigMaps

- A ConfigMap is an API object used to store non-confidential data in key-value pairs.

- Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.

- A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.

Secrets

- A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key.

- Such information might otherwise be put in a Pod specification or in a container image

Vanilla kubernetes can run on 5000 servers. Within single cluster we can run 5000 nodes

On each server we can run docker engine as CRI

Kubelet is node agent which manages containers

Kubernetes manages pods

Pod is collection of containers

We connect only to master ie kube-apiserver which redirect connection to one of the nodes

Pod storage etcd

Kube-schedular: which node of cluster pod to run application on

Single application can have multiple pods and replicas taken care by kube-controller mgr

Pods run on worker nodes

Pods runs on each node =110

Pods = one or more containers

Single cluster can be divided into multiple virtual clusters called namespace

Nested namespace is not supported

Always we run command from master node

If you delete NS ..all objects within NS will be deleted

Accidental deletion is not supported 

All pods need to be in same node .. As per requirement, we can't have some pod in one node and other in another node

Any object we create goes in default node if not specified

There is also constraint on how many pods can run on single nodes

CRD comes into picture when you want more features to kubernetes without impacting it

Logging directly to worker node and create pod is called static pod

All control pods are running as static pods -- usecase of static pods

What is CRD in kubernetes?

In Kubernetes, CRD stands for Custom Resource Definition. CRDs allow users to define custom resources and their schema in a Kubernetes cluster, extending the functionality of Kubernetes beyond its built-in resources like Pods, Services, Deployments, and others.

Here are some key points about CRDs:

1. **Custom Resources**: CRDs enable users to define their own custom resources, which can represent any kind of object or application-specific resource that is not provided by default in Kubernetes. Examples include databases, message queues, machine learning models, monitoring configurations, and more.

2. **Schema Definition**: When creating a CRD, users specify the schema for the custom resource, including its API fields, validation rules, default values, and other metadata. This allows Kubernetes to enforce consistency and validate the configuration of custom resources.

3. **Controller Logic**: After defining a custom resource using a CRD, users typically write controllers or operators to manage the lifecycle and behavior of the custom resource. Controllers watch for changes to custom resources and reconcile the actual state of the resource with its desired state, performing actions such as creating, updating, or deleting underlying Kubernetes objects.

4. **Extensibility**: CRDs provide a powerful mechanism for extending Kubernetes with domain-specific or application-specific functionality. By defining custom resources and controllers, users can encapsulate complex logic, automate operational tasks, and integrate third-party systems with Kubernetes seamlessly.

5. **Community and Ecosystem**: CRDs have become a popular tool for Kubernetes users and ecosystem developers to create and share custom extensions and integrations. Many open-source projects, vendors, and organizations provide CRDs and controllers for common use cases, enabling users to leverage pre-built solutions or develop their own customizations.

Overall, CRDs are a fundamental building block for extending Kubernetes with custom resources and domain-specific functionality, enabling users to adapt Kubernetes to their specific requirements and use cases.

Working with services

- Load balancer

- Kube-proxy

• Runs on every node in cluster

• Kube-proxy use iptables for loadbalancing

4 services

Clusterip is for accessing cluster

To access service from outside .. Change to NodePort on yml file. Service is also available internally

First port inside cluster and 2nd port outside cluster

To access externally: nodeip:2nd port

We can also have public ip ..don't want to reveal real ip

Kubectl get rs

Pods with less age will get delete

In case of edit etcd we don't have to apply

But make changes in yaml and apply is recommended

The older the pods the more are stable

Kube-poxy is used for both cluster ip and external ip

Kubectl explain pod

Along with application container .. We can have more containers like metric, logs basically helper container

Independent pod if created .. Will be deleted so better to have rs so that it will create once incase of deletion

So each tier label we can have single RS (replicaset)

Per application have 1 RS

Deployment  makes use of RS…when introduce new version ..its helps in rollout

Yaml for RS and deployment

Kubectl get apps all

Kubectl get rs

Kubectl get deploy

##k get deploy
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
authserver                     1/1     1            1           3d
cluster-api                    1/1     1            1           3d
common-agent                   1/1     1            1           3d

# k explain pods
KIND:     Pod
VERSION:  v1
DESCRIPTION:
     Pod is a collection of containers that can run on a host. This resource is
     created by clients and scheduled onto hosts.
FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
   metadata     <Object>
     Standard object's metadata. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
   spec <Object>
     Specification of the desired behavior of the pod. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
   status       <Object>
     Most recently observed status of the pod. This data may not be up to date.
     Populated by the system. Read-only. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status

Rollout means upgrade

Rollback means downgrade

Kubectl explain deployment ---recursive

We can generate yaml for any objects

Kubectl create --help

Kubectl expose deployment web-server --port=80

Kubectl get svc

Default type is clusterIP

Replicaset get details from etcd

Kube-proxy relies on all nodes

To get application ip, describe the pod

Deletion is support on pod level not container level

Init container start in sequence. If one init container fails other init container will not start. To satisfy some application pre-check. But application container start in parallel.

Sidecar is type of another application container. It is kind of plug and play

If there are multiple replicas .. Replicas are distributed equally across pods.it is default behaviour of schedular.

By default it is dynamic scheduling.

We can also do manual scheduling. Using node name. Kube schedular will not involve here.

We can have labels to any object

System pods runs on master.. Applications pods run worker

Terraform and ansible tools to create labels

Kube api server connect to schedular

By default kubernetes logs are not enabled in control plane. Schedular specific logs

Pod priority .. .default is 0 … higher the value higher is priority .. In case of resource issue .. Priority pod will schedule and other pod will get evicted and can be scheduled on other nodes.

In case of probes .. Schedular will not come into picture .. It is all about kubelet

Schedular comes in picture for complete pod

what is dynamic scheduling in kubernetes
Dynamic scheduling in Kubernetes refers to the process by which the Kubernetes scheduler automatically assigns Pods to nodes in the cluster based on available resources, constraints, and user-defined policies. The scheduler continuously monitors the state of the cluster and makes decisions about where to place Pods to ensure efficient resource utilization, high availability, and optimal performance.
Here are some key points about dynamic scheduling in Kubernetes:
1. **Node Selection**: When a new Pod is created or an existing Pod needs to be rescheduled (e.g., due to node failure or scaling), the Kubernetes scheduler evaluates the available nodes in the cluster to determine the best placement for the Pod. Factors considered during node selection may include available CPU and memory resources, node affinity and anti-affinity rules, pod affinity and anti-affinity rules, node taints and tolerations, and resource requests and limits specified in the Pod's configuration.
2. **Scalability and Resource Utilization**: Dynamic scheduling enables Kubernetes clusters to scale efficiently and automatically distribute workload across nodes to achieve optimal resource utilization. By dynamically placing Pods on nodes with available capacity, Kubernetes ensures that resources are utilized effectively and that Pods are not over-provisioned or under-provisioned.
3. **High Availability**: Dynamic scheduling helps improve the resilience and availability of applications by automatically redistributing workload in the event of node failures or disruptions. When a node becomes unavailable, the Kubernetes scheduler identifies alternative nodes where affected Pods can be rescheduled to maintain service availability.
4. **Integration with Cluster Autoscaler**: Dynamic scheduling works in conjunction with the Kubernetes Cluster Autoscaler, which automatically adjusts the size of the cluster by adding or removing nodes based on workload demands. The scheduler can schedule Pods on newly added nodes to accommodate increased workload or rebalance workload across existing nodes to optimize resource usage.
5. **Custom Schedulers and Policies**: Kubernetes allows users to customize the scheduling process by implementing custom schedulers or defining scheduling policies using features like node affinity, pod affinity, pod anti-affinity, node taints, and tolerations. These customizations enable users to align scheduling decisions with specific requirements, constraints, and business priorities.
Overall, dynamic scheduling plays a critical role in the efficient operation of Kubernetes clusters, enabling automated workload placement, resource optimization, and high availability of applications.

Static Pod:

Are scheduled on master nodes.

Kube-apiserver, kube-controller-manager, kube-schedular, etcd

Kube-proxy is also called load balancer

Pod restart (deleted + added)

Ready(1/1) : right side is total number of container

kubectl get nodes -o wide

watch kubectl get all -n nsxi-platform

kubectl get svc(services)

kubectl api-resources

systemctl status docker

kubectl get pods -A

kubectl describe nodes | grep -i -e "Name: " -e "Taints"

kubectl get pv(persistent volume)

kubectl get sc (storageclass)

kubectl get cm(configmap)

# k describe nodes | grep -i -e "Name: " -e "Taints"
Name:               napp-cluster-default-sd54b-68xhv
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-sd54b
Taints:             node-role.kubernetes.io/master:NoSchedule
  Hostname:    
Name:               napp-cluster-default-sd54b-7m2s6
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-sd54b
Taints:             node-role.kubernetes.io/master:NoSchedule
  Hostname:    
Name:               napp-cluster-default-sd54b-gn595
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-sd54b
Taints:             node-role.kubernetes.io/master:NoSchedule
  Hostname:    
Name:               napp-cluster-default-workers-kgvqh-766459bb7f-2tsm5
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-workers-kgvqh-766459bb7f
Taints:             <none>
  Hostname:    
Name:               napp-cluster-default-workers-kgvqh-766459bb7f-djjkj
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-workers-kgvqh-766459bb7f
Taints:             <none>
  Hostname:    
Name:               napp-cluster-default-workers-kgvqh-766459bb7f-mj7sh
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-workers-kgvqh-766459bb7f
Taints:             <none>
  Hostname:    
Name:               napp-cluster-default-workers-kgvqh-766459bb7f-mtx9f
Annotations:        cluster.x-k8s.io/cluster-name: napp-cluster-default
                    cluster.x-k8s.io/owner-name: napp-cluster-default-workers-kgvqh-766459bb7f
Taints:             <none>
  Hostname:    

# k api-resources

NAME                              SHORTNAMES                           APIVERSION                                           NAMESPACED   KIND
bindings                                                               v1                                                   true         Binding
componentstatuses                 cs                                   v1                                                   false        ComponentStatus
configmaps                        cm                                   v1                                                   true         ConfigMap
endpoints                         ep                                   v1                                                   true         Endpoints

# k get sc

NAME                                      PROVISIONER              RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE

vsan-default-storage-policy               csi.vsphere.vmware.com   Delete          Immediate              true                   119d

vsan-default-storage-policy-latebinding   csi.vsphere.vmware.com   Delete          WaitForFirstConsumer   true                   119d

# k get cm

NAME                                                                 DATA   AGE

cluster-api-application-config                                       2      3d

clusterapi-feature-ctrl                                              1      3d

Kubernetes version : +1 and -1 version upgrade is allowed

During upgrade in case of failures … it will automatically rollback to pervious version

During upgrade: kubeadm + kubelet + kubectl all needs upgrade

Pods has 3 directores: kube-proxy, netwroking, pod

In kubernetes every object gets unique uid

In docker every container has unique id

By default pods are ephemeral and data is not persistent.

Daemons check in config map after every 1 minutes .. So it will take 1 min to update same in pod

Secret and configmap should not exceed 1MB

Secret is encrypted using TLS during transition

ConfigMap can be used as volume as backend

By default all pods with same or different namespace can communicate with each other

By default service to service communication is restricted within namespace

To have control we can have network policies.

what is upstream kubernetes?

In the context of Kubernetes, "upstream" typically refers to the official development repository or source code of Kubernetes maintained by the Kubernetes project itself. It is the primary location where new features, enhancements, bug fixes, and improvements to Kubernetes are developed and released. Here are some key points about the upstream Kubernetes project: 1. **Official Repository**: The upstream Kubernetes project is hosted on GitHub under the organization "kubernetes" (https://github.com/kubernetes/kubernetes). This repository contains the source code, documentation, and other resources related to the Kubernetes project. 2. **Development Process**: The upstream Kubernetes project follows an open-source development model, where contributions are made by individual developers, companies, and organizations from around the world. Development activities, including code reviews, issue tracking, and release management, are coordinated through GitHub. 3. **Release Cycle**: Kubernetes follows a regular release cycle, with new versions (major, minor, and patch releases) being published approximately every three months. Each release undergoes a rigorous development, testing, and validation process before being made generally available to users. 4. **Community Involvement**: The upstream Kubernetes project has a large and active community of contributors and users who participate in various aspects of the project, including code contributions, testing, documentation, support, and advocacy. The Kubernetes community is inclusive and welcomes participation from anyone interested in contributing to the project. 5. **Distribution**: Many Kubernetes distributions, platforms, and vendors build their offerings based on the upstream Kubernetes project. These distributions may add additional features, integrations, and customizations on top of the core Kubernetes platform, but they often track the upstream project closely to incorporate new features and fixes. Overall, the upstream Kubernetes project serves as the central hub for development and innovation in the Kubernetes ecosystem, driving the evolution of the platform and enabling its adoption by organizations worldwide.

A Visual Overview of Kubernetes

Containers revolutionized modern application development and deployment. Unlike bulky virtual machines, containers package up just the application code and dependencies, making them lightweight and portable. However, running containers at scale brings challenges. Enter Kubernetes!

Kubernetes helps deploy, scale, and manage containerized applications across clusters of machines.

𝗖𝗼𝗿𝗲 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 𝗖𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀

Control Plane: The brains behind cluster management, handling scheduling, maintaining desired state, rolling updates etc. Runs on multiple machines for high availability.

Worker Nodes: The machines that run the containerized applications. Each node has components like kubelet and kube-proxy alongside the application containers.

The smallest deployable units in Kubernetes are Pods. A Pod encapsulates one or more tightly coupled containers that comprise an application. Kubernetes assigns Pods to worker nodes through its API server.

𝗞𝗲𝘆 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 𝗖𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀

- Scalability: It's easy to scale applications up and down on demand. Just specify the desired instance count, Kubernetes handles the rest!
- Portability: Applications can run anywhere - on premise, cloud, hybrid environments etc. No vendor lock-in!
- Resiliency: Kubernetes restarts failed containers, replaces unhealthy nodes, and maintains desired state, reducing downtime.
- Automation: Manual tasks like rolling updates, rollbacks are automated, freeing teams to focus on development.

𝗧𝗿𝗮𝗱𝗲𝗼𝗳𝗳𝘀

The power of Kubernetes comes with complexity. Installing, configuring, and operating Kubernetes has a steep learning curve. For many teams, it's overkill.

Managed Kubernetes services help by handling control plane management, letting teams focus only on applications and pay for just the worker resources used.

𝗜𝘀 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 𝗮 𝗚𝗼𝗼𝗱 𝗙𝗶𝘁?

Consider:
- Are you running containers already at meaningful scale?
- Will portability or resiliency resolve production issues?
- Is your team willing to invest in learning and operating Kubernetes?

If you answered yes, Kubernetes may suit your needs. Otherwise, containers without orchestration may still get the job done.