Pages

Wednesday, January 31, 2024

Kubernetes Commands

1. Nodes

CommandsDescription
kubectl get nodeTo list down all worker nodes.
kubectl delete node <node_name>Delete the given node in cluster.
kubectl top nodeShow metrics for a given node.
kubectl describe nodes | grep ALLOCATED -A 5Describe all the nodes in verbose.
kubectl get pods -o wide | grep <node_name>List all pods in the current namespace, with more details.
kubectl get no -o wideList all the nodes with mode details.
kubectl describe noDescribe the given node in verbose.
kubectl annotate node <node_name>Add an annotation for the given node.
kubectl uncordon node <node_name>Mark my-node as schedulable.
kubectl label nodeAdd a label to given node


2. Pods
CommandsDescription
kubectl get poTo list the available pods in the default namespace.
kubectl describe pod <pod_name>To list the detailed description of pod.
kubectl delete pod <pod_name>To delete a pod with the name.
kubectl create pod <pod_name>To create a pod with the name.
Kubectl get pod -n <name_space>To list all the pods in a namespace.
Kubectl create pod <pod_name> -n <name_space>To create a pod with the name in a namespace.

3. Namespaces
CommandsDescription
kubectl create namespace <namespace_name>To create a namespace by the given name.
kubectl get namespaceTo list the current namespace in a cluster.
kubectl describe namespace <namespace_name>To display the detailed state of one or more namespaces.
kubectl delete namespace <namespace_name>To delete a namespace.
kubectl edit namespace <namespace_name>To edit and update the definition of a namespace.

4. Services
CommandsDescription
kubectl get servicesTo list one or more services.
kubectl describe services <services_name>To list the detailed display of services.
kubectl delete services -o wideTo delete all the services.
kubectl delete service < service_name>To delete a particular service.

5. Deployments
CommandsDescription
kubectl create deployment <deployment_name>To create a new deployment.
kubectl get deploymentTo list one or more deployments.
kubectl describe deployment <deployment_name>To list a detailed state of one or more deployments.
kubectl delete deployment<deployment_name>To delete a deployment.
6. DaemonSets
CommandDescription
kubectl get dsTo list out all the daemon sets.
kubectl get ds -all-namespacesTo list out the daemon sets in a namespace.
kubectl describe ds [daemonset_name][namespace_name]To list out the detailed information for a daemon set inside a namespace.
7. Events
CommandsDescription
kubectl get eventsTo list down the recent events for all the resources in the system.
kubectl get events --field-selector involvedObject.kind != PodTo list down all the events except the pod events.
kubectl get events --field-selector type != NormalTo filter out normal events from a list of events.
8. Logs
CommandsDescription
kubectl logs <pod_name>To display the logs for a Pod with the given name.
kubectl logs --since=1h <pod_name>To display the logs of last 1 hour for the pod with the given name.
kubectl logs --tail-20 <pod_name>To display the most recent 20 lines of logs.
kubectl logs -c <container_name> <pod_name>To display the logs for a container in a pod with the given names.
kubectl logs <pod_name> pod.logTo save the logs into a file named as pod.log.

9. ReplicaSets
CommandsDescription
kubectl get replicasetsTo List down the ReplicaSets.
kubectl describe replicasets <replicaset_name>To list down the detailed state of one or more ReplicaSets.
kubectl scale --replace=[x]To scale a replica set.

10. Service Accounts
CommandsDescription
kubectl get serviceaccountsTo List Service Accounts.
kubectl describe serviceaccountsTo list the detailed state of one or more service accounts.
kubectl replace serviceaccountsTo replace a service account.
kubectl delete serviceaccounts <name>To delete a service account.

Changing Resource Attributes

Taints: They ensure that pods are not placed on inappropriate nodes.
CommandDescription
kubectl taint <node_name><taint_name>This is used to update the taints on one or more nodes.
Labels: They are used to identify pods.
CommandDescription
kubectl label pod <pod_name> Add or update the label of a pod
For Cluster Introspection
CommandsDescription
kubectl versionTo get the information related to the version.
kubectl cluster-infoTo get the information related to the cluster.
kubectl config g viewTo get the configuration details.
kubectl describe node <node_name>To get the information about a node.
Interacting with Deployments and Services
CommandsDescription
kubectl logs deploy/my-deploymentDump Pod logs for a Deployment (single-container case).
kubectl logs deploy/my-deployment -c my-containdump Pod logs for a Deployment (multi-container case).
kubectl port-forward svc/my-service 5000To listen on local port 5000 and forward to port 5000 on Service backend.
kubectl port-forward deploy/my-deployment 5000:6000To listen on local port 5000 and forward to port 6000 on a Pod created by <my-deployment>.
kubectl exec deploy/my-deployment -- lsTo run command in first Pod and first container in Deployment (single- or multi-container cases).



How Git Works?








There are 4 main location where our code lives in Git. 

The working directory: where we actively edit files locally.

The staging Area: It is the temporary holding spot for changes before committing. 

The local repository: This is where we store committed changes locally. 

The Remote repository: A server like Github for sharing and backing up code. 

Most Git commands move files between these four locations. 

The first step is git clone of existing repository so you have a local version of the project to work on, complete with all its history. With the repo clone locally, we are in the working directory. This is your local development environment where you make changes to your code. 

When you are ready to commit your changes, you'll use git add to stage a snapshot of those files in staging area. Think of this as a checkpoint, where your changes are gathered and ready to be finalized.

Next step is to use git commit, which takes the snapshot of the staging area and saves it to your local repository. This locks in those staged changes by creating a permanent record that you can refer back to, like a snapshot in time. 

Your code just doesn't stay on your local machine, When you are ready to share your progress with the team or backup your work, you use git push to send your commits to the remote repository. This is often a shared server where your team can collaborate, like github or bitbucket. 

Collaboration in git is a two way exchange, To integrate your team mates work, you use git pull, which fetches changes from the remote repository and merges them into your local repository. It combines two commands: git fetch, which grabs the latest updates, and git merge which integrates these updates with your work. 

There are times when you need to switch contexts, perhaps to fix a bug in another branch. Thats where the git checkout or git switch comes in. It allows you to switch between different branches to work on specific features. 

Git branching allows you to diverse from the main codebase to develop a new feature without impacting the main code. Some key concepts include creating a new branch with git branch and switching between branches using git switch, merging branches together with git merge and resolving merge conflicts when changes overlap. Branching enables isolate development and collaboration workflows. There are more nuance when merging or rebasing changes from others or managing branches. 

Many developers use graphical git tools like github desktop and sourceTree. These tools provide visual interfaces and shortcuts for common commands. 

###

Git is a distributed version control system (DVCS) widely used for tracking changes in source code during software development. Here's an overview of how Git works:

1. **Local Repository**:

   - Git operates primarily on a local repository, which resides on the user's computer. This repository contains the entire history of the project, including all files, commits, branches, and tags.

2. **Working Directory**:

   - The working directory is the directory on the user's computer where the project files are stored and edited. When changes are made to files in the working directory, Git tracks these changes and allows them to be staged for commit.

3. **Staging Area**:

   - The staging area, also known as the "index" or "cache," is a temporary storage area where changes to files are prepared (staged) for commit. Files in the staging area represent the next snapshot of the project that will be committed to the repository.

4. **Commits**:

   - A commit is a snapshot of the project's state at a specific point in time. Each commit records changes to files, along with metadata such as the author, timestamp, and commit message. Commits are immutable and uniquely identified by a cryptographic hash.

5. **Branches**:

   - A branch is a lightweight movable pointer to a specific commit in the repository's history. Branches allow developers to work on different features or bug fixes concurrently without affecting the main project. The default branch in Git is typically named "master" or "main."

6. **Merging and Rebasing**:

   - Merging combines changes from different branches into a single branch, typically the main branch. Git automatically resolves conflicts between conflicting changes during the merge process.

   - Rebasing is an alternative to merging that rewrites the commit history by moving, copying, or deleting commits. This can create a linear history and avoid merge commits.

7. **Remote Repository**:

   - Git supports remote repositories, which are copies of the repository hosted on remote servers, such as GitHub, GitLab, or Bitbucket. Remote repositories allow multiple developers to collaborate on the same project and synchronize changes across different machines.

8. **Pull and Push**:

   - Pull is the process of fetching changes from a remote repository and integrating them into the local repository. This updates the local repository with changes made by other developers.

   - Push is the process of sending local commits to a remote repository, making them available to other developers. This updates the remote repository with changes made locally.

9. **Distributed Nature**:

   - Git is a distributed version control system, meaning that each developer has a complete copy of the repository, including its entire history. This allows developers to work offline, commit changes locally, and synchronize with remote repositories later.

Overall, Git provides powerful version control capabilities, enabling efficient collaboration, code review, and software development workflows. Its flexibility, speed, and robustness make it one of the most widely used version control systems in the software development industry.



Tuesday, January 30, 2024

Vertical vs Horizontal Scaling

 


Vertical scaling means adding more power to your existing server. This could involve adding more CPU, RAM, storage or network bandwidth. For eg say your cloud database hits capacity limits on its starter 8 core server. You could upgrade to 32 core instance with faster SSD storage, 96GB of RAM and 10GB networking. Now the box can take on the extra load. 

Advantage: 

It is simple to implement. Upgrading existing hardware is easier than setting up new servers. It's cost effective in the short term. You only pay for the additional resources you need. Everything runs on one machine making maintenance and upgrade easier. 

Disadvantages:

It's a single point of failure. If the server fails, everything goes down. 
Limited scaling headroom. There are physical limit, how powerful single server can be. 
High cost at large scale. Upgrading to high end hardware can be expensive. 

Horizontal scaling means adding more servers to you infrastructure and distributing the workloads across them. This is also known as scaling out. Instead of cramming everything in one big box, we could spread the capacity across three 8 core- nodes. The popularity of cloud servers with auto-scaling and serverless computing has significantly simplified this approach to scaling for some workloads. 

Advantages:

High availability. Distribute systems offer increased availability through redundant servers and failover mechanisms. 
Predictable growth headroom. You can add more servers as needed, scaling your capacity as your needs grows. 
Improved performance. Spreading the workloads across multiple servers can increase overall performance. 
Lower cost over time.
Distributing workloads across more efficient servers can be cheaper than upgrading to high end hardware. 

Disadvantages

Complex to implement. Setting up and managing a distributed system is more complex than managing a single server. This is specially true for stateful systems like databases.
High upfront cost. There are several dimensions on the cost front. First, sharding your database or application to distribute the workload can be complex and require significant development effort. 
Maintaining data consistency across multiple nodes requires data replication mechanisms, which can add additional overhead to your system and increase operational cost. 
Distributing traffic efficiently across multiple servers requires a robust load balancing solution, which can add additional software or hardware cost to your infrastructure. 

Which one to choose. Things to consider.

Budget: Vertical scaling is generally cheaper in the short term and horizontal scaling can be more cost effective in the long run.

Workload: if the workload is unpredictable or bursty horizontal scaling can help you handle peak demand. 

Performance requirements: If your application is performance sensitive, horizontal scaling can help you distribute the load and improve responsiveness. 

If your application requires complex sharding or other horizontal scaling mechanism, the additional development and operational cost need to be factored in your decision.  

###

Vertical scaling and horizontal scaling are two different approaches to increasing the capacity and performance of a system, such as a web application, database server, or network infrastructure. Here's an overview of each approach and some common use cases:

1. **Vertical Scaling**:

   - **Definition**: Vertical scaling, also known as scaling up or scaling vertically, involves increasing the capacity of a single resource (e.g., CPU, memory, storage) within a server or virtual machine (VM).

   - **Method**: Vertical scaling typically involves upgrading hardware components, such as replacing the CPU with a faster one, adding more RAM, or upgrading to a larger storage device.   

   - **Use Cases**:

     - **Monolithic Applications**: Vertical scaling is commonly used for monolithic applications where all components of the application run on a single server. Increasing the capacity of the server can improve the performance of the entire application.

     - **Small to Medium Workloads**: Vertical scaling is suitable for small to medium workloads where the resource requirements can be met by a single server or VM. It is often the simplest and most cost-effective solution for scaling up.

     - **Database Servers**: Vertical scaling is often used to increase the capacity of database servers to handle larger datasets or higher query loads. Adding more CPU cores or memory can improve database performance and responsiveness.


2. **Horizontal Scaling**:

   - **Definition**: Horizontal scaling, also known as scaling out or scaling horizontally, involves adding more instances of a resource, such as servers or VMs, to distribute the workload across multiple machines.

   - **Method**: Horizontal scaling typically involves deploying multiple instances of an application or service across different servers or VMs. Load balancers are used to distribute incoming requests evenly across these instances.

   - **Use Cases**:

     - **Web Applications**: Horizontal scaling is commonly used for web applications where the workload can be distributed across multiple servers or VMs. Adding more instances allows the application to handle a higher volume of traffic and improve scalability.

     - **Microservices Architecture**: Horizontal scaling is well-suited for microservices architectures, where different components of the application are deployed as independent services. Each service can be scaled independently based on its resource requirements.

     - **High Availability**: Horizontal scaling improves fault tolerance and availability by distributing the workload across multiple instances. If one instance fails or becomes unavailable, the remaining instances can continue to handle requests without interruption.


3. **Hybrid Scaling**:

   - **Definition**: Hybrid scaling combines elements of both vertical and horizontal scaling to optimize performance and scalability.

   - **Method**: In hybrid scaling, vertical scaling may be used to increase the capacity of individual components within a system, while horizontal scaling is used to distribute the workload across multiple instances of these components.

   - **Use Cases**:

     - **Elasticity**: Hybrid scaling allows for dynamic scaling based on demand, where additional resources can be added vertically or horizontally as needed to meet changing workload requirements.

     - **Cost Optimization**: Hybrid scaling can help optimize costs by leveraging the benefits of both vertical and horizontal scaling. Vertical scaling may be more cost-effective for predictable workloads with steady resource requirements, while horizontal scaling provides flexibility and scalability for unpredictable workloads.

In summary, vertical scaling involves increasing the capacity of individual resources within a single server or VM, while horizontal scaling involves adding more instances of a resource to distribute the workload across multiple machines. The choice between vertical and horizontal scaling depends on factors such as the nature of the workload, scalability requirements, cost considerations, and architectural constraints.

###

Scale out vs Scale in

"Scale out" and "scale in" are two strategies for adjusting the capacity of a system, such as a network, server infrastructure, or cloud environment, to accommodate changes in workload or demand. Here's an overview of each approach:


1. **Scale Out**:

   - **Definition**: Scale out, also known as horizontal scaling or scaling horizontally, involves adding more instances or resources to a system to distribute the workload across multiple machines.

   - **Method**: In scale out, additional servers, virtual machines, or containers are deployed to handle the increased workload. Load balancers are used to evenly distribute incoming requests or tasks across the available instances. 

   - **Use Cases**:

     - **High Traffic Events**: Scale out is commonly used to handle sudden spikes in traffic or demand, such as during promotional events, sales peaks, or seasonal traffic surges.

     - **Scalable Architectures**: Scale out is well-suited for architectures designed to scale horizontally, such as microservices-based applications or distributed systems. Adding more instances allows these architectures to handle a higher volume of traffic and improve scalability.

     - **Fault Tolerance**: Scale out improves fault tolerance and availability by distributing the workload across multiple instances. If one instance fails or becomes unavailable, the remaining instances can continue to handle requests without interruption.


2. **Scale In**:

   - **Definition**: Scale in, also known as horizontal scaling or scaling horizontally, involves reducing the number of instances or resources in a system to optimize resource utilization and reduce costs.

   - **Method**: In scale in, excess or underutilized instances are removed from the system, typically through automation or manual intervention. Load balancers may adjust their configurations to distribute the workload across the remaining instances.

   - **Use Cases**:

     - **Cost Optimization**: Scale in helps optimize costs by eliminating unnecessary or underutilized resources. It allows organizations to align their infrastructure capacity with actual workload demands, reducing operational expenses.

     - **Off-Peak Hours**: Scale in is useful for reducing capacity during off-peak hours or periods of low demand. By scaling down resources when they are not needed, organizations can maximize resource utilization and minimize wastage.

     - **Resource Consolidation**: Scale in can be used to consolidate resources and streamline infrastructure management. It allows organizations to reduce complexity and overhead by eliminating redundant or obsolete instances.

In summary, scale out involves adding more instances or resources to a system to handle increased workload or demand, while scale in involves removing excess or underutilized instances to optimize resource utilization and reduce costs. The choice between scale out and scale in depends on factors such as workload patterns, scalability requirements, cost considerations, and operational constraints. Both strategies play important roles in capacity management and resource optimization for modern IT environments.