Pages

Friday, December 31, 2021

All Headers - TCP, IP, UDP, ARP, DNS, ICMP, Ethernet, IPSec, Vlan tagging



AH (Authentication Header) is one of the two main components of the IPsec (Internet Protocol Security) protocol suite, along with the ESP (Encapsulating Security Payload). AH provides integrity and authentication services for IP packets, ensuring that they have not been tampered with or altered during transmission. Here's an overview of AH in IPsec:

1. **Integrity and Authentication**:
   - AH provides integrity protection by calculating a cryptographic hash (usually using HMAC-SHA1 or HMAC-SHA256) over the entire IP packet (including the IP header and payload) and including the hash value in the AH header.
   - The hash value is generated using a shared secret key known to both the sender and receiver. This allows the receiver to verify the integrity of the packet by recalculating the hash value and comparing it to the value included in the AH header.

2. **No Confidentiality**:
   - Unlike ESP, which provides both integrity and confidentiality services, AH only provides integrity and authentication. It does not encrypt the packet payload, so the original payload is visible to anyone who intercepts the packet.

3. **AH Header Format**:
   - The AH header is inserted between the IP header and the packet payload. It includes fields such as:
     - Next Header: Specifies the type of the next header after the AH header (e.g., TCP, UDP).
     - Payload Length: Specifies the length of the AH header and the payload.
     - Security Parameters Index (SPI): A unique identifier that associates the packet with a particular security association.
     - Sequence Number: An optional field used to prevent replay attacks by ensuring that packets are received in the correct order.
     - Authentication Data (Hash): The computed hash value over the entire IP packet.

4. **Transport and Tunnel Mode**:
   - AH can operate in either transport mode or tunnel mode, depending on the requirements of the IPsec deployment.
   - In transport mode, AH protects the payload of the IP packet while leaving the IP header unchanged.
   - In tunnel mode, AH protects both the IP header and payload of the packet. The entire original IP packet is encapsulated within a new IP packet with a new IP header.

5. **Compatibility and Limitations**:
   - AH may encounter compatibility issues with certain network devices and NAT (Network Address Translation) configurations, as the IP addresses and certain fields in the IP header are protected by AH and cannot be modified without breaking the integrity check.
   - AH cannot be used in scenarios where NAT is involved or when there is a need for confidentiality in addition to integrity and authentication.

In summary, AH in IPsec provides integrity and authentication services for IP packets, ensuring that they have not been tampered with during transmission. However, it does not provide confidentiality, and it may encounter compatibility issues in certain network configurations.


ESP (Encapsulating Security Payload) is one of the two main components of the IPsec (Internet Protocol Security) protocol suite, along with the Authentication Header (AH). ESP provides confidentiality, integrity, and authentication services for IP packets, making it a versatile and widely used protocol for securing communications over IP networks. Here's an overview of ESP in IPsec:

1. **Confidentiality**:
   - One of the primary functions of ESP is to provide confidentiality by encrypting the payload (or inner IP packet) of the IP packet. This ensures that the contents of the packet are protected from eavesdropping or interception by unauthorized parties.
   - ESP achieves confidentiality through encryption algorithms such as AES (Advanced Encryption Standard), DES (Data Encryption Standard), or 3DES (Triple DES). The choice of encryption algorithm depends on the security requirements of the IPsec deployment.

2. **Integrity and Authentication**:
   - In addition to confidentiality, ESP also provides integrity and authentication services to ensure that the packet has not been tampered with during transmission.
   - ESP calculates a cryptographic hash (similar to AH) over the entire IP packet (including the ESP header, payload, and optional padding), and includes the hash value in the ESP header. This allows the receiver to verify the integrity of the packet by recalculating the hash value and comparing it to the value included in the ESP header.

3. **ESP Header Format**:
   - The ESP header is inserted between the IP header and the packet payload. It includes fields such as:
     - Security Parameters Index (SPI): A unique identifier that associates the packet with a particular security association.
     - Sequence Number: An optional field used to prevent replay attacks by ensuring that packets are received in the correct order.
     - Payload Data: The encrypted payload of the IP packet.
     - Authentication Data (Hash): The computed hash value over the entire IP packet.

4. **Transport and Tunnel Mode**:
   - Similar to AH, ESP can operate in either transport mode or tunnel mode.
   - In transport mode, ESP protects the payload of the IP packet while leaving the IP header unchanged.
   - In tunnel mode, ESP protects both the IP header and payload of the packet. The entire original IP packet is encapsulated within a new IP packet with a new IP header.

5. **Compatibility and Interoperability**:
   - ESP is widely supported by network devices and is interoperable with most IPsec implementations.
   - It can be used in conjunction with AH for enhanced security or individually based on the specific security requirements of the deployment.

In summary, ESP in IPsec provides confidentiality, integrity, and authentication services for IP packets, making it a powerful tool for securing communications over IP networks. It encrypts the payload of the IP packet to ensure confidentiality, and calculates a cryptographic hash over the entire packet to ensure integrity and authentication.



 VLAN (Virtual Local Area Network) tagging is a technique used in computer networking to allow multiple VLANs to coexist on the same physical network infrastructure, such as Ethernet. VLAN tagging involves adding an additional header to Ethernet frames to identify which VLAN the frame belongs to. This allows network switches to differentiate between traffic from different VLANs and appropriately forward the frames to their destination VLANs. 

The VLAN tagging header consists of two main components:

1. **VLAN Tag**: This is a 4-byte header inserted into the Ethernet frame between the source MAC address and the EtherType/length field. The VLAN tag contains the following fields:

   - **Priority**: 3 bits used to define the priority level of the frame (0-7). This is often referred to as the Quality of Service (QoS) priority.
   - **C (Canonical Format Identifier)**: 1 bit used to indicate whether the VLAN ID field is in canonical format (0) or non-canonical format (1). This bit is used to ensure interoperability between different network devices.
   - **VLAN ID (VID)**: 12 bits used to specify the VLAN to which the frame belongs. This field allows for up to 4096 unique VLANs (0-4095), although the actual number of usable VLANs may be lower depending on the specific networking equipment and configurations.

2. **EtherType/Length**: This field follows the VLAN tag and indicates the type of the payload data or the length of the frame. For VLAN-tagged frames, the EtherType field is set to a specific value (0x8100) known as the VLAN EtherType, indicating the presence of a VLAN tag.

When a network device receives a VLAN-tagged frame, it examines the VLAN tag to determine the appropriate VLAN for the frame. The device then forwards the frame only to ports associated with that VLAN, ensuring that the frame is delivered only to devices within the same VLAN. This segmentation helps to improve network security, reduce broadcast traffic, and facilitate network management.

VLAN tagging is commonly used in enterprise networks, data centers, and service provider networks to segment network traffic, isolate broadcast domains, and facilitate network management and troubleshooting. It is an essential feature of modern Ethernet networks and is supported by most managed switches and network devices.


The TCP (Transmission Control Protocol) header is a fundamental component of the TCP/IP protocol suite, which governs communication between devices over the Internet. It resides in the Transport Layer of the TCP/IP model and provides reliable, connection-oriented communication between hosts. The TCP header consists of various fields, each serving a specific purpose in managing the transmission of data. Here's an explanation of the key fields found in a TCP header:

1. **Source Port (16 bits)**:
   - This field identifies the source port number of the sending device. Port numbers range from 0 to 65535 and help to differentiate multiple connections on the same host.

2. **Destination Port (16 bits)**:
   - This field identifies the destination port number on the receiving device. Similar to the source port, it helps the receiving device identify the application or service to which the segment is destined.

3. **Sequence Number (32 bits)**:
   - The sequence number field specifies the sequence number of the first data byte in the TCP segment. It is used for ordering and reassembling received segments at the receiving end. 

4. **Acknowledgment Number (32 bits)**:
   - In TCP, acknowledgments (ACKs) are used to confirm receipt of data segments. This field contains the sequence number that the receiver expects to receive next. It acknowledges the receipt of all previous bytes.

5. **Data Offset (4 bits)**:
   - This field indicates the length of the TCP header in 32-bit words. Since the TCP header size can vary (due to options), this field is used to determine the start of the data payload.

6. **Reserved (3 bits)**:
   - These bits are reserved for future use and are set to zero in the current specification of TCP.

7. **Flags (9 bits)**:
   - TCP uses various control flags to manage the connection state and control the behavior of data transmission. These flags include:
     - **URG (Urgent Pointer)**: Indicates that the Urgent pointer field is significant.
     - **ACK (Acknowledgment)**: Indicates that the Acknowledgment field is significant.
     - **PSH (Push)**: Push function. Asks to push the buffered data to the receiving application.
     - **RST (Reset)**: Reset the connection.
     - **SYN (Synchronize)**: Synchronize sequence numbers. Used to initiate a connection.
     - **FIN (Finish)**: No more data from sender. Used to terminate a connection.

8. **Window Size (16 bits)**:
   - This field specifies the size of the receive window, which indicates the number of bytes that the sender can transmit before receiving an acknowledgment. It helps in flow control, allowing the sender to regulate the rate of data transmission.

9. **Checksum (16 bits)**:
   - The TCP checksum field is used to detect errors in the TCP segment during transmission. It is computed over the TCP header, data payload, and pseudo-header (including source and destination IP addresses) and helps ensure the integrity of the transmitted data.

10. **Urgent Pointer (16 bits)**:
    - If the URG flag is set, this field indicates the byte offset from the current sequence number where urgent data ends.

11. **Options**:
    - The TCP header may contain optional fields or options, which are used for various purposes such as timestamping, selective acknowledgment (SACK), window scaling, and maximum segment size (MSS).

The TCP header, along with the TCP segment payload, forms the TCP segment, which is encapsulated within an IP packet and transmitted over the network. By providing reliable, connection-oriented communication, TCP ensures that data is delivered accurately and in order between hosts on a network.



The IP (Internet Protocol) header is a crucial component of the TCP/IP protocol suite, responsible for routing data packets across networks. It resides in the Network Layer of the TCP/IP model and provides a set of essential fields that govern the delivery of packets from a source to a destination. Here's an explanation of the key fields found in an IPv4 header:

1. **Version (4 bits)**:
   - This field indicates the version of the IP protocol being used. For IPv4, the value is 4.

2. **Header Length (4 bits)**:
   - The header length field specifies the length of the IP header in 32-bit words. Since the IPv4 header size can vary (due to options), this field is used to determine the start of the data payload.

3. **Type of Service (8 bits)**:
   - The Type of Service (TOS) field is used to specify the quality of service (QoS) level for the packet. It includes subfields such as Precedence, Delay, Throughput, Reliability, and Cost.

4. **Total Length (16 bits)**:
   - This field specifies the total length of the IP packet, including the header and data payload. It ranges from 20 to 65,535 bytes.

5. **Identification (16 bits)**:
   - The identification field is used for fragmentation and reassembly of IP packets. It helps to uniquely identify fragments belonging to the same original packet.

6. **Flags (3 bits)**:
   - The Flags field contains control flags used for fragmentation. The three flags are:
     - **Reserved (Bit 0)**: Reserved for future use and must be set to 0.
     - **Don't Fragment (DF) (Bit 1)**: If set, indicates that the packet should not be fragmented.
     - **More Fragments (MF) (Bit 2)**: If set, indicates that more fragments follow the current fragment.

7. **Fragment Offset (13 bits)**:
   - This field indicates the offset of the current fragment relative to the beginning of the original packet. It is used in conjunction with the Identification field for reassembly.

8. **Time to Live (TTL) (8 bits)**:
   - The TTL field specifies the maximum number of hops (routers) that the packet can traverse before being discarded. Each router decrements the TTL value by at least one, and if it reaches zero, the packet is discarded.

9. **Protocol (8 bits)**:
   - This field identifies the protocol used in the data payload of the IP packet (e.g., TCP, UDP, ICMP). It helps the receiving device determine how to process the packet payload.

10. **Header Checksum (16 bits)**:
    - The header checksum field is used to detect errors in the IP header during transmission. It is computed over the IP header and helps ensure the integrity of the transmitted packet.

11. **Source IP Address (32 bits)**:
    - This field specifies the source IP address of the sending device, identifying the originator of the packet.

12. **Destination IP Address (32 bits)**:
    - This field specifies the destination IP address of the receiving device, identifying the intended recipient of the packet.

The IP header, along with the data payload, forms the IP packet, which is encapsulated within a data link layer frame (e.g., Ethernet frame) and transmitted over the network. By providing a set of essential fields for routing and delivery, the IP header ensures that data packets are properly routed across interconnected networks.

PING (Packet Internet Groper) is a utility used to test the reachability of a host on an Internet Protocol (IP) network and to measure the round-trip time (RTT) for packets sent from the originating host to a destination computer. The PING utility works by sending ICMP Echo Request packets to the target host and waiting for ICMP Echo Reply packets in response.

While PING itself doesn't have a "header" in the traditional sense like TCP or IP packets, ICMP (Internet Control Message Protocol) packets, including the Echo Request and Echo Reply packets used by PING, do have headers. Here's an explanation of the key fields found in an ICMP Echo Request and Echo Reply packet:

1. **Type (8 bits)**:
   - The Type field specifies the type of ICMP message. For PING, the Type is set to 8 for Echo Request packets and 0 for Echo Reply packets.

2. **Code (8 bits)**:
   - The Code field provides additional information or context for the ICMP message. For PING, the Code is typically set to 0.

3. **Checksum (16 bits)**:
   - The Checksum field is used to detect errors in the ICMP message during transmission. It is computed over the entire ICMP packet, including the header and data payload, and helps ensure the integrity of the transmitted packet.

4. **Identifier (16 bits)**:
   - The Identifier field is used to identify the PING process sending the Echo Request packet. It allows the sender to match Echo Reply packets to the corresponding Echo Request packets.

5. **Sequence Number (16 bits)**:
   - The Sequence Number field is incremented for each Echo Request packet sent by the PING process. It allows the sender to identify and match Echo Reply packets to the corresponding Echo Request packets.

6. **Data (Variable length)**:
   - The Data field contains optional data sent by the sender in the Echo Request packet. This data is typically used for diagnostic purposes or to measure network performance.

When the destination host receives an ICMP Echo Request packet, it responds with an ICMP Echo Reply packet. The Echo Reply packet contains the same Identifier and Sequence Number as the corresponding Echo Request packet, allowing the sender to match the reply to the original request.

In summary, while PING itself doesn't have a header, the ICMP packets used by PING do have headers that contain essential fields such as Type, Code, Checksum, Identifier, Sequence Number, and optional Data. These fields help facilitate communication between the PING sender and the destination host, allowing for network testing and troubleshooting.


ICMP (Internet Control Message Protocol) is a network layer protocol used to send control messages and error reporting between network devices in an IP network. ICMP messages are encapsulated within IP packets and are used for various purposes, including network diagnostics, error reporting, and management. The ICMP header consists of several fields that are used to convey information between network devices. Here's an explanation of the key fields found in an ICMP header:

1. **Type (8 bits)**:
   - The Type field specifies the type of ICMP message. It indicates the purpose or function of the ICMP packet. Common types of ICMP messages include:
     - **Echo Request (Type 8)**: Used by the PING utility to test reachability and measure round-trip time to a destination.
     - **Echo Reply (Type 0)**: Sent in response to an Echo Request message, confirming that the destination is reachable and providing round-trip time information.
     - **Destination Unreachable (Type 3)**: Indicates that the destination host or network is unreachable.
     - **Time Exceeded (Type 11)**: Indicates that the Time to Live (TTL) field of an IP packet has expired.
     - **Parameter Problem (Type 12)**: Indicates that there is an issue with one or more parameters in the IP header.
     - **Redirect (Type 5)**: Informs a host about a better route to a particular destination.

2. **Code (8 bits)**:
   - The Code field provides additional information or context for the ICMP message. It further specifies the type of ICMP message. For example, for an ICMP Destination Unreachable message, the Code field might indicate the reason for the unreachable condition (e.g., network unreachable, host unreachable).

3. **Checksum (16 bits)**:
   - The Checksum field is used to detect errors in the ICMP message during transmission. It is computed over the entire ICMP packet, including the header and data payload, and helps ensure the integrity of the transmitted packet.

4. **Data (Variable length)**:
   - The Data field contains optional data specific to the type and code of the ICMP message. The content of this field varies depending on the purpose of the ICMP message. For example, in an Echo Request message, the Data field may contain arbitrary data used for testing, while in a Destination Unreachable message, it may include an encapsulated IP header and a portion of the original packet that triggered the error.

The ICMP header is encapsulated within an IP packet, and ICMP messages are typically transmitted as IP datagrams over an IP network. ICMP plays a critical role in network troubleshooting and management by providing feedback about the status of network connections and assisting in diagnosing network-related issues.


The DNS (Domain Name System) protocol is responsible for translating domain names (e.g., www.example.com) into IP addresses (e.g., 192.0.2.1) and vice versa. The DNS operates through a client-server model, where DNS clients (such as web browsers or other applications) send DNS queries to DNS servers to resolve domain names to IP addresses.

The DNS header is a fundamental component of DNS messages and contains essential information about the DNS query or response. Here's an explanation of the key fields found in a DNS header:

1. **Transaction ID (16 bits)**:
   - The Transaction ID field uniquely identifies the DNS query and its corresponding response. It allows the DNS client to match responses with the original queries.

2. **Flags (16 bits)**:
   - The Flags field contains various control flags that specify the type of DNS message (query or response), query type, response code, and other parameters. Some important flags include:
     - **QR (Query/Response)**: Indicates whether the message is a query (0) or a response (1).
     - **Opcode**: Specifies the type of query (e.g., standard query, inverse query, status request).
     - **AA (Authoritative Answer)**: Indicates whether the responding DNS server is authoritative for the queried domain.
     - **TC (Truncated)**: Indicates that the DNS message was truncated due to being too large to fit in a single UDP packet.
     - **RD (Recursion Desired)**: Indicates whether the client requests recursive resolution from the DNS server.
     - **RA (Recursion Available)**: Indicates whether the DNS server supports recursive resolution.
     - **RCODE (Response Code)**: Specifies the result of the DNS query (e.g., no error, name error, server failure).

3. **Question Count (16 bits)**:
   - The Question Count field specifies the number of entries in the question section of the DNS message. Each entry represents a domain name that the client is querying.

4. **Answer Record Count (16 bits)**:
   - The Answer Record Count field specifies the number of resource records in the answer section of the DNS response. Each resource record contains information mapping domain names to IP addresses.

5. **Authority Record Count (16 bits)**:
   - The Authority Record Count field specifies the number of resource records in the authority section of the DNS response. Authority records provide information about the authoritative DNS servers for the queried domain.

6. **Additional Record Count (16 bits)**:
   - The Additional Record Count field specifies the number of additional resource records in the additional section of the DNS response. Additional records may include additional information relevant to the query, such as DNS server addresses or DNSSEC signatures.

The DNS header is followed by the question section, which contains the domain names being queried, and optional answer, authority, and additional sections containing resource records with DNS resolution information.

Overall, the DNS header contains critical metadata about DNS queries and responses, enabling efficient communication between DNS clients and servers and facilitating the resolution of domain names to IP addresses.


The ARP (Address Resolution Protocol) is a communication protocol used to map IP addresses to MAC addresses within a local network segment. It helps devices on the same network determine each other's hardware (MAC) addresses when only their IP addresses are known. ARP operates at the data link layer (Layer 2) of the OSI model.

The ARP protocol doesn't have a distinct header like some other protocols (e.g., TCP, UDP), but rather its fields are embedded within an Ethernet frame. When an ARP message is sent, it is encapsulated within an Ethernet frame, and the ARP information is contained within the payload of the Ethernet frame.

Here's an explanation of the key fields found in an ARP message:

1. **Hardware Type (2 bytes)**:
   - Specifies the type of hardware (e.g., Ethernet) being used on the network. Common values include 1 for Ethernet.

2. **Protocol Type (2 bytes)**:
   - Indicates the protocol type being used (e.g., IPv4). Common values include 0x0800 for IPv4.

3. **Hardware Address Length (1 byte)**:
   - Specifies the length of hardware addresses (MAC addresses) in bytes. For Ethernet, this value is typically 6 bytes.

4. **Protocol Address Length (1 byte)**:
   - Specifies the length of protocol addresses (IP addresses) in bytes. For IPv4, this value is typically 4 bytes.

5. **Operation (2 bytes)**:
   - Indicates the type of ARP message being sent. Common values include:
     - 1: ARP Request
     - 2: ARP Reply

6. **Sender Hardware Address (variable length)**:
   - Contains the MAC address of the sender of the ARP message.

7. **Sender Protocol Address (variable length)**:
   - Contains the IP address of the sender of the ARP message.

8. **Target Hardware Address (variable length)**:
   - In an ARP Request, this field is typically empty (all zeroes) since the sender is requesting the MAC address of another device. In an ARP Reply, it contains the MAC address of the target device.

9. **Target Protocol Address (variable length)**:
   - Contains the IP address of the target device.

When a device needs to send data to another device on the same local network segment, it checks its ARP cache to see if it has the MAC address of the destination IP address. If not, it sends out an ARP Request message broadcasted to all devices on the local network asking, "Who has this IP address? Please tell me your MAC address." The device with the matching IP address then responds with an ARP Reply message containing its MAC address.

Overall, ARP plays a crucial role in enabling communication between devices on the same network by resolving IP addresses to MAC addresses, facilitating the transmission of data at the data link layer.

The UDP (User Datagram Protocol) header is a fundamental component of the UDP protocol, which operates at the transport layer (Layer 4) of the OSI model. UDP is a connectionless and unreliable protocol used for transmitting datagrams between hosts on an IP network. The UDP header contains information necessary for delivering UDP datagrams from the source to the destination.

Here's an explanation of the key fields found in a UDP header:

1. **Source Port (16 bits)**:
   - This field specifies the source port number of the sending application or process. Port numbers range from 0 to 65535 and help identify the application or service sending the UDP datagram.

2. **Destination Port (16 bits)**:
   - This field specifies the destination port number on the receiving host. Similar to the source port, it helps identify the application or service to which the UDP datagram is destined.

3. **Length (16 bits)**:
   - The Length field specifies the length of the UDP datagram, including the header and data payload, in bytes. The minimum value for this field is 8 bytes (the size of the UDP header), and the maximum value is 65,535 bytes.

4. **Checksum (16 bits)**:
   - The Checksum field is used to detect errors in the UDP datagram during transmission. It is optional in UDP, and if not used, the field is set to zero. When checksumming is enabled, the sender calculates a checksum over the entire UDP datagram (including the UDP header, pseudo-header, and data payload) and includes it in this field. The receiver recalculates the checksum and compares it to the value in the UDP header to verify the integrity of the datagram.

The UDP header is relatively simple compared to other transport layer protocols like TCP. It provides minimal functionality, focusing primarily on source and destination port identification and datagram length. UDP is widely used for applications that prioritize speed and efficiency over reliability, such as real-time multimedia streaming, DNS (Domain Name System), and DHCP (Dynamic Host Configuration Protocol). However, because UDP lacks features like error correction, flow control, and congestion control, applications built on UDP must implement their own mechanisms for handling these issues if necessary.


Ethernet

The Ethernet header is a fundamental component of Ethernet frames, which are used to encapsulate data for transmission over Ethernet networks. Ethernet operates at the data link layer (Layer 2) of the OSI model and provides a mechanism for delivering data between devices on the same local network segment. The Ethernet header contains essential information necessary for the transmission and reception of Ethernet frames.

Here's an explanation of the key fields found in an Ethernet header:

1. **Destination MAC Address (6 bytes)**:
   - This field specifies the MAC (Media Access Control) address of the destination device to which the Ethernet frame is being sent. MAC addresses are unique identifiers assigned to network interfaces and are used for addressing and routing Ethernet frames within a local network.

2. **Source MAC Address (6 bytes)**:
   - This field specifies the MAC address of the sending device that originates the Ethernet frame. It identifies the source of the frame and allows the recipient to send responses or acknowledgments back to the sender.

3. **EtherType or Length (2 bytes)**:
   - The EtherType field specifies the type of the payload data encapsulated within the Ethernet frame. Common EtherType values include:
     - **IPv4 (0x0800)**: Indicates that the payload data is an IPv4 packet.
     - **IPv6 (0x86DD)**: Indicates that the payload data is an IPv6 packet.
     - **ARP (0x0806)**: Indicates that the payload data is an ARP (Address Resolution Protocol) message.
     - **VLAN Tagging (0x8100)**: Indicates the presence of VLAN tagging in the Ethernet frame.

   Alternatively, when Ethernet II framing is used, this field is referred to as Length and specifies the length of the payload data in bytes.

4. **Optional VLAN Tag (4 bytes)**:
   - If VLAN tagging is used, an additional 4-byte VLAN tag may be present in the Ethernet header. The VLAN tag contains information such as the VLAN identifier (VID) and priority level for the frame.

5. **Frame Check Sequence (FCS) (4 bytes)**:
   - The FCS field contains a cyclic redundancy check (CRC) value computed over the entire Ethernet frame (excluding the preamble and start frame delimiter) to detect transmission errors. The receiving device calculates its own CRC value based on the received frame and compares it to the FCS value in the Ethernet frame to verify the integrity of the data.

The Ethernet header, along with the payload data and FCS, forms the Ethernet frame, which is transmitted over the physical network medium. By providing addressing, error detection, and EtherType information, the Ethernet header facilitates the reliable transmission of data between devices on an Ethernet network.



Troubleshoot CLI's - ESX, NSX, NSX-Intelligence, Edge, FW, IDS/IPS

 On NSX mgr
==========
nsx-mgr-1> get managers
nsx-mgr-1> get management-cluster status
nsx-mgr-1> get cluster status
nsx-mgr-1> get nodes
nsx-mgr-1> get interfaces
nsx-mgr-1> get transport-nodes status 
nsx-mgr-1> get configuration
nsx-mgr-1> get cluster config
nsx-mgr-1> get firewall summary
nsx-mgr-1> get firewall published-entity
nsx-mgr-1> get firewall status
nsx-mgr-1> get logical-routers
nsx-mgr-1> get logical-switches
nsx-mgr-1> get service manager
nsx-mgr-1> get log-file syslog follow 
nsx-mgr-1> get logging-servers
root@nsx-mgr-1:~# tail -f ./var/log/proton/nsxapi.log
root@nsx-mgr-1:~# tail -f /var/log/policy/policy.log ## policy mgr logs
nsx-mgr-1> set service manager logging-level debug
nsx-mgr-1> get logical-switch <id> arp-table
 
nsxcli commands on ESX
===================
> get intelligence flow config
Displays current PACE flow configuration
> get intelligence flow stats
Displays flow export statistics
> get intelligence flow stats ack
Displays flow export acknowledgement stats

On ESX:
======
List the VIBs loaded on ESXi
[root@prom-05056b17085:~] esxcli software vib list | grep -e nsx -e vsip
Host to mgr communication
[root@prom-05056b17085:~] esxcli network ip connection list | grep 1235
[root@prom-05056b17085:~] esxcli network nic list
[root@prom-05056b17085:~] esxcli network firewall ruleset set -r syslog -e true
[root@prom-05056b17085:~] esxcfg-nics -l
[root@prom-05056b17085:~] tail -f /var/log/dfwpktlogs.log

Firewall logs on ESX:
===============
[root@prom-05056b17085:~] tail -f /var/log/dfwpktlogs.log
[root@prom-05056b17085:~]  summarize-dvfilter   ## get the VM interface  ID
[root@prom-05056b17085:~] vsipioctl getrules -f nic-6389857-eth2-vmware-sfw.2
[root@prom-05056b17085:~] cat /var/log/dfwpktlogs.log
[root@prom-05056b17085:~] nsxcli
prom-05056b17085.system.test> get logical-switches
prom-05056b17085.system.test> get firewall vifs
prom-05056b17085.system.test> get firewall rule-stats
prom-05056b17085.system.test> get firewall status
prom-05056b17085.system.test> get firewall <vifuuid> ruleset rules
prom-05056b17085.system.test> get firewall <vifuuid> addrsets
prom-05056b17085.system.test> get logical-router 51515dfa-b15a-45d6-9d60-90e4c40023e1 interfaces
prom-05056b17085.system.test> get logical-router 51515dfa-b15a-45d6-9d60-90e4c40023e1 interface 85d532cc-2627-4ae0-8ee6-e7ba38f67572
prom-05056b17085.system.test> get host-switch PROD-Overlay-NVDS tunnels

ESX:
===
[root@blrkv-hs1-b0606:~] esxcli network ip interface ipv4 get
[root@blrkv-hs1-b0606:~] esxcli vm process list
[root@blrkv-hs1-b0606:~] esxcli storage vmfs extent list

NSX Edge:
=========
edge> get service ssh
edge> get configuration 
edge> get interfaces
edge> get managers
edge> get host-switches
edge> get tunnel-ports
edge> get vteps
edge> get logical-routers
edge> get logical-switches
edge> get firewall interfaces
edge> get firewall <interfaceid> ruleset
edge> get firewall <interfaceid> ruleset rules
edge> get dhcp ip-pools
edge> get dhcp leases

On NSX-Intelligence:
===============
Intel> get service
Intel> get version
Intel> get services
get log-file syslog | find pace-monitor
Intel>restart service service-name
root@Intel:~# cat /var/log/pace/token-registration.log 
root@Intel:~# cat /var/log/pace/pace-server.log
root@Intel:~# tail -f /var/log/pace/nsx-config.log

Periodic health check service called pace-monitor that posts the overall appliance health to syslog.
root@Intel:~# journalctl -u pace-monitor -n 60 --no-pager
Intel> get service druid

API:
====
root@systest-runner:~[1154]# curl -k -u username:password  https://<nsx-ip>/api/v1/logical-ports
root@systest-runner:~[1156]# curl -k -u username:password https://<nsx-ip>/api/v1/firewall/status
root@systest-runner:~[1158]# curl -k -u username:password https://<nsx-ip>/api/v1/firewall/sections/<id>/rules
root@systest-runner:~[1168]# curl -I -k -u admin:VMware1VMware! https://<nsx-ip>/policy/api/v1/infra/realized-state/realized-entities

On worknode for scan:
=================
root@ubuntu-1804:~# nmap -T4 -Pn -A -v <server-ip>
Starting Nmap 7.60 ( https://nmap.org ) at 2021-12-07 11:01 UTC
NSE: Loaded 146 scripts for scanning.
-T<0-5>: Set timing template (higher is faster)
-A: Enable OS detection, version detection, script scanning, and traceroute
-v: Increase verbosity level (use -vv or more for greater effect)
 -V: Print version number
-Pn: Treat all hosts as online -- skip host discovery

IPS/IDS On ESX
===========
[root@prom-05056b17085:~] nsxcli -c get ids engine stats
[root@prom-05056b17085:~] nsxcli -c get ids events stats

Capture on the edge:
================
edge> get logical-routers
edge(tier1_sr[1])> get interfaces
edge(tier1_sr[1])> get interfaces stats
edge> start capture interface <id>
edge> start capture interface <id>

From client initiate traffic:
root@ubuntu-1804:~# nmap --script http-vuln-cve2014-3704 --script-args http-vuln-cve2014-3704.uri=“/drupal”,http-vuln-cve2014-3704.cleanup=false <server-ip> --disable-arp-ping

Logs:
[root@blrkv-hs1-b0606:~] cd /var/log/nsx-idps
 [root@blrkv-hs1-b0606:/var/log/nsx-idps] summarize-dvfilter
 
To confirm that IDS is enabled on this host, run the command get ids status
On ESX -> nsxcli

prom-05056b17085.system.test> get ids profile

To review IDS profile (engine) statistics including the number of packets processed and alerts generated, 
run the command get ids engine profilestats <tab_to_select_profile_ID>

[root@prom-05056b96efd:~] summarize-dvfilter | grep -A 3 vmm

Get all the DFW rules using "vsipioctl getrules -f <VM_interface_ID>"

Get the flow statistics for rule using command "vsipioctl getrules -f <VM_interface_ID> -s"

Get the addrset/groups used in the VM's Firewall rules

[root@prom-05056b17085:/var/log/nsx-idps] vsipioctl getaddrset -f <nic>

Get the active Firewall flow per VM. Trigger some traffic and verify
[root@prom-05056b1c8f2:/var/log/nsx-idps] vsipioctl getflows -f <nic>

Get the active Full Firewall config per VM
[root@prom-05056b17085:/var/log/nsx-idps] vsipioctl getfwconfig -f <nic>

Monday, December 27, 2021

DevOps Basics

Git vs Github
Git is a protocol or the version control system that helps us track and merge changes.
And Github is the GIT server that host remote repositories. 

Webserver introduction:

Say we have website, it’s an e-commerce application. It is hosted on a webserver. So when a user opens a browser and types in the address to the server, a request is sent to the server hosting the application.

On receiving the request, the server sends back its response which is the web page along with the data about a list of products available on the site. And that’s what the user sees on the page. The machine running the program is the web server and the browser is the client. The client sends a request to the server and the server returns a response. So there are two sets of code in work here.

A set of code that runs on the server side that is actually hosting the web server and that’s the code that’s listening for connection from users and querying databases, etc. So, this is called the server-side code or the backend code.

This is usually written in programming languages like Java, Python or NodeJS. And the response sent back by the server could be another set of code that runs on the client-side that is used to display webpages and information within the pages. These are called client-side code or also referred to as front-end code. These are usually in the form of HTML, CSS, and JavaScript.They run on the client-side in the user’s browsers. Users can even see that code inside their browser using the browser developer tools. So both these sets of code are developed together as part of the same application.


Say you have server-side code written in Java that uses a Spring Boot web framework.


Now you have developed the code of your web application, but how do you serve it? How do you run it so this code listens on a port and responds back to the users?

For some of these frameworks like Python, for example, flask, for instance, provides a way to run the application natively but it is not recommended for hosting the application in a production environment. Thats where a web server comes in so while a web framework is what helps you develop application code and web server is what hosts this application.


So you basically take all your code that you have developed and put it on a web server. Note that the code need not be in the original format, it could be in case of Python or NodeJS based application, but in case of JAVA, it is usually packaged. So Apache Tomcat, GlassFish, Nginx, GUnicorn, uWSGI are examples of servers that can host web applications. The webservers run one or more processes that listen on a particular port for requests and performs the required operations and responds back to the users. Once done, for example, Tomcat listens on port 8080 by default, whereas Nginx listens on port 80, and Gunicorn listens on port 8000 by default. And off-course these port can be change as per our wish. Also, web servers can host multiple applications at the same time. So, they can redirect requests based on the URL or hostname or a path to different applications.


Websites can be broadly classified into two types:

A website that only uses HTML pages, some CSS styling, and images such as a website that shows a list of pictures as an image gallery could be classified as a static website because it only uses static content. That is content that does not change like HTML files and CSS files. There is no real interaction with the server after the content is served. However, an e-commerce website that lists products and provides the ability for users to add a cart and make purchases are called dynamic websites. They often have a back-end and some logic running for processing payments, shipping orders, getting a list of products and purchases, etc. So dynamic websites usually require both static and dynamic content. That is static content as HTML pages and images are used to show the webpage, but at the same time, information about products are retrieved form the database before displaying them on the web page, So, web pages are likely to be displayed differently based on the data in the database. And that’s why they are called dynamic websites so they have code and databases in the backend. 


There are different web server technologies that serve different purposes. The Apache HTTP web server and NGinx are good example of static web servers that serve static content. Whereas the Apache Tomcat, uWSGI, Gunicorn are good examples of dynamic web servers or application servers.


Static web servers are usually referred to as web servers, and servers that host dynamic web sites are referred to as application servers. 


Apache Web server:

The Apache web server is an open-source HTTP server developed and maintained by the Apache software foundation. It is a web server, usually used to serve web content like HTML, CSS, and JavaScript files.

It also used in conjunction with an application server that acts as a back-end for running business logic, such as Apache Tomcat server. 


Install Apache web server:

yum install httpd

service httpd start

service httpd status


If you have firewall configured on your system, then make sure to add a rule to allow HTTP traffic.


View logs:

Every server has a path where it stores the logs. To view log files


cat /var/log/httpd/error_log

cat /var/log/httpd/access_log


The access logs are update whenever a user accesses the website, and the error logs are updated whenever there is an error. Also every server has a configuration file, where you configure the different parameters that govern

How the server operates, such as what port it listens on and where the static content is stored and any SSL or HTTPS configurations, where logs are stored etc. In case of apache web server, it is at, /etc/httpd/conf/httpd.conf path.

The DocumentRoot defines the location where static content is stored. So move all the static contents of your website to this directory. Another configuration is the server name for this you must have a DNS entry configured in your networks DNS server to point that host to the IP address of your system.


A single apache server can host multiple websites. To do that, the files and configuration of each website is configured as a virtual host within the apache configuration file. So, a virtual host is a logical division of the Apache web server.

Each virtual host can be configured with its own server name and document root. Each virtual host configuration file can also be moved to separate configuration file. 


Apache Tomcat:

The Apache Tomcat server provides a web server environment where we can host Java-based web applications. So as a prerequisite you must have java installed. 



By default apache web server runs on port 80 and Apache Tomcat on port 8080. The apache Tomcat server is a web server capable of hosting multiple Java web applications. Once you extract the package under bin you can find .bat files for windows and .sh for linux. Out of these startup.sh and shutdown.sh scripts are used to start and stop the Tomcat server. Under conf directory we have multiple conf files where we configure what port to listen and how to direct the traffic between the different web apps, etc. The server.xml file has entry for connector. The connector is the endpoint at which the requests are received. So it’s set to 8080 and that’s the default port that Tomcat listens on, and the port number can be changed if required. Any change will require a restart of Tomcat service. The web dot.xml file is used to deploy and configure web applications. The logs directory is where the logs are stored. And the web apps directory is where web applications hosted by Tomcat server are located and this is where you should place your application that you want apache to serve.



Database basics:

Databases are a key component of system Design

Operations engineers deploy and manage databases

Applications read and write to these databases


Mostly databases are categorised as SQL and non-SQL based. MySQL and PostgreSQL are SQL based and MongoDB, Cassandra are example of Non SQL DB.


SQL or relational DB store data in the form of rows and columns. Adding additional entry will affect complete data.


No SQL databases store information in the form of documents or pages, These files can be in any format or structure and changes to one file does not affect the others.


In summary in SQL each person data is represented in a row but in No SQL each person gets their own document. Together all rows in a SQL database form a table. In a no SQL database, all documents together form a collection.

We typically use SQL queries to query and filter data from a SQL database. 


Famous websites like fb, Goggle and Youtube use MySQl as their database. 



Log location is /var/log/mysqld.log

The default port that MySQL listens on is 3306


MongoDB is an Open source, NoSQL, scalable and high performance also known as document database as it stores data in JSON like document format. Multiple documents together form a collection. Multiple collections together form a database and you can have multiple such databases with a single MongoDB server. 




Logs are stored at path /var/log/mongodb/mongod.log


Default port that MongoDB listens on is 27017


These setting can be changes at configuration file located at /etc/mangod.conf


DevOps is a set of practices, principles, and cultural philosophies that aim to improve collaboration, communication, and integration between software development (Dev) and IT operations (Ops) teams. The primary goal of DevOps is to streamline the software delivery process, accelerate deployment cycles, and improve the quality and reliability of software applications. DevOps emphasizes automation, continuous integration (CI), continuous delivery (CD), and close collaboration between development, operations, and other stakeholders throughout the software development lifecycle (SDLC).


Key aspects of DevOps include:


1. **Culture and Collaboration**: DevOps promotes a culture of collaboration, transparency, and shared responsibility across development, operations, quality assurance (QA), and other teams involved in software delivery. It encourages breaking down silos between teams, fostering open communication, and promoting a "fail-fast, learn-fast" mentality.


2. **Automation**: DevOps advocates for the automation of repetitive tasks, such as code builds, testing, deployment, and infrastructure provisioning. Automation helps reduce manual errors, improve efficiency, and accelerate the software delivery process.


3. **Continuous Integration (CI)**: CI is a practice in DevOps where developers frequently integrate their code changes into a shared repository (e.g., version control system) multiple times a day. Each integration triggers automated builds and tests to ensure that the codebase remains stable and functional.


4. **Continuous Delivery (CD)**: CD extends CI by automating the release process to ensure that software can be deployed to production environments quickly, reliably, and frequently. CD pipelines automate deployment, testing, and validation tasks, enabling organizations to release new features and updates to end-users with minimal manual intervention.


5. **Infrastructure as Code (IaC)**: IaC is a DevOps practice where infrastructure resources, such as servers, networks, and databases, are defined and managed using code and configuration files. IaC enables the automation of infrastructure provisioning, configuration, and management, making it easier to scale, replicate, and maintain infrastructure environments.


6. **Monitoring and Feedback**: DevOps emphasizes the importance of monitoring application performance, infrastructure health, and user feedback. Continuous monitoring helps identify issues, track performance metrics, and gather insights for continuous improvement and optimization.


7. **Security and Compliance**: DevOps integrates security and compliance practices into the software delivery pipeline from the early stages of development. DevSecOps extends DevOps principles to include security considerations throughout the SDLC, ensuring that security is built into the software from the outset.


Overall, DevOps is a holistic approach to software development and operations that fosters agility, collaboration, and innovation, enabling organizations to deliver high-quality software products and services more efficiently and reliably in today's fast-paced digital economy.