DevOps

DevOps is the union of people, process and products to enable continuous delivery of value to end users.

Delivery in iterative and incremental ways to end users.

Collaborate effectively to achieve a common goal. Dev and Ops teams should work on collective organisation goals.

Dev and Ops team should have same goals.

Dev and Ops should follow process that should add continusous value. One way of adding value is by eliminating the waste or waiting time in project.

Get continuous feedback from the customer and use it in the production

Summary:
People coming together for common goals
Adding value to end users (process)
Using tools (products)
Devops => DEv + Ops

The goal of DevOps is to shorten the systems development life cycle while also delivering features, fixes, and updates frequently in close alignment with business objectives. The DevOps approach is to include automation and event monitoring at all steps of the software build. DevOps adoption is estimated to be around 81 percent for larger organizations.

Benefits of using DevOps for companies include:

Better team collaboration and trust
Decreased problem resolution time
Increased quality and stability of code

TCP HALF OPEN

Tcp half open is the state where the client sends SYN to server and server replies with a SYN ACK but the client never send ACK to the server. This will keep TCP session open. The server will keep resending SYN ACK until timeout expires.

This is mainly used for port scanning purposes, just to check ports open on server side. SYN packet will contain the port number of server which it wants to scan. Client will keep retrying with different port numbers until it receives SYN ACK from server. The firewall can be used to block such client to avoid port scan attack.

Forward Proxy vs Reverse Proxy Server

A forward proxy (or just proxy) is a server that sits between user devices and the internet. A forward proxy is commonly used for:

- Protecting client privacy and anonymity
- Avoiding browsing restrictions
- Blocking access to certain content

A reverse proxy is a server that accepts requests from clients, forwards them to backend web servers, and returns the responses to the clients. The clients interact with the reverse proxy as if it was the origin server. Reverse proxies are good for:

- Protecting backend servers from direct exposure to clients
- Load balancing requests across multiple backend servers
- Caching static content closer to clients for faster delivery
- Terminating SSL connections and offloading encryption/decryption tasks from the backend servers

The key difference is that a forward proxy acts on behalf of clients, while a reverse proxy acts on behalf of servers. Forward proxies handle outbound requests to external servers, while reverse proxies handle inbound requests coming from clients.

Two common types of proxy are forward and reverse proxy. 

A forward proxy is a server that sits between the group of client machines and the internet. When those clients makes request to websites on internet, the forward proxy acts as a middle man, intercepts those requests, and talk to web servers on behalf of those client machines. 

Why is this required?

First, Forward proxy protects the client's online identity. By using a forward proxy to connect to a website . the IP address of the client is hidden from the server. Only the ip address of the proxy is visible. It would be harder to trace back to the client. 

Second, Forward proxy can also used to bypass the browsing restrictions. Some institutions like governments, schools, and big business use firewalls to restrict access to the internet. By connecting to the forward proxy outside the firewalls, the client machine can potentially can around these restrictions. It does not always work because the firewalls could themselves block the connection to proxy. 

Three, A forward proxy can can used to block access to the certain content. This is not uncommon for schools and business to configure their network to connect to all clients to the web through the proxy and apply filtering rules to disallow sites like social networks. It is worth noting that a forward proxy normally requires a client to configure its application to point to it. For large institutions they usually apply  technique called transparent proxy to streamline the process. 

A transparent proxy works with L4 switches to redirect certain types of traffic to the proxy automatically. There is no need to configure client machine to use it. It is difficult to bypass a transparent proxy when the client is on the institution's network. In summary a forward proxy sits between the clients and the internet and acts on behalf of the client. 

Reverse proxy sits between the internet and the web servers. It intercepts the request from the clients and talks to the web server on behalf of the clients. 

Why would a website use a reverse proxy?

One, a reverse proxy could be used to protect website itself. The website's Ip address are hidden behind the reverse proxy and are not revealed to the clients. This makes it much harder to target a DDoS attack against a website. 

Second, a reverse proxy is used for load balancing. A popular website handling millions of users everyday is unlikely to be able to handle the traffic with a single server. A reverse proxy can balance a large amount of incoming requests by distributing the the traffic to the large pool of web servers, and effectively preventing any single of them from becoming overloaded. Services like Cloudflare put reverse proxy servers in hundreds of locations all around the world. This puts the reverse proxy close to the user and at the same time  provides large amount of processing capacity. 

Third, a reverse proxy caches static content. A piece of content can be cached in reverse proxy for a period of time. If the same piece of content requested again from the reverse proxy, the locally cached version could be quickly returned. 

Fourth, a reverse proxy can handle SSL encryption. SSL handshake is computationally expensive. A reverse proxy can free up the origin servers from these expensive operations. Instead of handling SSL for all clients, a website only needs to handle SSL handshake from a small number of reverse proxies.

Reverse proxy are everywhere. For a modern website it is not uncommon to have many layers of reverse proxy. The first layer could be edge service like cloudflare. The reverse proxies are deployed in hundred of locations worldwide close to the users. 

The second layer could be an API gateway or load balancer the the hosting provider. Many cloud providers combine these two layers into a single ingress service. The use would enter the cloud network at the edge close to the user, and from the edge the reverse proxy connect over a fast fiber network to the load balancer where the request is evenly distributed to the cluster of web servers. 

Proxy is a software that makes request on behalf of client

some organization uses proxy to block unwanted websites or to protect employees from bad websites like phishing websites. 

Another use case is caching. if already some information is cached, the proxy will not even connect to server proxy, it can serve cache contents.

Anonymity from final destination server point of view. Final destination doesn't know originating client.

Microservices ..having different types of proxy ..Http proxy, TCP proxy

In reverse proxy client doesn't know about final destination

proxy will make request still on your behalf. There is one case for load balancing ..proxy can talk to google server 1 or google server2

Q and A

* Can we use proxy and reverse proxy together?

yes

* Can I use Proxy to hide my identity instead of a VPN?

You can but vpn is more secure since it works more at low level. vpn can only see domain ..but proxy can see everything especially if it is working as TLS termination.

* Can I use proxy just for HTTP?

No, there are so many proxies as mentioned in above slide.

Cloudflare with free TLS certificate

Cloudflare is a worldwide cloud reverse proxy with caching, DDOs mitigation and much more

https://www.cloudflare.com/

Webbrowser will make DNS request and DNS server will respond the ip address of the webserver here tlscourse.com and web browser will connect to web server directly

But reverse proxy situation is different. In this case DNS server will resolve tlscourse.com to IP: 48.235.12.1

Webbrowser will make TCP/http connection with Reverse proxy and reverse proxy on behalf of web-browser will establish another TCP session with web server that hosts tlscourse.com

If there are multiple web servers, reverse proxy can establish connection with any of them but for all client it will appear that the web site tlscourse.com is hosted at 48.235.12.1

Benefits of using reverse proxy

================================

One benefit is you are hiding ip address of your real web server.

Another advantage is that a reverse proxy may perform different kind of checks that might eliminate any DOS attacks on web server

Any attacks discovered may be stopped at reverse proxy

Another advantage is reverse proxy may cache the web pages of web server

Reverse proxy will serve the cached web page if it is already cached

Another advantage is based on the location of customer web browser different nearby reverse proxy servers will serve request

Such approach usually significantly reduces latency. Web browser receives data faster from web server

DNS Reflection and Amplification

To establish a connection between your machine, and google, first thing you need to know is what is the IP address of google.com.

For this, first your machine will send udp packet to dns server (can be local dns server or cloudflare or google public dns 8.8.8.8) asking what is the IP address of google.com. As it is UDP so it be unencrypted packet.

The server will reply with another udp packet saying google ip address is eg: 1.2.3.4

Then the client will establish connection with google.com using this IP address

DNS reflection:
Say attacker A is a bad actor. A will send a dns request to DNS server but instead of providing its own IP address in udp request, it will send the IP address of server which it wants to attack.
Packet will be like : 80/B/google.com/D/53, where B is the IP address of actual webserver and D is the IP address of DNS server. This process is also called as ip spoofing.

DNS server will send reply to server B. Packet will look like
53/D/1.2.3.4/B/80

Web server B has no clue what's going on. It doesn't know what is 1.2.3.4, so it will drop the packet.
This is called dns reflection, reflecting request to another server

One UDP inbound request is fine for B, but if there are multiple DNS request and multiple DNS replies to B using different attacker machines and DNS servers, this will create a DDos attack.

Though dns reply packet is very small, but to make things worse, an attacker can ask for more information in dns request like multiple options. In this case dns server will reply with larger packet up to 4000 bytes. Doing this thousand of times..will occupy all resource on server
This is called DNS reflection with Amplification attack

To protect web server B we can place a firewall in between the DNS server and webserver B, but that might not be feasible option always.In this case your application might not work properly as some of your applications might be blocked

Sometime L7 Reverse proxy is used in between the DNS server and webserver, the connection will terminate at proxy, it will decrypt the traffic and establish another connection with the webserver.

OSI Model

At Source

At Destination

Note: Each client can have 7 connection to same server on same ip and port. Each connection will be unique based on the session id present in session layer.

Port forwarding and IPtables

*** Forward packet from one port to another in same machine sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.254.47:8080
-A action
-dport dynamic port
DNAT destination NAT

*** To clean the NAT table sudo iptables -t nat -F *** To Masquerade in case you want to forward to another ip address sudo iptables -t nat -A POSTROUTING -p tcp --dport 80 -j MASQUERADE
This is used if you want to forward to different IP.

*** To make these entries persistent. Once all configuration is done. Configure below command apt-get install iptables-persistent

In case of persistent, entries will remain even after reboot of device.

HTTP

HTTP is a protocol for transferring web pages, text, media, binary files and much more. It stands for hyper text transfer protocol and It is what the Internet pretty much runs on

Client / server Client makes HTTP requests to a server that supports HTTP, the server (web server, http server like apache tomcat, nodejs etc..) responds back with the content. This could be an HTML page, images, JSON and much more.

HTTP anatomy

Request (browser, web app) - URL - Method type - Headers - Body

Response (web server) - Status code - Headers - Body

HTTP 1.0 over tcp - Application Layer 7 - new connection with each request.

HTTP 1.1 over tcp - Persisted connection

HTTP/2 over tcp - Compression - Multiplexing - Server Push - SPDY - Mostly secure by default - Negotiates protocol during TLS (NPN/ALPN)

HTTP/2 over QUIC ( HTTP/3) - Replaces TCP with QUIC (udp with congestion control)

VPN vs Proxy

A VPN (virtual private network) and Proxy are two pieces of tech that each has their applications, differences and pros and cons. Both make requests on behave on the client but they are implemented very differently

In a VPN, the client becomes part of a completely different network . So if a client makes a request to a server, the new network makes the request. A VPN makes a machine part of a another network at the lowest level of the network (layer 2) you can a brand ip address a brand mac address. In A Proxy, the client makes the request to the proxy asking it to go to a destination server, and proxy makes the request to the server. VPN Pros - Encrypts traffic - Redirect all traffic at the lowest level (layer 2) - Access restricted content - Access Private networks (work) VPN Cons - Anonymity (uhmm) - Slow extra hops - Double Encryption - VPN can log your data (DNS) Proxy Pros - Caching Layer 7 - Anonymity - not in case of public proxy - Blocking Websites (transparent proxy) - Control and many applications (load balancing, service mesh, firewall proxy security) - Work on Layer 7 and Layer 4 Proxy Cons - Applications can bypass proxy - Not all traffic is routed. HTTP proxy, HTTPS proxy SOCKS proxy - No encryption by default

What is an HTTP Proxy?

A proxy is a software that intercepts traffic and forward it to the destination on behave of the client. This extra layer provide several advantages such as caching, load balancing, content filtering and much more.. We made a video about proxy vs reverse proxy check it out if you want to learn more about the difference. In this video we will explain the different types of HTTP proxies and the benefits and use cases of using each coming up. Transparent proxy (gateway) HTTP insecure proxy Service Mesh Proxy Transparent proxy It is mostly used by the ISPs, clients don’t know they are connected to transparent proxy. The way it works is it looks at TCP/IP layer 4/3 and forward it to the destination, it might do some content filtering based on the IP address or the port so it blocks certain sites. But thats pretty much it. transparent proxy cannot know which pages are you viewing or your what youtube videos are you watching. It can block you from watching youtube all together but it cannot block you from watching lets say a specific youtube channel that is critical of the government ISP is located at. Transparent proxy doesn’t change the content.

HTTP Proxy (insecure) This kind of proxy is used alot, especially in service meshes like linkerd. This kind of proxy have to be configured in the client to use it. Each request will always be targeted to the proxy IP address / port. So when want to make a GET request to husseinnasser.com, and you have a proxy configured, when you look at the TCP packet for that request the destination IP and port is those of the proxy. The proxy looks at the GET request and specifically the HOST header and establishes another TCP connection to the actual destination on husseinnasser.com. So this kind of proxy maintains two tcp connections. Client to proxy and proxy to destination. The proxy have access to the content, it can block the website. It can know what exact page you are viewing. It knows everything because HTTP is insecure. Assuming youtube uses just HTTP, if you have a proxy setup it can block a specific channel or even video from being viewed.

SSL/TLS Termination, TLS Forward Proxy

This offloads complex crypto TLS from the main server to this proxy so the main server can do what it does best

- TLS Termination Proxy

- Pros and Cons

- Pros - Off load crypto to proxy instead of main servers (no longer a problem) - Have TLS session close to the client - Decrypt data so HTTP accelerators like Varnish can function correctly. - Allow intrusion detection system to analyse traffic - Load balancers can make better decisions on the data (Layer 7) - Service Mesh in microservices architecture

- Cons - If proxy got compromised attacker can gains access to all data. - Limited by the maximum number of connections on the proxy.

All Headers

Perfect Forward Secrecy in TLS

Perfect Forward Secrecy is a property of an encryption system that prevents an attacker from decrypting past recorded sessions even after the private key of the server is leaked.

Problem TLS1.2

Attacker is listening to the conversation and he records all the encrypted GET request messages and saves it though he don't have private key to decrypt.

Say after few months later server had a buffer overflow problem. This allows attacker to ask for more data to sever. As private key is cached and while returning data to attacker, it might contain private key as well.

Now attacker has private key and he can decrypt and will have symmetric key. With symmetric key he can decrypt all the messages and read the contents.

To solve this problem we use PFS. Diffie Hellman is always PFS. This is used just for key exchange not for encryption.

In case of TLS1.3 symmetric key will be ephemeral. It will be used only for that particular session.

In this case even if private key is leaked nobody can read the message.

Note: In case of TLS1.2 there is 2 round of message exchange but in case of TLS1.3 there is single round of message exchange that is advantage of TLS1.3

Ciphers:

###Weak and insecure ciphers:

TLS_RSA_WITH_RC4_128_SHA 

RSA: key exchange algorithm

RC4: symmetric key algorithm

SHA: digital signature algorithm used to sign digital cert.

TLS_DHE_RSA_WITH_AES_256_CBC_SHA 

DHE: Deffe hellmen ephemeral

DH used with RSA is weak. Anything with RSA is not PFS

TLS_RSA_WITH_AES_128_CBC_SHA

##Strong ciphers:

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

ECDHE: ellliptic curve Deffe Hellmen Ephemeral

Anything with ECDHE is strong

TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256

TLS_DHE_RSA_WITH_ARIA_128_GCM_SHA256

TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

To check what all ciphers are used/supported by particular web server use:

https://www.ssllabs.com/ssltest/analyze.html