Sahil's Tech Trek 🧗
Essential Guide to Web Servers: Why Understanding Web Servers is Crucial

Essential Guide to Web Servers: Why Understanding Web Servers is Crucial

Sahil Naik's photo
Sahil Naik
·

16 min read

Table of contents

As a DevOps engineer, you'll frequently work with web servers because they are essential for deploying, managing, and scaling applications. Whether it's configuring servers, optimizing performance, automating setups, or securing them, web servers are part of your day-to-day tasks. Having a solid understanding of their fundamentals makes it much easier to troubleshoot issues, integrate with CI/CD pipelines, and ensure everything runs smoothly in production. Without this knowledge, managing infrastructure and deployments will be difficult.

Why we need a Server?

When you're building a website for example BookMyMovie, it's will be running on our localhost:8080, and it's only accessible from our own computer. If we want to share the website with the world, we need to host it on a server. For that we have two options, well, technically three.

Option 1: Configuring your laptop as a server

We can technically configure our laptop as a server by keeping it running 24/7, ensuring that it has a static IP address (or using a service like dynamic DNS), and forwarding ports from our router. This would allow others to access our website using our laptop's IP address.

However, this approach has several drawbacks:

  1. 24/7 Power and Internet: Our laptop needs to be on all the time. If we shut it down or our internet connection drops, our website will go offline. This isn't ideal for a website that needs to be accessible constantly.

  2. Limited Resources: Laptops are not designed to handle multiple incoming connections or serve high-traffic websites. As more users access our site, our laptop could become slow or crash.

  3. Security Risks: Exposing our personal laptop to the internet opens up potential security risks. We’ll need to constantly update and secure our system to avoid vulnerabilities.

  4. Poor Reliability: Laptops are not built for the uptime and stability that a professional server offers. There's a risk of overheating, failure, or network issues affecting the accessibility of our website.

Option 2: Buying a Dedicated Server

Instead of using our laptop, we can rent or buy a dedicated server from a hosting provider. A dedicated server is a machine designed specifically to host websites and applications, providing several advantages:

  1. 24/7 Availability: The server is always on, with constant internet connectivity. We don’t have to worry about manually switching anything on or off.

  2. Scalability: Dedicated servers are built to handle traffic efficiently. As our BookMyMovie website grows and gets more visitors, the server can scale to accommodate the load.

  3. Security: Hosting providers offer strong security measures, including firewalls, regular software patches, and network monitoring, helping to protect our site from cyberattacks.

  4. Reliability and Support: Servers from hosting companies are more reliable and designed to run without interruption. Plus, hosting providers often offer customer support in case any issues arise, ensuring minimal downtime for our website.

  5. Resources and Performance: Dedicated servers offer far more processing power, memory, and bandwidth than our laptop, ensuring smooth performance for users.

P.S: We will discuss about Option 3 in the later part.

Server and Web Server:

A server is just a powerful computer that provides services or resources to other computers (called "clients") over a network. A web server is a type of server specifically designed to handle websites. Its main job is to deliver web pages, images, videos, or any files to your browser when you visit a website.

Web servers can be located:

  • In huge data centers (big buildings full of servers) owned by companies like Google, AWS, or Microsoft.

  • Or they can even sit in someone's home or office if they are hosting their own website like the image above.

What Information Web Server contains?

  • HTML, CSS, JavaScript files are stored on the web server.

  • Images, videos, and documents that are part of the website.

  • Backend code and logic (if it’s a dynamic site that changes based on input, like a login page).

  • Database connection info to fetch and store user data.

  • Web servers store frontend files (HTML, CSS, JavaScript) that are sent to your browser.

  • If the website is dynamic, the backend code (e.g. Go, Python, PHP) is also stored and executed on the server.

  • Web servers store all the files (frontend and backend) on their hard drives, similar to how you store files on your computer. These files are organized in folders, ready to be sent when requested.

Working of a Web Server:

  1. You type a website link (e.g., www.bookmymovie.com) into your browser. It s like an address for the web server.

  2. When you enter a link, it helps your browser locate the correct web server where the website's files are stored.

  3. Your browser sends a request to the web server where the website is hosted.

  4. The web server receives the request, finds the files for the website, and sends them back to your browser.

  5. Your browser displays the website.

We will now understand the whole workflow with a example of Bookmymovie web app:

You have built an amazing web app called "bookmymovie" using Go for the backend and HTML, CSS, and JavaScript for the frontend. Right now, it works on your computer at localhost:8080, but no one else can access it except you. Let’s figure out how to share it with the world!

Step 1: Understanding the Problem

  • Your app is running on your personal computer, but it’s not accessible to others because:

    1. It’s on localhost, which means only your computer knows about it.

    2. Your computer isn’t designed to handle external traffic from users worldwide.

You’ll need a web server to host your app and make it accessible to the internet.

Step 2: Choosing a Web Server

You now have two options for hosting your app:

  1. Set Up Your Computer as a Web Server:

    • Your computer can act as a web server, but it must be ON 24/7, connected to the internet, and powerful enough to handle multiple users.

    • Install server software like Apache or Nginx to manage incoming requests.

  2. Get a Public IP Address:

    • Contact your internet provider and request a static public IP address so users can find your computer on the internet.

  3. Set Cooling and Power Backup:

    • Since your computer will be running 24/7, ensure it has proper cooling (like a fan or AC in the room) and a UPS (Uninterruptible Power Supply) to avoid downtime during power cuts.

  4. Open Ports on Your Router:

    • Forward port 8080 (or another port you choose) on your router so users can connect to your computer.

Step 3: Publishing Your Code

Now that you’ve chosen a hosting method, let’s get your app on the server:

  1. Prepare Your Code:

    • Package your Go backend and frontend files.

  2. Transfer Code to the Server:

    • Use tools like SCP (secure copy) or FileZilla to upload your code to the server.

  3. Install Dependencies:

    • If your backend needs libraries, install them on the server.

    • For example, install Go on the server to run your backend code.

  4. Run Your App on the Server:

    • Start your backend service by running something like:
go run main.go
  • Use a process manager like PM2 or systemd to keep your app running even if the server restarts.

  1. Serve Your Frontend:

    • Use a web server like Nginx to serve your HTML, CSS, and JavaScript files to users.

Step 4: Make It Public

Now your app is running on the server, but you want users to access it through the internet:

  1. Point Domain to Server:

    • Configure DNS settings to link bookmymovie.com to your server’s IP address.

  2. Use HTTPS:

    • Get a free SSL certificate (using Let’s Encrypt) to secure your website.

Step 5: How Users Access Your App

Here’s what happens when a user visits bookmymovie.com:

  1. User Types the Link:

  2. Request Reaches Your Server:

    • The request travels through the internet to your web server.

  3. Web Server Responds:

    • Your web server runs your backend code (Go) to handle requests like booking a turf.

    • It sends the necessary HTML, CSS, and JavaScript files to the user’s browser.

  4. Browser Displays the App:

    • The user sees your web app, browses turfs, and makes a booking.

Role of Apache and Nginx:

Imagine you have a website, and every time someone visits it, the browser sends a request to your server asking for resources — things like a web page, images, CSS files, and JavaScript. This is where HTTP, the protocol browsers use to request resources, comes into play.

Now, if you try to manage all of these requests yourself without any help, things could get pretty messy. For example, let’s say your website becomes popular, and you start getting a lot of visitors all at once. Your server might start getting overwhelmed trying to respond to each request — it could crash, slow down, or become unresponsive. That’s because the operating system (like Linux/Ubuntu) doesn't know how to properly handle multiple HTTP requests simultaneously. It lacks the specialized tools required to manage web traffic effectively, like:

  • How to respond to multiple users at once.

  • How to send the right resources (like a webpage or an image) efficiently.

  • How to handle dynamic content (e.g., PHP or database queries) and static content (e.g., HTML, images).

Here Comes the Savior: Apache and Nginx

Apache and Nginx are like the heroes of web traffic management! They step in to handle all of these complexities and save your server from being overwhelmed.

Apache:

  • Handling HTTP Requests: Apache steps up as a web server, intercepting all incoming HTTP requests. It knows exactly how to respond to requests, whether it's for a simple webpage (static content) or a dynamic page that needs processing (like PHP scripts).

  • Serving Dynamic Content: If you have content that changes (e.g., a contact form submission or a blog page that fetches data from a database), Apache can process that dynamically. It does this through modules like mod_php to run PHP scripts, turning your server into a dynamic content generator.

  • Easy Setup for Beginners: Apache is well-known for its ease of setup and configuration. It's like a dependable sidekick who makes sure everything works well for most use cases, especially if you're running smaller sites or using a CMS like WordPress.

Nginx:

  • Efficiently Handling Static Content: On the other hand, Nginx comes in with an asynchronous, event-drivenapproach. This means that Nginx can handle a large number of requests at once without using much memory. It's like a superhero who doesn't tire easily, even when many users are knocking on the door.

  • Lightweight and Fast: Nginx is extremely fast at serving static content like images, CSS, and HTML files. Because it doesn’t need to create a new process for each request (like Apache does), it can handle thousands of requests simultaneously with minimal resources. This makes Nginx ideal for websites with high traffic and static content.

Reverse Proxy:

A reverse proxy is like a middleman between the users and your servers.

  • What it does: When a user sends a request (e.g., visiting a website), the reverse proxy receives it first and then forwards it to the appropriate server.

  • Why it's useful:

    1. Hides servers: The user doesn’t directly see your servers; instead, they interact with the reverse proxy. This adds an extra layer of security.

    2. Improves performance: It can cache (store) responses for common requests, so it doesn’t need to ask the server every time.

    3. Load balancing: It distributes incoming traffic to different servers to prevent overloading (explained more below).

Example: Imagine a receptionist at an office. When a visitor arrives, the receptionist asks what they need and directs them to the correct department. The visitor doesn’t deal with the departments directly—they only interact with the receptionist.

Load Balancing:

Load balancing is like managing traffic to ensure no single server gets overwhelmed by too many requests.

  • What it does: It distributes user requests across multiple servers.

  • Why it's useful:

    1. Prevents overload: If one server gets too many requests, it might crash. Load balancing ensures traffic is shared evenly.

    2. Improves reliability: If one server fails, others can take over.

    3. Faster responses: By spreading the load, servers can respond to requests more quickly.

Example: Think of a restaurant with multiple counters. If all customers go to one counter, there will be long delays. A manager (load balancer) directs customers to the least busy counter, ensuring smooth service.

How Reverse Proxy and Load Balancing Work Together

A reverse proxy can act as a load balancer. It receives requests from users, decides which server is best suited to handle the request (load balancing), and forwards the request to that server.

  • Benefits of using both:

    1. Better performance.

    2. Higher reliability.

    3. Enhanced security.

Real-life Example: Popular services like Netflix, Amazon, or Google use reverse proxies and load balancing to handle millions of requests efficiently without letting their servers crash.

WebSockets:

WebSockets is a communication protocol that enables real-time, two-way communication between a client (like a web browser) and a server.

How It Works:

  1. The client establishes a WebSocket connection with the server over a single TCP (Transmission Control Protocol) connection.

  2. Once the connection is open, both the client and server can send and receive messages at any time without repeatedly requesting or waiting for a response.

Key Features:

  1. Full-Duplex Communication: Both the client and server can exchange data simultaneously in both directions.

  2. Persistent Connection: After the connection is established, it remains open until explicitly closed by either side.

  3. Efficient: It eliminates the need for repeated HTTP requests (as in traditional polling) and reduces latency and overhead.

  4. Use Cases:

    • Real-time chat applications (e.g., WhatsApp Web, Slack).

    • Online multiplayer games.

    • Live financial data updates, such as stock prices.

    • Real-time notifications.

Example:

  • In a chat app, as soon as User A sends a message, it’s instantly delivered to User B without either user refreshing the page.

HTTPS:

HTTPS (Hypertext Transfer Protocol Secure) is a secure version of HTTP that ensures safe communication over the internet.

How It Works:

  1. HTTPS uses encryption protocols (like TLS—Transport Layer Security) to encrypt the data being transmitted between the client (browser) and the server.

  2. When a user connects to a website using HTTPS, the server presents a digital certificate to prove its identity.

  3. Once the connection is secure, all data (like passwords or credit card information) is encrypted and transmitted securely.

Key Features:

  1. Data Encryption: Prevents attackers from intercepting and reading the data exchanged between a user and the server.

  2. Authentication: Ensures the website the user is interacting with is legitimate through the use of SSL/TLS certificates.

  3. Data Integrity: Protects the data from being altered during transmission.

Benefits:

  • Protects sensitive information like passwords, credit card numbers, and personal details.

  • Helps websites gain trust (indicated by a lock icon in the browser address bar).

  • Improves SEO (search engine optimization) rankings as search engines favor HTTPS websites.

Example:

  • When you access a banking website, the connection is secured with HTTPS to ensure that your login credentials and account details are encrypted and safe from attackers.

Caching :

Caching is the process of storing copies of frequently accessed data in a temporary storage (cache) so it can be retrieved quickly without fetching it repeatedly from the original source.

Example of Caching

Scenario: Website Loading

  1. Without Caching:

    • You visit a website for the first time.

    • The browser fetches all the resources (HTML, CSS, images, etc.) directly from the server.

    • This process takes time because each file must be downloaded from the server.

  2. With Caching:

    • The browser saves a copy of the resources (like images and CSS files) locally in its cache.

    • On your next visit to the website:

      • Instead of downloading the resources again, the browser loads them from the cache.

      • This makes the website load much faster.

Example:

  • First Visit:

    • You visit example.com.

    • The website loads, and the browser downloads a logo (logo.png) from the server.

  • Second Visit:

    • You visit example.com again.

    • The browser fetches logo.png from its cache instead of downloading it from the server.

Example in Real-Life Applications

Online Shopping Website

  1. Scenario: A customer searches for "laptops" on an e-commerce website.

  2. Without Caching:

    • Every time a user searches for "laptops," the server queries the database to fetch product details.

    • This process takes time and increases the load on the database.

  3. With Caching:

    • The server caches the search results for "laptops."

    • When another user searches for "laptops," the cached result is served instead of querying the database.

    • This reduces the database load and makes the search results appear faster.

Types of Caching in the Example

  1. Browser Caching:

    • The user's browser caches static assets like images, CSS files, and JavaScript files for the shopping website.

    • If the user revisits the site, these assets are loaded from the cache instead of being downloaded again.

  2. Server-Side Caching:

    • The server caches the product details for laptops in its memory (e.g., using a tool like Redis).

    • When the same search query comes in, the server serves the cached results.

  3. Content Delivery Network (CDN) Caching:

    • Images of laptops are cached on CDN servers worldwide.

    • Users download images from the nearest CDN server instead of the main website server, making the process faster.

Logging:

Logging involves capturing and storing data about system events, errors, or actions in a structured format, typically in log files. These logs provide insights into how an application or system is functioning.

Purpose of Logging:

  • Troubleshooting: Helps diagnose issues when something goes wrong.

  • Debugging: Assists developers in understanding the flow of an application and identifying bugs.

  • Auditing: Provides a record of activities, often for compliance or security purposes.

  • Analysis: Offers insights into system usage or user behavior.

Components of a Log Entry:

A typical log entry might include:

  • Timestamp: When the event occurred.

  • Severity Level: Indicates the importance of the event (e.g., INFO, WARN, ERROR, FATAL).

  • Message: A descriptive message about the event.

  • Source: The component or service that generated the log.

Example of Logging:

  • Web Application:

    • When a user logs in, the system logs:
2025-01-01 10:00:00 [INFO] User 'john_doe' logged in successfully from IP 192.168.1.1.

  • If an error occurs during login:

  •   2025-01-01 10:01:00 [ERROR] Failed login attempt for user 'john_doe' from IP 192.168.1.2: Invalid password.

Monitoring:

Monitoring involves continuously tracking the performance and health of systems, applications, and infrastructure. Monitoring tools collect metrics, set thresholds, and generate alerts for anomalies or failures.

Purpose of Monitoring:

  • Performance Tracking: Ensures systems are running efficiently.

  • Proactive Issue Detection: Alerts you before small issues become critical problems.

  • Capacity Planning: Helps predict future resource needs by analyzing trends.

  • User Experience: Monitors key aspects like website uptime and response times.

What is Monitored?

  • System Metrics:

    • CPU usage, memory usage, disk space.

  • Application Metrics:

    • Response times, error rates, database query performance.

  • Network Metrics:

    • Bandwidth, latency, packet loss.

  • Business Metrics:

    • Sales transactions, active users, or order processing rates.

Example of Monitoring:

  • Web Application Monitoring:

    • The system tracks the average response time for HTTP requests.

    • If response time exceeds 2 seconds, it sends an alert:

Alert: High response time detected on 'checkout' API. Current avg: 2.5s

Do you remember Option 3 we talked about earlier? Option 3 removes the need to buy a personal server and instead suggests renting one.

Option 3: Use a Cloud-Based Web Server

This is much easier and more reliable:

  1. Find a Hosting Provider:

    • Choose a service like AWS, DigitalOcean, Google Cloud, or Heroku.

  2. Set Up a Virtual Machine:

    • Create a virtual computer (called a VM) in the cloud. It will act as your web server.

    • Choose a Linux-based server (like Ubuntu) for simplicity.

  3. Domain Name:

    • Buy a domain name like bookmymovie.com from providers like GoDaddy or Namecheap.

    • Connect your domain to your cloud server using a DNS service.

Why Cloud Hosting is Better

  • No need to worry about cooling, power, or hardware failures.

  • Reliable and scalable (easily handle more users).

  • Easier setup and management.

Thank you so much for reading my blog! But that's not all—there are more exciting blogs on the way as I continue exploring new technologies. Feel free to connect with me on LinkedIn or Twitter, and don't forget to subscribe to my newsletter to get my latest posts straight to your inbox.

Wishing you all a very Happy New Year ✨!

LinkedIn: https://www.linkedin.com/in/nsahil992

Twitter/X: https://twitter.com/nsahil992

web serversapachenginxDevopsWeb DevelopmentDeveloper