Self hosting services behind Cloudflare tunnels

Diagram from Cloudflare Docs

What is self hosting and why should we do it?

Self hosting is the process of hosting your own services, instead of using a third party. For example, instead of using Google Drive, you can use Nextcloud to host your files. Or instead of using Netflix, you can use Plex to host your media.

It is a great way to reduce your reliance on third parties and to take control of your data. It also helps to decentralize the internet making it more resilient.

Prerequisites

• Basic knowledge of Docker and Docker Compose.
• Docker environment with Portainer installed (see this guide).

Abstract overview

We’ll be setting up a Grafana dashboard (open source analytics & monitoring solution) as an example, using the grafana/grafana Docker image. We’ll be using Cloudflare Tunnels to expose the Grafana dashboard to the internet, without exposing any ports.

Cloudflare Tunnel is tunneling software that lets you quickly secure and encrypt application traffic to any type of infrastructure, so you can hide your web server IP addresses, block direct attacks, and get back to delivering great applications.

Lastly, for data sources we’ll be using Prometheus and cAdvisor images respectively.

Configuring Portainer Network

We need to create a separate network for Grafana, Prometheus and cAdvisor. To do this, we’ll be using Portainer (container management tool).

Head over to Portainer, and click on the “Networks” tab. Then click on the “Add network” button.

Name the network web, and leave the rest of the settings as default. Click on the “Create the network” button.

Once the network is created, we can start configuring Grafana.

Configuring Grafana

In the sidebar, click on the “Stacks” tab. Then click on the “Add stack” button.

Name it monitoring, and set the stack file to the following:

monitoring.yml

version: '3.8'
 
networks:
  # This is the network we created earlier
  web:
    external: true
 
volumes:
  # These are the volumes we'll be using for storing data
  prometheus_data: {}
  grafana_data: {}
 
services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    # These are the ports we'll be using for Prometheus
    ports:
      - 9090:9090/tcp
    networks:
      - web
    volumes:
      - /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    restart: unless-stopped
    command:
      # This is the configuration file for Prometheus
      # We'll be editing this file later for setting up cAdvisor
      - '--config.file=/etc/prometheus/prometheus.yml'
 
  cadvisor:
    image: zcube/cadvisor:latest
    container_name: cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    # These are the ports we'll be using for cAdvisor
    ports:
      - 8080:8080/tcp
    networks:
      - web
    privileged: true
    restart: unless-stopped
 
  grafana:
    image: grafana/grafana
    container_name: grafana
    volumes:
      - grafana_data:/var/lib/grafana
    # These are the ports we'll be using for Grafana
    ports:
      - 3000:3000/tcp
    networks:
      - web
    restart: unless-stopped

Once this is done, click on the “Deploy the stack” button.

All the three services should be up and running now. You can check this by clicking on the “Containers” tab.

Configuring Prometheus

Now we need to configure Prometheus to scrape data from cAdvisor. To do this, we’ll be editing the prometheus.yml file.

To do this, we’ll need to access our server or machine using SSH. Once we’ve logged in, run the following command:

# We need to navigate to the prometheus directory
cd /etc/prometheus/
# Then we need to edit the prometheus.yml file
nano prometheus.yml

Now copy and paste the following configuration into the file:

prometheus.yml

global:
  scrape_interval: 15s # By default, scrape targets every 15 seconds.
 
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']
 
  # Job for cadvisor
  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8080']

This will tell Prometheus to scrape data from cAdvisor. Once you’re done, press Ctrl + X to exit the editor. Then press Y to save the changes.

Once all the changes are saved, go back to Portainer. Click on the “Stacks” tab, and then click on the “monitoring” stack.

Select the editor tab on the right, click on the “Update the stack” button (enable Re-pull image and redeploy) and finally press the “Update” button .

This should restart all the monitoring services, which should update the Prometheus configuration.

Configuring Cloudflare Tunnel

Now, we’ll need to create a Cloudflare zero trust account (to authenticate with tunnels). To do this, go to Cloudflare Zero Trust.

Once you’ve created your account, you’ll be able to create a tunnel. To do this, click on the “Create a tunnel” button.

Once you’ve created your tunnel, you’ll be navigated to the connector installation page. Here, you’ll be able to download the connector for your operating system. For this guide, we’ll be using the Docker connector.

Once the connector is installed, we can specify the grafana service (we have set the port to 3000) to expose.

After the tunnel is created, you’ll be able to see the container inside portainer in the “Stacks” tab.

Accessing Grafana

To access Grafana, you’ll need to navigate to https://<tunnel_name>.yourdomain.com/. For example, my tunnel name is grafana, we’ll need to navigate to https://grafana.yashgarg.dev/.

You should be able to see the Grafana login page. The default username and password are both admin.

Setting Prometheus as Data Source

Once you’ve logged in, we need to setup Prometheus as a data source. To do this, click on the “Configuration” tab on the left, and then click on the “Data Sources” tab.

Set the name to Prometheus, and set the URL to http://prometheus:9090. Then click on the “Save & Test” button.

Now that we’ve set up Prometheus as a data source, we can add the cadvisor dashboard.

Adding cAdvisor Dashboard

To add the cAdvisor dashboard, click on the “Create” button on the left. Then click on the “Import” button.

Enter the dashboard ID as 14282, and click on the “Load” button. You should now see the cAdvisor dashboard.

It will start to populate the data after a few minutes, and should look like this after a few hours.

Conclusion

We have successfully setup a monitoring stack using Prometheus, cAdvisor, and Grafana. We’ve exposed the Grafana dashboard using Cloudflare Tunnels without exposing any ports externally.

Similarly, you can also use Cloudflare Tunnels to expose other services like qBittorrent, Plex, etc.

For reference, some of my commonly used docker-compose files can be found here.