Difference between pages "Web-server-stack" and "Package:Varnish"

From Funtoo
(Difference between pages)
Jump to navigation Jump to search
m
 
m
 
Line 1: Line 1:
{{Ebuild
|Summary=Varnish is a state-of-the-art, high-performance HTTP accelerator
|CatPkg=www-servers/varnish
|Maintainer=
}}
'''Varnish''' is a webcache & http accelerator.  Varnish will either serve cached content, or retireve content from the server, and cache it.  Varnish will reduce I/O pressure from webservers.


== Pre-install considerations ==
==Install==
=== ssl ===
===Emerge===
Ssl [http://en.wikipedia.org/wiki/Wildcard_certificate wild card certificates] can use the same certificate to cover several subdomain names.  As in https://wiki.funtoo.org https://www.funtoo.org https://forums.funtoo.org can all use the same certificate.  https://funtoo.org would not be covered under the wildcard, so [[User:Threesixes|Threesixes]] ([[User talk:Threesixes|talk]]) suggests using http://domain.tld as a http navigation splash page directory.  All that is required to setup a ca signed ssl certificate is an email on the server.  https://www.startssl.com offers free ssl ca certificates, though there are several other certificate [http://en.wikipedia.org/wiki/Certificate_authority#Providers providers].  Many web apps require you set your URL & will have problems if your URL is set to http://, rather than https://


=== sockets vs tcp stack ===
Install {{Package|www-servers/varnish}}:
Sockets have less overhead but can not be shared across jails, or to other machines.  Tcp stack has more overhead but is far more flexible.
<console>###i## emerge www-servers/varnish</console>


=== Email Servers ===
==Configuration==
* {{Package|mail-mta/postfix}} <-- default in Funtoo
* {{Package|mail-mta/ssmtp}}
* {{Package|mail-mta/exim}}
* {{Package|mail-mta/sendmail}}
* {{Package|mail-mta/nullmailer}}


=== FTP Servers ===
{{note|as your varnish is local to your server, your server needs to be aware that it is behind a proxy, and configured for [http://en.wikipedia.org/wiki/X-Forwarded-For x-forwarded-for] or similar to fetch real users ip addresses instead of displaying 127.0.0.1 or localhost.}}
It is common practice to use FTP servers to host files for downloading.


* {{Package|net-ftp/vsftpd}}
Configuration is controlled by /etc/varnish/default.vcl & /etc/conf.d/varnishd
* {{Package|net-ftp/proftpd}}
* {{Package|net-ftp/pure-ftpd}}
* {{Package|net-ftp/qshare}}


== Webserver ==
{{file|name=/etc/varnish/default.vcl|desc=varnish configuration file|body=
Web servers come in several varieties. The most common stack is known as LAMP which stands for linux apache mysql php. [[User:Threesixes|Threesixes]] ([[User talk:Threesixes|talk]]) suggests setting up the web server stack by selecting the database first, then scripting language second, and web server 3rd.
vcl 4.0;
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}
}}


=== Databases ===
{{file|name=/etc/conf.d/varnishd|desc=varnish configuration file|body=
* {{Package|dev-db/mysql}}
VARNISHD="/usr/sbin/varnishd"
mariadb is a drop in replacement for mysql <-- default in Funtoo
VARNISHADM="/usr/bin/varnishadm"
* {{Package|dev-db/mariadb}}
CONFIGFILE="/etc/varnish/default.vcl"
percona is a drop in replacement for mysql
VARNISHD_OPTS="-a 127.0.0.1:80"
* {{Package|dev-db/percona-server}}
VARNISHD_OPTS="${VARNISHD_OPTS} -u varnish -g varnish"
* {{Package|dev-db/postgresql-server}}
}}
* {{Package|dev-db/sqlite}}


=== Languages ===
Varnish will fetch data from localhost:8080 and serve accelerated proxy data on localhost:80
* {{Package|dev-lang/php}}
* {{Package|dev-lang/perl}}
* {{Package|dev-lang/python}}


=== Web Servers ===
=== c10k ===
* {{Package|www-servers/apache}}
For 10,000 concurrent connections (or not) a few configuration settings control the power of varnish.  If you're having varnish directly serve to the outside world, dial back concurrency to say 50 or 100 connections per ip. You must take into account corporations, and universities hammering several connections from a singular ip.  c10k is useful information for if varnish is behind a load balancer such as pound, nginx, or tengine, and all requests are internal and local.
* {{Package|www-servers/cherokee}}
* {{Package|www-servers/nginx}}
* {{Package|www-servers/tengine}}
* {{Package|www-servers/lighttpd}}


{{file|name=/etc/conf.d/varnishd|desc=varnish concurrency settings|body=
VARNISHD_OPTS="-a 127.0.0.1:80 -p thread_pool_min=20 -p thread_pool_max=1000 -p thread_pool_add_delay=2 -s malloc,700M"
}}


=== SSL Termination, Reverse Proxies, & load balancing ===
{{file|name=/etc/varnish/default.vcl|desc=varnish concurrency settings|body=
Reverse proxies are useful, some cache static data, and shuck out cached pages rather than hitting the web server. Some pass requests to backend nodes high availability clustering your website, some web servers have this functionality built in.
backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 10000;
    # .port = "80" led to issues with competing for the port with apache.
}
}}


* {{Package|www-servers/nginx}}
== BootService ==
* {{package|net-misc/stunnel}}
 
* {{package|www-servers/pound}}
To start varnish immediately:
* {{Package|www-servers/varnish}}
<console>###i## rc-service varnishd start</console>
* {{Package|net-proxy/squid}}
 
To start varnish at boot:
<console>###i## rc-update add varnishd default</console>
 
== Verification ==
To verify that your traffic is going through varnish, and see if you're getting hits or misses:
<console>$##i## curl -I http://www.funtoo.org/Welcome</console>
 
== Benchmarking ==
{{package|app-admin/apache-tools}} apache benchmark can show the power of varnish.  The examples shown are running 500 requests with concurrency of 100 hits.
 
 
ab against a 3 worker cluster mode puma server
<console>###i## ab -n 500 -c 100 http://127.0.0.1:3000/index.html | grep Request</console>
Requests per second:    110.92 [#/sec] (mean)


== Post install ==
There are several considerations to take into account with a web server install, such as setting up an email server, setting up a firewall, firewalling web applications, and dynamically firewalling attackers.


=== Firewalls ===
ab against the same server served through varnish
* {{Package|net-firewall/nftables}}
<console>###i## ab -n 500 -c 100 http://127.0.0.1/index.html | grep Request</console>
* {{Package|net-firewall/iptables}}
Requests per second:    10268.42 [#/sec] (mean)
* {{Package|net-firewall/firewalld}}
* {{Package|net-firewall/ufw}}


=== Dynamic Firewalling ===
== SSL support ==
* {{Package|app-admin/sshguard}}
Varnish does not support ssl.  There are packages to get around this limitation:
* {{Package|net-analyzer/fail2ban}}
* {{package|net-misc/stunnel}}
* {{package|www-servers/pound}}


=== Benchmarking ===
== Media ==
It's a good idea to benchmark your system, server, & websites.  There are several tools to assist you in doing this.
{{#widget:YouTube16x9|id=JEF6_XC-2ZU}}


* http://yslow.org/
{{EbuildFooter}}
* http://getfirebug.com/
* {{Package|app-admin/apache-tools}}
* {{Package|app-benchmarks/sysbench}}
* {{Package|app-benchmarks/phoronix-test-suite}}
* {{Package|app-benchmarks/iozone}}
* {{Package|app-benchmarks/piozone}}
* {{Package|app-benchmarks/siege}}
* {{Package|app-benchmarks/ramspeed}}
* {{Package|app-benchmarks/jmeter}}

Revision as of 08:15, December 1, 2014

Varnish

   Tip

We welcome improvements to this page. To edit this page, Create a Funtoo account. Then log in and then click here to edit this page. See our editing guidelines to becoming a wiki-editing pro.

Varnish is a webcache & http accelerator. Varnish will either serve cached content, or retireve content from the server, and cache it. Varnish will reduce I/O pressure from webservers.

Install

Emerge

Install www-servers/varnish: 

root # emerge www-servers/varnish

Configuration

   Note

as your varnish is local to your server, your server needs to be aware that it is behind a proxy, and configured for x-forwarded-for or similar to fetch real users ip addresses instead of displaying 127.0.0.1 or localhost.

Configuration is controlled by /etc/varnish/default.vcl & /etc/conf.d/varnishd

   /etc/varnish/default.vcl - varnish configuration file
vcl 4.0;
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}
   /etc/conf.d/varnishd - varnish configuration file
VARNISHD="/usr/sbin/varnishd"
VARNISHADM="/usr/bin/varnishadm"
CONFIGFILE="/etc/varnish/default.vcl"
VARNISHD_OPTS="-a 127.0.0.1:80"
VARNISHD_OPTS="${VARNISHD_OPTS} -u varnish -g varnish"

Varnish will fetch data from localhost:8080 and serve accelerated proxy data on localhost:80

c10k

For 10,000 concurrent connections (or not) a few configuration settings control the power of varnish. If you're having varnish directly serve to the outside world, dial back concurrency to say 50 or 100 connections per ip. You must take into account corporations, and universities hammering several connections from a singular ip. c10k is useful information for if varnish is behind a load balancer such as pound, nginx, or tengine, and all requests are internal and local.

   /etc/conf.d/varnishd - varnish concurrency settings
VARNISHD_OPTS="-a 127.0.0.1:80 -p thread_pool_min=20 -p thread_pool_max=1000 -p thread_pool_add_delay=2 -s malloc,700M"
   /etc/varnish/default.vcl - varnish concurrency settings
backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 10000;
    # .port = "80" led to issues with competing for the port with apache.
}

BootService

To start varnish immediately: 

root # rc-service varnishd start

To start varnish at boot: 

root # rc-update add varnishd default

Verification

To verify that your traffic is going through varnish, and see if you're getting hits or misses: 

user $ curl -I http://www.funtoo.org/Welcome

Benchmarking

app-admin/apache-tools apache benchmark can show the power of varnish. The examples shown are running 500 requests with concurrency of 100 hits.


ab against a 3 worker cluster mode puma server 

root # ab -n 500 -c 100 http://127.0.0.1:3000/index.html | grep Request

Requests per second: 110.92 [#/sec] (mean)


ab against the same server served through varnish 

root # ab -n 500 -c 100 http://127.0.0.1/index.html | grep Request

Requests per second: 10268.42 [#/sec] (mean)

SSL support

Varnish does not support ssl. There are packages to get around this limitation:

  • No results
  • No results

Media


Retrieved from "https://www.funtoo.org/index.php?title=Web-server-stack&oldid=7243"