Difference between pages "Traffic Control" and "Package:Nginx"

(Difference between pages)
(Other Links of Interest)
 
m (add media playlist)
 
Line 1: Line 1:
== Introduction ==
+
{{Ebuild
 +
|Summary=Robust, small and high performance HTTP and reverse proxy server
 +
|CatPkg=www-servers/nginx
 +
|Maintainer=Drobbins
 +
|Repository=Funtoo Overlay
 +
|Overlay=Funtoo
 +
}}
 +
[[Image:nginx.gif|frame]]
  
Linux's traffic control functionality offers a lot of capabilities related to influencing the rate of flow, as well as latency, of primarily outgoing but also in some cases incoming network traffic. It is designed to be a "construction kit" rather than a turn-key system, where complex network traffic policing and shaping decisions can be made using a variety of algorithms. The Linux traffic control code is also often used by academia for research purposes, where is it can be a useful mechanism to simulate and explore the impact of a variety of different network behaviors. See [http://www.linuxfoundation.org/collaborate/workgroups/networking/netem netem] for an example of a simulation framework that can be used for this purpose.  
+
nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in [http://funtoo.org funtoo]), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.  
  
Of course, Linux traffic control can also be extremely useful in an IT context, and this document is intended to focus on the practical, useful applications of Linux traffic control, where these capabilities can be applied to solve problems that are often experienced on modern networks.
+
=== USE Expanded flags ===
  
== Incoming and Outgoing Traffic ==
+
Furthermore, you can set the nginx modules you like to use in ''/etc/make.conf'' in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".
  
One common use of Linux traffic control is to configure a Linux system as a Linux router or bridge, so that the Linux system sits between two networks, or between the "inside" of the network and the real router, so that it can shape traffic going to local machines as well as out to the Internet. This provides a way to prioritize, shape and police both incoming (from the Internet) and outgoing (from local machines) network traffic, because it is easiest to create traffic control rules for traffic flowing ''out'' of an interface, since we can control when the system ''sends'' data, but controlling when we ''receive'' data requires an additional ''intermediate queue'' to be created to buffer incoming data. When a Linux system is configured as a firewall or router with a physical interface for each part of the network, we can avoid using intermediate queues.
+
nginx USE flags go into ''/etc/portage/package.use'' or ''/etc/portage/package.use/nginx'', while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.
  
A simple way to set up a layer 2 bridge using Linux involves creating a bridge device with <tt>brctl</tt>, adding two Ethernet ports to this bridge (again using <tt>brctl</tt>), and then apply prioritization, shaping and policing rules to both interfaces. The rules will apply to ''outgoing'' traffic on each interface. One physical interface will be connected to an upstream router on the same network, while the other network port will be connected to a layer 2 access switch to which local machines are connected. This allows powerful egress shaping policies to be created on both interfaces, to control the flows in and out of the network.
+
Example:
 +
<console>
 +
###i## echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx
 +
</console>
  
== Recommended Resources ==
+
=== Emerging nginx ===
  
Resources you should take a look at, in order:
+
Now you are ready to install nginx with php and xcache support:
 +
<console>
 +
###i## emerge -avt nginx php xcache
 +
</console>
 +
so now just check your useflags and press enter to start emerge.
  
* [http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm HTB documentation] by Martin Devera. Best way to create different priority classes and bandwidth allocations.
+
== Configuring ==
* [http://www.opalsoft.net/qos/DS.htm Differentiated Services On Linux HOWTO] by Leonardo Balliache. Good general docs.
+
* [http://blog.edseek.com/~jasonb/articles/traffic_shaping/index.html A Practical Guide to Linux Traffic Control] by Jason Boxman. Good general docs.
+
* [http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb IFB - replacement for Linux IMQ], with examples. This is the official best way to do ''inbound'' traffic control, when you don't have dedicated in/out interfaces.
+
* [http://seclists.org/fulldisclosure/2006/Feb/702 Use of iptables hashlimit] - Great functionality in iptables. There's a hashlimit example below as well.
+
  
Related Interesting Links:
+
All configuration is done in ''/etc/nginx'' with ''nginx.conf'' as the main configuration file and all virtual hosts in ''/etc/nginx/sites/available'' while you have to symlink ''/etc/nginx/sites-available/{VHOST}'' to ''/etc/nginx/sites-enabled/{VHOST}'' to activate them. An example config for such a {VHOST} looks like that:
  
* [http://wiki.secondlife.com/wiki/BLT Second Life Bandwidth Testing Protocol] - example of Netem
+
<pre>
* [http://www.29west.com/docs/THPM/udp-buffer-sizing.html UDP Buffer Sizing], part of [http://www.29west.com/docs/THPM/index.html Topics in High Performance Messaging]
+
server {
 +
    listen          80;
 +
    server_name    www.example.com;
  
== Recommended Approaches ==
+
    access_log      /var/log/nginx/www.example.com.access_log main;
 +
    error_log      /var/log/nginx/www.example.com.error_log info;
  
Daniel Robbins has had very good results with the [http://luxik.cdi.cz/~devik/qos/htb/ HTB queuing discipline] - it has very good features, and also has [http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm very good documentation], which is just as important, and is designed to deliver useful results in a production environment. And it works. If you use traffic control under Funtoo Linux, please use the HTB queuing discipline as the root queuing discipline because you will get good results in very little time. Avoid using any other queuing discipline under Funtoo Linux as the ''root'' queuing discipline on any interface. If you are creating a tree of classes and qdiscs, HTB should be at the top, and you should avoid hanging classes under any other qdisc unless you have plenty of time to experiment and verify that your QoS rules are working as expected. Please see [[#State_of_the_Code|State of the Code]] for more info on what Daniel Robbins considers to be the current state of the traffic control implementation in Linux.
+
    root /var/www/www.example.com/htdocs;
 +
}
 +
</pre>
  
== State of the Code ==
+
The ''nginx.conf'' and ''sites-available/localhost'' file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.
  
If you are using enterprise kernels, especially any RHEL5-based kernels, you must be aware that the traffic control code in these kernels is about 5 years old and contains many significant bugs. In general, it is possible to avoid these bugs by using HTB as your root queueing discipline and testing things carefully to ensure that you are getting the proper behavior. The <tt>prio</tt> queueing discipline is known to not work reliably in RHEL5 kernels. See [[Broken Traffic Control]] for more information on known bugs with older kernels.
+
=== php-fpm ===
  
If you are using a more modern kernel, Linux traffic control should be fairly robust. The examples below should work with RHEL5 as well as newer kernels.
+
nginx does not natively support php, so we delegate that responsibility to [[Package:Php#Fpm | php-fpm]]
  
== Inspect Your Rules ==
+
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm configuration|body=
 +
server {
 +
        ...
 +
index index.php index.cgi index.htm index.html;
 +
location ~ .php$ {
 +
        fastcgi_pass 127.0.0.1:9000;
 +
include fastcgi.conf;
 +
        }
 +
        ...
 +
}
 +
}}
  
If you are implementing Linux traffic control, you should be running these commands frequently to monitor the behavior of your queuing discipline. Replace <tt>$wanif</tt> with the actual network interface name.
+
==== php caching ====
 +
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm cache configuration|body=
 +
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
 +
fastcgi_cache_key "$scheme$request_method$host$request_uri";
 +
server {
 +
...
 +
        location ~ \.php$ {
 +
...
 +
fastcgi_cache MYAPP;
 +
fastcgi_cache_valid 200 60m;
 +
...
 +
}}
 +
[https://www.digitalocean.com/community/tutorials/how-to-setup-fastcgi-caching-with-nginx-on-your-vps for more information on php caching]
  
<source lang="bash">
+
=== proxy_pass===
tc -s qdisc ls dev $wanif
+
This configuration proxies to other webservers.  In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails
tc -s class ls dev $wanif
+
</source>
+
  
== Matching ==
+
{{file|name=/etc/nginx/sites-available/localhost|desc=rails or python configurations|body=
 +
server {
 +
        ...
 +
location /rails/ {
 +
    proxy_set_header Host $host;
 +
    proxy_set_header X-Real-IP $remote_addr;
 +
    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
 +
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
 +
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
 +
}
 +
        ...
 +
}
 +
}}
  
Here are some examples you can use as the basis for your own filters/classifiers:
+
== Location Processing Order ==
 +
One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.
  
# <tt>protocol arp u32 match u32 0 0</tt> - match ARP packets
+
=== Two basic types of Location directives ===
# <tt>protocol ip u32 match ip protocol 0x11 0xff</tt> - match UDP packets
+
There are two basic types of location directives. The first is called a "conventional string", and looks something like this:
# <tt>protocol ip u32 match ip protocol 17 0xff</tt> - (also) match UDP packets
+
location /foo { deny all; }
# <tt>protocol ip u32 match ip protocol 0x6 0xff</tt> - match TCP packets
+
The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:
# <tt>protocol ip u32 match ip protocol 1 0xff</tt> - match ICMP (ping) packets
+
location ~ \.php$ { blah; }
# <tt>protocol ip u32 match ip dst 4.3.2.1/32</tt> - match all IP traffic headed for IP 4.3.2.1
+
# <tt>protocol ip u32 match ip src 4.3.2.1/32 match ip sport 80 0xffff</tt> - match all IP traffic from 4.3.2.1 port 80
+
# <tt>protocol ip u32 match ip sport 53 0xffff</tt> - match originating DNS (both TCP and UDP)
+
# <tt>protocol ip u32 match ip dport 53 0xffff</tt> - match response DNS (both TCP and UDP)
+
# <tt>protocol ip u32 match ip protocol 6 0xff match u8 0x10 0xff at nexthdr+13</tt> - match packets with ACK bit set
+
# <tt>protocol ip u32 match ip protocol 6 0xff match u8 0x10 0xff at nexthdr+13 match u16 0x0000 0xffc0 at 2</tt> - packets less than 64 bytes in size with ACK bit set
+
# <tt>protocol ip u32 match ip tos 0x10 0xff</tt> - match IP packets with "type of service" set to "Minimize delay"/"Interactive"
+
# <tt>protocol ip u32 match ip tos 0x08 0xff</tt> - match IP packets with "type of service" set to "Maximize throughput"/"Bulk" (see "QDISC PARAMETERS" in <tt>tc-prio</tt> man page)
+
# <tt>protocol ip u32 match tcp dport 53 0xffff match ip protocol 0x6 0xff</tt> - match TCP packets heading for dest. port 53 (my not work)
+
  
== Sample Traffic Control Code ==
+
=== The basic algorithm ===
 +
Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most ''specific'' match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.
  
<source lang="bash">
+
This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.
modemif=eth4
+
  
iptables -t mangle -A POSTROUTING -o $modemif -p tcp -m tos --tos Minimize-Delay -j CLASSIFY --set-class 1:10
+
== Advanced Location Processing ==
iptables -t mangle -A POSTROUTING -o $modemif -p tcp --dport 53 -j CLASSIFY --set-class 1:10
+
Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.
iptables -t mangle -A POSTROUTING -o $modemif -p tcp --dport 80 -j CLASSIFY --set-class 1:10
+
iptables -t mangle -A POSTROUTING -o $modemif -p tcp --dport 443 -j CLASSIFY --set-class 1:10
+
  
tc qdisc add dev $modemif root handle 1: htb default 12
+
=== = (equals) Location ===
tc class add dev $modemif parent 1: classid 1:1 htb rate 1500kbit ceil 1500kbit burst 10k
+
One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path ''exactly'' and ''completely''. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:
tc class add dev $modemif parent 1:1 classid 1:10 htb rate 700kbit ceil 1500kbit prio 1 burst 10k
+
location = /foo/bar { deny all; }
tc class add dev $modemif parent 1:1 classid 1:12 htb rate 800kbit ceil 800kbit prio 2
+
tc filter add dev $modemif protocol ip parent 1:0 prio 1 u32 match ip protocol 0x11 0xff flowid 1:10
+
tc qdisc add dev $modemif parent 1:10 handle 20: sfq perturb 10
+
tc qdisc add dev $modemif parent 1:12 handle 30: sfq perturb 10
+
</source>
+
  
The code above is a working traffic control script that is even compatible with RHEL5 kernels, for a 1500kbps outbound link (T1, Cable or similar.) In this example, <tt>eth4</tt> is part of a bridge. The code above should work regardless of whether <tt>eth4</tt> is in a bridge or not -- just make sure that <tt>modemif</tt> is set to the interface on which traffic is flowing ''out'' and you wish to apply traffic control.
+
=== ~* (case-insensitive regex) Location ===
 +
A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.
  
=== <tt>tc</tt> code walkthrough ===
+
=== ^~ (short-circuit conventional string) Location ===
 +
You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the ''most specific'' match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans ''all'' conventional string matches to find the ''best'' match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the ''best'' match.
  
This script uses the <tt>tc</tt> command to create two priority classes - 1:10 and 1:12. By default, all traffic goes into the low-priority class, 1:12. 1:10 has priority over 1:12 (<tt>prio 1</tt> vs. <tt>prio 2</tt>,) so if there is any traffic in 1:10 ready to be sent, it will be sent ahead of 1:12. 1:10 has a rate of 700kbit but can use up to the full outbound bandwidth of 1500kbit by borrowing from 1:12.
+
== Ebuild Update Protocol ==
  
UDP traffic (traffic that matches <tt>ip protocol 0x11 0xff</tt>) will be put in the high priority class 1:10. This can be good for things like FPS games, to ensure that latency is low and not drowned out by lower-priority traffic.
+
To work on a new version of the ebuild, perform the following steps.
  
If we stopped here, however, we would get a bit worse results than if we didn't use <tt>tc</tt> at all. We have basically created two outgoing sub-channels of different priorities. The higher priority class ''can'' drown out the lower-priority class, and this is intentional so it isn't the issue -- in this case we ''want'' that functionality. The problem is that the high priority and low priority classes can both be dominated by high-bandwidth flows, causing other traffic flows of the same priority to be drowned out. To fix this, two <tt>sfq</tt> queuing disciplines are added to the high and low priority classes and will ensure that individual traffic flows are identified and each given a fair shot at sending data out of their respective classes. This should prevent starvation within the classes themselves.
+
First, temporarily set the following settings in <tt>/etc/make.conf</tt>:
  
=== <tt>iptables</tt> code walkthrough ===
+
<syntaxhighlight lang="bash">
 +
NGINX_MODULES_HTTP="*"
 +
NGINX_MODULES_MAIL="*"
 +
</syntaxhighlight>
  
First note that we are adding netfilter rules to the <tt>POSTROUTING</tt> chain, in the <tt>mangle</tt> table. This table allows us to modify the packets ''right before'' they are queued to be sent out of an interface, which is exactly what we want. At this point, these packets could have been locally-generated or forwarded -- as long as they are on their way to going out of <tt>modemif</tt> (eth4 in this case), the <tt>mangle</tt> <tt>POSTROUTING</tt> chain will see them and we can classify them and perform other useful tweaks.
+
This will enable all available modules for nginx.
  
The iptables code puts all traffic with the "minimize-delay" flag (interactive ssh traffic, for example) in the high priority traffic class. In addition, all HTTP, HTTPS and DNS TCP traffic will be classified as high-priority. Remember that all UDP traffic is being classified as high priority via the <tt>tc</tt> rule described above, so this will take care of DNS UDP traffic automatically.
+
Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run <tt>ebuild nginx-x.y.ebuild clean install</tt> to ensure that all modules patch/build properly. Basic build testing is now complete.
  
=== Further optimizations ===
+
== Media ==
 
+
{{#widget:YouTube|playlist=PL0k5C_Zqzft0QyD3G4l9cp8h1waSuNWlm}}
==== SSH ====
+
{{EbuildFooter}}
 
+
<source lang="bash">
+
iptables -t mangle -N tosfix
+
iptables -t mangle -A tosfix -p tcp -m length --length 0:512 -j RETURN
+
#allow screen redraws under interactive SSH sessions to be fast:
+
iptables -t mangle -A tosfix -m hashlimit --hashlimit 20/sec --hashlimit-burst 20 \
+
--hashlimit-mode srcip,srcport,dstip,dstport --hashlimit-name minlat -j RETURN
+
iptables -t mangle -A tosfix -j TOS --set-tos Maximize-Throughput
+
iptables -t mangle -A tosfix -j RETURN
+
 
+
iptables -t mangle -A POSTROUTING -p tcp -m tos --tos Minimize-Delay -j tosfix
+
</source>
+
 
+
To use this code, place it ''near the top of the file'', just below the <tt>modemif="eth4"</tt> line, but ''before'' the main <tt>iptables</tt> and <tt>tc</tt> rules. These rules will apply to ''all'' packets about to get queued to any interface, but this is not necessarily a bad thing, since the TCP flags being set are not just specific to our traffic control functionality. To make these rules specific to <tt>modemif</tt>, add "-o $modemif" after "-A POSTROUTING" on the last line, above. As-is, the rules above will set the TCP flags on all packets flowing out of all interfaces, but the the traffic control rules will only take effect for <tt>modemif</tt>, because they are only configured for that interface.
+
 
+
SSH is a tricky protocol. By default, all the outgoing SSH traffic is classified as "minimize-delay" traffic, which will cause it to all flow into our high-priority class, even if it is a bulk <tt>scp</tt> transfer running in the background. This code will grab all "minimize-delay" traffic such as SSH and telnet and route it through some special rules. Any individual keystrokes (small packets) will be left as "minimize-delay" packets. For anything else, we will run the <tt>hashlimit</tt> iptables module, which will identify individual outbound flows and allow small bursts of traffic (even big packets) to remain "minimize-delay" packets. These settings have been specifically tuned so that most <tt>GNU screen</tt> screen changes (^A^N) when logging into your server(s) remotely will be fast. Any traffic over these burst limits will be reclassified as "maximize-throughput" and thus will drop to our lower-priority class 1:12. Combined with the traffic control rules, this will allow you to have very responsive SSH sessions into your servers, even if they are doing some kind of bulk outbound copy, like rsync over SSH.
+
 
+
Code in our main <tt>iptables</tt> rules will ensure that any "minimize-delay" traffic is tagged to be in the high-priority 1:10 class.
+
 
+
What this does is keep interactive SSH and telnet keystrokes in the high-priority class, allow GNU screen full redraws and reasonable full-screen editor scrolling to remain in the high-priority class, while forcing bulk transfers into the lower-priority class.
+
 
+
==== ACKs ====
+
 
+
<source lang="bash">
+
iptables -t mangle -N ack
+
iptables -t mangle -A ack -m tos ! --tos Normal-Service -j RETURN
+
iptables -t mangle -A ack -p tcp -m length --length 0:128 -j TOS --set-tos Minimize-Delay
+
iptables -t mangle -A ack -p tcp -m length --length 128: -j TOS --set-tos Maximize-Throughput
+
iptables -t mangle -A ack -j RETURN
+
 
+
iptables -t mangle -A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK ACK -j ack
+
</source>
+
 
+
To use this code, place it ''near the top of the file, just below the <tt>modemif="eth4"</tt> line, but ''before'' the main <tt>iptables</tt> and <tt>tc</tt> rules.
+
 
+
ACK optimization is another useful thing to do. If we prioritize small ACKs heading out to the modem, it will allow TCP traffic to flow more smoothly without unnecessary delay.  The lines above accomplish this.
+
 
+
This code basically sets the "minimize-delay" flag on small ACKs. Code in our main <tt>iptables</tt> rules will then tag these packets so they enter high-priority traffic class 1:10.
+
 
+
== Other Links of Interest ==
+
* http://manpages.ubuntu.com/manpages/maverick/en/man8/ufw.8.html
+
* https://help.ubuntu.com/community/UFW
+
 
+
[[Category:Investigations]]
+
[[Category:Articles]]
+
[[Category:Featured]]
+
[[Category:Networking]]
+

Revision as of 22:59, January 2, 2015

www-servers/nginx


Current Maintainer(s):Daniel Robbins
Source Repository:Repository:Funtoo Overlay


Summary: Robust, small and high performance HTTP and reverse proxy server

Use Flags

addition
Enables HTTP addition filter module
aio
Enables file AIO support
flv
Enables special processing module for flv files
http
Enable HTTP core support
http-cache
Enable HTTP cache support
libatomic
Use libatomic instead of builtin atomic operations
pop
Enables POP3 proxy support
random-index
Enables HTTP random index module
realip
Enables realip module
smtp
Enables SMTP proxy support
static-gzip
Enables support for gzipping static content
status
Enables stub_status module
sub
Enables sub_filter module
webdav
Enable webdav support

News

Drobbins

Perl Updates

Gentoo has bumped perl from 5.20 to 5.22. Be sure to run perl-cleaner --all after the upgrade.
2015-07-25 by Drobbins
Drobbins

ARM Rebuild

ARM systems will use new stage3's that are not compatible with earlier versions.
2015-06-27 by Drobbins
Drobbins

ABI X86 64 and 32

Funtoo Linux has new 32-bit compatibility libraries inherited from Gentoo. Learn about them here.
2015-06-18 by Drobbins
More...

Nginx

Tip

We welcome improvements to this page. To edit this page, Create a Funtoo account. Then log in and then click here to edit this page. See our editing guidelines to becoming a wiki-editing pro.

Nginx.gif

nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in funtoo), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.

USE Expanded flags

Furthermore, you can set the nginx modules you like to use in /etc/make.conf in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".

nginx USE flags go into /etc/portage/package.use or /etc/portage/package.use/nginx, while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.

Example:

# echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx

Emerging nginx

Now you are ready to install nginx with php and xcache support:

# emerge -avt nginx php xcache

so now just check your useflags and press enter to start emerge.

Configuring

All configuration is done in /etc/nginx with nginx.conf as the main configuration file and all virtual hosts in /etc/nginx/sites/available while you have to symlink /etc/nginx/sites-available/{VHOST} to /etc/nginx/sites-enabled/{VHOST} to activate them. An example config for such a {VHOST} looks like that:

server {
    listen          80;
    server_name     www.example.com;

    access_log      /var/log/nginx/www.example.com.access_log main;
    error_log       /var/log/nginx/www.example.com.error_log info;

    root /var/www/www.example.com/htdocs;
}

The nginx.conf and sites-available/localhost file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.

php-fpm

nginx does not natively support php, so we delegate that responsibility to php-fpm

/etc/nginx/sites-available/localhost - fpm configuration
server {
        ...
	index index.php index.cgi index.htm index.html;
	location ~ .php$ {
	        fastcgi_pass 127.0.0.1:9000;
		include fastcgi.conf;
        }
        ...
}

php caching

/etc/nginx/sites-available/localhost - fpm cache configuration
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
server {
...
        location ~ \.php$ {
...
		fastcgi_cache MYAPP;
		fastcgi_cache_valid 200 60m;
...

for more information on php caching

proxy_pass

This configuration proxies to other webservers. In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails

/etc/nginx/sites-available/localhost - rails or python configurations
server {
        ...
	location /rails/ {
	    proxy_set_header Host $host;
	    proxy_set_header X-Real-IP $remote_addr;
	    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
	}
        ...
}

Location Processing Order

One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.

Two basic types of Location directives

There are two basic types of location directives. The first is called a "conventional string", and looks something like this:

location /foo { deny all; }

The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:

location ~ \.php$ { blah; }

The basic algorithm

Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most specific match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.

This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.

Advanced Location Processing

Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.

= (equals) Location

One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path exactly and completely. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:

location = /foo/bar { deny all; }

~* (case-insensitive regex) Location

A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.

^~ (short-circuit conventional string) Location

You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the most specific match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans all conventional string matches to find the best match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the best match.

Ebuild Update Protocol

To work on a new version of the ebuild, perform the following steps.

First, temporarily set the following settings in /etc/make.conf:

NGINX_MODULES_HTTP="*"
NGINX_MODULES_MAIL="*"

This will enable all available modules for nginx.

Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run ebuild nginx-x.y.ebuild clean install to ensure that all modules patch/build properly. Basic build testing is now complete.

Media