Difference between pages "Usability Testing" and "Package:Nginx"

From Funtoo
(Difference between pages)
Jump to navigation Jump to search
 
m (add media playlist)
 
Line 1: Line 1:
This page provides a place to document your install experiences, highlighting pain points so that they can be addressed in future versions of Funtoo Linux.
{{Ebuild
|Summary=Robust, small and high performance HTTP and reverse proxy server
|CatPkg=www-servers/nginx
|Maintainer=Drobbins
|Repository=Funtoo Overlay
|Overlay=Funtoo
}}
[[Image:nginx.gif|frame]]


== Daniel Robbins, ninja2, 12/24/2010 ==
nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in [http://funtoo.org funtoo]), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.


Installation of ninja2 (Dell R710) using rhel5-openvz-binaries kernel/modules/initrd .tbz2 (compiled on another machine using gcc-4.1.2) and GPT/GUID partitions. Use of binary kernel was successful although directing Portage to use the binary package was difficult - it wanted to find it inside a package repository rather than accept it directly from the command-line. Forgot to downgrade udev-160 to udev-146-r4 which resulted in a missing /etc/udev/rules.d/70-persistent-net.rules. Also didn't have brctl (bridge-utils) or ifenslave on the stage3 to allow me to set up the network correctly. Also ran into an issue with boot-update where I got the configuration wrong, because I used "kernel-[-v]" instead of "kernel[-v]". This could be a usability issue with the boot.conf syntax.
=== USE Expanded flags ===


After getting the system booted, I ran into an issue with macaddr renaming - you can't rename a device to eth1 if eth1 already exists! So I realized the limitations of macaddr renaming using nameif (built in to the Funtoo Linux network configuration scripts.) After fixing 70-persistent-net.rules and thinking of ways we could have some kind of default assignment rather than the currently random approach, I rebooted, configured my network, firewall, OpenVZ using the masked vzctl-3.0.25-r2 -- without issue -- and had the system up and running. I found that "bc" is no longer installed by default (I thought it was) -- so I added it to the default Metro stage3. Only ~90K.
Furthermore, you can set the nginx modules you like to use in ''/etc/make.conf'' in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".


Possible usability improvements:
nginx USE flags go into ''/etc/portage/package.use'' or ''/etc/portage/package.use/nginx'', while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.


* document use of rhel5-openvz-binaries
Example:
* improve Portage's ability to install binary packages by having the .tbz2 specified on the command-line
<console>
* improve management of udev to either force downgrade or allow multiple versions to exist on the same system (this is becoming more appealing....)
###i## echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx
* add bridge-utils to default stage3 (DONE)
</console>
* add ifenslave to default stage3
* add bc to default stage3 (DONE)
* integrate some scheme for logically ordering ethX devices when you have many on a machine
* look into improving boot-update configuration syntax to avoid "-[-v]" issues that can cause thorny config problems.


Successes:
=== Emerging nginx ===


* the test of rhel5-openvz-binaries tbz2 was successful overall. It worked, and saved a ton of effort.
Now you are ready to install nginx with php and xcache support:
* GPT/GUID partitioning worked fine.
<console>
* testing of vzctl-3.0.25-r2 (currently in testing) worked perfectly.
###i## emerge -avt nginx php xcache
* install was relatively pain-free for a pro but still needs work.
</console>
so now just check your useflags and press enter to start emerge.


[[Category:QA]]
== Configuring ==
 
All configuration is done in ''/etc/nginx'' with ''nginx.conf'' as the main configuration file and all virtual hosts in ''/etc/nginx/sites/available'' while you have to symlink ''/etc/nginx/sites-available/{VHOST}'' to ''/etc/nginx/sites-enabled/{VHOST}'' to activate them. An example config for such a {VHOST} looks like that:
 
<pre>
server {
    listen          80;
    server_name    www.example.com;
 
    access_log      /var/log/nginx/www.example.com.access_log main;
    error_log      /var/log/nginx/www.example.com.error_log info;
 
    root /var/www/www.example.com/htdocs;
}
</pre>
 
The ''nginx.conf'' and ''sites-available/localhost'' file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.
 
=== php-fpm ===
 
nginx does not natively support php, so we delegate that responsibility to [[Package:Php#Fpm | php-fpm]]
 
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm configuration|body=
server {
        ...
index index.php index.cgi index.htm index.html;
location ~ .php$ {
        fastcgi_pass 127.0.0.1:9000;
include fastcgi.conf;
        }
        ...
}
}}
 
==== php caching ====
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm cache configuration|body=
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
server {
...
        location ~ \.php$ {
...
fastcgi_cache MYAPP;
fastcgi_cache_valid 200 60m;
...
}}
[https://www.digitalocean.com/community/tutorials/how-to-setup-fastcgi-caching-with-nginx-on-your-vps for more information on php caching]
 
=== proxy_pass===
This configuration proxies to other webservers.  In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails
 
{{file|name=/etc/nginx/sites-available/localhost|desc=rails or python configurations|body=
server {
        ...
location /rails/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
}
        ...
}
}}
 
== Location Processing Order ==
One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.
 
=== Two basic types of Location directives ===
There are two basic types of location directives. The first is called a "conventional string", and looks something like this:
location /foo { deny all; }
The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:
location ~ \.php$ { blah; }
 
=== The basic algorithm ===
Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most ''specific'' match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.
 
This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.
 
== Advanced Location Processing ==
Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.
 
=== = (equals) Location ===
One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path ''exactly'' and ''completely''. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:
location = /foo/bar { deny all; }
 
=== ~* (case-insensitive regex) Location ===
A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.
 
=== ^~ (short-circuit conventional string) Location ===
You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the ''most specific'' match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans ''all'' conventional string matches to find the ''best'' match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the ''best'' match.
 
== Ebuild Update Protocol ==
 
To work on a new version of the ebuild, perform the following steps.
 
First, temporarily set the following settings in <tt>/etc/make.conf</tt>:
 
<syntaxhighlight lang="bash">
NGINX_MODULES_HTTP="*"
NGINX_MODULES_MAIL="*"
</syntaxhighlight>
 
This will enable all available modules for nginx.
 
Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run <tt>ebuild nginx-x.y.ebuild clean install</tt> to ensure that all modules patch/build properly. Basic build testing is now complete.
 
== Media ==
{{#widget:YouTube|playlist=PL0k5C_Zqzft0QyD3G4l9cp8h1waSuNWlm}}
{{EbuildFooter}}

Revision as of 22:59, January 2, 2015

Nginx

   Tip

We welcome improvements to this page. To edit this page, Create a Funtoo account. Then log in and then click here to edit this page. See our editing guidelines to becoming a wiki-editing pro.

Nginx.gif

nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in funtoo), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.

USE Expanded flags

Furthermore, you can set the nginx modules you like to use in /etc/make.conf in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".

nginx USE flags go into /etc/portage/package.use or /etc/portage/package.use/nginx, while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.

Example:

root # echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx

Emerging nginx

Now you are ready to install nginx with php and xcache support:

root # emerge -avt nginx php xcache

so now just check your useflags and press enter to start emerge.

Configuring

All configuration is done in /etc/nginx with nginx.conf as the main configuration file and all virtual hosts in /etc/nginx/sites/available while you have to symlink /etc/nginx/sites-available/{VHOST} to /etc/nginx/sites-enabled/{VHOST} to activate them. An example config for such a {VHOST} looks like that:

server {
    listen          80;
    server_name     www.example.com;

    access_log      /var/log/nginx/www.example.com.access_log main;
    error_log       /var/log/nginx/www.example.com.error_log info;

    root /var/www/www.example.com/htdocs;
}

The nginx.conf and sites-available/localhost file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.

php-fpm

nginx does not natively support php, so we delegate that responsibility to php-fpm

   /etc/nginx/sites-available/localhost - fpm configuration
server {
        ...
	index index.php index.cgi index.htm index.html;
	location ~ .php$ {
	        fastcgi_pass 127.0.0.1:9000;
		include fastcgi.conf;
        }
        ...
}

php caching

   /etc/nginx/sites-available/localhost - fpm cache configuration
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
server {
...
        location ~ \.php$ {
...
		fastcgi_cache MYAPP;
		fastcgi_cache_valid 200 60m;
...

for more information on php caching

proxy_pass

This configuration proxies to other webservers. In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails

   /etc/nginx/sites-available/localhost - rails or python configurations
server {
        ...
	location /rails/ {
	    proxy_set_header Host $host;
	    proxy_set_header X-Real-IP $remote_addr;
	    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
	}
        ...
}

Location Processing Order

One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.

Two basic types of Location directives

There are two basic types of location directives. The first is called a "conventional string", and looks something like this:

location /foo { deny all; }

The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:

location ~ \.php$ { blah; }

The basic algorithm

Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most specific match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.

This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.

Advanced Location Processing

Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.

= (equals) Location

One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path exactly and completely. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:

location = /foo/bar { deny all; }

~* (case-insensitive regex) Location

A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.

^~ (short-circuit conventional string) Location

You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the most specific match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans all conventional string matches to find the best match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the best match.

Ebuild Update Protocol

To work on a new version of the ebuild, perform the following steps.

First, temporarily set the following settings in /etc/make.conf:

NGINX_MODULES_HTTP="*"
NGINX_MODULES_MAIL="*"

This will enable all available modules for nginx.

Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run ebuild nginx-x.y.ebuild clean install to ensure that all modules patch/build properly. Basic build testing is now complete.

Media