Difference between pages "GUID Booting Guide" and "Package:Nginx"

From Funtoo
(Difference between pages)
Jump to navigation Jump to search
 
m (add media playlist)
 
Line 1: Line 1:
== Introduction ==
{{Ebuild
|Summary=Robust, small and high performance HTTP and reverse proxy server
|CatPkg=www-servers/nginx
|Maintainer=Drobbins
|Repository=Funtoo Overlay
|Overlay=Funtoo
}}
[[Image:nginx.gif|frame]]


GPT, which stands for GUID Partition Table, is a disk partitioning scheme that was introduced by Intel for Itanium architecture systems, as part of EFI, the Extensible Firmware Interface. While you are probably not using an Itanium architecture computer, and you are likely using a BIOS-based rather than an EFI-based system, you still may want to use GPT partitioning. Why? Because the standard MBR-based partitioning scheme only supports system disks that are less than 2TiB in size. On modern systems, especially systems with hardware RAID logical volumes, it is very easy to go beyond the 2TiB limit. GUID partition tables support disks that are larger than 2TiB in size.
nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in [http://funtoo.org funtoo]), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.  


=== GPT Technology Overview ===
=== USE Expanded flags ===


This section contains a technical overview of GPT technology.
Furthermore, you can set the nginx modules you like to use in ''/etc/make.conf'' in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".


GUID partition tables support up to 512 partitions. GPT data structures are stored in the first sectors of the drive with a secondary copy stored at the end of the drive. This allows the partitioning scheme of your disk to be recovered in situations where the primary partition table has been corrupted.
nginx USE flags go into ''/etc/portage/package.use'' or ''/etc/portage/package.use/nginx'', while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.


For compatibility with legacy partitioning tools, GPT partitioning tools typically rewrite the MBR partition table (generally located in the first sector of the disk) in a way those tools will interpret it like ''"This disk has only one partition (of an unknown type) covering the whole disk".''
Example:
<console>
###i## echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx
</console>


It is possible to convert an existing MBR-partitioned disk to GPT format using the <tt>gdisk</tt> command. Please carefully read the <tt>gdisk</tt> man page before using this capability, as it is potentially dangerous, particularly if you are performing it on your boot disk.
=== Emerging nginx ===


{{ fancyimportant|Funtoo Linux fully supports GPT on x86-32bit and x86-64-bit systems. GPT is supported on SPARC systems, but currently only for non-boot disks.
Now you are ready to install nginx with php and xcache support:
}}
<console>
###i## emerge -avt nginx php xcache
</console>
so now just check your useflags and press enter to start emerge.


=== Booting GPT ===
== Configuring ==


If you decide to use a GPT-based partitioning scheme for your system disk, either out of necessity due to a 2TiB+ disk, or because you want to try GPT out, then the question arises -- how do you get the darn thing to boot? This is where the new <tt>GRUB</tt> boot loader comes in. The new <tt>GRUB</tt> (version 1.9x, found at <tt>sys-boot/grub</tt>) is a redesign of the original <tt>GRUB</tt> (version 0.9x, now called <tt>sys-boot/grub-legacy</tt> in Funtoo) boot-loader that includes very mature support for booting from GPT-based disks.
All configuration is done in ''/etc/nginx'' with ''nginx.conf'' as the main configuration file and all virtual hosts in ''/etc/nginx/sites/available'' while you have to symlink ''/etc/nginx/sites-available/{VHOST}'' to ''/etc/nginx/sites-enabled/{VHOST}'' to activate them. An example config for such a {VHOST} looks like that:


Now, let's take a look at how to get GPT-based booting working under Funtoo Linux.
<pre>
server {
    listen          80;
    server_name    www.example.com;


== Getting Started ==
    access_log      /var/log/nginx/www.example.com.access_log main;
    error_log      /var/log/nginx/www.example.com.error_log info;


The first thing you'll need to do is to use a LiveCD. I recommend [http://www.sysresccd.org/Main_Page System Rescue CD] for this task as it is Gentoo-based and includes all the proper tools. Go ahead and boot the LiveCD, and then get to the point where you are ready to partition your system disk.
    root /var/www/www.example.com/htdocs;
}
</pre>


At this point, you have two choices as to what partitioning tool to use. You can use either <tt>gdisk</tt> or <tt>parted</tt>. <tt>gdisk</tt> is a very nice <tt>fdisk</tt>-like partitioning tool that supports GPT partitioning. It is rather new software but seems to work quite well. The other tool you can use, GNU <tt>parted</tt>, has been around for a while and is more mature, but is harder to use.
The ''nginx.conf'' and ''sites-available/localhost'' file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.  


We'll take a look at how to create partitions using <tt>gdisk</tt>. Alternatively, <code>cgdisk</code>, curses-based gdisk for users familiar with cfdisk or <code>sgdisk</code>, command-line tool can be used for creating and managing GPT partitions.
=== php-fpm ===


== Partitioning Using Gdisk ==
nginx does not natively support php, so we delegate that responsibility to [[Package:Php#Fpm | php-fpm]]


OK, the first step is using <tt>gdisk</tt> is to start it up, specifying the disk you want to modify:
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm configuration|body=
server {
        ...
index index.php index.cgi index.htm index.html;
location ~ .php$ {
        fastcgi_pass 127.0.0.1:9000;
include fastcgi.conf;
        }
        ...
}
}}


<pre># gdisk /dev/sda</pre>
==== php caching ====
You should find <tt>gdisk</tt> very familiar to <tt>fdisk</tt>. Here is the partition table we want to end up with:
{{file|name=/etc/nginx/sites-available/localhost|desc=fpm cache configuration|body=
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
server {
...
        location ~ \.php$ {
...
fastcgi_cache MYAPP;
fastcgi_cache_valid 200 60m;
...
}}
[https://www.digitalocean.com/community/tutorials/how-to-setup-fastcgi-caching-with-nginx-on-your-vps for more information on php caching]


<pre>Command (? for help): p
=== proxy_pass===
Disk /dev/sda: 312581808 sectors, 149.1 GiB
This configuration proxies to other webservers.  In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails
Disk identifier (GUID): 17
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 312581774
Total free space is 0 sectors (0 bytes)


Number  Start (sector)    End (sector)  Size      Code  Name
{{file|name=/etc/nginx/sites-available/localhost|desc=rails or python configurations|body=
  1             34          204833  100.0 MiB  0700  Linux/Windows data
server {
  2          204834          270369  512.0 kiB  EF02  BIOS boot partition
        ...
  3          270370        1318945  512.0 MiB  8200  Linux swap
location /rails/ {
  4         1318946      312581774  148.4 GiB  0700  Linux/Windows data
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
}
         ...
}
}}


Command (? for help): </pre>
== Location Processing Order ==
Above, you'll see that we have a 100 MiB boot partition, a 512 kiB &quot;BIOS boot partition&quot;, 512 MiB of swap, and the remaining disk used by a 148.4 GiB root partition.
One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.  


The one new thing here is the &quot;BIOS boot partition.&quot; What is it? In GRUB-speak, this BIOS boot partition is basically the location of the meat of GRUB's boot loading code - the quivalent of the <tt>stage1_5</tt> and <tt>stage2</tt> files in legacy GRUB. Since GPT-based partition tables have less &quot;bonus&quot; space than their MBR equivalents, and explicit partition of code <tt>EF02</tt> is required to hold the guts of the boot loader.
=== Two basic types of Location directives ===
There are two basic types of location directives. The first is called a "conventional string", and looks something like this:
location /foo { deny all; }
The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:
location ~ \.php$ { blah; }


In all other respects, the partition table is similar to that of an MBR-based disk. We have a boot and root partition with code <tt>0700</tt>, and a Linux swap partition with code <tt>8200</tt>. One this partition table has been written to disk and appropriate <tt>mkfs</tt> and <tt>mkswap</tt> commands are issued, <tt>/dev/sda1</tt> will be used to hold <tt>/boot</tt>, <tt>/dev/sda2</tt> will be used by the new GRUB directly, <tt>/dev/sda3</tt> will house our swap and <tt>/dev/sda4</tt> will hold our root filesystem.
=== The basic algorithm ===
Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most ''specific'' match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.


Go ahead and create filesystems on these partitions, and then mount the root and boot filesystems to <tt>/mnt/gentoo</tt> and <tt>/mnt/gentoo/boot</tt> respectively. Now go ahead and unpack a stage3 tarball to <tt>/mnt/gentoo</tt> and chroot in as you normally do.
This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.


== Configuring The Kernel ==
== Advanced Location Processing ==
Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.


Your kernel will need a couple of extra GPT-related options enabled in order for it to make sense of your GPT partitions and find your filesystems. These options can be found under <tt>Enable the block layer ---&gt; Partition Types</tt>:
=== = (equals) Location ===
One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path ''exactly'' and ''completely''. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:
location = /foo/bar { deny all; }


<pre>[*] Advanced Partition Selection (PARTITION_ADVANCED)
=== ~* (case-insensitive regex) Location ===
[*] EFI GUID Partition Support (EFI_PARTITION)</pre>
A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.
If you are using a non-Funtoo distribution then you may need to append a proper <tt>rootfstype=</tt> option to your kernel boot options to allow Linux to properly mount the root filesystem when <tt>Advanced Partition Selection</tt> is enabled. [[Boot-Update]] does this for you automatically.


Now just go ahead and compile and install your kernel, and copy it to <tt>/boot/bzImage</tt>.
=== ^~ (short-circuit conventional string) Location ===
You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the ''most specific'' match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans ''all'' conventional string matches to find the ''best'' match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the ''best'' match.


== Booting The System ==
== Ebuild Update Protocol ==


To get the system booted, you will want to first edit <tt>/etc/fstab</tt> inside the chroot so that it reflects the partitions and filesystems you just created. Then, emerge <tt>boot-update</tt> version 1.4_beta2 or later:
To work on a new version of the ebuild, perform the following steps.


<pre># emerge boot-update</pre>
First, temporarily set the following settings in <tt>/etc/make.conf</tt>:
[[Boot-Update]] is a front-end for the GRUB 1.9x boot loader and provides a necessary simplified configuration interface. <tt>boot-update</tt> is used to generate boot loader configuration files. But before we get to <tt>boot-update</tt>, we first need to install GRUB to your hard disk. This is done as follows:
 
<pre># grub-install /dev/sda</pre>
<tt>grub-install</tt> will detect and use <tt>/dev/sda2</tt> and use it to store its boot loader logic.
 
Now it's time to create an <tt>/etc/boot.conf</tt> file. For more information on all available options, consult the [[Boot-Update]] guide -- I'll show you a sample configuration for the sample GPT partition scheme above:
 
<pre>boot {
        generate grub
        default bzImage
}


&quot;Funtoo Linux&quot; {
<syntaxhighlight lang="bash">
        kernel bzImage
NGINX_MODULES_HTTP="*"
}</pre>
NGINX_MODULES_MAIL="*"
Once <tt>/etc/boot.conf</tt> has been created, then type:
</syntaxhighlight>


<pre># boot-update</pre>
This will enable all available modules for nginx.
This will auto-generate a <tt>/boot/grub/grub.cfg</tt> file for you, and you will now be able to reboot into Funtoo Linux using a GPT partitioning scheme.


For more information on all the options available for <tt>/etc/boot.conf</tt>, please consult the [[Boot-Update]] guide.
Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run <tt>ebuild nginx-x.y.ebuild clean install</tt> to ensure that all modules patch/build properly. Basic build testing is now complete.


[[Category:Funtoo features]]
== Media ==
{{#widget:YouTube|playlist=PL0k5C_Zqzft0QyD3G4l9cp8h1waSuNWlm}}
{{EbuildFooter}}

Revision as of 22:59, January 2, 2015

Nginx

   Tip

We welcome improvements to this page. To edit this page, Create a Funtoo account. Then log in and then click here to edit this page. See our editing guidelines to becoming a wiki-editing pro.

Nginx.gif

nginx (pronounced "engin-x") is a Web and reverse proxy server for HTTP, SMTP, POP3 and IMAP protocols. It focuses on high concurrency, performance and low memory usage. Nginx quickly delivers static content with efficient use of system resources, also dynamic content is delivered on a network using FastCGI, SCGI handlers for scripts, uWSGI application servers or Phusion Passenger module (atm broken in funtoo), further more it can serve a very capable software load balancer. It uses an asynchronos event-driven approach to handle requests which provides more predictable performance under load, in contrast to the Apache HTTP server model, that uses a threaded or process-oriented approach to handling request. Nginx is licensed under a BSD-like license and it runs on Unix, Linux, BSD variants, Mac OS X, Solaris, AIX and Microsoft Windows.

USE Expanded flags

Furthermore, you can set the nginx modules you like to use in /etc/make.conf in the NGINX_MODULES_HTTP variable as NGINX_MODULES_HTTP="variables".

nginx USE flags go into /etc/portage/package.use or /etc/portage/package.use/nginx, while the HTTP and MAIL modules go as NGINX_MODULES_HTTP or NGINX_MODULES_MAIL are stored in /etc/make.conf. And as you wouldn't server only static html files, but most commonly also php files/scripts you should also install php with fpm enabled and xcache for caching the content, what makes your nginx setup way faster. For xcache you need to set PHP_TARGETS="php5-3" in '/etc/make.conf'.

Example:

root # echo "www-servers/nginx USE-FLAG-List" >> /etc/portage/package.use/nginx

Emerging nginx

Now you are ready to install nginx with php and xcache support:

root # emerge -avt nginx php xcache

so now just check your useflags and press enter to start emerge.

Configuring

All configuration is done in /etc/nginx with nginx.conf as the main configuration file and all virtual hosts in /etc/nginx/sites/available while you have to symlink /etc/nginx/sites-available/{VHOST} to /etc/nginx/sites-enabled/{VHOST} to activate them. An example config for such a {VHOST} looks like that:

server {
    listen          80;
    server_name     www.example.com;

    access_log      /var/log/nginx/www.example.com.access_log main;
    error_log       /var/log/nginx/www.example.com.error_log info;

    root /var/www/www.example.com/htdocs;
}

The nginx.conf and sites-available/localhost file is well commented. Customize it to your needs. Make sure you set the listen option correctly. By default, the listen option is set to listen on the loopback interface. If you leave this unchanged other computers on the network will not be able to connect to the server.

php-fpm

nginx does not natively support php, so we delegate that responsibility to php-fpm

   /etc/nginx/sites-available/localhost - fpm configuration
server {
        ...
	index index.php index.cgi index.htm index.html;
	location ~ .php$ {
	        fastcgi_pass 127.0.0.1:9000;
		include fastcgi.conf;
        }
        ...
}

php caching

   /etc/nginx/sites-available/localhost - fpm cache configuration
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
server {
...
        location ~ \.php$ {
...
		fastcgi_cache MYAPP;
		fastcgi_cache_valid 200 60m;
...

for more information on php caching

proxy_pass

This configuration proxies to other webservers. In this example we have webrick running on port 3000 behind nginx producing the live link http://localhost/rails

   /etc/nginx/sites-available/localhost - rails or python configurations
server {
        ...
	location /rails/ {
	    proxy_set_header Host $host;
	    proxy_set_header X-Real-IP $remote_addr;
	    proxy_pass http://127.0.0.1:3000/; #for ruby on rails webrick
            #proxy_pass http://127.0.0.1:8000/; #for python -m http.server
            #proxy_pass http://127.0.0.1:8080/; #for other web servers like apache, lighttpd, tengine, cherokee, etc...
	}
        ...
}

Location Processing Order

One often confusing aspect of nginx configuration is the order in which it processes location directives. This section is intended to clarify the confusion and help you to write secure nginx location directives.

Two basic types of Location directives

There are two basic types of location directives. The first is called a "conventional string", and looks something like this:

location /foo { deny all; }

The second basic type of location directive is a regex, or regular expression block. In its most basic form, it looks like this, with a "~" and then a regular expression that is matched against the request path. "^" can be used to match the beginning of the request path, and "$" can be used to match the end of the request path. If you need to match a ".", you must escape it as "\." as per regular expression matching rules:

location ~ \.php$ { blah; }

The basic algorithm

Nginx uses a special algorithm to find the proper location string to match the incoming request. The basic concept to remember is that conventional string directives are placed in one "bucket", and then regular expression strings are placed in another "bucket". Nginx will use the first regular expression match that it finds, when scanning the file from top to bottom. If no matching regular expression is found, nginx will look in its "conventional string" bucket, and try to find a match. In the case of the conventional string matches, the most specific match will be used, in other words, the one will be used that matches the greatest number of characters in the request path.

This is the foundation for nginx location processing, so always use these rules as a starting point for understanding location matching order. Nginx then provides various sub-types of location directives which modify this default behavior in a number of ways. This will be covered in the next section.

Advanced Location Processing

Always use the location processing logic described in the previous section as the foundation for understanding how nginx finds a matching location directive, and then once you are comfortable with how this works, read about these more advanced directives and understand how they fit into nginx's overall logic.

= (equals) Location

One advanced location directive is the "=" location, which can be considered a variant of a "conventional string" directive. "=" directives are searched before all other directives, and if a match found, then the corresponding location block is used. A "=" location must the requested path exactly and completely. For example, the following location block will match only the request /foo/bar, but not /foo/bar/oni.html:

location = /foo/bar { deny all; }

~* (case-insensitive regex) Location

A "~*" regex match is just like a regular "~" regex match, except matches will be performed in a case-insensitive manner. "~*" location directives, being regex directives, fall into the regex "bucket" and are processed along other regex directives. This means that they are processed in the order they appear in your configuration file and the first match will be used -- assuming no "=" directives match.

^~ (short-circuit conventional string) Location

You may think that a "^~" location is a regex location, but it is not. It is a variant of a conventional string location. If you recall, nginx will search for conventional string matches by finding the most specific match. However, when you use a "^~" location, nginx behavior is modified. Imagine the way a conventional string match works. Nginx scans your configuration file, looking at each conventional string match from line 1 to the end of file, but it scans all conventional string matches to find the best match. Well, the "~^" location match short-circuits this process. If, in the process of scanning each conventional string match in the config file, nginx encounters a "^~" match that matches the current request path, then nginx will apply this match, and stop looking for the best match.

Ebuild Update Protocol

To work on a new version of the ebuild, perform the following steps.

First, temporarily set the following settings in /etc/make.conf:

NGINX_MODULES_HTTP="*"
NGINX_MODULES_MAIL="*"

This will enable all available modules for nginx.

Now, create a new version of the ebuild in your overlay, and look at all the modules listed at the top of the ebuild. Visit the URLs in the comments above each one and ensure that the latest versions of each are included. Now run ebuild nginx-x.y.ebuild clean install to ensure that all modules patch/build properly. Basic build testing is now complete.

Media