Difference between pages "Awk by Example, Part 2" and "Funtoo:Keychain"

From Funtoo
(Difference between pages)
Jump to navigation Jump to search
 
 
Line 1: Line 1:
{{Article
{{Article
|Subtitle=Records, loops, and arrays
|Subtitle=Official Project Page
|Keywords=command,unix,variables,print,space
|Summary=Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. Download and learn how to use Keychain on your Linux, Unix or MacOS system.
|Keywords=keychain,ssh,rsa,dsa,gpg,linux,gentoo,macos,download,source code
|Author=Drobbins
|Author=Drobbins
|Previous in Series=Awk by Example, Part 1
|Next in Series=Awk by Example, Part 3
}}
}}
=== Multi-line records ===
<tt>Keychain</tt> helps you to manage SSH and GPG keys in a convenient and secure manner. It acts as a frontend to <tt>ssh-agent</tt> and <tt>ssh-add</tt>, but allows you to easily have one long running <tt>ssh-agent</tt> process per system, rather than the norm of one <tt>ssh-agent</tt> per login session.
Awk is an excellent tool for reading in and processing structured data, such as the system's /etc/passwd file. /etc/passwd is the UNIX user database, and is a colon-delimited text file, containing a lot of important information, including all existing user accounts and user IDs, among other things. In my previous article, I showed you how awk could easily parse this file. All we had to do was to set the FS (field separator) variable to ":".


By setting the FS variable correctly, awk can be configured to parse almost any kind of structured data, as long as there is one record per line. However, just setting FS won't do us any good if we want to parse a record that exists over multiple lines. In these situations, we also need to modify the RS record separator variable. The RS variable tells awk when the current record ends and a new record begins.
This dramatically reduces the number of times you need to enter your passphrase. With <tt>keychain</tt>, you only need to enter a passphrase once every time your local machine is rebooted. <tt>Keychain</tt> also makes it easy for remote cron jobs to securely "hook in" to a long-running <tt>ssh-agent</tt> process, allowing your scripts to take advantage of key-based logins.


As an example, let's look at how we'd handle the task of processing an address list of Federal Witness Protection Program participants:
Those who are new to OpenSSH and the use of public/private keys for authentication may want to check out the following articles by Daniel Robbins, which will provide a gentle introduction to the concepts used by Keychain:
<pre>
* [[OpenSSH Key Management, Part_1]]
Jimmy the Weasel
* [[OpenSSH Key Management, Part_2]]
100 Pleasant Drive
* [[OpenSSH Key Management, Part_3]]
San Francisco, CA 12345


Big Tony
== Download and Resources ==
200 Incognito Ave.
Suburbia, WA 67890
</pre>
Ideally, we'd like awk to recognize each 3-line address as an individual record, rather than as three separate records. It would make our code a lot simpler if awk would recognize the first line of the address as the first field ($1), the street address as the second field ($2), and the city, state, and zip code as field $3. The following code will do just what we want:
<pre>
BEGIN {
    FS="\n"
    RS=""
}
</pre>
Above, setting FS to "\n" tells awk that each field appears on its own line. By setting RS to "", we also tell awk that each address record is separated by a blank line. Once awk knows how the input is formatted, it can do all the parsing work for us, and the rest of the script is simple. Let's look at a complete script that will parse this address list and print out each address record on a single line, separating each field with a comma.
<pre>
BEGIN {
    FS="\n"
    RS=""
}
{ print $1 ", " $2 ", " $3 }
</pre>
If this script is saved as address.awk, and the address data is stored in a file called address.txt, you can execute this script by typing awk -f address.awk address.txt. This code produces the following output:
<pre>
Jimmy the Weasel, 100 Pleasant Drive, San Francisco, CA 12345
Big Tony, 200 Incognito Ave., Suburbia, WA 67890
</pre>


=== OFS and ORS ===
The latest release of keychain is version <tt>2.7.2_beta1</tt>, and was released on July 7, 2014. The current version of keychain supports <tt>gpg-agent</tt> as well as <tt>ssh-agent</tt>.
In address.awk's print statement, you can see that awk concatenates (joins) strings that are placed next to each other on a line. We used this feature to insert a comma and a space (", ") between the three address fields that appeared on the line. While this method works, it's a bit ugly looking. Rather than inserting literal ", " strings between our fields, we can have awk do it for us by setting a special awk variable called OFS. Take a look at this code snippet.
<pre>
print "Hello", "there", "Jim!"
</pre>


The commas on this line are not part of the actual literal strings. Instead, they tell awk that "Hello", "there", and "Jim!" are separate fields, and that the OFS variable should be printed between each string. By default, awk produces the following output:
Keychain is compatible with many operating systems, including <tt>AIX</tt>, <tt>*BSD</tt>, <tt>Cygwin</tt>, <tt>MacOS X</tt>, <tt>Linux</tt>, <tt>HP/UX</tt>, <tt>Tru64 UNIX</tt>, <tt>IRIX</tt>, <tt>Solaris</tt> and <tt>GNU Hurd</tt>.
<pre>
Hello there Jim!
</pre>
This shows us that by default, OFS is set to " ", a single space. However, we can easily redefine OFS so that awk will insert our favorite field separator. Here's a revised version of our original address.awk program that uses OFS to output those intermediate ", " strings:
<pre>
BEGIN {
    FS="\n"
    RS=""
    OFS=", "
}
{ print $1, $2, $3 }
</pre>
Awk also has a special variable called ORS, called the "output record separator". By setting ORS, which defaults to a newline ("\n"), we can control the character that's automatically printed at the end of a print statement. The default ORS value causes awk to output each new print statement on a new line. If we wanted to make the output double-spaced, we would set ORS to "\n\n". Or, if we wanted records to be separated by a single space (and no newline), we would set ORS to " ".


=== Multi-line to tabbed ===
=== Download ===
Let's say that we wrote a script that converted our address list to a single-line per record, tab-delimited format for import into a spreadsheet. After using a slightly modified version of address.awk, it would become clear that our program only works for three-line addresses. If awk encountered the following address, the fourth line would be thrown away and not printed:
<pre>
Cousin Vinnie
Vinnie's Auto Shop
300 City Alley
Sosueme, OR 76543
</pre>
To handle situations like this, it would be good if our code took the number of records per field into account, printing each one in order. Right now, the code only prints the first three fields of the address. Here's some code that does what we want:
<pre>
BEGIN {
    FS="\n"
    RS=""
    ORS=""
}
{ 
    x=1
    while ( x<NF ) {
        print $x "\t"
        x++
    }
    print $NF "\n"
}
</pre>
First, we set the field separator FS to "\n" and the record separator RS to "" so that awk parses the multi-line addresses correctly, as before. Then, we set the output record separator ORS to "", which will cause the print statement to not output a newline at the end of each call. This means that if we want any text to start on a new line, we need to explicitly write print "\n".


In the main code block, we create a variable called x that holds the number of current field that we're processing. Initially, it's set to 1. Then, we use a while loop (an awk looping construct identical to that found in the C language) to iterate through all but the last record, printing the record and a tab character. Finally, we print the last record and a literal newline; again, since ORS is set to "", print won't output newlines for us. Program output looks like this, which is exactly what we wanted:
* ''Release Archive''
<pre>
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.2_beta1.tar.bz2 keychain 2.7.2_beta1]
Jimmy the Weasel        100 Pleasant Drive      San Francisco, CA 12345
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1.tar.bz2 keychain 2.7.1]
Big Tony        200 Incognito Ave.     Suburbia, WA 67890
Cousin Vinnie  Vinnie's Auto Shop      300 City Alley  Sosueme, OR 76543
</pre>


=== Looping constructs ===
* ''Apple MacOS X Packages''
We've already seen awk's while loop construct, which is identical to its C counterpart. Awk also has a "do...while" loop that evaluates the condition at the end of the code block, rather than at the beginning like a standard while loop. It's similar to "repeat...until" loops that can be found in other languages. Here's an example:
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1-macosx.tar.gz keychain 2.7.1 MacOS X package]
<pre>
{
    count=1
    do {
        print "I get printed at least once no matter what"
    } while ( count != 1 )
}
</pre>
Because the condition is evaluated after the code block, a "do...while" loop, unlike a normal while loop, will always execute at least once. On the other hand, a normal while loop will never execute if its condition is false when the loop is first encountered.


=== for loops ===
Keychain development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [https://bugs.funtoo.org Funtoo Linux bug tracker] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.
Awk allows you to create for loops, which like while loops are identical to their C counterpart:
<pre>
for ( initial assignment; comparison; increment ) {
    code block
}
</pre>
Here's a quick example:
<pre>
for ( x = 1; x <= 4; x++ ) {
    print "iteration",x
}
</pre>
This snippet will print:
<pre>
iteration 1
iteration 2
iteration 3
iteration 4
</pre>


=== Break and continue ===
=== Project History ===
Again, just like C, awk provides break and continue statements. These statements provide better control over awk's various looping constructs. Here's a code snippet that desperately needs a break statement:
<pre>
while (1) {
    print "forever and ever..."
}
</pre>
Because 1 is always true, this while loop runs forever. Here's a loop that only executes ten times:
<pre>
x=1
while(1) {
    print "iteration",x
    if ( x == 10 ) {
        break
    }
    x++
}
</pre>
Here, the break statement is used to "break out" of the innermost loop. "break" causes the loop to immediately terminate and execution to continue at the line after the loop's code block.


The continue statement complements break, and works like this:
Daniel Robbins originally wrote <tt>keychain</tt> 1.0 through 2.0.3. 1.0 was written around June 2001, and 2.0.3 was released in late August, 2002.
<pre>
x=1
while (1) {
    if ( x == 4 ) {
        x++
        continue
    }
    print "iteration",x
    if ( x > 20 ) {
        break
    }
    x++
}
</pre>
This code will print "iteration 1" through "iteration 21", except for "iteration 4". If iteration equals 4, x is incremented and the continue statement is called, which immediately causes awk to start to the next loop iteration without executing the rest of the code block. The continue statement works for every kind of awk iterative loop, just as break does. When used in the body of a for loop, continue will cause the loop control variable to be automatically incremented. Here's an equivalent for loop:
<pre>
for ( x=1; x<=21; x++ ) {
    if ( x == 4 ) {
        continue
    }
    print "iteration",x
}
</pre>
It wasn't necessary to increment x just before calling continue as it was in our while loop, since the for loop increments x automatically.


=== Arrays ===
After 2.0.3, <tt>keychain</tt> was maintained by various Gentoo developers, including Seth Chandler, Mike Frysinger and Robin H. Johnson, through July 3, 2003.
You'll be pleased to know that awk has arrays. However, under awk, it's customary to start array indices at 1, rather than 0:
<pre>
myarray[1]="jim"
myarray[2]=456
</pre>
When awk encounters the first assignment, myarray is created and the element myarray[1] is set to "jim". After the second assignment is evaluated, the array has two elements.


Once defined, awk has a handy mechanism to iterate over the elements of an array, as follows:
On April 21, 2004, Aron Griffis committed a major rewrite of <tt>keychain</tt> which was released as 2.2.0. Aron continued to actively maintain and improve <tt>keychain</tt> through October 2006 and the <tt>keychain</tt> 2.6.8 release. He also made a few commits after that date, up through mid-July, 2007. At this point, <tt>keychain</tt> had reached a point of maturity.
<pre>
for ( x in myarray ) {
    print myarray[x]
}
</pre>
This code will print out every element in the array myarray. When you use this special "in" form of a for loop, awk will assign every existing index of myarray to x (the loop control variable) in turn, executing the loop's code block once after each assignment. While this is a very handy awk feature, it does have one drawback -- when awk cycles through the array indices, it doesn't follow any particular order. That means that there's no way for us to know whether the output of above code will be:
<pre>
jim
456
</pre>
or
<pre>
456
jim
</pre>
To loosely paraphrase Forrest Gump, iterating over the contents of an array is like a box of chocolates -- you never know what you're going to get. This has something to do with the "stringiness" of awk arrays, which we'll now take a look at.


=== Array index stringiness ===
In mid-July, 2009, Daniel Robbins migrated Aron's mercurial repository to git and set up a new project page on funtoo.org, and made a few bug fix commits to the git repo that had been collecting in [http://bugs.gentoo.org bugs.gentoo.org]. Daniel continues to maintain <tt>keychain</tt> and supporting documentation on funtoo.org, and plans to make regular maintenance releases of <tt>keychain</tt> as needed.
[[Awk by example, Part1 |In my previous article]], I showed you that awk actually stores numeric values in a string format. While awk performs the necessary conversions to make this work, it does open the door for some odd-looking code:
<pre>
a="1"
b="2"
c=a+b+3
</pre>
After this code executes, c is equal to 6. Since awk is "stringy", adding strings "1" and "2" is functionally no different than adding the numbers 1 and 2. In both cases, awk will successfully perform the math. Awk's "stringy" nature is pretty intriguing -- you may wonder what happens if we use string indexes for arrays. For instance, take the following code:
<pre>
myarr["1"]="Mr. Whipple"
print myarr["1"]
</pre>
As you might expect, this code will print "Mr. Whipple". But how about if we drop the quotes around the second "1" index?
<pre>
myarr["1"]="Mr. Whipple"
print myarr[1]
</pre>
Guessing the result of this code snippet is a bit more difficult. Does awk consider myarr["1"] and myarr[1] to be two separate elements of the array, or do they refer to the same element? The answer is that they refer to the same element, and awk will print "Mr. Whipple", just as in the first code snippet. Although it may seem strange, behind the scenes awk has been using string indexes for its arrays all this time!


After learning this strange fact, some of us may be tempted to execute some wacky code that looks like this:
== Quick Setup ==
<pre>
myarr["name"]="Mr. Whipple"
print myarr["name"]
</pre>
Not only does this code not raise an error, but it's functionally identical to our previous examples, and will print "Mr. Whipple" just as before! As you can see, awk doesn't limit us to using pure integer indexes; we can use string indexes if we want to, without creating any problems. Whenever we use non-integer array indices like myarr["name"], we're using associative arrays. Technically, awk isn't doing anything different behind the scenes than when we use a string index (since even if you use an "integer" index, awk still treats it as a string). However, you should still call 'em associative arrays -- it sounds cool and will impress your boss. The stringy index thing will be our little secret. ;)


=== Array tools ===
=== Linux ===
When it comes to arrays, awk gives us a lot of flexibility. We can use string indexes, and we aren't required to have a continuous numeric sequence of indices (for example, we can define myarr[1] and myarr[1000], but leave all other elements undefined). While all this can be very helpful, in some circumstances it can create confusion. Fortunately, awk offers a couple of handy features to help make arrays more manageable.


First, we can delete array elements. If you want to delete element 1 of your array fooarray, type:
To install under Gentoo or Funtoo Linux, type  
<pre>
<console>
delete fooarray[1]
###i## emerge keychain
</pre>
</console>
And, if you want to see if a particular array element exists, you can use the special "in" boolean operator as follows:
<pre>
if ( 1 in fooarray ) {
    print "Ayep!  It's there."
} else {
    print "Nope!  Can't find it."
}
</pre>


=== Next time ===
For other Linux distributions, use your distribution's package manager, or download and install using the source tarball above. Then generate RSA/DSA keys if necessary. The quick install docs assume you have a DSA key pair named <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> in your <tt>~/.ssh/</tt> directory. Add the following to your <tt>~/.bash_profile</tt>:
We've covered a lot of ground in this article. Next time, I'll round out your awk knowledge by showing you how to use awk's math and string functions and how to create your own functions. I'll also walk you through the creation of a checkbook balancing program. Until then, I encourage you to write some of your own awk programs, and to check out the following resources.


== Resources ==
{{file|name=~/.bash_profile|body=
* Read Daniel's other awk articles on Funtoo: Awk By Example, [[Awk by example, Part1|Part 1]] and [[Awk by example, Part3|Part 3]].
eval `keychain --eval --agents ssh id_rsa`
* If you'd like a good old-fashioned book, [http://www.oreilly.com/catalog/sed2/ O'Reilly's sed & awk, 2nd Edition] is a wonderful choice.
}}
* Be sure to check out the [http://www.faqs.org/faqs/computer-lang/awk/faq/ comp.lang.awk FAQ]. It also contains lots of additional awk links.
 
* Patrick Hartigan's [http://sparky.rice.edu/~hartigan/awk.html awk tutorial] is packed with handy awk scripts.
If you want to take advantage of GPG functionality, ensure that GNU Privacy Guard is installed and omit the <tt>--agents ssh</tt> option above.
* [http://www.tasoft.com/tawk.html Thompson's TAWK Compiler] compiles awk scripts into fast binary executables. Versions are available for Windows, OS/2, DOS, and UNIX.
 
* [http://www.gnu.org/software/gawk/manual/gawk.html The GNU Awk User's Guide] is available for online reference.
=== Apple MacOS X ===
 
To install under MacOS X, install the MacOS X package for keychain. Assuming you have an <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> key pair in your <tt>~/.ssh/</tt> directory, add the following to your <tt>~/.bash_profile</tt>:
 
{{file|name=~/.bash_profile|body=
eval `keychain --eval --agents ssh --inherit any id_dsa`
}}
 
{{Fancynote|The <tt>--inherit any</tt> option above causes keychain to inherit any ssh key passphrases stored in your Apple MacOS Keychain. If you would prefer for this to not happen, then this option can be omitted.}}
 
== Background ==
 
You're probably familiar with <tt>ssh</tt>, which has become a secure replacement for the venerable <tt>telnet</tt> and <tt>rsh</tt> commands.
 
Typically, when one uses <tt>ssh</tt> to connect to a remote system, one supplies a secret passphrase to <tt>ssh</tt>, which is then passed in encrypted form over the network to the remote server. This passphrase is used by the remote <tt>sshd</tt> server to determine if you should be granted access to the system.
 
However, OpenSSH and nearly all other SSH clients and servers have the ability to perform another type of authentication, called asymmetric public key authentication, using the RSA or DSA authentication algorithms. They are very useful, but can also be complicated to use. <tt>keychain</tt> has been designed to make it easy to take advantage of the benefits of RSA and DSA authentication.
 
== Generating a Key Pair ==
 
To use RSA and DSA authentication, first you use a program called <tt>ssh-keygen</tt> (included with OpenSSH) to generate a ''key pair'' -- two small files. One of the files is the ''public key''. The other small file contains the ''private key''. <tt>ssh-keygen</tt> will ask you for a passphrase, and this passphrase will be used to encrypt your private key. You will need to supply this passphrase to use your private key. If you wanted to generate a DSA key pair, you would do this:
 
<console># ##i##ssh-keygen -t dsa
Generating public/private dsa key pair.</console>
You would then be prompted for a location to store your key pair. If you do not have one currently stored in <tt>~/.ssh</tt>, it is fine to accept the default location:
 
<console>Enter file in which to save the key (/root/.ssh/id_dsa): </console>
Then, you are prompted for a passphrase. This passphrase is used to encrypt the ''private key'' on disk, so even if it is stolen, it will be difficult for someone else to use it to successfully authenticate as you with any accounts that have been configured to recognize your public key.
 
Note that conversely, if you '''do not''' provide a passphrase for your private key file, then your private key file '''will not''' be encrypted. This means that if someone steals your private key file, ''they will have the full ability to authenticate with any remote accounts that are set up with your public key.''
 
Below, I have supplied a passphrase so that my private key file will be encrypted on disk:
 
<console>Enter passphrase (empty for no passphrase): ##i#########
Enter same passphrase again: ##i#########
Your identification has been saved in /var/tmp/id_dsa.
Your public key has been saved in /var/tmp/id_dsa.pub.
The key fingerprint is:
5c:13:ff:46:7d:b3:bf:0e:37:1e:5e:8c:7b:a3:88:f4 root@devbox-ve
The key's randomart image is:
+--[ DSA 1024]----+
|          .      |
|          o  . |
|          o . ..o|
|      . . . o  +|
|        S    o. |
|            . o.|
|        .  ..++|
|       . o . =o*|
|        . E .+*.|
+-----------------+</console>
 
== Setting up Authentication ==
 
Here's how you use these files to authenticate with a remote server. On the remote server, you would append the contents of your ''public key'' to the <tt>~.ssh/authorized_keys</tt> file, if such a file exists. If it doesn't exist, you can simply create a new <tt>authorized_keys</tt> file in the remote account's <tt>~/.ssh</tt> directory that contains the contents of your local <tt>id_dsa.pub</tt> file.
 
Then, if you weren't going to use <tt>keychain</tt>, you'd perform the following steps. On your local client, you would start a program called <tt>ssh-agent</tt>, which runs in the background. Then you would use a program called <tt>ssh-add</tt> to tell <tt>ssh-agent</tt> about your secret private key. Then, if you've set up your environment properly, the next time you run <tt>ssh</tt>, it will find <tt>ssh-agent</tt> running, grab the private key that you added to <tt>ssh-agent</tt> using <tt>ssh-add</tt>, and use this key to authenticate with the remote server.
 
Again, the steps in the previous paragraph is what you'd do if <tt>keychain</tt> wasn't around to help. If you are using <tt>keychain</tt>, and I hope you are, you would simply add the following line to your <tt>~/.bash_profile</tt> or if a regular user to<tt>~/.bashrc</tt> :
 
{{file|name=~/.bash_profile|body=
eval `keychain --eval id_dsa`
}}
 
The next time you log in or source your <tt>~/.bash_profile</tt> or if you use <tt>~/.bashrc</tt>, <tt>keychain</tt> will start, start <tt>ssh-agent</tt> for you if it has not yet been started, use <tt>ssh-add</tt> to add your <tt>id_dsa</tt> private key file to <tt>ssh-agent</tt>, and set up your shell environment so that <tt>ssh</tt> will be able to find <tt>ssh-agent</tt>. If <tt>ssh-agent</tt> is already running, <tt>keychain</tt> will ensure that your <tt>id_dsa</tt> private key has been added to <tt>ssh-agent</tt> and then set up your environment so that <tt>ssh</tt> can find the already-running <tt>ssh-agent</tt>. It will look something like this:
 
Note that when <tt>keychain</tt> runs for the first time after your local system has booted, you will be prompted for a passphrase for your private key file if it is encrypted. But here's the nice thing about using <tt>keychain</tt> -- even if you are using an encrypted private key file, you will only need to enter your passphrase when your system first boots (or in the case of a server, when you first log in.) After that, <tt>ssh-agent</tt> is already running and has your decrypted private key cached in memory. So if you open a new shell, you will see something like this:
 
This means that you can now <tt>ssh</tt> to your heart's content, without supplying a passphrase.
 
You can also execute batch <tt>cron</tt> jobs and scripts that need to use <tt>ssh</tt> or <tt>scp</tt>, and they can take advantage of passwordless RSA/DSA authentication as well. To do this, you would add the following line to the top of a bash script:
 
{{file|name=example-script.sh|body=
eval `keychain --noask --eval id_dsa` || exit 1
}}
 
The extra <tt>--noask</tt> option tells <tt>keychain</tt> that it should not prompt for a passphrase if one is needed. Since it is not running interactively, it is better for the script to fail if the decrypted private key isn't cached in memory via <tt>ssh-agent</tt>.
 
== Keychain Options ==
 
=== Specifying Agents ===
 
In the images above, you will note that <tt>keychain</tt> starts <tt>ssh-agent</tt>, but also starts <tt>gpg-agent</tt>. Modern versions of <tt>keychain</tt> also support caching decrypted GPG keys via use of <tt>gpg-agent</tt>, and will start <tt>gpg-agent</tt> by default if it is available on your system. To avoid this behavior and only start <tt>ssh-agent</tt>, modify your <tt>~/.bash_profile</tt> as follows:
 
{{file|name=~/.bash_profile|body=
eval `keychain --agents ssh --eval id_dsa` || exit 1
}}
 
The additional <tt>--agents ssh</tt> option tells <tt>keychain</tt> just to manage <tt>ssh-agent</tt>, and ignore <tt>gpg-agent</tt> even if it is available.
 
=== Clearing Keys ===
 
Sometimes, it might be necessary to flush all cached keys in memory. To do this, type:
 
<console># ##i##keychain --clear</console>
Any agent(s) will continue to run.
 
=== Improving Security ===
 
To improve the security of <tt>keychain</tt>, some people add the <tt>--clear</tt> option to their <tt>~/.bash_profile</tt> <tt>keychain</tt> invocation. The rationale behind this is that any user logging in should be assumed to be an intruder until proven otherwise. This means that you will need to re-enter any passphrases when you log in, but cron jobs will still be able to run when you log out.
 
=== Stopping Agents ===
 
If you want to stop all agents, which will also of course cause your keys/identities to be flushed from memory, you can do this as follows:
 
<console># ##i##keychain -k all</console>
If you have other agents running under your user account, you can also tell <tt>keychain</tt> to just stop only the agents that <tt>keychain</tt> started:
 
<console># ##i##keychain -k mine</console>
 
=== GPG ===
 
Keychain can ask you for your GPG passphrase if you provide it the GPG key ID. To find it out:
<console>
$##i## gpg -k
pub  2048R/DEADBEEF 2012-08-16
uid                  Name (Comment) <email@host.tld>
sub  2048R/86D2FAC6 2012-08-16
</console>
 
Note the '''DEADBEEF''' above is the ID. Then, in your login script, do your usual
 
<console>
$##i## keychain --dir ~/.ssh/.keychain ~/.ssh/id_rsa DEADBEEF
$##i## source ~/.ssh/.keychain/$HOST-sh
$##i## source ~/.ssh/.keychain/$HOST-sh-gpg
</console>
 
=== Learning More ===
 
The instructions above will work on any system that uses <tt>bash</tt> as its default shell, such as most Linux systems and Mac OS X.
 
To learn more about the many things that <tt>keychain</tt> can do, including alternate shell support, consult the keychain man page, or type <tt>keychain --help | less</tt> for a full list of command options.
 
I also recommend you read my original series of articles about [http://www.openssh.com OpenSSH] that I wrote for IBM developerWorks, called <tt>OpenSSH Key Management</tt>. Please note that <tt>keychain</tt> 1.0 was released along with Part 2 of this article, which was written in 2001. <tt>keychain</tt> has changed quite a bit since then. In other words, read these articles for the conceptual and [http://www.openssh.com OpenSSH] information, but consult the <tt>keychain</tt> man page for command-line options and usage instructions :)
 
* [http://www.ibm.com/developerworks/library/l-keyc.html Common Threads: OpenSSH key management, Part 1] - Understanding RSA/DSA Authentication
* [http://www.ibm.com/developerworks/library/l-keyc2/ Common Threads: OpenSSH key management, Part 2] - Introducing <tt>ssh-agent</tt> and <tt>keychain</tt>
* [http://www.ibm.com/developerworks/library/l-keyc3/ Common Threads: OpenSSH key management, Part 3] - Agent forwarding and <tt>keychain</tt> improvements
 
As mentioned at the top of the page, <tt>keychain</tt> development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [http://groups.google.com/group/funtoo-dev funtoo-dev mailing list] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.


[[Category:Linux Core Concepts]]
[[Category:HOWTO]]
[[Category:Projects]]
[[Category:First Steps]]
[[Category:Articles]]
[[Category:Articles]]
{{ArticleFooter}}
{{ArticleFooter}}

Revision as of 16:22, January 5, 2015

Official Project Page

Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. Download and learn how to use Keychain on your Linux, Unix or MacOS system.
   Support Funtoo!
Get an awesome Funtoo container and support Funtoo! See Funtoo Containers for more information.

Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. It acts as a frontend to ssh-agent and ssh-add, but allows you to easily have one long running ssh-agent process per system, rather than the norm of one ssh-agent per login session.

This dramatically reduces the number of times you need to enter your passphrase. With keychain, you only need to enter a passphrase once every time your local machine is rebooted. Keychain also makes it easy for remote cron jobs to securely "hook in" to a long-running ssh-agent process, allowing your scripts to take advantage of key-based logins.

Those who are new to OpenSSH and the use of public/private keys for authentication may want to check out the following articles by Daniel Robbins, which will provide a gentle introduction to the concepts used by Keychain:

Download and Resources

The latest release of keychain is version 2.7.2_beta1, and was released on July 7, 2014. The current version of keychain supports gpg-agent as well as ssh-agent.

Keychain is compatible with many operating systems, including AIX, *BSD, Cygwin, MacOS X, Linux, HP/UX, Tru64 UNIX, IRIX, Solaris and GNU Hurd.

Download

Keychain development sources can be found in the keychain git repository. Please use the Funtoo Linux bug tracker and #funtoo irc channel for keychain support questions as well as bug reports.

Project History

Daniel Robbins originally wrote keychain 1.0 through 2.0.3. 1.0 was written around June 2001, and 2.0.3 was released in late August, 2002.

After 2.0.3, keychain was maintained by various Gentoo developers, including Seth Chandler, Mike Frysinger and Robin H. Johnson, through July 3, 2003.

On April 21, 2004, Aron Griffis committed a major rewrite of keychain which was released as 2.2.0. Aron continued to actively maintain and improve keychain through October 2006 and the keychain 2.6.8 release. He also made a few commits after that date, up through mid-July, 2007. At this point, keychain had reached a point of maturity.

In mid-July, 2009, Daniel Robbins migrated Aron's mercurial repository to git and set up a new project page on funtoo.org, and made a few bug fix commits to the git repo that had been collecting in bugs.gentoo.org. Daniel continues to maintain keychain and supporting documentation on funtoo.org, and plans to make regular maintenance releases of keychain as needed.

Quick Setup

Linux

To install under Gentoo or Funtoo Linux, type 

root # emerge keychain

For other Linux distributions, use your distribution's package manager, or download and install using the source tarball above. Then generate RSA/DSA keys if necessary. The quick install docs assume you have a DSA key pair named id_dsa and id_dsa.pub in your ~/.ssh/ directory. Add the following to your ~/.bash_profile:

   ~/.bash_profile 
eval `keychain --eval --agents ssh id_rsa`

If you want to take advantage of GPG functionality, ensure that GNU Privacy Guard is installed and omit the --agents ssh option above.

Apple MacOS X

To install under MacOS X, install the MacOS X package for keychain. Assuming you have an id_dsa and id_dsa.pub key pair in your ~/.ssh/ directory, add the following to your ~/.bash_profile:

   ~/.bash_profile 
eval `keychain --eval --agents ssh --inherit any id_dsa`
   Note

The --inherit any option above causes keychain to inherit any ssh key passphrases stored in your Apple MacOS Keychain. If you would prefer for this to not happen, then this option can be omitted.

Background

You're probably familiar with ssh, which has become a secure replacement for the venerable telnet and rsh commands.

Typically, when one uses ssh to connect to a remote system, one supplies a secret passphrase to ssh, which is then passed in encrypted form over the network to the remote server. This passphrase is used by the remote sshd server to determine if you should be granted access to the system.

However, OpenSSH and nearly all other SSH clients and servers have the ability to perform another type of authentication, called asymmetric public key authentication, using the RSA or DSA authentication algorithms. They are very useful, but can also be complicated to use. keychain has been designed to make it easy to take advantage of the benefits of RSA and DSA authentication.

Generating a Key Pair

To use RSA and DSA authentication, first you use a program called ssh-keygen (included with OpenSSH) to generate a key pair -- two small files. One of the files is the public key. The other small file contains the private key. ssh-keygen will ask you for a passphrase, and this passphrase will be used to encrypt your private key. You will need to supply this passphrase to use your private key. If you wanted to generate a DSA key pair, you would do this: 

root # ssh-keygen -t dsa
Generating public/private dsa key pair.

You would then be prompted for a location to store your key pair. If you do not have one currently stored in ~/.ssh, it is fine to accept the default location: 

Enter file in which to save the key (/root/.ssh/id_dsa):

Then, you are prompted for a passphrase. This passphrase is used to encrypt the private key on disk, so even if it is stolen, it will be difficult for someone else to use it to successfully authenticate as you with any accounts that have been configured to recognize your public key.

Note that conversely, if you do not provide a passphrase for your private key file, then your private key file will not be encrypted. This means that if someone steals your private key file, they will have the full ability to authenticate with any remote accounts that are set up with your public key.

Below, I have supplied a passphrase so that my private key file will be encrypted on disk: 

Enter passphrase (empty for no passphrase): #######
Enter same passphrase again: #######
Your identification has been saved in /var/tmp/id_dsa.
Your public key has been saved in /var/tmp/id_dsa.pub.
The key fingerprint is:
5c:13:ff:46:7d:b3:bf:0e:37:1e:5e:8c:7b:a3:88:f4 root@devbox-ve
The key's randomart image is:
+--[ DSA 1024]----+
|          .      |
|           o   . |
|          o . ..o|
|       . . . o  +|
|        S     o. |
|             . o.|
|         .   ..++|
|        . o . =o*|
|         . E .+*.|
+-----------------+

Setting up Authentication

Here's how you use these files to authenticate with a remote server. On the remote server, you would append the contents of your public key to the ~.ssh/authorized_keys file, if such a file exists. If it doesn't exist, you can simply create a new authorized_keys file in the remote account's ~/.ssh directory that contains the contents of your local id_dsa.pub file.

Then, if you weren't going to use keychain, you'd perform the following steps. On your local client, you would start a program called ssh-agent, which runs in the background. Then you would use a program called ssh-add to tell ssh-agent about your secret private key. Then, if you've set up your environment properly, the next time you run ssh, it will find ssh-agent running, grab the private key that you added to ssh-agent using ssh-add, and use this key to authenticate with the remote server.

Again, the steps in the previous paragraph is what you'd do if keychain wasn't around to help. If you are using keychain, and I hope you are, you would simply add the following line to your ~/.bash_profile or if a regular user to~/.bashrc :

   ~/.bash_profile 
eval `keychain --eval id_dsa`

The next time you log in or source your ~/.bash_profile or if you use ~/.bashrc, keychain will start, start ssh-agent for you if it has not yet been started, use ssh-add to add your id_dsa private key file to ssh-agent, and set up your shell environment so that ssh will be able to find ssh-agent. If ssh-agent is already running, keychain will ensure that your id_dsa private key has been added to ssh-agent and then set up your environment so that ssh can find the already-running ssh-agent. It will look something like this:

Note that when keychain runs for the first time after your local system has booted, you will be prompted for a passphrase for your private key file if it is encrypted. But here's the nice thing about using keychain -- even if you are using an encrypted private key file, you will only need to enter your passphrase when your system first boots (or in the case of a server, when you first log in.) After that, ssh-agent is already running and has your decrypted private key cached in memory. So if you open a new shell, you will see something like this:

This means that you can now ssh to your heart's content, without supplying a passphrase.

You can also execute batch cron jobs and scripts that need to use ssh or scp, and they can take advantage of passwordless RSA/DSA authentication as well. To do this, you would add the following line to the top of a bash script:

   example-script.sh 
eval `keychain --noask --eval id_dsa`

The extra --noask option tells keychain that it should not prompt for a passphrase if one is needed. Since it is not running interactively, it is better for the script to fail if the decrypted private key isn't cached in memory via ssh-agent.

Keychain Options

Specifying Agents

In the images above, you will note that keychain starts ssh-agent, but also starts gpg-agent. Modern versions of keychain also support caching decrypted GPG keys via use of gpg-agent, and will start gpg-agent by default if it is available on your system. To avoid this behavior and only start ssh-agent, modify your ~/.bash_profile as follows:

   ~/.bash_profile 
eval `keychain --agents ssh --eval id_dsa`

The additional --agents ssh option tells keychain just to manage ssh-agent, and ignore gpg-agent even if it is available.

Clearing Keys

Sometimes, it might be necessary to flush all cached keys in memory. To do this, type: 

root # keychain --clear

Any agent(s) will continue to run.

Improving Security

To improve the security of keychain, some people add the --clear option to their ~/.bash_profile keychain invocation. The rationale behind this is that any user logging in should be assumed to be an intruder until proven otherwise. This means that you will need to re-enter any passphrases when you log in, but cron jobs will still be able to run when you log out.

Stopping Agents

If you want to stop all agents, which will also of course cause your keys/identities to be flushed from memory, you can do this as follows: 

root # keychain -k all

If you have other agents running under your user account, you can also tell keychain to just stop only the agents that keychain started: 

root # keychain -k mine

GPG

Keychain can ask you for your GPG passphrase if you provide it the GPG key ID. To find it out: 

user $ gpg -k
pub   2048R/DEADBEEF 2012-08-16
uid                  Name (Comment) <email@host.tld>
sub   2048R/86D2FAC6 2012-08-16

Note the DEADBEEF above is the ID. Then, in your login script, do your usual 

user $ keychain --dir ~/.ssh/.keychain ~/.ssh/id_rsa DEADBEEF
user $ source ~/.ssh/.keychain/$HOST-sh
user $ source ~/.ssh/.keychain/$HOST-sh-gpg

Learning More

The instructions above will work on any system that uses bash as its default shell, such as most Linux systems and Mac OS X.

To learn more about the many things that keychain can do, including alternate shell support, consult the keychain man page, or type keychain --help | less for a full list of command options.

I also recommend you read my original series of articles about OpenSSH that I wrote for IBM developerWorks, called OpenSSH Key Management. Please note that keychain 1.0 was released along with Part 2 of this article, which was written in 2001. keychain has changed quite a bit since then. In other words, read these articles for the conceptual and OpenSSH information, but consult the keychain man page for command-line options and usage instructions :)

As mentioned at the top of the page, keychain development sources can be found in the keychain git repository. Please use the funtoo-dev mailing list and #funtoo irc channel for keychain support questions as well as bug reports.


   Note

Browse all our available articles below. Use the search field to search for topics and keywords in real-time.

Article Subtitle
Article Subtitle
Awk by Example, Part 1 An intro to the great language with the strange name
Awk by Example, Part 2 Records, loops, and arrays
Awk by Example, Part 3 String functions and ... checkbooks?
Bash by Example, Part 1 Fundamental programming in the Bourne again shell (bash)
Bash by Example, Part 2 More bash programming fundamentals
Bash by Example, Part 3 Exploring the ebuild system
BTRFS Fun
Funtoo Filesystem Guide, Part 1 Journaling and ReiserFS
Funtoo Filesystem Guide, Part 2 Using ReiserFS and Linux
Funtoo Filesystem Guide, Part 3 Tmpfs and Bind Mounts
Funtoo Filesystem Guide, Part 4 Introducing Ext3
Funtoo Filesystem Guide, Part 5 Ext3 in Action
GUID Booting Guide
Learning Linux LVM, Part 1 Storage management magic with Logical Volume Management
Learning Linux LVM, Part 2 The cvs.gentoo.org upgrade
Libvirt
Linux Fundamentals, Part 1
Linux Fundamentals, Part 2
Linux Fundamentals, Part 3
Linux Fundamentals, Part 4
LVM Fun
Making the Distribution, Part 1
Making the Distribution, Part 2
Making the Distribution, Part 3
Maximum Swappage Getting the most out of swap
On screen annotation Write on top of apps on your screen
OpenSSH Key Management, Part 1 Understanding RSA/DSA Authentication
OpenSSH Key Management, Part 2 Introducing ssh-agent and keychain
OpenSSH Key Management, Part 3 Agent Forwarding
Partition Planning Tips Keeping things organized on disk
Partitioning in Action, Part 1 Moving /home
Partitioning in Action, Part 2 Consolidating data
POSIX Threads Explained, Part 1 A simple and nimble tool for memory sharing
POSIX Threads Explained, Part 2
POSIX Threads Explained, Part 3 Improve efficiency with condition variables
Sed by Example, Part 1
Sed by Example, Part 2
Sed by Example, Part 3
Successful booting with UUID Guide to use UUID for consistent booting.
The Gentoo.org Redesign, Part 1 A site reborn
The Gentoo.org Redesign, Part 2 The Documentation System
The Gentoo.org Redesign, Part 3 The New Main Pages
The Gentoo.org Redesign, Part 4 The Final Touch of XML
Traffic Control
Windows 10 Virtualization with KVM
Retrieved from "https://www.funtoo.org/index.php?title=Awk_by_Example,_Part_2&oldid=8324"