Difference between pages "Awk by Example, Part 3" and "Funtoo:Keychain"

From Funtoo
(Difference between pages)
Jump to navigation Jump to search
 
 
Line 1: Line 1:
{{Article
{{Article
|Subtitle=String functions and ... checkbooks?
|Subtitle=Official Project Page
|Keywords=command,unix,variables,print,space,gsub
|Summary=Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. Download and learn how to use Keychain on your Linux, Unix or MacOS system.
|Keywords=keychain,ssh,rsa,dsa,gpg,linux,gentoo,macos,download,source code
|Author=Drobbins
|Author=Drobbins
|Previous in Series=Awk by Example, Part 2
}}
}}
=== Formatting output ===
<tt>Keychain</tt> helps you to manage SSH and GPG keys in a convenient and secure manner. It acts as a frontend to <tt>ssh-agent</tt> and <tt>ssh-add</tt>, but allows you to easily have one long running <tt>ssh-agent</tt> process per system, rather than the norm of one <tt>ssh-agent</tt> per login session.
While awk's print statement does do the job most of the time, sometimes more is needed. For those times, awk offers two good old friends called printf() and sprintf(). Yes, these functions, like so many other awk parts, are identical to their C counterparts. printf() will print a formatted string to stdout, while sprintf() returns a formatted string that can be assigned to a variable. If you're not familiar with printf() and sprintf(), an introductory C text will quickly get you up to speed on these two essential printing functions. You can view the printf() man page by typing "man 3 printf" on your Linux system.


Here's some sample awk sprintf() and printf() code. As you can see in the following script, everything looks almost identical to C.
This dramatically reduces the number of times you need to enter your passphrase. With <tt>keychain</tt>, you only need to enter a passphrase once every time your local machine is rebooted. <tt>Keychain</tt> also makes it easy for remote cron jobs to securely "hook in" to a long-running <tt>ssh-agent</tt> process, allowing your scripts to take advantage of key-based logins.
<pre>
#!/usr/bin/awk -f
BEGIN {
x=1
b="foo"
printf("%s got a %d on the last test\n","Jim",83)
myout=sprintf("%s-%d",b,x)
print myout
}
</pre>
This code will print:
<pre>
Jim got a 83 on the last test
foo-1
</pre>


=== String functions ===
Those who are new to OpenSSH and the use of public/private keys for authentication may want to check out the following articles by Daniel Robbins, which will provide a gentle introduction to the concepts used by Keychain:
Awk has a plethora of string functions, and that's a good thing. In awk, you really need string functions, since you can't treat a string as an array of characters as you can in other languages like C, C++, and Python. For example, if you execute the following code:
* [[OpenSSH Key Management, Part_1]]
<pre>
* [[OpenSSH Key Management, Part_2]]
mystring="How are you doing today?"
* [[OpenSSH Key Management, Part_3]]
print mystring[3]
</pre>
You'll receive an error that looks something like this:
<pre>
awk: string.gawk:59: fatal: attempt to use scalar as array
</pre>
Oh, well. While not as convenient as Python's sequence types, awk's string functions get the job done. Let's take a look at them.


First, we have the basic length() function, which returns the length of a string. Here's how to use it:
== Download and Resources ==
<pre>
print length(mystring)
</pre>
This code will print the value:
<pre>
24
</pre>
OK, let's keep going. The next string function is called index, and will return the position of the occurrence of a substring in another string, or it will return 0 if the string isn't found. Using mystring, we can call it this way:
<pre>
print index(mystring,"you")
</pre>
Awk prints:
<pre>
9
</pre>
We move on to two more easy functions, tolower() and toupper(). As you might guess, these functions will return the string with all characters converted to lowercase or uppercase respectively. Notice that tolower() and toupper() return the new string, and don't modify the original. This code:
<pre>
print tolower(mystring)
print toupper(mystring)
print mystring
</pre>
....will produce this output:
<pre>
how are you doing today?
HOW ARE YOU DOING TODAY?
How are you doing today?
</pre>
So far so good, but how exactly do we select a substring or even a single character from a string? That's where substr() comes in. Here's how to call substr():
<pre>
mysub=substr(mystring,startpos,maxlen)
</pre>
mystring should be either a string variable or a literal string from which you'd like to extract a substring. startpos should be set to the starting character position, and maxlen should contain the maximum length of the string you'd like to extract. Notice that I said maximum length; if length(mystring) is shorter than startpos+maxlen, your result will be truncated. substr() won't modify the original string, but returns the substring instead. Here's an example:
<pre>
print substr(mystring,9,3)
</pre>
Awk will print:
<pre>
you
</pre>
If you regularly program in a language that uses array indices to access parts of a string (and who doesn't), make a mental note that substr() is your awk substitute. You'll need to use it to extract single characters and substrings; because awk is a string-based language, you'll be using it often.


Now, we move on to some meatier functions, the first of which is called match(). match() is a lot like index(), except instead of searching for a substring like index() does, it searches for a regular expression. The match() function will return the starting position of the match, or zero if no match is found. In addition, match() will set two variables called RSTART and RLENGTH. RSTART contains the return value (the location of the first match), and RLENGTH specifies its span in characters (or -1 if no match was found). Using RSTART, RLENGTH, substr(), and a small loop, you can easily iterate through every match in your string. Here's an example match() call:
The latest release of keychain is version <tt>2.7.2_beta1</tt>, and was released on July 7, 2014. The current version of keychain supports <tt>gpg-agent</tt> as well as <tt>ssh-agent</tt>.
<pre>
print match(mystring,/you/), RSTART, RLENGTH
</pre>
Awk will print:
<pre>
9 9 3
</pre>


=== String substitution ===
Keychain is compatible with many operating systems, including <tt>AIX</tt>, <tt>*BSD</tt>, <tt>Cygwin</tt>, <tt>MacOS X</tt>, <tt>Linux</tt>, <tt>HP/UX</tt>, <tt>Tru64 UNIX</tt>, <tt>IRIX</tt>, <tt>Solaris</tt> and <tt>GNU Hurd</tt>.
Now, we're going to look at a couple of string substitution functions, sub() and gsub(). These guys differ slightly from the functions we've looked at so far in that they actually modify the original string. Here's a template that shows how to call sub():
<pre>
sub(regexp,replstring,mystring)
</pre>
When you call sub(), it'll find the first sequence of characters in mystring that matches regexp, and it'll replace that sequence with replstring. sub() and gsub() have identical arguments; the only way they differ is that sub() will replace the first regexp match (if any), and gsub() will perform a global replace, swapping out all matches in the string. Here's an example sub() and gsub() call:
<pre>
sub(/o/,"O",mystring)
print mystring
mystring="How are you doing today?"
gsub(/o/,"O",mystring)
print mystring
</pre>
We had to reset mystring to its original value because the first sub() call modified mystring directly. When executed, this code will cause awk to output:
<pre>
HOw are you doing today?
HOw are yOu dOing tOday?
</pre>
Of course, more complex regular expressions are possible. I'll leave it up to you to test out some complicated regexps.


We wrap up our string function coverage by introducing you to a function called split(). split()'s job is to "chop up" a string and place the various parts into an integer-indexed array. Here's an example split() call:
=== Download ===
<pre>
numelements=split("Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec",mymonths,",")
</pre>
When calling split(), the first argument contains the literal string or string variable to be chopped. In the second argument, you should specify the name of the array that split() will stuff the chopped parts into. In the third element, specify the separator that will be used to chop the strings up. When split() returns, it'll return the number of string elements that were split. split() assigns each one to an array index starting with one, so the following code:
<pre>
print mymonths[1],mymonths[numelements]
</pre>
....will print:
<pre>
Jan Dec
</pre>


=== Special string forms ===
* ''Release Archive''
A quick note -- when calling length(), sub(), or gsub(), you can drop the last argument and awk will apply the function call to $0 (the entire current line). To print the length of each line in a file, use this awk script:
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.2_beta1.tar.bz2 keychain 2.7.2_beta1]
<pre>
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1.tar.bz2 keychain 2.7.1]
{
    print length()
}
</pre>


=== Financial fun ===
* ''Apple MacOS X Packages''
A few weeks ago, I decided to write my own checkbook balancing program in awk. I decided that I'd like to have a simple tab-delimited text file into which I can enter my most recent deposits and withdrawals. The idea was to hand this data to an awk script that would automatically add up all the amounts and tell me my balance. Here's how I decided to record all my transactions into my "ASCII checkbook":
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1-macosx.tar.gz keychain 2.7.1 MacOS X package]
<pre>
23 Aug 2000    food    -    -    Y    Jimmy's Buffet    30.25
</pre>
Every field in this file is separated by one or more tabs. After the date (field 1, $1), there are two fields called "expense category" and "income category". When I'm entering an expense like on the above line, I put a four-letter nickname in the exp field, and a "-" (blank entry) in the inc field. This signifies that this particular item is a "food expense" :) Here's what a deposit looks like:
<pre>
23 Aug 2000    -    inco    -    Y    Boss Man        2001.00
</pre>
In this case, I put a "-" (blank) in the exp category, and put "inco" in the inc category. "inco" is my nickname for generic (paycheck-style) income. Using category nicknames allows me to generate a breakdown of my income and expenditures by category. As far as the rest of the records, all the other fields are fairly self-explanatory. The cleared? field ("Y" or "N") records whether the transaction has been posted to my account; beyond that, there's a transaction description, and a positive dollar amount.


The algorithm used to compute the current balance isn't too hard. Awk simply needs to read in each line, one by one. If an expense category is listed but there is no income category (denoted by "-"), then this item is a debit. If an income category is listed, but no expense category (denoted by "-") is present, then the dollar amount is a credit. And, if there is both an expense and income category listed, then this amount is a "category transfer"; that is, the dollar amount will be subtracted from the expense category and added to the income category. Again, all these categories are virtual, but are very useful for tracking income and expenditures, as well as for budgeting.
Keychain development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [https://bugs.funtoo.org Funtoo Linux bug tracker] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.


=== The code ===
=== Project History ===
Time to look at the code. We'll start off with the first line, the BEGIN block and a function definition:
<pre>
#!/usr/bin/awk -f
BEGIN {
    FS="\t+"
    months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
}


function monthdigit(mymonth) {
Daniel Robbins originally wrote <tt>keychain</tt> 1.0 through 2.0.3. 1.0 was written around June 2001, and 2.0.3 was released in late August, 2002.
    return (index(months,mymonth)+3)/4
}
</pre>
Adding the first "#!..." line to any awk script will allow it to be directly executed from the shell, provided that you "chmod +x myscript" first. The remaining lines define our BEGIN block, which gets executed before awk starts processing our checkbook file. We set FS (the field separator) to "\t+", which tells awk that the fields will be separated by one or more tabs. In addition, we define a string called months that's used by our monthdigit() function, which appears next.


The last three lines show you how to define your own awk function. The format is simple -- type "function", then the function name, and then the parameters separated by commas, inside parentheses. After this, a "{ }" code block contains the code that you'd like this function to execute. All functions can access global variables (like our months variable). In addition, awk provides a "return" statement that allows the function to return a value, and operates similarly to the "return" found in C, Python, and other languages. This particular function converts a month name in a 3-letter string format into its numeric equivalent. For example, this:
After 2.0.3, <tt>keychain</tt> was maintained by various Gentoo developers, including Seth Chandler, Mike Frysinger and Robin H. Johnson, through July 3, 2003.
<pre>
print monthdigit("Mar")
</pre>
....will print this:
<pre>
3
</pre>
Now, let's move on to some more functions.


=== Financial functions ===
On April 21, 2004, Aron Griffis committed a major rewrite of <tt>keychain</tt> which was released as 2.2.0. Aron continued to actively maintain and improve <tt>keychain</tt> through October 2006 and the <tt>keychain</tt> 2.6.8 release. He also made a few commits after that date, up through mid-July, 2007. At this point, <tt>keychain</tt> had reached a point of maturity.
Here are three more functions that perform the bookkeeping for us. Our main code block, which we'll see soon, will process each line of the checkbook file sequentially, calling one of these functions so that the appropriate transactions are recorded in an awk array. There are three basic kinds of transactions, credit (doincome), debit (doexpense) and transfer (dotransfer). You'll notice that all three functions accept one argument, called mybalance. mybalance is a placeholder for a two-dimensional array, which we'll pass in as an argument. Up until now, we haven't dealt with two-dimensional arrays; however, as you can see below, the syntax is quite simple. Just separate each dimension with a comma, and you're in business.


We'll record information into "mybalance" as follows. The first dimension of the array ranges from 0 to 12, and specifies the month, or zero for the entire year. Our second dimension is a four-letter category, like "food" or "inco"; this is the actual category we're dealing with. So, to find the entire year's balance for the food category, you'd look in mybalance[0,"food"]. To find June's income, you'd look in mybalance[6,"inco"].
In mid-July, 2009, Daniel Robbins migrated Aron's mercurial repository to git and set up a new project page on funtoo.org, and made a few bug fix commits to the git repo that had been collecting in [http://bugs.gentoo.org bugs.gentoo.org]. Daniel continues to maintain <tt>keychain</tt> and supporting documentation on funtoo.org, and plans to make regular maintenance releases of <tt>keychain</tt> as needed.
<pre>      
function doincome(mybalance) {
    mybalance[curmonth,$3] += amount
    mybalance[0,$3] += amount       
}


function doexpense(mybalance) {
== Quick Setup ==
    mybalance[curmonth,$2] -= amount
    mybalance[0,$2] -= amount       
}


function dotransfer(mybalance) {
=== Linux ===
    mybalance[0,$2] -= amount
    mybalance[curmonth,$2] -= amount
    mybalance[0,$3] += amount
    mybalance[curmonth,$3] += amount
}
</pre>
When doincome() or any of the other functions are called, we record the transaction in two places -- mybalance[0,category] and mybalance[curmonth, category], the entire year's category balance and the current month's category balance, respectively. This allows us to easily generate either an annual or monthly breakdown of income/expenditures later on.


If you look at these functions, you'll notice that the array referenced by mybalance is passed in by reference. In addition, we also refer to several global variables: curmonth, which holds the numeric value of the month of the current record, $2 (the expense category), $3 (the income category), and amount ($7, the dollar amount). When doincome() and friends are called, all these variables have already been set correctly for the current record (line) being processed.
To install under Gentoo or Funtoo Linux, type
<console>
###i## emerge keychain
</console>


=== The main block ===
For other Linux distributions, use your distribution's package manager, or download and install using the source tarball above. Then generate RSA/DSA keys if necessary. The quick install docs assume you have a DSA key pair named <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> in your <tt>~/.ssh/</tt> directory. Add the following to your <tt>~/.bash_profile</tt>:
Here's the main code block that contains the code that parses each line of input data. Remember, because we have set FS correctly, we can refer to the first field as $1, the second field as $2, etc. When doincome() and friends are called, the functions can access the current values of curmonth, $2, $3 and amount from inside the function. Take a look at the code and meet me on the other side for an explanation.
<pre>
{
    curmonth=monthdigit(substr($1,4,3))
    amount=$7
     
    #record all the categories encountered
    if ( $2 != "-" )
        globcat[$2]="yes"
    if ( $3 != "-" )
        globcat[$3]="yes"


    #tally up the transaction properly
{{file|name=~/.bash_profile|body=
    if ( $2 == "-" ) {
eval `keychain --eval --agents ssh id_rsa`
        if ( $3 == "-" ) {
}}
            print "Error: inc and exp fields are both blank!"
 
            exit 1
If you want to take advantage of GPG functionality, ensure that GNU Privacy Guard is installed and omit the <tt>--agents ssh</tt> option above.
        } else {
 
            #this is income
=== Apple MacOS X ===
            doincome(balance)
 
            if ( $5 == "Y" )
To install under MacOS X, install the MacOS X package for keychain. Assuming you have an <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> key pair in your <tt>~/.ssh/</tt> directory, add the following to your <tt>~/.bash_profile</tt>:
                doincome(balance2)
 
        }
{{file|name=~/.bash_profile|body=
    } else if ( $3 == "-" ) {
eval `keychain --eval --agents ssh --inherit any id_dsa`
        #this is an expense
}}
        doexpense(balance)
 
        if ( $5 == "Y" )  
{{Fancynote|The <tt>--inherit any</tt> option above causes keychain to inherit any ssh key passphrases stored in your Apple MacOS Keychain. If you would prefer for this to not happen, then this option can be omitted.}}
            doexpense(balance2)
 
     } else {
== Background ==
         #this is a transfer
 
         dotransfer(balance)
You're probably familiar with <tt>ssh</tt>, which has become a secure replacement for the venerable <tt>telnet</tt> and <tt>rsh</tt> commands.
        if ( $5 == "Y" )
 
            dotransfer(balance2)
Typically, when one uses <tt>ssh</tt> to connect to a remote system, one supplies a secret passphrase to <tt>ssh</tt>, which is then passed in encrypted form over the network to the remote server. This passphrase is used by the remote <tt>sshd</tt> server to determine if you should be granted access to the system.
    }                       
 
}
However, OpenSSH and nearly all other SSH clients and servers have the ability to perform another type of authentication, called asymmetric public key authentication, using the RSA or DSA authentication algorithms. They are very useful, but can also be complicated to use. <tt>keychain</tt> has been designed to make it easy to take advantage of the benefits of RSA and DSA authentication.
</pre>
 
In the main block, the first two lines set curmonth to an integer between 1 and 12, and set amount to field 7 (to make the code easier to understand). Then, we have four interesting lines, where we write values into an array called globcat. globcat, or the global categories array, is used to record all those categories encountered in the file -- "inco", "misc", "food", "util", etc. For example, if $2 == "inco", we set globcat["inco"] to "yes". Later on, we can iterate through our list of categories with a simple "for (x in globcat)" loop.
== Generating a Key Pair ==
 
To use RSA and DSA authentication, first you use a program called <tt>ssh-keygen</tt> (included with OpenSSH) to generate a ''key pair'' -- two small files. One of the files is the ''public key''. The other small file contains the ''private key''. <tt>ssh-keygen</tt> will ask you for a passphrase, and this passphrase will be used to encrypt your private key. You will need to supply this passphrase to use your private key. If you wanted to generate a DSA key pair, you would do this:
 
<console># ##i##ssh-keygen -t dsa
Generating public/private dsa key pair.</console>
You would then be prompted for a location to store your key pair. If you do not have one currently stored in <tt>~/.ssh</tt>, it is fine to accept the default location:
 
<console>Enter file in which to save the key (/root/.ssh/id_dsa): </console>
Then, you are prompted for a passphrase. This passphrase is used to encrypt the ''private key'' on disk, so even if it is stolen, it will be difficult for someone else to use it to successfully authenticate as you with any accounts that have been configured to recognize your public key.
 
Note that conversely, if you '''do not''' provide a passphrase for your private key file, then your private key file '''will not''' be encrypted. This means that if someone steals your private key file, ''they will have the full ability to authenticate with any remote accounts that are set up with your public key.''
 
Below, I have supplied a passphrase so that my private key file will be encrypted on disk:
 
<console>Enter passphrase (empty for no passphrase): ##i#########
Enter same passphrase again: ##i#########
Your identification has been saved in /var/tmp/id_dsa.
Your public key has been saved in /var/tmp/id_dsa.pub.
The key fingerprint is:
5c:13:ff:46:7d:b3:bf:0e:37:1e:5e:8c:7b:a3:88:f4 root@devbox-ve
The key's randomart image is:
+--[ DSA 1024]----+
|          .      |
|          o  . |
|          o . ..o|
|      . . . o  +|
|        S     o. |
|            . o.|
|         .  ..++|
|        . o . =o*|
|         . E .+*.|
+-----------------+</console>
 
== Setting up Authentication ==
 
Here's how you use these files to authenticate with a remote server. On the remote server, you would append the contents of your ''public key'' to the <tt>~.ssh/authorized_keys</tt> file, if such a file exists. If it doesn't exist, you can simply create a new <tt>authorized_keys</tt> file in the remote account's <tt>~/.ssh</tt> directory that contains the contents of your local <tt>id_dsa.pub</tt> file.
 
Then, if you weren't going to use <tt>keychain</tt>, you'd perform the following steps. On your local client, you would start a program called <tt>ssh-agent</tt>, which runs in the background. Then you would use a program called <tt>ssh-add</tt> to tell <tt>ssh-agent</tt> about your secret private key. Then, if you've set up your environment properly, the next time you run <tt>ssh</tt>, it will find <tt>ssh-agent</tt> running, grab the private key that you added to <tt>ssh-agent</tt> using <tt>ssh-add</tt>, and use this key to authenticate with the remote server.
 
Again, the steps in the previous paragraph is what you'd do if <tt>keychain</tt> wasn't around to help. If you are using <tt>keychain</tt>, and I hope you are, you would simply add the following line to your <tt>~/.bash_profile</tt> or if a regular user to<tt>~/.bashrc</tt> :
 
{{file|name=~/.bash_profile|body=
eval `keychain --eval id_dsa`
}}
 
The next time you log in or source your <tt>~/.bash_profile</tt> or if you use <tt>~/.bashrc</tt>, <tt>keychain</tt> will start, start <tt>ssh-agent</tt> for you if it has not yet been started, use <tt>ssh-add</tt> to add your <tt>id_dsa</tt> private key file to <tt>ssh-agent</tt>, and set up your shell environment so that <tt>ssh</tt> will be able to find <tt>ssh-agent</tt>. If <tt>ssh-agent</tt> is already running, <tt>keychain</tt> will ensure that your <tt>id_dsa</tt> private key has been added to <tt>ssh-agent</tt> and then set up your environment so that <tt>ssh</tt> can find the already-running <tt>ssh-agent</tt>. It will look something like this:
 
Note that when <tt>keychain</tt> runs for the first time after your local system has booted, you will be prompted for a passphrase for your private key file if it is encrypted. But here's the nice thing about using <tt>keychain</tt> -- even if you are using an encrypted private key file, you will only need to enter your passphrase when your system first boots (or in the case of a server, when you first log in.) After that, <tt>ssh-agent</tt> is already running and has your decrypted private key cached in memory. So if you open a new shell, you will see something like this:
 
This means that you can now <tt>ssh</tt> to your heart's content, without supplying a passphrase.
 
You can also execute batch <tt>cron</tt> jobs and scripts that need to use <tt>ssh</tt> or <tt>scp</tt>, and they can take advantage of passwordless RSA/DSA authentication as well. To do this, you would add the following line to the top of a bash script:
 
{{file|name=example-script.sh|body=
eval `keychain --noask --eval id_dsa` || exit 1
}}
 
The extra <tt>--noask</tt> option tells <tt>keychain</tt> that it should not prompt for a passphrase if one is needed. Since it is not running interactively, it is better for the script to fail if the decrypted private key isn't cached in memory via <tt>ssh-agent</tt>.
 
== Keychain Options ==
 
=== Specifying Agents ===
 
In the images above, you will note that <tt>keychain</tt> starts <tt>ssh-agent</tt>, but also starts <tt>gpg-agent</tt>. Modern versions of <tt>keychain</tt> also support caching decrypted GPG keys via use of <tt>gpg-agent</tt>, and will start <tt>gpg-agent</tt> by default if it is available on your system. To avoid this behavior and only start <tt>ssh-agent</tt>, modify your <tt>~/.bash_profile</tt> as follows:
 
{{file|name=~/.bash_profile|body=
eval `keychain --agents ssh --eval id_dsa` || exit 1
}}
 
The additional <tt>--agents ssh</tt> option tells <tt>keychain</tt> just to manage <tt>ssh-agent</tt>, and ignore <tt>gpg-agent</tt> even if it is available.
 
=== Clearing Keys ===
 
Sometimes, it might be necessary to flush all cached keys in memory. To do this, type:
 
<console># ##i##keychain --clear</console>
Any agent(s) will continue to run.
 
=== Improving Security ===
 
To improve the security of <tt>keychain</tt>, some people add the <tt>--clear</tt> option to their <tt>~/.bash_profile</tt> <tt>keychain</tt> invocation. The rationale behind this is that any user logging in should be assumed to be an intruder until proven otherwise. This means that you will need to re-enter any passphrases when you log in, but cron jobs will still be able to run when you log out.
 
=== Stopping Agents ===
 
If you want to stop all agents, which will also of course cause your keys/identities to be flushed from memory, you can do this as follows:
 
<console># ##i##keychain -k all</console>
If you have other agents running under your user account, you can also tell <tt>keychain</tt> to just stop only the agents that <tt>keychain</tt> started:
 
<console># ##i##keychain -k mine</console>
 
=== GPG ===
 
Keychain can ask you for your GPG passphrase if you provide it the GPG key ID. To find it out:
<console>
$##i## gpg -k
pub  2048R/DEADBEEF 2012-08-16
uid                  Name (Comment) <email@host.tld>
sub  2048R/86D2FAC6 2012-08-16
</console>
 
Note the '''DEADBEEF''' above is the ID. Then, in your login script, do your usual
 
<console>
$##i## keychain --dir ~/.ssh/.keychain ~/.ssh/id_rsa DEADBEEF
$##i## source ~/.ssh/.keychain/$HOST-sh
$##i## source ~/.ssh/.keychain/$HOST-sh-gpg
</console>


On the next twenty or so lines, we analyze fields $2 and $3, and record the transaction appropriately. If $2=="-" and $3!="-", we have some income, so we call doincome(). If the situation is reversed, we call doexpense(); and if both $2 and $3 contain categories, we call dotransfer(). Each time, we pass the "balance" array to these functions so that the appropriate data is recorded there.
=== Learning More ===


You'll also notice several lines that say "if ( $5 == "Y" ), record that same transaction in balance2". What exactly are we doing here? You'll recall that $5 contains either a "Y" or a "N", and records whether the transaction has been posted to the account. Because we record the transaction to balance2 only if the transaction has been posted, balance2 will contain the actual account balance, while "balance" will contain all transactions, whether they have been posted or not. You can use balance2 to verify your data entry (since it should match with your current account balance according to your bank), and use "balance" to make sure that you don't overdraw your account (since it will take into account any checks you have written that have not yet been cashed).
The instructions above will work on any system that uses <tt>bash</tt> as its default shell, such as most Linux systems and Mac OS X.


=== Generating the report ===
To learn more about the many things that <tt>keychain</tt> can do, including alternate shell support, consult the keychain man page, or type <tt>keychain --help | less</tt> for a full list of command options.
After the main block repeatedly processes each input record, we now have a fairly comprehensive record of debits and credits broken down by category and by month. Now, all we need to do is define an END block that will generate a report, in this case a modest one:
<pre>
END {
    bal=0
    bal2=0       
    for (x in globcat) {
        bal=bal+balance[0,x]
        bal2=bal2+balance2[0,x]   
    }
    printf("Your available funds: %10.2f\n", bal)
    printf("Your account balance: %10.2f\n", bal2)       
}
</pre>
This report prints out a summary that looks something like this:
<pre>
Your available funds:    1174.22
Your account balance:    2399.33
</pre>
In our END block, we used the "for (x in globcat)" construct to iterate through every category, tallying up a master balance based on all the transactions recorded. We actually tally up two balances, one for available funds, and another for the account balance. To execute the program and process your own financial goodies that you've entered into a file called '''mycheckbook.txt''', put all the above code into a text file called '''balance''' and do <span style="color:green;">"chmod +x balance"</span>, and then type <span style="color:green;">"./balance mycheckbook.txt"</span>. The balance script will then add up all your transactions and print out a two-line balance summary for you.


=== Upgrades ===
I also recommend you read my original series of articles about [http://www.openssh.com OpenSSH] that I wrote for IBM developerWorks, called <tt>OpenSSH Key Management</tt>. Please note that <tt>keychain</tt> 1.0 was released along with Part 2 of this article, which was written in 2001. <tt>keychain</tt> has changed quite a bit since then. In other words, read these articles for the conceptual and [http://www.openssh.com OpenSSH] information, but consult the <tt>keychain</tt> man page for command-line options and usage instructions :)
I use a more advanced version of this program to manage my personal and business finances. My version (which I couldn't include here due to space limitations) prints out a monthly breakdown of income and expenses, including annual totals, net income and a bunch of other stuff. Even better, it outputs the data in HTML format, so that I can view it in a Web browser :) If you find this program useful, I encourage you to add these features to this script. You won't need to configure it to record any additional information; all the information you need is already in balance and balance2. Just upgrade the END block, and you're in business!


I hope you've enjoyed this series. For more information on awk, check out the resources listed below.
* [http://www.ibm.com/developerworks/library/l-keyc.html Common Threads: OpenSSH key management, Part 1] - Understanding RSA/DSA Authentication
* [http://www.ibm.com/developerworks/library/l-keyc2/ Common Threads: OpenSSH key management, Part 2] - Introducing <tt>ssh-agent</tt> and <tt>keychain</tt>
* [http://www.ibm.com/developerworks/library/l-keyc3/ Common Threads: OpenSSH key management, Part 3] - Agent forwarding and <tt>keychain</tt> improvements


== Resources ==
As mentioned at the top of the page, <tt>keychain</tt> development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [http://groups.google.com/group/funtoo-dev funtoo-dev mailing list] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.
* Read Daniel's other awk articles on Funtoo: Awk By Example, [[Awk by example, Part1|Part 1]] and [[Awk by example, Part2|Part 2]].
* If you'd like a good old-fashioned book, [http://www.oreilly.com/catalog/sed2/ O'Reilly's sed & awk, 2nd Edition] is a wonderful choice.
* Be sure to check out the [http://www.faqs.org/faqs/computer-lang/awk/faq/ comp.lang.awk FAQ]. It also contains lots of additional awk links.
* Patrick Hartigan's [http://sparky.rice.edu/~hartigan/awk.html awk tutorial] is packed with handy awk scripts.
* [http://www.tasoft.com/tawk.html Thompson's TAWK Compiler] compiles awk scripts into fast binary executables. Versions are available for Windows, OS/2, DOS, and UNIX.
* [http://www.gnu.org/software/gawk/manual/gawk.html The GNU Awk User's Guide] is available for online reference.


[[ Category:Linux Core Concepts ]]
[[Category:HOWTO]]
[[Category:Projects]]
[[Category:First Steps]]
[[Category:Articles]]
[[Category:Articles]]
{{ArticleFooter}}
{{ArticleFooter}}

Revision as of 16:22, January 5, 2015

Official Project Page

Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. Download and learn how to use Keychain on your Linux, Unix or MacOS system.
   Support Funtoo!
Get an awesome Funtoo container and support Funtoo! See Funtoo Containers for more information.

Keychain helps you to manage SSH and GPG keys in a convenient and secure manner. It acts as a frontend to ssh-agent and ssh-add, but allows you to easily have one long running ssh-agent process per system, rather than the norm of one ssh-agent per login session.

This dramatically reduces the number of times you need to enter your passphrase. With keychain, you only need to enter a passphrase once every time your local machine is rebooted. Keychain also makes it easy for remote cron jobs to securely "hook in" to a long-running ssh-agent process, allowing your scripts to take advantage of key-based logins.

Those who are new to OpenSSH and the use of public/private keys for authentication may want to check out the following articles by Daniel Robbins, which will provide a gentle introduction to the concepts used by Keychain:

Download and Resources

The latest release of keychain is version 2.7.2_beta1, and was released on July 7, 2014. The current version of keychain supports gpg-agent as well as ssh-agent.

Keychain is compatible with many operating systems, including AIX, *BSD, Cygwin, MacOS X, Linux, HP/UX, Tru64 UNIX, IRIX, Solaris and GNU Hurd.

Download

Keychain development sources can be found in the keychain git repository. Please use the Funtoo Linux bug tracker and #funtoo irc channel for keychain support questions as well as bug reports.

Project History

Daniel Robbins originally wrote keychain 1.0 through 2.0.3. 1.0 was written around June 2001, and 2.0.3 was released in late August, 2002.

After 2.0.3, keychain was maintained by various Gentoo developers, including Seth Chandler, Mike Frysinger and Robin H. Johnson, through July 3, 2003.

On April 21, 2004, Aron Griffis committed a major rewrite of keychain which was released as 2.2.0. Aron continued to actively maintain and improve keychain through October 2006 and the keychain 2.6.8 release. He also made a few commits after that date, up through mid-July, 2007. At this point, keychain had reached a point of maturity.

In mid-July, 2009, Daniel Robbins migrated Aron's mercurial repository to git and set up a new project page on funtoo.org, and made a few bug fix commits to the git repo that had been collecting in bugs.gentoo.org. Daniel continues to maintain keychain and supporting documentation on funtoo.org, and plans to make regular maintenance releases of keychain as needed.

Quick Setup

Linux

To install under Gentoo or Funtoo Linux, type 

root # emerge keychain

For other Linux distributions, use your distribution's package manager, or download and install using the source tarball above. Then generate RSA/DSA keys if necessary. The quick install docs assume you have a DSA key pair named id_dsa and id_dsa.pub in your ~/.ssh/ directory. Add the following to your ~/.bash_profile:

   ~/.bash_profile 
eval `keychain --eval --agents ssh id_rsa`

If you want to take advantage of GPG functionality, ensure that GNU Privacy Guard is installed and omit the --agents ssh option above.

Apple MacOS X

To install under MacOS X, install the MacOS X package for keychain. Assuming you have an id_dsa and id_dsa.pub key pair in your ~/.ssh/ directory, add the following to your ~/.bash_profile:

   ~/.bash_profile 
eval `keychain --eval --agents ssh --inherit any id_dsa`
   Note

The --inherit any option above causes keychain to inherit any ssh key passphrases stored in your Apple MacOS Keychain. If you would prefer for this to not happen, then this option can be omitted.

Background

You're probably familiar with ssh, which has become a secure replacement for the venerable telnet and rsh commands.

Typically, when one uses ssh to connect to a remote system, one supplies a secret passphrase to ssh, which is then passed in encrypted form over the network to the remote server. This passphrase is used by the remote sshd server to determine if you should be granted access to the system.

However, OpenSSH and nearly all other SSH clients and servers have the ability to perform another type of authentication, called asymmetric public key authentication, using the RSA or DSA authentication algorithms. They are very useful, but can also be complicated to use. keychain has been designed to make it easy to take advantage of the benefits of RSA and DSA authentication.

Generating a Key Pair

To use RSA and DSA authentication, first you use a program called ssh-keygen (included with OpenSSH) to generate a key pair -- two small files. One of the files is the public key. The other small file contains the private key. ssh-keygen will ask you for a passphrase, and this passphrase will be used to encrypt your private key. You will need to supply this passphrase to use your private key. If you wanted to generate a DSA key pair, you would do this: 

root # ssh-keygen -t dsa
Generating public/private dsa key pair.

You would then be prompted for a location to store your key pair. If you do not have one currently stored in ~/.ssh, it is fine to accept the default location: 

Enter file in which to save the key (/root/.ssh/id_dsa):

Then, you are prompted for a passphrase. This passphrase is used to encrypt the private key on disk, so even if it is stolen, it will be difficult for someone else to use it to successfully authenticate as you with any accounts that have been configured to recognize your public key.

Note that conversely, if you do not provide a passphrase for your private key file, then your private key file will not be encrypted. This means that if someone steals your private key file, they will have the full ability to authenticate with any remote accounts that are set up with your public key.

Below, I have supplied a passphrase so that my private key file will be encrypted on disk: 

Enter passphrase (empty for no passphrase): #######
Enter same passphrase again: #######
Your identification has been saved in /var/tmp/id_dsa.
Your public key has been saved in /var/tmp/id_dsa.pub.
The key fingerprint is:
5c:13:ff:46:7d:b3:bf:0e:37:1e:5e:8c:7b:a3:88:f4 root@devbox-ve
The key's randomart image is:
+--[ DSA 1024]----+
|          .      |
|           o   . |
|          o . ..o|
|       . . . o  +|
|        S     o. |
|             . o.|
|         .   ..++|
|        . o . =o*|
|         . E .+*.|
+-----------------+

Setting up Authentication

Here's how you use these files to authenticate with a remote server. On the remote server, you would append the contents of your public key to the ~.ssh/authorized_keys file, if such a file exists. If it doesn't exist, you can simply create a new authorized_keys file in the remote account's ~/.ssh directory that contains the contents of your local id_dsa.pub file.

Then, if you weren't going to use keychain, you'd perform the following steps. On your local client, you would start a program called ssh-agent, which runs in the background. Then you would use a program called ssh-add to tell ssh-agent about your secret private key. Then, if you've set up your environment properly, the next time you run ssh, it will find ssh-agent running, grab the private key that you added to ssh-agent using ssh-add, and use this key to authenticate with the remote server.

Again, the steps in the previous paragraph is what you'd do if keychain wasn't around to help. If you are using keychain, and I hope you are, you would simply add the following line to your ~/.bash_profile or if a regular user to~/.bashrc :

   ~/.bash_profile 
eval `keychain --eval id_dsa`

The next time you log in or source your ~/.bash_profile or if you use ~/.bashrc, keychain will start, start ssh-agent for you if it has not yet been started, use ssh-add to add your id_dsa private key file to ssh-agent, and set up your shell environment so that ssh will be able to find ssh-agent. If ssh-agent is already running, keychain will ensure that your id_dsa private key has been added to ssh-agent and then set up your environment so that ssh can find the already-running ssh-agent. It will look something like this:

Note that when keychain runs for the first time after your local system has booted, you will be prompted for a passphrase for your private key file if it is encrypted. But here's the nice thing about using keychain -- even if you are using an encrypted private key file, you will only need to enter your passphrase when your system first boots (or in the case of a server, when you first log in.) After that, ssh-agent is already running and has your decrypted private key cached in memory. So if you open a new shell, you will see something like this:

This means that you can now ssh to your heart's content, without supplying a passphrase.

You can also execute batch cron jobs and scripts that need to use ssh or scp, and they can take advantage of passwordless RSA/DSA authentication as well. To do this, you would add the following line to the top of a bash script:

   example-script.sh 
eval `keychain --noask --eval id_dsa`

The extra --noask option tells keychain that it should not prompt for a passphrase if one is needed. Since it is not running interactively, it is better for the script to fail if the decrypted private key isn't cached in memory via ssh-agent.

Keychain Options

Specifying Agents

In the images above, you will note that keychain starts ssh-agent, but also starts gpg-agent. Modern versions of keychain also support caching decrypted GPG keys via use of gpg-agent, and will start gpg-agent by default if it is available on your system. To avoid this behavior and only start ssh-agent, modify your ~/.bash_profile as follows:

   ~/.bash_profile 
eval `keychain --agents ssh --eval id_dsa`

The additional --agents ssh option tells keychain just to manage ssh-agent, and ignore gpg-agent even if it is available.

Clearing Keys

Sometimes, it might be necessary to flush all cached keys in memory. To do this, type: 

root # keychain --clear

Any agent(s) will continue to run.

Improving Security

To improve the security of keychain, some people add the --clear option to their ~/.bash_profile keychain invocation. The rationale behind this is that any user logging in should be assumed to be an intruder until proven otherwise. This means that you will need to re-enter any passphrases when you log in, but cron jobs will still be able to run when you log out.

Stopping Agents

If you want to stop all agents, which will also of course cause your keys/identities to be flushed from memory, you can do this as follows: 

root # keychain -k all

If you have other agents running under your user account, you can also tell keychain to just stop only the agents that keychain started: 

root # keychain -k mine

GPG

Keychain can ask you for your GPG passphrase if you provide it the GPG key ID. To find it out: 

user $ gpg -k
pub   2048R/DEADBEEF 2012-08-16
uid                  Name (Comment) <email@host.tld>
sub   2048R/86D2FAC6 2012-08-16

Note the DEADBEEF above is the ID. Then, in your login script, do your usual 

user $ keychain --dir ~/.ssh/.keychain ~/.ssh/id_rsa DEADBEEF
user $ source ~/.ssh/.keychain/$HOST-sh
user $ source ~/.ssh/.keychain/$HOST-sh-gpg

Learning More

The instructions above will work on any system that uses bash as its default shell, such as most Linux systems and Mac OS X.

To learn more about the many things that keychain can do, including alternate shell support, consult the keychain man page, or type keychain --help | less for a full list of command options.

I also recommend you read my original series of articles about OpenSSH that I wrote for IBM developerWorks, called OpenSSH Key Management. Please note that keychain 1.0 was released along with Part 2 of this article, which was written in 2001. keychain has changed quite a bit since then. In other words, read these articles for the conceptual and OpenSSH information, but consult the keychain man page for command-line options and usage instructions :)

As mentioned at the top of the page, keychain development sources can be found in the keychain git repository. Please use the funtoo-dev mailing list and #funtoo irc channel for keychain support questions as well as bug reports.


   Note

Browse all our available articles below. Use the search field to search for topics and keywords in real-time.

Article Subtitle
Article Subtitle
Awk by Example, Part 1 An intro to the great language with the strange name
Awk by Example, Part 2 Records, loops, and arrays
Awk by Example, Part 3 String functions and ... checkbooks?
Bash by Example, Part 1 Fundamental programming in the Bourne again shell (bash)
Bash by Example, Part 2 More bash programming fundamentals
Bash by Example, Part 3 Exploring the ebuild system
BTRFS Fun
Funtoo Filesystem Guide, Part 1 Journaling and ReiserFS
Funtoo Filesystem Guide, Part 2 Using ReiserFS and Linux
Funtoo Filesystem Guide, Part 3 Tmpfs and Bind Mounts
Funtoo Filesystem Guide, Part 4 Introducing Ext3
Funtoo Filesystem Guide, Part 5 Ext3 in Action
GUID Booting Guide
Learning Linux LVM, Part 1 Storage management magic with Logical Volume Management
Learning Linux LVM, Part 2 The cvs.gentoo.org upgrade
Libvirt
Linux Fundamentals, Part 1
Linux Fundamentals, Part 2
Linux Fundamentals, Part 3
Linux Fundamentals, Part 4
LVM Fun
Making the Distribution, Part 1
Making the Distribution, Part 2
Making the Distribution, Part 3
Maximum Swappage Getting the most out of swap
On screen annotation Write on top of apps on your screen
OpenSSH Key Management, Part 1 Understanding RSA/DSA Authentication
OpenSSH Key Management, Part 2 Introducing ssh-agent and keychain
OpenSSH Key Management, Part 3 Agent Forwarding
Partition Planning Tips Keeping things organized on disk
Partitioning in Action, Part 1 Moving /home
Partitioning in Action, Part 2 Consolidating data
POSIX Threads Explained, Part 1 A simple and nimble tool for memory sharing
POSIX Threads Explained, Part 2
POSIX Threads Explained, Part 3 Improve efficiency with condition variables
Sed by Example, Part 1
Sed by Example, Part 2
Sed by Example, Part 3
Successful booting with UUID Guide to use UUID for consistent booting.
The Gentoo.org Redesign, Part 1 A site reborn
The Gentoo.org Redesign, Part 2 The Documentation System
The Gentoo.org Redesign, Part 3 The New Main Pages
The Gentoo.org Redesign, Part 4 The Final Touch of XML
Traffic Control
Windows 10 Virtualization with KVM
Retrieved from "https://www.funtoo.org/index.php?title=Awk_by_Example,_Part_3&oldid=8324"