Difference between pages "Keychain" and "The Gentoo.org Redesign, Part 1"

(Difference between pages)
 
(Created page with "{{Article |Subtitle=A site reborn |Summary=Have you ever woken up one morning and suddenly realized that your cute little personal development Web site isn't really that great...")
 
Line 1: Line 1:
== Introduction ==
+
{{Article
 
+
|Subtitle=A site reborn
<tt>Keychain</tt> helps you to manage SSH and GPG keys in a convenient and secure manner. It acts as a frontend to <tt>ssh-agent</tt> and <tt>ssh-add</tt>, but allows you to easily have one long running <tt>ssh-agent</tt> process per system, rather than the norm of one <tt>ssh-agent</tt> per login session.
+
|Summary=Have you ever woken up one morning and suddenly realized that your cute little personal development Web site isn't really that great? If so, you're in good company. In this series, Daniel Robbins shares his experiences as he redesigns the www.gentoo.org Web site using technologies like XML, XSLT, and Python. Along the way, you may find some excellent approaches to use for your next Web site redesign. In this article, Daniel creates a user-centric action plan and introduces pytext, an embedded Python interpreter.
__TOC__
+
|Author=Drobbins
This dramatically reduces the number of times you need to enter your passphrase. With <tt>keychain</tt>, you only need to enter a passphrase once every time your local machine is rebooted. <tt>Keychain</tt> also makes it easy for remote cron jobs to securely &quot;hook in&quot; to a long running <tt>ssh-agent</tt> process, allowing your scripts to take advantage of key-based logins.
+
{{#seo:
+
|title=Keychain (SSH/GPG Key Management)
+
|keywords=keychain,ssh,gpg,funtoo,linux,gentoo,Daniel Robbins
+
|description=Keychain is a shell script that helps you to manage your SSH and GPG keys more easily.
+
 
}}
 
}}
== Download and Resources ==
+
== An unruly horde ==
  
The latest release of keychain is version <tt>2.7.2_beta1</tt>, and was released on July 7, 2014. The current version of keychain supports <tt>gpg-agent</tt> as well as <tt>ssh-agent</tt>.
+
Fellow software developer, may I ask you a question? Why is it that although many of us are intimately familiar with Web technologies such as HTML, CGI, Perl, Python, Java technology, and XML, our very own Web sites -- the ones devoted to our precious development projects -- look like they were thrown together by an unruly horde of hyperactive 12-year-olds? Why, oh why, is this so?
  
Keychain is compatible with many operating systems, including <tt>AIX</tt>, <tt>*BSD</tt>, <tt>Cygwin</tt>, <tt>MacOS X</tt>, <tt>Linux</tt>, <tt>HP/UX</tt>, <tt>Tru64 UNIX</tt>, <tt>IRIX</tt>, <tt>Solaris</tt> and <tt>GNU Hurd</tt>.
+
Could it be because most of the time, we've left our Web site out to rot while we squander our precious time hacking away on our free software projects? The answer, at least in my case, is a most definite "Yes."
  
=== Download ===
+
When I'm not writing articles for IBM developerWorks or being a new dad, I'm feverishly working on the next release of Gentoo Linux, along with my skilled team of volunteers. And, yes, Gentoo Linux has its own Web site (see Resources). As of right now (March 2001), our Web site isn't that special; that's because we don't spend much time working on it because we're generally engrossed in improving Gentoo Linux itself. Sure, our site does have several admittedly cute logos that I whipped up using Xara X (see Resources), but when you look past the eye candy, our site leaves a lot to be desired. Maybe yours does too. If so, I have one thing to say to you -- welcome to the club.
  
* ''Release Archive''
+
==  www.gentoo.org ==
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.2_beta1.tar.bz2 keychain 2.7.2_beta1]
+
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1.tar.bz2 keychain 2.7.1]
+
  
* ''Apple MacOS X Packages''
+
In our case, our Web site dilemma exists because our project has been growing, and our Web site hasn't. Now that Gentoo Linux is approaching the 1.0 release (when it'll be officially ready for non-developers) and is growing in popularity, we need to start seriously looking at how our Web site can better serve its users. Here's a snapshot of www.gentoo.org:
** [http://www.funtoo.org/distfiles/keychain/keychain-2.7.1-macosx.tar.gz keychain 2.7.1 MacOS X package]
+
  
Keychain development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [https://bugs.funtoo.org Funtoo Linux bug tracker] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.
+
<div style="margin: 10px;">[[File:L-redesign-01.gif|frame|class=img-responsive|The current (March 2001) state of affairs at www.gentoo.org]]</div>
  
=== Project History ===
+
As you can see, we have all the bare essentials -- a description of Gentoo Linux, a features list, a daily Changelog (automatically updated thanks to Python), and a bunch of important links (to the download sites, to our mailing list sign-up pages, and to cvsWeb). We also have links to three documentation resources -- the Gentoo Linux Install Guide and Development Guides, and Christian Zander's NVIDIA Troubleshooting Guide.
  
Daniel Robbins originally wrote <tt>keychain</tt> 1.0 through 2.0.3. 1.0 was written around June 2001, and 2.0.3 was released in late August, 2002.
+
However, while the site seems O.K., we're missing a lot of things. The most obvious is documentation -- our installation and development guides need a lot of work. And then we need to add an FAQ, new links, new user information...the list is endless.
  
After 2.0.3, <tt>keychain</tt> was maintained by various Gentoo developers, including Seth Chandler, Mike Frysinger and Robin H. Johnson, through July 3, 2003.
+
== Content vs. display ==
  
On April 21, 2004, Aron Griffis committed a major rewrite of <tt>keychain</tt> which was released as 2.2.0. Aron continued to actively maintain and improve <tt>keychain</tt> through October 2006 and the <tt>keychain</tt> 2.6.8 release. He also made a few commits after that date, up through mid-July, 2007. At this point, <tt>keychain</tt> had reached a point of maturity.
+
And now we come to our second problem. Right now, all of our work is done in raw HTML; I hack away at the index.html file until it looks O.K. Even worse, our Web documentation is written in raw HTML. This isn't a good thing from a development perspective because our raw content (consisting of paragraphs, sections, chapters) is garbled together with a bunch of display-related HTML tags. This, of course, makes it difficult to change both the content and the look of our site. While this approach has worked so far, it is bound to cause problems as our site continues to grow.
  
In mid-July, 2009, Daniel Robbins migrated Aron's mercurial repository to git and set up a new project page on funtoo.org, and made a few bug fix commits to the git repo that had been collecting in [http://bugs.gentoo.org bugs.gentoo.org]. Daniel continues to maintain <tt>keychain</tt> and supporting documentation on funtoo.org, and plans to make regular maintenance releases of <tt>keychain</tt> as needed.
+
Clearly, we need to be using better technologies behind the scenes. Instead of using HTML directly, we need to start using things like XML, XSLT, and Python. The goal is to automate as much as possible so that we can add and expand our site with ease. If we do our job well, even major future changes to our site should be relatively painless.
  
== Quick Setup ==
+
== A strategy! ==
  
=== Linux ===
+
It was clear that we had a lot of work ahead of us. In fact, there was so much to be done that I didn't know where to begin. Just as I was trying to sort out everything in my head, I came across Laura Wonnacott's "Site Savvy" InfoWorld column (see Resources). In it, she explained the concept of "user-centric" design -- how to improve a Web site while keeping the needs of your target audience (in this case, Gentoo Linux users and developers) in focus. Reading the article and taking a look at the "Handbook of User-Centered Design" link from the article helped me to formulate a strategy -- an action plan -- for the redesign:
  
To install under Gentoo or Funtoo Linux, type
+
# First, clearly define the official goal of the Web site -- in writing. What's it there for, and what's it supposed to do?
<console>
+
#  Identify the different categories of users who will be using your site -- your target audience. Rank them in order of priority: Which ones are most important to you?
###i## emerge keychain
+
# Set up a system for getting feedback from your target audience, so they can let you know what you're doing right and wrong.
</console>
+
# Evaluate the feedback, and use it to determine what parts of the site need to be improved or redesigned. Tackle high-priority sections first.
 +
# Once you've selected the part of the site to improve, get to work! During your implementation, make sure that the content and design of the new section caters specifically to the needs of your target audience and fixes all known deficiencies.
 +
# When the section redesign is complete, add it to your live site, even if it has a look that's markedly different from your current site. This way, your users can begin benefitting from the newly redesigned section immediately. If there's a problem with the redesign, you'll get user feedback more quickly. Finally, making incremental improvements to your site (rather than revamping the whole site and then rolling it out all at once -- surprise!) will help prevent your users from feeling alienated by your (possibly dramatic) site changes.
 +
#  After completing step 6, jump to step 4 and repeat.
  
For other Linux distributions, use your distribution's package manager, or download and install using the source tarball above. Then generate RSA/DSA keys if necessary. The quick install docs assume you have a DSA key pair named <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> in your <tt>~/.ssh/</tt> directory. Add the following to your <tt>~/.bash_profile</tt>:
+
== The mission statement ==
  
{{file|name=~/.bash_profile|body=
+
I was happy to discover that we already had step 3 in place. We had received several e-mail suggestions from visitors to the site, and our developer mailing list also served as a way of exchanging suggestions and comments. However, I had never really completed steps 1 or 2. While the answers may seem obvious, I did find it helpful to actually sit down and write out our mission statement:
eval `keychain --eval --agents ssh id_rsa`
+
}}
+
  
If you want to take advantage of GPG functionality, ensure that GNU Privacy Guard is installed and omit the <tt>--agents ssh</tt> option above.
+
www.gentoo.org exists to assist those who use and develop for Gentoo Linux by providing relevant, up-to-date information about Gentoo Linux and Linux in general, focusing on topics related to Gentoo Linux installation, use, administration, and development. As the central hub for all things Gentoo, the site should also feature important news relevant to Gentoo Linux users and developers. In addition to catering to Gentoo Linux users and developers, www.gentoo.org has the secondary purpose of meeting the needs of potential Gentoo Linux users, providing the information they need to decide whether Gentoo Linux is right for them.
  
=== Apple MacOS X ===
+
== The target audience ==
  
To install under MacOS X, install the MacOS X package for keychain. Assuming you have an <tt>id_dsa</tt> and <tt>id_dsa.pub</tt> key pair in your <tt>~/.ssh/</tt> directory, add the following to your <tt>~/.bash_profile</tt>:
+
So far, so good. Now for step 2 -- defining our target audience:
  
{{file|name=~/.bash_profile|body=
+
www.gentoo.org has three target audiences -- Gentoo Linux developers, users, and potential users. While no one group is absolutely a higher priority than another, right now the needs of Gentoo Linux developers are our highest priority, followed by Gentoo Linux users, and then potential users. This is because Gentoo Linux is currently in a prerelease state. When Gentoo Linux reaches version 1.0, Gentoo Linux users and potential users will also become a priority.
eval `keychain --eval --agents ssh --inherit any id_dsa`
+
}}
+
  
{{Fancynote|The <tt>--inherit any</tt> option above causes keychain to inherit any ssh key passphrases stored in your Apple MacOS Keychain. If you would prefer for this to not happen, then this option can be omitted.}}
+
== Comments and suggestions ==
  
== Background ==
+
O.K., now it's time to evaluate the suggestions and comments we've collected:
  
You're probably familiar with <tt>ssh</tt>, which has become a secure replacement for the venerable <tt>telnet</tt> and <tt>rsh</tt> commands.
+
Over the past few months, we've received a number of suggestions from Web site visitors. Overwhelmingly, people are requesting better documentation -- for both developers and users. Several developers have asked if we could create a mailing list that would be devoted exclusively to describing CVS commits.
  
Typically, when one uses <tt>ssh</tt> to connect to a remote system, one supplies a secret passphrase to <tt>ssh</tt>, which is then passed in encrypted form over the network to the remote server. This passphrase is used by the remote <tt>sshd</tt> server to determine if you should be granted access to the system.
+
Interestingly, we've also received a couple of e-mails asking whether Gentoo Linux is a commercial or free product. I'm guessing that because our main logo is inscribed with the name "Gentoo Technologies, Inc." (our legal corporation name), people assume that we have a commercial focus. Modifying our logo so that it reads "Gentoo Linux" and adding small opening paragraph to the main page explaining that we are a free software project should help.
  
However, OpenSSH and nearly all other SSH clients and servers have the ability to perform another type of authentication, called asymmetric public key authentication, using the RSA or DSA authentication algorithms. They are very useful, but can also be complicated to use. <tt>keychain</tt> has been designed to make it easy to take advantage of the benefits of RSA and DSA authentication.
+
== The improvement list ==
  
== Generating a Key Pair ==
+
O.K., now let's turn these suggestions into a list of possible improvements:
  
To use RSA and DSA authentication, first you use a program called <tt>ssh-keygen</tt> (included with OpenSSH) to generate a ''key pair'' -- two small files. One of the files is the ''public key''. The other small file contains the ''private key''. <tt>ssh-keygen</tt> will ask you for a passphrase, and this passphrase will be used to encrypt your private key. You will need to supply this passphrase to use your private key. If you wanted to generate a DSA key pair, you would do this:
+
* Revamp main page
 +
** Implementation: update logo and add free software blurb
 +
** Goal: to clearly state that we are a free software project
 +
** Target group: potential users
 +
**  Difficulty: medium
 +
* Improve basic user documentation
 +
**  Implementation: new XML/XSLT system, verbose documentation
 +
** Goal: to make it easier for users to install Gentoo Linux
 +
** Target group: new users
 +
** Difficulty: medium
 +
*Improve/create developer documentation
 +
** Implementation: new XML/XSLT system, CVS guide, dev guide, Portage guide
 +
**  Goal: to help our developers to do a great job
 +
** Target group: developers
 +
** Difficulty: hard
 +
*Add a CVS mailing list
 +
** Implementation: use our existing mailman mailing list manager
 +
** Goal: to better inform our developers
 +
** Target group: developers
 +
** Difficulty: easy
  
<console># ##i##ssh-keygen -t dsa
+
== A selection! ==
Generating public/private dsa key pair.</console>
+
You would then be prompted for a location to store your key pair. If you do not have one currently stored in <tt>~/.ssh</tt>, it is fine to accept the default location:
+
  
<console>Enter file in which to save the key (/root/.ssh/id_dsa): </console>
+
Two things leap out from the list, for different reasons. The first is the CVS mailing list -- this one is a no-brainer because it's so easy to implement. Often, it makes sense to implement the easiest changes first so that users can benefit from them right away.
Then, you are prompted for a passphrase. This passphrase is used to encrypt the ''private key'' on disk, so even if it is stolen, it will be difficult for someone else to use it to successfully authenticate as you with any accounts that have been configured to recognize your public key.
+
  
Note that conversely, if you '''do not''' provide a passphrase for your private key file, then your private key file '''will not''' be encrypted. This means that if someone steals your private key file, ''they will have the full ability to authenticate with any remote accounts that are set up with your public key.''
+
The second big thing that leaps out from the list is the need for developer documentation. This is a longer-term project that will require much more work. From my conversations with the other developers, we all appear to be in agreement that some kind of XML/XSL approach is the right solution.
  
Below, I have supplied a passphrase so that my private key file will be encrypted on disk:
+
== The XML/XSL prototype ==
  
<console>Enter passphrase (empty for no passphrase): ##i#########
+
To help start the process, I developed a prototype XML syntax to be used for all our online documentation. By using this XML syntax (called "guide"), our documentation will be clearly organized into paragraphs, sections, and chapters (using XML tags like <section>, <chapter>, etc.) while remaining free of any display-related tags. To create the HTML for display on our site, I created a prototype set of XSL transforms. By using an XSLT processor such as Sablotron, our guide XML files can be converted into HTML as follows:
Enter same passphrase again: ##i#########
+
Your identification has been saved in /var/tmp/id_dsa.
+
Your public key has been saved in /var/tmp/id_dsa.pub.
+
The key fingerprint is:
+
5c:13:ff:46:7d:b3:bf:0e:37:1e:5e:8c:7b:a3:88:f4 root@devbox-ve
+
The key's randomart image is:
+
+--[ DSA 1024]----+
+
|          .      |
+
|          o  . |
+
|          o . ..o|
+
|      . . . o  +|
+
|        S    o. |
+
|            . o.|
+
|        .  ..++|
+
|        . o . =o*|
+
|        . E .+*.|
+
+-----------------+</console>
+
  
== Setting up Authentication ==
+
devguide.xml + guide.xsl ---XSLT processor---> devguide.html
  
Here's how you use these files to authenticate with a remote server. On the remote server, you would append the contents of your ''public key'' to the <tt>~.ssh/authorized_keys</tt> file, if such a file exists. If it doesn't exist, you can simply create a new <tt>authorized_keys</tt> file in the remote account's <tt>~/.ssh</tt> directory that contains the contents of your local <tt>id_dsa.pub</tt> file.
+
The great thing about this XML/XSLT approach is that it separates our raw content (XML) from the display-related information contained in the guide.xsl (XSLT) file. If we ever need to update the look of our Web pages, we simply modify the guide.xsl file and run all our XML through the XSLT processor (Sablotron), creating updated HTML pages. Or, if we need to add a few chapters to the development guide, we can modify devguide.xml. Once we're done, we then run the XML through Sablotron, which then spits out a fully-formatted devguide.html file with several added chapters. Think of XML as the content and XSLT as the display-related formatting macros.
  
Then, if you weren't going to use <tt>keychain</tt>, you'd perform the following steps. On your local client, you would start a program called <tt>ssh-agent</tt>, which runs in the background. Then you would use a program called <tt>ssh-add</tt> to tell <tt>ssh-agent</tt> about your secret private key. Then, if you've set up your environment properly, the next time you run <tt>ssh</tt>, it will find <tt>ssh-agent</tt> running, grab the private key that you added to <tt>ssh-agent</tt> using <tt>ssh-add</tt>, and use this key to authenticate with the remote server.
+
While our entire team is convinced that XML/XSLT is the way to go, we haven't yet agreed upon an official XML syntax. Achim, our development lead, suggested that we use docbook instead of rolling our own XML syntax. However, the prototype guide XML format has helped to start the decision-making process. Because we developers are going to be the ones using the XML/XSL on a daily basis, it's important to choose a solution that we're comfortable with and meets all of our needs. By my next article, I should have a working XML/XSL doc system to show off to you.
  
Again, the steps in the previous paragraph is what you'd do if <tt>keychain</tt> wasn't around to help. If you are using <tt>keychain</tt>, and I hope you are, you would simply add the following line to your <tt>~/.bash_profile</tt> or if a regular user to<tt>~/.bashrc</tt> :
+
== Technology demo: pytext ==
  
{{file|name=~/.bash_profile|body=
+
For the most part, our current Web site isn't using any new or super-cool technologies that are worth mentioning. However, there's one notable exception -- our tiny pytext embedded Python interpreter.
eval `keychain --eval id_dsa`
+
}}
+
  
The next time you log in or source your <tt>~/.bash_profile</tt> or if you use <tt>~/.bashrc</tt>, <tt>keychain</tt> will start, start <tt>ssh-agent</tt> for you if it has not yet been started, use <tt>ssh-add</tt> to add your <tt>id_dsa</tt> private key file to <tt>ssh-agent</tt>, and set up your shell environment so that <tt>ssh</tt> will be able to find <tt>ssh-agent</tt>. If <tt>ssh-agent</tt> is already running, <tt>keychain</tt> will ensure that your <tt>id_dsa</tt> private key has been added to <tt>ssh-agent</tt> and then set up your environment so that <tt>ssh</tt> can find the already-running <tt>ssh-agent</tt>. It will look something like this:
+
Like many of you, I'm a huge Python fan and much prefer it over other scripting languages, so when it came time to add some dynamic content to our Web site, I naturally wanted to use Python. And, as you probably know, when coding dynamic HTML content, it's usually much more convenient to embed the language commands inside the HTML, rather than the other way around. Thus, the need for an embedded Python interpreter that can take a document like this:
  
Note that when <tt>keychain</tt> runs for the first time after your local system has booted, you will be prompted for a passphrase for your private key file if it is encrypted. But here's the nice thing about using <tt>keychain</tt> -- even if you are using an encrypted private key file, you will only need to enter your passphrase when your system first boots (or in the case of a server, when you first log in.) After that, <tt>ssh-agent</tt> is already running and has your decrypted private key cached in memory. So if you open a new shell, you will see something like this:
+
<pre>
 +
<p>
 +
Yeah, sure; I got some questions:<br>
 +
<!--code
 +
names=["bob","jimmy","ralph"]
 +
items=["socks","lunch","accordion"]
 +
for x in items:
 +
for y in names:
 +
print "Anyone seen",y+"'s",x+"?<br>"
 +
-->
 +
See, told you so.
 +
</pre>
  
This means that you can now <tt>ssh</tt> to your heart's content, without supplying a passphrase.
+
....and transform it into this:
  
You can also execute batch <tt>cron</tt> jobs and scripts that need to use <tt>ssh</tt> or <tt>scp</tt>, and they can take advantage of passwordless RSA/DSA authentication as well. To do this, you would add the following line to the top of a bash script:
+
<pre>
 +
<p>
 +
Yeah, sure; I got some questions:<br>
 +
Anyone seen bob's socks?<br>
 +
Anyone seen jimmy's socks?<br>
 +
Anyone seen ralph's socks?<br>
 +
Anyone seen bob's lunch?<br>
 +
Anyone seen jimmy's lunch?<br>
 +
Anyone seen ralph's lunch?<br>
 +
Anyone seen bob's accordion?<br>
 +
Anyone seen jimmy's accordion?<br>
 +
Anyone seen ralph's accordion?<br>
 +
See, told you so.
 +
</pre>
  
{{file|name=example-script.sh|body=
+
Here's the source code for pytext:
eval `keychain --noask --eval id_dsa` || exit 1
+
}}
+
  
The extra <tt>--noask</tt> option tells <tt>keychain</tt> that it should not prompt for a passphrase if one is needed. Since it is not running interactively, it is better for the script to fail if the decrypted private key isn't cached in memory via <tt>ssh-agent</tt>.
+
Code Listing 2.4:
 +
{{file|name=pytext|lang=python|desc=The pytext embedded Python interpreter|body=
 +
#!/usr/bin/env python2
  
== Keychain Options ==
+
# pytext 2.1
 +
# Copyright 1999-2001 Daniel Robbins
 +
# Distributed under the GPL
  
=== Specifying Agents ===
+
import sys
  
In the images above, you will note that <tt>keychain</tt> starts <tt>ssh-agent</tt>, but also starts <tt>gpg-agent</tt>. Modern versions of <tt>keychain</tt> also support caching decrypted GPG keys via use of <tt>gpg-agent</tt>, and will start <tt>gpg-agent</tt> by default if it is available on your system. To avoid this behavior and only start <tt>ssh-agent</tt>, modify your <tt>~/.bash_profile</tt> as follows:
+
def runfile(myarg):
 +
  "interprets a text file with embedded elements"
 +
  mylocals={}
 +
  try:
 +
      a=open(myarg,'r')
 +
  except IOError:
 +
      sys.stderr.write("!!! Error opening "+myarg+"!\n")
 +
      return
 +
  mylines=a.readlines()
 +
  a.close()
 +
  pos=0
 +
  while pos<len(mylines):
 +
      if mylines[pos][0:8]=="<!--code":
 +
  mycode=""
 +
  pos=pos+1
 +
  while (pos<len(mylines)) and (mylines[pos][0:3]!="-->"):
 +
      mycode=mycode+mylines[pos]
 +
      pos=pos+1
 +
  exec(mycode,globals(),mylocals)
 +
      else:
 +
  sys.stdout.write(mylines[pos])
 +
      pos=pos+1
  
{{file|name=~/.bash_profile|body=
+
if len(sys.argv)>1:
eval `keychain --agents ssh --eval id_dsa` || exit 1
+
  for x in sys.argv[1:]:
 +
      runfile(x)
 +
  sys.exit(0)
 +
else:
 +
  sys.stderr.write
 +
    ("pytext 2.1 -- Copyright 1999-2001 Daniel Robbins. ")
 +
  sys.stderr.write
 +
    ("Distributed under the\nGNU Public License\n\n")
 +
  sys.stderr.write
 +
    ("Usage: "+sys.argv[0]+" file0 [file1]...\n")
 +
  sys.exit(1)
 
}}
 
}}
  
The additional <tt>--agents ssh</tt> option tells <tt>keychain</tt> just to manage <tt>ssh-agent</tt>, and ignore <tt>gpg-agent</tt> even if it is available.
+
== How pytext works ==
  
=== Clearing Keys ===
+
Here's how it works. It scans each input line, and most of the time, each input line is simply echoed to stdout. However, if pytext encounters a line beginning with <!--code, then the contents of every line up to the first line beginning with --> are appended to a string called mycode. Pytext then executes the mycode string using the built-in exec() function, effectively creating an embedded Python interpreter.
  
Sometimes, it might be necessary to flush all cached keys in memory. To do this, type:
+
There's something really beautiful about this particular implementation -- we call exec() in such a way that all modifications to the global and local namespaces are saved. This makes it possible to import a module or define a variable in one embedded block, and then access this previously-created object in a later block, as this example clearly demonstrates:
  
<console># ##i##keychain --clear</console>
+
<pre>
Any agent(s) will continue to run.
+
<!--code
 +
import os
 +
foo=23
 +
-->
  
=== Improving Security ===
+
Hello
  
To improve the security of <tt>keychain</tt>, some people add the <tt>--clear</tt> option to their <tt>~/.bash_profile</tt> <tt>keychain</tt> invocation. The rationale behind this is that any user logging in should be assumed to be an intruder until proven otherwise. This means that you will need to re-enter any passphrases when you log in, but cron jobs will still be able to run when you log out.
+
<!--code
 +
print foo
 +
if os.path.exists("/tmp/mytmpfile"):
 +
print "it exists"
 +
else:
 +
print "I don't see it"
 +
-->
 +
</pre>
  
=== Stopping Agents ===
+
Handy, eh? pytext serves is an excellent demonstration of the power of Python, and is an extremely useful tool for Python fans. For our current site, we call pytext from a cron job, using it to periodically generate the HTML code for our main page Changelog:
  
If you want to stop all agents, which will also of course cause your keys/identities to be flushed from memory, you can do this as follows:
 
 
<console># ##i##keychain -k all</console>
 
If you have other agents running under your user account, you can also tell <tt>keychain</tt> to just stop only the agents that <tt>keychain</tt> started:
 
 
<console># ##i##keychain -k mine</console>
 
 
=== GPG ===
 
 
Keychain can ask you for your GPG passphrase if you provide it the GPG key ID. To find it out:
 
 
<console>
 
<console>
$##i## gpg -k
+
$ ##i##pytext index.ehtml > index.html
pub  2048R/DEADBEEF 2012-08-16
+
uid                  Name (Comment) <email@host.tld>
+
sub  2048R/86D2FAC6 2012-08-16
+
 
</console>
 
</console>
  
Note the '''DEADBEEF''' above is the ID. Then, in your login script, do your usual
+
That's it for now; I'll see you next time when we'll take a look at the first stage of the www.gentoo.org redesign!
 
+
{{ArticleFooter}}
<console>
+
$##i## keychain --dir ~/.ssh/.keychain ~/.ssh/id_rsa DEADBEEF
+
$##i## source ~/.ssh/.keychain/$HOST-sh
+
$##i## source ~/.ssh/.keychain/$HOST-sh-gpg
+
</console>
+
 
+
=== Learning More ===
+
 
+
The instructions above will work on any system that uses <tt>bash</tt> as its default shell, such as most Linux systems and Mac OS X.
+
 
+
To learn more about the many things that <tt>keychain</tt> can do, including alternate shell support, consult the keychain man page, or type <tt>keychain --help | less</tt> for a full list of command options.
+
 
+
I also recommend you read my original series of articles about [http://www.openssh.com OpenSSH] that I wrote for IBM developerWorks, called <tt>OpenSSH Key Management</tt>. Please note that <tt>keychain</tt> 1.0 was released along with Part 2 of this article, which was written in 2001. <tt>keychain</tt> has changed quite a bit since then. In other words, read these articles for the conceptual and [http://www.openssh.com OpenSSH] information, but consult the <tt>keychain</tt> man page for command-line options and usage instructions :)
+
 
+
* [http://www.ibm.com/developerworks/library/l-keyc.html Common Threads: OpenSSH key management, Part 1] - Understanding RSA/DSA Authentication
+
* [http://www.ibm.com/developerworks/library/l-keyc2/ Common Threads: OpenSSH key management, Part 2] - Introducing <tt>ssh-agent</tt> and <tt>keychain</tt>
+
* [http://www.ibm.com/developerworks/library/l-keyc3/ Common Threads: OpenSSH key management, Part 3] - Agent forwarding and <tt>keychain</tt> improvements
+
 
+
As mentioned at the top of the page, <tt>keychain</tt> development sources can be found in the [http://www.github.com/funtoo/keychain keychain git repository]. Please use the [http://groups.google.com/group/funtoo-dev funtoo-dev mailing list] and [irc://irc.freenode.net/funtoo #funtoo irc channel] for keychain support questions as well as bug reports.
+
 
+
[[Category:HOWTO]]
+
[[Category:Projects]]
+
[[Category:First Steps]]
+
[[Category:Articles]]
+

Revision as of 08:21, December 31, 2014

A site reborn

Have you ever woken up one morning and suddenly realized that your cute little personal development Web site isn't really that great? If so, you're in good company. In this series, Daniel Robbins shares his experiences as he redesigns the www.gentoo.org Web site using technologies like XML, XSLT, and Python. Along the way, you may find some excellent approaches to use for your next Web site redesign. In this article, Daniel creates a user-centric action plan and introduces pytext, an embedded Python interpreter.

Support Funtoo and help us grow! Donate $15 per month and get a free SSD-based Funtoo Virtual Container.
Looking for people interested in testing and documenting Docker support! Contact Daniel Robbins for more info.

An unruly horde

Fellow software developer, may I ask you a question? Why is it that although many of us are intimately familiar with Web technologies such as HTML, CGI, Perl, Python, Java technology, and XML, our very own Web sites -- the ones devoted to our precious development projects -- look like they were thrown together by an unruly horde of hyperactive 12-year-olds? Why, oh why, is this so?

Could it be because most of the time, we've left our Web site out to rot while we squander our precious time hacking away on our free software projects? The answer, at least in my case, is a most definite "Yes."

When I'm not writing articles for IBM developerWorks or being a new dad, I'm feverishly working on the next release of Gentoo Linux, along with my skilled team of volunteers. And, yes, Gentoo Linux has its own Web site (see Resources). As of right now (March 2001), our Web site isn't that special; that's because we don't spend much time working on it because we're generally engrossed in improving Gentoo Linux itself. Sure, our site does have several admittedly cute logos that I whipped up using Xara X (see Resources), but when you look past the eye candy, our site leaves a lot to be desired. Maybe yours does too. If so, I have one thing to say to you -- welcome to the club.

www.gentoo.org

In our case, our Web site dilemma exists because our project has been growing, and our Web site hasn't. Now that Gentoo Linux is approaching the 1.0 release (when it'll be officially ready for non-developers) and is growing in popularity, we need to start seriously looking at how our Web site can better serve its users. Here's a snapshot of www.gentoo.org:

The current (March 2001) state of affairs at www.gentoo.org

As you can see, we have all the bare essentials -- a description of Gentoo Linux, a features list, a daily Changelog (automatically updated thanks to Python), and a bunch of important links (to the download sites, to our mailing list sign-up pages, and to cvsWeb). We also have links to three documentation resources -- the Gentoo Linux Install Guide and Development Guides, and Christian Zander's NVIDIA Troubleshooting Guide.

However, while the site seems O.K., we're missing a lot of things. The most obvious is documentation -- our installation and development guides need a lot of work. And then we need to add an FAQ, new links, new user information...the list is endless.

Content vs. display

And now we come to our second problem. Right now, all of our work is done in raw HTML; I hack away at the index.html file until it looks O.K. Even worse, our Web documentation is written in raw HTML. This isn't a good thing from a development perspective because our raw content (consisting of paragraphs, sections, chapters) is garbled together with a bunch of display-related HTML tags. This, of course, makes it difficult to change both the content and the look of our site. While this approach has worked so far, it is bound to cause problems as our site continues to grow.

Clearly, we need to be using better technologies behind the scenes. Instead of using HTML directly, we need to start using things like XML, XSLT, and Python. The goal is to automate as much as possible so that we can add and expand our site with ease. If we do our job well, even major future changes to our site should be relatively painless.

A strategy!

It was clear that we had a lot of work ahead of us. In fact, there was so much to be done that I didn't know where to begin. Just as I was trying to sort out everything in my head, I came across Laura Wonnacott's "Site Savvy" InfoWorld column (see Resources). In it, she explained the concept of "user-centric" design -- how to improve a Web site while keeping the needs of your target audience (in this case, Gentoo Linux users and developers) in focus. Reading the article and taking a look at the "Handbook of User-Centered Design" link from the article helped me to formulate a strategy -- an action plan -- for the redesign:

  1. First, clearly define the official goal of the Web site -- in writing. What's it there for, and what's it supposed to do?
  2. Identify the different categories of users who will be using your site -- your target audience. Rank them in order of priority: Which ones are most important to you?
  3. Set up a system for getting feedback from your target audience, so they can let you know what you're doing right and wrong.
  4. Evaluate the feedback, and use it to determine what parts of the site need to be improved or redesigned. Tackle high-priority sections first.
  5. Once you've selected the part of the site to improve, get to work! During your implementation, make sure that the content and design of the new section caters specifically to the needs of your target audience and fixes all known deficiencies.
  6. When the section redesign is complete, add it to your live site, even if it has a look that's markedly different from your current site. This way, your users can begin benefitting from the newly redesigned section immediately. If there's a problem with the redesign, you'll get user feedback more quickly. Finally, making incremental improvements to your site (rather than revamping the whole site and then rolling it out all at once -- surprise!) will help prevent your users from feeling alienated by your (possibly dramatic) site changes.
  7. After completing step 6, jump to step 4 and repeat.

The mission statement

I was happy to discover that we already had step 3 in place. We had received several e-mail suggestions from visitors to the site, and our developer mailing list also served as a way of exchanging suggestions and comments. However, I had never really completed steps 1 or 2. While the answers may seem obvious, I did find it helpful to actually sit down and write out our mission statement:

www.gentoo.org exists to assist those who use and develop for Gentoo Linux by providing relevant, up-to-date information about Gentoo Linux and Linux in general, focusing on topics related to Gentoo Linux installation, use, administration, and development. As the central hub for all things Gentoo, the site should also feature important news relevant to Gentoo Linux users and developers. In addition to catering to Gentoo Linux users and developers, www.gentoo.org has the secondary purpose of meeting the needs of potential Gentoo Linux users, providing the information they need to decide whether Gentoo Linux is right for them.

The target audience

So far, so good. Now for step 2 -- defining our target audience:

www.gentoo.org has three target audiences -- Gentoo Linux developers, users, and potential users. While no one group is absolutely a higher priority than another, right now the needs of Gentoo Linux developers are our highest priority, followed by Gentoo Linux users, and then potential users. This is because Gentoo Linux is currently in a prerelease state. When Gentoo Linux reaches version 1.0, Gentoo Linux users and potential users will also become a priority.

Comments and suggestions

O.K., now it's time to evaluate the suggestions and comments we've collected:

Over the past few months, we've received a number of suggestions from Web site visitors. Overwhelmingly, people are requesting better documentation -- for both developers and users. Several developers have asked if we could create a mailing list that would be devoted exclusively to describing CVS commits.

Interestingly, we've also received a couple of e-mails asking whether Gentoo Linux is a commercial or free product. I'm guessing that because our main logo is inscribed with the name "Gentoo Technologies, Inc." (our legal corporation name), people assume that we have a commercial focus. Modifying our logo so that it reads "Gentoo Linux" and adding small opening paragraph to the main page explaining that we are a free software project should help.

The improvement list

O.K., now let's turn these suggestions into a list of possible improvements:

  • Revamp main page
    • Implementation: update logo and add free software blurb
    • Goal: to clearly state that we are a free software project
    • Target group: potential users
    • Difficulty: medium
  • Improve basic user documentation
    • Implementation: new XML/XSLT system, verbose documentation
    • Goal: to make it easier for users to install Gentoo Linux
    • Target group: new users
    • Difficulty: medium
  • Improve/create developer documentation
    • Implementation: new XML/XSLT system, CVS guide, dev guide, Portage guide
    • Goal: to help our developers to do a great job
    • Target group: developers
    • Difficulty: hard
  • Add a CVS mailing list
    • Implementation: use our existing mailman mailing list manager
    • Goal: to better inform our developers
    • Target group: developers
    • Difficulty: easy

A selection!

Two things leap out from the list, for different reasons. The first is the CVS mailing list -- this one is a no-brainer because it's so easy to implement. Often, it makes sense to implement the easiest changes first so that users can benefit from them right away.

The second big thing that leaps out from the list is the need for developer documentation. This is a longer-term project that will require much more work. From my conversations with the other developers, we all appear to be in agreement that some kind of XML/XSL approach is the right solution.

The XML/XSL prototype

To help start the process, I developed a prototype XML syntax to be used for all our online documentation. By using this XML syntax (called "guide"), our documentation will be clearly organized into paragraphs, sections, and chapters (using XML tags like <section>, <chapter>, etc.) while remaining free of any display-related tags. To create the HTML for display on our site, I created a prototype set of XSL transforms. By using an XSLT processor such as Sablotron, our guide XML files can be converted into HTML as follows:

devguide.xml + guide.xsl ---XSLT processor---> devguide.html

The great thing about this XML/XSLT approach is that it separates our raw content (XML) from the display-related information contained in the guide.xsl (XSLT) file. If we ever need to update the look of our Web pages, we simply modify the guide.xsl file and run all our XML through the XSLT processor (Sablotron), creating updated HTML pages. Or, if we need to add a few chapters to the development guide, we can modify devguide.xml. Once we're done, we then run the XML through Sablotron, which then spits out a fully-formatted devguide.html file with several added chapters. Think of XML as the content and XSLT as the display-related formatting macros.

While our entire team is convinced that XML/XSLT is the way to go, we haven't yet agreed upon an official XML syntax. Achim, our development lead, suggested that we use docbook instead of rolling our own XML syntax. However, the prototype guide XML format has helped to start the decision-making process. Because we developers are going to be the ones using the XML/XSL on a daily basis, it's important to choose a solution that we're comfortable with and meets all of our needs. By my next article, I should have a working XML/XSL doc system to show off to you.

Technology demo: pytext

For the most part, our current Web site isn't using any new or super-cool technologies that are worth mentioning. However, there's one notable exception -- our tiny pytext embedded Python interpreter.

Like many of you, I'm a huge Python fan and much prefer it over other scripting languages, so when it came time to add some dynamic content to our Web site, I naturally wanted to use Python. And, as you probably know, when coding dynamic HTML content, it's usually much more convenient to embed the language commands inside the HTML, rather than the other way around. Thus, the need for an embedded Python interpreter that can take a document like this:

<p>
Yeah, sure; I got some questions:<br>
<!--code
names=["bob","jimmy","ralph"]
items=["socks","lunch","accordion"]
for x in items:
for y in names:
print "Anyone seen",y+"'s",x+"?<br>"
-->
See, told you so.

....and transform it into this:

<p>
Yeah, sure; I got some questions:<br>
Anyone seen bob's socks?<br>
Anyone seen jimmy's socks?<br>
Anyone seen ralph's socks?<br>
Anyone seen bob's lunch?<br>
Anyone seen jimmy's lunch?<br>
Anyone seen ralph's lunch?<br>
Anyone seen bob's accordion?<br>
Anyone seen jimmy's accordion?<br>
Anyone seen ralph's accordion?<br>
See, told you so.

Here's the source code for pytext:

Code Listing 2.4:

pytext (python source code) - The pytext embedded Python interpreter
#!/usr/bin/env python2
 
# pytext 2.1
# Copyright 1999-2001 Daniel Robbins
# Distributed under the GPL
 
import sys
 
def runfile(myarg):
   "interprets a text file with embedded elements"
   mylocals={}
   try:
      a=open(myarg,'r')
   except IOError:
      sys.stderr.write("!!! Error opening "+myarg+"!\n")
      return
   mylines=a.readlines()
   a.close()
   pos=0
   while pos<len(mylines):
      if mylines[pos][0:8]==""):
       mycode=mycode+mylines[pos]
       pos=pos+1
  exec(mycode,globals(),mylocals)
       else:
  sys.stdout.write(mylines[pos])
       pos=pos+1
 
if len(sys.argv)>1:
   for x in sys.argv[1:]:
       runfile(x)
   sys.exit(0)
else:
   sys.stderr.write
     ("pytext 2.1 -- Copyright 1999-2001 Daniel Robbins. ")
   sys.stderr.write
     ("Distributed under the\nGNU Public License\n\n")
   sys.stderr.write
     ("Usage: "+sys.argv[0]+" file0 [file1]...\n")
   sys.exit(1)

How pytext works

Here's how it works. It scans each input line, and most of the time, each input line is simply echoed to stdout. However, if pytext encounters a line beginning with are appended to a string called mycode. Pytext then executes the mycode string using the built-in exec() function, effectively creating an embedded Python interpreter.

There's something really beautiful about this particular implementation -- we call exec() in such a way that all modifications to the global and local namespaces are saved. This makes it possible to import a module or define a variable in one embedded block, and then access this previously-created object in a later block, as this example clearly demonstrates:

<!--code
import os
foo=23
-->

Hello

<!--code
print foo
if os.path.exists("/tmp/mytmpfile"):
print "it exists"
else:
print "I don't see it"
-->

Handy, eh? pytext serves is an excellent demonstration of the power of Python, and is an extremely useful tool for Python fans. For our current site, we call pytext from a cron job, using it to periodically generate the HTML code for our main page Changelog:

$ pytext index.ehtml > index.html

That's it for now; I'll see you next time when we'll take a look at the first stage of the www.gentoo.org redesign!

Next >>>

Read the next article in this series: The Gentoo.org Redesign, Part 2

Support Funtoo and help us grow! Donate $15 per month and get a free SSD-based Funtoo Virtual Container.
Looking for people interested in testing and documenting Docker support! Contact Daniel Robbins for more info.

About the Author

Daniel Robbins is best known as the creator of Gentoo Linux and author of many IBM developerWorks articles about Linux. Daniel currently serves as Benevolent Dictator for Life (BDFL) of Funtoo Linux. Funtoo Linux is a Gentoo-based distribution and continuation of Daniel's original Gentoo vision.

Got Funtoo?

Have you installed Funtoo Linux yet? Discover the power of a from-source meta-distribution optimized for your hardware! See our installation instructions and browse our CPU-optimized builds.

Funtoo News

Drobbins

How We're Keeping You At the Center of the Funtoo Universe

Read about recent developments that keep you, our users, at the forefront of our focus as Funtoo moves forward.
10 April 2015 by Drobbins
Mgorny

New OpenGL management in Funtoo

Funtoo is switching to an improved system for managing multiple OpenGL providers (Mesa/Xorg, AMD and NVIDIA). The update may involve blockers and file collisions.
30 March 2015 by Mgorny
Drobbins

Subarch Profiles are coming...

Subarch profiles are on their way! Learn more here.
29 March 2015 by Drobbins
View More News...

More Articles

Browse all our Linux-related articles, below:

A

B

F

G

K

L

M

O

P

S

T

W

X

Z