Difference between pages "System resurrection" and "Zope HOWTO"

(Difference between pages)
(Fixing broken portage)
 
(First Steps)
 
Line 1: Line 1:
Although it is always possible to resurrect a machine back to life by reinstalling it, it is not always suitable to reinstall from scratch. Sometimes the best approach is to try to repair, or "resurrect," a broken system so that is it fully functional again. This document will show you how to resurrect a broken Funtoo system without reinstalling everything from scratch.
+
This page documents how to use Zope with Funtoo Experimental, which currently has good Zope support thanks to [[Progress Overlay Python]] integration.
+
== Building binary packages ==
+
  
The best approach for critical system repair is to boot on a SystemRescueCD or other similar LiveCD, and use this as working platform to resurrect your system.
+
== About Zope ==
  
For creating binary packages, you can use any of the following for a ''source environment'':
+
Zope is an Open Source application server framework written in Python. It has an interesting history which you should familiarize yourself with before starting Zope development, as it contains several interesting twists and turns.
  
* Use the most recent Funtoo stage3 for your architecture
+
=== Zope History ===
* If available, use a recent system backup (tar or cpio archive) or snapshot
+
  
On a working Linux system, which can be either your broken system booted with a LiveCD, or any type of Linux system on your network with similar processor, you will want to build a chroot environment using the "source environment" you selected above, and use this as a platform for building binary packages to restore your system. Once these packages are created, they can be copied to your broken system and installed using the steps later in this document.
+
{{fancynote| This HOWTO targets Zope 2.13, which includes Five. It is typically the version you should be using for new Zope projects.}}
  
The binary package creation environment would typically be set up as follows:
+
* There are two versions of Zope, Zope 2 and Zope 3. One might assume that Zope 3 is the version that people should use for new software development projects by default, but this is not the case. Most Zope-based projects continue to use Zope 2. Zope 3 was an attempt to redesign Zope 2 from scratch, and is completely different from Zope 2, but it was not adopted by the community.
  
 +
* There is also something called [http://codespeak.net/z3/five/ Five] (named because it is "2 + 3") that backports many of the new features of Zope 3 into the Zope 2 framework. Several projects will use Zope 2 plus Five in order to use some of the newer features in Zope. Five was merged into mainline Zope 2 in early 2010, and first appeared in Zope 2.8.
 +
 +
* You can learn more about the history of Zope 2, 3 and Five in the [http://svn.zope.org/Zope/trunk/src/Products/Five/README.txt?view=markup Five README].
 +
 +
* To make things even more interesting, work on [http://docs.zope.org/zope2/releases/4.0/ Zope 4] is underway, and it will be based on 2.13 rather than 3.x. It includes a number of [http://docs.zope.org/zope2/releases/4.0/CHANGES.html#restructuring incompatible changes] with prior versions.
 +
=== Zope Resources ===
 +
 +
Now that you understand what version of Zope you should be targeting (2.13), we can point you towards the correct documentation :)
 +
 +
; '''[http://docs.zope.org/zope2/zope2book/ The Zope 2 Book]'''
 +
: This book provides a general introduction to Zope concepts and ZMI. It is a good place to start, but doesn't provide a direct introduction to Zope development. It's recommended that you skim through this book to familiarize yourself with Zope. It generally does not assume much prior knowledge about Web development or Python.
 +
; '''[http://docs.zope.org/zope2/zdgbook/ Zope Developer's Guide]'''
 +
: This guide will give you a better introduction to Zope development. It assumes you already know Python. Skip chapters 1 and 2 and start in [http://docs.zope.org/zope2/zdgbook/ComponentsAndInterfaces.html chapter 3], which covers components and interfaces. [http://docs.zope.org/zope2/zdgbook/Products.html Chapter 5] covers the creation of your first product.
 +
; '''[http://codespeak.net/z3/five/manual.html The Five Manual]'''
 +
: We're not done yet. There is a bunch of stuff in Zope 2.13 that is not in the official documentation. Namely, the stuff in Five.
 +
; '''[http://docs.zope.org/ztkpackages.html ZTK Documentation]'''
 +
: ZTK 
 +
; '''ZCA'''
 +
: [http://www.muthukadan.net/docs/zca.html A Comprehensive Guide to Zope Component Architecture] offers a good introduction to the programming concepts of ZCA. We also have a new page on [[Zope Component Architecture]] which will help you to understand the big picture of ZCA and why it is useful. ZCML ("Z-camel") is a part of ZCA and  was introduced in Zope 3, so typically you will find ZCML documented within Zope 3 documentation and book.
 +
; '''Content Components'''
 +
: Views and Viewlets: [http://docs.zope.org/zope.viewlet/index.html This tutorial on viewlets] also contains some viewlet-related ZCML examples near the end. The "Content Component way" of developing in Zope seems to be a Zope 3 thing and tied to ZCML. Chapter 13+ of Stephan Richter's ''Zope 3 Developer's Handbook'' (book) seems to cover this quite well. You will probably also want to check out Philipp Weitershausen's ''Web Component Development with Zope 3'' (book).
 +
; '''[http://wiki.zope.org/zope2/Zope2Wiki Zope 2 Wiki]'''
 +
: Main wiki page for all things related to Zope 2.
 +
; '''[http://docs.zope.org docs.zope.org]'''
 +
: This is the main site for Zope documentation.
 +
 +
== First Steps ==
 +
 +
First, you will need to emerge {{Package|net-zope/zope}}:
 
<console>
 
<console>
# ##i##install -d /mnt/rescue
+
###i## emerge zope
# ##i##tar xpvf backup.tar.bz2 -C /mnt/rescue
+
# ##i##cp /etc/resolv.conf /mnt/rescue/etc
+
# ##i##mount --bind /proc /mnt/rescue/proc
+
# ##i##mount --bind /sys /mnt/rescue/sys
+
# ##i##mount --bind /dev /mnt/rescue/dev
+
# ##i##mount --bind /dev/pts /mnt/rescue/dev/pts
+
# ##i##chroot /mnt/rescue
+
# ##i##source /etc/profile
+
# ##i##env-update
+
 
</console>
 
</console>
  
No matter of the way you jump in a functional Funtoo/Gentoo environment, the magic command to create a binary package archive once inside is to use the <tt>quickpkg</tt> command. <tt>quickpkg</tt> will capture the package in the exact form it is actually deployed on the environment and create an archive of it placed in <tt>/usr/portage/packages/<package-category>/<package-name>-<package-version>.tbz2</tt>.  
+
Zope is now installed.
 +
 
 +
== Project Skeleton ==
  
In the following example capture everything installed within the "source environment" that is related {{Package|sys-devel/gcc}} (4.8.1-r3 is present on the system) is captured in a single archive named <tt>gcc-4.8.1-r3.tbz2</tt> located in <tt>/usr/portage/packages/sys-devel</tt>:
+
{{fancynote| Zope should be run by a regular user account, not as the root user.}}
  
 +
The first step in using Zope is to ensure that you are using a regular user account. Create a new directory called <tt>zope_test</tt>:
 
<console>
 
<console>
# ##i##quickpkg sys-devel/gcc
+
$##bl## cd
 +
$##bl## mkdir zope_test
 
</console>
 
</console>
  
If you need to recompile a package instead of archiving an already deployed version (and of course without installing it on your "source environment"), just do:
+
Now, enter the directory, and create an "instance", which is a set of files and directories that are used to contain a Zope project:
 
+
 
<console>
 
<console>
# ##i##emerge --buildpkgonly sys-devel/gcc
+
$##bl## cd zope_test
 +
$##bl## /usr/lib/zope-2.13/bin/mkzopeinstance
 
</console>
 
</console>
  
== Restoring the binary packages on the broken system ==
+
You will see the following output, and will be prompted to answer a few questions:
 +
<console>
 +
Please choose a directory in which you'd like to install
 +
Zope "instance home" files such as database files, configuration
 +
files, etc.
  
There are a couple of methods that can be used to restore binary packages to a broken system.
+
Directory: instance
 +
Please choose a username and password for the initial user.
 +
These will be the credentials you use to initially manage
 +
your new Zope instance.
  
=== Chroot/Emerge Method ===
+
Username: admin
 +
Password: ****
 +
Verify password: ****
 +
</console>
  
This first approach can be used for lightly damaged systems that still have a functional Portage and to which you can still <tt>chroot</tt> and perform all basic Linux commands. To use this method, you would mount your broken system to <tt>/mnt/broken</tt> using steps similar to the way we set up <tt>/mnt/rescue</tt>, above.
+
Now, we will start our Zope instance:
 +
<console>
 +
$##bl## cd instance
 +
$##bl## bin/runzope
 +
</console>
  
Before or after chrooting, copy the binary packages created in the step above in the exact same location on your broken system (e.g. in <tt>/usr/portage/packages/'''sys-devel'''</tt> in the case of <tt>'''sys-devel'''/gcc</tt>).
+
Now that Zope is running, you can visit <tt>localhost:8080</tt> in your Web browser. You will see a nice introductory page to Zope.
  
Once chrooted inside your system, you will be able to merge your packages using <tt>emerge</tt> as follows:
+
If you now go to the <tt>localhost:8080/manage</tt> URL, you will be prompted to log in. Enter the username and password you specified. You are now logged in to the ZMI (Zope Management Interface.)
 +
 
 +
You can stop your application by pressing Control-C. In the future, you can start and stop your Zope instance using the following commands:
  
 
<console>
 
<console>
# ##i##emerge -k  sys-devel/gcc
+
$##bl## zopectl start
 +
$##bl## zopectl stop
 
</console>
 
</console>
  
=== Tbz2 Extract Method ===
+
{{fancynote| <tt>zopectl start</tt> will cause your instance to run in the background rather than consuming a shell console.}}
  
This alternate method for installing binary packages is simpler and does not require the use of the <tt>chroot</tt> command. To prepare for using this approach, you just need to mount all key filesystems to <tt>/mnt/broken</tt> as follows -- and bind mounts are not necessary:
+
== First Project ==
  
<console>
+
We will create a single very primitive Zope package, consisting of an Interface for a TODO class, and a TODO class.
# ##i##install -d /mnt/broken
+
# ##i##mount /dev/sdaX /mnt/broken
+
# ##i##mount /dev/sdaY /mnt/broken/usr
+
</console>
+
  
Then, use the following commands to extract the .tbz2 archive to your broken filesystem that you mounted at <tt>/mnt/broken</tt>:
+
Create the following files and directories relative to your project root:
  
<console>
+
* Create the directory <tt>lib/python/example</tt>.
# ##i##tar xjpvf gcc-4.4.5.tbz2 -C /mnt/broken
+
* Create the file <tt>lib/python/example/__init__.py</tt> by typing <tt>touch lib/python/example/__init__.py</tt>.
</console>
+
* Create these files:
  
You will see a note about the trailing garbage at the end of the file being ignored. This is normal -- tar is ignoring the Portage .tbz2 metadata that is tacked on to the end of the file.
+
=== <tt>example-configure.zcml</tt> ===
  
At this point, you can set up bind mounts (see <tt>/mnt/rescue</tt> example earlier in this document for those steps), <tt>chroot</tt> inside <tt>/mnt/broken</tt>, and perform a few tests to determine if your issue has been resolved.
+
This file registers the <tt>example</tt> directory you created in <tt>lib/python</tt> as a ''package'', so that it is seen by Zope. Edit <code>/etc/package-includes/example-configure.zcml</code>:
 +
{{File
 +
|/etc/package-includes/example-configure.zcml|<pre>
 +
<include package="example" />
 +
</pre>}}
  
== Fixing broken portage ==
+
=== <tt>interfaces.py</tt> ===
Sometimes {{Package|sys-apps/portage}} may fail and the above <tt>binpkg</tt> mentioned way to ressurect system will not work because of portage being broken itself. To fix portage manually the following steps required:
+
<console>
+
# ##i##cd /tmp
+
# ##i##wget ftp://ftp.osuosl.org/pub/funtoo/funtoo-current/snapshots/portage-latest.tar.xz
+
# ##i##tar xf portage-latest.tar.xz
+
# ##i##cd portage
+
# ##i##rm -rf /usr/lib/portage/*
+
# ##i##cp -a bin pym /usr/lib/portage/
+
# ##i##ln -s /usr/lib/portage/bin/emerge /usr/bin/emerge
+
</console>
+
This will back a working portage again. To ensure everything is consistent re-emerge portage itself immediately.
+
<console>
+
# ##i##emerge portage </console>
+
File collision warning, if any, can be skipped at this point.
+
  
= Critical System Packages =
+
The following file defines the <tt>ITODO</tt> interface, and also uses some Zope Schema functions to define what kind of data we expect to store in objects that implement <tt>ITODO</tt>. Edit <code>/lib/python/example/interfaces.py</code> with your favorite text editor:
  
 +
{{File
 +
|/lib/python/example/interfaces.py|<pre>
 +
from zope.interface import Interface
 +
from zope.schema import List, Text, TextLine, Int
  
Several packages at the heart of your Funtoo system, these mainly are:
+
class ITODO(Interface):
 +
    name = TextLine(title=u'Name', required=True)
 +
    todo = List(title=u"TODO Items", required=True, value_type=TextLine(title=u'TODO'))
 +
    daysleft = Int(title=u'Days left to complete', required=True)
 +
    description = Text(title=u'Description', required=True)
 +
</pre>}}
  
{| {{Table}}
+
=== <tt>TODO.py</tt> ===
|-
+
!Component
+
!Package
+
!Functional role
+
|-
+
|GNU Binutils
+
|{{Package|sys-devel/binutils}}
+
|Binutils are a set a tools (linker, assembler...) used behind the scene by the GNU Compiler Collection to produce executable files.
+
|-
+
|GNU Compiler collection (GCC)
+
|{{Package|sys-devel/gcc}}
+
|GCC is a collection of compilers for several languages (FORTRAN, Java, C and C++, plus some libraries like the Standard Template Library or ''STL''). A wide spectrum of software in a Funtoo system is written in C/C++.
+
|-
+
|C Library
+
|{{Package|sys-libs/glibc}}
+
|The C library contains an implementation of a wide range of commonly needed functionalities like memory allocation, strings manipulation, I/O operations and so on. It is maybe one of the most critical system components as nearly everything on a Funtoo system depends on this component (including the Python interpreter which executes the vairous Python scripts at the heart of the Funtoo core utilities and package management system).
+
|-
+
|Z library
+
|{{Package|sys-libs/zlib}}
+
|This library contain several lossless compression/decompression routines.It is used by many other components on a Funtoo system like the bzip2/bunzip2 commands (app-arch/bzip2) for example (xz archive utilities depends on another standalone set of libraries).
+
|-
+
|Python
+
|{{Package|dev-lang/python}}
+
|The Python interpreter is the heart of Portage as is written in Python (the same is also true for various second line utilities).
+
|}
+
  
= Various weird issues =
+
Now, we define <tt>TODO</tt> to be a ''persistent'' object, meaning it can be stored in the ZODB. We specify that it implements our previously-defined <tt>ITODO</tt> interface, and provide reasonable defaults for all values when we create a new TODO object. Edit <code>/lib/python/example/TODO.py<code> using your favorite text editor:
 +
{{File
 +
|/lib/python/example/TODO.py|<pre>
 +
from persistent import Persistent
 +
from zope.interface import implements
 +
from example.interfaces import ITODO
  
'''P1:''' I have a local distfiles mirror and wget complains about not being able to resolve 'localhost'. <br>
+
class TODO(Persistent):
'''S1:''' libnss (which handles name resolution) is probably damaged or suffers of inconsistencies, in your /etc/make.conf, change 'localhost' in GENTOO_MIRRORS for 127.0.0.1 (IPv4) or ::1 (IPv6)<br>
+
    implements(ITODO)
 +
    name = u''
 +
    todo = []
 +
    daysleft = 0
 +
    description = u''
 +
</pre>}}
  
'''P2:''' Same problem as described in P1 but I use another machine on my network or a public mirror on the Internet  <br>
+
=== <tt>configure.zcml</tt> ===
'''S2:''' See the solution given in S1 but with providing the IP of the machine you are downloading for.
+
 
 +
Create the <tt>/lib/python/example/configure.zcml</tt> configuration file:
 +
{{File
 +
|/lib/python/example/configure.zcml|<pre>
 +
<configure xmlns="http://namespaces.zope.org/zope"
 +
    xmlns:five="http://namespaces.zope.org/five"
 +
    xmlns:browser="http://namespaces.zope.org/browser">
 +
</configure>
 +
</pre>}}
 +
 
 +
== Debug Mode ==
 +
 
 +
We can test our first project by entering debug mode:
 +
<console>
 +
$##bl## bin/zopectl debug
 +
Starting debugger (the name "app" is bound to the top-level Zope object)
 +
</console>
 +
 
 +
Now, let's try creating a new TODO object and writing it out to a ZODB database:
 +
<console>
 +
>>> from ZODB import FileStorage, DB
 +
>>> storage = FileStorage.FileStorage('mydatabase.fs')
 +
>>> db = DB(storage)
 +
>>> connection = db.open()
 +
>>> import transaction
 +
>>> root = connection.root()
 +
>>> from example.TODO import TODO
 +
>>> a = TODO
 +
>>> a.name = u'My TODOs'
 +
>>> a.TODOS = [ u'Do Laundry', u'Wash Dishes' ]
 +
>>> a.daysleft = 1
 +
>>> a.description = u'Things I need to do today.'
 +
>>> root[u'today'] = a
 +
>>> transaction.commit()
 +
</console>
  
[[Category:Internals]]
 
[[Category:Portage]]
 
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]
 +
[[Category:Featured]]

Revision as of 22:32, 28 January 2014

This page documents how to use Zope with Funtoo Experimental, which currently has good Zope support thanks to Progress Overlay Python integration.

About Zope

Zope is an Open Source application server framework written in Python. It has an interesting history which you should familiarize yourself with before starting Zope development, as it contains several interesting twists and turns.

Zope History

Note

This HOWTO targets Zope 2.13, which includes Five. It is typically the version you should be using for new Zope projects.
  • There are two versions of Zope, Zope 2 and Zope 3. One might assume that Zope 3 is the version that people should use for new software development projects by default, but this is not the case. Most Zope-based projects continue to use Zope 2. Zope 3 was an attempt to redesign Zope 2 from scratch, and is completely different from Zope 2, but it was not adopted by the community.
  • There is also something called Five (named because it is "2 + 3") that backports many of the new features of Zope 3 into the Zope 2 framework. Several projects will use Zope 2 plus Five in order to use some of the newer features in Zope. Five was merged into mainline Zope 2 in early 2010, and first appeared in Zope 2.8.
  • You can learn more about the history of Zope 2, 3 and Five in the Five README.
  • To make things even more interesting, work on Zope 4 is underway, and it will be based on 2.13 rather than 3.x. It includes a number of incompatible changes with prior versions.

Zope Resources

Now that you understand what version of Zope you should be targeting (2.13), we can point you towards the correct documentation :)

The Zope 2 Book
This book provides a general introduction to Zope concepts and ZMI. It is a good place to start, but doesn't provide a direct introduction to Zope development. It's recommended that you skim through this book to familiarize yourself with Zope. It generally does not assume much prior knowledge about Web development or Python.
Zope Developer's Guide
This guide will give you a better introduction to Zope development. It assumes you already know Python. Skip chapters 1 and 2 and start in chapter 3, which covers components and interfaces. Chapter 5 covers the creation of your first product.
The Five Manual
We're not done yet. There is a bunch of stuff in Zope 2.13 that is not in the official documentation. Namely, the stuff in Five.
ZTK Documentation
ZTK
ZCA
A Comprehensive Guide to Zope Component Architecture offers a good introduction to the programming concepts of ZCA. We also have a new page on Zope Component Architecture which will help you to understand the big picture of ZCA and why it is useful. ZCML ("Z-camel") is a part of ZCA and was introduced in Zope 3, so typically you will find ZCML documented within Zope 3 documentation and book.
Content Components
Views and Viewlets: This tutorial on viewlets also contains some viewlet-related ZCML examples near the end. The "Content Component way" of developing in Zope seems to be a Zope 3 thing and tied to ZCML. Chapter 13+ of Stephan Richter's Zope 3 Developer's Handbook (book) seems to cover this quite well. You will probably also want to check out Philipp Weitershausen's Web Component Development with Zope 3 (book).
Zope 2 Wiki
Main wiki page for all things related to Zope 2.
docs.zope.org
This is the main site for Zope documentation.

First Steps

First, you will need to emerge net-zope/zope (package not on wiki - please add):

# emerge zope

Zope is now installed.

Project Skeleton

Note

Zope should be run by a regular user account, not as the root user.

The first step in using Zope is to ensure that you are using a regular user account. Create a new directory called zope_test:

$ cd
$ mkdir zope_test

Now, enter the directory, and create an "instance", which is a set of files and directories that are used to contain a Zope project:

$ cd zope_test
$ /usr/lib/zope-2.13/bin/mkzopeinstance

You will see the following output, and will be prompted to answer a few questions:

Please choose a directory in which you'd like to install
Zope "instance home" files such as database files, configuration
files, etc.

Directory: instance
Please choose a username and password for the initial user.
These will be the credentials you use to initially manage
your new Zope instance.

Username: admin
Password: ****
Verify password: **** 

Now, we will start our Zope instance:

$ cd instance
$ bin/runzope

Now that Zope is running, you can visit localhost:8080 in your Web browser. You will see a nice introductory page to Zope.

If you now go to the localhost:8080/manage URL, you will be prompted to log in. Enter the username and password you specified. You are now logged in to the ZMI (Zope Management Interface.)

You can stop your application by pressing Control-C. In the future, you can start and stop your Zope instance using the following commands:

$ zopectl start
$ zopectl stop

Note

zopectl start will cause your instance to run in the background rather than consuming a shell console.

First Project

We will create a single very primitive Zope package, consisting of an Interface for a TODO class, and a TODO class.

Create the following files and directories relative to your project root:

  • Create the directory lib/python/example.
  • Create the file lib/python/example/__init__.py by typing touch lib/python/example/__init__.py.
  • Create these files:

example-configure.zcml

This file registers the example directory you created in lib/python as a package, so that it is seen by Zope. Edit /etc/package-includes/example-configure.zcml:

{{{name}}}
{{{body}}}

interfaces.py

The following file defines the ITODO interface, and also uses some Zope Schema functions to define what kind of data we expect to store in objects that implement ITODO. Edit /lib/python/example/interfaces.py with your favorite text editor:

{{{name}}}
{{{body}}}

TODO.py

Now, we define TODO to be a persistent object, meaning it can be stored in the ZODB. We specify that it implements our previously-defined ITODO interface, and provide reasonable defaults for all values when we create a new TODO object. Edit /lib/python/example/TODO.py<code> using your favorite text editor:

{{{name}}}
{{{body}}}

configure.zcml

Create the /lib/python/example/configure.zcml configuration file:

{{{name}}}
{{{body}}}

Debug Mode

We can test our first project by entering debug mode:

$ bin/zopectl debug
Starting debugger (the name "app" is bound to the top-level Zope object)

Now, let's try creating a new TODO object and writing it out to a ZODB database:

>>> from ZODB import FileStorage, DB
>>> storage = FileStorage.FileStorage('mydatabase.fs')
>>> db = DB(storage)
>>> connection = db.open()
>>> import transaction
>>> root = connection.root()
>>> from example.TODO import TODO
>>> a = TODO
>>> a.name = u'My TODOs'
>>> a.TODOS = [ u'Do Laundry', u'Wash Dishes' ]
>>> a.daysleft = 1
>>> a.description = u'Things I need to do today.'
>>> root[u'today'] = a
>>> transaction.commit()