Thursday, December 31, 2009

How to find the latest Plone release

The versions of Plone listed on http://plone.org/products/plone/releases/ aren't always up-to-date. This is because it's updated only when the complete installer is available.

Us folks who just want to run a buildout can just look at http://dist.plone.org/release/, and add this to the find-links and extends in their buildout. It's just as stable a release.

Tuesday, December 29, 2009

Overriding a view class

I had some trouble overriding one of Plone (4)'s view classes. The tutorial at
http://www.slideshare.net/wooda/martin-aspeli-extending-and-customising-plone-3
were a great help.

The step below should not be needed anymore.,A fix for it has been committed to Plone 4 trunk. You can just use Interface
But the missing step was to subclass the theme marker interface from IDefaultPloneLayer, instead of from zope.interface.Interface

from plone.theme.interfaces import IDefaultPloneLayer

class IPloneInvitePolicy(IDefaultPloneLayer):
""" A marker interface for the theme layer """

Friday, December 18, 2009

The difference between view classes and skin templates, from a security point of view

Did some Googling for this, couldn't figure it out. Found help on the chat:

(03:31:51 PM) khink: So, does anyone else know about if ZCML-registered templates are executed with other permissions than skin templates?
(03:31:56 PM) ender_ left the room (quit: ).
(03:33:02 PM) optilude: khink: templates for views (such as those registered with browser:page) are considered filesystem code
(03:33:08 PM) optilude: so they are not executed in restrictedpython
(03:33:13 PM) optilude: and so you can do whatever you want there
(03:33:19 PM) optilude: they still have a view permission, obviously
(03:33:28 PM) khink: optilude: But permission restrictions should apply, right?
(03:33:39 PM) optilude: khink: what kind of permission restrictions do you mean?
(03:33:59 PM) cbess [n=christop@office.newmediagateway.com] entered the room.
(03:34:17 PM) khink: If I'm Anonymous, i shouldn't be able to see a field (on an AT type) which is protected by a permission, i thought.
(03:34:35 PM) khink: optilude: Permission 'Set own password', that is.
(03:34:37 PM) optilude: mmm
(03:34:45 PM) optilude: khink: you might
(03:34:59 PM) khink: It seems to happen: http://bit.ly/5mYWJ1
(03:35:00 PM) optilude: if the template accesses the data (getWhatever) then sure
(03:35:12 PM) optilude: khink: if you use the AT display widget it may make an explicit permission check
(03:35:25 PM) optilude: but basically, you can do context/getFoo with impunity
(03:35:34 PM) khink: But the same template in the skins folder does not show the field.
(03:35:44 PM) khink: That surprised me.
(03:36:09 PM) optilude: khink: because in a skin script, whenever you do traversal, zope does security
(03:36:22 PM) optilude: in filesystem code, there's no such check
(03:37:10 PM) khink: optilude: So in fact, fs code renders permissions defined on individual fields useless?
(03:37:40 PM) optilude: no
(03:37:48 PM) optilude: they still work
(03:38:00 PM) optilude: they still stop someone from going http://mysite/foo/bar/getPassword
(03:38:11 PM) khink: optilude: true
(03:38:14 PM) optilude: or someone with ZMI access from writing a script/template TTW that accesses stuff they shouldn't
(03:38:30 PM) optilude: the zope security model is that if you have filesystem access, you're not subjected to the sandbox
(03:38:42 PM) optilude: ZCML-registered browser views (with templates or not) are filesystem code
(03:38:44 PM) optilude: you can't make them through the web
(03:39:08 PM) optilude: so you can also do things in there that you can't do in a TTW template, e.g. use the re module or access a variable starting with an underscore
(03:39:17 PM) optilude: and, importantly, you can do this:
(03:39:29 PM) optilude: tal:condition="checkPermission('Read foo', context)"
(03:39:30 PM) optilude: right
(03:39:33 PM) optilude: or do that in a view class
(03:39:36 PM) FinnArild: I have said it before, and I will say it again: I just LOVE zsyncer.
(03:39:39 PM) khink: optilude: Yes, i see the advantage of that.
(03:39:52 PM) optilude: FinnArild: blog about it
(03:39:54 PM) optilude: people don't know about it
(03:39:56 PM) khink: optilude: Thanks for explaining!


Bottom line: anyone who is allowed to use the view gets access to the data it renders. (Of course, this also means that permissions for individual fields are not checked.) So you can't rely on any pre-defined security or permission settings.

Saturday, December 5, 2009

Plone as a DMS

In order to use Plone as a Document Management System, you'll want Plone to be accessible through the desktop. Users won't want do download a file from the website.

On Windows, Enfold Desktop is a nice solution, it blends in with your Folder and Network browsing. On Linux and Mac it can be accomplished with WebDav (which is available on Windows XP and 2003 Server, but ED is much nicer).

Zope has to be configured as a WebDav server. Add this to your buildout:

zope-conf-additional =
enable-ms-author-via on

address 8484
force-connection-close off


See http://plone.org/documentation/how-to/webdav/ for more on Webdav and see http://svn.zope.org/Zope/trunk/src/Zope2/utilities/skel/etc/zope.conf.in?view=markup for details about configuring your Zope's settings.

When testing WebDav locally, i found that going to localhost doesn't get me anywhere: i need to specify an IP (127.0.0.1).

Changed documents are uploaded immediately. "Page" types are shown in the WebDav folder as ".html" files, but they're really just text files with an HTML part in it, so they're not easily editable for users.

Sunday, November 22, 2009

Plone 3.3.2 on Lenny requires libc6-dev

Trying to buildout Plone 3.3.2 on Debian Lenny, i got this error:

/usr/lib/gcc/i486-linux-gnu/4.3.2/include-fixed/limits.h:122:61: error: limits.h: No such file or directory
...
recipe/zope2install/__init__.py", line 247, in install
'build_ext', '-i',
AssertionError


Some googling hinted i had to install the package libc6-dev, which worked.

Edit: What possibly happened is that libc6-dev was installed before as a dependency
of some other package, which i removed. I would have been better if i'd installed build-essential from the beginning.

Tuesday, November 17, 2009

The SHV5 rootkit

I found out yesterday evening that my home server was cracked. This is an old desktop which runs in a cupboard in my house, and it contains no sensitive data whatsoever. Maybe that's why i've been slacking on security: I hadn't done updates for some time until i tried a dist-upgrade to Debian Lenny, mainly for fun. I had noticed some funny behaviour before, but thought nothing of it.

While dist-upgrading, i found out that the SHV5 rootkit had been installed. I found more information on this here: http://blog.gnist.org/article.php?story=hollidaycracking Still it seems they must have gained root access in order to install the rootkit, and how that happened worries me.

I thought i was secured, having at some point installed DenyHosts, having disallowed ssh root access, etcetera. Maybe the rootkit was installed before that.

Monday, October 5, 2009

Shell script for moving files to trash

(updated 06-10-2009, bottom paragraph)

Ubuntu keeps its trash in ~/.local/share/Trash/files, and stores metadata (ie. deleted from where, deleted when) in a sibling info/ folder. This script tries to move files to trash just as the file browser would do, so you can easily restore it later.

Save it as trash (or whatever you like) in your $HOME/bin folder (don't forget to chmod u+x it), and use it instead of rm.

#!/bin/bash
#
# A shell script for moving files and folders to your (Gnome) trash can,
# allowing to restore files from the file browser.

TRASH="$HOME/.local/share/Trash/"
TRASH_FILES="$TRASH/files/"
TRASH_INFO="$TRASH/info/"
pwd=`pwd`;
until [ "$#" = "0" ];do
fullpath=`find $pwd -maxdepth 1 -name "$1"`;
trashinfofile="$TRASH_INFO$1.trashinfo";
trashdate=`date +%FT%T`;
mv $1 "$TRASH_FILES" &&
echo "[Trash Info]
Path=$fullpath
DeletionDate=$trashdate" > "$trashinfofile" &&
echo "Moved $fullpath to $TRASH_FILES and updated trash metadata." ||
echo "Could not move $fullpath to $TRASH_FILES, or could not update trash metadata!";
shift
done

Thanks to george9233 for the beginning of this script.

That was fun! But the really easy version is
sudo apt-get install trash-cli
man trash

Wednesday, September 30, 2009

Folders with quotas

Another step in getting control over the total size of stuff that users upload, is the quota product. It's not yet eggified, a bit old, but works fine on Plone 3.1.7. It's available from the collective at https://svn.plone.org/svn/collective/quota. I've added a Dutch translation.

Plone Dutch users' day (Gebruikersdag) talk

Here is a link to the slides of the talk i gave at the Dutch Plone users' day: Deliverance for Plone, a use case

Tuesday, September 29, 2009

How to limit Plone's file upload size?

It would be nice if we could somehow limit the size of members' uploads. One step in this would be to limit the size for an individual file upload.

Follow-up: See /2009/09/folders-with-quotas.html for more along this track.

After digging in ATContentTypes' ATFile class, it turns out there's a validator for this, but it's disabled by default. It can be modified by tweaking the file etc/atcontenttypes.conf.in in the ATContentTypes product:

<archetype ATFile>
# maximum file size in byte, kb or mb
max_file_size 10mb
</archetype>

You can set it to just 1024 for 1024 bytes, or use kb's or mb's as above.

Next: how to change this value without touching ATContentTypes? There's a recipe called plone.recipe.atcontenttypes which sets ATContentTypes' configuration from the buildout. Just add this to your buildout:

[buildout]
parts =
...
atcontenttypes-conf

[atcontenttypes-conf]
recipe = plone.recipe.atcontenttypes
zope-instance-location = ${instance:location}
max-file-size = ATFile:10mb

Thursday, September 24, 2009

PloneGazette: exporting all subscribers as csv

A custom script. It's worth noting that in PloneGazette 3.0, calling getFolderContents on a NewsletterBTree will give its parent's contents. That's acquisition for you. So we use the getSubcribers method.

There's also a nice trick to copy the results to the Plone log.


## Script (Python) "exportEmailSubscriptions"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=
##title=
##
response = context.REQUEST.RESPONSE
response.setHeader('Content-type','application/csv;; charset=utf-8')
response.addHeader("Content-Disposition","inline; filename=export.csv")
response.setHeader('Cache-Control','no-cache')
response.setHeader('Pragma','No-Cache')

def copy_to_log(message):
message = str(message)
script_name = "EXPORT_SUBSCRIBERS"
context.plone_log("%s: %s" % (script_name, message))
return message

from Products.CMFCore.utils import getToolByName
catalog = getToolByName(context, 'portal_catalog')
results = catalog(portal_type="NewsletterBTree")
for result in results:
folder = result.getObject()
subscr_brains = folder.getSubscribers()
for subscr_brain in subscr_brains:
subscr = subscr_brain.getObject()
print copy_to_log("%s,%s,%s" % ( subscr.Title(), subscr.active, subscr.format, ))

return printed

AttributeError: validateSingleEmailAddress with MailDropHost

I use MailDropHost in a Plone 2.5 buildout for development. The nice thing for development is, you send emails to all users from Plone, and they will not be sent, just kept in MailDrop's spool folder, until you start maildrophost.

However, it gave an error when sending mail:
  Module Products.CMFPlone.PloneTool, line 190, in validateSingleEmailAddress
AttributeError: validateSingleEmailAddress


It seems PloneTool calls this method, which is present in Plone2.5's SecureMailHost, but not in the MailDropHost. It was solved using Jarn's SecureMaildropHost. Here is my dvl.cfg:

[buildout]
extends = base.cfg
parts +=
maildrophost
eggs +=
Products.Clouseau
plone.reload
Products.DocFinderTab

[maildrophost]
recipe = infrae.maildrophost
smtp_host = mysmtphost
smtp_port = 25
version = 1.22

[productdistros]
urls +=
http://plone.org/products/securemaildrophost/releases/0.1/SecureMaildropHost-0.1.tgz

[instance]
products +=
${maildrophost:location}

Thursday, September 3, 2009

Product shows up twice

A product showed up twice: once as "Products.MyProductName" and once as "MyProductName". This happened in the ZMI > portal_quickinstaller, ZMI > Control Panel > Products, and in the Plone Add-on Products panel.

The trick was to remove the line "five:registerPackage" from configure.zcml, as explained in the weblion site. Reloading the zcml configuration with plone.reload was also handy.

Friday, August 28, 2009

Installing Plone on Ubuntu 9.04 Jaunty / 9.10 Karmic (using buildout)

(modified Mar. 19, 2010: add note about UnifiedInstaller, link to this script at plone.org)
(modified Jan. 11, 2010: added script header, put plone's buildout dir in a variable)
(modified Jan. 08, 2010: PIL as egg in buildout)
(modified Oct. 30, 2009: use http://dist.plone.org/thirdparty/)

Note: the recommended way for beginners to set up a Plone site is to use the installers, available from plone.org. Simply download and unpack the .tgz file, and read the README for what to to next. Most likely all you'll want to do at first is go into the unpacked directory (Plone-x.x.x-UnifiedInstaller) and run ./install.sh standalone. This will install a Plone folder in your homedir. Go over there, into the zinstance folder, run ./bin/instance fg and you're on your way.

Below is a more do-it-yourself approach, it's probably interesting to developers and maintainers only. The information below will be updated at plone.org.


#!/bin/bash
# install_plone_3_on_ubuntu.sh
#
# A quickstart guide to install Plone 3 on your Ubuntu system.

# 1) Plone <4 only works with Python 2.4.
sudo apt-get install python2.4

# 2a) Get setuptools / easy_install for python 2.4
#wget http://peak.telecommunity.com/dist/ez_setup.py
#sudo python2.4 ez_setup.py -U setuptools
#
# 2b) As an alternative to Setuptools, you may want to consider using
# Distribute (http://pypi.python.org/pypi/distribute/) instead.
# The Plone community nowadays tends to use Distribute.
wget http://python-distribute.org/distribute_setup.py
sudo python2.4 distribute_setup.py

# 3) Get ZopeSkel paster scripts
sudo easy_install-2.4 ZopeSkel

# 4.1) Create a buildout.
PLONEDIR='myplonebuildout'
paster create -t plone3_buildout $PLONEDIR plone_version=3.3.3 zope2_install='' plone_products_install='' zope_user=admin zope_password=admin http_port=8080 debug_mode=off verbose_security=off
# See http://dist.plone.org/release/ for available releases.
# You might also do
# paster create -t plone3_buildout
# to get a dialog prompting you for the variables.
cd $PLONEDIR
# Run bootstrap, this will create the bin/ directory
python2.4 bootstrap.py
#
# 4.2) Install header files
# You need to have python header files installed in order to build Zope.
sudo apt-get install python2.4-dev
#
# We may need these header files to build PIL (see below).
# Without the header files, you might see this after running buildout:
# --------------------------------------------------------------------
# *** TKINTER support not available
# *** JPEG support not available
# --- ZLIB (PNG/ZIP) support ok
# *** FREETYPE2 support not available
# --------------------------------------------------------------------
# This means you'll get errors like these for uploaded JPEG images:
# IOError: decoder jpeg not available
# and you won't be able to see previews of these images, or modify them.
sudo apt-get install libjpeg-dev libfreetype6-dev
# Now you'll see
# --------------------------------------------------------------------
# *** TKINTER support not available
# --- JPEG support ok
# --- ZLIB (PNG/ZIP) support ok
# --- FREETYPE2 support ok
# --------------------------------------------------------------------

# 4.3) Run buildout.
./bin/buildout
# (See http://www.netsight.co.uk/junk/xkcd-buildout.png.)

# 5a) For running the code, we need the Python imaging module (PIL), which is
# no longer packaged for Python 2.4 since Ubuntu 9.04 (Jaunty Jackalope).
wget http://dist.plone.org/thirdparty/PILwoTk-1.1.6.4.tar.gz
sudo easy_install-2.4 PILwoTk-1.1.6.4.tar.gz
#
# 5b) As an alternative, you may also do as Alex Clarke suggests on
# http://blog.aclark.net/?p=31, and add this to your buildout.cfg:
# [buildout]
# find-links +=
# find-links = http://dist.plone.org/thirdparty/PILwoTk-1.1.6.4.tar.gz
# [instance]
# eggs +=
# PILwoTk
# This means you don't have to install PILwoTk system-wide, which is nice.
# However, you still have to install the header files on your system!

# 6) Start Plone!
./bin/instance fg

Friday, July 17, 2009

Authenticating in the Plone debug prompt

While debugging through ./bin/instance debug, a catalog call would only yield published objects:

>>> len(site.portal_catalog(review_state='private'))
0

I'm sure i had more than that! There's surely some problem with authentication.

How to tell your debug instance that you are, in fact, The Man?

>>> from AccessControl.SecurityManagement import newSecurityManager
>>> admin = app.acl_users.getUser('admin')
>>> from zope.publisher.browser import TestRequest
>>> request = TestRequest()
>>> newSecurityManager(request,admin)
>>> len(site.portal_catalog(review_state='private'))
640

That's more like it.

You can also pass a request object to portal_catalog.searchResults, as in portal_catalog.searchResults(REQUEST=request,review_state='private').

You could also have used portal_catalog.unrestrictedSearchResults(...), as Gilles Lenfant pointed out. See Products.CMFCore.utils.CatalogTool.CatalogTool for details.

Thursday, July 16, 2009

Setup zcml from plone debug prompt

Some multiadapters are not available on the plone debug prompt by default. For an example, start an instance in debug mode and type:

site = app.plone-id
from zope.component import getMultiAdapter
from zope.publisher.browser import TestRequest
request = TestRequest()
#from Products.CMFPlone.browser.interfaces import IPlone
view = getMultiAdapter((plonesite, request), name="plone")

The above works.

However, trying to get a portlet manager to render will give an error:

context = site
provider = getMultiAdapter((context, request, view), name="plone.rightcolumn")

outputs zope.component.interfaces.ComponentLookupError: ((<PloneSite at /plone-id>, <zope.publisher.browser.TestRequest instance URL=http://127.0.0.1>, <Products.Five.metaclass.Plone object at 0x6e84750>), <InterfaceClass zope.interface.Interface>, 'plone.rightcolumn')

Now try this:

from zope.app.component.site import setSite
setSite(site)

and fetch the portlet manager again:

provider = getMultiAdapter((context, request, view), name="plone.rightcolumn")
provider

This will return something like <plone.app.portlets.manager.ColumnPortletManagerRenderer object at 0xec3c110>.

Thursday, July 2, 2009

How do Plone's expiration/publication dates work?

Update 2011-11-28: Controlling permission: Access inactive portal content .

It's simple enough, but worth a summary:


  • When an item is past its expiration date, it's marked "expired" in red in its document byline when viewed.

  • An item whose publication date is before the current date doesn't get extra text in its byline.

  • In both cases, the item is "unpublished", which is not to be confused with a workflow state.

  • It merely means the item doesn't show up in listings and searches.

  • These listings include folder listings.

  • However, the owner of the item will keep seeing it, which is handy because you like to know what you have lying around in your site.

  • The permission that controls this is Access inactive portal content.

  • Expired items in a folder are marked as such when viewing the folder_contents.

  • There's no quick way of seeing if items in a folder listing are not yet published.

  • When you set an unpublished item as the default view for a folder, that item will be shown.

  • Unpublishing an item doesn't have any effect for admins. They will always see unpublished items in their listings and searches.

  • Giving another regular users rights ("can add", can edit", "can review") on the item doesn't make it any less unpublished for those users.

  • A practical way for a non-admin user to access an unpublished item is directly through its URL.

Friday, June 19, 2009

Countdown shell (bash) script


function countdown () {
a=$1
while [ $a -gt 0 ]; do
echo $a;
a=$(($a-1));
done
}
for i in `countdown 10`; do echo $i; sleep 0.1; done

Thursday, June 18, 2009

How to change Plone's default folder listing sort order

Note: you may also want to look at this article, which show how to add new content on the top of a folder instead of at the bottom.

The easy, all-Plone way to change Plone's folder sort order is to create a Collection (or Smart Folder, or Topic) which you can configure through the web to display items from wherever you please, in whatever order you please. I think you should consider using a Collection first. For example, look how the "News" folder is set up in in a fresh Plone install. It contains a Collection which is used as the default view for the folder.

See the documentation on using Collections.

And now, the entering-a-world-of-pain way:

The folder_listing and folder_contents views use the script getFolderContents (in CMFPlone) to get brains objects.

If you want to modify the default order in which items are listed in your site, you can customize this script by adding these lines:

# Modification to alter sort order
contentFilter['sort_on'] = "modified"
contentFilter['sort_order'] = "descending"


If you wanted a different sort order for different folders, you could place the script in the folder itself. Acquisition should take care of that. (Haven't tried this myself yet.)

I have tested this on Plone 2.5, but it should work equally well on Plone 3.2.2, as the getFolderContents script is still present there.

Note that this will disable the feauture that allows you to sort the folder by hand!

Monday, June 15, 2009

Creating a Plone 2.5 buildout with slideshowfolder 4.0

Slideshowfolder 4.0 can work on Plone 2.5, but it requires plone.app.form.

The default buildout from paster create -t plone2.5_buildout requires some tweaking. I checked out http://svn.plone.org/svn/plone/plone.app.form/branches/plone-2.5 in develop-eggs/ and used the buildout.cfg below. Note that you have to explicitly include Five 1.4.4 (Zope ships with a 1.3 version) and CMFonFive.

[buildout]
parts =
plone
zope2
productdistros
instance
zopepy


# Add additional egg download sources here. dist.plone.org contains archives
# of Plone packages.
find-links =
http://dist.plone.org
http://download.zope.org/ppix/
http://download.zope.org/distribution/
http://effbot.org/downloads

# Add additional eggs here
# elementtree is required by Plone
eggs =
elementtree
plone.app.form
Products.slideshowfolder

# Reference any eggs you are developing here, one per line
# e.g.: develop = src/my.package
develop =
develop-eggs/plone.app.form

[plone]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.distros
recipe = plone.recipe.distros
urls = http://launchpad.net/plone/2.5/2.5.5/+download/Plone-2.5.5.tar.gz
nested-packages = Plone-2.5.5.tar.gz
version-suffix-packages = Plone-2.5.5.tar.gz

[zope2]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.zope2install
recipe = plone.recipe.zope2install
fake-zope-eggs = true
additional-fake-eggs =
ZODB3
url = http://www.zope.org/Products/Zope/2.9.10/Zope-2.9.10-final.tgz

# Use this section to download additional old-style products.
# List any number of URLs for product tarballs under URLs (separate
# with whitespace, or break over several lines, with subsequent lines
# indented). If any archives contain several products inside a top-level
# directory, list the archive file name (i.e. the last part of the URL,
# normally with a .tar.gz suffix or similar) under 'nested-packages'.
# If any archives extract to a product directory with a version suffix, list
# the archive name under 'version-suffix-packages'.
[productdistros]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.distros
recipe = plone.recipe.distros
urls =
http://codespeak.net/z3/five/release/Five-1.4.4.tgz
http://codespeak.net/z3/cmfonfive/release/CMFonFive-1.3.4.tgz
nested-packages =
version-suffix-packages =

[instance]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.zope2instance
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 1235
debug-mode = on
verbose-security = on

# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = Plone my.package
eggs =
${buildout:eggs}


# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =
plone.app.form

products =
${buildout:directory}/products
${productdistros:location}
${plone:location}

[zopepy]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/zc.recipe.egg
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

Thursday, June 11, 2009

Testing cookies with wget

I noticed an odd behaviour on one of our websites, hosted by an external party: When cookies exceeded a certain length, our Plone site didn't get any cookies in the request.

An excellent way to find the point of failure here is to use wget in combination with a script which prints the cookies it gets. Create a file cookie.txt:
your.domain.com  FALSE  / FALSE 0 cookie_id "SomeCookieValue"

This file must be tab-separated! Wget will not display error messages if your cookie file is corrupt.

Next, run
wget -qO - --load-cookie cookie.txt http://your.domain.com/showCookies

where showCookies is a page that wil simply print the cookies it gets. (-O - sends output to terminal instead of file, q turns off other info.)

In my case, i used several cookie files: for each domain, a short cookie (8 chars value) and a long one (2000 characters). This way, i was able to determine that the current Debian/Ubuntu (Intrepid) Pound package (version 2.4.3) doesn't deal well with cookies: when cookie data exceeds about 500 bytes, the cookies aren't passed to the application. Hand-compiling version 2.4.4 remedies this.

Update: More info on limited cookie(s) size with Pound.

Monday, April 27, 2009

ATI woes after upgrade to Jaunty

After upgrading to Ubuntu 9.04, i would get a scrambled screen and a freeze after X starts. It was still possible to SSH to the machine, but it didn't react to the keyboard. I have an ATI Radeon HD2350. I had problems with earlier versions of Ubuntu, but my friend Google really didn't give me much hope. (I'll never ever buy something with an ATI card in it again!)

I found a fix here, and it consisted of installing envyng-core and running envyng -t ro remove all ati drivers. That was all i had to do to get a normal login screen.

Update: After several issues with this graphics card (system freezes, faulty colours after user switching, never works out of the box) i ditched it. I'm in nVidia heaven now.

Friday, April 24, 2009

A text-field with formatting

Sometimes, you want your text-field (meaning "field" in an Archetypes context) to preserve line breaks. I used to think this was not possible: if you want line breaks, or other formatting stuff, you'll have to use a rich text field.

However, my colleague Huub pointed out this neat way of preserving formatting in a text field:

You set the tagged values:
default_content_type=text/x-web-intelligent
allowable_content_types=python:text/x-web-intelligent
default_output_type=text/html


This will even turn e-mail addresses and urls into links.

Wednesday, April 22, 2009

Why is @@manage-portlets not available on my custom content type?

It's something to do with not extending an Archetypes class.

I had a custom type which i created which ArchGenXML, using the <<large>> stereotype in my UML. The view wasn't available on my type. When i removed the stereotype and created a generalization from (<<stub>> class) ATBTreeFolder instead (tagged value import_from = Products.ATContentTypes.content.folder), it worked.

Thursday, April 9, 2009

Creating a separate instance for debugging

Make debugging Zope/Plone more fun using a live environment!

Update: A separate config file for a two-instance setup



You may want to use the file below to create a two-instances setup from a very basic config file. In this case, my config extends dvl.cfg, which is a single-instance, non-cluster setup. It's important that this buildout have an [instance] section. (Almost all simple buildouts, notably those created by paster, do.)

Contents of dvl-debug.cfg:

# Development setup with separate instance for debugging

[buildout]
extends = dvl.cfg

parts +=
zeoserver
debug-instance

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 127.0.0.1:8000

[instance]
recipe = collective.recipe.zope2cluster
instance-clone = instance
zeo-client = true
zeo-address = ${zeoserver:zeo-address}

[debug-instance]
recipe = collective.recipe.zope2cluster
instance-clone = instance
http-address = 8081


Original post



I recently modified my buildout-dvl.config to read
[buildout]
# I have some base configuration in another file
extends = buildout-base.cfg

eggs +=
plone.reload
Products.DocFinderTab

parts +=
zeoserver
instance0
instance1

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${instance-settings:zope2-location}
zeo-address = ${instance-settings:zeo-address}

[instance-settings]
# Most instance-settings are defined in my buildout-base.cfg
zeo-address = 127.0.0.101:8000

[instance0]
# My development instance
recipe = collective.recipe.zope2cluster
instance-clone = instance-settings
http-address = 5432

debug-mode = on
verbose-security = on

zcml =
plone.reload

[instance1]
# My debugging instance
recipe = collective.recipe.zope2cluster
instance-clone = instance-settings
http-address = 8101

Where buildout-base.cfg contains, among other things:
[instance-settings]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = zope-prd:secret
effective-user = zope-prd
zeo-client = True
zeo-address = 192.168.62.39:8000

Just fire up your debugging instance: ./bin/instance1 debug. Change something, using only Python code, save it by doing import transaction; transaction.commit() and watch your changes take effect in your live environment.

Friday, March 27, 2009

How to set a ReferenceField programmatically

I'm creating a method for importing content into Plone. The content types that are created have a reference field on them, but how to set that from your import method (or import script)?

A ReferenceField simply stores the UID of the referenced object, so you can do it like this (if your field is called myReference:
    obj.setMyReference(referenced_object.UID())

Wednesday, March 11, 2009

Installing deliverance

Following the steps on the deliverance site didn't work for me: I get stuff like /usr/bin/ld: i386 architecture of input file ... is incompatible with i386:x86-64 output I'm guessing it's because i'm on a 64 bit platform, and the build doesn't take this into account. If you are on a 32 bit platform, the steps on the deliverance site may be easier for you.

To circumvent it, i installed deliverance from svn, using these steps:

# Get development libraries (i'm on Ubuntu)
sudo apt-get install libxml2-dev libxslt-dev
# Create virtualenv
virtualenv --no-site-packages deliverance_svn
cd deliverance_svn
# Install latest version of setuptools to prevent error "global name 'log' is not defined"
. bin/activate
easy_install -U setuptools
deactivate
# Checkout deliverance from svn
sudo apt-get install subversion
svn co http://codespeak.net/svn/z3/deliverance/trunk/ deliverance
cd deliverance
# Install deliverance
../bin/python setup.py install
# Install paster in virtualenv
../bin/easy_install PasteScript
cd ..
# Create deliverance instance
./bin/paster create -t deliverance DelivTest
# Answer the questions...
./bin/deliverance-proxy DelivTest/etc/deliverance.xml

Monday, March 9, 2009

Dit en dat

Soms is het leven mooi. Bijvoorbeeld wanneer iemand deze voorbeeldtekst aanlevert:

"Ik heb veel ervaring met dit en dat, maar mijn echte specialiteit is zus en zo. Dus daarom juist nu meteen en ondanks alles, zou ik willen zeggen, evenwel nochtans, desalniettemin."

Het heeft iets weg van Ajuinen & Look, had ook iets van Koot & Bie kunnen zijn, of maarwatishet.com. Maar een snelle Google levert niets op, dus voorlopig schrijf ik 'm toe aan Philip Smits.

Monday, March 2, 2009

xml.parsers.expat.ExpatError when running ArchGenXML

I got this error:

$ ~/bin/archgenxml-virtualenv/bin/archgenxml uml/OPSB.uml
...
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 51, column 0

For a more complete stacktrace see http://paste.plone.org/26649.
Turns out it's a bug in ArgoUML 0.28 BETA 2: When saving as a .uml file, a sequence of "^@"'s is inserted. Remove it manually, or use 0.26 for now.

See http://argouml.tigris.org/issues/show_bug.cgi?id=5710 for the bug report.

edit:
The folks at Tigris were quick to follow this up. They've targeted it for the next beta release (0.28 beta 3).
another edit:
Beta 3 is out, this bug is fixed.

ArchGenXml: How to get the latest profile.xmi?

Use the agx_argouml_profile script that came with your version of ArchGenXML.

$ cd ~/bin
$ virtualenv archgenxml-latest
$ cd archgenxml-latest
$ . bin/activate
$ easy_install archgenxml
$ ./bin/agx_argouml_profile
$ ls
archgenxml_profile.xmi bin include lib
$ deactivate

Friday, February 27, 2009

Linking to translated content items from zope page templates

This document is now a how-to on plone.org.

I had a situation where a custom footer links to a "Terms of Service" page. This page was a Page (Document) object with LinguaPlone enabled.

What happens is this:

  1. Set your browser's language to Dutch.

  2. Log in. Note that all texts are in Dutch.

  3. Click the link "Voorwaarden" (or whatever, it links to 'terms-of-service'

  4. You get the item in English instead of Dutch.


This is not desirable. We should get the translation, which has been created.

So how to link to it so it gets the right language? Two things:

  • Add a script getContentTranslation:
    ## Script (Python) "getContentTranslation
    ##bind context=context
    ##parameters=
    ##title=Get the preferred translation, if it's there.
    ##


    served_languages = context.portal_languages.listSupportedLanguages()
    boundLanguages = context.portal_languages.getLanguageBindings()
    prefLang = boundLanguages[0]
    return (hasattr(context, 'getTranslation') and context.getTranslation(prefLang)) or context;


  • In you TAL, do this:
            <span id=""
    tal:define="link_object python: context.portal_url.get(content_id, None);"
    tal:condition="python: link_object is not None"
    tal:attributes="id content_id">
    <a href=""
    tal:define="
    translated_object link_object/getContentTranslation"
    tal:content="translated_object/title"
    tal:attributes="href translated_object/absolute_url">
    [Title]
    </a>
    </span>



This also takes care of inserting the translated title as the link title, and it checks for existence of the content object (whose id is defined in content_id).

Monday, February 23, 2009

My final word on RewriteRules

See this excellent page: The RewriteRuleWitch

Thursday, February 12, 2009

Convert number to scientific notation

Here's a snip to convert a number to scientific notation in Python:

def convertToScientific(nr):
coefficient = float(nr)
exponent = 0
while abs(coefficient) >= 10:
exponent += 1
coefficient = coefficient / 10
while abs(coefficient) < 1:
exponent -= 1
coefficient = coefficient * 10
return (coefficient, exponent)

Wednesday, February 4, 2009

Creating portal content in setuphandlers.py

Updated; see http://keeshink.blogspot.com/2011/05/creating-plone-content-when-installing.html

To add portal content, you can use GenericSetup's profiles/default/structure mechanism. However, this does not allow to set the default view of folders. To gain more flexibility, i added some methods to setuphandlers.py:


from Products.CMFPlone.utils import _createObjectByType

def deletePloneFolders(p):
"""Delete the standard Plone stuff that we don't need
"""
# Delete standard Plone stuff..
existing = p.objectIds()
itemsToDelete = ['Members', 'news', 'events']
for item in itemsToDelete:
if item in existing:
p.manage_delObjects(item)

def createFolderStructure(portal):
"""Define which objects we want to create in the site.
"""
profile_children = [
{ 'id': 'meters',
'title': 'My Meters',
'description': '',
'type': 'Folder',
'layout': 'metertypeform',
},
]
top_folders = [
{ 'id': 'profile',
'title': 'My Profile',
'description': '',
'type': 'Folder',
'layout': 'wm-profile',
'children': profile_children,
},
{ 'id': 'meterreadings',
'title': 'My Meter Readings',
'description': '',
'type': 'Folder',
'layout': 'meterreadings',
},
{ 'id': 'analysis',
'title': 'My Consumption',
'description': '',
'type': 'Folder',
'layout': 'analysis',
},
]
createObjects(parent=portal, children=top_folders)

def createObjects(parent, children):
"""This will create new objects, or modify existing ones if id's and type
match.
"""
parent.plone_log("Creating %s in %s" % (children, parent))
existing = parent.objectIds()
parent.plone_log("Existing ids: %s" % existing)
for new_object in children:
if new_object['id'] in existing:
parent.plone_log("%s exists, skipping" % new_object['id'])
else:
_createObjectByType(new_object['type'], parent, \
id=new_object['id'], title=new_object['title'], \
description=new_object['description'])
parent.plone_log("Now to modify the new_object...")
obj = parent.get(new_object['id'], None)
if obj is None:
parent.plone_log("can't get new_object %s to modify it!" % new_object['id'])
else:
if obj.Type() != new_object['type']:
parent.plone_log("types don't match!")
else:
obj.setLayout(new_object['layout'])
obj.reindexObject()
children = new_object.get('children',[])
if len(children) > 0:
createObjects(obj, children)

def setupVarious(context):

# Ordinarily, GenericSetup handlers check for the existence of XML files.
# Here, we are not parsing an XML file, but we use this text file as a
# flag to check that we actually meant for this import step to be run.
# The file is found in profiles/default.

portal = context.getSite()

if context.readDataFile('MyProduct.theme_various.txt') is None:
return

# Add additional setup code here
deletePloneFolders(portal)
createFolderStructure(portal)

Thursday, January 29, 2009

Debugging Apache rewrite rules using a local environment

On a production site, I got an error when accessing Zope/Plone through Apache. Without Apache in front, it worked just fine. When reading Zope's instance log, it showed IndexError: string index out of range. Probably something to do with a rewrite rule. Apache rewrite rules with the VirtualHostMonster can be a mystery.

A colleague showed me how to set up a local environment for testing. This allows us to test a rewrite rule without risking downtime on the production environment.

Edit: See the Rewrite rule witch for a quick start with rewrite rules.

Modify your hosts file



  • Add 127.0.0.1 test-rewrite to your /etc/hosts file



Set up apache



  • Install Apache locally, if you hadn't already. :)

  • Add an Apache config file test to /etc/apache2/sites-available, which looks like
    <VirtualHost *>

    ServerName test-rewrite

    RewriteEngine On

    # The rewrite rule under scrutiny
    # Assumes your Zope is running on port 8080
    RewriteRule ^/(.*) http://127.0.0.1:8080/VirtualHostBase/http/test-rewrite:80/VirtualHostRoot/$1 [P,L,NC]

    RewriteLog /var/log/apache2/rewrite.log
    RewriteLogLevel 9

    CustomLog /var/log/apache2/test-rewriterule.log combined

    </VirtualHost>

  • Enable modules proxy, rewrite, proxy-http (using a2enmod)

  • In proxy.conf, set "Allow from all"

  • Enable the new config: a2ensite test

  • Modify /etc/apache2/sites-available/default so it begins with
    NameVirtualHost *
    <VirtualHost *>



Test it!



  • Run apache2ctl -t and apache2ctl -S

  • Run a tail -f on /var/log/apache2/rewrite.log

  • Tinker on your rewrite rule, and restart apache (/etc/init.d/apache2 restart) after modifications.

  • Go to http://test-rewrite and watch the log to see what happens

History: "compare to previous revision" shows nothing

That's because you have to tell the Plone difftool which fields to compare.

Go to the ZMI, portal_diff, select Document (or whatever your content type is), select the field ('text' most likely), Diff Type is 'Compound type for AT Types', and add.

Makefile for translating page templates using i18ndude


# Makefile for creating translation files (.pot and .po) in Plone products.
# Kees Hink , 2009.
#
# Place this inside the 'i18n' directory in you Product (which should be empty
# otherwise). Adjust the definitions, and run 'make'.
# A .pot file will be created. Edit it, and run 'make xx-translation' to create
# a translation file for language xx.
#
# I have chosen to not rebuild the .pot-file, but to merge it with a newly
# created temporary file. This assures the settings (author, e-mail, package
# etc.) are kept.

# DEFINITIONS
# Modify these to suit your product.
#
# The base name for your .pot and .po files.
# Beware of trailing whitespace!
FILENAMEBASE = yourproduct_here
POTFILE := ${FILENAMEBASE}.pot
TEMPORARY = temp.pot
# The i18n domain should match the i18n.domain="" statement in your .pt,
# .cpt, etc. files.
I18NDOMAIN = yourproduct
# Where to find the translatable strings.
# For a typical Plone 2.5 product, this will be skins/{product}_templates/
# For Plone 3, it will be browser/templates/ or just browser/
TEMPLATE_DIRECTORY = '../browser/templates'

# Update the main translations file.
# This is run by default when you run 'make'.
default:
# Create temporary .pot file.
# We only search the browser/templates directory.
i18ndude rebuild-pot --pot ${TEMPORARY} \
--create ${I18NDOMAIN} ${TEMPLATE_DIRECTORY}
# Check if the .pot file exists:
# - If it exists, create a backup.
# - If no, this must be the first run. Use a copy of the temporary .pot file
# as basis. Edit the .pot file after the first run.
if [ -e ${POTFILE} ]; \
then cp ${POTFILE} ${POTFILE}.backup; \
else cp ${TEMPORARY} ${POTFILE}; \
fi
# Merge temporary into the .pot.
# Merging will print a warning 'Merge-Warning: Key is already in
# target-catalog' for each string that is in both the temporary and the real
# .pot file (almost all strings, in effect).
# You can safely ignore these warnings.
i18ndude merge --pot ${POTFILE} --merge ${TEMPORARY}

# Update a specific translation file.
# Call as 'make xx-translation', replace xx with your country code.
%-translation:
# Create backup, or else create initial file.
if [ ! -e ${FILENAMEBASE}-$*.po ]; \
then touch ${FILENAMEBASE}-$*.po;\
else cp ${FILENAMEBASE}-$*.po ${FILENAMEBASE}-$*.po.backup;\
fi
# Sync translation strings in translation file with .pot
# This may delete translations. If you see something like '0 added, 531
# removed, consider re-examining the backup.
i18ndude sync --pot ${POTFILE} ${FILENAMEBASE}-$*.po

Wednesday, January 21, 2009

Apache2 VirtualHost doesn't work

I had a setup with 2 virtual hosts via SSL on the same server, but when trying to add a third i couldn't get it to work. Turns out using named virtual hosts is guaranteed not to work with SSL, and this was SSL. So why did it work for 2 virtual hosts before? This was because the domain names were pointing to different IP addresses. (One to the load balancer and one to one of the two balanced machines.)

Lesson learned: read the ouput of 'apache2ctl -S' carefully!

Sunday, January 18, 2009

Noorderslag 2009

Seen on saturday January 17th:


  • Marike Jager: Singer/songwriter. Don't remember any features.

  • Das Aldi Combo: Funk/soul with bass, drums, organ and trombone/sousaphone.

  • Giants of Husavik: Maybe that's just not my thing. Why were they clad like KKK?

  • The Fringe: Rap/hiphop. Well, that's really not my thing.

  • De Dijk: Winner of the Popprijs 2008. They played a short set, very slick as we know them. The songs from their new album 'Brussel' were the most interesting. Always a great live band.

  • Face Tomorrow: Rock. Energetic, good stuff. Wouldn't mind seeing more of them.

  • zZz: Drums + keyboards. Original! However, my ears were tired.


Great evening, when do you get a chance to see so many bands live? Nice crowd too.

Eurosonic 2009 (day 2)

Seen friday January 16th:

  • De Fuckups (De Walrus): Punk! Bass, drums, 2 guitars, singer.

  • The 69'ers (De Walrus): Drums/guitar duo. Well done, but lacked power or variation after a few songs.

  • Universe 217 (Huis de Beurs): Metal, which is not my area of interest. Interesting still.

  • Awkward I (Huis de Beurs): Acoustic multivocal songs with guitar(s), cello, organ, mandolin. Original songs, good show

Where does cups-pdf put my pdf file?

Cups-pdf is a great tool if you want to print files to pdf format. It installs very easily in Ubuntu (Intrepid): just do sudo apt-get install cups-pdf, and you will see a 'PDF' printer in your printers overview. It works like a charm, but where do the PDF files go?

You'll have to create a directory called 'PDF' in your home folder. It's strange that cups-pdf exits without error for cases where there's no PDF folder yet, but there you go. This bug is already known.

Friday, January 16, 2009

How to be careful when banking online

Many people would love to be careful, but don't know how. A few hints are explained here.
When banking online:

  • Don't have more than one browser window or tab open when banking.

  • When you're asked to type a password, always check that that request comes from the real site.

  • Consider downloading the NoScript plugin for Firefox

Eurosonic 2009 (day 1)

Seen Thursday January 15:


  • Them Holy Rollers (Pacific Bar): Good, but it should have been way louder.

  • Killteam (De Kar): Hard metal-ish stuff, and loud enough. Liked it.

  • Polarkreis 18 (Grand Theatre Main): 80's pop, bombastic and slightly gay, mostly due to the singer. Nice to see pros at work.

  • John & Jehn (Spieghel Down): Singer/songwriterish duo, didn't sound interesting.

  • Barbie Bangkok (Spieghel Up): Couldn't see a thing, place was full. Music sounded flavorless.

  • Triggerfinger (Vera): Trio.The evening's winner. A good mixture of the first two acts.

Thursday, January 15, 2009

Python password generator

Probably been done before, whatever.

(Update 2010-06-24: Maybe you want to use a command line program like pwgen)


from random import choice

default_number_of_passwords = 100
default_password_length = 8

def generate(length):
"""Make a password.

Password length is an argument. Which characters are used can be set below.
"""
# Define character classes.
az = range(ord('a'),ord('z')+1)
AZ = range(ord('A'),ord('Z')+1)
zero_to_9 = range(ord('0'),ord('9')+1)
special = [ord(x) for x in '!@#$%^&*()-_=+[]{};:,.<>/?\|`~']
# Select character classes to use.
# Special characters may not be valid on the system where you want to use
# these passwords! They are therefore not used by default. Modify the string
# on the line below accordingly, and probably also the 'special' list
# above.
characters = az + AZ + zero_to_9
pw = ''
for i in range(length):
pw += chr(choice(characters))
return pw

def main_program(number_of_passwords=None,password_length=None):
if number_of_passwords is None:
number_of_passwords = default_number_of_passwords
if password_length is None:
password_length = default_password_length
for i in range(number_of_passwords):
print generate(password_length)

if '__main__' in __name__:
main_program()

Monday, January 12, 2009

Nomination for Least Prolific Blogger

I hereby would like to nominate Wichert Akkerman for the Least Prolific Blogger Award. His last entry comes after a 16-month silence. And of course, when you don't blog often, people pay extra attention when you do!

Monday, January 5, 2009

Parcours NK Marathon op natuurijs

Hier de locatie: De Oostvaardersplassen. Rechtsboven Lelystad.

En hier het parcours, van de site van de KNSB: