Archives for the ‘Python’ Category

Python Memory Usage using __slots__

I have been trying to reduce memory usage on our application server, and experimented with using __slots__ to reduce memory usage.
The memory overhead of using __dict__ vs __slots__ turned out to be 147 bytes per object.
Here are my results

Test
Total Memory
Bytes per object

slots
40880K
41

dicts
183840K
188

Using the following test script

class C(object):
#__slots__ = ('abc', 'defg')
pass

a = []
for i in xrange(1000000):
c [...]

Converting a .dotx file to a .docx file with Python

It used to be that one could simply rename a Word Template .dot file to .doc. However, Microsoft has made it somewhat harder today.
One way to do this is through using Word Automation. However, another way is to do some XML manipulation, given that .dotx and .dotcx files are actually zip archives.

import xml.dom.minidom
import zipfile

def dotx2docx(src, [...]

Printing the current running stack in Python

The traceback module has utility functions for printing the currently running stack, not just tracebacks.

import traceback
import logging

# somewhere in your buggy code
logging.log(1, "".join(traceback.format_stack())

unable to remap C:\cygwin\bin\tk84.dll to same address as parent

I was checking out Tauber’s Pinax, and came across VirtualEnv for the first time. Those are two cool projects.
However, building PIL on Cygwin Python 2.5 yields the following error message:
unable to remap C:\cygwin\bin\tk84.dll to same address as parent
Thanks to the power of Google, DataHammer has already found a resolution

C:\>cd \cygwin\bin
C:\cygwin\bin>ash
$ ./rebase -b 0×1000000000 tk84.dll

Cygwin Python and sqlite3

Apparently, cygwin’s distribution of Python 2.5.1 doesn’t come with sqlite3. This is a pity for those who would like to work with Django using cygwin Python.
Building _sqlite3.dll on cygwin is not too difficult. If you are in a hurry, heres the download _sqlite3.dll.
Prerequisites

/bin/cygsqlite3-0.dll
/bin/libpython2.5.dll
/usr/include/sqlite3.h
/usr/include/python2.5/Python.h [et.c]

tar jxf Python-2.5.2.tar.bz2 Python-2.5.2/Modules/_sqlite/
cd Python-2.5.2/Modules/_sqlite/
gcc -shared -o _sqlite3.dll -I /usr/include/python2.5 -L /bin [...]

First Encounter with Genshi

I was playing around with code generation of some scaffolding-style code for CodeIgniter. Initially, python’s string interpolation operator % sufficed. As time progressed, it became clearer that resorting to a templating library buys me cleaner code.
Problem #1
The $ used by Genshi for string interpolation conflicts with PHP’s $variables.
Solution #1
After 10 minutes of source [...]

Python subclassing file types

I was hoping to write a simple tee class by subclassing file, but there must be further type magic getting in the way when print >> is used.

import sys

class tee(file):
def write(self, text):
sys.stdout.write(text)
file.write(self, text)

fd = tee("output.txt", "w")
print >> fd, "Test 1"
fd.write("Test 2\n")

$ python [...]

Rich Metadata Mediated UI development

What would you automate into your boilerplate code after having 10 years of writing database applications? Here are some links to promising projects/essays:

Andromeda
Django Admin
Dataphor
Promises: User interface “hints” integrated with the data model

Naked Objects
MS Access and Query By Form
I like how CodeIgniter makes it possible to compose declarative data validations e.g. “valid_email|matches[email_confirm]|min_length[6]“
We need standard ValueConverters [...]

Managing Conflict Errors in Zope

Sometimes, your Zope application may be unnecessarily writing too many times to the database. An easy way to track what objects are being written to is to use the analyze.py utility. Look at the Count and Pct, and make sure that relatively static objects do not get too many writes.
$ ../../bin/python analyze.py Data.fs
Processed 13382 records [...]

C# 3.0 and 3.5 for experienced developers.

Python developers are probably already familiar with List Comprehensions, and lambda expressions. The interesting twist with LINQ is these expressions are translated into SQL and executed on the RDBMS instead of being done on the client side. There was a pretty clever Python project that achieved this in Python through dissassembly of python bytecodes, but [...]

Compact Inverted Calendar for Business Cards

Here’s a Compact Inverted Calendar for Business Cards, good for 2008 and 2009.

1 2 3 4 5 6 7
8 9 [...]

Escaping XML and CDATA

I don’t know what got into the head of the designers of XML, that CDATA escape sections have their own escaping syntax. Lshift has a low down on the details: Escaping XML and CDATA. Good to know that xml.dom.minidom gets this right.

Python, XmlrpcLib, ExpatParser and Encodings

I had to deal with an XMLRPC server that returned data in ‘iso-8859-1′ encoded, but didn’t have the encoding wired into the XML declaration. After some digging around, this hardcoded changes fixed the problem.

class ExpatParser:
# fast expat parser for Python 2.0 and later. [...]

Prolog For Python Programmers

Differences in the Interpreter
One gotcha is that the Prolog interpreter runs in a query mode, while code in file runs as assertions. Python, on the other hand, makes no such distinction. Code that you can compile in a file can equally be typed into the interpreter.
i.e. You have to write the rules in a text [...]

Building Python Extensions With ActivePython 2.5 And Visual Studio 2005

Actually, the title should be “you shouldn’t bother trying to build python extensions with Visual Studio 2005″. All sorts of complications arise and I haven’t got it working at all. Here’s some references if you enjoy a little self-flaggelation.
1) Make sure you understand manifests. Building DLLs will never be the same again! Richard Grimes has [...]

Serving Static Repositories Using Mercurial

I have been playing with Mercurial for a little web project. It works fine locally, but when I tried to publish a static copy on my web host, it started playing up.
I was unable to clone the remote repository using:
hg clone static-http://www.redmountainsw.com/phpar/ new_local_dirname
I kept getting this message:

requesting all changes
adding changesets
transaction abort!
rollback completed
abort: integrity check failed [...]

Django shortcuts

I’ve exceedingly poor memory, so here’s my shortcuts.

python manage.py dumpdata > data.json
python manage.py reset registration
python manage.py loaddata data.json

More Spreadsheet Innovation

Readers are probably aware how fond I am of spreadsheets as a tool for rapidly prototyping applications. Here’s another implementation of spreadsheets based around IronPython, but addresses issues like shared updates.

A Python curl

It happens that I often have to dial into users sites to troubleshoot problems, and they usually run our Python software on Windows. Since I often don’t have access to tools like curl, I’d like to present the next best thing. A command-line interface to urllib2 which uses the same options as curl. This script [...]

Exactly how hard is it to encode and decode RFC2396 in ASP.net?

Yang Xing reports:
Developer [sic] should avoid encoding Space into “+” or double encoded into “%2b”. It is recommended that when encode [sic] URL use “System.Uri.EscapeDataString”, when decode URL use “HttpUtility.UrlDecode”
Sigh. There are days when one just wishes one’s back in Python-land.
Contrast with the following Python-equivalent:

>>> urllib.quote(‘255 m’)
‘255%20m’
>>> urllib.quote_plus(‘255 m’)
‘255+m’
>>> urllib.unquote(‘255%20m’)
‘255 m’
>>> urllib.unquote_plus(‘255%20m’)
‘255 m’
>>> urllib.unquote_plus(‘255+m’)
‘255 m’