Python 3 and PyObjC

My main programming platform is OS X 10.8 and Python 3 (currently Python 3.3). In earlier blog posts I have discussed installing and using PyCrypto in this environment. My next attempt was to find a GUI solutions for this platform. I usually prefer native approaches when programming, and for OS X the Cocoa framework is the native approach. To get access to this from Python we need a bridge, and the bridge is called PyObjC. At the home page PyObjC seems like a dead project. But Ronald Oussoren is still doing some development, including Python 3 support. It is not straight forward to compile and install PyObjC for Python 3, but Giacomo Lacava provides the necessary steps. To summarize I have done the installation using the following steps:

1. Fetch the latest code from Ronald Oussoren:

   hg clone https://bitbucket.org/ronaldoussoren/pyobjc
   cd pyobjc

2. Compile and install PyObjC core:

   (cd pyobjc-core && python3 setup.py install)

3. Compile and install the Cocoa, CoreData and Quartz frameworks:

   for i in Cocoa Quartz CoreData; do
     (cd pyobjc-framework-$i && python3 setup.py install)
   done

4. Compile and install the rest of PyObjC (except the XgridFoundation framework):

   for i in pyobjc-framework-[^X]*; do
     (cd $i && python3 setup.py install)
   done

So far everything seems to work OK. You need Mercurial (hg) to perform the first step.

AES and RSA PyCrypto examples with Python 3

Earlier today I wrote about installing PyCrypto for Python 3 on OS X 10.8. I will use the opportunity to provide a few examples using PyCrypto to do AES and RSA encryption and decryption. I have created 3 programs. The first one, pycrypto-mkkey.py [src], generates an RSA key and saves it in two different files. One including the private key and one with only the public key (using the .pub extension). The stored private key is password protected with the given password. The program is used like this:

python3 pycrypto-mkkey.py rsa-key "passwd"

The second program, pycrypto-encrypt.py [src], use the public RSA key to encrypt a file. Actually, the file is encrypted using AES, and the AES key is then encrypted with RSA. First the program generates a new fresh AES key and saves it to disk encrypted with the RSA key. Then it uses this AES key to encrypt the file (from stdin) and write the encrypted data to stdout. The program is used like this:

python3 pycrypto-encrypt.py rsa-key.pub aes-key < plain > cipher

The third program, pycrypto-decrypt.py [src], use the private RSA key to decrypt the file. Actually, the file is decrypted using AES, and the AES key is decrypted using RSA. First the program decrypts the AES key with the private RSA key. Then it uses this AES key to decrypt the file (from stdin) and write the decrypted data to stdout. The program is used like this:

python3 pycrypto-decrypt.py rsa-key aes-key "passwd" < cipher

Se also the README file distributed together with the programs.

PyCrypto with Python 3 on OS X 10.8

On an updated Mac (in my case 10.8.2) with an updated version of Xcode  (in my case 4.5.2) compiling and installing PyCrypto for Python 3 (in my case 3.3) is not just simply running the setup.py script. The latest release of PyCrypto is 2.6 and it supports Python 3 (it has supported Python 3 since version 2.4). But my earlier attempts using Python 3.2 on OS X 10.8 failed since it couldn't locate the SDKs and the SDK for OS X 10.6 was not installed (see issue14498 and related posts to solve these issues). In Python 3.3 this is not an issue any more, but using the correct version of the compiler is. To get the build process to use the correct compiler I tried with the following build command (llvm-gcc-4.2 is the compiler we want to use on current OS X):

> CC=llvm-gcc-4.2 python3.3 setup.py build

In the beginning of the build process it seemed to use llvm-gcc-4.2, but finally it failed with this message:

unable to execute gcc-4.2: No such file or directory

The obvious solution is the create a symlink from llvm-gcc-4.2 to gcc-4.2:

> sudo ln -s /usr/bin/llvm-gcc-4.2 /usr/bin/gcc-4.2
> python3.3 setup.py build

Success! Then install it:

> python3.3 setup.py install

And we are done. A small example using PyCrypto to encrypt a text with AES:

> python3.3
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from hashlib import sha256
>>> from Crypto import Random
>>> from Crypto.Cipher import AES
>>> from Crypto.Util import Counter
>>> pwd = "My secret password"
>>> key = sha256(pwd.encode()).digest()
>>> irv = Random.new().read(AES.block_size)
>>> icv = int.from_bytes(irv, "little")
>>> ctr = Counter.new(AES.block_size*8, initial_value=icv)
>>> cip = AES.new(key, AES.MODE_CTR, counter=ctr)
>>> msg = cip.encrypt('Attack at dawn'.encode())

To decrypt the message msg you need the password pwd or the key (the secret) and the initial counter value icv (can be public). The Python code to do this is left as an programming exercise for the reader (or you kan peek at pycryptex.py [src]).

Coding fonts

I consider myself a programmer, and to perform this task I rely on a set of tools. I need a text editor, compilers and/or run-times, build-systems, code repositories, documentation, and sometimes even an IDE. Since the late 80s my text editor of choice has been Emacs (and Aquamacs on OS X), but I am also comfortable with vi (including Vim) and ed. Currently, I am investigating time in newer text editors, like Sublime Text (writing editor plugins in Python should be fun) and Chocolat, but we are not close friends yet (but I will spend more time with subl to see how our relationship evolves). More on these tools later.

When programming, the code is viewed in a text editor (or IDE) in a font (the size and style of a particular typeface). The default fonts for most code editors are OK, but we can do better than OK. On OS X I have tried and used a lot of different fonts for coding. Since yesterday my preferred font for coding Python was Menlo Regular 13 (also used by Xcode):



I believe Apple's Menlo has evolved from DejaVu Sans Serif Mono (common on Linux) and Bitstream Vera Sans Mono. Previously, I used Monaco Regular 12, but I switched to Menlo since I find it less noisy. Yesterday I came over this announcement of the open source font Source Code Pro from Adobe (via Daring Fireball):



Now I have used Source Code Pro Regular 13 for one day programming in Python, and it is too early to conclude that the font is a good coding font option for me. My first impression is good. The font has a lightness to it that I like, and it is readable and clean. I will use it for a few more hours and then conclude if it should replace Menlo Regular 13 as my preferred coding font. I might even end up using different fonts depending on programming language and text editor. However, I recommend that you download and try this new open source font from Adobe.

Adobe also have other open source fonts available, including Source Sans Pro (revise and hosted on GitHub) and Kenten Generic.

Update 20140721: Other interesting coding fonts are Inconsolata and Fira Mono (from Mozilla's typeface Fira). This is Inconsolata:

And this is Fira Mono (Regular):

Update 20140805: You could also consider Input:

Input is a flexible system of fonts designed specifically for code by David Jonathan Ross. It offers both monospaced and proportional fonts, all with a large range of widths, weights, and styles for richer code formatting.

Generating a BibTeX file from Cristin data

Based on the previous script generating HTML from Cristin data, I created another script generating a BibTeX file from the same JSON data. The script has the same limitations as the previous one, but it should be easy to modify for your needs. Have fun!

Fetch and process information from Cristin

Yesterday I wrote a small Python script to fetch and generate a publication list for web-pages. I use a web-services (ws) provided by Cristin (Current research information system in Norway). Cristin is a research information system for hospitals, research institutes, and universities and university colleges. From the provided ws I use the method hentVarbeiderPerson (see Brukerdokumentasjon Cristin Web Service for details). The argument lopenr is set to the unique id of the researcher (who's publications I want to fetch). The argument format is set to json and the argument sortering is set to to AAR_PERSON_TITTEL (meaning sort the results by year, name and title). The query string then becomes (replace <ID> with the unique id of the user):

http://www.cristin.no/ws/hentVarbeiderPerson?lopenr=<ID>&format=json&sortering=AAR_PERSON_TITTEL

This query will return the publications in the JSON format. You can try this URL (with a real value for the id) in a browser or with command line programs like curl. The Python script processes it and generate HTML. The script is tailored towards publications registered in my name. However, it should be easy to modify the script to match your type of publications.  It reads the fetched JSON file from stdin and write the generated HTML to stdout. Errors and warnings are written to stderr. You should check the warnings. They might give you a hint of type of publications my script doesn't support.

In the returned data from Cristin each publication has two data sets. The "fellesdata" (common data, see commonmap in the script) part includes title, authors, year and some information on what type of publication it is. The "kategoridata" (category data, see catmap in the script) includes information about how the publication was published.  Based on these two sets of information I group the publications in different sections (like Web pages, Conference papers, Journal papers, and Reports, see tmap for details).  The order attribute specifies the order of the information inside each entry, and the porder attribute specifies the order of the sections (different type of publications). Have fun with it. The Python script is written for Python 3, but it should work with Python 2.

Almost two years in silence

I restart my blog today.  November 16th 2010 was my last post (on Beatles available in iTunes). Since many of my old post were dated, I've deleted them all.  The main reason for the silence is that other tools have replaced the purpose of the blog for me. I used the blog to remind myself on interesting software, web-pages, music and articles/text that I wanted to dig into later. Kippt and Instapaper has replaced the blog for these purposes.