Open Emacs from a link in an HTML file on Mac OS X

Since I do most of my writing in LaTeX, I have all my references (publications I cite when I write papers) in a series of bib-files (for BibTeX). To easier browse and search these references I have created a small Python script that generates a HTML document with all these references. Each reference also includes a link to the bib-file this reference was found in. In the link I include the key of the reference as a fragment identifier. And to specify that I want this link to be open in Emacs, I identify the URL protocol as emacs. The following is an example of such a URL:

emacs:///path/to/file/refs.bib#smith1999a

This link should open Emacs at the start of the reference with the key smith1999a in the file /path/to/file/mybib.bib. The next step is to create an application doing the necessary work and register the URL protocol emacs to activate this application. The easiest approach on a Mac is the write a short AppleScript Application. I have followed the instructions found at these two articles: Launch Scripts from Webpage Links and OS X URL handler to open links to local files. First I created the following AppleScript application:

on open location emacsURL

  -- Find file path and BibTeX key positions in URL
  set p to the offset of "///" in emacsURL
  set i to the offset of "#" in emacsURL
	
  -- File path
  set fn to text (p + 2) thru (i - 1) of emacsURL
	
  -- BibTeX key
  set bk to text (i + 1) thru -1 of emacsURL
	
  -- Line number of BibTeX key, e.g. 
  -- grep -n {smith199a, refs.bib | awk -F:'{ print $1 }'
  set ln to (do shell script "grep -n {" & bk
    & ", " & fn & " | awk -F: '{ print $1 }'")
	
  -- Open file in Emacs on correct line using 
  -- "+line:ln <filename>"
  if application "Emacs" is not running then
    tell application "Emacs" to activate
    delay 2 -- To ensure Emacs is ready
    do shell script "/usr/local/bin/emacsclient -n +"
      & ln & " " & fn
  else
    do shell script "/usr/local/bin/emacsclient -c -n +"
      & ln & " " & fn
  end if
	
end open location

From the Script Editor I saved the above AppelScript code as an Application. Then I had to edit the property list file of the application. In Finder, click on the application with the Control key held down and choose Show Package Contents. Go to the Contents folder and edit the Info.plist file (using Emacs or other suitable programs like Vim, Coda or Xcode). I changed the CFBundleIdentifier and added the following lines to register this application to handle the emacs URLs:

<key>CFBundleURLTypes</key>
<array>
  <dict>
    <key>CFBundleURLName</key>
    <string>Emacs Helper</string>
    <key>CFBundleURLSchemes</key>
    <array>
      <string>emacs</string>
    </array>
  </dict>
</array>

The application has to execute once the register (activate) the URL handling.

Non breakable spaces in Word

I prefer TeX and friends (LaTeX, MetaPost, BibTeX) when writing papers. But sometimes I have to use Microsoft Word. One annoyance I have observed in many documents written in Word is line breaks (word wrapping) on wrong places. Not all spaces are a correct place to break lines in a text. In TeX I would write the tilde (~) character instead of a space, and in HTML I would use the character entity  . This would create a space were a line break will not occur. However, this is also easily possible in Microsoft Word. Instead of inserting the space by typing on the Spacebar, press Ctrl-Shift-Spacebar, and a non breakable space is inserted in your document.

Cryptography with Python 3

In 2012 I posted a post on using PyCrypto with Python 3 and some AES and RSA examples. Now, I usually use the cryptography Python library (implemented for both Python 2 and 3). I have ported all my PyCrypto examples from 2012 (see the README file) to the cryptography library.

pycryptex.py / pycryptex-cbc.py

pycryptex.py [src] is a small example using AES to encrypt and decrypt a text:

> python3 pycryptex.py

The example includes two versions, one using the high level Fernet class and the other using the more low level hazmat functions. In pycryptex-cbc.py [src] the second version is implemented using CBC mode (where padding is necessary):

> python3 pycryptex-cbc.py

pycrypto-mkkey.py / pycrypto-encrypt.py / pycrypto-decrypt.py

An example with three programs. pycrypto-mkkey.py [src] is used to generatea RSA key-pair. To generate an RSA key pair stored in the file k1 and protected with the password "passwd" is done with the following command (the public key is stored in the k1.pub file):

> python3 pycrypto-mkkey.py k1 "passwd"

pycrypto-encrypt.py [src] generates an AES key and use this key to encrypt plaintext data read from stdin (README in the example below). The ciphertext is written to stdout (CIPHER in the example below). The AES key is encrypted using the public RSA key k1.pub generated above and then saved to file k2 (no password needed since the public key is not password protected):

> python3 pycrypto-encrypt.py k1.pub k2 < README > CIPHER

pycrypto-decrypt.py [src] reads the encrypted AES key k2 end decrypts it using the RSA key k1 (k1 is protected with the password "passwd"). It then use the AES key to decrypt the ciphertext data read from stdin (CIPHER in the example below). The plaintext is written to stdout:

> python3 pycrypto-decrypt.py k1 k2 "passwd" < CIPHER

pwsec-server.py / pwsec-client.py

An example with two programs, a server pwsec-server.py [src] and a client pwsec-client.py [src]. The example demonstrates secure communication using AES. The shared key is generated from a password (the shared secret). We are using CTR mode, and the initial value (for the counter) is sent first in the first message. First start the server then the client:

> python3 pwsec-server.py localhost 3456 "mypass" &
> python3 pwsec-client.py localhost 3456 "mypass"

These two programs are using the tcp module from NOOP project (currently, only a few of the modules from the NOOP project are released, May 2015).

pubsec-send.py / pubsec-receive.py

An example with two programs, a sender pubsec-send.py [src] and a receiver pubsec-receive.py [src]. The example demonstrates secure communication using a combination of RSA and AES. The sender use the public RSA key of the receiver to encrypt the first message sent to the receiver. This message contains the shared secret AES key of the session. Then the sender sends a message encrypted with this key. First start the receiver then the sender:

> python3 pubsec-receive.py k1 localhost 3456 "passwd" &
> python3 pubsec-send.py k1.pub localhost 3456

These two programs are also using the tcp module from NOOP project.

Note

This code is not meant to be robust. All error checking is ignored.

Run automator workflow with keyboard shortcuts

In the last post we demonstrated how to automate a task using AppleScript and the Fake scriptable web browser. Often, I use automator included on every Mac to perform such task. To activate them, we bind them to a keyboard shortcut in the given application. The example we will use today is to press the a keyboard shortcut to load the remote content of an email in Mail.app (you have of course turned off loading all remote content in emails by unselecting Load remote content in messages in the Viewing part of Mail.app Preferences).

Create the automator workflow

We start by creating the automator workflow:

1. Start Automator and select New Document.
2. Choose the type of workflow to be Services.
3. Choose that service receives no input in Mail.app.
4. Open Mail.app and open an email with (unloaded) remote content.
5. In Automator, press the Record button.
6. In Mail.app, press the Load Remote Content button.
7. In Automator, terminate recording.
8. Save the service workflow with the name Load Remote Content.

You should now be able to run this from the Automator to test it (try it by selecting a message with remote content, where the remote content is not loaded). Before you test it be sure that you have given Automator (and Mail.app) access to control your computer. You do that in System Preferences (Security & Privacy → Privacy → Accessibility).

The keyboard shortcut

You should check that the Service you have created is visible in the Mail → Services menu (the Load Remote Content menu is there). If it is there, let us create the keyboard shortcut for it. Open System Preferences and select Keyboard → Shortcut. In App Shortcut add a new with the plus (+) button. Choose Mail.app for the Application, and in the menu title type exactly as it was written above: Load Remote Content. Choose your Keyboard Shortcut (I used Shift-Control-Cmd-I) and press Add.

Now, whenever you read as message with (unloaded) remote content, pressing the keyboard shortcut will load the content in the message.

Automate tasks with AppleScript, Fake, and the keychain

You can make life a lot easier on your Mac if you learn how to automate things. The standard approach to do this on Macs are to use Automator (see How to use Automator: What Automator is and how it works from Macworld UK). If you include AppleScript, Fake, and the keychain in your toolchain, you can achieve even more. As an example I will develop an automated task to create a new email alias using a web portal. This example is used since it includes user input (email alias), access of password in keychain, filling in and submitting forms on web pages, and return data to the user though the clipboard. An example using Automator is also available.

AppleScript

The main parts of this example are an AppleScript script and a Fake workflow. We also expect that the password of the user is stored in the keychain [1]. The first thing we do is to fetch the password for the web portal from the keychain:

  set domain_name to "mydomain.com"
  set user_name to "myusername"
  set pwd_cmd to "security find-generic-password"
  set pwd_script to pwd_cmd & " -a " & user_name & " -w"
  set pass_wd to do shell script pwd_script

The next step is to prompt the user for the email alias (including converting the input to lower case [2]):

  set dialog_txt to "New email alias (<alias>@" & domain_name & "):"
  display dialog dialog_txt default answer ""
  set email_alias_case to text returned of result
  set email_alias to _string's lowerString(email_alias_case)

The typical use for the user of a new email alias is to type the new email address in to a web form. To make this easier for the user we'll copy the new email address to the clipboard:

  set the clipboard to email_alias & "@" & domain_name

The final part of the AppleScript is to execute the Fake workflow. We have to transfer three parameters (variables) to the workflow:

  tell application "Fake"
    set variable with name "emailAlias" to email_alias
    set variable with name "userName" to user_name
    set variable with name "passWd" to pass_wd
    activate
    open "Users:aa:Applications:MakeEmailAlias.fakeworkflow"
    delay 1
    run workflow with variables { \
      emailAlias:email_alias, \
      userName:user_name, \
      passWd:pass_wd \
    }
    wait until done
    close front document
    quit
  end tell

The only thing missing in the AppleScript code above is loading of the text string manipulation library _string.scpt [2]:

  set _string to load script alias ( \
    (path to library folder from user domain as text) & \
    "Scripts:Libraries:" & "_string.scpt")

Fake workflow

The Fake workflow receives 3 parameters (variables) from the AppleScript; the new email alias (emailAlias), the username (userName), and the password (passWd) to log in to the portal. In the Fake workflow we use these variables to fill in the correct values at web form. The following Fake workflow is an example, and it as to be updated based on the actual web portal you are using. The example workflow consists of 4 steps (and 10 Fake workflow actions):

1. Go to login web page:
   • Load URL: example.com
2. Log in with username and password (and wait a second):
   • Set Value of HTML Element:
     • with id: user
     • to: {userName}
   • Set Value of HTML Element:
     • with id: password
     • to: {passWd}
   • Click HTML Element:
     • for XPath: /html/body/div/...
   • Delay:
     • for: 1.0 seconds
3. Select web page where email aliases can be added:
   • Click HTML Link:
     • with text: email@mydomain.com
4. Add email alias (and wait tw seconds):
• Focus HTML Element:
  • with id: newalias
• Set Value of HTML Element:
  • with id: newalias
  • to: {emailAlias}
• Click HTML Element:
  • with id: submit
• Delay:
  • for: 2.0 seconds

In the example above we use different approaches to identify the elements on web pages (with id, for XPath, with text). In your case you should use the approach that is easiest for the web pages you are scripting. Fake provide a feature where you can drag the id of an element to the workflow action id value.

Notes

1. To store and access data in the keychain we use the security command line interface. To add a new account name with password pwd to the keychain you can do the following command:

     security add-generic-password -a name -s service -w pwd

   Then we can print this password with the following command:

     security find-generic-password -a name -w

   (In the first command service is a human readable name describing your service.)
2. In the examples above I expect the AppleScript library _string.scpt from Brati's Lover AppleScript library.

Including source code in papers and on the web

I've written many papers and a lot of web pages that includes source code. Since Python code was an important part of my Dr. thesis, I even wrote a tool to generate a pretty-printed Python code for LaTeX (this was written in 1997, but since I've been using it ever since it has had a few minor bug fixes since then; however, it needs Python 1.6 and it was the first Python program I ever wrote!). Just by accident (being sidetracked when I was checking out Middleman extensions) I rediscovered Pygments yesterday. I know I've seen it before, but I had forgotten all about it. This time I checked it out, and I think it will finally replace my home-written tool. Since I write most of my papers in LaTeX and I use BSDmake to do the magic, I have included a few new rules in my Makefile (using my old ltk.mk file):

PY2TEX =	pygmentize -f latex -P "verboptions=frame=single" -O linenos
SRCSTYLE =	srcstyle.sty
STYLETYPE =	default
MKSTYLE =	$(PY2TEX) -S $(STYLETYPE)

.include	"$(HOME)/lib/mk/ltx.mk"

$(LTXSRC):	$(SRCSTYLE)

$(SRCSTYLE):
		$(MKSTYLE) > $(.TARGET)

In the LaTeX document I have to include the generated style file and the fancyvrb package (and for most styles the color package). If the source code included in the document is in the file src/mypycode.py, then you will find this in the LaTeX document (ltx.mk expects to find source code files in the src directory):

\usepackage{fancyvrb}
\usepackage{color}
\usepackage{srcstyle}
...
\begin{figure}
  \begin{minipage}{0.9\linewidth}
    \input{src/mypycode.tex}
  \end{minipage}
  \caption{The caption text describing mypycode.}
\end{figure}

Pygments can be used to generate TeX, LaTeX, HTML and many other formats. It supports a wide range of programming languages, and can easily be extended to other formats and other programming languages. It is highly configurable and flexible, and the command line way of using it is the simple approach. For full control and flexibility you write your own tool using Pygments as a Python library (and it supports both Python 2 and 3).

The styles, programming languages, formats, and filters available can easily be listed with these commands:

pygmentize -L styles		# styles
pygmentize -L lexers		# programming languages
pygmentize -L formatters	# formats
pygmentize -L filters		# filters
pygmentize -L			# all