Using PlistBuddy when I want to open Emacs from a link in an HTML file on OS X

Previously, I have discussed How to open Emacs from a link in an HTML file on Mac OS X. This post is just to give a complete overview of how this can be done without to many manual tasks. My complete AppleScript program and its Makefile is available for download. It should only be used with your own modification.  It includes the usage of cvs to commit the changes in the bib files to my local cvs-server and the establishment of a VPN connection if I am not on my local university network (since my cvs-server is not available outside the university network). This part should be completely recoded for your usage.

I use BSD make to compile and install the software, and I use PlistBuddy to manipulate the Info.plist file of the AppleScript application. The focus of this post is how to modify the Info.plist file of the AppleScript application using PlistBuddy. A lot of resources for PlistBuddy is available online: A Simple PlistBuddy Tutorial, How To Manage Plist Files With PlistBuddy, and OSX: Dealing with property list files.

In the previous post I explained the changes needed to be done in the Info.plist file of the application. The application is compiled using osacompile:

osacompile -o EmacsURL.app EmacsURL.applescript

The Info.plist is now located in the EmacsURL.app/Contents directory. To add the necessary properties in the Info.plist file we perform the following PlistBuddy commands (in the EmacsURL.app/Contents directory):

PlistBuddy -c 'Add :CFBundleIdentifier string com.apple.AppleScript.EmacsURL’ Info.plist
PlistBuddy -c 'Add :CFBundleURLTypes array’ Info.plist
PlistBuddy -c 'Add :CFBundleURLTypes:0:CFBundleURLName string "Emacs Helper"' Info.plist
PlistBuddy -c 'Add :CFBundleURLTypes:0:CFBundleURLSchemes array' Info.plist
PlistBuddy -c 'Add :CFBundleURLTypes:0:CFBundleURLSchemes:0 string emacs' Info.plist

First we add an identifier for the application (com.apple.AppleScript.EmacsURL). Then we add the URL-handling as an array that contains one dictionary with a name (Emacs Helper) and a scheme. The scheme is an array with a single string element (emacs). The consequence is that this application will handle the emacs URL protocol (handle URLs with the prefix emacs:). To activate this on OS X, the application should be run once after it is installed. For other details regarding compiling, modifying the Info.plist file, and installing the program, see the Makefile file.

Open Emacs from a link in an HTML file on Mac OS X

Since I do most of my writing in LaTeX, I have all my references (publications I cite when I write papers) in a series of bib-files (for BibTeX). To easier browse and search these references I have created a small Python script that generates a HTML document with all these references. Each reference also includes a link to the bib-file this reference was found in. In the link I include the key of the reference as a fragment identifier. And to specify that I want this link to be open in Emacs, I identify the URL protocol as emacs. The following is an example of such a URL:

emacs:///path/to/file/refs.bib#smith1999a

This link should open Emacs at the start of the reference with the key smith1999a in the file /path/to/file/mybib.bib. The next step is to create an application doing the necessary work and register the URL protocol emacs to activate this application. The easiest approach on a Mac is the write a short AppleScript Application. I have followed the instructions found at these two articles: Launch Scripts from Webpage Links and OS X URL handler to open links to local files. First I created the following AppleScript application:

on open location emacsURL

  -- Find file path and BibTeX key positions in URL
  set p to the offset of "///" in emacsURL
  set i to the offset of "#" in emacsURL
	
  -- File path
  set fn to text (p + 2) thru (i - 1) of emacsURL
	
  -- BibTeX key
  set bk to text (i + 1) thru -1 of emacsURL
	
  -- Line number of BibTeX key, e.g. 
  -- grep -n {smith199a, refs.bib | awk -F:'{ print $1 }'
  set ln to (do shell script "grep -n {" & bk
    & ", " & fn & " | awk -F: '{ print $1 }'")
	
  -- Open file in Emacs on correct line using 
  -- "+line:ln <filename>"
  if application "Emacs" is not running then
    tell application "Emacs" to activate
    delay 2 -- To ensure Emacs is ready
    do shell script "/usr/local/bin/emacsclient -n +"
      & ln & " " & fn
  else
    do shell script "/usr/local/bin/emacsclient -c -n +"
      & ln & " " & fn
  end if
	
end open location

From the Script Editor I saved the above AppelScript code as an Application. Then I had to edit the property list file of the application. In Finder, click on the application with the Control key held down and choose Show Package Contents. Go to the Contents folder and edit the Info.plist file (using Emacs or other suitable programs like Vim, Coda or Xcode). I changed the CFBundleIdentifier and added the following lines to register this application to handle the emacs URLs:

<key>CFBundleURLTypes</key>
<array>
  <dict>
    <key>CFBundleURLName</key>
    <string>Emacs Helper</string>
    <key>CFBundleURLSchemes</key>
    <array>
      <string>emacs</string>
    </array>
  </dict>
</array>

The application has to execute once the register (activate) the URL handling.

Automate tasks with AppleScript, Fake, and the keychain

You can make life a lot easier on your Mac if you learn how to automate things. The standard approach to do this on Macs are to use Automator (see How to use Automator: What Automator is and how it works from Macworld UK). If you include AppleScript, Fake, and the keychain in your toolchain, you can achieve even more. As an example I will develop an automated task to create a new email alias using a web portal. This example is used since it includes user input (email alias), access of password in keychain, filling in and submitting forms on web pages, and return data to the user though the clipboard. An example using Automator is also available.

AppleScript

The main parts of this example are an AppleScript script and a Fake workflow. We also expect that the password of the user is stored in the keychain [1]. The first thing we do is to fetch the password for the web portal from the keychain:

  set domain_name to "mydomain.com"
  set user_name to "myusername"
  set pwd_cmd to "security find-generic-password"
  set pwd_script to pwd_cmd & " -a " & user_name & " -w"
  set pass_wd to do shell script pwd_script

The next step is to prompt the user for the email alias (including converting the input to lower case [2]):

  set dialog_txt to "New email alias (<alias>@" & domain_name & "):"
  display dialog dialog_txt default answer ""
  set email_alias_case to text returned of result
  set email_alias to _string's lowerString(email_alias_case)

The typical use for the user of a new email alias is to type the new email address in to a web form. To make this easier for the user we'll copy the new email address to the clipboard:

  set the clipboard to email_alias & "@" & domain_name

The final part of the AppleScript is to execute the Fake workflow. We have to transfer three parameters (variables) to the workflow:

  tell application "Fake"
    set variable with name "emailAlias" to email_alias
    set variable with name "userName" to user_name
    set variable with name "passWd" to pass_wd
    activate
    open "Users:aa:Applications:MakeEmailAlias.fakeworkflow"
    delay 1
    run workflow with variables { \
      emailAlias:email_alias, \
      userName:user_name, \
      passWd:pass_wd \
    }
    wait until done
    close front document
    quit
  end tell

The only thing missing in the AppleScript code above is loading of the text string manipulation library _string.scpt [2]:

  set _string to load script alias ( \
    (path to library folder from user domain as text) & \
    "Scripts:Libraries:" & "_string.scpt")

Fake workflow

The Fake workflow receives 3 parameters (variables) from the AppleScript; the new email alias (emailAlias), the username (userName), and the password (passWd) to log in to the portal. In the Fake workflow we use these variables to fill in the correct values at web form. The following Fake workflow is an example, and it as to be updated based on the actual web portal you are using. The example workflow consists of 4 steps (and 10 Fake workflow actions):

1. Go to login web page:
   • Load URL: example.com
2. Log in with username and password (and wait a second):
   • Set Value of HTML Element:
     • with id: user
     • to: {userName}
   • Set Value of HTML Element:
     • with id: password
     • to: {passWd}
   • Click HTML Element:
     • for XPath: /html/body/div/...
   • Delay:
     • for: 1.0 seconds
3. Select web page where email aliases can be added:
   • Click HTML Link:
     • with text: email@mydomain.com
4. Add email alias (and wait tw seconds):
• Focus HTML Element:
  • with id: newalias
• Set Value of HTML Element:
  • with id: newalias
  • to: {emailAlias}
• Click HTML Element:
  • with id: submit
• Delay:
  • for: 2.0 seconds

In the example above we use different approaches to identify the elements on web pages (with id, for XPath, with text). In your case you should use the approach that is easiest for the web pages you are scripting. Fake provide a feature where you can drag the id of an element to the workflow action id value.

Notes

1. To store and access data in the keychain we use the security command line interface. To add a new account name with password pwd to the keychain you can do the following command:

     security add-generic-password -a name -s service -w pwd

   Then we can print this password with the following command:

     security find-generic-password -a name -w

   (In the first command service is a human readable name describing your service.)
2. In the examples above I expect the AppleScript library _string.scpt from Brati's Lover AppleScript library.

Including source code in papers and on the web

I've written many papers and a lot of web pages that includes source code. Since Python code was an important part of my Dr. thesis, I even wrote a tool to generate a pretty-printed Python code for LaTeX (this was written in 1997, but since I've been using it ever since it has had a few minor bug fixes since then; however, it needs Python 1.6 and it was the first Python program I ever wrote!). Just by accident (being sidetracked when I was checking out Middleman extensions) I rediscovered Pygments yesterday. I know I've seen it before, but I had forgotten all about it. This time I checked it out, and I think it will finally replace my home-written tool. Since I write most of my papers in LaTeX and I use BSDmake to do the magic, I have included a few new rules in my Makefile (using my old ltk.mk file):

PY2TEX =	pygmentize -f latex -P "verboptions=frame=single" -O linenos
SRCSTYLE =	srcstyle.sty
STYLETYPE =	default
MKSTYLE =	$(PY2TEX) -S $(STYLETYPE)

.include	"$(HOME)/lib/mk/ltx.mk"

$(LTXSRC):	$(SRCSTYLE)

$(SRCSTYLE):
		$(MKSTYLE) > $(.TARGET)

In the LaTeX document I have to include the generated style file and the fancyvrb package (and for most styles the color package). If the source code included in the document is in the file src/mypycode.py, then you will find this in the LaTeX document (ltx.mk expects to find source code files in the src directory):

\usepackage{fancyvrb}
\usepackage{color}
\usepackage{srcstyle}
...
\begin{figure}
  \begin{minipage}{0.9\linewidth}
    \input{src/mypycode.tex}
  \end{minipage}
  \caption{The caption text describing mypycode.}
\end{figure}

Pygments can be used to generate TeX, LaTeX, HTML and many other formats. It supports a wide range of programming languages, and can easily be extended to other formats and other programming languages. It is highly configurable and flexible, and the command line way of using it is the simple approach. For full control and flexibility you write your own tool using Pygments as a Python library (and it supports both Python 2 and 3).

The styles, programming languages, formats, and filters available can easily be listed with these commands:

pygmentize -L styles		# styles
pygmentize -L lexers		# programming languages
pygmentize -L formatters	# formats
pygmentize -L filters		# filters
pygmentize -L			# all

Generating a BibTeX file from Cristin data

Based on the previous script generating HTML from Cristin data, I created another script generating a BibTeX file from the same JSON data. The script has the same limitations as the previous one, but it should be easy to modify for your needs. Have fun!

Fetch and process information from Cristin

Yesterday I wrote a small Python script to fetch and generate a publication list for web-pages. I use a web-services (ws) provided by Cristin (Current research information system in Norway). Cristin is a research information system for hospitals, research institutes, and universities and university colleges. From the provided ws I use the method hentVarbeiderPerson (see Brukerdokumentasjon Cristin Web Service for details). The argument lopenr is set to the unique id of the researcher (who's publications I want to fetch). The argument format is set to json and the argument sortering is set to to AAR_PERSON_TITTEL (meaning sort the results by year, name and title). The query string then becomes (replace <ID> with the unique id of the user):

http://www.cristin.no/ws/hentVarbeiderPerson?lopenr=<ID>&format=json&sortering=AAR_PERSON_TITTEL

This query will return the publications in the JSON format. You can try this URL (with a real value for the id) in a browser or with command line programs like curl. The Python script processes it and generate HTML. The script is tailored towards publications registered in my name. However, it should be easy to modify the script to match your type of publications.  It reads the fetched JSON file from stdin and write the generated HTML to stdout. Errors and warnings are written to stderr. You should check the warnings. They might give you a hint of type of publications my script doesn't support.

In the returned data from Cristin each publication has two data sets. The "fellesdata" (common data, see commonmap in the script) part includes title, authors, year and some information on what type of publication it is. The "kategoridata" (category data, see catmap in the script) includes information about how the publication was published.  Based on these two sets of information I group the publications in different sections (like Web pages, Conference papers, Journal papers, and Reports, see tmap for details).  The order attribute specifies the order of the information inside each entry, and the porder attribute specifies the order of the sections (different type of publications). Have fun with it. The Python script is written for Python 3, but it should work with Python 2.