5 An introduction to pyffi

The pyffi library makes it possible to use Python libraries from a Racket program.

A Racket program can start a Python process by requiring pyffi and calling initialize. After the initialization run and run* can be used to evaluate expressions and statements in the Python process.

> (require pyffi)

> (initialize)

> (post-initialize)

> (run "1+2")

3

Here Racket starts an embed Python process. The Python "1+2" is parsed, compiled and evaluated by Python. The resulting Python value 3 is then converted to the Racket value 3.

5.1 Atomic values: numbers, Booleans and None

Atomic Python values (numbers, Booleans and None) are automatically converted to their corresponding Racket values.

> (require pyffi)

> (initialize)

> (post-initialize)

> (run "12")

12

> (run "34.")

34.0

> (run "5+6j")

5.0+6.0i

> (run "False")

#f

> (run "True")

#t

> (list (run "None"))

'(#<void>)

5.2 Compound Values: strings, tuples, lists, and, dictionaries

Compound (non-atomic) Python values such as strings, tuples, lists and dicts are not converted to Racket values. Instead they are wrapped in a struct named obj. Due to a custom printer handler these wrapped values print nicely.

> (run "'Hello World'")

(obj "str" : 'Hello World')

> (run "(1,2,3)")

(obj "tuple" : (1, 2, 3))

> (run "[1,2,3]")

(obj "list" : [1, 2, 3])

> (run "{'a': 1, 'b': 2}")

(obj "dict" : {'a': 1, 'b': 2})

The values display nicely too:

> (displayln (run "'Hello World'"))

Hello World

> (displayln (run "(1,2,3)"))

(1, 2, 3)

> (displayln (run "[1,2,3]"))

[1, 2, 3]

> (displayln (run "{'a': 1, 'b': 2}"))

{'a': 1, 'b': 2}

Printing and displaying a Python object use the __repr__ and __str__ methods of the object respectively.

The idea is that Racket gains four new data types: pystring, pytuple, pylist and pydict.

To convert a compound value use pystring->string, pytuple->vector, pylist->list or pydict->hash.

> (pystring->string (run "'Hello World'"))

"Hello World"

> (pytuple->vector (run "(1,2,3)"))

'#(1 2 3)

> (pylist->list (run "[1,2,3]"))

'(1 2 3)

> (pydict->hash (run "{'a': 1, 'b': 2}"))

'#hash(("a" . 1) ("b" . 2))

Similarly, you can convert Racket values to Python ones.

> (string->pystring "Hello World")

(obj "str" : 'Hello World')

> (vector->pytuple #(1 2 3))

(obj "tuple" : (1, 2, 3))

> (list->pylist '(1 2 3))

(obj "list" : [1, 2, 3])

> (hash->pydict (hash "a" 1 "b" 2))

(obj "dict" : {'b': 2, 'a': 1})

It’s important to note that creating Python values using string->pystring, vector->pytuple, list->pylist and hash->pydict is much more efficient than using run. The overhead of run is due to the parsing and compiling of its input string. In contrast string->pystring and friends use the C API to create the Python values directly.

The data types have also have constructors:

> (pystring #\H #\e #\l #\l #\o)

(obj "str" : 'Hello')

> (pytuple 1 2 3)

(obj "tuple" : (1, 2, 3))

> (pylist 1 2 3)

(obj "list" : [1, 2, 3])

> (pydict "a" 1 "b" 2)

(obj "dict" : {'a': 1, 'b': 2})

The new types pystring, pytuple, pylist and pydict can be used with for.

> (for/list ([x (in-pystring (string->pystring "Hello"))]) x)

'(#\H #\e #\l #\l #\o)

> (for/list ([x (in-pytuple (vector->pytuple #(1 2 3)))]) x)

'(1 2 3)

> (for/list ([x (in-pylist (list->pylist '(1 2 3)))]) x)

'(1 2 3)

> (for/list ([(k v) (in-pydict (hash->pydict (hash "a" 1 "b" 2)))]) (list k v))

'(((obj "str" : 'b') 2) ((obj "str" : 'a') 1))

5.3 Builtin functions and modules

> (run* "x = 1+2")

Here the statement x = 1+2 is parsed, compiled and executed. The result of the expression 1+2 is stored in the global variable x.

But due to the overhead of run it is better to make a direct variable reference.

> main.x

3

Here main is the name we have given to the module used for the global namespace of the Python interpreter. The dotted identifier main.x thus references the variable x in the global namespace.

Since Python modules are first class values, we can see their printed representations:

> main

(obj "module" : <module '__main__' (built-in)>)

> builtins

(obj "module" : <module 'builtins' (built-in)>)

> (builtins.abs -7)

7

> (builtins.list "Hello")

(obj "list" : ['H', 'e', 'l', 'l', 'o'])

> (builtins.range 2 5)

(obj "range" : range(2, 5))

> (builtins.list (builtins.range 2 5))

(obj "list" : [2, 3, 4])

If you find the name builtins too long, then you can give it a new, shorter name.

> (define b builtins)

> (b.abs -7)

7

If you access the abs functions directly, you get a callable object:

> b.abs

(obj callable "builtin_function_or_method" : <built-in function abs>)

A callable object can be used just like a normal Racket function:

> (map b.abs '(1 -2 3 -4))

'(1 2 3 4)

To use functions from the Python standard library, you need to import it before you can use it. The standard library sys provide a lot of system information. Let’s use it to find the version of the Python interpreter.

> (import sys)

> sys.version_info

(obj "version_info" : sys.version_info(major=3, minor=10, micro=2, releaselevel='final', serial=0))

The list of modules in The Python Standard Library is long, so let’s just try one more.

We want to print a text calendar for the current month.

The expression (calendar.TextCalendar) instantiates a TextCalendar object. The syntax (object .method arg ...) is used to invoke the method formatmonth with the arguments 2022 and 7 (for July).

The documentation for formatmonth shows its signature:

formatmonth(theyear, themonth, w=0, l=0)

The two first arguments theyear and themonth are postional arguments and the two last arguments w and l are keyword arguments both has 0 has as default value.

The keyword argument w specifies the width of the date columns. We can get full names of the week days with a width of 9.

> (displayln ((calendar.TextCalendar) .formatmonth 2022 7 #:w 9))

July 2022   Monday   Tuesday  Wednesday  Thursday   Friday   Saturday   Sunday                                              1         2         3      4         5         6         7         8         9        10     11        12        13        14        15        16        17     18        19        20        21        22        23        24     25        26        27        28        29        30        31

5.4 Objects, Callable objects, Functions, Methods and Properties

All values in Python are represented as objects. This differs from Racket, where most data types (e.g. numbers and strings) aren’t objects.

In the data model used by Python, all objects have an identity, a type and a value. In practice the identity of a Python object is represented by its address in memory [the CPython implementation never moves objects].

To represent a Python object in Racket it is wrapped in an obj struct. The structure contains the type name as a string and a pointer to the object. The wrapper set the struct property gen:custom-write to display, write and print the wrapped objects nicely.

The result of repr() is used to write an object.
The result of str() is used to display an object.

> (define s (string->pystring "foo"))

> (repr s)

"'foo'"

> (writeln s)

(obj "str" : 'foo')

> (str s)

"foo"

> (displayln s)

foo

Functions and in general callable objects support the well-known syntax f(a,b,c). Such callable objects are wrapped in an callable-obj struct, which has obj as a super type. The callable-obj use the struct property prop:procedure to make the wrapper applicable.

> (run* "def f(x): return x+1")

> (define f main.f)

> f

(obj callable "function" : <function f at 0x115213e20>)

> (f 41)

42

Function calls with keywords work as expected. A Python keyword is simply prefixed with #: to turn it into a Racket keyword, as this example shows:

> (run* "def hello(name, title='Mr'): return 'Hello ' + title + ' ' + name")

> (displayln (main.hello "Foo"))

Hello Mr Foo

> (displayln (main.hello #:title "Mrs" "Bar"))

Hello Mrs Bar

In order to illustrate methods, let’s look at the Calendar class in the calendar module.

> (import calendar)

> calendar.Calendar

(obj callable "type" : <class 'calendar.Calendar'>)

Calling the class gives us an instance object. We pass 0 to make Monday the first week day.

> (calendar.Calendar #:firstweekday 0)

(obj "Calendar" : <calendar.Calendar object at 0x1184eb430>)

One of the methods of a calendar object is monthdatescalendar.

> (define cal (calendar.Calendar #:firstweekday 0))

> cal.monthdatescalendar

(obj callable "method" : <bound method Calendar.monthdatescalendar of <calendar.Calendar object at 0x118533b20>>)

The syntax obj.method gives us a bound method, which we can call. Bound methods are wrapped in method-obj to make them applicable.

However, we can also invoke the monthdatescalendar method directly with the help of the syntax (obj .method argument ...).

> (pyfirst (pyfirst (cal .monthdatescalendar year month)))

(obj "date" : datetime.date(2022, 8, 29))

Method invocations can be chained. That is, if a method call returns an object, we can invoke a method on it. The fist element of a list can be retrieved by the pop method, so we can replace the two calls to pyfirst with two invocations of .pop.

> (cal .monthdatescalendar year month  .pop 0 .pop 0)

(obj "date" : datetime.date(2022, 8, 29))

Besides methods an object can have properties (atributes). The syntax is obj.attribute. Most Python objects carry a little documentation in the oddly named __doc__ attribute.

> (displayln cal.__doc__)

Base calendar class. This class doesn't do any formatting. It simply     provides data to subclasses.

5.5 Exceptions

An exception on the Python side is converted to an exception on the Racket side. The exception will be printed with a full traceback.

> (run "1/0")

run: Python exception occurred;

ZeroDivisionError: division by zero

File "<string>", line 1, in <module>