8 Datatypes

In Python all values are stored as objects. An object has an identity, a type and a value.

The identity of an object is given by the object’s address in memory. The is operator in Python corresponds to eq? in Racket.

Objects whose value can change are mutable. If the value of an object can’t change the object is called immutable. Note: In Racket an immutable vector can contain (mutable) boxes. Even though the vector is immutable the contents of the boxes can change. What doesn’t change is the identity of the boxes in the vector. In the same way an immutable Python container might contain mutable objects.

In Python numbers, strings and tuples are immutable. Python dictionaries and lists are mutable.

Python objects aren’t explicitly destroyed. When an object becomes unreachable, the memory of an object is reclaimed by the garbage collector. The current Python interpreter uses reference counting to keep track of an objects reachability.

When a Python object is returned from Python to Racket as a new reference, the Python system won’t deallocate the object until Racket says it’s okay to do so. Since we don’t want manually to keep track of the lifetime of Python objects, the low-level C-bindings register all new references will a will executor. When the Racket garbage collector detects a value is unreachable, the garbage collector will execute any wills associated with the value. The will executor we are using, simply decrements the reference count of the Python object.

If you stick to the high level functions in pyffi you don’t need to worry about reference counting. However it might be relevant if you need to use the low-level C-API.

8.1 Python Lists - pylist

Despite the name a Python list is not a singly linked list, but an array. Objects with the type "list" will be called pylist to tell them apart from standard Racket lists.

The operations pylist, list->pylist and vector->pylist can be used to construct pylists from Racket values.

In in for-loops, use in-pylist to iterate through the elements.

> (pylist 1 2 3)

(obj "list" : [1, 2, 3])

> (pylist? (pylist 1 2 3))

#t

> (pylist? (pylist))

#t

> (pylist? '(1 2 3))

#f

> (pylist 1 2 3 4)

(obj "list" : [1, 2, 3, 4])

> (pylist)

(obj "list" : [])

> (pylist (pylist 1 2 3 4) 5 (pylist 6 7))

(obj "list" : [[1, 2, 3, 4], 5, [6, 7]])

This function takes constant time.

> (pylist-ref (pylist "a" "b" "c" "d") 1)

"b"

> (pyfirst (pylist "a" "b" "c" "d"))

"a"

> (pysecond (pylist "a" "b" "c" "d"))

"b"

This function takes constant time.

> (define xs (pylist 0 1 2 3 4))

> (pylist-set! xs 1 #t)

> xs

(obj "list" : [0, True, 2, 3, 4])

> (list->pylist '(1 2 3 4))

(obj "list" : [1, 2, 3, 4])

> (list->pylist '(1 "foo" #(3 4)))

(obj "list" : [1, 'foo', (3, 4)])

> (list->pylist '(1 (2 3) #(4 (5 6))))

(obj "list" : [1, [2, 3], (4, [5, 6])])

> (vector->pylist '#(1 2 3 4))

(obj "list" : [1, 2, 3, 4])

> (vector->pylist '#(1 "foo" #(3 4)))

(obj "list" : [1, 'foo', (3, 4)])

> (vector->pylist '#(1 (2 3) #(4 (5 6))))

(obj "list" : [1, [2, 3], (4, [5, 6])])

> (pylist-length (pylist 1 2 3))

3

This function takes time proportional to the size of xs.

> (pylist->list (pylist 1 2 3 #t #f "a"))

'(1 2 3 #t #f (obj "str" : 'a'))

This function takes time proportional to the size of xs.

> (pylist->vector (pylist 1 2 3 #t #f "a"))

'#(1 2 3 #t #f (obj "str" : 'a'))

This function takes time proportional to the size of xs.

> (pylist->pytuple (pylist 1 2 3 #t #f "a"))

(obj "tuple" : (1, 2, 3, True, False, 'a'))

> (define xs (pylist 0 1 2 3))

> (for/list ([x (in-pylist xs)])     x)

'(0 1 2 3)

Worst case this function takes time proportional to the size of xs.

> (define xs (pylist 0 1 2 3))

> (pylist-insert! xs 2 #t)

> xs

(obj "list" : [0, 1, True, 2, 3])

> (define xs (pylist 10 11 12 13))

> (pylist-length xs)

4

> (pylist-append-item! xs 14)

> xs

(obj "list" : [10, 11, 12, 13, 14])

> (pylist-length xs)

5

> (define xs (pylist 1 2 3 4))

> (pylist-reverse! xs)

> xs

(obj "list" : [4, 3, 2, 1])

> (define xs (pylist 3 2 4 1))

> (pylist-sort! xs)

> xs

(obj "list" : [1, 2, 3, 4])

> (define ys (pylist 3 #t 2 4 #f #f 1))

> (pylist-sort! ys)

> ys

(obj "list" : [False, False, True, 1, 2, 3, 4])

In Python notation: list[low:high].

> (define xs (pylist 1 2 3 #t #f "a"))

> (pylist-get-slice xs 1 3)

(obj "list" : [2, 3])

8.2 Python Tuples - pytuple

Python tuples correspond to immutable Racket vectors.

Even though there is no datastructure in Racket called "tuple", Python tuples will have the name "putuple" to match the names of pylist and pydict.

The operations pytuple, list->pytuple and vector->pytuple can be used to construct pytuples from Racket values.

In in for-loops, use in-pytuple to iterate through the elements.

> (pytuple 1 2 3)

(obj "tuple" : (1, 2, 3))

> (pytuple? (pytuple 1 2 3))

#t

> (pytuple? (pytuple))

#t

> (pytuple? '(1 2 3))

#f

> (pytuple 1 2 3 4)

(obj "tuple" : (1, 2, 3, 4))

> (pytuple)

(obj "tuple" : ())

> (pytuple (pytuple 1 2 3 4) 5 (pytuple 6 7))

(obj "tuple" : ((1, 2, 3, 4), 5, (6, 7)))

This function takes constant time.

> (pytuple-ref (pytuple "a" "b" "c" "d") 1)

"b"

> (list->pytuple '(1 2 3 4))

(obj "tuple" : (1, 2, 3, 4))

> (list->pytuple '(1 "foo" #(3 4)))

(obj "tuple" : (1, 'foo', (3, 4)))

> (list->pytuple '(1 (2 3) #(4 (5 6))))

(obj "tuple" : (1, [2, 3], (4, [5, 6])))

> (vector->pytuple '#(1 2 3 4))

(obj "tuple" : (1, 2, 3, 4))

> (vector->pytuple '#(1 "foo" #(3 4)))

(obj "tuple" : (1, 'foo', (3, 4)))

> (vector->pytuple '#(1 (2 3) #(4 (5 6))))

(obj "tuple" : (1, [2, 3], (4, [5, 6])))

> (pytuple-length (pytuple 1 2 3))

3

This function takes time proportional to the size of xs.

> (pytuple->list (pytuple 1 2 3 #t #f "a"))

'(1 2 3 #t #f (obj "str" : 'a'))

This function takes time proportional to the size of xs.

> (pytuple->vector (pytuple 1 2 3 #t #f "a"))

'#(1 2 3 #t #f (obj "str" : 'a'))

This function takes time proportional to the size of xs.

> (define xs (pytuple->immutable-vector (pytuple 1 2 3 #t #f "a")))

> xs

'#(1 2 3 #t #f (obj "str" : 'a'))

> (immutable? xs)

#t

This function takes time proportional to the size of xs.

> (pytuple->pylist (pytuple 1 2 3 #t #f "a"))

(obj "list" : [1, 2, 3, True, False, 'a'])

> (define xs (pytuple 0 1 2 3))

> (for/list ([x (in-pytuple xs)])     x)

'(0 1 2 3)

In Python notation: list[low:high].

> (define xs (pytuple 1 2 3 #t #f "a"))

> (pytuple-get-slice xs 1 3)

(obj "tuple" : (2, 3))

8.3 Python Dictionaries - pydict

Dictionaries in Python are associative arrays indexed by keys. Given a key one can lookup a value. Think of dictionaries as sets of key/value pairs.

Any immutable value can be used as a key, so strings, numbers and tuples can always be used as keys.

The corresponding data structure in Racket is the hash table.

Use the operations pydict->hash and hash->pydict to convert back and forth between pydicts and hash tables.

> (pydict? (hash->pydict (hash "a" 1  "b" 2)))

#t

> (hash->pydict (hash "a" 1  "b" 2))

(obj "dict" : {'b': 2, 'a': 1})

> (hash->pydict (hash 1 "x"  2 "y"))

(obj "dict" : {1: 'x', 2: 'y'})

> (hash->pydict (hash #(1 2) "tuple used as key"))

(obj "dict" : {(1, 2): 'tuple used as key'})

The function convert-key is used to convert the keys from Python values to Racket ones.
The function convert-value is used to convert the values.

The default value for the key conversion is pr/key.
The default value for the value conversion is pr.

> (pydict->hash (hash->pydict (hash "a" 1  "b" 2)))

'#hash(("a" . 1) ("b" . 2))

> (pydict->hash (hash->pydict (hash 1 "x"  2 "y")))

'#hash((1 . (obj "str" : 'x')) (2 . (obj "str" : 'y')))

> (pydict->hash (hash->pydict (hash #(1 2) "tuple used as key")))

'#hash((#(1 2) . (obj "str" : 'tuple used as key')))

The key to val mappings are added to the table in the order that they appear in the argument list, so later mappings can hide earlier mappings if the keys are equal.

The default value for the key and value conversion is rp.

> (pydict "a" 1  "b" 2)

(obj "dict" : {'a': 1, 'b': 2})

> (pydict 1 "x"  2 "y")

(obj "dict" : {1: 'x', 2: 'y'})

> (pydict #(1 2) "tuple used as key")

(obj "dict" : {(1, 2): 'tuple used as key'})

If failure-result is a procedure, it is called (through a tail call) with no arguments to produce the result.

Otherwise, failure-result is returned as the result.

> (define d (pydict "a" 1  "b" 2))

> d

(obj "dict" : {'a': 1, 'b': 2})

> (pydict-ref d "a")

1

> (pydict-ref d "b")

2

> (pydict-ref d "c")

pydict-ref: no value found for key

key: "c"

> (define d (pydict "a" 1))

> (pydict-set! d "a" 11)

> (pydict-set! d "b" 22)

> d

(obj "dict" : {'a': 11, 'b': 22})

> (define d (pydict "a" 1 "b" 2))

> (pydict-remove! d "b")

> (pydict-remove! d "c")

> d

(obj "dict" : {'a': 1})

> (define d (pydict "a" 1  "b" 2))

> d

(obj "dict" : {'a': 1, 'b': 2})

> (pydict-clear! d)

> d

(obj "dict" : {})

> (define d (pydict "a" 1  "b" 2))

> (pydict-contains? d "a")

#t

> (pydict-contains? d "x")

#f

> (define d1 (pydict "a" 1  "b" 2))

> (define d2 (pydict-copy d1))

> (list d1 d2)

'((obj "dict" : {'a': 1, 'b': 2}) (obj "dict" : {'a': 1, 'b': 2}))

> (pydict-set! d1 "a" 11)

> (list d1 d2)

'((obj "dict" : {'a': 11, 'b': 2}) (obj "dict" : {'a': 1, 'b': 2}))

> (define d (pydict "a" 1  "b" 2))

> (pydict-keys d)

(obj "list" : ['a', 'b'])

> (define d (pydict "a" 1  "b" 2 "c" 1))

> (pydict-values d)

(obj "list" : [1, 2, 1])

> (define d (pydict "a" 1  "b" 2 "c" 1))

> (pydict-count d)

3

If a key k is present in both pydicts and is mapped to values v1 and v2 in d1 and d2 respectively, then k is mapped to v2 if override? is true, and mapped to v1 otherwise.

> (define d1 (pydict "a" 1  "b" 2))

> (define d2 (pydict "b" 22 "c" 33))

> (pydict-merge! d1 d2)

> d1

(obj "dict" : {'a': 1, 'b': 22, 'c': 33})

> (define d1 (pydict "a" 1  "b" 2))

> (define d2 (pydict "b" 22 "c" 33))

> (pydict-merge! d1 d2 #f)

> d1

(obj "dict" : {'a': 1, 'b': 2, 'c': 33})

> (define d (pydict "a" 1 "b" 2))

> (for/list ([(key value) (in-pydict d)])     (list key value))

'(((obj "str" : 'a') 1) ((obj "str" : 'b') 2))

8.4 Python Strings - pystring

Strings in Python are represented as objects with type str. Python strings are immutable sequences of Unicode code points.

There is no character type in Python, so various Python libraries use strings of length 1 to represent characters.

> (string->pystring "foo")

(obj "str" : 'foo')

> (pystring? (string->pystring "foo"))

#t

> (pystring #\f #\o #\o)

(obj "str" : 'foo')

> (string->pystring "foo")

(obj "str" : 'foo')

> (pystring->string (pystring #\f #\o #\o))

"foo"

> (pystring-length (pystring #\f #\o #\o))

3

> (define foo (pystring #\f #\o #\o))

> (pystring-ref foo 0)

#\f

> (pystring-ref foo 10)

pystring-ref: index 10 out of range for the string "foo"

The first position in a string corresponds to 0, so the start and end arguments must be less than or equal to the length of x, and end must be greater than or equal to start, otherwise the exception exn:fail:contract is raised.

> (subpystring (string->pystring "foobarbaz") 3 6)

(obj "str" : 'bar')

> (define x (string->pystring "foo"))

> (for/list ([c (in-pystring x)])     c)

'(#\f #\o #\o)

8.5 Python Generators

Python generator correspond to Racket generators from racket/generator. Think of a generator as a function that can produce a series of values.

If the body of Python function contains a yield statement, calling the function returns a generator object. Use next repeatedly on the generator to generate the series of values.

The yield expr statement both "pauses" the computation of the generator and returns the result of evaluating the expression. The computation is resumed by next.

This example shows a generator that produces the natural numbers.

This produces the list (list 1 2 3).

Usually the most convenient way of using such a generator is to use in-pygenerator.

Generators are automatically wrapped in an generator-obj struct, which has obj as super type. The wrapping allows us to make the in-generator implicit in for loops.