The distance between

Squeeze my hand
let me feel strength in your fingers
Squeeze my hand
prove to me that you are more than a cold lump of
clay - Squeeze my hand show
me that you still have
feeling Squeeze my hand because you
can still hear me, feel me and I can still
hear you, feel you Squeeze my hand
goddammit Squeeze

-A.Badger 2017

 

Advertisements

Python testing: Asserting raw byte output with pytest

The Code to Test

When writing code that can run on both Python2 and Python3, I’ve sometimes found that I need to send and receive bytes to stdout. Here’s some typical code I might write to do that:

# byte_writer.py
import sys
import six

def write_bytes(some_bytes):
    # some_bytes must be a byte string
    if six.PY3:
        stdout = sys.stdout.buffer
    else:
        stdout = sys.stdout
    stdout.write(some_bytes)

if __name__ == '__main__':
    write_bytes(b'\xff')

In this example, my code needs to write a raw byte to stdout. To do this, it uses sys.stdout.buffer on Python3 to circumvent the automatic encoding/decoding that occurs on Python3’s sys.stdout. So far so good. Python2 expects bytes to be written to sys.stdout by default so we can write the byte string directly to sys.stdout in that case.

The First Attempt: Pytest newb, but willing to learn!

Recently I wanted to write a unittest for some code like that. I had never done this in pytest before so my first try looked a lot like my experience with nose or unittest2: override sys.stdout with an io.BytesIO object and then assert that the right values showed up in sys.stdout:

# test_byte_writer.py
import io
import sys

import mock
import pytest
import six

from byte_writer import write_bytes

@pytest.fixture
def stdout():
    real_stdout = sys.stdout
    fake_stdout = io.BytesIO()
    if six.PY3:
        sys.stdout = mock.MagicMock()
        sys.stdout.buffer = fake_stdout
    else:
        sys.stdout = fake_stdout

    yield fake_stdout

    sys.stdout = real_stdout

def test_write_bytes(stdout):
    write_bytes()
    assert stdout.getvalue() == b'a'

This gave me an error:

[pts/38@roan /var/tmp/py3_coverage]$ pytest              (07:46:36)
_________________________ test_write_byte __________________________

stdout = 

    def test_write_byte(stdout):
        write_bytes(b'a')
>       assert stdout.getvalue() == b'a'
E       AssertionError: assert b'' == b'a'
E         Right contains more items, first extra item: 97
E         Use -v to get the full diff

test_byte_writer.py:27: AssertionError
----------------------- Captured stdout call -----------------------
a
===================== 1 failed in 0.03 seconds =====================

I could plainly see from pytest’s “Captured stdout” output that my test value had been printed to stdout. So it appeared that my stdout fixture just wasn’t capturing what was printed there. What could be the problem? Hmmm…. Captured stdout… If pytest is capturing stdout, then perhaps my fixture is getting overridden by pytest’s internal facility. Let’s google and see if there’s a solution.

The Second Attempt: Hey, that’s really neat!

Wow, not only did I find that there is a way to capture stdout with pytest, I found that you don’t have to write your own fixture to do so. You can just hook into pytest’s builtin capfd fixture to do so. Cool, that should be much simpler:

# test_byte_writer.py
import io
import sys

from byte_writer import write_bytes

def test_write_byte(capfd):
    write_bytes(b'a')
    out, err = capfd.readouterr()
    assert out == b'a'

Okay, that works fine on Python2 but on Python3 it gives:

[pts/38@roan /var/tmp/py3_coverage]$ pytest              (07:46:41)
_________________________ test_write_byte __________________________

capfd = 

    def test_write_byte(capfd):
        write_bytes(b'a')
        out, err = capfd.readouterr()
>       assert out == b'a'
E       AssertionError: assert 'a' == b'a'

test_byte_writer.py:10: AssertionError
===================== 1 failed in 0.02 seconds =====================

The assert looks innocuous enough. So if I was an insufficiently paranoid person I might be tempted to think that this was just stdout using python native string types (bytes on Python2 and text on Python3) so the solution would be to use a native string here ("a" instead of b"a". However, where the correctness of anyone else’s bytes <=> text string code is concerned, I subscribe to the philosophy that you can never be too paranoid. So….

The Third Attempt: I bet I can break this more!

Rather than make the seemingly easy fix of switching the test expectation from b"a" to "a" I decided that I should test whether some harder test data would break either pytest or my code. Now my code is intended to push bytes out to stdout even if those bytes are non-decodable in the user’s selected encoding. On modern UNIX systems this is usually controlled by the user’s locale. And most of the time, the locale setting specifies a UTF-8 compatible encoding. With that in mind, what happens when I pass a byte string that is not legal in UTF-8 to write_bytes() in my test function?

# test_byte_writer.py
import io
import sys

from byte_writer import write_bytes

def test_write_byte(capfd):
    write_bytes(b'\xff')
    out, err = capfd.readouterr()
    assert out == b'\xff'

Here I adapted the test function to attempt writing the byte 0xff (255) to stdout. In UTF-8, this is an illegal byte (ie: by itself, that byte cannot be mapped to any unicode code point) which makes it good for testing this. (If you want to make a truly robust unittest, you should probably standardize on the locale settings (and hence, the encoding) to use when running the tests. However, that deserves a blog post of its own.) Anyone want to guess what happens when I run this test?

[pts/38@roan /var/tmp/py3_coverage]$ pytest              (08:19:52)
_________________________ test_write_byte __________________________

capfd = 

    def test_write_byte(capfd):
        write_bytes(b'\xff')
        out, err = capfd.readouterr()
>       assert out == b'\xff'
E       AssertionError: assert '�' == b'\xff'

test_byte_writer.py:10: AssertionError
===================== 1 failed in 0.02 seconds =====================

On Python3, we see that the undecodable byte is replaced with the unicode replacement character. Pytest is likely running the equivalent of b"Byte string".decode(errors="replace") on stdout. This is good when capfd is used to display the Captured stdout call information to the console. Unfortunately, it is not what we need when we want to check that our exact byte string was emitted to stdout.

With this change, it also becomes apparent that the test isn’t doing the right thing on Python2 either:

[pts/38@roan /var/tmp/py3_coverage]$ pytest-2            (08:59:37)
_________________________ test_write_byte __________________________

capfd = 

    def test_write_byte(capfd):
        write_bytes(b'\xff')
        out, err = capfd.readouterr()
>       assert out == b'\xff'
E       AssertionError: assert '�' == '\xff'
E         - �
E         + \xff

test_byte_writer.py:10: AssertionError
========================= warnings summary =========================
test_byte_writer.py::test_write_byte
  /var/tmp/py3_coverage/test_byte_writer.py:10: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
    assert out == b'\xff'

-- Docs: http://doc.pytest.org/en/latest/warnings.html
=============== 1 failed, 1 warnings in 0.02 seconds ===============

In the previous version, this test passed. Now we see that the test was passing because Python2 evaluates u"a" == b"a" as True. However, that’s not really what we want to test; we want to test that the byte string we passed to write_bytes() is the actual byte string that was emitted on stdout. The new data shows that instead, the test is converting the value that got to stdout into a text string and then trying to compare that. So a fix is needed on both Python2 and Python3.

These problems are down in the guts of pytest. How are we going to fix them? Will we have to seek out a different strategy that lets us capture stdout, overriding pytest’s builtin?

The Fourth Attempt: Fortuitous Timing!

Well, as it turns out, the pytest maintainers merged a pull request four days ago which implements a capfdbinary fixture. capfdbinary is like the capfd fixture that I was using in the above example but returns data as byte strings instead of as text strings. Let’s install it and see what happens:

$ pip install --user git+git://github.com/pytest-dev/pytest.git@6161bcff6e3f07359c94a7be52ad32ecb8822142
$ mv ~/.local/bin/pytest ~/.local/bin/pytest-2
$ pip3 install --user git+git://github.com/pytest-dev/pytest.git@6161bcff6e3f07359c94a7be52ad32ecb8822142

And then update the test to use capfdbinary instead of capfd:

# test_byte_writer.py
import io
import sys

from byte_writer import write_bytes

def test_write_byte(capfdbinary):
    write_bytes(b'\xff')
    out, err = capfdbinary.readouterr()
    assert out == b'\xff'

And with those changes, the tests now pass:

[pts/38@roan /var/tmp/py3_coverage]$ pytest              (11:42:06)
======================= test session starts ========================
platform linux -- Python 3.5.4, pytest-3.2.5.dev194+ng6161bcf, py-1.4.34, pluggy-0.5.2
rootdir: /var/tmp/py3_coverage, inifile:
plugins: xdist-1.15.0, mock-1.5.0, cov-2.4.0, asyncio-0.5.0
collected 1 item                                                    

test_byte_writer.py .

===================== 1 passed in 0.01 seconds =====================

Yay! Mission accomplished.

The music filled everything and I was gone for a year

Sitting on a bench a little apart from the revelry, I feel the gulf that’s grown in the past year.  Sisters, brothers, cousins, aunts, and uncles.  Laughing, dancing, drinking, enjoying….  Lily is telling Joseph about the building her new job is in.  Chris is relating how his daughter cried nonstop her first two days in kindergarten then didn’t want to come home on the third.  Aunty Jamie is dancing with the kids….

Everyone fits here but I am sitting a little apart.

Have you ever spent a whole day without uttering a word?  Have you ever carried your whole life in 70 liters of space?  Have you ever lost the path at night, above treeline, with Hurricane Floyd bearing down on you, and no idea whether the next human will come by in the morning or next month?

When you go out into the world the world changes you.  When you return home there’s sometimes trouble finding your place.  When you go out on a long distance hike, a grand pilgrimage, you sometimes find that there’s no one at home who can even help you find that place.  You have stories to tell but for everyone but yourself, they’re just stories.  For you, they’re memories of a life.  A life that you still live even though your feet are now treading trimmed and mown grass instead of dirt, rock, and moss.  A life that you still live despite waking up on a soft mattress instead of a thin foam pad spread over a carpet of pine needles.  A life that you still breathe despite the scents of burnt oil, shampoos, and laundry soap that attempt to mask, overlay, and dismiss it.

Laughing voices combine for a moment to push deeper into the night-shrouded grounds of the estate.  A seven-year locust quickly reasserts its mastery of the nighttime sounds with a long, winding call that starts with a quick thrum and ends with a slow, dribbling churr, churr…  chuuu-uuurr.

A call and response from man’s artificial lighting to nature’s great unknown.

During the daylight, the neatly manicured lawns and precisely sculpted hedge animals might seem as far from the woods and forests as a walk down New York’s Madison Avenue but at night — even New York City has Central Park.

The band starts to play again.  Not a rendition of a pop number this time, but a slow improvisation.  My brother Roy starts them out with a long, drawn-out B that hangs in the air as poignantly as only a violin’s bow can coerce.  The crowd’s attention directs itself languidly towards the band as their ears respond to the magicalness of the single note.  Just as the hint emerges that the note can sustain no longer, just as the noise of conversation begins to fall off, Cousin Maggie’s flute and Uncle Vic’s cello enter the fray, expanding the note, enriching it, strengthening it, adding flavours without changing its central qualities.  And then both Maggie and Roy’s instruments fall silent along with the talking.  All eyes are now on the little platform where two generations of their kin are reaching out to strum their heartstrings.  It is Uncle Vic’s cello which rumbles on without its partners and then, judging precisely the moment before the crowd’s interest will wane, shifts to the next note, and then the next; the next.  A soft sigh of recognition arises from the crowd.  Pachelbel.

Unlike the rendition during the ceremony whose stilted perfection reflected the solemn gravity of that occasion, Roy, Maggie, and Vic now let the music mutate and transform.  Uncle Vic’s bow strokes out a rich deep foundation full of warm harmonics into which Roy’s violin and Maggie’s flute flirt and cavort.  Running through the audience like a series of childhood friends.  Stopping to grip one shoulder companionably, brush ghostly lips along another’s cheek, press a third into a tight embrace; the music reaches out, surrounds, and entangles the audience and then carries them along on a shared journey.  The violin soars playfully, the flute punctuates moments of perfect clarity, and below it all, the cello keeps time and provides the base from which all the other music springs.

Midway through, Maggie and Roy exchange a look and then the music changes.  Pachelbel is usually playful.  More lively the further into the piece one penetrates.  But this performance infuses it with melancholy.  The violin adds a layer of loss; the flute covers that with longing.  The violin ignites restrained passion; the flute replies with a clear eyed acknowledgement of reality.  Duetting, they produce a piercing, soothing melody of old regrets and fresh acceptance.

The last notes die away.  Horsehair bows are raised from strings.  Maggie exchanges pursed lipped communion with her flute for a flashing smile seemingly directed at no one.  Eyes, for a moment, directed at someone she dares not turn to look at.  Suddenly, nature, as if in agreement with the sentiment, voices one final accent.  A loon’s singular voice echoes up from the lake below the estate.  Calling, calling.  Tugging my feet away from the world of complicated man, back to enveloping mystery.

 

-Anonymous Badger, 2017

The Prairie

Youth
views a series of vignettes
passing before the windows of its eyes
poignant tableaus
big blue sky and laughing grasses
heartache and mirth comingle
dispersing quickly in the passing winds

Maturity
waits for a train hauling boxcars through the night
cachunkety chunk
click clack click clack
alternating shadows and light
passing before the windows of its eyes
no car standing out
neither the front nor the back in sight

Age
stands alone on the edge of the slough
a single cicada winds an unanswered call to the coming night
no breeze dares to whisper
as the light slowly leaves the land
green fades to grey
no differences now apparent
through the windows of its eyes

 

-Toshio, 2017