Someone showed me this today:
If the author prints a book with that poem, please, let me know. I’ll buy a copy to support such genius
All packed up and waiting for my plane to Raleigh. Going there to work on enabling two-factor authentication for the hosts that give shell access inside of Fedora’s Infrastructure. For the first round, I think we’re planning on going for simple and minimal to show what we can do. Briefly, the simplest and minimalist is:
* Server to verify a one time password (we already have one for yubikeys)
* CGI to take a username, password, and otp to verify in fas and the otp server
* pam module for sudo that verifies the user via the cgi
* database to store the secret keys for the otp generation and associate them with the fas username
We’re hoping to go a little beyond the minimal at the FAD:
* Have a web frontend to configure the secret keys that are stored for an account.
* Presently we’re thinking that this is a FAS frontend but we may end up re-evaluating this depending on what we decide to do for web apps and what to require for changing an auth source.
* Allow both yubikey and google-authenticator as otp
I’m also hoping that since we’ll have most of the sysadmin side of infrastructure present that we’ll get a chance to discuss and write down a few OTP policies for the future:
* Do we want to make two-factor optional for some people and required for others?
* How many auth sources do we require in order to change a separate auth source (email address, password, secret for otp generation, phone number, gpg key, etc)?
If we manage to get through all of that work, there’s a few other things we could work on as well:
* Design and implement OTP for our web apps
I’ve spent a good part of the last week working on the python3 port of kitchen. It’s now to the point where I’ve reviewed all of the code and got the unittests passing. I still need to add some deprecation warnings and a gettext object that mirrors the python3 API instead of the python2 API. Then it’ll be ready for an alpha release. Still a lot of work to do before a final release. Most of the documentation will need to be updated to change from
bytes and the best practices sections will need a major overhaul since a lot of the problems with python2 and unicode have either been fixed, mitigated, or moved to a different level.
It was both an easy and hard undertaking. The easy part was that kitchen is largely a collection of dependent but unrelated functions. So it’s reasonably easy to pick a set of functions, figure out that they don’t depend on anything else in kitchen, and then port them one by one.
The hard part is that a lot of those functions deal with things that are explicitly unicode and things that are explicitly byte strings; an area that has both changed dramatically in python3 and that
2to3 doesn’t handle very well. Here’s a couple of things I ended up doing to help out:
Detecting String Types
Kitchen has several places that need to know whether an object it’s been given is a byte string, unicode string, or a generic string. The python2 idioms for this are:
if isinstance(obj, basestring): pass # object is any of the string types if isinstance(obj, str): pass # object is a byte string elif isinstance(obj, unicode): pass # object is a unicode string else: pass # object was not a string type
In python3, a couple things have changed.
basestringtype as byte strings and unicode strings are no longer meant to be related types.
bytes) and mutable (
With these changes, the python3 idioms equivalent to the python2 ones look something like this:
if isinstance(obj, str) or isinstance(obj, bytes) or isinstance(obj, bytearray): pass # any string type if isinstance(obj, bytes) or isinstance(obj, bytearray): pass # byte string elif isinstance(obj, str): pass # unicode string
There’s two issues with these changes:
2to3can correctly change all occurrences of
isinstance(obj, str)but occurrences of
isinstance(obj, str)will also be rendered as
isinstance(obj, str)in the
2to3output. This is correct for a lot of normal python2 code that is trying to separate strings from
floats, or other types but it is incorrect for code that’s trying to explicitly separate bytes from unicode. So you’ll need to hand-audit and fix your code wherever these idioms are being used.
For kitchen, I added a few helper functions into
kitchen.text.misc that encapsulate the python2 and python3 idioms. For instance:
def isbasestring(obj): if isinstance(obj, str) or isinstance(obj, bytes) or isinstance(obj, bytearray): return True return False
and similar for
isbytestring(). [In case you're curious, I broke with PEP8 style for these function names because of the long history of
is* functions and methods in python and other programming languages.] By pushing these into functions, I can use
if isbasetring(obj): on both python2 and python3. I only have to change the implementation of the
is*string() functions in a single place when porting from python2 to python3.
Now let’s mention some of the caveats to using this:
isbasestring()) is somewhat expensive. So if you use this in any hot inner loops, you may want to benchmark with the function and with the expanded version to see whether you take a noticable loss of speed.
bytearrays are mutable so maybe your code shouldn’t include those with the “normal” string types.
str). In python3 byte strings are not as ubiquitous as they were in python2 so maybe your code can be changed to stop checking for the type of the object altogether or reduced to a single
isinstance(obj, str). The language has evolved so when possible, evolve your code to adapt as well.
Next time: Literals
I’ve been hacking on other people’s python3 code for a while doing porting and bugfixes but so far my own code has been tied to python2 because of dependencies. Yesterday I ported my first personal script from python2 to python3. This was just a simple, one file script that hacks together a way to track how long my kids are using the computer and log them off after they’ve hit a quota. The kind of thing that many a home sysadmin has probably hacked together to automate just a little bit of their routine. For that use, it seemed very straightforward to make the switch. There were only three changes in the language that I encountered when making the transition:
except Exception, exc. The new syntax is:
except Exception as exc.
print 'hello world'. In python3, it’s a function so you write it this way:
When I’ve worked on porting libraries that needed to maintain some form of compat between python2 (older versions… no nice shiny python-2.7 for you!) and python3 these concerns were harder to address as there needed to be two versions of the code (usually, maintained via automatic build-time invocation of 2to3). With this application/script, throwing out python2 compatibility was possible so switching over was just a matter of getting an error when the code executed and switching the syntax over.
This script also didn’t use any modules that had either not ported, been dropped, or been restructured in the switch from python2 to python3. Unlike my day job where urllib’s restructuring would affect many of the things that we’ve written and lack of ported third-party libraries would prevent even more things from being ported, this script (and many other of my simple-home-use scripts) didn’t require any changes due to library changes.
Verdict? Within these constraints, porting to python3 was as painless as porting between some python2.x releases has been. I don’t see any reason I won’t use python3 for new programming tasks like this. I’ll probably port other existing scripts as I need to enhance them.
Congratulations and many thanks to everyone who was involved in the effort to unbundle zlib from rsync! Looks like this long standing bug that’s been a sore spot for many distributions is finally being addressed. It almost makes me want to create a Fedora 18 Feature page for it
If you were at the first day of PyCon Santa Clara this year, you might have seen the picketing employees outside the main hotel/conference center. Looks like the reason for that protest has made it to the attention of a group that’s going to send a petition to the Hyatt management.
This is something I’ve been noticing for a while and am finally getting around to blogging.
In the first days of FESCo, Thorsten Leemhuis was the chairman. One of the quirks of his time was that we’d encounter a topic where we voted on a solution and found that a majority agreed with one sentiment but it wasn’t unanimous. When that happened, Thorsten would be sure to ask if there was anything we could do to make the solution more acceptable to the dissenters even if they still wouldn’t vote for the proposal.
This sometimes lead to discussions of a proposal that had been approved with margins like 7 to 2 and after the discussion and changes, the vote was still 7 to 2. So from an external standpoint, this might be seen as unproductive. Why don’t we just get a decision made and move on?
But over the years I’ve watched a lot of other split decisions be made on several committees from both the inside and the outside and it’s struck me that, perhaps, we don’t do nearly enough of this sort of examination. Making changes after it was clear that a majority agreed with the basic proposal had several beneficial effects:
So what can we do with this? Maybe it’s too much to ask that we look over every little decision we make where there’s disagreement and attempt to find every last bit of common ground that we can (There were certainly times when it seemed to take forever to make a decision) but what about decisions that are close votes? What about decisions that have days-long threads as part of their backstory? In these cases, consider the proposal that the majority agrees on to be a strawman. A starting point from which to start chipping away to see what changes can be made that are still acceptable to the majority while addressing many of the issues that the minority has. Remember that the goal is to craft a compromise that addresses as many concerns as possible.
We’ve just deployed a new Fedora Account System to production. This release just pulls a few new features that didn’t quite make the 0.8.10 release:
That’s it for this minor update.
/me goes to play with the audio captcha some more.
Are you on IRC? Are you at FUDCon? Do you have a project that you want to happen in Fedora?
The Fedora Board is working on choosing goals that each individual member wants to champion and bring to fruition this year in Fedora. So if you have some idea that you think a Board member’s help will make work better, grab your nearest Board member[*] and ask them to bring it up on their Sunday meeting. One of them may take it up as something they think they can work on and help accomplish in the coming year.
[*] Board members you may see wandering around at FUDCon:
As mentioned last week a new kitchen release went out today. Since last week some small changes were made to the documentation and a few changes to make building kitchen easier were implemented but nothing has changed in the code. Here’s the text of the release announcement:
== Kitchen 1.1.0 has been released ==
Kitchen is a python library that brings together small snippets of code that you might otherwise find yourself reimplementing via cut and paste. Each little bit is useful and important but they usually feel too small and too trivial to create a whole module just for that one little function. However, experience has shown that any code implemented by copying will inevitably be shown to have bugs. And when you fix those bugs, you’ll wish you had created the module so you could fix the bug in one place rather than two (or five.. or ten…). Kitchen aims to be that one place.
Kitchen currently has code for easily setting up gettext so it won’t throw UnicodeErrors in corner cases, compatibility modules for different python2 stdlib versions (2.4, 2.5, 2.7), a little bit of iterators, and a whole lot of code for unicode-byte string conversion. In addition to the code, kitchen contains documentation that explains some of the common problems that arise when dealing with unicode in python2 and how to fix them through changes in coding practices and/or making use of functions from kitchen.
The 1.1.0 release enhances the gettext portion of kitchen. The major enhancements are:
For the complete set of changes, see the NEWS file.