New kitchen release coming soon

[EDIT]For those who are curious, kitchen is a python module for miscellaneous code snippets. Things that people end up reimplementing via cut and paste because they seem to be too small to write a module for but are so useful that they need them in many places. Currently, it has code for i18n, compatibility modules for different python2 stdlib versions, a little bit of iterators, and a whole lot of code for unicode-byte string conversions.

Over the recent vacation I put the finishing (code) touches on a new kitchen release. I’ve scheduled the release of this code for next week on January 10, 2012. This is mainly since I just added the kitchen module on transifex.net and I’d like to see if any translations show up before next week. If anyone finds any bugs in the code on python-2.3.1 through python-2.7.x, please bring them up on the mailing list, on irc.freenode.net (I hang out in #fedora-admin and #fedora-python), or in the kitchen bug tracker so that I can address them before the release date.

The beta code is available from fedorahosted.org at: https://fedorahosted.org/releases/k/i/kitchen/kitchen-1.1.0b1.tar.gz

or from the bzr repository:

  bzr branch bzr://bzr.fedorahosted.org/bzr/kitchen/devel

The major changes are in the kitchen.i18n module. Previously, kitchen.i18n.*Translations objects guaranteed that they would return byte str when requested (via gettext(), ngettext(), lgettext(), and lngettext() methods) and unicode strings when requested (via ugettext() and ungettext()). The new code makes the additional guarantee that byte str‘s that are returned are valid in the requested output charset.

Here’s an example of the old behaviour vs new behaviour:

   >>> from kitchen.i18n import get_translation_object
   >>> translations = get_translation_object('kitchen')
   >>> b_ = translations.lgettext
   >>> translations.set_output_charset('utf-8')
   >>> translations.input_charset = 'latin-1'
   >>> # This would be: 'Café does not exist in the message catalog'
   >>> print repr(b_('Caf\xe9 does not exist in the message catalog'))
   # Old behaviour =>
   'Caf\xe9 does not exist in the message catalog'
   # New behaviour =>
   'Caf\xc3\xa9 does not exist in the message catalog'

   # Example 2: with wrong input_charset =>
   >>> translations.input_charset = 'utf-8'
   >>> print repr(b_('Caf\xe9 does not exist in the message catalog'))
   # New behaviour yields valid utf-8 bytes even when input_charset is wrong =>
   'Caf\xef\xbf\xbd does not exist in the message catalog'

Notice that this is not a magical panacea. The second example, shows that if input_encoding does not match the byte encoding of the strings that are given, the output string will be mangled (replacement characters or garbage characters). However, all the bytes in the output string will be valid in the chosen encoding so you won’t have to worry about exceptions if you attempt to transform the string again.

The other major change is that the kitchen.i18n.get_translation_object() function has been rewritten to be a drop in replacement for the stdlib’s gettext.translations(). The behaviour changes from that include the code now attempting to discover translations in every message catalog that it finds in the paths given to it. Additionally, those code changes lead to bugs in the *Translations classes fallback code being discovered and squashed.

See the NEWS file for other changes.

Brought to you by a cast of thousands (well, tens, at least)

The past few weeks I’ve been coordinating a new release of the Fedora Account System(FAS). Since FAS is something used within Fedora but not a whole lot of other places, development is usually driven by a relatively small handful of people: Ricky Zhou, Mike Mcgrath, and I. This release saw a large number of other contributors which has been very good as the three of us have been increasingly pulled into other projects so our time for FAS has steadily decreased.

• Adam M. Dutko fixed several long standing bugs and feature requests
• Luis Bazán updated several pieces of the UI
• Sijis Aviles switched us from signing the CLA to the new FPCA
• Pierre-Yves Chibon created a new captcha to replace the universally hated one that we were employing
• Jun Chen added a means to clear a user’s ssh key
• Nick Bebout started the work of removing copyright phrase-ology that we no longer want to use (“All rights reserved”) and tracking down which people needed to agree that we could switch licenses from GPLv2-only to GPLv2-or-later
• Jim Lieb contributed code to make handling of languages easier and made FAS more configurable for use in other sites.
• and for the first time in several releases we coordinated our release with the Fedora Translation Team on transifex so that the translations they contributed could go out with the first release instead of when a subsequent bugfix was released.

So let’s take a brief tour of some of these new features.

More Translations

Although we’ve been using transifex to manage translations for FAS for a while now, I hadn’t really understood how to leverage the full power of the Fedora Translation Team to get translations.  Thanks to some prodding by pingou, I got in touch with the translation team this time around and arranged for a string freeze before release during which they worked hard to translate FAS into their native languages. Thanks to transifex, I can show you this nice graph of their hard work:

Top translations: fas » faspot Full graph of translation stats

Clearing SSH Keys

When Fedora Infrastructure recently made the decision to invalidate public ssh keys because we had no way to tell which users might have hosted their ssh private keys on other projects servers which had been attacked and infiltrated, one of the options was for a user who didn’t actually need to use ssh to simply remove their ssh. Unfortunately, the web interface didn’t include the ability to do that so user’s who wanted to go this route had to contact one of the admins and have them remove it for them manually. Thanks to Jun Chen, users can now perform this step for themselves:

New Captcha!

There have always been many times more accounts in FAS than there were active contributors to Fedora. In itself, this wasn’t a problem. However, at some point, spammers started signing their bots up for Fedora accounts as they found that with that, they could write to the Fedora Wiki. To combat this, we added a captcha to the signup process. However, we quickly found that the captcha we added was too hard. Many people came to us to complain that they could not answer the captcha successfully. Thanks to pingou, we have a new captcha which displays a simple math equation in a much less distorted image. Writing the correct answer to the equation is all you need to do.

These are just some of the more user visible changes. If you’re interested in the more behind the scenes changes (SELinux fixes from ricky, password strength checking, and more), check out the changes in FAS’s git repository.

Porting to TurboGears2

Fedora Infrastructure was an early adopter of the Turbogears web framework. As such, we have a large number of applications that were built targeting the TurboGears-1.0.x/1.1.x release series. Unfortunately, while the upstream TurboGears project has released new features, optimizations, and other enhancements, this work has had to be done in new releases which aren’t backwards compatible with TurboGears-1.1.x. So we’ve been faced with the task of porting our apps to the newer releases for some time… and proven remarkably success at procrastinating on that .

Well, thanks to the efforts of pingou we’re starting to take steps to fix that. He’s gone through the Fedora Elections application and ported it to TurboGears2.x. This is a great step that shows that there’s no blockers to getting all our apps running with TG2 (our custom auth layer was a big worry before this work was done.)

We still have four more applications to port that are more complex than elections but now we have some experience to show what needs to be done. Having an idea of what we’re in for is great for breaking through the mental reservations about starting the process.

If you see pingou online, be sure to thank him for a job well done!

Fedora 16 naming vote delay

It’s been asked a few times so just putting the answer out onto the intarwebs:

Q: When are we going to hold the election for the Fedora 16 release name?
A: Sometime soon. We’re behind schedule because we (the Board) did the pre-filtering of names for easily found collisions on March 22 instead of on March 8th. The names have been submitted to Red Hat legal and we’re currently waiting on them to give us back a list of approved names. Once that’s done, jsmith will add an election to the Fedora Election App and announce that we’re ready for people to start voting. I’ll put in a disclaimer here that you’ve probably all heard: “Past performance is no guarantee of future results” but judging by the time we normally allot to going from submission to legal to having the election, I’m hoping that we’ll be able to start the voting at the end of this week or the start of the next.

Best Lightning Talk

At about 8:30 into this video, you’ll see a great lightning talk: PyCon 2011: Friday Afternoon Lightning Talks

It struck me on several levels.  It was told by an excellent straight man.  It had a great story.  It was gently deprecating.   In short it was very entertaining.

But beyond that, it also highlighted one of the core philosophies of open source that sometimes gets lost in mailing list threads about user base and popularity.  Open source is powered by people wanting to scratch their own itches.  Having a popular product is fun but people will work on something that only has a small amount of users as long as it is a good base for them to do what they want.

How to lose customers

Dining room of typical house.  March, 2011 8:30PM PST

Phone:  Ring.  Ring.  Ring.

<Toshio picks up phone>

Toshio: Hello?

BoA_human1: Hello, this is Lewis at Bank of America Home Loans, calling about your account.  Can you give me the last four digits of your social security number to verify your loan?

Toshio: Can I call you back to verify you are actually from Bank of America?

BoA_human1: Yes.  You can call 1-800-669-0102.

Toshio: And what’s your name again?

BoA_human1: Lewis.  But you can actually talk to anyone here and they’ll have this information and be able to talk to you.

Toshio: Thanks.

<Toshio hangs up the phone.  Dials 1-800-669-0102>

BoA_automated_system:  Thank you for calling BAC Home loan servicing LP, the Bank of America company that services your home loan.  Para español, por favor, pressione número uno.  We are a debt collector.  Please enter or say your account number.

<Toshio enters in the digits of his account number>

Phone:  Beep beep beep beep beep.

BoA_automated_system:  Okay [digit][digit][digit][digit], is that right?

Toshio: Yes.

BoA_automated_system: Are you an agent calling on behalf of a customer

Toshio: No.

BoA_automated_system: If you are calling to confirm if your payment has been recieved press 1 now.  If you would like to pay by phone press 2 now.  If you are calling to tell us when you will make a payment press 3 now.

<Toshio ….  Pause… Finally hits star>

BoA_automated_system: Do you want the main menu?

Toshio: yes

BoA_automated_system: Ring.  Ring.

BoA_human2: Hello, Home Retention, this is Malik can I get the last four digits of your social security number to verify your identity?

Toshio: [digit] [digit] [digit] [digit]

BoA_human2: Thank you.  I see that your account can only be serviced by the advocacy department and their office closes at 5:00 PM Pacific standard time.  I can give you their number to call back tomorrow.

[...]

 

Date of first contact with HUD approved housing counselor: June 2009

Date of initial packet for eligibility for the Home Affordable Modification Program sent to Bank of America:  November 2009

Date for three trial payments: December 2009, January 2010, February 2010

Date that Bank of America sent us the “Modification Agreement” with new loan terms to sign according to the date on their letter: 19 August 2010

Date we received the “Modification Agreement” from FedEx: 23rd of August, 2010

Date we had to return it by: 29th of August 2010

Number of days spent on the telephone with Bank of America and HUD counselors trying to get answers to why there was a “Deferred Amount” of $11,852.52 : 3 days

Number of unique (does not count being transferred back to a previous phone number) Bank of America phone numbers  transferred to in those 3 days: 23

Number of unique Bank of America Departments contacted by calling those numbers: 9

Date I called Bank of America to find out why we hadn’t gotten new payment coupons or a copy of the “Modification Agreement” that Bank of America had signed: December 2010

Story I got from Advocacy at Bank of America: “The underwriters have to look at the agreement and approve it.”

Information I got from talking to the company in charge of sending and receiving  the Modification Agrement packets whose number was the one given on the Bank of America packet: “The packet you received was the final agreement.  You’re just waiting for Bank of America to sign it and send you back the “Executed Agreement”.  The underwriters have already looked at it so they just need to send you a copy.”

Story I got from Advocacy at Bank of America when I gave them the new information: “It’s with the negotiator.  We’ll have them call you within 5 to 10 business days”.

Date of that phone call: end of December 2010, just before Christmas.

Today’s date: 8 March, 2011

 

Need more Memory

For those installing the Fedora 15 Alpha into a VM, just a quick note — virt-manager currently defaults to 512MB of RAM for its VMs.  This isn’t enough RAM to boot the Alpha installer.

[11:37:59] <wwoods> F15 will need at least 640MB
[11:38:02] <wwoods> maybe 768
[11:38:42] <wwoods> I have a fix for that in the works but it’s kind of invasive and may not be ready ’til F16

Symptoms of trying to boot with too little memory may include a kernel oops when installing the rootfs.  Hope this helps people avoid head scratching later .