logging.config.listen() in Lib/logging/config.py, doctstring updated with more description.
Issue #15452 Added verify option for logging configuration socket listener. Now we have a verify argument which should be a callable which should verify whether bytes received across the socket are valid and should be processed.
Issue #15609 Objects/unicodeobject.c Attached patch adds tests to check that str%args and str.format(args) take all optimization opportunities for simple format strings. It takes also the fast-path when Unicode string argument when width and/or precision is set. For example, “%.3s” % “abc” now returns “abc” unmodified (instead of creating a copy). The patch also takes the fast-path when formatting integers with str%args when the argument is not an int, but an object with int method.
Issue #15979 http://docs.python.org/dev/library/timeit.html Documentation for the timeit module got updated.
Issue #15766 Python/dynload_shlib.c Catch exceptions while raising the ImportError in imp.load_dynamic()
Issue #15609 Optimize str%args for integer argument
Code changed in this commit Objects/unicodeobject.c & Python/formatter_unicode.c Lib/test/test_format.py is also updated to have new tests for the changes.
Updated code to use “yield from” in various places - Lib/http/cookiejar.py - Lib/json/encoder.py - Lib/xml/etree/ElementPath.py - Lib/xml/etree/ElementTree.py - Tools/importbench/importbench.py
Issue #16089 _elementtree.TreeBuilder broken with a non-C-deriving element_factory Patches went in with tests
Issue #15599 test_circular_imports() of test_threaded_import fails on FreeBSD 9.0 Lib/test/test_threaded_import.py Several systems cannot handle gil_interval == 1 in extreme situations. So it is now changed to sys.setswitchinterval(1e-5).
Issue #16304: Further optimize BZ2File.readlines?() In Lib/bz2.py
Anotjher patch set to utilize yeild from in the codebase. - Lib/argparse.py - Lib/argparse.py - Lib/concurrent/futures/_base.py - Lib/difflib.py - Lib/email/_header_value_parser.py - Lib/email/iterators.py - Lib/glob.py - Lib/mailbox.py - Lib/pkgutil.py - Lib/traceback.py - Lib/weakref.py
In Lib/test/test_sysconfig.py we now have tests with subprocess.DEVNULL
Previously darkserver project had a koji plugin, which used to collect the information from the koji-hubs itself. It used to process each rpm, find ELF files and extract GNU BUILD-ID from them.
But the major disadvantage on this method we found was package build time increase. For bigger packages like kernel it used to take more than one hour extra. Koji does not allow to do direct async-plugins, so I had to look for a complete separate import process.
The following image explains the new process
We have a darkproducer process running, which is checking koji for any new build (using API(s) provided by koji), if it finds a build is on, then it creates an entry in the Buildqueue, if the build is completed then it creates an entery in the Jobqueue.
darkbuildworker process is monitoring Buildqueue, and if it finds any completed job then it moves it to the Jobqueue or else puts it back in the Buildqueue.
Then finally we can have many darkjobworker processes, they can monitor the Jobqueue and imports any jobs landing in the queue. All of the processes work as daemons.
Broke the previous monolithic design into simple producer-consumer one. Using Redis to maintain the queues. I am also running a dev instance to to test the design
The codebase which contains queue creation and task management, is being moved into a separate python module to create/use redis based queues. I hope it will be useful for someone else trying to solve similar kind of problem.
Last night in the FESCo meeting the feature proposal of having a full Clojure stack got accepted. Yon can find the feature page here. /me and hircus are currently working on this and more help are always welcome :)
Day 2 started with a talk from Joshua Wulf about the Fedora book written by the attendees. The idea seems to easy enough for the newbies to contribute but still they will require a bit knowledge of docbook I think.
Rest of the first half was spent discussing package Lisp modules in Fedora with Christoph Wickert. Now I am going to setup a repo with all the packages so that people can play around with them.
During lunch time I met another fellow Red Hatter, Luc de Louw. Lunch table discussions went on about various other FUDCons and funny incidents in them. The lighting talks session went really well. People submitted various ideas and started demoing various projects. While CWickert was showing LXDE, I decided to try it out and still now the performance looks amazing, though there are few small things needs to fixed.
The cloud talk by CWickert was hilarious. After the day finished we slowly moved to the hotel and FUDPub was supposed to start from 8pm in the same hotel. The food was nice and cake was delicious.
The number of attendees in the day 3 was smaller compared to the last two days. I had my talk on Darkserver in the morning. We had some good discussion about the future roadmap of the project during the talk. You can find the slides here. There was another good talk on various create applications available in FOSS world. In the second half of the day we decided to put up the talks and workshops for voting as number of people was less. I went ahead to do the python workshop as planned. Abu Mansur Manaf gave the ending keynote which should boost the newbies to become contributors.
Spent the evening in the hotel room discussing various functional programming languages and features of the languages and darkserver roadmap with hircus. Later in the night went out for dinner with the rest of the event crew members to a local food joint. Stayed up for the rest of the night as we had catch an early morning flight back to India.
Met many old friends and made many new ones, long discussions on project ideas, great food, sleep for only few hours everyday, this event was full on. The last FUDCon in India and this one, a large portion of the audience are total newbies and I hope they got good ideas on how to start contribute. My personal goals were to find new contributors for darkserver project and discuss the packaging ideas with CWickert and both went well.
Getting up early in the morning, eating breakfast while sleeping is the most common thing of every event’s day 1. It was exactly the same, but we managed to start from the hotel to the FUDCon venue on time. The registration desk was already opened. We slowly moved into Audi 3 for the keynote by Christoph Wickert.
In the last few days we managed to get Darkserver live in production. Thanks to Nirik and Dgilmore, without their help it was not possible.
This service is providing the API(s) based on which other developers can write tools to query details of build-ids. We Have build-id in every ELF we build from Fedora 8, but there was no good way to query them. More details on why we need the service can be found at the wiki page.
For now it is handling only primary architecture builds. Within last 4 days we have around 118k build-id(s) in the database.
I am working on a small command line client for the same, called darkclient. Hope to have the first release out in next week. API details can be found at the home page of the service.
The project page is here.
Why you need it ? Say you are writing unittests for your favourite application and in some function you are using xmlrpclib module to interact with a web-application. Now for testing you can easily mock that interaction out, which will help to run your code in fast and easy way. Mocking database calls can be another example.
Example:
import unittest
import xmlrpclib
from mock import patch
class TestFoo(unittest.TestCase):
"""
A simple test
"""
@patch('xmlrpclib.Server')
def test_first(self, mock_xmlrpc):
m = mock_xmlrpc.return_value
m.multiply.return_value = 6
server = xmlrpclib.Server("http://kushaldas.in/")
res = server.multiply(2, 3)
self.assertEqual(res, 6)
if __name__ == '__main__':
unittest.main()
In Fedora you can just yum install python-mock.