A Bit Behind Schedule

Life seems intent on limiting the time I can spend on my GSoC project. The first wave of family arrived at my house today, and more still will be coming. I have a lot to do to help my family host. Last week we set the goal that I should have NumPy working on PyPy by the end of last week. (which we’ll call Tuesday since that’s when we set those goals forward) In addition I was intended to have started fixing up micronumpy.

While I haven’t started on micronumpy, I have made significant strides in getting NumPy to work on PyPy. Unfortunately, CPyExt isn’t as mature as we’d hoped, and rather than working on NumPy, I’m spending my time implementing C API functions in PyPy. NumPy still doesn’t import, but I can see from the output of nm -u the number of missing symbols is being slowly whittled down. I should also note that NumPy’s setup.py has caused me significant grief since its ‘clean’ target doesn’t clean up after setup.py build_ext -i at the moment I haven’t identified whether that’s a bug in NumPy’s copy of distutils (gross in and of itself) or if it’s a nasty interaction between it and CPyExt’s presetup.py which creates a stub dll/so for testing purposes.

Now on to the positive side of things, since I began writing this post (about a week ago) I’ve satisfied the last of NumPy’s Symbol dependencies it seems, now there are still issues with importing the module, for instance some API functions get passed non-Python objects (probably being double freed) but I’m not sure how best to track this all down. I’m not sure whether I just haven’t found the source, or if it has to do with output redirection and buffering, but no matter where i’ve put printfs (in c) and prints (in python) they always end up after the exception is thrown in the output…

EDIT: It’s almost certainly turned out to be a buffering/stderr vs. stdout issue, how I’m going to get Python buffering to play nice with C is a mystery to me though… Perhaps if I turn off buffering…

So, sorry for the absence, things are still moving along, though I definitely have a bit of catching up to do.

A Bit Behind Schedule


To start, there was a question where my progress could be followed. There are several, unique projects which affect my ultimate goal, and each one is going to be hosted accordingly. There’s nothing interesting there right now but a woefully out of sync trunk and some basic array code (hey, it works though). but modifications to PyPy will live at codespeak.net http://codespeak.net/svn/pypy/branch/micronumpy is the url, to browse the source you can navigate a browser to http://codespeak.net/viewvc/pypy/branch/micronumpy. My modifications to NumPy will live here though the trivial changes are already in trunk (Changeset #8448 and Changeset #8418). I will have a few changes for Cython by the time the summer is through, but for the moment I may mostly ignore the mtrand.c file which has alot of problems for non-CPython implementations.

As far as plans go, it’s been decided that this week I should finish up my NumPy compatibility work as fast as possible, I am quite close, though I noticed I had some other hackish things I did to get it to build, which I need to fix. After that it will be time to port micronumpy to lltype arrays, which are PyPy’s way of creating raw arrays/buffers. Without this, the lists used in the current implementation can be moved by the garbage collector (a very useful thing). However, normal C code does not have any method for dealing with moving objects, and so in order to interact with NumPy’s C code, we need to migrate to lltype arrays.