Moving On

A little about my life, just for a second, I’ve come down with some sort of sickness, which sucks. And I’ll be gone the 23rd through the 25th on a camping trip

On a more technical note, I was struggling a little bit with NumPy’s handling of Fortran ordered arrays conceptually, so my stride_column code didn’t seem to work right, but I think I’ve fixed it now. It looks like the “right” thing to do, however, is going to be to precompute the strides of an array at creation time, that should provide a pretty significant speed boost for multi-dimensional indexing. (NumPy has always been doing this, I haven’t)

I want to keep pushing on NumPy on PyPy, but I haven’t made any progress since my last blog post, that’s a very important part of my GSoC project though, so it must get done.

I’ll be updating this post later, for now I want to get some work done.

Cheers,

Dan
Advertisements
Moving On

Overview

My mentor asked me to write about what we’ve done so far, how we got here, and where we’re going.

Most of the story of what’s happened from May 24 to now, is chronicled in this blog, however I’m going to make an attempt to summarize it a little to make a nice, digestible byte for you to consume.

Past

To start, my school didn’t end until May 28th, and so I didn’t get anything done during my final’s week. Then, the next week or so my family from Nebraska and San Diego came to stay at my house for my younger brother’s celebrations. That is, my little brother achieved the rank of Eagle Scout in Boy Scouts, which is the highest rank, and he had also graduated highschool. Both celebrations were in the span of one hectic weekend, so, though I did shut myself in my room to get work done, my work really began after everyone left.

On the technical side of things, I’d been working on getting NumPy working on top of PyPy. Unfortunately, CPyExt, PyPy’s CPython compatibility layer is not as mature or complete as we had hoped, and so I spent the better part of June hammering on it and NumPy. My work on NumPy/CPyExt has been very tough. Implementing mostly wrapper functions in CPyExt for the sake of NumPy was the easiest part, but there are bugs lurking in CPyExt, and subtle incompatibilities with the CPython interface. I have a few functions sitting in my working copy that I want to commit to PyPy, but won’t until I take another look at them.

On top of this, NumPy doesn’t always play within the API and will go around touching your structs in their private members. I tried to eliminate NumPy’s bad touch, and those changes, (most heinous of which was in PyArray_Scalar()) are in my github repository for NumPy.

Present

In an attempt to make up for lost time, I’ve been doing a lot of work lately. I quickly (re)implemented micronumpy and it passes 50% of its tests. It does seem to have a somewhat obscure bug where it segfaults if you allocate (and subsequently collect?) too many micronumpy arrays. I haven’t had a chance to look into this just yet, but that’s my #1 bug to squash right now. Thanks to the awesome work from the PyPy guys with the JIT generator, my incredibly naive implementation of micronumpy arrays is already twice as fast as normal NumPy on CPython for the convolve benchmark. This is part of the beauty of PyPy, I didn’t do any JIT specific work, and the JIT has yielded a significant improvement. There are hints that I can and will give the JIT, to hopefully further improve performance. (After all, we want to beat Cython (well, that’d be nice, I’ll be happy to be within 20% or so))

A little about Subversion. I attempted to merge trunk back into the micronumpy branch. Mind you, not one file has a conflicting edit. The merge resulted in a series of conflicts. Subsequently, I tried to commit my latest changes including the convolve benchmark I used, and svn would not co-operate. Eventually I got a bit of help on #svn on irc.freenode.net and started svn reverting, rming, and svn updateing, and after around three hours, I finally was able to commit again. During the whole process I ended up writing a few bash one-liners and brainlessly svn reverted my working copy of micronumpy. Luckily I still had vim open with the latest changes that I could write back out, but that had me scared…

Future

I’ve been developing some tools to make debugging CPyExt easier, so I’m very hopeful that I’ll have NumPy working on PyPy. The first is a little indented printer, which allows you to push/pop levels of indentation. I implemented it as a decorator which outputs function arguments, so I’m able to somewhat usefully, see the CPyExt call stack. Right now I’m interested in implementing a more useful printer for lltype.Struct, which currently only tells us it’s a struct, and what its fields are. I’d like to make it certain key fields from the struct as name/value pairs.

On top of this, micronumpy is coming together nicely, I still have to implement some of the tougher array access methods (multi-dimensional slices for instance) and those will be coming, hopefully I can scavenge them from the original micronumpy. My first priority, for micronumpy is to sort out this segfault that keeps occuring.

Closing

So that’s what has been, what is, and what will be. I hope someone out there is finding this interesting, and eventually people will find my work useful. Also writing this post has made me think more closely about how much time I’ve put into this summer of code. For a while I was feeling that maybe I wasn’t working enough, or hard enough, but writing this, and realizing how much time I’ve actually had to put in, puts it in perspective, and I feel much better.

Cheers,

Dan
Overview

Unofficial Results

So, in order to make up for lost time, I’ve been coding a significant amount the past 48 hours, near non-stop. My labor has finally bore fruit.

Micronumpy

Micronumpy found at https://codespeak.net/svn/pypy/branch (Click to browse the source, copy the link name for the checkout address, full instructions here) has been rewritten to use lltype.Array and now supports enough to run the convolve benchmark found in pypy/tool/numpybenchmark.py which is a slightly modified version of this. Unfortunately, pypy segfaults if the test is run more than 20 times by timeit. However, with 20 repetitions, a 200 by 200 image, and a 3×3 kernel, PyPy handily beats CPython and plain NumPy.

Command Average Time per run (Seconds)
./pypy-c ~/Projects/micronumpy/pypy/tool/numpybench.py 200 200 3 3 0.38495
python ~/Projects/micronumpy/pypy/tool/numpybench.py 200 200 3 3 0.85705

That’s an improvement of about 55% shorter runtime!

That’s not bad for a first iteration (Even if the iteration’s been a while in the works). Now note that the benchmark as it is right now does not provide for a warmup period, which should improve PyPy’s score even more. Whether or not a warmup period is more or less of a measurement of what we care about, I’m not sure. Also note that, while the times have remained stable, they are likely susceptible to my other processes on this computer. I plan on writing a follow up article which will hopefully have more interesting results, and maybe even a graph!

NumPy on PyPy

Though it’s taken far longer than any of us expected(Except maybe Maciej). I’m very hopeful that in the next week or so I can have NumPy running in a somewhat stable fashion on PyPy. Currently I’m hacking on CPyExt/lltype to give more useful str() values, rather than <* <Array of Char> at 0xDEADBEEF> I hope it will look more like <* <Char array = "spam"> at 0xDEADBEEF> which I think will be a useful debugging tool for everyone.

More on everything after the midterm reviews 🙂

Cheers,
Dan
Unofficial Results

The day of Reckoning(Midterm Evaluations) is upon us!

As evidenced by my previous blog posts, I’m not particularly happy with my own progress. Hopefully my recent progress will be sufficient to appease the Google gods. (Maybe with some benchmarks too)

I can report that I’ve completed quite a bit since my last blog post, for instance I implemented a toy micronumpy array implementation using lltype.Array and have now grown it into a “full” implementation. I hope that my approach will work well with the JIT, simple element lookups are a couple of dynamic dispatches away (which is a little less than ideal in my opinion) but it should JIT well since the JIT will remove the overhead of those calls.

Unfortunately, this implementation isn’t re-using much of the original code, considering that the core storage of the array is being altered, not a whole lot of the original code is proving useful, but I will be attempting to re-use as much as possible (if for no other reason than not having to reconsider all of the math). I currently have one of our original tests passing now, which is better than nothing, but not particularly satisfying…

I’ve had to deal with alot of bugs in PyPy’s CPyExt CPython extension compatibility layer, which is certainly what slowed progress the most. The time spent on that was almost entirely unexpected, and represents the biggest setback (other than family obligations…)

Trying to keep this short so that I can get back to work, I want to have some benchmarks before evaluations are over to justify my existence 🙂

The day of Reckoning(Midterm Evaluations) is upon us!