Fernando Perez

Scipy advanced tutorials results

2009-06-30T23:35:00.000-07:00

We recently conducted a poll on Doodle, soliciting feedback on the preferred topics for the advanced track, which is meant to contain 2 days with 8 2-hour sessions focusing on one specific topic at a time. The table below shows the complete results, which I've only sorted for convenient viewing and anonymized (the raw Doodle output contains the names given by each person voting). If anyone would like the raw spreadsheet, just drop me a line.

The score was computed as #yes-#no (i.e., yes=+1, neutral=0, no=-1), from a total of 30 responses, and the results are in the table below, ranked from highest to lowest score. In my personal opinion, all the topics offered would have made for very good and interesting tutorials, but the point of asking for feedback is obviously to follow it to some degree, which we will now do. I think it's worth noting --though not particularly surprising-- that the ranking roughly follows the generality of the tools: matplotlib and numpy are at the top, with finite elements and graph theory at the bottom. While I personally use NetworkX and love it, it's a specialized tool that for many probably offers no compelling reason to learn it, while pretty much every single numerical python user needs numpy and matplotlib.

We are now in the process of contacting possible speakers for the top topics, and will communicate on the mailing list a final list of topics once we have confirmed speakers for all.

Note: the html formatting of this table is hideous and for some odd reason it drops to the bottom of the page, so you need to scroll way down to the bottom of this page to see the results table. Sorry. I generated it from OpenOffice and it looks fine in Firefox, but it renders horrible here. If anyone can send me a note on how to fix it (such that I can copy/paste the corrected html), I'll be happy to do so.

	Yes	Neutral	No	Score	Rank
Advanced topics in matplotlib use	18	10	2	16	1
Advanced numpy	18	10	2	16	2
Designing scientific interfaces with Traits	15	11	4	11	3
Mayavi/TVTK	13	11	6	7	4
Cython	14	8	8	6	5
Symbolic computing with sympy	15	6	9	6	6
Statistics with Scipy	9	15	6	3	7
Using GPUs with PyCUDA	13	7	10	3	8
Testing strategies for scientific codes	11	11	8	3	9
Parallel computing in Python and mpi4py	12	8	10	2	10
Sparse Linear Algebra with Scipy	9	12	9	0	11
Structured and record arrays in numpy	8	14	8	0	12
Design patterns for efficient iterator-based scientific codes	9	7	14	-5	13
Sage	8	6	16	-8	14
The TimeSeries scikit	4	13	13	-9	15
Hermes: high order Finite Element Methods	6	9	15	-9	16
Graph theory with NetworkX	5	9	16	-11	17

Python at the SIAM CSE'09 meeting

2009-03-09T13:48:00.000-07:00

After the success of last year's Python minisymposium at the annual SIAM meeting, this year we had a repeat: Simula's Hans-Petter Langtangen (author of the well-known Python Scripting for Computational Science), U. Washington's Randy LeVeque and I co-organized another minisymposium on Python for Scientific computing.

At he Computational Science and Engineering 2009 meeting, held in downtown Miami March 2-6, we had again 3 sessions with 4 talks each (part I, II and III), with a different mix of speakers and focus than last year. While last year we spent some effort introducing the language and to a certain extent justyfing its use in real-world scientific work, we felt that this time, the growth of the many python projects out there speaks for itself and that we should instead turn our attention to actual tools and projects useful for specific work. Thus, we had no 'why python for science' talk, although obviously most speakers spent some time providing this kind of information in the context of their own projects.

I think that this is the right path to follow: for a number of years, many of us have been developing tools and justyfing Python as a viable alternative to tools such as Matlab or IDL, but we need to start moving away from that mode. There is no doubt in my mind now that, while still immature in certain areas, Python is a credible, production alternative to said packages. In many contexts it can actually be far superior to its propietary and expensive counterparts, as many of us have found in practice.

So hopefully as we move forward, we will do less of 'you can use python for scientific work' and more of 'here is a great scientific project/tool that happens to be implemented using Python'. Eventually, I hope we will not have any Python-specific sessions at scientific meetings, just like we don't see any "Fortran for scientific computing" special sessions.

This was a smaller meeting than the annual one, but attendance for our sessions was good. There were interesting discussions and questions from audience members who were obviously new to the tools and curious to learn more. Personally, I had a great time both catching up with some familiar faces and meeting some new ones, all of whom I hope will become more regular contributors to the greater ecosystem of scientific python tools. Ondrej Certik, always full of energy, promptly posted his own report, along with some video from where we were staying.

An aside: when I tried to book the conference hotel all available rooms at the conference rates were taken, so a plan B was in order. I found online a room across the street that was cheaper than the Miami Hilton's conference rate and rented it. It turned out to be a great condo on the 50th floor of this tower, with free high quality internet, lots of work space we could use and a gorgeous view. Lesson learned: in this economic crisis, shop around: we got a great place and still saved money over the normal conference rate... The conference hotel is just barely visible, in the lower right hand corner of this shot:

I am currently in the process of collecting slides from the speakers; check that page later if you are interested in any talk for which I haven't posted them yet: I'll continue to update it as I receive them.

Until the next meeting...

[ Image credits: John D. Hunter and F.P., full album available here ]

Book review: Expert Python Programming

2009-03-09T12:06:00.000-07:00

Update: I've slightly modified the language of this review, which as my wife correctly pointed out to me, was unnecessarily harsh. While I stand by my previous evaluation of the book, I think the same things can be said in a more constructive tone.

While this isn't strictly a SciPy post, I've already have a few questions about this book, so I guess I'll tag it as 'scipy' as well, for those interested.

I recently reviewed the book Expert Python Programming by Tarek Ziadé. While not aimed at a scientific audience, the book covers a number of topics that we frequently discuss on the Numpy and Scipy lists (such as documentation and testing, workflows, API decisions, etc). Since I really prefer to write longer text in reST using Emacs than in a blog editor, I've posted the review over at my static site. Feel free to head over there if you are interested in the full review, I've only reproduced here the summary:

Summary

Expert Python Programming covers a list of very interesting topics regarding real-world development using Python. It assumes a reader who already knows the basics of the language and covers a number of important topics, both in the more advanced parts of the language and in terms of developing applications using Python. There is a strong emphasis on agile development practices, testing, documentation, application life cycle management and other aspects of 'real world' work. The list of topics covered is excellent, and the book is well printed and bound.

However, unfortunately it suffers from rather poor editing throughout, with a broken idea flow that makes for choppy reading. Very few ideas are properly developed, as the book relies excessively on code snippets, bullet lists and stand-out info boxes. Ultimately, this gives it more the feel of a set of notes than that of a coherent volume.

This should not be read as an indictment of the book: the table of contents alone is a list of 'right things to do' when using Python, and there is a great deal of useful material in all of the chapters. If you are looking for reference material, links and starting points for further reading, Expert Python Programming can be an excellent resource and well worth your investment. However, if you are searching for a text that develops complex concepts at length, delving into details and subtleties, it might not be the ideal tool for you. I hope this provides a useful picture of the qualities of this book.

Python tools for science go to SIAM: a trip report

2008-07-28T12:22:00.000-07:00

Update (2008.08.06): I have posted the slides I have available from the minisymposium described below.

The 2008 SIAM annual meeting was held July 7-11 in San Diego, and we had a healthy presence of Python-based tools for scientific computing. Randy LeVeque from U. Washington and I co-organized a 3-part minisymposium entitled Python and Sage: Open Source Scientific Computing. You can see the list of talks for parts I, II and III online. I should add that for the talks where we had cancellations, we ended up with excellent impromptu replacements by Bill Hart from Sandia National Labs and Travis Oliphant from Enthought.

Our sessions were well attended (especially considering the very large number of parallel sessions at the conference) and generated lively discussion. There's clearly a need in the scientific and mathematical community for open source, high level tools that integrate well with existing high performance codes written in Fortran, C or C++, while providing interactivity, visualization, access to the network, etc. This is the kind of idea that many of us in the SciPy and Sage worlds have been preaching for a while, but now the response from the wider community is much more positive than before (admittedly, years ago the tools were immature enough that they were very much of the "user assembly required" variety).

I am extremely pleased to report that we were chosen for the conference highlights page (many thanks to John Hunter, of matplotlib fame, for the great screenshot)! Given that this page mostly lists the invited plenaries and other special events, I am very happy about this (SIAM had literally hundreds of talks from parallel sessions to choose from and only two of those were highlighted). I think it is both a recognition of the work done by all our speakers, and of the relevance of the topic today. Thanks again to all who contributed.

We should take this level of interest as both a recognition an a challenge: as more scientists become interested in these tools for their everyday research, the pressure on high quality documentation, easy installation and deployment, good tutorials, etc, will only increase. Everyone is working hard on this, and in particular it's worth mentioning the excellent work that is being done this summer as part of the NumPy/SciPy Documentation Marathon 2008, where a lot of new documentation has been added to NumPy. This is an effort where anyone can contribute! The infrastructure is just a wiki where you can register to edit docs, and developers will later review and merge your contributions into the core project for the next release.

So let's all keep up the momentum going, and I hope to see many more interested faces, both new and old, at the upcoming SciPy 2008 conference at Caltech in August!