Right now the theano unit tests don't check performance, just correctness. This is reasonable, as checking performance (a) isn't necessary on new installs and (b) would slow down development. However, performance testing is important -- cf. http://groups.google.com/group/theano-dev/browse_thread/thread/c03365bb03d19604. The deep learning tutorials, which run the same code over the same data set are an excellent 'realistic use' test case for theano. I propose that a buildbot should be set up that runs this code, and that significant variations in runtime vs. the prior run or vs. some known standard should trigger an investigation into the change (whether good or bad).