It's unwieldy to put enormous blocks of c code into python code with the string substitutions and stuff. Editing and testing a pure C plugin is sometimes much easier: I'm thinking of giants like conv and gemm in particular.

Consider writing these ops in pure C++ in pure C++ files, and getting theano to build them like is normal for mixed C/python libraries.

Also, consider an auto-generated cache for elemwise ops. The code is pretty stable, so lets just generate all ops for all types in each of 0,1,2,3,4,5-D tensors, and dump it to a file. This would speed compilation and keep the number of .so files in the theano compile cache from getting too big.