Currently, gradient expression generation will fail if any op in the graph fails hasattr(op, 'grad'). This is too strict. It would be sufficient to only complain if this check fails for any op on the topological path from 'wrt' parameters to 'cost' parameters. Consider:


b + a[op(c)]* differentiable wrt 'b', even if op.grad() is not defined