#514
- Created on2010-10-23 01:20
- Reported by
- StatusNew
- PriorityNormal (3)
- Assigned to-
- Milestone
- Plan Level-
- Due Date-
Detect theano bug in old experiment.
Here is a way to allow detecting Theano bug in old experiment. This can be usefull to discover old experiment that was not behaving correctly due to some Theano bug! This also help raise the confidence of other people in Theano.
Here is how.
We add a parameter record=FILEPATH to theano compilation function. When we have this parameter and we execute this function for the first time, we record into this file the following information:
1) The function before optimization(The signature of each op(op, version, inputs,...) in toposort order)
2) The function after optimization(as 1)
3) All optimization event
So when we discover a bug in Theano, we can just check our past recorded file for the bad case. This way we can know if this bug affected us or not.
We record the data in the file when we use the Theano function the first time only. This allow to compile function that we don't use without generating false problem report when we parse the file.
Here is how.
We add a parameter record=FILEPATH to theano compilation function. When we have this parameter and we execute this function for the first time, we record into this file the following information:
1) The function before optimization(The signature of each op(op, version, inputs,...) in toposort order)
2) The function after optimization(as 1)
3) All optimization event
So when we discover a bug in Theano, we can just check our past recorded file for the bad case. This way we can know if this bug affected us or not.
We record the data in the file when we use the Theano function the first time only. This allow to compile function that we don't use without generating false problem report when we parse the file.
Leave a comment
Should we record that automatically in the user home directory somewhere? That would have the problem that the user won't easily be able to trac down what caused this problem. To be able to trac it correctly, the user must put that file with the result of the experiments.
Should jobman deal record it automatically? That way it is more tractable and automatic.
Should jobman deal record it automatically? That way it is more tractable and automatic.
We could use a Theano flags to set the path to this file. We can make it default to something in the current directory. That way as most experience are computed with jobman they will have that file in their experiment directory. How this will affect jobs launched on other backend or with condor without jobman?
We
We
We could use a Theano flags to set the path to this file. We can make it default to something in the current directory. That way as most experience are computed with jobman they will have that file in their experiment directory. How this will affect jobs launched on other backend or with condor without jobman?
We
We