#436
  • Created on
    2009-10-20 18:33
  • Reported by
    Anonymous
  • Status
    New
  • Priority
    Normal (3)
  • Assigned to
    -
  • Milestone
    0.2
  • Plan Level
    -
  • Due Date
    -

Attachments

Related Tickets

Add new subtask or bug

cmodule take too long on NFS to find all old precomputed file

Right now, cmodule put each compiled op in a separate direcotory. when cmodule start it list thousands of direcotory and read thousands of file over NFS just to find witch op was previously compiled. This take too much time on NFS. I went from 40s of compilatio time to 5s just by deleting the .theano.NOBAKCUP direcotory.

We should make a resume file of all directory. When cmodule start it would read this file and make the directory listing of .theano/arch/ for each directory not in the summary, add it. After write new new summary. This should work correctly with multiple process on different computer as event if we lose a copy of the summary file, we see what we don't have and complete it.
User picture
  on 2011-03-21 20:19 *
By nouiz
ticket* 638 should help that by having less module to load the key. ticket #635 could allow us to don't load the key.plk file of each precompiled module
User picture
  on 2011-03-31 19:47 *
By lamblin
ticket* 638 should help that by having less module to load the key. ticket #635 could allow us to don't load the key.plk file of each precompiled module
User picture
  on 2011-04-07 16:18 *
By nouiz
ticket* 638 should help that by having less module to load the key. ticket #635 could allow us to don't load the key.plk file of each precompiled module