implement incremental reconcile in ScalaReconcilingStrategy
Right now ScalaReconcilingStrategy doesn't support incremental reconcile. It effects at least outline view, which doesn't update itself when scala source code changes.
Leave a comment
on 2015-09-16 08:23 *
By Iulian Dragos
Not sure if that's the right diagnostic. At least, quick outline (CMD-O on Mac) does show an up to date view of the class. But indeed, the Outline view is not up to date, so it might simply miss a refresh/other event trigger.
The call sequence is something like this (I dropped insignificant details):
IResourceChanedEvent ElementChangedEvent
Document --> Reconciler --> IReconcileStrategy --> ICompilationUnit --> JavaOutlinePage
depending on the configuration Reconciler calls either
IReconcilerStrategy.reconcile(DirtyRegion dirtyRegion, IRegion subRegion);
which is not implemented or
IReconcilerStrategy.reconcile(IRegion partition)
In java implementation last method calls ICompilationUnit which triggers ElementChangedEvent.
Scala Implementation doesn't do it. Even if it does, scala ICompilationUnit implementation (ScalaSourceFile) is empty:
/**
override def reconcile(
astLevel : Int,
reconcileFlags : Int,
workingCopyOwner : WorkingCopyOwner,
monitor : IProgressMonitor) : org.eclipse.jdt.core.dom.CompilationUnit = {
null
}
So, ResourceChangedEvent is never propagated to outline view.
IResourceChanedEvent ElementChangedEvent
Document --> Reconciler --> IReconcileStrategy --> ICompilationUnit --> JavaOutlinePage
depending on the configuration Reconciler calls either
IReconcilerStrategy.reconcile(DirtyRegion dirtyRegion, IRegion subRegion);
which is not implemented or
IReconcilerStrategy.reconcile(IRegion partition)
In java implementation last method calls ICompilationUnit which triggers ElementChangedEvent.
Scala Implementation doesn't do it. Even if it does, scala ICompilationUnit implementation (ScalaSourceFile) is empty:
/**
- We cut short this call since reconciliation is performed through the usual mechanism in the
- editor. Calls arriving here come from the JDT, for instance from the breadcrumb view, and end
- up doing expensive computation on the UI thread.
- @see #1002412
override def reconcile(
astLevel : Int,
reconcileFlags : Int,
workingCopyOwner : WorkingCopyOwner,
monitor : IProgressMonitor) : org.eclipse.jdt.core.dom.CompilationUnit = {
null
}
So, ResourceChangedEvent is never propagated to outline view.
on 2015-09-17 06:52 *
By Iulian Dragos
Thanks for the thorough analysis! I still wonder why the Quick Outline is always up to date, though.
BTW, the shortcut was necessary because our "structure builder" is very slow, and can't afford doing this on the UI thread (for instance, during a Save this would block the user, sometimes for a second or more). However, the structure buildier is called on a background thread whenever Scala is done type-checking, so maybe we can publish the right event on that code-path. This area of the IDE is a minefield, so be prepared for some detective work!
BTW, the shortcut was necessary because our "structure builder" is very slow, and can't afford doing this on the UI thread (for instance, during a Save this would block the user, sometimes for a second or more). However, the structure buildier is called on a background thread whenever Scala is done type-checking, so maybe we can publish the right event on that code-path. This area of the IDE is a minefield, so be prepared for some detective work!
on 2015-09-18 06:50 *
By Iulian Dragos
Here's another idea, which I think is waaay better (and more interesting to code ;-)). In eclipse, editors can set their own implementation of an outline view. Right now we use the Java one, via a translation from Scala syntactic elements to Java ones. This mapping is approximate, and adds synthetic fields/methods, etc.
It would be way more efficient to create a new Outline view, specific to the Scala editor. This would use the parse tree (as opposed to Java elements), so it would be 1:1 to the source code. No more synthetics, no more getters and setters instead of vars, etc. Plus, since it doesn't need typechecking, it's instantly fast.
But a lot more work. However, it's clean "state" work ;-)
Here's a few tickets #1000863, #1001711, #1000500, #1001144, #1000746, #1002356 all related to the outline view.
It would be way more efficient to create a new Outline view, specific to the Scala editor. This would use the parse tree (as opposed to Java elements), so it would be 1:1 to the source code. No more synthetics, no more getters and setters instead of vars, etc. Plus, since it doesn't need typechecking, it's instantly fast.
But a lot more work. However, it's clean "state" work ;-)
Here's a few tickets #1000863, #1001711, #1000500, #1001144, #1000746, #1002356 all related to the outline view.
on 2015-09-18 06:51 *
By Iulian Dragos
And the good news is that Eclipse has an API for that: https://wiki.eclipse.org/FAQ_How_do_I_create_an_Outline_view_for_my_own_language_editor%3F
Do you mean using scala Tree as a model for OutlineView? Yes, it would be more natural. I'm confused a little bit with scala Tree. Who keeps it in memory? Is it rebuild every time we build Java model? Or it is cached somewhere? What happens when source code gets modified?
And why "structure builder" is slow? It looks like every Java element is a pretty much wrapper around scala element. Why does wrapper creation take much time?
And why "structure builder" is slow? It looks like every Java element is a pretty much wrapper around scala element. Why does wrapper creation take much time?
on 2015-09-19 09:25 *
By Simon Schäfer
The IDE parses every Scala file the moment it is modified. The resulting tree can be used in a Scala outline view. This operation is fast, there is no caching needed not even for Scala files with 10k lines of code. The Structure Builder on the other side doesn't work with an arbitrary tree, but with a typechecked one, therefore we first have to ask the compiler to typecheck the tree. This is slow and can't be done in the Ui thread. Converting the resulting typechecked tree in Java elements isn't that expensive anymore.
The IDE also uses different trees than the compiler. After typechecking the compiler returns scala.reflect trees, whereas the IDE can work with anything. In the past we used scalariform to parse Scala files (I suggest that for a Scala outline view it should be used too). For partition creation we do scanning of Scala code even by ourselves but we don't create a tree afterwards, just a list of regions.
The IDE also uses different trees than the compiler. After typechecking the compiler returns scala.reflect trees, whereas the IDE can work with anything. In the past we used scalariform to parse Scala files (I suggest that for a Scala outline view it should be used too). For partition creation we do scanning of Scala code even by ourselves but we don't create a tree afterwards, just a list of regions.
on 2015-09-19 17:02 *
By Simon Schäfer
We don't even have to use scalariform as I just noticed. If we have our own Outline View, we could easily fill it with semantic information in the background.
on 2015-09-20 09:24 *
By Iulian Dragos
My take is that the outline view should really follow the source code. If the return type is unspecified, it should be reflected in the outline view. We can improve on this later, if we want, but as a first step it's fine to not show any return type.
Additionally, I'd suggest using Scala parse trees rather than scalariform, given that the Scala parser is considerably faster and more resilient to errors. Scalariform doesn't return anything (None) if there's any syntax error, which is the case most of the times when editing. It would be bad to not have any outline if the user didn't close a string or parenthesis.
Additionally, I'd suggest using Scala parse trees rather than scalariform, given that the Scala parser is considerably faster and more resilient to errors. Scalariform doesn't return anything (None) if there's any syntax error, which is the case most of the times when editing. It would be bad to not have any outline if the user didn't close a string or parenthesis.
I implemented initial version of the Outline View based on parsed tree. It is not final yet. It doesn't have any filters. It should be straightforward. But before doing this I want to sync up. I have it in my branch:
https://github.com/andrey-ilinykh/scala-ide/tree/feature/incr-reconcile-1002545
Could someone have a look? Basically, I created one more reconciling strategy which parses source code and build a model which is used by ContentProvider. It is pretty fast- 10 20 ms even for big files (several thousand of lines)
https://github.com/andrey-ilinykh/scala-ide/tree/feature/incr-reconcile-1002545
Could someone have a look? Basically, I created one more reconciling strategy which parses source code and build a model which is used by ContentProvider. It is pretty fast- 10 20 ms even for big files (several thousand of lines)
on 2015-10-09 07:53 *
By Simon Schäfer
Please open a PR, that is for what these things were made.
on 2015-10-09 16:40 *
By Simon Schäfer
PR = pull request on github
Fixed by #1014