#231
  • Created on
    2010-05-21 12:35
  • Reported by
    Jonas Bonér
  • Status
    Duplicate
  • Priority
    Normal (3)
  • Assigned to
    -
  • Milestone
    2.0-M1
  • Plan Level
    -
  • Due Date
    -

Attachments

Related Tickets

Add new subtask or bug

REMOTE: Support trap exit of remote actor crash generically

Currently we only track trap exit when sending the reply message after a ? message.

Proposed solution:

Clustered actor has two parts:
  • The client/ClusterActorRef on the "client" cluster node
  • The remote instance(s) (that the client talks to) on another cluster node

Part 1 - The remote instance side of things
1. Create a hidden "system" Supervisor actor on the cluster node
2. Link the remote actor instance to this Supervisor
3. ZK is used to set a Watch (ephemeral node) for linked actor
4. The path for this Watch is send back to the ClusterActorRef that represents this instance (see part 2 for how that is managed on in the ClusterActorRef)
5. Now when remote actor fails the Watch should be removed (which will trigger the ClusterActorRef on the client side)
6. ... wait for ClusterActorRef to do its job
7. When Supervisor receives a Restart message from the ClusterActorRef then it sends an Exit message to its linked actor to restart the remote actor.

Part 2 - The client/ClusterActorRef side of things
1. The ClusterActorRef receives the path to the Watch from the remote instance
2. It then adds a Listener to this path (Watch)
3. When Watch is triggered, notifying our Listener then we send the remote actor a Restart message to the "system" Supervisor on this node (go to part 1 item 7)

Use the RemoteDaemonActor channel (in ClusterNode) for all "system" communication. Add necessary commands to the ClusterProtocol.proto.

All life-cycle messages (Restart, Exit, etc.) should be sent to a the EventHandler.
User picture
  on 2010-07-02 11:11 *
By Jonas Bonér
Milestone changed from 0.10 to 0.11
Updating tickets (#135, #257, #258, #259, #261, #262, #263, #264, #265, #207, #273, #208, #194, #225, #227, #228, #231, #198, #299, #300, #255, #117)
User picture
  on 2010-08-19 12:08 *
By Jonas Bonér
Milestone changed from 0.11 to 1.0
Updating tickets (#135, #181, #194, #207, #208, #209, #228, #230, #231, #250, #255, #257, #258, #259, #261, #262, #263, #264, #265, #273, #298, #299, #300, #322, #323, #326, #343, #346, #363, #364, #373, #378, #380, #381, #384, #385, #387, #399)

Skipping 0.11. Next release is 1.0.
User picture
  on 2010-09-02 07:03 *
By Jonas Bonér
Description changed from Currently we only track tra... to Currently we only track tra...
Priority changed from Normal (3) to Highest (1)
User picture
  on 2010-09-23 06:53 *
By viktorklang
I'm guessing that this comment is linked to this: -)

private def notifySupervisorWithMessage(notification: LifeCycleMessage) = {
// FIXME to fix supervisor restart of remote actor for oneway calls, inject a supervisor proxy that can send notification back to client
_supervisor.foreach { sup =>
if (sup.isShutdown) { // if supervisor is shut down, game over for all linked actors
shutdownLinkedActors
stop
} else sup ! notification // else notify supervisor
}
}

Couldn't it be solved in implementing "notifySupervisorWithMessage" differently in RemoteActorRef (which I am assuming that _supervisor is transformed into)?
User picture
  on 2010-10-15 05:17 *
By viktorklang
Milestone changed from 1.0-MILESTONE1 to 1.0-MILESTONE2
User picture
  on 2010-10-28 07:49 *
By Jonas Bonér
Assigned to set to viktorklang
This is highest prio IMHO.
User picture
  on 2010-10-28 07:54 *
By viktorklang
I'd like some assistance with this, what's needed and which solution is most viable?
User picture
  on 2010-11-21 12:11 *
By viktorklang
Milestone changed from 1.0-MILESTONE2 to Backlog
User picture
  on 2010-11-23 00:24 *
By Jonas Bonér
Milestone changed from Backlog to 1.1
Updating tickets (#231, #87, #429, #439, #461, #470, #503, #512, #514, #520, #521, #534, #538, #539, #550, #209, #384, #387, #416, #455, #487, #497)
User picture
  on 2011-02-23 07:28 *
By Jonas Bonér
Assigned to changed from viktorklang to -none-
Milestone changed from 1.1 to 1.2
Priority changed from Highest (1) to Normal (3)
User picture
  on 2011-04-06 00:10 *
By Jonas Bonér
Milestone changed from 1.2 to 2.0
Updating tickets (#684, #87, #231, #439, #461, #470, #503, #592, #630, #637, #639, #641, #642, #657, #691, #692, #693, #719, #724, #725, #728, #729, #384, #455, #487, #601)
User picture
  on 2011-06-25 07:39 *
By Jonas Bonér
Component changed from None to cluster
User picture
  on 2011-07-06 01:47 *
By Jonas Bonér
Description changed from Currently we only track tra... to Currently we only track tra...
User picture
  on 2011-07-06 02:03 *
By Jonas Bonér
Description changed from Currently we only track tra... to Currently we only track tra...
User picture
  on 2011-10-07 07:13 *
By viktorklang
Summary changed from Support trap exit of remote actor crash generically to REMOTE: Support trap exit of remote actor crash generically
User picture
  on 2011-10-19 15:39 *
By viktorklang
Status changed from New to Duplicate
This is superceded by the Remote DeathWatch
User picture
  on 2011-12-14 16:54 *
By viktorklang
Milestone changed from 2.0 to 2.0-M1
Updating tickets (#684, #883, #886, #995, #1017, #968, #969, #973, #1002, #1016, #1021, #1035, #1048, #1119, #1156, #1451, #231, #470, #595, #599, #639, #655, #687, #691, #692, #693, #724, #728, #729, #749, #751, #755, #765, #766, #767, #769, #770, #771, #772, #784, #785, #807)