ClusterSingleton may appear duplicated when seed nodes die
Bsically, if we have a few seed nodes, a partition happens so that a singleton is running on a "detached cluster", and nodes "come back" by forming a new cluster - they may decide to run the singleton - because they don't know about the node that's running the singleton, nor is any of the seed nodes in contact with it.
This task is about introducing patterns to remove this problem. One pattern is using roles to delay starting the singleton until more seed nodes know the nodes that have joined. Another one is manualy joining nodes and using an external discovery service (zookeeper for example) to find nodes.
Note: We currently do not do autodiscovery in akka, which would also help to kill this problem.
Discussed in detail here: https://groups.google.com/forum/#!topic/akka-user/ns7DPHGYbIk
This task is about introducing patterns to remove this problem. One pattern is using roles to delay starting the singleton until more seed nodes know the nodes that have joined. Another one is manualy joining nodes and using an external discovery service (zookeeper for example) to find nodes.
Note: We currently do not do autodiscovery in akka, which would also help to kill this problem.
Discussed in detail here: https://groups.google.com/forum/#!topic/akka-user/ns7DPHGYbIk
Leave a comment
on 2014-04-08 13:19 *
By Patrik Nordwall
and in the end it was because auto-down was used without caution, and then you can always end up with separate clusters and multiple singletons
on 2014-04-08 14:18 *
By Konrad Malawski
I'll draft something up to improve http://doc.akka.io/docs/akka/2.3.1/contrib/cluster-singleton.html#cluster-singleton so people are aware about these issues.