#2266
  • Created on
    2012-06-25 07:17
  • Reported by
    Patrik Nordwall
  • Status
    Invalid
  • Priority
    Normal (3)
  • Assigned to
    -
  • Milestone
    2.3-M1
  • Plan Level
    -
  • Due Date
    -

Attachments

Related Tickets

Add new subtask or bug

Auto-down threshold

To avoid auto-down of small group of nodes in case of network partition, see #2265, we need a configurable threshold to disable auto-down.
Configuration property could be a percentage of all nodes.
E.g. disable auto-down if unreachable / (unreachable + liveMembers) > 70%.
or enable auto-down only when unreachable / (unreachable + liveMembers) < 30%
Any personal preferences?

I think we need to introduce some delay of the auto-down action, otherwise it will not have the full picture of unreachable when taking this decision.
User picture
  on 2012-06-25 15:39 *
By Peter Vlugter
Might be interesting to see how Riak decided to auto-down. From memory, only explicit user down was supported there when we were thinking about this.
User picture
  on 2012-06-25 16:17 *
By Jonas Bonér
Yeah. The safest is only allow user-down, but that would be limiting I think. If we can get auto-down to work then it would be great. We need more tests to verify.
User picture
  on 2012-06-26 11:41 *
By Patrik Nordwall
We redefined how network partition should be handled. Is this still something we need?
Changing to lower prio, and we can think about it.
User picture
  on 2012-06-26 11:41 *
By Patrik Nordwall
Priority changed from Normal (3) to Low (4)
User picture
  on 2012-06-27 04:15 *
By Patrik Nordwall
Priority changed from Low (4) to High (2)
This is probably important. We should discuss the split brain semantics with the whole team.
User picture
  on 2012-06-27 04:49 *
By Jonas Bonér
Let's discuss tomorrow.
User picture
  on 2012-07-03 08:46 *
By Patrik Nordwall
Priority changed from High (2) to Low (4)
We discussed how to handle network partitions again, and the conclusion is that current implementation is fine. Changing priority to low, threshold for auto-down is something we can add later.
User picture
  on 2012-11-01 07:46 *
By viktorklang
Milestone changed from Coltrane to Rollins
User picture
  on 2012-11-01 07:46 *
By viktorklang
Priority changed from Low (4) to Normal (3)
User picture
  on 2013-09-24 04:05 *
By rkuhn
Assigned to set to login
Status changed from New to Invalid
closing this ticket for now: what users will need to do is to write little policy actors which decide what to do based on MembershipEvents, and once we see successful patterns we can incorporate them
User picture
  on 2013-10-18 01:24 *
By Patrik Nordwall
Milestone changed from Rollins to 2.3-M1