TailChoppingRouter
"In Dean’s experiments, BigTable’s 99.9th percentile latency dropped to 50ms when he sent out a second, redundant request if the initial request hadn’t come back in 10ms—a 40x improvement." - http://www.bailis.org/blog/doing-redundant-work-to-speed-up-distributed-queries/
So the idea is to have a router that:
This should have lower resource use than a ScatterGatherFirstCompletedRouter but would still serve to reduce variance in most cases.
It might also be possible to generalise the ScatterGatherFirstCompletedRouter to support this use case.
So the idea is to have a router that:
- Routes a message to one routee at first.
- If it hasn't received a reply in time x then route the message to a second routee.
- Reply with the first result received; discard any redundant replies.
This should have lower resource use than a ScatterGatherFirstCompletedRouter but would still serve to reduce variance in most cases.
It might also be possible to generalise the ScatterGatherFirstCompletedRouter to support this use case.
Leave a comment
This is a specific case of http://en.wikipedia.org/wiki/Speculative_execution.
This optimization only applies for large batch kind of work with enough available workers -- it deserves some thought, could be useful for someone!
This optimization only applies for large batch kind of work with enough available workers -- it deserves some thought, could be useful for someone!