#2534
- Created on2012-09-21 00:40
- Reported by
- StatusInvalid
- PriorityHigh (2)
- Assigned to
- Milestone
- Plan Level-
- Due Date-
become singleton cluster when one node is shutdown *** FAILED
I've seen this error on a few runs.
Replicate with: akka-cluster-experimental > test
On: mac OS
[JVM-Node1] [WARN] [09/20/2012 18:23:27.137] [SingletonClusterSpec-akka.actor.default-dispatcher-2] [akka://SingletonClusterSpec/user/TestConductorClient] received dead letter from Actor[akka://SingletonClusterSpec/deadLetters]: PoisonPill
[JVM-Node1] - become singleton cluster when one node is shutdown *** FAILED *** (4 seconds, 412 milliseconds)
[JVM-Node1] false was not true (MultiNodeClusterSpec.scala:167)
[JVM-Node1] org.scalatest.exceptions.TestFailedException:
[JVM-Node1] at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:155)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.newTestFailedException(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.matchers.MustMatchers$MustMethodHelper$.mustMatcher(MustMatchers.scala:884)
[JVM-Node1] at org.scalatest.matchers.MustMatchers$AnyMustWrapper.must(MustMatchers.scala:912)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.assertLeaderIn(MultiNodeClusterSpec.scala:167)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.assertLeaderIn(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.assertLeader(MultiNodeClusterSpec.scala:156)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.assertLeader(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3$$anonfun$apply$mcV$sp$4.apply$mcV$sp(SingletonClusterSpec.scala:69)
[JVM-Node1] at akka.remote.testkit.MultiNodeSpec.runOn(MultiNodeSpec.scala:253)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply$mcV$sp(SingletonClusterSpec.scala:61)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(SingletonClusterSpec.scala:60)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(SingletonClusterSpec.scala:60)
[JVM-Node1] at org.scalatest.WordSpec$$anon$2.apply(WordSpec.scala:2179)
[JVM-Node1] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.akka$cluster$MultiNodeClusterSpec$$super$withFixture(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.withFixture(MultiNodeClusterSpec.scala:87)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.withFixture(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$class.invokeWithFixture$1(WordSpec.scala:2176)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTest$1.apply(WordSpec.scala:2185)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTest$1.apply(WordSpec.scala:2185)
[JVM-Node1] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:198)
[JVM-Node1] at org.scalatest.WordSpec$class.runTest(WordSpec.scala:2185)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.runTest(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTests$1.apply(WordSpec.scala:2250)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTests$1.apply(WordSpec.scala:2250)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:260)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:265)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249)
[JVM-Node1] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:326)
[JVM-Node1] at org.scalatest.WordSpec$class.runTests(WordSpec.scala:2250)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.runTests(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.Suite$class.run(Suite.scala:2303)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.org$scalatest$WordSpec$$super$run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$run$1.apply(WordSpec.scala:2297)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$run$1.apply(WordSpec.scala:2297)
[JVM-Node1] at org.scalatest.SuperEngine.runImpl(Engine.scala:362)
[JVM-Node1] at org.scalatest.WordSpec$class.run(WordSpec.scala:2297)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.org$scalatest$BeforeAndAfterAll$$super$run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:213)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:60)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:1560)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:1557)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1557)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:697)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:696)
[JVM-Node1] at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1601)
[JVM-Node1] at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:695)
[JVM-Node1] at org.scalatest.tools.Runner$.main(Runner.scala:571)
[JVM-Node1] at org.scalatest.tools.Runner.main(Runner.scala)
[JVM-Node1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JVM-Node1] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[JVM-Node1] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[JVM-Node1] at java.lang.reflect.Method.invoke(Method.java:597)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:71)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:139)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:71)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:139)
[JVM-Node1] at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:28)
[JVM-Node1] at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:45)
[JVM-Node1] at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:35)
[JVM-Node1] at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:45)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:74)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:105)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Replicate with: akka-cluster-experimental > test
On: mac OS
[JVM-Node1] [WARN] [09/20/2012 18:23:27.137] [SingletonClusterSpec-akka.actor.default-dispatcher-2] [akka://SingletonClusterSpec/user/TestConductorClient] received dead letter from Actor[akka://SingletonClusterSpec/deadLetters]: PoisonPill
[JVM-Node1] - become singleton cluster when one node is shutdown *** FAILED *** (4 seconds, 412 milliseconds)
[JVM-Node1] false was not true (MultiNodeClusterSpec.scala:167)
[JVM-Node1] org.scalatest.exceptions.TestFailedException:
[JVM-Node1] at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:155)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.newTestFailedException(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.matchers.MustMatchers$MustMethodHelper$.mustMatcher(MustMatchers.scala:884)
[JVM-Node1] at org.scalatest.matchers.MustMatchers$AnyMustWrapper.must(MustMatchers.scala:912)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.assertLeaderIn(MultiNodeClusterSpec.scala:167)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.assertLeaderIn(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.assertLeader(MultiNodeClusterSpec.scala:156)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.assertLeader(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3$$anonfun$apply$mcV$sp$4.apply$mcV$sp(SingletonClusterSpec.scala:69)
[JVM-Node1] at akka.remote.testkit.MultiNodeSpec.runOn(MultiNodeSpec.scala:253)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply$mcV$sp(SingletonClusterSpec.scala:61)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(SingletonClusterSpec.scala:60)
[JVM-Node1] at akka.cluster.SingletonClusterSpec$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(SingletonClusterSpec.scala:60)
[JVM-Node1] at org.scalatest.WordSpec$$anon$2.apply(WordSpec.scala:2179)
[JVM-Node1] at org.scalatest.Suite$class.withFixture(Suite.scala:1974)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.akka$cluster$MultiNodeClusterSpec$$super$withFixture(SingletonClusterSpec.scala:34)
[JVM-Node1] at akka.cluster.MultiNodeClusterSpec$class.withFixture(MultiNodeClusterSpec.scala:87)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.withFixture(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$class.invokeWithFixture$1(WordSpec.scala:2176)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTest$1.apply(WordSpec.scala:2185)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTest$1.apply(WordSpec.scala:2185)
[JVM-Node1] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:198)
[JVM-Node1] at org.scalatest.WordSpec$class.runTest(WordSpec.scala:2185)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.runTest(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTests$1.apply(WordSpec.scala:2250)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$runTests$1.apply(WordSpec.scala:2250)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:260)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:265)
[JVM-Node1] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249)
[JVM-Node1] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:326)
[JVM-Node1] at org.scalatest.WordSpec$class.runTests(WordSpec.scala:2250)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.runTests(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.Suite$class.run(Suite.scala:2303)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.org$scalatest$WordSpec$$super$run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$run$1.apply(WordSpec.scala:2297)
[JVM-Node1] at org.scalatest.WordSpec$$anonfun$run$1.apply(WordSpec.scala:2297)
[JVM-Node1] at org.scalatest.SuperEngine.runImpl(Engine.scala:362)
[JVM-Node1] at org.scalatest.WordSpec$class.run(WordSpec.scala:2297)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.org$scalatest$BeforeAndAfterAll$$super$run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:213)
[JVM-Node1] at akka.cluster.SingletonClusterSpec.run(SingletonClusterSpec.scala:34)
[JVM-Node1] at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:60)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:1560)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:1557)
[JVM-Node1] at scala.collection.immutable.List.foreach(List.scala:309)
[JVM-Node1] at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1557)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:697)
[JVM-Node1] at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:696)
[JVM-Node1] at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1601)
[JVM-Node1] at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:695)
[JVM-Node1] at org.scalatest.tools.Runner$.main(Runner.scala:571)
[JVM-Node1] at org.scalatest.tools.Runner.main(Runner.scala)
[JVM-Node1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JVM-Node1] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[JVM-Node1] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[JVM-Node1] at java.lang.reflect.Method.invoke(Method.java:597)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:71)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:139)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:71)
[JVM-Node1] at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:139)
[JVM-Node1] at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:28)
[JVM-Node1] at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:45)
[JVM-Node1] at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:35)
[JVM-Node1] at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:45)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:74)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:105)
[JVM-Node1] at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
Leave a comment
on 2012-09-21 05:53 *
By Patrik Nordwall
Assigned to set to Patrik Nordwall
Status changed from New to Accepted
Thanks, I see what the problem is. It's a residue of that we update the clusterView with published events, which can arrive later than before. I will fix the test.
on 2012-09-21 11:11 *
By viktorklang
2.0-M3?
on 2012-09-21 15:27 *
By Patrik Nordwall
I don't think my first hypothesis holds. I will investigate more thoroughly next week.
on 2012-09-24 09:38 *
By Patrik Nordwall
I can't reproduce this and I can't see that it should be a problem. Did some println debugging as well. The LeaderChanged notifications has been rewritten lately so it might be that this was fixed by that. Please provide the full log of the failing test if you see it again.
I assume it is akka.cluster.SingletonClusterWithAccrualFailureDetector
I assume it is akka.cluster.SingletonClusterWithAccrualFailureDetector
Can't see that it could go wrong with latest master. Missing full log. Can't reproduce. Re-open if you see it again.
Updating tickets (#939, #940, #1941, #2213, #2214, #2215, #2219, #2222, #2223, #2239, #2240, #2249, #2250, #2252, #2253, #2254, #2256, #2259, #2263, #2264, #2265, #2267, #2270, #2271, #2275, #2277, #2286, #2287, #2289, #2290, #2303, #2304, #2308, #2310, #2311, #2317, #2323, #2331, #2374, #2392, #2405, #2423, #2425, #2440, #2444, #2445, #2453, #2456, #2459, #2473, #2477, #2491, #2495, #2523, #2534, #2541, #2544, #2545, #2549, #2582, #2583, #2589, #2626)
Updating tickets (#939, #940, #1941, #2081, #2126, #2213, #2214, #2215, #2219, #2222, #2223, #2239, #2240, #2249, #2250, #2252, #2253, #2254, #2256, #2259, #2263, #2264, #2265, #2267, #2270, #2271, #2275, #2277, #2286, #2287, #2289, #2290, #2303, #2304, #2308, #2310, #2311, #2317, #2323, #2331, #2374, #2392, #2394, #2405, #2408, #2423, #2424, #2425, #2440, #2444, #2445, #2449, #2453, #2456, #2459, #2461, #2473, #2477, #2485, #2491, #2495, #2498, #2501, #2505, #2515, #2517, #2523, #2534, #2541, #2544, #2545, #2549, #2582, #2583, #2588, #2589, #2598, #2599, #2618, #2623, #2626, #2627, #2630, #2631, #2633, #2634, #2635, #2637, #2638, #2642, #2643, #2646, #2647, #2648, #2649, #2650, #2653, #2655, #2657, #2658)