I suspect this is serious, not only a problem in the tests.
I looked at SupervisorMiscSpec.
The problem is creating a child inside handleChildTerminated

override def handleChildTerminated(context: ActorContext, child: ActorRef, children: Iterable[ActorRef]): Unit = {
            val newKid = context.actorOf(Props.empty, child.path.name)

That leaves something hanging and threads not removed.

The system.shutdown() terminates normally, but something is still left.

Probably a very strange use case, but anyway.

In BalancingDispatcherModelSpec it is "handle waves of actors" that cause the leak.

Yeah, could definitely be true since the Dispatcher won't shut down if it's still inhabited.

Question is if this can solve it:

diff --git a/akka-actor/src/main/scala/akka/actor/ActorCell.scala b/akka-actor/src/main/scala/akka/actor/ActorCell.scala
index f922dae..8bbb149 100644
--- a/akka-actor/src/main/scala/akka/actor/ActorCell.scala
+++ b/akka-actor/src/main/scala/akka/actor/ActorCell.scala
@@ -508,8 +508,7 @@ private[akka] class ActorCell(
}

private def isTerminating = childrenRefs match {
- case TerminatingChildrenContainer(_, , Termination) ⇒ true
- case TerminatedChildrenContainer ⇒ true
+ case TerminatedChildrenContainer | TerminatingChildrenContainer(, , Termination) ⇒ true
case _ ⇒ false
}
private def isNormal = childrenRefs match {
@@ -981,11 +980,10 @@ private[akka] class ActorCell(
childrenRefs match {
case tc @ TerminatingChildrenContainer(, _, reason) ⇒
val n = removeChild(child)
- actor.supervisorStrategy.handleChildTerminated(this, child, children)
if (!n.isInstanceOf[TerminatingChildrenContainer]) reason match {
case Recreation(cause) ⇒ doRecreate(cause, actor) // doRecreate since this is the continuation of "recreate"
case Termination ⇒ doTerminate()
- case _ ⇒
+ case _ ⇒ actor.supervisorStrategy.handleChildTerminated(this, child, children)
}
case _ ⇒
removeChild(child)

The problem is some missing fault handling logic which discards failures while shutting down (no point to escalate them at that point). I have the proper fix on my branch for #2212.

with the first fix, ActorModelSpec still leaks, don’t know the reason yet

found it: if an actor dies real fast (e.g. context.stop(self) within the constructor), then Supervise might arrive after ChildTerminated at the parent, leaving a stale entry in the children list.

I also wonder why BalancingDispatcherModelSpec does not fail sometimes, because the cachedMessage may not be delivered in case the child dies too fast. Did not see that happen on my machine, though. It’s probably the deadLetters race which saves us here …

Did you verify that the problem with leaking perm gen was fixed also. Run all tests several times and make sure that perm gen doesn't incr and no OOME.

/Patrik

Updating tickets (#2186, #2187, #2189, #2192, #2196, #2197, #2198, #2200, #2203, #2208, #2212, #2217, #2218, #2221, #2224, #2225, #2226, #2227, #2228, #2232, #2236, #2237, #2247, #2248, #2255, #2257, #2260, #2262, #2268, #2269, #2272, #2276, #2285, #2288, #2292, #2294, #2295, #2298, #2301, #2302, #2312, #2315, #2318, #2319, #2332, #2333, #2335, #2337, #2340, #2341, #2342, #2345)