I've tried to configure my own mailbox type to be used by routees within BalancingPool.
I've tried a lot of ways to do it without positive result:

• via akka.actor.deployment.<actor-name> section and pool-dispatcher subsection in application.conf
• directly from the code

After investigation of the code I've found the working way, but it seems not to be forethought by developers.
There is configuration and code snippets:

application.conf:

akka {
  actor {
    deployment {
      /parent/child {
        router = balancing-pool
        nr-of-instances = 5
        pool-dispatcher {
          mailbox-type = "my.Mailbox"
       }
    }
  }
}  

BalancingPool-/parent/child {
  type = akka.dispatch.BalancingDispatcherConfigurator
  mailbox-type = my.Mailbox
  mailbox-capacity = 30000
  # rest of the dispatcher configuration
}

Looks strange, isn't it?

I think there is a bug in the BalancingPool.newRoute(...) and BalancingDispatcherConfigurator bundle.

The fact is that when BalancingDispatcherConfigurator configures internal BalancingDispatcher instance it obtains maibox type erroneously:

class BalancingDispatcherConfigurator(_config: Config, _prerequisites: DispatcherPrerequisites)
  extends MessageDispatcherConfigurator(BalancingDispatcherConfigurator.amendConfig(_config), _prerequisites) {

  private val instance = {
    val mailboxes = prerequisites.mailboxes
    val id = config.getString("id")
    val requirement = mailboxes.getMailboxRequirement(config)
    if (!classOf[MultipleConsumerSemantics].isAssignableFrom(requirement))
      throw new IllegalArgumentException(
        "BalancingDispatcher must have 'mailbox-requirement' which implements akka.dispatch.MultipleConsumerSemantics; " +
          s"dispatcher [$id] has [$requirement]")
    val mailboxType =
      if (config.hasPath("mailbox-type")) {
        val mt = mailboxes.lookup(id)
        if (!requirement.isAssignableFrom(mailboxes.getProducedMessageQueueType(mt)))
          throw new IllegalArgumentException(
            s"BalancingDispatcher [$id] has 'mailbox-type' [${mt.getClass}] which is incompatible with 'mailbox-requirement' [$requirement]")
        mt
      } else mailboxes.lookupByQueueType(requirement)
    create(mailboxType)
  }

  // rest of the class

Precisely it tries to find out mailbox configuration by pool's dispatcher ID, e.g. BalancingPool-<actor path>. Obviously, it's bug.

I expect next behavior for mailbox type resolving:
1. If there is 'mailbox-type' path in the self config (pool-dispatcher config) mailbox type should be looked up from this config
2. if there is 'mailbox' path in the self config it can be recognized as mailbox ID and used mailbox type should be looked up by this ID
3. lookupByQueueType

Or in the code snippet:

 val mailboxType =
      if (config.hasPath("mailbox-type")) {
        val mt = mailboxes.lookup(<path-to-self-config>)
        if (!requirement.isAssignableFrom(mailboxes.getProducedMessageQueueType(mt)))
          throw new IllegalArgumentException(
            s"BalancingDispatcher [$id] has 'mailbox-type' [${mt.getClass}] which is incompatible with 'mailbox-requirement' [$requirement]")
        mt
      } else if (config.hasPath("mailbox")) {
         val mt = mailboxes.lookup(config.getString("mailbox"))
         if (!requirement.isAssignableFrom(mailboxes.getProducedMessageQueueType(mt)))
          throw new IllegalArgumentException(
            s"BalancingDispatcher [$id] has 'mailbox-type' [${mt.getClass}] which is incompatible with 'mailbox-requirement' [$requirement]")
        mt  
      }  else mailboxes.lookupByQueueType(requirement)