Hi @drewhk, do you consider this as bug?

This is expected behaviour. I took care to make ByteArrayIterator consistent with the Scala collections framework. Converting an iterator to a collection consumes it, therefore it should be empty afterwards. Try with toVector on a Scala List iterator, for example.

That is how iterators work: when you consume their elements then they are empty. toByteString always consumes everything that’s left in this ByteIterator. But as I tried to say, that is not related to the issue observed by the creator of the ticket. That issue is caused by assuming that take() copies the iterator, which it does not. The latter has probably been chosen for performance reasons, it does not match standard Iterator semantics. But interleaving calls to the original iterator and the one returned by take() in the case of scala.collection.Iterator is also not well-defined. Therefore I think that the current behavior is consistent enough

scala> var i = 0
i: Int = 0

scala> Iterator.continually({i += 1; i})
res5: Iterator[Int] = non-empty iterator

scala> res5.take(2).toList
res6: List[Int] = List(1, 2)

scala> res5.take(2).toList
res7: List[Int] = List(3, 4)

scala> Iterator(1,2,3,4)
res8: Iterator[Int] = non-empty iterator

scala> res8.take(2).toList
res9: List[Int] = List(1, 2)

scala> res8.take(2).toList
res10: List[Int] = List(1, 2)

Exactly - the Scala docs for Iterator.take (and many other similar methods) state "Reuse: After calling this method, one should discard the iterator it was called on, and use only the iterator that was returned. [...]". So there's freedom in how to implement it, and it was indeed done this way for performance reasons, to avoid object creation (working at the byte level, performance is important ... :-) ).

:(
why not create 2 methods, toByteString and asByteString
my personal opinion is that if I have 'toByteString' I expect new object created, and for asByteString it will clean iterator.
Thus later on you can see which one is being used... this approach also will give you ability to document them both properly and highlight the difference

Hi @oschulz, @drewhk

Example to convince you:
val size = iterator.getLong
process( iterator.take(size).toByteString() )
val size2 = iterator.getLong
process( iterator.take(size2).toByteString() )
and so on....but instead I have to iterator.take(size2).clone.toByteString() which is not very consistent with iterator.getLong

Or maybe add new method like iterator.getByteString(size:Int) similar to iterator.getLong ?

@yklymko, the code you present would not work in any case, toByteString has nothing to do with your problem. The problem is rather that after iterator.take(size) that iterator will end after size, thus your line with size2 will never work.

Your final suggestion is the proper fix, I’ll rename and reopen the ticket.

thx @rkuhn

Yes, getByteString(n: Int) makes sense. Or maybe simply call it "getBytes(n: Int)", in addition to the "getBytes(xs: Array[Byte], offset: Int, n: Int)" that we already have?

moving remainder of Coltrane to Rollins