What are "zip" methods in Scala and Spark?

In Scala, Spark and a lot of other "big data"-type frameworks, languages, libraries I see methods named "zip*". For instance, in Scala, List types have an inherent zipWithIndex method that you can use like so:

val listOfNames : List[String] = getSomehow() for((name,i) <- listOfNames.zipWithIndex) { println(s"Names #${i+1}: ${name}") } 

Similarly Spark has RDD methods like zip, zipPartitions, etc.

But the method name "zip" is totally throwing me off. Is this a concept in computing or discrete math?! What's the motivation for all these methods with "zip" in their names?

1 Answer

They are named zip because you are zipping two datasets like a zipper.

To visualize it, take two datasets:

x = [1,2,3,4,5,6] y = [a,b,c,d,e,f] 

and then zip them together to get

 1 a 2 b 3 c 4 d 5 e 6 f 

I put the extra spacing just give the zipper illusion as you move down the dataset :)