I find the backward version slightly more intuitive. Here’s why:
Suppose I want to uniformly randomly shuffle a deck of cards in a single pass. I stick the deck on the table and call it the non-shuffled pile. My goal is to move the deck, one card at a time, into the shuffled pile. First I need to select a card, uniformly at random, to be the bottom card of the new pile, and I move it over. Then I select another card, uniformly at random from the still non-shuffled cards, and put it on top of the bottom shuffled card. I repeat this until I’ve moved all the cards, so that each card in the shuffled pile is a uniform random selection from all of the cards it could have been. And that’s it.
One can think of this as random selection, whereas the “forward” version is like random insertion of a not-random card into a shuffled pile. And for whatever reason I tend to think of the selection version first.
Huh I didn't know the backwards version was more common, it seems odd.
You could also call the last version the online version, as it will ensure the partial list is random at any point in time (and can be used for inputs with indeterminate length, or to extend a random list with new elements, sample k elements etc.)
Not too sure if the enumerate is necessary. I usually dislike using it just to have an index to play around with. A similar way of doing the same thing is:
for x in source:
a.append(x)
i = random.randint(0, len(a))
a[i], a[-1] = a[-1], a[i]
Which makes the intention a bit clearer. You could even avoid the swap entirely but you would need to handle the case where i is at the end of the list separately.
Not quite sure what you have in mind here, but you need reservoir sampling for this in order to make the selection uniformly random (which I assume is what's desired)
You can just use this algorithm but ignore everything after the first k elements. The algorithm still works if you don't store anything beyond the first k elements but just pretend they are there.
enumerate() is just an awkward way to get len(a). In theory, you could somehow be in an environment where you have dynamically resizing arrays (vectors) that don't track their length internally. But in this case it's probably because OP doesn't have a firm grasp what's happening (which is why they wrote the blog post).
For me, the reason for reaching for the “backwards” version first is that it wasn’t as clear to me that the “forward” version makes a uniform distribution.
Even after reading the article.
I appreciated the comment here about inserting a card at a random location, but that also isn’t quite right, because you swap cards not insert. Nevertheless, that did it for me.
That’s funny. I’ve always done it the forwards way. I didn’t even realise that wasn’t the usual way.
I suppose one of the benefits of having a poor memory is that one sometimes improves things in the course of rederiving them from an imperfect recollection.
The article didn't mention it, but I think you can derive the FY shuffle using base-factorial.
Separate point: One way of deriving 4 different variants of Fisher-Yates is to take a random bijection f:[n]->[n] (where [n] means {0,1,...,n}) drawn using whichever variant of the FY shuffle, and then potentially pre-composing or post-composing with the bijection reverse:[n]->[n] that sends k to n - k - 1.
I find the backward version slightly more intuitive. Here’s why:
Suppose I want to uniformly randomly shuffle a deck of cards in a single pass. I stick the deck on the table and call it the non-shuffled pile. My goal is to move the deck, one card at a time, into the shuffled pile. First I need to select a card, uniformly at random, to be the bottom card of the new pile, and I move it over. Then I select another card, uniformly at random from the still non-shuffled cards, and put it on top of the bottom shuffled card. I repeat this until I’ve moved all the cards, so that each card in the shuffled pile is a uniform random selection from all of the cards it could have been. And that’s it.
One can think of this as random selection, whereas the “forward” version is like random insertion of a not-random card into a shuffled pile. And for whatever reason I tend to think of the selection version first.
For what it's worth, I think of the forward version as randomly selecting a card and putting it at the bottom of the new pile (0..i in the array).
There are actually four variants:
• loop counts downwards vs upwards
• the processed part of the array is a uniform sample of the whole array, or it is a segment that has been uniformly shuffled
Knuth described only the downwards sampling version, which is probably why it’s the most common.
The variants are compared quite well on wikipedia https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle
Huh I didn't know the backwards version was more common, it seems odd.
You could also call the last version the online version, as it will ensure the partial list is random at any point in time (and can be used for inputs with indeterminate length, or to extend a random list with new elements, sample k elements etc.)
Not too sure if the enumerate is necessary. I usually dislike using it just to have an index to play around with. A similar way of doing the same thing is:
Which makes the intention a bit clearer. You could even avoid the swap entirely but you would need to handle the case where i is at the end of the list separately.> sample k elements
Not quite sure what you have in mind here, but you need reservoir sampling for this in order to make the selection uniformly random (which I assume is what's desired)
You can just use this algorithm but ignore everything after the first k elements. The algorithm still works if you don't store anything beyond the first k elements but just pretend they are there.
enumerate() is just an awkward way to get len(a). In theory, you could somehow be in an environment where you have dynamically resizing arrays (vectors) that don't track their length internally. But in this case it's probably because OP doesn't have a firm grasp what's happening (which is why they wrote the blog post).
Very interesting article!
For me, the reason for reaching for the “backwards” version first is that it wasn’t as clear to me that the “forward” version makes a uniform distribution.
Even after reading the article.
I appreciated the comment here about inserting a card at a random location, but that also isn’t quite right, because you swap cards not insert. Nevertheless, that did it for me.
That’s funny. I’ve always done it the forwards way. I didn’t even realise that wasn’t the usual way.
I suppose one of the benefits of having a poor memory is that one sometimes improves things in the course of rederiving them from an imperfect recollection.
one algo "creates" shuffled subarray and grows it - the other "chooses" random cards one at a time
both seem intuitive from individual perspectives
The article didn't mention it, but I think you can derive the FY shuffle using base-factorial.
Separate point: One way of deriving 4 different variants of Fisher-Yates is to take a random bijection f:[n]->[n] (where [n] means {0,1,...,n}) drawn using whichever variant of the FY shuffle, and then potentially pre-composing or post-composing with the bijection reverse:[n]->[n] that sends k to n - k - 1.