They need to know the information which functions as key. Because many people don't trust government secrets, the information used for this purpose on a passport is actually just facts about you which were already printed in your passport, plus the passport number. The machine summarises these in a "Machine readable zone" but they're nothing you didn't know.
For a random traveller you can probably guess roughly how old they are, which is a few bits for the date-of-birth, and maybe you could strike up conversation and discover their name (or maybe it's printed on baggage, called out by fellow travellers etc.) but yeah it'll be very hard
For a very well known person you can likely discover everything except the passport number and you might get a decent guess at that from knowing roughly when it would be issued.
"For a very well known person you can likely discover everything except the passport number and you might get a decent guess at that from knowing roughly when it would be issued."
From a very well known person you could probably also steal everything you need directly, if your purpose is to create damage.
For a random traveller you can probably guess roughly how old they are, which is a few bits for the date-of-birth, and maybe you could strike up conversation and discover their name (or maybe it's printed on baggage, called out by fellow travellers etc.) but yeah it'll be very hard
For a very well known person you can likely discover everything except the passport number and you might get a decent guess at that from knowing roughly when it would be issued.