Rue grew up in a family of mediums in Lavender town, and I’m her family they receive a “spectral familiar” which is essentially their first Pokémon.
These Pokémon aren’t actually caught or kept in a pokeball however! They are instead bound to their partners. (For example, Gengar stays in Rue’s shadow most of the time.)

The Pokémon is first found as a spirit who is either;
A) Showing great interest in the human already
B) Currently haunting someone

After finding a suitable spirit, there is a seance (which includes the haunted individual if there is one) where the two are linked together permanently.

In Rue’s case, there was a Haunter who would secretly follow child!Rue around, it’s unknown why or if there was any eventual I’ll intent. But it was essentially busted and given the offer to be selected as her spectral familiar. It accepted Rue because binding with her gave it the power to evolve into Gengar.