From a later flashback chapter from Farewell Wanderlust:

And he would have to watch. Watch Akira being handed over like a trophy. Watch his spouse treat him however they wanted - good or bad. Watching, watching, watching, but never touching, never be more than a shadow that would make sure Akira could continue to laugh like he did when they did things like today.