It had been a month since Xavier was busted out of prison. Leonard couldn't bring him to his mansion, Skye was always snooping around and he didn't have the time and energy to start explaining why the man who was said to be dead was alive in his home.
He wasn't ready to answer questions or even tell lies that would stress him out, Skye did that on her own. Leonard got a penthouse under a celebrity's name. And placed two guards with him, who act as the people living there when deliveries are being made. Everything has been provided for, all Xavier needed to do was plan.
Plan under the skies, plan like a ghost and plan behind the shadows. Leonard trusted Xavier, not completely but enough to get information from him and study his patterns.
A man like Xavier had something that made him noticeable by Milan, and he also did something that made Milan feel threatened, he noticed the body language Milan displayed when he asked a while back, he was angry.