[ It's a rejection for something Jack was starting to get hopeful about so it does sting. He wants Castiel nearby, obviously. He wants to be with him, even if he doesn't know or love Jack. It doesn't feel right without him, not if Castiel is alive and out there. So it's another reminder this gap between them will never fully be closed. ]

You're dead now. In my time. I thought I'd never see you again. The Shadow likes me okay in the Empty but they really don't like you.

[ So it's unlikely they would let Jack and Castiel be together in the Empty. They're all supposed to be asleep there, Jack was only awake and aware because the Shadow allowed Billie in there to talk with him. He knows that what he has to look forward is endless emptiness. Not a place to rest with his father. Not even Heaven truly has that, outside of an exception like the Winchesters. ]

I don't know what I'm doing. I want to be here for you, but it feels like I'm making things worse.