"If you were a servant I wouldn't be taking you shopping or putting up with your mouth."

As if he doesn't actually enjoy it... Midorima wouldn't put up with it if he didn't actually appreciate Takao, that's the reality. And he'll call him a servant to other people anyway because what is admitting friendship. But now Takao's said it so he can allude to it as proof he's right. Not that it's stopped him before...

Regardless, he's going to get ready to go again without Takao if the other boy doesn't catch up. Prismatica is open late, they can definitely go now that Midorima has had some food to put energy in his system for a bit.