Is it toxic to continue hanging out with this guy?

Get out ... each of you is serving a purpose in his life. You are serving second fiddle. All of his needs are being met by more than one woman. And you can tell yourself there is nothing ual there ... but then you mention feeling jealous ... if this was truly platonic ... jealousy would not be an issue.