I also have had Vonage for a long time. The one problem I had is when I added a home security camera system to my network and started running out of available upload bandwidth. Then I could hear callers just fine but they couldn't hear me.
In my case I scaled back the camera frame per second speeds and all was well. For Vonage you will need to have about 100 kbps of upload bandwidth per Vonage primary phone number.
Also, yes obviously the ISP having constant outages will be a problem for Vonage. But you can have it ring your cell phone instead in the case of an outage. I don't know if it will effect Comcast's phone service but you might be no better off. I know Road Runner claims their phone service works even when there is an internet outage and only goes out if there is a systemwide outage of service, meaning no cable either.