Do you have any colleges nearby that might have boards where you could post an opening? That is how I got my summer nannying position. It might not work so well for year round, but you never know if there is someone who only goes part time that might be interested.
We looked at an agency, care.com (there were actually some profiles on there that were also on the agency's site), and a local mom's forum. We had almost 20 people contact us from care.com. Some were great, some were eh. Our nanny is part time, she works for another family on the days she isnt with us. We did 2 different background checks and met with 2 of her references. If you want a list of questions we asked before hiring, I can send them to you. I was taken aback by one candidate who looked awesome on paper and then basically said she just wanted the job to make extra money to go out drinking with her friends.
We spend a ton of money paying our nanny over the table using a payroll agency, but thats because daycares around here dont take infants for only 2 days a week. And I didnt want to worry about drop off and pick up with twins. If you choose to hire one, keep in mind the cost of an agency finders fee, and payroll set up, and taxes, workers comp, unemployment insurance, etc. May have you rethinking hiring one for 4 months once you add it all up.