Should pregnant women avoid cats
Yes and No. The reason people say to avoid cats during pregnancy is because they worry you can contract toxoplasmosis, a rare parasitic disease, that could spread to the baby.
The truth is you are very unlikely to get this from your cat. It is most often contracted eating undercooked meat or vegetables that aren't washed properly. Even then, it's very rare.
The reason cat's are singled out is because they are one of the few species who pass the infectious phase of the parasite in their feces, so coming into contact with cat litter could be dangerous. However, you'd have to ingest the parasite in order to become infected. Also, most indoor cats have no way of getting the parasite themselves unless you have mice and they contracted it from eating a mouse. If your can has been indoors for at least 2 years you're probably safe.
Still, It's best to avoid changing the cat litter if you're pregnant just to be safe. If you do have to change it, be sure to wash your hands thoroughly afterwards. Make sure anyone else in your household does as well, so they don't inadvertently pass it to you. If you have a high-risk outdoor, hunting cat then I would also wash your hand after petting/brushing the cat. Avoid touching your face afterwards and keep the cat away from surfaces you eat or prepare food on.
Chances are, if you haven't gotten the parasite from your cat before you got pregnant, you won't suddenly contract it now. Still, it's better to be safe then sorry.