I used to use Blogger, but I recently decided to move my blog to Wordpress. The primary reason I decided to leave Blogger is Blogger's pathetic security, mostly due to the lack of SSL access. I picked Wordpress for my blog's new home because Wordpress has some of the best features and positive overall experience out of all blogging services I know. In fact, Wordpress's only minor drawback in my mind is the lack of manual control over the templates, but I'm not a customization freak, so this isn't a big concern for me.
Blogger doesn't even let you log in over SSL, not to mention keeping your session over SSL while you're editing your blog. When you change your password, Blogger doesn't even send you a validation email. What does that mean? Every 12 year old hacker armed with Ethereal or tcpdump can steal your password by eavesdropping on your connection, and can then go ahead and change your password and thereby hijack your blog.
Your blog is a large part of your your online identity. It's often the first thing that shows on search engines when people search for your name. It's valuable. I'm not comfortable with the thought that my blog could be hijacked so easily and there's nothing I can do to prevent it. (I did read that certain blogging applications let you use Blogger over SSL, but that's one more hoop than I'm willing to jump.)
I dread the day when somebody stages a large scale attack on Blogger and hijacks thousands if not millions of blogs. Maybe such an event would kick Google's butt into action, getting it to turn on the SSL switch on the Blogger servers. I suppose that if this happened, Blogger could mitigate the disaster by rolling back all changes that happened during the attack, and then resetting all passwords. The damage would be significant, but not irreversible. I'm actually more concerned about individual blogs getting hijacked without Blogger's knowing or caring.
Wordpress has SSL access, so this problem largely doesn't affect Wordpress users (I say "largely" because the Wordpress servers could always be cracked and the user data could be stolen, but the risk is very small). That's a huge advantage for Wordpress, and is the primary reason I moved here. I must say I'm happy here so far. I may decide to host my blog on my own server eventually, which would have is downsides, but it's likely that Wordpress will remain my blog's permanent home.