Solution for all these troubles are Widget Logic plugin by Alan. This is a fantastic plugin for WordPress blogger. It works very well. I highly recommend this plugin for controlling content on sidebar. It takes WordPress conditional tags and output according to the conditional tag(s) you provided for the content.
It will not display the recent posts on Home Page. Conditional tag is_home() returns true when you are on home page.
To display a content for a particular Category or all categories, you can check with category conditional tag.
is_category() - Is a Category archive page ?
is_category('4') - Is a category Id 4 page? Check in Category edit page for Category id.
is_category('php') - Is PHP category page?
in_category('5') - Is the current Post in this category?
Similar conditional check for tags are also available and can easily be used with Widget Logic plugin to control sidebar contents.
is_tag() - Is a tag archive page?
is_tag('wordpress') - Is a WordPress tag archive page?
If you have set your front page for static page or something else (Setting->Reading->Front Page display) then also you have the conditional tag - is_front_page().
For displaying content on all blog post pages only, use this tag:
is_single() - Is the page displayed in browser is single/post page
is_single('title of the post') - When title of the post match with the entered text then it will show content.
For controlling widget on wordpress page, use is_page().
Now, you may be filling power in your hand. Check here for all possible way of controlling content in sidebar of your WordPress blog.
Here is another post for controlling sidebar content.