Read the title again. Let it seep in. Now exhale...
Bounce metrics reported on analytics are not a ranking factor. The represent any visitor who visited only one page of your website, then left. Perhaps they only needed your phone number or address? Or perhaps, they discovered you and bookmarked you for a future visit? Both are legit bounce scenarios what may have lead to further website visits or even better, a sale.
This notion that all site bounce is a bad thing is very widely published myth and assumption by amateurs.
To be fair, some bounce may be bad, like if visitors to a page were expecting something else or if the experience on that page was so awful, they left your site for someone else. But even in these cases - it's not a ranking factor.
One of the most common causes for bounce occur when ads point to a landing page where the visitor performs an action and isn't required or expected to do anything else. Like filling out a form. In this case, sending a visitor after submitting a form to a conversion page where they can see/read an acknowledgement that their form was submitted as opposed to displaying this message on the same URL that the form was on - is a great way to reduce bounce. It's also a super way of reporting leads and conversions in Google Analytics via Google Goals.
So do keep any eye on it, and learn what bounce means for your website. Sometimes, reducing it provides a better user experience for your visitors. Other times, meh... no biggie.