Rare Mushrooms are found in only a few places on the Ark. They are used in some of the Rockwell Recipes, though collecting a decent number of these can be a challenge.
I've found that Rare Mushrooms are best found in the Swamp. So far I've not found them elsewhere. They are found by chopping down certain trees in the swamps. They are a rare drop, so you'll get a ton of wood as you take drown trees trying to get a Rare Mushroom.
This doesn't have to be quite as dangerous as it sounds, however. Along the East coast of the island there is a beach with some swamp trees not far from it. Along this strip of beach I've noticed that it's not very dangerous.
Below are two screenshots of the different tree types that can provide Rare Mushrooms. Please be aware that harvesting with a Metal Hatchet is far better than Stone. I am not even sure you'll have much chance to get Rare Mushrooms with a Stone Hatchet. The third image is a map of Ark, on which I've marked the safest area for harvesting Rare Mushrooms.
Here is a map that shows a good area to search for Rare Mushrooms in Ark. This location is on the main map The Island. Beware, this area can be extremely dangerous.
The fastest and easiest method of harvesting Rare Mushrooms is with Dino Power. And surprisingly enough, the Mammoth, isn't the only good option. Sure it's the best for getting Wood. But, I find it faster and easier to get Rare Mushrooms with a berry harvesting Dino. Yup, it's in the tail. Sure I don't get much Wood that way, tons of Thatch. Keep in mind, I'm talking Rare Mushrooms here.
A long tail, oh yeah, that does the job. Hits a wide area faster than a Mammoth. In the screenshots below I am using a Stego, but a Bronto would be better. Regardless of better, the Stego will still have me going home with a bunch of Rare Mushrooms and it won't take long.
In the screenshots below I show harvesting Rare Mushrooms around Map GPS 56.3 - 83.3. It's the trees that look like they are standing on their roots that are the best ones for Rare Mushrooms.