While rocks in Vue seem more varied in shape and look better in their default state than the ones in Bryce, they are generally not suitable for closeups unless you tweak them a bit.
What you will need to complete this tutorial
A copy of Vue 5 or 6 and the free lichen rock material available in the textures section.
What is the issue with Vue rocks?
Switching to a bitmap based texture
While the above is more acceptable than the default setup, it still does not look much like a rock. In many cases, the procedural textures of Vue make a very good job, but generally using a bitmap texture will get you far more realistic results. To try it, download and install the lichen rock material from the texture section of this website.
The lichen rock texture is made with a high res photo taken from the CGTextures website. I edited the photo with Image Synth to make it tileable. I then generated several additional maps, including a detailed bump map with the beta version of Crazy Bump, a superb program that enables you to make a bump map from an ordinary photo (I summarize a lot, it is far more powerful than this). The Vue material is a combination of the different bitmap images I exported from Crazy Bump.
In order to make the rock look really realistic, activate the displacement in the bump tab and give it a quality boost of 1. Of course, you can still use the rock texture without this, but the bumps look somewhat fake.
Thanks to the displacement material, you should now have a rock looking more realistic than the default Vue one and suitable for closeups. The bumps look realistic as they make real bumps on the object surface. Be aware that this kind of material is longer to render.
Changing rock materials in ecosystems
You can include rocks in ecosystems, but they generally all have the default material. You cannot edit ecosystem components individually, but you can access their material via the material browser. Ecosystem materials are at the bottom. It is possible to use high res bitmap textures in ecosystems, but do not try to enable displacement mapping !