To avoid the scribbly look, keep the texture lines consistent. Instead of trying to draw all the little grain details, only draw the larger ring structures and the like. Try to keep them from intersecting, since they usually don't. Boards and panels are usually cut with the grain of the trunk, but often not perfectly with it, so they usually have these very long lengthwise "rings", plus some smaller ones where branches used to grow.

To keep the texture from being distracting, you should use a low-contrast colour. If you can't use a low-contrast colour, it's better to have no texture at all, in my opinion.


Here's a door I did a few years back:

The colour choice is a bit weird due to the limited palette, but hopefully you can see what I mean about keeping the lines all connected and keeping the rings in mind as you draw the texture.
You could elongate the shapes more vertically, this would give the look of boards cut with the grain rather than straight/diagonally across it (that was a mistake I made with this door). You could also play with the scale of the texture. Trees come in different sizes, and so do boards.