I've never tried brown rice syrup in coffee or tea, but I think you could do it. The flavor of BRS is so mild that it's mostly just sweetness. So you'd probably be fine. The flavor reminds me of a very, very mild honey, but less sweet.
Agave dissolves in hot or cold liquids and it's easily pourable...I think it makes a great substitution for tea! I always use agave in my tea and the lattes I make for myself. It's wonderful. As for coffee, I bet it would be great too although I don't drink coffee so I can't tell you for sure. I think agave tastes great in anything that calls for sugar; it's very versatile.
Thanks! I cannot wait to try the agave. I'll have to go out and buy some. I hear it's tasty if you add a little vanilla and agave to plane, unsweetend almond milk. I know Whole Foods and Trader Joe's sells it.