The responsibility on both sides is taxing; hence, the very act of speaking in public breaks the natural rules of human discourse and is thus unnatural.
The result of this unnatural act is to make both the speaker and the audience somewhat nervous in their new roles as dominant speaker and submissive audience.
And over time, those connections that I just mentioned, where you're moderating the panel, you're not necessarily on the panel, you're not the keynote speaker, that's okay.