The definition of formal system is not itself a purely formal affair. It depends on the meanings of terms such as ``abstract'' and ``formal.'' It is also widely misunderstood, even by mathematicians who profess formalist foundations for their work (but most of whom have never studied such a definition). Finally, the definition is disturbing, because there seems to be no absolutely direct way to express it---rather, we define a formal system to be the abstraction of something that can be described in a certain way. On the other hand, with careful study it appears to be one of the most precise and robust definitions that I can find for any concept.
Having written the definition above, I'm not very impressed by it. But I hope that it can serve as a guide to figuring out what is and is not essential to the nature of a formal system. In essence, I've only been able to describe the abstract formal nature of a system in terms of the concrete ceremonies that we go through to specify it, while encouraging you to see through those ceremonies into the systems that lie behind them. In particular, notice that the distinction between the ``description'' that ``delimits'' the arrangements in a formal language and the ``rule'' that ``allows'' certain sentences to be introduced in a derivation is not very precise. Derivations are just formal abstract arrangements. We can rearrange the description of a particular formal system by moving information between the description of its formal language and the description of its formal rule without affecting the essence of the system. There are lots of other rearrangements of description that don't affect the underlying abstract system.
A formal system is exactly the same thing as a computational system. The two phrases merely emphasize different applications and desirable qualities for systems, they do not distinguish different defining characters. Programming languages are particular formal languages, computations are derivations. There's really no difference. Although the idea of formal/computational system arose and had great conceptual value long before the construction of automated computing engines (what we now call ``computers''), widespread familiarity with computers probably helps a lot in spreading an understanding of formal systems.
Before computers, many very intelligent thinkers were amazed at the sense of a priori certainty derived from formal systems, and perhaps let their amazement distract them from a clearer understanding of the essential nature of those systems. The notion of formal certainty conveyed an almost mystical feeling to many (e.g., consider the greek attitude toward number). Familiarity with computers does not by any means automatically bring a correct understanding of formal systems. In many cases it may do the reverse. But I think that familiarity at least gives more people the gumption to think seriously about computation/derivation, and I expect such serious thought to eventually produce some insight.