I think affairs are as much about sex, as sex is about love. Affairs cover more than the act of physical, pleasurable contact. Having an affair means somebody wants you and to spend very intimate time with you, which is a compliment for most people.
It’s more egoistic than running a relationship, but it’s not like both parts get something out of it. An affair is less about devotion and more about satisfying your own needs. You find something someone you’re attracted to and enjoy falling in love a little instead of loving long term and the immorality attached to it by society is actually fading.
It’s important to understand, that you actually have more experiences in a short carry on than sex. Of course it’s a part of it, but it’s also the interest of the counter part in you and hopefully yours in him/her. For people who like extremes and very high quality time it can actually be much easier than having a long-term relationship they have to care for. Notice I say easier, not better, which it also can be, but most people know how it is, to be in love very deeply and not wanting to let go somebody.
Other aspects than the sex are actually defined by what you are looking for. Depending on what you are looking for, there can be a lot of things to share. Common interests are always healthy. A problematic case is work or persons you see on a regular basis anyways. There’s always the chance/risk one really falls in love and the other part does not, which would result in a very unpleasant situation for both. Though there is the (a lot smaller) chance you both fall in love, have kids and live happily ever after. (Keep in mind, tiny chance)