Credit card issuers can be pretty annoying. They stuff our mailboxes with credit card solicitations, make us endure endless commercials about taking their cards, and seem to nickel and dime us with fees at every turn. The media continually reminds us of poor slobs who are up to their eyeballs in debt because they’ve maxxed out dozens of credit cards. Not surprisingly, the financial crisis has breathed new life into the legislation to regulate fees, disclosures, and various nooks and crannies of this business.

Congress should carefully consider the other side of the story in writing new regulation for the card business. Here are some points to bear in mind:

o Most people who use credit cards use them responsibly and actually need the credit. That’s more true now than it ever was. Many consumers are encountering temporary financial problems as a result of job loss and temporary reductions in income. Yes, a small fraction of the people who provide the fodder for all those newspaper stories get into trouble. But unlike the liquor industry which doesn’t necessarily care whether people drink too much, the card issuers have strong financial incentives to prevent this from happening—which, in part, is why it doesn’t happen much. Many people need and want credit. Just knowing that credit is available is important to them.

o Lots of small businesses are turning to their cards—always a source of credit for this sector that accounts for most American jobs—as banks have cut off other sources of credit. Credit cards are a major source of funding for entrepreneurial efforts that lead to the creation of new jobs. That’s always been the case. Cut off credit lines to small businesses and we will cut off job growth and lower economic prosperity.

o While we may not like particular credit card fees, it isn’t as if these guys are making a lot of money. This is a highly competitive industry and when you take the risk of credit card lending into account it is not a very profitable one for most issuers. Regulations of various rates such as late fees may curtail those fees but the card issuers will have to raise other fees to make up for it or reduce their card issuance which would be bad for the reasons above.

The various pieces of proposed legislation for regulating fees and granting of credit are going to have two main effects. First, they are going to result in card issuers raising fees that haven’t been regulated. The days of no annual fee cards will go away and maybe some of the reward programs will be curtailed. There’s no free lunch for card regulation. Second, they are going to result in card issuers reducing the provision of credit card debt. If they can’t charge extra for late fees and impose other penalties late payment, or if they just can’t make as much money on cards as they once did, they will cut back on who gets cards and how much of a credit line they get. That’s going to make a lot of American consumers worse off and its going to cost jobs.

What’s unfortunate about all this is that some card issuers have brought this on themselves. Some of the banks just got way too cute at imposing hidden charges in their quest to make money on cards in a highly competitive environment. But, as with everything involving the financial crisis, we need to be careful not to cut off our noses to spite our face. We shouldn’t cause consumer misery and job loss just because we’re mad at a handful of greedy bankers who should have known better. The economy doesn’t need to dry up yet another source of credit for consumers and small business owners.