The cheapest way to get into the banking business might be to buy a failed bank. When the US Federal Deposit Insurance Corp. takes over a failed bank, it tries to find a buyer, and buying a bank from the FDIC has some attractive features. For one thing, you get a bank, with a brand name and customer relationships and branches and stuff; building all of that from scratch would take you a lot of time and money. You get it cheap, probably, because it failed. And, critically, you don’t buy the whole bank; you get to pick and choose what you want. You take over the failed bank’s customer deposits, but you can avoid many of its other liabilities (long-term debt, lawsuits, etc.), making it a fairly clean thing to buy. And you can buy most of the bank’s assets, but if it has any particularly hairy assets that you don’t want, you probably leave those with the FDIC.
There is one problem with this approach, though, which is that the FDIC won’t sell you a failed bank unless you are already a bank. This is the flip side of the good things in the previous paragraph: If you buy a failed bank from the FDIC, you are not actually buying the bank, taking over the entire corporate entity with its bank charter. Instead, you are buying some of its assets and assuming some of its liabilities, including particularly deposits: The trade with the FDIC is that you agree to be liable for $X of the bank’s deposits, you agree to pay the FDIC $Y for the assets you want, and the FDIC credits the $X of assumed liabilities against the $Y purchase price.